Convenience vs. Authenticity: User Testing Internally

A common question I get a lot is, "can we user test this internally?" And the answer is always "No, but…".

Sometimes we forget the purpose of user testing. Are we doing it to understand customer needs? Validate ideas or concepts in the early stage? If so, that requires testing with actual users, people who use the product.

Employees can be users of your product, but they come with a set of biases and obstacles that will probably create a deviated reflection of actual users. Employees can be affected by:

1. A set of biases

They might have prior knowledge about the project or client. Their knowledge could impact the way they react to a prototype. Employees could also potentially be more likely to be loyal to a brand. Their behavior in a test could become overly positive (or negative) because of that loyalty. It might become less about the prototype and more about their feelings and relationship with the company.

2. Stepping outside their role

A visual designer might focus on the visual aspect versus the product's full experience. A developer might provide very technical feedback. This doesn't necessarily reflect an actual user and their process of thinking.

3. Internal politics

Employees could have relationships with the designers, and they don't want to hurt their feelings. They don't want to disappoint them or insult them in any capacity. Or testing with someone more influential in the company who may have their own agenda of the company's best direction. They could have their own set of motivations that could affect the results.

Ultimately, the biggest risk with testing with employees is that we don’t see the authentic behaviour of real users.

So why does the question come up a lot? Primarily for two reasons: how costly it can be and the convenience of recruiting internally versus externally. It can also be about confidentiality, that an idea is so new that they don't want the concept to go to market yet, even though there are NDA's in place. It's a more complex situation to control. NDA's that are already in place with internal employees might bring more ease of mind.

So when is it an excellent time to test with internal people?

Pre-test
Internal Product
Confidential Concept
Resistance to research

Pre-Test

Before testing it with actual users, it might be useful to do a pre-test with someone internally to ensure that all is in place. There are no bugs in the prototype, no spelling mistakes, "no oopsies that aren't supposed to be there." We want to make sure there are no red flags. It's vital to ensure the testee understands what you wish to feedback on during the pre-test. It could be the structure of the test, it could be the test script (e.g., are you asking the right questions? Should you frame it differently?), and let them know it's not precisely the interface or the design itself you want feedback on.

Internal product

If the product is for employees, then yes, please do test it internally. Websites such as an intranet are specifically developed for this target audience. If the product's target audience is your employees, then make sure to invite employees who don't know much about the project to minimize any biases.

Confidential Concept

When ideas or concepts are in the early/initial stages, stakeholders may be concerned about them being leaked to the market. By introducing external participants to a test, it may feel like an announcement to the market. When the confidentiality of the product is too high, stick to using internal people. You can always test with actual users later in the process.

Resistance to research

If the company you work for is having some resistance to research, such as it's too early, too costly, or is considered an inconvenience, then a good option is to start by testing internally. Highlight the challenges that come from the internal test to convince stakeholders to do another test with real users.

If you do end up using internal employees as testees, key criteria to look out for when asking them to participate:

Minimum engagement with the project or product
Minimum knowledge about design or tech (developers, designers, researchers). Employees such as Human Resources or in Finance might be the right choice as a testee.
Minimum knowledge about the company, such as new hires or interns, to get a fresh perspective.

There are always exceptions, such as the target audience and the test purpose. But these are the essential items to watch out for.

At the end of the day, real users should be used instead of internal employees. In situations where this isn't possible, try to follow the above criteria as closely as possible. This will ensure you have the most valuable, insightful, and accurate results possible.

Give us a call if you need help on when or who to test with. We're more than happy to help and ensure that your product or service is customer centric.