Expert Guide for Running Successful User Tests

Mariana Dourado | Feb 01, 2022

Reading Time: 4 minutes

User tests are a gold mine for any ecommerce business. Especially for stores with low traffic, where A/B tests cannot be run and would take forever to achieve any relevant data, user testing is what will guide major changes to the website. But don’t assume that if you have enough traffic you don’t have to worry about user testing. If your site gets plenty of traffic, you should be combining both strategies to maximize results.

In this article, we will cover the main things you need to know to set a good user test round and what to keep in mind when analyzing it. Remember: listening to customers and potential customers will give you the information you need to make your business successful.

1. Set a goal. 

The first thing you need to do when it comes to user testing is set a goal. 

What do you want to find out? Did you push a new version of your site live and want to see what your users’ first impressions of it are? Have you identified an underperforming page or step and want to find out what’s going on? Do you want to test a new functionality?

Make sure the round of user tests connects to a single main objective, otherwise you won’t be able to go deep into the problem you’re facing. 

Note: You can (and should) run tests just to make sure things are running smoothly, in which case, verifying that nothing is falling apart on your site will be the primary goal. 

2. Create clear instructions based on the selected objective. 

Make sure your instructions are clear and connected to the main goal of your tests. Users should be able to easily understand what you expect from them, and their input should help you figure out the issue you’re facing. It doesn’t make sense to force testers to spend a lot of time on a page that doesn’t need to be analyzed at this point. 

3. Find the instructional “sweet spot.” 

Clearly let your user testers know WHAT to do, but not exactly HOW to do it. Let’s say you noticed users are not interacting with your search bar. The most likely scenario is that they are not seeing the functionality—a major issue, as people who use the search tool convert several times more than the overall audience (which is why you should always incentivize its usage or you’re throwing money down the drain). 

If you simply tell them, “Type ‘red t-shirt’ in the search bar and choose your favorite item from the results,” they’ll know they need to find the tool and will very likely provide valuable insights when trying to find it or while using it. Here are some examples of possible feedback:

• “I didn’t even see the search bar there.”
• “I can’t find the search bar.”
• “The search icon is too small.”

Expert tip: Never (ever) hide your search behind an icon; always provide an actual bar! The more prominent your search feature is, the higher the chances of people interacting with it.

On the other hand, if you tell them, “Go to the search bar located on the middle of the header just below the logo at the top,” you will never know if finding it by themselves is extremely easy or almost impossible, and you’ll likely miss the source of the problem at hand.

Ideally, the tasks you assign should neither be too broad nor too narrow. Make sure the audience understands what needs to be done, but let them figure out how to do it by themselves so you can evaluate the clarity and usability of your site.

Note: Don’t ask for their opinion about the website, the layout, etc. Give them the task with instructions and observe what they do. 

4. Segment users based on your audience.

Choose user testers who are similar to your target audience. User testing platforms will allow you to filter the kind of people you need, so choose wisely to ensure the testing scenario is as similar to reality as possible. However, if you find that the audience who’s actually buying from you doesn’t line up with who you’re attempting to target, then focus your testing on the audience that actually turns into paying customers. Then, make a point to reassess your target audience, because it should always match the people who are actually buying from you.

5. Prioritize the most relevant device. 

You want to focus either on whatever device you’re seeing the most issues on or the one represents the majority of your traffic. If that’s mobile, then focus on mobile user tests. Now, that doesn’t mean you will “ignore” the other devices (like desktop or tablet), but they may not be worth prioritizing. You can always catch them on future testing rounds. 

6. Run 5-15 tests per round.

Despite what it may seem, more is not always better. If you run more than 15 tests, you’ll notice that issues will be repeated and you’ll essentially be confirming what the first tests have already shown you. Also keep in mind that you’ll need time to analyze each response, so it’s not recommended to run a bunch of them in one round and be overwhelmed with analysis. Typically, 10 is the sweet spot (as you will be doing that regularly), and you shouldn’t run fewer than five on each device.  

7. Remember user testing is an ongoing process.

Ideally, you will run a round of tests, fix the issues and gather insights, run a new round, fix new issues and gather new insights, and continue this process indefinitely (or at least until you reach the desired level of optimization and success with your store). 

8. Lay the groundwork before you order user tests.

First, check to make sure the tasks can be done by following the instructions you provided. Once the information is clear, if there are no typos and you’ve ensured the pages you need checking can be accessed, I highly recommend ordering one test as a sample, just to confirm everything is in order (it’s better to be safe than sorry.) 

As soon as you analyze the sample test and make sure everything is working properly, move forward with ordering the rest of the user tests.  

Expert tip: Pay more attention to what users “do” than what they “say.” They might say your website is great and easy to use, but if they’re having a hard time doing what you’ve asked them to, you’ll know that’s not exactly true and that improvements are necessary.

Wrap-up

User testing is one of the most valuable tools you have to optimize your ecom store. Take advantage of it by running user tests consistently and taking time to analyze the insights. I also highly recommend you keep track of all the invaluable information the tests provide so you can go back to it easily when you need testing ideas and a list of topics to update. 

All that’s left now is for you to get the ball rolling with user testing on your own store and watch your business soar to the next level!