Business owners have long struggled to measure the impact of change.
A business could make a change, and sales could drop … But the reason for the drop could be the change itself, or it could be 100 other factors. That’s why attribution is so important in the business world. You need to know whether you can attribute a result to a change you made so you can proceed accordingly.
The same thing applies to ecommerce stores. If you make a change on your homepage and suddenly sales increase, you don’t really know if sales increased because of the change you made or if the increase was due to something totally unrelated.
And that’s where testing comes in.
How Does Testing Work?
Testing allows you to split traffic between an original version of your site and one or more new versions so you can assess changes you made in a controlled environment and record the results of each version separately.
There are two main types of tests you can run on your ecom store:
- A/B test (aka “split test”): Compares two or more completely different versions of a webpage
- Multivariate test (MVT): Compares multiple variations of a webpage, where specifically chosen sections differ
Note: This type of test requires more traffic (than an A/B test) for the results to be reliable because of the additional variations involved.
Why Test My Store?
The graph above shows the conversion rate of an ecommerce store. As you can see, there was a marked drop from August 8 to August 10. So, someone must have made a change on the 8th that was detrimental to the store … right?
Actually, no. There were no changes made to this store during that time frame; that’s just regular fluctuation.
However, if you had just implemented a change on your store on the 8th, that change most likely would have been mistakenly blamed for the sudden poor performance.
Furthermore, even if the change was massively helpful, the conversion rate would have looked flat because the decrease (due to natural fluctuation) would have canceled it out. And because of that, you’d probably assume the change had no effect, maintain the status quo on your webpage, and subsequently lose out on the positive results that would’ve followed.
This is exactly the kind of mistake that can be avoided by testing. When you test, you are sending traffic to each variation at the same time. The conversion rate can go up or down, but, regardless, you’ll always know which variation is winning because both variations are live at the same time.
How Do I Start Testing?
If you have an ecommerce store, you can use Google Optimize for free. Within this platform, you’re able to set up multiple types of tests on your website.
The challenge is to know what kinds of changes to start testing.
As a general guideline, the changes that typically have the most impact are:
- Based on customer research
- Big and bold
- On high-traffic pages
- Above the fold (i.e., the part of the page visible without scrolling down)
- Involving elements such as offer, product, price, or layout
- Solutions to existing problems
Testing Tips
Keep in mind that tests need to run for a certain amount of time for the data to be reliable.
Think of it like this: If you flip a coin, there’s a 50/50 chance it’ll land on either side, right? But, if you flip a coin two times and both times it lands on heads, would you conclude that 100% of the time it’ll land on heads? Of course not. You know if you tossed it more times, it would eventually land on tails.
Well, the same thing happens with tests. You need more than one day (or coin toss) to have any certainty about the results.
To learn more about how long your tests should take, simply Google “Significance calculator A/B testing.” Many websites can help calculate how long to run a test for it to yield reliable results.
Conclusion
Coming up with good changes and testing those changes will help you stay ahead in the highly competitive ecommerce world. Each win stacks on top of the previous win.
And really, even the changes that don’t pan out in your favor are advantageous because they help you avoid missteps and potentially harmful business decisions along the way.
So, what are you waiting for? Get testing!
