A/B Testing for CRO — CXL Conversion Optimization Minidegree Review
A/B testing is crucial for conversion rate optimization (CRO), but it can be difficult to know where to start.
You might be running A/B tests on your site, but if you aren’t getting the results that you want, it could be for one of many reasons.
Maybe you’re not using the right tools or knowing how to analyze your results correctly.
We’ll take a look at everything from the A/B testing process to specific tools and criteria to help you get more out of your tests.
What to test?
If you want to improve your website, you have to test. But what exactly should you test?
There are plenty of articles about running different kinds of tests. You can run many different types of A/B, multivariate or responsive tests on your website.
The one you should run depends on many things, so the correct answer is: it depends.
Testing the wrong things results in wasted time, effort, and money. You have to know what actually makes a difference.
Test prioritization
We often hear people frame the challenge of A/B testing in these terms: I have limited time and money.
How do I maximize conversion lift while minimizing my investment?
That is an important question, but in my experience, a test prioritization question that’s a more common one is: Which tests should I prioritize and why?
The PXL Framework is a simple yet powerful framework for prioritizing ideas and running A/B tests.
A/B testing statistics
A/B testing tries to estimate the impact of a change by comparing it to change (b) when there was no change (A).
The problem is that we could be tempted to conclude there is an improvement when there isn’t.
Statistical significance is an important concept when doing A/B testing. It is highly vital when communicating the results of an A/B test to others.
It can also be useful in determining whether you are making the right decisions for your business based on the testing you do.
Statistical Power for A/B testing is a guide to using statistical tools to determine how much confidence we can estimate.
While concepts like an “alpha” or a “beta” coefficient can be a little daunting, these metrics help us assess how precise an experiment was.
Testing strategies
Optimizers can use numerous testing strategies while running an A/B test.
There are numerous testing strategies that optimizers can use while running an A/B test.
Multivariate testing
Multivariate testing is a method of carrying out statistical analysis on the performance of landing pages.
It helps us determine whether these pages are effective and which elements contribute most positively to their conversion rates.
It works for conversion rate optimization by changing the various elements on a page (such as text, image, call-to-action). It measures the outcome on the visible changes it brings about in bounce rate and conversion rate.
Bandits
Bandit testing is different from a/b tests, so the traffic to different variations is not allocated evenly.
So it’s not 50/50, or 33/33/33, the traffic allocation, or how many people see which variation is dynamically changed based on the performance of a specific variation.
If a variation A is outperforming variation B, the Bandit would start sending more traffic to A and less to B.
And if the opposite is true, it would send less traffic to A and more to B.
But it isn’t just about “winning” variants getting more traffic, that is definitely not the case at all.
The results you get from bandit testing are completely dependent on how you set up your test environment.
You could be sending 10% of traffic on Tuesday, and 90% of traffic on Wednesday, even though you have the same winner each week.
Existence Testing
The principal goal of existence testing is to ensure that a variation can improve the test objectives, not just be better than your baseline.
This is why it’s important to note that the purpose of existence testing is to identify the crucial elements of a page for conversion rate optimization, not merely identify a statistically significant element.
If you change your page and then get an increase in your conversion rate, that doesn’t necessarily mean that you have an improvement.
Iterative Testing
The process of iterative testing is very similar to the scientific method. You start with a test, analyze it, and then improve your hypothesis.
Then you run it again — rinse and repeat until you have a significant win.
Testing on steroids might be another good description.
If you’re testing just the “big picture” (one page, one hypothesis at a time), you may think you’ve run an A/B Test.
However, if you can test all parts of your website and continually improve your hypotheses, I would call that Iterative Testing for A/B testing.
Innovative Testing
Innovative testing is when we’re not just changing one or another element, we’re rethinking how this page or how a certain flow on your site should be.
So this is not a radical redesign of the whole site. It’s about innovating a certain portion of your site and running an experiment to see if that idea has merit or not.
Split Path Testing
Split path testing is the type of testing where we’re taking the user onto a different path (split), and then secondly, sending users to different variations of pages.
We do split path testing because we want to perform an A/B test on a single webpage, not multiple versions of a page.
Conclusion
Internet marketers, conversion experts, and web-analytics professionals use heuristic analysis frameworks when doing conversion-optimization audits.
Having a framework is important because it ensures a consistent approach in your efforts to improve conversions.
If you’re looking to improve your marketing skills, I strongly recommend the CXL Conversion Optimization Minidegree. It’s a great way to get up to speed on marketing.
This program is for students looking to get advanced-level skills.
Learn from industry experts, not just conference speakers and theory peddlers.
It’s for people serious about building skills and ready to put in the effort required to complete it.
It’s for people who want a holistic understanding of growth across all channels and on an organizational level.
Even if you’ve never taken a business course, the lessons and activities in this program will make you feel like an expert. Growth Marketing Minidegree is a significant career boost for me and one of the best decisions I’ve ever made.
