What is A/B testing? 6 steps to conduct an A/B testing

If you are a marketer, you may use A/B Testing a lot. It’s an effective method to test between two values. So what is A/B testing and how to do it? Let’s find out in this article!

What is A/B Testing?

There are quite a few definitions of A/B Testing, but in general, they all say that: A/B Testing is running tests between two different versions, under the same conditions, to find the “winning” version before deploying this version on a larger scale. Here is another definition of A/B Testing:

• A/B testing is a method to compare two versions of a webpage or an application to find out which version works better.
• A/B Testing is a method of running different samples in a short period of time, measuring which model customers like.
• A/B test is a technique of dividing the object to be tested into 2 versions A and B to choose the version that users are more interested in through how the user interacts with each of those versions.
• A/B testing is a process where 2 versions A and B are compared together in the same environment/situation to assess which version is more effective.

One opinion is that A/B testing is not given much importance in practice compared to SEO or Ads. Because among A/B testing, SEO and (Digital) Ads, the latter two are directly related to revenue, a vital decision for any businesses or dropshipping agents, so it is almost always focused first and then on optimization.

Why should we do A/B Testing?

• Allow changes to be made judiciously, preventing a bad impact on the user experience
• Help determine the influencing factors and the level of each factor on the behavior of customers/users;
• The test execution cost is low, but the results/rewards are many times higher (if achieved);
• Make communication between sales, marketing, superiors more grounded, based on specific data/data instead of feelings.

Looking at it from a specific perspective, A/B testing can:

• Website: optimize UI, UX of the website, find an interface that attracts users
• Advertising and selling online: measuring the effectiveness of each different advertising model
• Offline advertising: evaluate the effectiveness of advertising channels
• Mobile apps: help improve UI, UX
• Email marketing: determine which subject, sending time brings the best results

Process of conducting A/B Testing

There are many different A/B processes, but an A/B test will often follow these steps:

Step 1: Collect (and analyze) data

The goal of this step is to identify problem pages, such as high bounce rate/drop off, low time on page, or conversions.

Step 2: Provide orientation and goals for post-test improvement

For example, increase the traffic to the web by how much %, increase the conversion rate, reduce the bounce rate, reduce the cart abandonment rate (for E-commerce websites).

Step 3: Make a hypothesis

List ideas and hypotheses to perform A/B Testing, for example, putting the subscription box on the top left will increase subscriptions, changing the color of the subscribe button will increase subscriptions…

Step 4: Determine sample size and A/B Testing runtime

Usually can test with at least 2000 page views or 1000 visitors accessing the page.

Step 5: Create a new version to conduct A/B Testing

The new version should change a single factor (variable) so that you can compare and conclude how that factor affects the goal set out in step 2. In fact, if you are an expert in the field. A/B is possible and there are tools that run multivariate (to save time), but not everyone can control the entire test.

Step 6: Analyze the results of the experiment and draw conclusions

If the new version works better than making this change. If not, proceed to the next test to find the winning version.

Note:  Steps 1 and 2 can be interchanged depending on the situation. In a situation where you see a relatively clear goal of improvement, the next step might be to collect data to pinpoint the problem or to convince your superiors or other departments. If the situation already has analytical tools at hand, usually Step 1 will be performed first, in the nature of problem discovery, and then Step 2 is performed.

A/B Testing in practice?

That’s a good definition, even though there is a process as well as a way to do 1 A/B, but most marketers do it subjectively, without the necessary monitoring and measurement to produce results. objective results.

For example, A/B does say that based on confidence is how many % to choose the winning version, but how many people can do this. Or when you send an email marketing campaign and “feel” it is not working, so in the next campaign you can not only change the subject line, but also adjust some small content in the email, so that’s it. I’ve got another version that may or may not be better. While the way we run can make us feel like we’re doing an A/B, the reality doesn’t help us discover which change (variable) has the biggest impact on the outcome.

Notes when doing A/B Testing

Each writer has different notes when doing A/B Testing, below is a summary of important notes, including:

• Make sure the environment and conditions when doing A/B Testing must be the same between versions.
• If possible, it must be possible to separate the traffic between desktop, mobile, and tablet because visitors in each place may have different behavior.
• Visitors coming from any source should also be distinguished because each source can have different behavior, different needs.
• When the set test run time is not completed, do not jump to conclusions or worst, end the test.
• Understand and make the best use of the tools used throughout the process.
• Run the test only for new visitors, limiting the display to existing visitors/users or customers.

Conclusion

A/B Testing is a necessary activity in optimization, specifically CRO – Conversion Rate Optimization, and mastering the entire process needs the performer to understand the process well and practice a lot to have experience and lessons needed to produce accurate test results.