How to create an A/B test?

Introduction

Welcome! This article will provide you with a simple step-by-step guide to creating an AB test in the EB control panel.

 

What is XO A/B Testing? 

With XO, build your personalisation strategies in a test and learn approach.

Our native A/B Testing tool allows you to create and manage tests on all aspects of your personalisation (models, algorithms, widgets) to deploy performance-focused strategies.

MODEL LEVEL	Try various algorithm configurations and define the relevant data to feed it.
ALGORITHM LEVEL	Evaluate the best algorithm to use: collaborative filtering, content-based, graph... And define the most adapted strategy for each use case.
MERCHANDISING LEVEL	Identify merchandising rules that generate additional revenue and enhance each rule to optimise your strategies and focus on what creates value.

 

Prerequisites

Before we start, make sure you have access to the XO control panel and that your items feed is available in XO. If you do not have access you can request access by clicking here or choosing “I/my colleague require access to an Attraqt service” from the use case drop down on the “Submit a request” page in the Support Centre. For security reasons, we require access for 3rd parties to be requested by a member of the organisation whose control panel access is being requested for. This is to ensure the organisation has granted access to the control panel.

 

Scenario

We will base the scenario on a specific example, using an XO demo account. 

Here, our brand will sponsor London Marathon so I’m expecting a higher volume of traffic in the upcoming days that may be interested in my Running products. Best way to understand user preferences is to test a new strategy forcing running products on my home page versus the current one which is based on EB most popular algorithm. 

 

Step-by-step guide

Step 1

To access the A/B Test interface, log into the XO console panel and click on 'Merchandising' then 'Widgets'.

Click on the A/B test icon on the widget that you wish to set up an A/B test for, then on 'Setup an A/B Test' to confirm the creation.

 

Step 2

Next we need to define test name and start and end dates. You can test different variations but on this guide, we will set up an AB test with a control tested against one variation. If you want to test more variations click on the green plus "button"

If the A/B Test was saved now it would start running at 00:01 on December 4th until manually stopped, because I didn’t define any end date. If you want it to automatically stop on a precise date you just need to add it on the “End Date”  box.

 

Step 3

On this test we are using as the control group the current widget strategy already in place in the merchandising tab. We just need to copy the existing configuration on Population 1.

 

Because we copied the default configuration there is no need to change the merchandising options settings. We just need to define a Population name to easily understand what we are testing once we will check the test performances:

For Population 1 we leave displayed title, template, location by default because we want to respect the default configuration.

 

Step 4

We now must set up the configuration for our second population. As we explained at the beginning of this guide we want to display Ruming products for the remaining 50% of the users.  

We need to define the population name and displayed title to be linked with the applied strategy. 

In order to keep the original widget template and location, we leave those options boxes by default.

 

Step 5

 Now let’s set the number of recommended products as 3 and to add the Running category TAG on Required tags box.

 

Step 6

The last step to complete the configuration of this AB test is to save it as the below explanation: 

 

Step 7

 The end result should look as follows:

 

Conclusion

Congratulations! You now know how to create an A/B Test in XO.

Let us know in the comment section if there are other test scenarios you would like us to create how-to guides for.

Enjoy your XO experience

Should you encounter any difficulty setting up an A/B Test of your own please don’t hesitate to contact us by clicking here or clicking the submit a case link on the Support Centre and choosing “I have a question” from the list of use cases.