A/B Testing for Search

This article will answer questions about our A/B testing functionality for Search, including prerequisites, caching considerations and supported triggers.  

 

What value does A/B testing for Search add?

A/B testing functionality for FHR Search and AI Search gives customers the ability to test and optimize search configurations and measure the impact these changes have on conversion. 

In FHR, search configuration is controlled within a search profile. Two separate search profiles can be configured. One profile will remain unchanged as a control. The other profile will use new or modified search passes: utilizing new technology, data, or weightings to complete the search.

 

What are the prerequisites for A/B testing for search?

Fredhopper Access Server (FAS) 21.1 & Insights + Common Activity Data Pipeline (CADP) + Installation of Attraqt’s variant attribution model into your backend system (customer work required)

 

What is the effect on caching?

Caching is typically a blocker to A/B testing as if ‘A’ is cached, typically ‘B’ will not show. We’ve developed a solution to this, enabling customers to retrieve and store the running A/B tests so that multiple backend caches can be created for the pages running tests. A/B testing will still increase the cache, but only on pages that are being A/B tested and at most will double.

The only limitation to this is that some customers like to do CDN or other front-end caches. They can continue to do so, however, any Fredhopper request would have to be made available in their backend, where Attraqt’s variant attribution model can assign the traffic.

 

What trigger types are supported?

• Universe (applied to the parent scope/s)
• Locale (applied to the parent scope/s)

 

What trigger types are not supported?

• Keyword trigger
• Coverage trigger

 

How often do the reports update?

The reports update twice a day. The time of the last update is highlighted at the top of the report ("Data was last refreshed at:").

 

How long after publishing before an A/B test appear on my front end?

This is a choice made by you, the customer, at point of integrating. This can be set to update every 5 minutes (as advised in the documentation). If set to update at 5-minute intervals, that would be the maximum you would have to wait.