A/B tests are a great way to accurately measure the impact that Chameleon is having and / or compare different versions of a product tour. 

There are a two ways this can be accomplished:

  • You can control where and for whom the Chameleon snippet is loaded. This will be managed by your developers. Learn more here.
  • You can use the 'Audience targeting' capability within Chameleon to target a proportion of users, randomly. See below. 

How it works

If you have Chameleon's A/B testing feature enabled (based on your plan), Chameleon will automatically append a random value ("Percentage value") between 0 and 100 to each of your users. This will be persistent and you can leverage this to target users.

How to create A/B test groups

To create a test group, simply add an extra filter ("sampling filter") to your Target Audience segment that selects a random sample of users of the desired size. 

This means you can still target users based on other conditions, such as user properties, events, data sources etc. 

To add the sampling filter:

  1. Select 'Default properties' as the type of filter
  2. Select 'Percentage' in the next dropdown
  3. Use 'more than' or 'less than' to define the range of users
  4. Set the boundary number for this "Percentage value"

In the above example, users that have a "Percentage value" between 50 and 100 would be targeted. This would constitute 50% of the users within the group defined by the other segment filters. 

To target 10% of users, you could use either:

  • Percentage more than 90
  • Percentage less than 10

You could also use multiple filters to target another 10% group (e.g. Percentage more than 10, and Percentage less than 20). 

You can use the same filter configuration in another segment (by re-creating the filter) to target the same user group. This means you can run multiple A/B tests to the same user group. 

How to A/B test tour variations

To test two variations of a tour:

  1. Create the control variation of the tour, including the target audience, with the sampling filter above
  2. Duplicate the control tour, update the tour name (using the variant name / label) and then re-create the target audience. This time use the opposite sampling filter, to target the alternative user group
  3. Set both tours live

You will be able to analyze the results through Chameleon, or within the analytics software you have connected. 

You will be able to use the tour name to segregate the data, as each Chameleon event contains additional attributes, such as Tour name, User ID, URL etc.

How to A/B test Chameleon vs. nothing

If you want to test a Chameleon tour against no tour, simply launch a single version to a sample group (using the same method as above). 

To know whether a user is included in the sample (regardless of whether or not they saw a tour) you can send the Chameleon "Percentage value" to your database / analytics solution, using a script like the below (with examples for Segment and Mixpanel):

chmln.on('after:profile', function() {
  // Now you have full access to chmln.data.profile
 
  var percent = chmln.data.profile.get('percent'));
  // Segment.com
  // analytics.identify({chameleon_testing_id: percent})

  // Mixpanel
  // mixpanel.people.set({chameleon_testing_id: percent})
})

You can easily adapt this based on where you'd like to send this data in your system.

You'll then be able to measure the impact on product activity based on the users that started a tour, using our conversion goal or by syncing Chameleon tour data with your product data through one of our native analytics integrations.

Alternatively the Chameleon team can help you run a custom test based on your existing user data. For help with this, just send us an email

Did this answer your question?