How do I A/B test campaigns with a control group?
After creating a Resource A/B test with a control group, target the variants in your marketing campaigns. This enables you to compare the effectiveness of your campaign(s) against a true control group.
How to include a control group in campaigns
Step 1: In the Campaigns center, select Create campaign.
Step 2: On the Choose your marketing channel screen, select In-app message as your campaign channel, then select Continue.
Step 3: Complete the Overview and Add Content screens of the in-app message workflow (for more information on completing these screens, see Creating in-app messages).
Step 4: On the Set Target screen, under the Define Target Audience section, select Custom Audience and then define the following values:
- Under User Profile Filters, select Resource Tests.
- Select the test from the list of available resource A/B tests.
- In the next list, select have been exposed to.
- Select one of the non-control variants you created (for example, Activities Page).
- Note: When creating your A/B Resource test, the control population size should be set to ‘100’ minus the percentage of users you want to expose to the A/B test campaigns. For example, if you want to expose 20 percent of your user base to the campaigns, set the control population size to ’80’.
- To add the audience filter, select Apply filter.
Step 5: In the Set Display Triggers section, select the event that you want to trigger the in-app message (for example, Session Start or activity_page_opened).
Step 6: In the Select Campaign Goals section, select any additional campaign goal you want to track.
Step 7: Select Save as Draft: Preview & Test to proceed, and then schedule and launch the campaign as required.
Step 8: Create and launch another in-app message campaign. Follow the same steps, but this time use the non-control variant you didn’t use in Step 4 (Messages Page).
Step 9: After users have had a chance to engage with the app, return to the Resource A/B Testing screen and select the A/B test you created to compare how each of the variants are performing.
