Last modified November 22, 2014 by Gordon Glauser

A/B Testing Push Notifications

Swrve’s push notification A/B testing feature enables you create up to 10 variations of a push notification and measure user response to different message content and actions. You can then track and compare which variants have higher engagement rates and adjust future campaigns accordingly.

A/B tested push notification

You can also create multilingual versions of a push notification and automatically target users of each language group using the Localize feature. However, it is not possible to A/B test a localized campaign at present—you can only select one or the other. For more information about creating a localized push notification campaign, see Localizing Push Notifications.

A/B Testing Push Notifications vs. Resources

The push notifications A/B testing feature is for testing variations of the content of your push notifications only. For information on A/B testing native app content using variants of different resources, see Intro to Resource A/B Testing.


How to A/B Test a Push Notification

The A/B test feature is available on the Add Content screen of the push notification campaign workflow. For more information on how to create basic push notifications, see Creating Push Notifications.

Step 1: On the Add Content screen, add the content of your control push notification as required (for example, text, emoji, media, buttons, alert sound and custom parameters).

Step 2: Click Add A/B Testing. The A/B test variant tabs are displayed and by default include a copy of the control push notification labeled as Variant B.

push notification A/B test

Step 3: To create additional variants:

  • Click the Add (+) icon next to the current variant. A copy of the current notification is created and the banner displays the percentage of the target audience that will see that notification. The target audience is divided evenly among the variants, based on the number of variants you create. For example, if you create four variants, 25% of the target audience sees each variant.
  • Click the edit icon or variant name to enter a specific name for the variant. The name is displayed on the tab above as you type.
  • Change the content of the notification variant as required (for example, different text, emojis, media, buttons, alert sound/no sound or custom parameters).
  • After you have created your notification variants, click Next: Set Targeting to proceed.

Step 4: Complete the rest of the campaign workflow as you would for a basic push notification, including setting campaign goals and checking each variant on a QA device. Note that the target audience is split evenly among the number of variants, however each variant includes the same campaign goals for all users.

Step 5: To view a preview of the push notification variants on the Preview & Test screen, click each variant tab. If you are happy with the preview of your push notification, schedule and launch the campaign as you would for a basic push notification.


Next Steps