Swrve’s resource A/B testing functionality enables you to test specific aspects of your app by comparing how a section of your app’s user base responds to variants of in-app resources. In Swrve, a resource is a set of key/value attributes that represents, for example, a purchasable item, configuration settings, or any other aspect of your application that you want to test (for more information, see What is a Swrve resource?).  One of the main features of Swrve’s resource A/B testing functionality is the ability to target any particular segment of users that you want. You can also target users by their app version.

How resource A/B tests work in Swrve

In short, resource A/B tests work by serving one group with an unmodified resource (called the control) and serving the other groups with modified resources (which are called variants). An example is a resource called House which has the key/value attribute of cost:100. You might want to create an A/B test that changes cost:100 to cost:200 for 70% of your user base. In this example, 30% are in the control group and see no changes, while the other 70% are in the variant and receive the new value (cost:200).

You can create two types of resource A/B tests in Swrve:

• Conversion Tests – these tests enable you to compare variants based on the proportion of users who send a given event (the conversion event). In conversion tests, each user can contribute at most once to the success of that variant.
• Engagement Test – these tests enable you to compare variants based on the total number of times users have sent an event (the counting event). Take the example of an app with a social feature whereby users can share the results of multiple achievements with their friends. You might want to test various calls to action which differ in terms of strength (such as SHARE NOW! versus Why not share with friends?) to see which variant encourages the most repeat sharing by your users. You can also use engagement tests to perform single item virtual currency price sensitivity tests. If you have a common item in your app that is bought often by your users for a fixed virtual currency price, you might want to test whether increasing the price removes more virtual currency from the economy. By performing an engagement A/B test, you can determine whether users keep buying the item at the same rate per session or over their lifetime after a price increase. Swrve recommends that you use engagement tests on segments of users who have already converted (that is, who have already performed the action in question at least once).

You can test multiple resources as part of a single A/B test. This feature enables you to create a single conceptual test over multiple Swrve resources; for example, you might want to test the effect of increasing the price of a class of resources or you might want to adjust multiple aspects of the app’s starting flow and the configuration of this flow is shared over multiple Swrve resources.

Swrve uses a Bayesian approach to A/B testing. For detailed information about the Bayesian testing model in Swrve, see Why use a Bayesian approach to A/B testing?

Managing resource A/B tests

Swrve provides two main views to enable you to manage resource A/B tests:

• The Resource A/B Testing screen provides an overview of all your resource A/B tests.
• The Resource A/B Test Details screen provides a detailed view of a particular resource A/B test.

To view the Resource A/B Testing screen, on the Optimization menu, select Resource A/B testing.

You can perform the following actions on the Resource A/B Testing screen:

• Create a new resource A/B test. For information about creating resource A/B tests, see Creating resource A/B tests.
• Delete a resource A/B test in Staging status by means of the delete icon.
• Archive a resource A/B test in Aborted, Finished, or Finished, but changes of winning variant not yet applied status by means of the archive icon.
• Create a copy of a resource A/B test in Aborted or Finished status by means of the clone icon.

For information about archiving and deleting resource A/B tests, see Aborting and deleting A/B tests.

Resource A/B test status

The A/B test status can be one of the following:

• Scheduled – the A/B test has been scheduled but has not yet been started.
• Staging – the A/B test has been created but has not yet been started.
• Started – the A/B test has been started.
• Aborted – execution of the A/B test has been stopped.
• Archived – the A/B test has been archived.
• Finished and control chosen – the A/B test has been finished and the control has been selected as the winner.
• Finished, but changes of winning variant not yet applied – the A/B test has been finished and the variant has been selected as the winner but the changes have not yet been applied (app resources have not been updated to match the resources of the winning variant).
• Finished and changes of winning variant applied – the A/B test has been finished, the variant has been selected as the winner and the changes have been applied (app resources have been updated to match the resources of the winning variant).

Resource A/B tests details

To view the Resource A/B Test Details screen for a particular test, select the A/B test name on the Resource A/B Testing screen.

The Resource A/B Test Details screen enables you to edit, monitor and manage your resource A/B test. The A/B Test Progress section enables you to monitor your test and access KPI data so you can best decide how to select a winning variant, if you haven’t already done so. For more information, see Creating resource A/B tests.

The actions you can perform on the Resource A/B Test Details screen depend on the A/B test status:

• Edit or start an A/B test in Staging status.
• Unschedule an A/B test in Scheduled status, returning the test to Staging status.
• Abort or finish an A/B test in Started status.
• Archive or create a copy of an A/B test in Aborted status.
• Delete a test in Archived status.
• Archive or create a copy of an A/B test in Finished and control chosen status.
• Apply the changes to an A/B test in Finished, but changes of winning variant not yet applied status.
• Archive or create a copy of an A/B test in Finished and changes of winning variant applied status.
• Assign a QA device (for testing purposes) to an A/B test in Staging or Started status.

Prerequisites

Before you can create resource A/B tests in Swrve, you and your development team must complete the following prerequisites:

• Integrate your app to enable A/B testing. For A/B test integration information for developers, see the platform-specific integration guide.
• If you want to be able to target users by app version for a Unity app, your development team may need to set the application version when initializing the Unity SDK (as it is not always automatically set). The app version is automatically set on iOS and Android.
• To create a resource A/B test, you must set up the resources on which you want to base the test, as detailed in Creating resource A/B tests. Swrve enables you to create resources manually or by means of the Swrve Items API. Although you can manually add a resource on the Resources screen whenever you like, your development team must configure the resource and its associated attributes in the code before you can use it as the basis for an A/B test.

Next steps

• View A/B testing best practices. For more information, see A/B testing best practices.
• Create a resource A/B test. For more information, see Creating resource A/B tests.
• Find information about targeting users in A/B tests and running tests in parallel with no overlap of users. For more information, see Targeting users in A/B tests.
• QA test an A/B test. For more information, see QA testing A/B tests.