Leveraging A/B/n testing has long been the gold standard for developing sharper, more effective marketing campaigns. Using control groups helps marketers measure the effect of given variables—such as messaging, design, and CTA placement—on overall campaign performance, generating insights that can be used to optimize campaigns and fuel gains in customer engagement, conversions, and revenue.

As the future of experimentation keeps evolving and savvy brands are leaning into predictive modeling to step up their marketing testing practice, Braze has introduced another method that’s even more advanced than traditional A/B and multivariate testing: Braze Personalized Variant.

What Is Braze Personalized Variant?

Typical A/B and multivariate tests take a one-size-fits-all approach to determine “winning” variables or combinations of variables (i.e. variants), identifying the one variable or variant that has the potential for the greatest impact in the aggregate. Braze Personalized Variant, on the other hand, enables brands to adopt a personalized approach to campaign optimization, making it possible to carry out a more sophisticated marketing personalization strategy.

Braze Personalized Variant makes it possible to:

#1: Determine the winning campaign variant for each individual user at the personal level, not just the aggregate winning variant.

#2: Send the right campaign to the right recipient based on their individual likelihood to engage.

#3: Increase the likelihood of engagement and conversions at the 1:1 level compared to traditional A/B and multivariate testing.

#4: Potentially drive even greater engagement and conversions at scale than traditional A/B and multivariate testing, thanks to incremental gains at the 1:1 level.

How Does Braze Personalized Variant Work?

Typical A/B and multivariate testing involves marketers distributing a test campaign—made up of one or more test variants along with a control campaign—to a percentage of a given segment of users during a set testing window. Ideally, this testing window is at least 24 hours long in order to ensure statistically significant and meaningful results. Within the Braze platform, marketers also get the chance to choose the metric they want to optimize the campaign for, such as unique opens, unique clicks, or a primary conversion event.

After the testing period ends, the winning variant is identified—within the Braze platform, this is the version that outperforms all of the others (i.e. generates more of the preferred metric) with a better than 95% confidence level. This winning version then gets sent to the remainder of the recipients of that segment.

Step 1: Select segment.

Step 2: Send “control” campaign to X% of segment; send A, B, etc., variants to additional percentages of segment.

Step 3: Send the “winning” segment to the remainder of the audience.

With Personalized Variant, instead of sending the “winning” segment to the remainder of the audience, each user receives the variant they’re most likely to engage with.

With Personalized Variant test analytics available in the Braze Campaigns dashboard, marketers can see the positive impact that sending a Personalized Variant can have on the campaign’s performance.

• Projected lift: Shows the anticipated percentage improvement associated with using a Personalized Variant (rather than the winning variant) in the selected optimization metric associated with this send.

• Overall results: Reveals the performance of the selected optimization metric as a result of sending the Personalized Variant.

• Projected results: Estimates what the results of the selected optimization metric would have been if the winning variant were to have been sent instead.

How Does Braze Determine the Best Variant for Each Customer?

During the initial testing period of a given campaign, Braze determines each user’s Personalized Variant by looking for associations between user characteristics and message preferences. Braze evaluates how customers respond to each variant (i.e. whether they engage, convert, etc.) used in the test and then identifies which of the remaining users are most likely to exhibit similar behavior and then sends the variant most likely to result in the desired action.

Personalized Variants are determined based on a combination of characteristics, including:

• Recency: The date of the customer’s last session

• Frequency: How often a given customer has sessions

• Tenure: How long the campaign recipient has been a customer

Patterns of recency, frequency, and tenure help shape a customer’s Personalized Variant.

If no Personalized Variant is found during a test period, marketers have the choice between sending the winning variant instead or simply ending the experiment.

Key Benefits of Braze Personalized Variant

By using Braze Personalized Variant instead of traditional A/B and multivariate testing methods, marketers may be able to:

1. Develop more effective marketing campaigns that are more likely to resonate at the 1:1 level.

2. Improve marketing personalization efforts for customers.

3. Improve critical customer engagement KPIs, such as conversions and revenue.

Interested in learning more about how to make the most out of your customer journeys? Check out our orchestration guide for insight into how to build a truly coordinated messaging flow that speaks to your customers and supports your key business goals.