In an A/B test in customer.io, if there is an A/B test in an recurring campaign, would the users in the control group consistently be bucketed into the control group & similarly for the treatment group?
I couldn’t find any documentation on this.
Solved
A/B testing - are users split consistently?
Best answer by Ramy
Hello!
This is Ramy from the Customer.io Support team; happy to help you with your question here!
Jumping right into the answer, the distribution of people into each path when A/B testing as well as Random Cohort Branch workflow actions is truly random. This means that in a recurring campaign, profile journeys will be re-evaluated as per the random percentage set, and journeys are not guaranteed to continue down the same path each time.
I hope this helps! Please let us know if you’ve got any follow-ups we can help with. :)
Best,
Hello!
This is Ramy from the Customer.io Support team; happy to help you with your question here!
Jumping right into the answer, the distribution of people into each path when A/B testing as well as Random Cohort Branch workflow actions is truly random. This means that in a recurring campaign, profile journeys will be re-evaluated as per the random percentage set, and journeys are not guaranteed to continue down the same path each time.
I hope this helps! Please let us know if you’ve got any follow-ups we can help with. :)
Best,
To take Ramy’s response one step further, if you want to keep that a/b test consistent over time, just create an attribute that is tagged with the test, that way you can make sure it goes to the same group each time. A very easy work around to get what you need!
Reply
Most helpful members this week
Login to the community
No account yet? Create an account
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.