How does it work?
Is it really A/B testing?
Technically, Birdy uses a method called "switchback testing". Instead of bucketing a profile visitor to experiment A or B, Birdy alternates between "treatment" and "control" periodically. In other words, Birdy buckets groups of profiles visitors over time, rather than individual visitors. This approach of split testing has initially been popularized by DoorDash.
In many ways, switchback testing is exactly like A/B testing, but instead of randomizing based on something like deliveries, we randomize based on time-region units. —DoorDash
How does the "statistical significance" meter work?
The meter's progress is defined by a time and a profile clicks threshold. After a certain amount of time and a certain amount of profile clicks, the results are considered statistically significant by Birdy 😊 These thresholds are somewhat subjective, but a good baseline nonetheless. You can choose to leave your test longer for an even greater chance at statistical significance 💪