What you are looking at
What
Controlled experiments that isolate the effect of a single change
on a key metric. Each user is randomly assigned to Control (A) or Variant (B) — everything else stays
identical so any difference in conversion is attributable to the change being tested.
Why
Recommendations and intuition tell you what might work.
A/B tests tell you whether it worked, with a measurable confidence level.
Statistical significance (p < 0.05) means there is less than a 5% probability
the observed difference is due to chance alone.
How to read results
Uplift % = relative improvement of B over A.
p-value = probability the result is random noise (lower = more confident).
Winner declared only when p < 0.05 AND the uplift direction aligns with
the business goal.
3
Active Experiments
3
Significant Results
72,000
Total Observations
3×
Tests Won by Variant B
95%
Confidence Threshold
AB-001: Conversion by Device
Price-first layout (B) outperforms image-first (A) on every device type.
Mobile shows the biggest relative uplift — users on small screens benefit most from seeing price
before scrolling through images.
AB-001: Weekly Conversion Trend
Variant B consistently leads throughout the test period. A stable,
consistent gap (not just a spike in one week) confirms the result is not driven by a
single seasonal event.
AB-003: App Retention Cohort
Price-anchor creative (B) acquires users with higher intent — they
know the price before installing, so they are less likely to churn after seeing it in-app.
Day-30 retention gap of ~5pp compounds significantly over the user lifecycle.
Uplift Summary Across All Tests
Each bar shows the relative uplift of Variant B over Control A on
the primary metric. All three are statistically significant (p < 0.05). Revenue uplift
for AB-002 is highest because it affects a high-value step (checkout completion).