Calculator Inputs
Use proportion mode for rates like conversion. Use continuous mode for metrics like revenue, time, or scores.
Example Data Table
| Scenario | Metric type | Baseline | Control n | Variant n | Alpha | Power | Approx. MDE |
|---|---|---|---|---|---|---|---|
| Homepage signup test | Proportion | 8.50% | 6,000 | 6,000 | 0.05 | 0.80 | 1.43 percentage points |
| Checkout conversion test | Proportion | 12.00% | 10,000 | 10,000 | 0.05 | 0.90 | 1.12 percentage points |
| Revenue per user test | Continuous | 120.00 | 4,500 | 4,500 | 0.05 | 0.80 | 2.92 units |
| Time-on-site experiment | Continuous | 5.40 | 3,500 | 3,500 | 0.01 | 0.90 | 0.37 units |
These rows illustrate typical planning inputs. Your actual result depends on the assumptions entered above.
Formula Used
General structure: MDE = (zalpha + zpower) × standard error
For proportions: SE = √[p(1-p)/nc + p(1-p)/nv] × variance adjustment
For continuous means: SE = s × √[(1/nc) + (1/nv)] × variance adjustment
Effective sample size: neffective = n × (1 - attrition) ÷ design effect
The chart estimates power across a range of candidate effects using the same test assumptions.
How to Use This Calculator
- Select whether your outcome is a proportion or a continuous mean.
- Enter the control and variant sample sizes you expect to analyze.
- Set alpha and target power for your study design.
- For proportions, enter the baseline rate percentage.
- For continuous metrics, enter the baseline mean and pooled standard deviation.
- Add any design effect, expected attrition, or variance reduction adjustments.
- Click Calculate MDE to see the smallest detectable effect.
- Use the download buttons to save a CSV or PDF summary.
Frequently Asked Questions
1. What is minimum detectable effect?
It is the smallest real difference your test setup can reliably identify at the chosen alpha and power. Smaller MDE values mean a more sensitive study design.
2. Why does a larger sample reduce MDE?
Bigger samples reduce sampling noise. Lower noise shrinks the standard error, which allows the test to detect smaller effects with the same error thresholds.
3. When should I use proportion mode?
Use proportion mode when the outcome is a rate or share, such as conversion, retention, click-through, completion, or defect rate.
4. When should I use continuous mode?
Use continuous mode for numeric averages such as revenue, duration, score, spend, order value, latency, or satisfaction rating.
5. What does design effect change?
Design effect reduces effective sample size when observations are not fully independent. Clustered, repeated, or correlated data often need this adjustment.
6. Why include attrition?
Attrition accounts for users or records lost before final analysis. A higher dropout rate lowers effective sample size and increases the detectable threshold.
7. What is variance reduction?
Variance reduction reflects techniques that make outcomes less noisy, such as covariate adjustment. Lower variance improves sensitivity and reduces MDE.
8. Should I use one-tailed or two-tailed tests?
Two-tailed tests are more conservative and common. One-tailed tests can detect smaller positive effects, but only when a one-direction hypothesis is justified before analysis.