Calculator Inputs
Plotly Graph
The chart tracks how family-wise false positive risk rises as the number of tests increases.
Example Data Table
These sample rows show how multiple testing can inflate false positive risk.
| Scenario | Alpha | Tests | FWER | Bonferroni Alpha | Expected False Positives |
|---|---|---|---|---|---|
| Clinical pilot | 0.0500 | 1 | 5.00% | 0.0500 | 0.0500 |
| A/B variants | 0.0500 | 5 | 22.62% | 0.0100 | 0.2500 |
| Feature screening | 0.0100 | 10 | 9.56% | 0.0050 | 0.1000 |
| Genome panel | 0.0500 | 20 | 64.15% | 0.0025 | 1.0000 |
Formula Used
1) Type One error probability
Alpha = P(reject H0 | H0 is true)
2) Confidence level
Confidence level = 1 − alpha
3) Family-wise error rate for independent tests
FWER = 1 − (1 − alpha)m, where m is the number of tests.
4) Expected false positives
Expected false positives = m × alpha
5) Bonferroni correction
Adjusted alpha = target FWER / m
6) Sidak correction
Adjusted alpha = 1 − (1 − target FWER)1/m
7) Critical z values
One-tailed z = z1−alpha and Two-tailed z = z1−alpha/2
How to Use This Calculator
- Enter your chosen significance level alpha.
- Set the number of independent tests in your study.
- Enter a target family-wise error rate for correction methods.
- Select one-tailed or two-tailed testing.
- Optionally enter an observed p-value to review significance decisions.
- Choose how many test counts the graph should display.
- Press the calculate button to place results above the form.
- Use the CSV or PDF buttons to export the results.
FAQs
1) What is a Type One error?
A Type One error happens when you reject a true null hypothesis. In practice, it is a false positive. The alpha level sets the probability of making that mistake for one test.
2) Why does multiple testing matter?
Running many tests raises the chance of getting at least one false positive. Even when each test uses the same alpha, the combined family-wise risk can become much larger.
3) What does family-wise error rate mean?
Family-wise error rate is the probability of making at least one Type One error across a set of related tests. It helps you evaluate overall risk instead of only per-test risk.
4) When should I use Bonferroni correction?
Use Bonferroni when you want a simple and conservative adjustment for multiple comparisons. It controls overall false positive risk well, though it can reduce sensitivity for real effects.
5) How is Sidak different from Bonferroni?
Sidak is slightly less conservative when tests are independent. Both methods control family-wise error, but Sidak usually gives a marginally larger adjusted alpha than Bonferroni.
6) What does the observed p-value section do?
It compares your entered p-value with the raw alpha, Bonferroni alpha, and Sidak alpha. That gives a fast view of whether significance remains after multiple-testing correction.
7) Why are one-tailed and two-tailed critical values shown?
They help interpret the threshold in z-score form. One-tailed tests place all alpha in one tail, while two-tailed tests split alpha across both tails.
8) Does this calculator assume independent tests?
Yes. The family-wise error curve uses the independence formula. Correlated tests can change the true overall risk, so results should be interpreted as a clean analytical approximation.