Calculator Inputs
Formula Used
Unadjusted family wise error rate:
FWER = 1 - (1 - alpha)m
Bonferroni corrected alpha:
alphaBonferroni = target family alpha / m
Sidak corrected alpha:
alphaSidak = 1 - (1 - target family alpha)1 / m
Expected false positives:
Expected false positives = true null tests × per-comparison alpha
Holm threshold:
For ordered p-value p(i), threshold = target family alpha / (m - i + 1)
How to Use This Calculator
- Enter the total number of planned hypotheses or comparisons.
- Enter the per-comparison alpha used by each single test.
- Enter the desired family level alpha, often 0.05.
- Enter the expected proportion of true null hypotheses.
- Add optional p-values separated by commas, spaces, or new lines.
- Press Calculate to view the result above the form.
- Use CSV or PDF download buttons to export the same calculation.
Example Data Table
| Scenario | Tests | Single-test alpha | Unadjusted FWER | Bonferroni alpha | Sidak alpha |
|---|---|---|---|---|---|
| Small study | 5 | 0.05 | 0.226219 | 0.01 | 0.010206 |
| Medium study | 10 | 0.05 | 0.401263 | 0.005 | 0.005116 |
| Large screen | 25 | 0.05 | 0.72261 | 0.002 | 0.00205 |
Understanding Family Wise Error Rate
Family wise error rate measures the chance of making at least one false positive across a family of related tests. A single test may use alpha of 0.05. Many tests raise the total risk. Ten independent tests at 0.05 create a much larger family risk than one test.
Why It Matters
Multiple testing appears in clinical trials, surveys, experiments, quality checks, and machine learning feature screens. Each comparison can wrongly reject a true null hypothesis. When the same study reports many comparisons, readers need one family level error view. FWER control keeps the chance of any false claim near the chosen target.
How This Tool Helps
This calculator estimates unadjusted family risk with the independence formula. It also reports Bonferroni and Sidak per comparison limits. The Bonferroni method is simple and conservative. It divides the target family alpha by the number of tests. Sidak is slightly less conservative when tests are independent. Holm uses ordered p values. It gives stronger power than plain Bonferroni while still controlling the family rate.
Choosing Inputs
Use the number of planned hypotheses as the test count. Use the per comparison alpha for the raw test level. Use the target family alpha for the protection level, often 0.05. Enter p values when you want adjusted decisions. Keep all tests from the same analysis family together. Do not split a family only to make results pass.
Reading Results
The unadjusted FWER shows the cost of running tests without correction. The adjusted alpha values show stricter cutoffs. The expected false positives field is a simple planning estimate. It uses the proportion of true null hypotheses. The p value table compares raw, Bonferroni, Sidak, and Holm outcomes. A significant label means the p value survives that correction rule.
Good Practice
Plan the correction method before looking at results. Report the number of tests and the family definition. Mention whether the independence assumption is reasonable. Use FWER control when any false positive would be costly. For exploratory work, also consider false discovery rate methods. They answer a different question. Document every exclusion rule. Store exports with the analysis file. This helps reviewers repeat checks, confirm thresholds, and understand decisions later clearly.
FAQs
What is family wise error rate?
Family wise error rate is the probability of making at least one false positive across a related group of hypothesis tests. It grows as the number of tests increases.
Why is FWER higher than single-test alpha?
Each test carries its own false positive risk. When many tests are run together, those risks combine. The family risk can become much larger than the alpha for one test.
When should I use Bonferroni correction?
Use Bonferroni when you need a simple and conservative correction. It works under broad dependence conditions, but it may reduce statistical power when many tests are included.
When is Sidak correction useful?
Sidak correction is useful when tests are independent or close to independent. It is usually slightly less strict than Bonferroni while still targeting the chosen family error rate.
What does Holm correction do?
Holm correction sorts p-values from smallest to largest. It applies step-down thresholds. It often gives more power than Bonferroni while controlling family wise error rate.
Can I enter fewer p-values than planned tests?
Yes. The calculator still uses the planned test count for correction thresholds. This helps keep the family definition consistent with the original analysis plan.
What does expected false positives mean?
It estimates the average number of false positive findings among true null tests. It is a planning estimate, not a guarantee for a single study.
Does this calculator assume independent tests?
The exact FWER and Sidak formulas assume independent tests. Bonferroni and Holm are more conservative and are commonly used even when tests are not independent.