Example Data Table
| Scenario |
Tests |
Alpha |
Target FWER |
Method |
Example p-values |
| Clinical endpoints |
5 |
0.05 |
0.05 |
Holm Bonferroni |
0.004, 0.018, 0.031, 0.070, 0.210 |
| Survey comparisons |
12 |
0.05 |
0.05 |
Bonferroni |
0.002, 0.009, 0.026, 0.055 |
| Independent lab tests |
8 |
0.01 |
0.05 |
Sidak |
0.001, 0.006, 0.014, 0.022 |
Formula Used
Independent family wise error rate:
FWER = 1 - (1 - α)m
Bonferroni upper bound:
FWER ≤ m × α
Bonferroni adjusted alpha:
αBonferroni = αfamily / m
Sidak adjusted alpha:
αSidak = 1 - (1 - αfamily)1 / m
Holm Bonferroni threshold:
Sorted pi is compared with αfamily / (m - i + 1).
How to Use This Calculator
Enter the total number of planned tests. Add the effective number of tests when tests are correlated. Use the same value as the total count when you do not have an estimate.
Enter the per-comparison alpha for the uncorrected testing plan. Then enter the target family wise error rate. Most confirmatory studies use 0.05, but your field may require a different value.
Select a correction method. Paste p-values separated by commas, spaces, or new lines. Press calculate. Review the estimated FWER, adjusted alpha values, adjusted p-values, and reject decisions.
Use the CSV and PDF buttons to save the calculated result for reports, appendices, or review notes.
Family Wise Error Rate in Multiple Testing
Family wise error rate, often called FWER, measures the chance of making at least one false positive decision across a group of tests. It matters when a study checks many hypotheses at the same time. A single alpha level may look safe for one test. Yet repeated testing can raise the overall false alarm risk quickly.
Why Control Matters
Researchers often compare treatments, groups, genes, survey items, or model coefficients. Each added test creates another chance to reject a true null hypothesis. Without control, a table of many small p-values may look stronger than it really is. FWER control helps keep the whole analysis honest. It is most useful when even one false claim would be serious.
Common Correction Methods
The Bonferroni method divides the target family alpha by the number of tests. It is simple and conservative. The Sidak method assumes independent tests and gives a slightly larger threshold. Holm Bonferroni sorts p-values and applies step-down limits. It usually has better power than plain Bonferroni while still controlling FWER.
Independent and Correlated Tests
The exact independent formula is one minus the probability that every test avoids a type one error. That is why the risk grows with more tests. Real tests may be correlated. Positive correlation can reduce the effective number of separate tests. The calculator includes an effective test count so analysts can explore this situation carefully. Use it as a planning aid, not as automatic proof.
Interpreting Results
The adjusted alpha shows the largest p-value allowed for each comparison under a chosen family goal. Adjusted p-values show how each result changes after correction. A rejected result should still match the study design, assumptions, and practical context. Statistical significance is not the same as importance.
Good Reporting Practice
Report the number of tests, the planned family, the correction method, and the target FWER. Explain whether tests were planned before seeing the data. Keep exploratory findings separate from confirmatory findings. This practice makes your work clearer and easier to review. Use this tool during study planning and final reporting. It supports quick sensitivity checks, so teams can see how decisions change when test counts or family goals change before publishing final results.
FAQs
What is family wise error rate?
Family wise error rate is the chance of making at least one false positive conclusion across a group of related hypothesis tests.
Why does FWER increase with more tests?
Each test adds another opportunity for a type one error. Even small alpha values can create a large overall false positive risk when repeated many times.
When should I use Bonferroni correction?
Use Bonferroni when you need a simple and conservative correction. It works without strong independence assumptions, but it may reduce statistical power.
When is Sidak correction useful?
Sidak correction is useful when tests are independent or close to independent. It is slightly less conservative than Bonferroni for the same family target.
What makes Holm Bonferroni different?
Holm Bonferroni sorts p-values and tests them step by step. It often rejects more true effects than plain Bonferroni while controlling family wise error.
What is an effective number of tests?
It is an estimated count of independent tests after considering correlation. It can help with sensitivity checks, but it should be justified carefully.
Can I use this for exploratory research?
Yes, but label exploratory results clearly. FWER control helps reduce false positives, yet exploratory findings still need confirmation in later studies.
Does a corrected p-value prove importance?
No. It supports statistical evidence under a method. Practical size, study quality, assumptions, and subject knowledge still matter.