Calculator Form
Formula Used
Raw comparisons: N = hypotheses × planned looks
Effective comparisons: Ne = 1 + (N - 1) × (1 - dependence)
No correction: adjusted alpha = alpha
Bonferroni: adjusted alpha = alpha / Ne
Sidak: adjusted alpha = 1 - (1 - alpha)^(1 / Ne)
Familywise Type I error rate: FWER = 1 - (1 - adjusted alpha)^Ne
Expected false positives: true null tests × adjusted alpha
How to Use This Calculator
Enter the alpha level planned for the study. Add the number of hypotheses or comparisons. Enter planned looks if the data will be checked more than once. Pick a correction method. Use dependence when tests are related. Press calculate to view adjusted alpha, familywise risk, expected false positives, and critical z value.
Example Data Table
| Scenario | Alpha | Tests | Looks | Method | Approximate FWER |
|---|---|---|---|---|---|
| Single planned test | 0.05 | 1 | 1 | No correction | 0.0500 |
| Five independent tests | 0.05 | 5 | 1 | No correction | 0.2262 |
| Five controlled tests | 0.05 | 5 | 1 | Bonferroni | 0.0490 |
| Twenty controlled tests | 0.05 | 20 | 1 | Sidak | 0.0500 |
Understanding Type I Error Rate
A Type I error happens when a test rejects a true null hypothesis. It is also called a false positive. In practice, the chosen alpha level is the planned chance of making this error for one test. A smaller alpha gives stronger protection, but it can also make real effects harder to detect.
Why The Rate Matters
Researchers often run more than one comparison. Each extra comparison adds another chance to find a false signal. When many true null hypotheses are tested, the family error rate can rise quickly. This is why a study with ten tests at 0.05 does not have only a five percent overall false positive risk.
Corrections And Study Design
Common corrections reduce the per comparison alpha. Bonferroni divides the target alpha by the number of comparisons. Sidak uses a probability formula and assumes independence. Both methods protect the study better than using the same alpha repeatedly. They can be conservative when tests are strongly related.
Interim looks also matter. A trial checked many times has more chances to stop early by chance. This calculator treats planned looks like additional opportunities for error. It also lets you enter an average dependence value. Higher dependence lowers the effective number of comparisons in the estimate.
Reading The Results
The adjusted alpha is the limit used for each comparison. The familywise Type I error rate is the chance of at least one false positive across the study. Expected false positives estimate the average number of false findings among true null tests. The critical z value shows the cutoff for a normal test.
Good Use Of Alpha
Alpha should be planned before analysis. It should match the cost of a false positive. Medical safety studies may need strict control. Early screening work may accept a larger rate. The best choice depends on risk, sample size, and scientific purpose.
This tool is for planning and education. It does not replace a full protocol or expert review. Use it to compare designs before data collection. Save results with the export buttons.
Report assumptions clearly. Readers should know which tests were planned, which correction was selected, and whether repeated looks were included in the final error estimate.
FAQs
What is a Type I error?
A Type I error is a false positive. It occurs when a statistical test rejects a null hypothesis that is actually true.
Is alpha the same as Type I error rate?
For one planned test, alpha is the planned Type I error rate. For many tests, the overall error rate can become larger.
What does familywise error rate mean?
Familywise error rate is the chance of getting at least one false positive across a group of tests or comparisons.
Why does the calculator include planned looks?
Each planned look gives another chance to reject the null by chance. Repeated looks can increase false positive risk.
When should I use Bonferroni correction?
Use Bonferroni when you need simple and strict control across several comparisons. It is easy but can be conservative.
When should I use Sidak correction?
Sidak is useful for independent or near independent comparisons. It is often slightly less conservative than Bonferroni.
What does dependence mean here?
Dependence estimates how related the tests are. Higher dependence reduces the effective comparison count in this calculator.
Can this replace a statistical plan?
No. It supports planning and teaching. Formal research should still use a complete protocol and qualified statistical review.