Permutation Confidence Interval Calculator

Calculator

Study design

Controls which permutation scheme is used.

Statistic

Choose what you want an interval for.

Confidence level

Use 0.95 for a 95% interval.

Permutations

Higher counts reduce Monte Carlo noise.

Interval type

One-sided gives a bound plus infinity.

Alternative for p-value

Does not change the interval computation.

Trim proportion

Used only for trimmed-mean statistics.

Random seed

Set for reproducible permutations.

Paired input mode

Differences use sign-flip permutations.

Group A

Example: 12.1, 10.9, 11.4

Group B

Example: 10.2, 9.9, 10.5

Example data table

This sample shows two independent groups. Click “Use example data” to load it into the calculator.

Group A	Group B
12.1	10.2
10.9	9.9
11.4	10.5
13	10.7
12.7	11
11.8	9.8
12.3	10.1
13.2	10.6

Formula used

Let θ̂ be your chosen statistic (for example, a mean difference). Build a permutation distribution T* by repeatedly reshuffling labels (or flipping signs for paired differences) under exchangeability.

With confidence level 1−α, compute permutation quantiles q_α/2 and q_1−α/2. The interval is computed by inversion:

CI = [ θ̂ − q_1−α/2(T*) , θ̂ − q_α/2(T*) ]

Lower one-sided: θ̂ − q_1−α(T*) • Upper one-sided: θ̂ − q_α(T*)

This is a practical, permutation-based approximation to an interval for the true effect; it works best when the permutation distribution is centered near zero and reasonably symmetric.

How to use this calculator

Pick a study design: independent groups, paired data, or correlation.
Choose a statistic you want an interval for.
Paste numbers using commas, spaces, or new lines.
Set confidence level and permutations, then press Submit.
Download the summary as CSV or PDF for reporting.

FAQs

1) What is a permutation confidence interval?

It uses random reshuffling consistent with your design to approximate uncertainty around a statistic, then converts permutation quantiles into an interval around the observed effect.

2) When should I prefer this over a t-interval?

When sample sizes are small, distributions are non-normal, or you want inference tied to the randomization mechanism, permutation-based methods can be more robust and transparent.

3) How many permutations are enough?

5,000 to 20,000 is a good start. If your bounds change noticeably between runs, increase the count. Seeds help reproduce results for reports.

4) Does the p-value match the interval?

The p-value comes from the same permutation distribution, but it targets a null hypothesis. The interval here is an inversion-style approximation and may not perfectly align for all statistics.

5) What assumptions are still required?

You need exchangeability under the design: label shuffling should be valid for independent groups, and sign flips should be valid for paired differences.

6) Can I use medians or trimmed means?

Yes. These can be more resistant to outliers, but they run slower with large permutation counts because they require sorting within each resample.

7) What does “Hedges' g” mean?

It is a small-sample corrected standardized mean difference. It behaves like Cohen’s d, but applies a correction to reduce bias when samples are small.

8) Why can the interval include infinity?

That happens when you select a one-sided interval type. One-sided bounds report only a lower or upper limit, leaving the other side open.