Permutation Test Stata Calculate and Summarize Calculator

Calculator Inputs

Group A Values

Group B Values

Statistic

Tail Direction

Permutation Mode

Random Repetitions

Random Seed

Alpha Level

Output

Example Data Table

Row	Control Value	Treatment Value
1	10	12
2	11	15
3	9	14
4	13	18
5	12	20
6	8	17
7	14	16
8	11	19

Formula Used

The calculator first computes an observed statistic from the two original groups.

Observed statistic: Tobs = S(Group A) - S(Group B)

For each permutation, all values are pooled. Labels are then reassigned. The same statistic is computed again.

Two sided p value: count(|Tperm| >= |Tobs|) / total permutations

Right tail p value: count(Tperm >= Tobs) / total permutations

Left tail p value: count(Tperm <= Tobs) / total permutations

For random simulation, the calculator uses a plus-one adjustment: p = (extreme + 1) / (repetitions + 1).

How to Use This Calculator

Paste numeric values for Group A and Group B.
Use commas, spaces, tabs, lines, pipes, or semicolons between values.
Choose the statistic you want to test.
Select two sided, right tail, or left tail testing.
Choose exact mode for small data, or random mode for larger data.
Enter repetitions, seed, and alpha level.
Press the calculate button.
Download the CSV or PDF summary when needed.

Understanding the Calculator

A permutation test is a direct resampling method. It asks a clear question. Would the observed group difference look unusual if labels were random? The calculator answers that question with exact or simulated shuffles.

Why Permutation Testing Helps

Many tests rely on distribution rules. Those rules can fail with small samples. They can also fail with skewed values. A permutation test uses the observed data itself. It keeps the values fixed. It only rearranges group membership. This makes the method useful for classroom work, audits, experiments, and quick checks.

What This Tool Summarizes

The tool compares two samples. You can paste control values and treatment values. You can choose a statistic. Mean difference is common. Median difference is robust. Sum difference is useful for totals. A Welch style statistic standardizes the mean difference. The page then reports group counts, averages, medians, spreads, the observed statistic, and a permutation p value.

Exact Versus Simulated Runs

Exact mode checks every possible reassignment when the data set is small enough. That gives a complete reference distribution. Random mode uses many shuffled samples. It is faster for larger data. More repetitions usually reduce simulation noise. A fixed seed makes results repeatable.

Reading the P Value

The p value measures how often shuffled data produced a statistic as extreme as the observed statistic. A small value suggests the observed grouping is not easily explained by random labels. It does not prove a cause. Good design, clean data, and subject knowledge still matter.

Practical Workflow

Start with the example data. Then replace it with your values. Select the statistic and tail direction. Use two sided testing when either direction matters. Use right or left tests only when the direction was planned before analysis. After calculation, download the CSV or PDF summary. Save the table with your notes, so the result stays reproducible.

Stata Style Thinking

Stata users often think in commands, returned statistics, and summaries. This page follows that habit. It shows the observed statistic first. Then it shows the resampling distribution. It also lists tail choice, repetitions, seed, and alpha. These details help you compare runs and explain the calculation in reports. It is not a replacement for careful statistical judgment alone.

FAQs

What is a permutation test?

A permutation test is a resampling test. It compares the observed statistic with statistics created by rearranging group labels. It is useful when normal assumptions are weak.

Can this replace formal software output?

It can support learning, checking, and reporting. For regulated or published work, confirm results with your approved statistical workflow and documented data rules.

Which statistic should I choose?

Use mean difference for average effects. Use median difference for skewed data. Use sum difference for totals. Use Welch style t when scale-adjusted mean difference is preferred.

What does two sided mean?

Two sided testing checks whether the observed result is extreme in either direction. Use it when both positive and negative differences matter.

When should I use exact mode?

Use exact mode for small samples. It checks every possible reassignment. The calculator switches away from exact mode when combinations become too large.

Why use a random seed?

A seed makes simulated results repeatable. The same data, settings, repetitions, and seed should return the same simulated summary.

What is the p value here?

It is the share of shuffled results that are at least as extreme as the observed statistic, based on the selected tail direction.

Why export CSV and PDF?

CSV is useful for spreadsheets and records. PDF is useful for sharing a clean summary. Both help preserve your settings and result.