Enter Paired Data
Example Data Table
| Observation | X value | Y value |
|---|---|---|
| 1 | 1.2 | 2.0 |
| 2 | 1.8 | 2.1 |
| 3 | 2.1 | 2.4 |
| 4 | 2.5 | 2.7 |
| 5 | 3.0 | 3.2 |
| 6 | 3.4 | 3.5 |
| 7 | 4.1 | 4.2 |
| 8 | 4.5 | 4.4 |
This sample shows positively related numeric observations. Auto mode will usually switch to binned estimation for this pattern.
Formula Used
Mutual information measures how much uncertainty in one variable is reduced by knowing the other.
I(X;Y) = Σ p(x,y) × log [ p(x,y) / ( p(x) × p(y) ) ]
Entropy: H(X) = −Σ p(x) log p(x), and H(Y) follows the same form.
Joint entropy: H(X,Y) = −Σ p(x,y) log p(x,y)
Variation of information: VI = H(X) + H(Y) − 2I(X;Y)
Normalized mutual information can be scaled several ways, such as dividing by min(Hx, Hy), max(Hx, Hy), average entropy, or √(HxHy).
Binned estimator: when values are continuous, the calculator first converts each variable into bins, then estimates probabilities from the resulting contingency table.
How to Use This Calculator
- Paste paired X values into the first field.
- Paste paired Y values into the second field.
- Choose auto, discrete, or binned estimation.
- Select a bin rule when using numeric data.
- Choose your preferred log base and normalization method.
- Set decimal precision for the report.
- Click Estimate Mutual Information.
- Review the summary metrics, heatmap, and joint probability table.
- Use the CSV or PDF buttons to export the result.
FAQs
1. What does mutual information measure?
It measures how much knowing one variable reduces uncertainty about another. Unlike correlation, it can detect nonlinear and non-monotonic dependence patterns.
2. When should I use discrete mode?
Use discrete mode when both variables are already categories, labels, or small sets of repeated values, such as classes, outcomes, or encoded states.
3. When should I use binned mode?
Use binned mode for continuous numeric variables. The calculator groups values into intervals, then estimates probabilities from those grouped observations.
4. Why does the bin rule matter?
Different bin rules change interval width and count. That changes estimated probabilities, so mutual information values may shift slightly between rules.
5. What is normalized mutual information?
It rescales mutual information to improve comparability across datasets. Higher normalized values usually indicate stronger shared structure between the variables.
6. Why can mutual information be zero?
A zero result means the observed joint distribution does not show information gain beyond the marginals. In practice, tiny nonzero values may appear from sampling noise.
7. Does the log base change interpretation?
The base changes units only. Base 2 gives bits, base e gives nats, and base 10 gives hartleys. Relative dependence stays the same.
8. Can this replace correlation analysis?
Not always. Mutual information captures broader dependence, while correlation focuses on linear association. Using both often gives a more complete statistical picture.