Roughly 80% of GitHub stars sit on 20% of projects. That claim feels right. Open source fame is lopsided. But how lopsided, exactly?

If you plot a histogram of repository star counts, the right tail stretches far beyond the median. A small number of projects attract huge attention, while most sit quietly in the long tail.

Heavy-tailed data like this appears in wealth, city sizes, neural network gradients, website traffic, file sizes, and more. One of the simplest continuous models for such behaviour is the Pareto (Type I) distribution.

2. Pareto principle vs Pareto distribution

It is easy to mix up the Pareto principle with the Pareto distribution, but they are not the same thing. Pareto principle (80/20 rule): an empirical heuristic. In many systems, roughly 80% of outcomes come from about 20% of causes.
Pareto distribution: two-parameter power law model with shape parameter alpha and scale parameter x_m (the minimum value).

The Pareto distribution has a probability density function (PDF):

f(x; alpha, x_m) = alpha * x_m**alpha / x**(alpha + 1), for x >= x_m

Only for a specific shape parameter

alpha = log_4(5) approx 1.16

Does the model reproduce an exact 80/20 split? In real data, you will usually see an approximate 80/20 pattern rather than a perfect match.

3. Mathematical essentials

Here are the core formulas you need, plus what they mean in practice.

Concept	Formula	Practical takeaway
PDF	f(x) = alpha * x_m**alpha / x**(alpha + 1)	Simple two parameter power law.
CDF	F(x) = 1 - (x_m / x)**alpha	Closed form CDF makes simulation and quantiles easy.
Survival	S(x) = (x_m / x)**alpha	Straight line on a log-log plot for a true power law.
Mean	E[X] = (alpha * x_m) / (alpha - 1) if alpha > 1	For alpha <= 1, the mean does not exist.
Variance	finite only if alpha > 2	For 1 < alpha <= 2 the mean exists but variance is infinite.

Practical rule of thumb:

• If your fitted alpha is just above 1, your data are extremely heavy-tailed
• If alpha is well above 2, the tail is still heavy but more manageable.

4. Simulating a Pareto in Python

A convenient way to simulate a Pareto variable is inverse CDF (inverse transform) sampling. For U ~ Uniform(0, 1), the transform

X = x_m * (1 - U)**(-1 / alpha)

generates X ~ Pareto(alpha, x_m).

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import pareto

alpha, x_m = 2.5, 1.0   # shape, scale (minimum)
n = 50_000

Inverse-CDF sampling

u = np.random.rand(n)

samples = x_m * (1 - u) ** (-1 / alpha)

Plot histogram and theoretical PDF

plt.hist(samples, bins=200, density=True, alpha=0.4)

x = np.linspace(x_m, samples.max(), 400)

plt.plot(x, pareto.pdf(x, alpha, scale=x_m), linewidth=2)

• plt.xscale("log")
• plt.yscale("log")
• plt.xlabel("x")
• plt.ylabel("PDF (log-log scale)")
• plt.title("Simulated Pareto(alpha=2.5)")
• plt.show()

scipy.stats.Pareto provides sampling, PDF, CDF, and fitting utilities, but the explicit inverse transform shows what is happening under the hood.

5. Fitting the distribution

5.1 Maximum likelihood with SciPy

For a genuine Pareto sample, the maximum likelihood estimator (MLE) for alpha is asymptotically efficient. SciPy can fit alpha and x_m directly.

from scipy.stats import pareto

# data: 1D NumPy array of positive observations
alpha_hat, loc_hat, scale_hat = pareto.fit(data, floc=0)

print(f"alpha_hat = {alpha_hat:.3f}")
print(f"x_m (scale) = {scale_hat:.3f}")

Here we fix loc to 0 so that the support starts at x_m = scale_hat > 0. This matches the standard Type I Pareto parameterisation.

5.2 Hill estimator (tail only)

Sometimes you care only about the extreme tail. The Hill estimator fits alpha using the k largest observations.

import numpy as np

def hill(x, k):
    x = np.sort(np.asarray(x))[::-1]  # descending
    return k / np.sum(np.log(x[:k] / x[k]))

alpha_hat = hill(data, k=1_000)
print(f"Hill estimate of alpha: {alpha_hat:.3f}")

You choose k via stability plots: vary k and look for a flat region in the estimated alpha. Using too few points yields high variance. Using too many points includes non-tail behaviour.

5.3 Bootstrap confidence interval

Because alpha controls tail risk, a confidence interval is often more informative than a point estimate. A simple option is a non-parametric bootstrap.

import random
import numpy as np
from scipy.stats import pareto

n_boot = 999
alphas = []

for _ in range(n_boot):
    resample = random.choices(data, k=len(data))
    alpha_resample, _, _ = pareto.fit(resample, floc=0)
    alphas.append(alpha_resample)

ci_low, ci_high = np.percentile(alphas, [2.5, 97.5])
print(f"95% CI for alpha: {ci_low:.3f}-{ci_high:.3f}")

This gives you a quick sense of estimation uncertainty without strong additional assumptions.

6. Model diagnostics

Fitting a Pareto is only half the story. You should also check whether the model is reasonable.

• Q Q plot on log scale: compare log data quantiles against the fitted Pareto.
  -Kolmogorov-Smirnov test: for example, scipy.stats.kstest with the fitted parameters.
• Tail comparison: overlay Pareto and alternatives such as lognormal on log-log axes. A genuine power law appears close to a straight line.

from scipy.stats import kstest, lognorm, pareto
import numpy as np

# KS test for Pareto fit
ks_p = kstest(data, "pareto", args=(alpha_hat, 0, scale_hat)).pvalue
print(f"KS p-value for Pareto: {ks_p:.3f}")

# Optional: compare log-likelihoods against a lognormal
shape_logn, loc_logn, scale_logn = lognorm.fit(data, floc=0)

If a lognormal model gives a clearly higher log likelihood and better QQ behaviour, your data might only look heavy-tailed on a simple plot.

7. Case study: GitHub stars

As a concrete example, consider the distribution of stars on public GitHub repositories.

import pandas as pd
import seaborn as sns
from scipy.stats import pareto
import matplotlib.pyplot as plt
import numpy as np

# Load data
stars = pd.read_csv("most_starred_repos.csv")["Stars"].values

# Filter and rescale
xmin = 50                      # only repos with >= 50 stars
data = stars[stars >= xmin] / xmin   # rescale so x_m = 1

# Fit Pareto via MLE
alpha_hat, loc_hat, scale_hat = pareto.fit(data, floc=0, fscale=1)
print(f"alpha_hat = {alpha_hat:.2f}")

# Visual check: empirical CDF vs fitted CDF
sns.ecdfplot(data=data, log_scale=True)

x_sorted = np.sort(data)
plt.plot(x_sorted, pareto.cdf(x_sorted, alpha_hat, scale=1.0))

plt.xlabel("Stars / 50")
plt.ylabel("CDF")
plt.title("GitHub stars tail vs fitted Pareto")
plt.show()

- Download a public snapshot of the most starred repositories as a CSV, for example, via the GitHub API or a community-maintained dataset.  
- Filter to repositories with at least a modest number of stars.  
- Rescale so that the minimum in your tail region equals `1`.

In practice, you will often see alpha around 1 to 2 for this kind of popularity data. That confirms a fat right tail that is reminiscent of the 80/20 story but not identical to it.

8. When not to use a Pareto

Pareto is powerful but easy to overuse. Be cautious in at least these cases:

• Finite upper bound (percentages, ratings): A distribution with infinite support is a poor fit.
• Distinct clusters or modes: a single power law cannot capture multimodal data.
• Variance sensitive metrics: if alpha <= 2, variance is infinite, so variance-based KPIs or traditional confidence intervals break down.
• Strong regulatory or actuarial constraints: heavy tails can violate solvency requirements unless carefully modeled and stress tested.

When in doubt, compare against lognormal, Weibull, or other candidates and use domain knowledge to guide your choice.

9. Pareto behavior in machine learning

Heavy tails show up in modern ML systems more often than you might expect.

• Stochastic gradient magnitudes in deep networks can follow power law-like spectra, which motivates adaptive learning rate schedules and gradient clipping strategies.
• Attention weights in large language models often have sparse, Pareto-like patterns, supporting mixed precision storage and pruning.
• Neural scaling laws empirically link model size, data, and compute via power law exponents, echoing Pareto-style behavior in performance curves.

You do not need an exact Pareto fit to benefit from this intuition. The key is to recognize that a small fraction of parameters, tokens, or activations may dominate behavior.

10. One-page cheat sheet

Keep this section handy when you are working with heavy tails in practice.

Topic	Formula or command
PDF	pareto.pdf(x, alpha, scale=x_m)
CDF	pareto.cdf(x, alpha, scale=x_m)
Draw samples	pareto.rvs(alpha, scale=x_m, size=n)
Fit MLE	pareto.fit(data, floc=0)
Mean exists	alpha > 1
Variance exists	alpha > 2
80/20 shape	alpha approx 1.16

11. Key takeaways

• Pareto vs 80/20: the Pareto principle is a rule of thumb, while the Pareto distribution is a parametric model with tunable alpha.
• Always inspect the tail: log log survival plots and tail Q Q plots reveal whether a power law is plausible.
• Fit responsibly: use MLE or Hill estimators, bootstrap confidence intervals, and compare against alternatives such as the lognormal.
• Mind the moments: when alpha <= 2, variance is infinite; when alpha <= 1, even the mean does not exist.
• Heavy tails are common: understanding Pareto behavior prepares you for real-world, messy data in finance, web analytics, and modern ML.

Ready to dive deeper?

Want to go beyond toy examples and apply these ideas in real projects?

Level up your probabilistic modeling skills with Code Labs Academy's Data Science and AI Bootcamp. You will move from core statistics to deploying machine learning models, including working with real heavy-tailed data.

Learn more about the Data Science and AI Bootcamp Explore all Code Labs Academy Bootcamps:

• Explore Programs