Distributions

Difficulty: Intermediate

A probability distribution describes how the values of a random variable are distributed. It tells you the probability (or probability density) of each possible outcome. Understanding distributions is fundamental to data science because real-world data often follows known distribution patterns, and many statistical methods assume specific distributions for the data they analyze.

The normal (Gaussian) distribution is the most important distribution in statistics. It is symmetric and bell-shaped, fully described by two parameters: the mean (center) and standard deviation (spread). The empirical rule states that approximately 68% of data falls within one standard deviation of the mean, 95% within two, and 99.7% within three. The Central Limit Theorem tells us that the sampling distribution of the mean approaches a normal distribution as the sample size grows, regardless of the population's distribution -- which is why the normal distribution appears everywhere in statistics.

The binomial distribution models the number of successes in n independent trials, each with the same probability p of success. For example, the number of heads in 10 coin flips follows a binomial distribution with n=10 and p=0.5. The Poisson distribution models the number of events occurring in a fixed interval of time or space, given a known average rate. It is commonly used for modeling rare events like customer arrivals, server requests, or equipment failures.

The uniform distribution assigns equal probability to all outcomes in a range. For a continuous uniform distribution between a and b, each sub-interval of the same length has the same probability. Discrete uniform distributions include fair dice rolls. While simple, the uniform distribution is important as a baseline and is used in random number generation and simulation.

Skewness measures the asymmetry of a distribution. A positively skewed distribution has a long right tail (mean > median), while a negatively skewed distribution has a long left tail (mean < median). Kurtosis measures the 'tailedness' of a distribution -- how much of the data is in the tails. A normal distribution has a kurtosis of 3 (excess kurtosis of 0). High kurtosis (leptokurtic) means heavy tails with more outliers, while low kurtosis (platykurtic) means light tails. Understanding skewness and kurtosis helps you assess whether data meets normality assumptions required by many statistical tests.

Code examples

Normal Distribution Properties

import numpy as np

np.random.seed(42)
data = np.random.normal(loc=100, scale=15, size=10000)

mean = np.mean(data)
std = np.std(data, ddof=1)

within_1std = np.sum((data >= mean - std) & (data <= mean + std)) / len(data)
within_2std = np.sum((data >= mean - 2*std) & (data <= mean + 2*std)) / len(data)
within_3std = np.sum((data >= mean - 3*std) & (data <= mean + 3*std)) / len(data)

print(f"Mean: {round(mean, 2)}")
print(f"Std Dev: {round(std, 2)}")
print(f"Within 1 std: {round(within_1std * 100, 1)}%")
print(f"Within 2 std: {round(within_2std * 100, 1)}%")
print(f"Within 3 std: {round(within_3std * 100, 1)}%")

We generate 10,000 samples from a normal distribution with mean 100 and std 15. The empirical rule is confirmed: approximately 68%, 95%, and 99.7% of values fall within 1, 2, and 3 standard deviations of the mean.

Binomial Distribution

import math

def binomial_pmf(n, k, p):
    """P(X = k) for binomial distribution"""
    comb = math.comb(n, k)
    return comb * (p ** k) * ((1 - p) ** (n - k))

# 10 coin flips, P(heads) = 0.5
n, p = 10, 0.5

for k in range(11):
    prob = binomial_pmf(n, k, p)
    if prob >= 0.05:  # Only show significant probabilities
        print(f"P(X={k:2d}) = {prob:.4f}")

mean = n * p
variance = n * p * (1 - p)
print(f"\nMean: {mean}")
print(f"Variance: {variance}")

The binomial PMF gives exact probabilities for each count of successes. With n=10 and p=0.5, the distribution is symmetric around 5. The mean is n*p = 5 and variance is n*p*(1-p) = 2.5.

Poisson Distribution

import math

def poisson_pmf(lam, k):
    """P(X = k) for Poisson distribution"""
    return (lam ** k) * math.exp(-lam) / math.factorial(k)

# Average 3 customers per hour
lam = 3

print("Customers/hour probabilities (lambda=3):")
for k in range(8):
    prob = poisson_pmf(lam, k)
    print(f"P(X={k}) = {prob:.4f}")

print(f"\nMean = Variance = {lam}")

The Poisson distribution models events per interval. With lambda=3, the most likely outcomes are 2 or 3 customers per hour. A unique property of the Poisson distribution is that its mean equals its variance.

Skewness and Kurtosis

import numpy as np

np.random.seed(0)

# Symmetric (normal)
normal_data = np.random.normal(0, 1, 10000)

# Right-skewed (exponential)
skewed_data = np.random.exponential(1, 10000)

def calc_skewness(data):
    n = len(data)
    mean = np.mean(data)
    std = np.std(data, ddof=1)
    return (n / ((n-1)*(n-2))) * np.sum(((data - mean) / std) ** 3)

def calc_kurtosis(data):
    n = len(data)
    mean = np.mean(data)
    std = np.std(data, ddof=1)
    k4 = np.mean(((data - mean) / std) ** 4)
    return k4 - 3  # Excess kurtosis

print(f"Normal - Skewness: {round(calc_skewness(normal_data), 4)}")
print(f"Normal - Excess Kurtosis: {round(calc_kurtosis(normal_data), 4)}")
print(f"Exponential - Skewness: {round(calc_skewness(skewed_data), 4)}")
print(f"Exponential - Excess Kurtosis: {round(calc_kurtosis(skewed_data), 4)}")

The normal distribution has skewness near 0 (symmetric) and excess kurtosis near 0. The exponential distribution is positively skewed (long right tail) with high excess kurtosis (heavy tail). These metrics help assess the shape of your data distribution.

Key points

Concepts covered

Normal distribution, Binomial distribution, Poisson distribution, Uniform distribution, Skewness, Kurtosis