Difficulty: Intermediate
Feature analysis is the process of examining individual variables in a dataset to understand their characteristics, distributions, and potential usefulness for analysis or modeling. Each feature (column) has properties like data type, cardinality (number of unique values), distribution shape, and relationship strength with other features. Understanding these properties is essential for making informed decisions about feature engineering, selection, and preprocessing.
Features are broadly categorized as numeric (continuous or discrete) or categorical (nominal or ordinal). Continuous features like temperature or salary can take any value within a range, while discrete features like number of children take only integer values. Nominal categorical features like color or department have no inherent order, whereas ordinal features like education level or satisfaction rating have a meaningful sequence.
Cardinality -- the number of distinct values a feature takes -- has significant implications for analysis and modeling. Low-cardinality categorical features (e.g., gender with 2 values) are easy to encode and analyze. High-cardinality features (e.g., user IDs with millions of values) require special handling such as frequency encoding, target encoding, or hashing. Numeric features with very low cardinality might actually be better treated as categorical.
Value distribution analysis reveals the shape of your data: whether it is normally distributed, skewed, bimodal, or uniform. Skewness and kurtosis are quantitative measures of distribution shape. Positively skewed data (long right tail) is common in financial datasets, while negatively skewed data appears in test scores near maximum. Understanding distributions guides your choice of summary statistics, visualization methods, and machine learning algorithms.
import pandas as pd
import numpy as np
data = {
'age': [25, 30, 35, 28, 32],
'salary': [50000.50, 65000.75, 80000.00, 55000.25, 72000.00],
'department': ['Eng', 'Mkt', 'Eng', 'HR', 'Mkt'],
'rating': [1, 2, 3, 2, 3],
'employee_id': ['E001', 'E002', 'E003', 'E004', 'E005']
}
df = pd.DataFrame(data)
print("Feature Classification:")
for col in df.columns:
dtype = df[col].dtype
nunique = df[col].nunique()
ratio = nunique / len(df)
if dtype == 'object':
if ratio > 0.8:
ftype = "Identifier (high cardinality)"
else:
ftype = "Categorical (nominal)"
elif dtype in ['int64', 'float64']:
if nunique <= 5:
ftype = "Categorical (ordinal/discrete)"
else:
ftype = "Numeric (continuous)"
else:
ftype = "Unknown"
print(f" {col}: {ftype} (unique={nunique}, dtype={dtype})")This classifier uses dtype and cardinality ratio to determine feature types. High unique ratio in string columns suggests identifiers. Low unique counts in numeric columns suggest discrete/ordinal features. This classification guides how each feature should be processed and encoded.
import pandas as pd
data = {
'gender': ['M', 'F', 'M', 'F', 'M', 'F', 'M', 'M', 'F', 'M'],
'city': ['NYC', 'LA', 'NYC', 'Chicago', 'LA', 'NYC', 'Boston', 'NYC', 'LA', 'Chicago'],
'user_id': ['U1', 'U2', 'U3', 'U4', 'U5', 'U6', 'U7', 'U8', 'U9', 'U10'],
'score': [85, 92, 78, 88, 95, 82, 90, 87, 91, 84]
}
df = pd.DataFrame(data)
print("Cardinality Report:")
print(f"{'Column':<12} {'Unique':>6} {'Total':>6} {'Ratio':>8} {'Level':<15}")
print("-" * 50)
for col in df.columns:
nunique = df[col].nunique()
total = len(df)
ratio = nunique / total
if ratio <= 0.05:
level = "Very Low"
elif ratio <= 0.2:
level = "Low"
elif ratio <= 0.5:
level = "Medium"
elif ratio < 1.0:
level = "High"
else:
level = "Unique (ID)"
print(f"{col:<12} {nunique:>6} {total:>6} {ratio:>8.2f} {level:<15}")Cardinality analysis reveals the diversity of values in each column. Features with very low cardinality (like gender) are easy to one-hot encode. Features with cardinality equal to dataset size (like user_id) are identifiers and usually should not be used as model features.
import numpy as np
def analyze_distribution(name, data):
arr = np.array(data, dtype=float)
n = len(arr)
mean = np.mean(arr)
median = np.median(arr)
std = np.std(arr)
# Skewness (Fisher's formula)
skew = np.mean(((arr - mean) / std) ** 3)
# Kurtosis (excess)
kurt = np.mean(((arr - mean) / std) ** 4) - 3
if abs(skew) < 0.5:
shape = "Symmetric"
elif skew > 0:
shape = "Right-skewed"
else:
shape = "Left-skewed"
print(f"{name}:")
print(f" Mean: {mean:.1f}, Median: {median:.1f}")
print(f" Skewness: {skew:.2f} ({shape})")
print(f" Kurtosis: {kurt:.2f}")
# Normal-ish data
normal = [48, 49, 50, 50, 51, 51, 52, 50, 49, 51]
analyze_distribution("Normal-ish", normal)
print()
# Right-skewed (income-like)
right_skew = [30, 35, 40, 42, 45, 50, 55, 80, 120, 200]
analyze_distribution("Right-skewed", right_skew)
print()
# Left-skewed (test scores near max)
left_skew = [60, 85, 88, 90, 92, 93, 94, 95, 96, 97]
analyze_distribution("Left-skewed", left_skew)Skewness near 0 indicates symmetric data. Positive skewness means a long right tail (mean > median), common in income data. Negative skewness means a long left tail (mean < median), common in exam scores. Kurtosis measures tail heaviness, with 0 being the normal distribution baseline.
import numpy as np
def pearson_corr(x, y):
x = np.array(x, dtype=float)
y = np.array(y, dtype=float)
mx, my = np.mean(x), np.mean(y)
num = np.sum((x - mx) * (y - my))
den = np.sqrt(np.sum((x - mx)**2) * np.sum((y - my)**2))
return num / den if den != 0 else 0
# Features
years_exp = [1, 2, 3, 5, 7, 10, 12, 15, 18, 20]
salary = [35, 42, 48, 55, 65, 78, 85, 92, 98, 105]
shoe_size = [9, 10, 8, 11, 9, 10, 12, 8, 11, 10]
pairs = [
("years_exp vs salary", years_exp, salary),
("years_exp vs shoe_size", years_exp, shoe_size),
("salary vs shoe_size", salary, shoe_size),
]
print("Correlation Analysis:")
for name, x, y in pairs:
r = pearson_corr(x, y)
strength = "Strong" if abs(r) > 0.7 else "Moderate" if abs(r) > 0.4 else "Weak"
direction = "positive" if r > 0 else "negative"
print(f" {name}: r={r:.3f} ({strength} {direction})")Pearson correlation ranges from -1 to 1. The near-perfect correlation between years_exp and salary (0.998) suggests a strong linear relationship. Shoe size has negligible correlation with both, confirming it is not a useful predictor. This type of analysis helps identify which features to keep for modeling.
feature types, cardinality, value distributions, feature importance