DataFrame Basics

Difficulty: Beginner

Once you have created a DataFrame, the next step is to inspect and understand its structure and content. Pandas provides a rich set of attributes and methods specifically designed for quick data exploration. These inspection tools are the first thing every data analyst uses when they encounter a new dataset, and mastering them will make you significantly faster at understanding unfamiliar data.

The head() and tail() methods are your starting point. head(n) returns the first n rows of the DataFrame (default is 5), while tail(n) returns the last n rows. These methods let you quickly glance at the data without printing the entire DataFrame, which could be millions of rows. In practice, you will call df.head() dozens of times during any analysis session to verify that your transformations are producing the expected results.

The shape attribute returns a tuple of (rows, columns), giving you an immediate sense of the dataset's size. The columns attribute returns an Index object listing all column names, and dtypes returns a Series showing the data type of each column. Together, these three attributes tell you the dimensions, the column names, and the types of data you are working with, all without looking at a single data value.

The info() method provides a concise summary that combines shape, column names, data types, and non-null counts into a single output. It is particularly useful for spotting missing data: if a column shows fewer non-null values than the total row count, you know there are NaN values in that column. The info() method also shows the memory usage of the DataFrame, which matters when working with large datasets.

The describe() method generates descriptive statistics for numeric columns by default. It shows count, mean, standard deviation, minimum, 25th percentile, median (50th percentile), 75th percentile, and maximum. For non-numeric columns, you can call describe(include='object') to get count, unique count, top value, and frequency. The describe() method is your quickest path to understanding the distribution and range of values in your dataset.

Code examples

Using head() and tail()

import pandas as pd

df = pd.DataFrame({
    'Name': ['Alice', 'Bob', 'Charlie', 'Diana', 'Eve'],
    'Age': [25, 30, 35, 28, 32],
    'Salary': [50000, 60000, 70000, 55000, 65000]
})

print("First 3 rows:")
print(df.head(3))
print("---")
print("Last 2 rows:")
print(df.tail(2))

head(3) returns the first 3 rows, and tail(2) returns the last 2 rows. These are non-destructive operations that return new DataFrames without modifying the original.

Inspecting shape, columns, and dtypes

import pandas as pd

df = pd.DataFrame({
    'Product': ['Laptop', 'Mouse', 'Keyboard'],
    'Price': [50000.0, 500.0, 1500.0],
    'InStock': [True, True, False]
})

print("Shape:", df.shape)
print("Columns:", list(df.columns))
print("---")
print(df.dtypes)

shape returns (rows, columns) as a tuple. columns lists all column names. dtypes shows the data type of each column: object for strings, float64 for decimals, and bool for boolean values.

Using describe() for Statistics

import pandas as pd

df = pd.DataFrame({
    'Math': [85, 90, 78, 92, 88],
    'Science': [80, 85, 75, 95, 90],
    'English': [70, 88, 82, 79, 91]
})

print(df.describe())

describe() automatically computes count, mean, standard deviation, min, 25th/50th/75th percentiles, and max for all numeric columns. This gives you a quick statistical overview of your data.

Checking Column Types and Renaming

import pandas as pd

df = pd.DataFrame({
    'emp_name': ['Alice', 'Bob', 'Charlie'],
    'emp_age': [25, 30, 35],
    'emp_dept': ['HR', 'IT', 'Finance']
})

print("Original columns:", list(df.columns))

# Rename columns
df = df.rename(columns={'emp_name': 'Name', 'emp_age': 'Age', 'emp_dept': 'Department'})
print("Renamed columns:", list(df.columns))
print("---")
print(df)

The rename() method accepts a dictionary mapping old column names to new ones. By default it returns a new DataFrame, so we reassign it. You can also pass inplace=True to modify the DataFrame in place.

Key points

• head() and tail() default to 5 rows if no argument is passed. Always use them before printing an entire large DataFrame.
• describe() only includes numeric columns by default. Use describe(include='all') to include non-numeric columns as well.
• The dtypes attribute is crucial for catching type issues early. A column you expect to be numeric might be 'object' if it contains mixed data or missing values represented as strings.

Concepts covered

head(), tail(), info(), describe(), shape, columns, dtypes