Python Logging Module

Difficulty: Intermediate

Question

How does the Python logging module work? Why is it preferred over print statements?

Answer

The logging module provides a flexible, configurable system for emitting log records.

Advantages over print(): - Log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL - Handlers: route to file, console, HTTP, email - Formatters: add timestamps, module names, line numbers - Hierarchical loggers: inherit configuration from parent - Disable without changing code (set level to WARNING to silence DEBUG) - Thread-safe

Best practices: - Use __name__ as logger name (creates hierarchy) - Configure in the entry point, not in libraries - Use structured logging for production (JSON output)

Code examples

Basic Logging Setup

import logging

# Basic configuration (for scripts)
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S'
)

# Module-level logger - use __name__
logger = logging.getLogger(__name__)

logger.debug('Debug message - detailed info')
logger.info('Info message - general info')
logger.warning('Warning message - unexpected event')
logger.error('Error message - serious problem')
logger.critical('Critical - program may not continue')

# Log exceptions with traceback
try:
    1 / 0
except ZeroDivisionError:
    logger.exception('Division failed')  # Includes traceback
    # Same as: logger.error('msg', exc_info=True)

# Lazy formatting - use % style, not f-strings!
user_id = 42
# GOOD: string only formatted if level is active
logger.debug('User %s logged in', user_id)
# BAD: always formats even if DEBUG is disabled
logger.debug(f'User {user_id} logged in')

logger.exception() logs ERROR level with full traceback. Use % style formatting - the string is only constructed if the log level is active.

File Handler and Logger Hierarchy

import logging
import sys

# Professional logging setup
def setup_logging(level=logging.INFO):
    root_logger = logging.getLogger()
    root_logger.setLevel(level)

    # Console handler
    console = logging.StreamHandler(sys.stdout)
    console.setLevel(logging.INFO)
    console_fmt = logging.Formatter(
        '%(levelname)s - %(message)s'
    )
    console.setFormatter(console_fmt)

    # File handler
    file_handler = logging.FileHandler('app.log', encoding='utf-8')
    file_handler.setLevel(logging.DEBUG)
    file_fmt = logging.Formatter(
        '%(asctime)s - %(name)s:%(lineno)d - %(levelname)s - %(message)s'
    )
    file_handler.setFormatter(file_fmt)

    root_logger.addHandler(console)
    root_logger.addHandler(file_handler)

# Logger hierarchy
# root (logging.getLogger())
# app
# app.database ← inherits from 'app' which inherits from root
# app.api

setup_logging()

db_logger = logging.getLogger('app.database')
api_logger = logging.getLogger('app.api')

db_logger.debug('Query executed')  # Goes to file only
api_logger.info('Request received')  # Goes to both

# Silence noisy library
logging.getLogger('urllib3').setLevel(logging.WARNING)

Handlers can have different levels. The hierarchy allows silencing specific modules. File handler captures DEBUG while console only shows INFO+.

Key points

• Never use logging.basicConfig in library code - only in application entry points
• logging.getLogger(__name__) creates a hierarchy matching package structure
• Structured logging (JSON) is preferred for production: use python-json-logger library
• logging is thread-safe; print() is not

Concepts covered

logging, Logger, Handler, Formatter, Level