Python Packaging and Project Setup

Difficulty: Intermediate

Question

How do you structure and package a Python project? What is pyproject.toml?

Answer

pyproject.toml (PEP 517/518/621) is the modern standard for Python project configuration, replacing setup.py and setup.cfg.

Tools: - pip + setuptools: standard package installer and build backend - poetry: all-in-one dependency management, virtual envs, publishing - uv: ultra-fast pip/poetry replacement (written in Rust) - hatch: modern project manager

Virtual environments: - Isolate project dependencies from system Python - python -m venv .venv, then activate - .venv/bin/activate (Unix) or .venv\Scripts\activate (Windows)

Structure: src layout is preferred for packaging (src/mypackage/); avoids import confusion during development.

Code examples

pyproject.toml Structure

# pyproject.toml (modern standard)
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.backends.legacy:build"

[project]
name = "my-package"
version = "1.0.0"
description = "A sample Python package"
readme = "README.md"
requires-python = ">=3.10"
license = {text = "MIT"}
authors = [
    {name = "Alice", email = "alice@example.com"}
]
dependencies = [
    "requests>=2.28",
    "pydantic>=2.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "black",
    "mypy",
    "ruff",
]

[project.scripts]  # CLI entry points
my-cli = "my_package.cli:main"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.mypy]
strict = true

[tool.ruff]
line-length = 88

pyproject.toml consolidates all project config. [project.scripts] creates CLI commands. [tool.*] sections configure development tools.

Virtual Environments and Dependency Management

# Create virtual environment
# python -m venv .venv

# Activate (Unix/Mac)
# source .venv/bin/activate

# Activate (Windows)
# .venv\Scripts\activate

# Install dependencies
# pip install -e .[dev]  # Install in editable mode with dev deps

# requirements.txt (for deployment)
# pip freeze > requirements.txt
# pip install -r requirements.txt

# poetry workflow
# poetry init
# poetry add requests pydantic
# poetry add --dev pytest black
# poetry install
# poetry run python script.py
# poetry shell  # Activate venv

# src layout (recommended)
# my-project/
# ├── src/
# │   └── my_package/
# │       ├── __init__.py
# │       ├── core.py
# │       └── utils.py
# ├── tests/
# │   ├── test_core.py
# │   └── conftest.py
# ├── pyproject.toml
# └── README.md

# Why src layout?
# - tests import installed package, not local files
# - Avoids name conflicts with stdlib
# - Forces proper installation before use
print('Project structure defined in pyproject.toml')

The src layout prevents accidentally importing local source instead of the installed package. This surfaces packaging issues early.

Key points

• uv (by Astral) is the fastest package installer in 2024 - 10-100x faster than pip
• __init__.py is required to make a directory a Python package
• PEP 660 enables editable installs (pip install -e .) with pyproject.toml
• Semantic versioning: MAJOR.MINOR.PATCH; bump MAJOR for breaking changes

Concepts covered

pyproject.toml, pip, virtual environments, poetry, setuptools