Chapter 34: Packaging and Distribution

You've written useful code. Now you want to share it — with a colleague, with your team, or with the world on PyPI so anyone can install it with pip install yourpackage. This chapter shows you how.

Packaging is also important even for internal projects. A properly packaged project is easier to install, version, test, and maintain than a loose collection of scripts.

What a Package Actually Is

When someone runs pip install requests, pip downloads a wheel (or source distribution) from PyPI, unpacks it, and copies files into their Python's site-packages directory. After that, import requests works anywhere in that environment.

A Python package is fundamentally:

1. A directory with an __init__.py
2. A pyproject.toml that describes the package (name, version, dependencies, entry points)
3. Optionally, a build backend that converts those files into a distributable artifact

Modern Project Structure

The modern, recommended layout for a Python package:

my_package/                  <- project root (git repo)
├── src/
│   └── my_package/          <- the actual package (importable code)
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── cli.py
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── test_utils.py
├── docs/
│   └── index.md
├── pyproject.toml           <- the single source of truth for everything
├── README.md
├── LICENSE
└── .gitignore

The src/ layout (putting your package inside a src/ directory) prevents the common mistake where Python imports your local source instead of your installed package during testing.

pyproject.toml — The Modern Configuration File

pyproject.toml replaced setup.py, setup.cfg, and requirements.txt in one file. It's the standard for every new Python project.

[build-system]
requires      = ["hatchling"]
build-backend = "hatchling.build"


[project]
name        = "my-awesome-package"
version     = "0.1.0"
description = "A short description of what this package does"
readme      = "README.md"
license     = { file = "LICENSE" }
requires-python = ">=3.10"
authors = [
    { name = "Alice Smith", email = "alice@example.com" },
]
keywords = ["python", "utility", "example"]
classifiers = [
    "Development Status :: 3 - Alpha",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.10",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
]

# Runtime dependencies — what your package needs to work
dependencies = [
    "requests>=2.28",
    "pydantic>=2.0",
]


[project.optional-dependencies]
# pip install my-awesome-package[dev]
dev = [
    "pytest>=7",
    "pytest-cov",
    "mypy",
    "ruff",
]
# pip install my-awesome-package[docs]
docs = [
    "mkdocs",
    "mkdocs-material",
]


[project.urls]
Homepage      = "https://github.com/alice/my-awesome-package"
Documentation = "https://my-awesome-package.readthedocs.io"
Repository    = "https://github.com/alice/my-awesome-package"
"Bug Tracker" = "https://github.com/alice/my-awesome-package/issues"


# Command-line scripts — creates `my-tool` command after pip install
[project.scripts]
my-tool = "my_package.cli:main"


# Tool configuration lives here too — one file for everything
[tool.pytest.ini_options]
testpaths  = ["tests"]
addopts    = "--cov=src --cov-report=term-missing"

[tool.mypy]
python_version        = "3.12"
strict                = true
ignore_missing_imports = true

[tool.ruff]
line-length = 88
target-version = "py310"
select  = ["E", "F", "I", "N", "W"]
ignore  = ["E501"]

[tool.coverage.run]
source = ["src"]
omit   = ["tests/*"]

Choosing a build backend

Backend	Install	Best for
hatchling	pip install hatch	Most projects — simple, modern
setuptools	Pre-installed	Legacy projects, C extensions
flit	pip install flit	Pure Python, minimal config
poetry	Separate install	Integrated dependency management

For most new projects, use hatchling. It's fast, simple, and well-maintained.

Building a Real Package

Let's build a package called texttools — a collection of text processing utilities.

Directory structure

texttools/
├── src/
│   └── texttools/
│       ├── __init__.py
│       ├── analysis.py
│       ├── transform.py
│       └── cli.py
├── tests/
│   ├── test_analysis.py
│   └── test_transform.py
├── pyproject.toml
├── README.md
└── LICENSE

The package code

# src/texttools/__init__.py
"""
texttools — A collection of text processing utilities.
"""
from .analysis  import word_count, char_frequency, readability_score
from .transform import clean, titlecase, truncate, slug

__version__ = "1.0.0"
__all__     = [
    "word_count", "char_frequency", "readability_score",
    "clean", "titlecase", "truncate", "slug",
]

# src/texttools/analysis.py
"""Text analysis utilities."""
import re
from collections import Counter
from typing import Counter as CounterType


def word_count(text: str) -> int:
    """Return the number of words in text."""
    return len(text.split())


def char_frequency(text: str, ignore_case: bool = True) -> CounterType[str]:
    """Return a Counter of character frequencies, excluding whitespace."""
    if ignore_case:
        text = text.lower()
    return Counter(c for c in text if not c.isspace())


def readability_score(text: str) -> float:
    """
    Flesch Reading Ease score.
    90-100: Very easy. 0-30: Very difficult.
    """
    sentences = max(1, len(re.findall(r"[.!?]+", text)))
    words     = max(1, word_count(text))
    syllables = max(1, sum(_count_syllables(w) for w in text.split()))

    return 206.835 - 1.015 * (words / sentences) - 84.6 * (syllables / words)


def _count_syllables(word: str) -> int:
    """Rough syllable counter."""
    word    = word.lower().strip(".,!?;:")
    vowels  = "aeiou"
    count   = 0
    prev_v  = False
    for ch in word:
        is_v = ch in vowels
        if is_v and not prev_v:
            count += 1
        prev_v = is_v
    return max(1, count)

# src/texttools/transform.py
"""Text transformation utilities."""
import re
import unicodedata


def clean(text: str) -> str:
    """Strip leading/trailing whitespace and normalize internal spaces."""
    return " ".join(text.split())


def titlecase(text: str) -> str:
    """Title case, but don't capitalize short words like 'a', 'the', 'of'."""
    small = {"a", "an", "and", "as", "at", "but", "by", "for",
             "if", "in", "nor", "of", "on", "or", "so", "the",
             "to", "up", "yet"}
    words  = text.lower().split()
    result = []
    for i, word in enumerate(words):
        if i == 0 or word not in small:
            result.append(word.capitalize())
        else:
            result.append(word)
    return " ".join(result)


def truncate(text: str, max_length: int, suffix: str = "...") -> str:
    """Truncate text to max_length characters, appending suffix if truncated."""
    if len(text) <= max_length:
        return text
    return text[: max_length - len(suffix)].rstrip() + suffix


def slug(text: str) -> str:
    """Convert text to a URL-friendly slug."""
    text = unicodedata.normalize("NFKD", text)
    text = text.encode("ascii", "ignore").decode("ascii")
    text = text.lower()
    text = re.sub(r"[^\w\s-]", "", text)
    text = re.sub(r"[\s_-]+", "-", text)
    return text.strip("-")

# src/texttools/cli.py
"""Command-line interface for texttools."""
import sys
import argparse
from . import word_count, readability_score, slug, truncate


def main() -> None:
    parser = argparse.ArgumentParser(
        prog        = "texttools",
        description = "Text processing utilities",
    )
    subparsers = parser.add_subparsers(dest="command", required=True)

    # texttools count <text>
    count_p = subparsers.add_parser("count", help="Count words in text")
    count_p.add_argument("text", nargs="?", help="Text to analyze (or stdin)")

    # texttools score <text>
    score_p = subparsers.add_parser("score", help="Flesch readability score")
    score_p.add_argument("text", nargs="?")

    # texttools slug <text>
    slug_p = subparsers.add_parser("slug", help="Convert to URL slug")
    slug_p.add_argument("text", nargs="?")

    # texttools truncate <text> --length 50
    trunc_p = subparsers.add_parser("truncate", help="Truncate text")
    trunc_p.add_argument("text",   nargs="?")
    trunc_p.add_argument("--length", type=int, default=80)
    trunc_p.add_argument("--suffix", default="...")

    args = parser.parse_args()

    text = args.text or sys.stdin.read().strip()
    if not text:
        parser.error("Provide text as an argument or via stdin")

    if args.command == "count":
        print(word_count(text))
    elif args.command == "score":
        score = readability_score(text)
        print(f"{score:.1f}")
    elif args.command == "slug":
        print(slug(text))
    elif args.command == "truncate":
        print(truncate(text, args.length, args.suffix))


if __name__ == "__main__":
    main()

pyproject.toml for texttools

[build-system]
requires      = ["hatchling"]
build-backend = "hatchling.build"

[project]
name            = "texttools"
version         = "1.0.0"
description     = "A collection of text processing utilities"
readme          = "README.md"
requires-python = ">=3.10"
license         = { text = "MIT" }
authors         = [{ name = "Alice Smith", email = "alice@example.com" }]
keywords        = ["text", "nlp", "utilities"]
classifiers     = [
    "License :: OSI Approved :: MIT License",
    "Programming Language :: Python :: 3",
    "Topic :: Text Processing",
]
dependencies    = []   # no external dependencies

[project.optional-dependencies]
dev = ["pytest", "pytest-cov", "mypy", "ruff"]

[project.scripts]
texttools = "texttools.cli:main"

[project.urls]
Repository = "https://github.com/alice/texttools"

[tool.hatch.build.targets.wheel]
packages = ["src/texttools"]

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts   = "-v --cov=src/texttools --cov-report=term-missing"

Installing in Development Mode

While developing, install the package in editable mode — changes to source files are reflected immediately without reinstalling:

# From the project root
pip install -e .

# With dev dependencies
pip install -e ".[dev]"

After this, you can run:

python -c "import texttools; print(texttools.word_count('hello world'))"
# 2

texttools count "The quick brown fox jumps over the lazy dog"
# 9

texttools slug "Hello World! This is a URL slug."
# hello-world-this-is-a-url-slug

texttools score "Python is easy to learn. It has simple syntax. Beginners love it."
# 74.2

Writing a Good README

A README is the first thing people see. It should answer five questions in order:

# texttools

A collection of text processing utilities for Python.

## Installation

```bash
pip install texttools

Quick Start

import texttools

print(texttools.word_count("Hello world"))       # 2
print(texttools.slug("Hello World!"))            # hello-world
print(texttools.truncate("Long text...", 10))    # Long te...

CLI

echo "Hello World" | texttools slug
# hello-world

texttools score "Python is a great language for beginners."
# 68.4

API Reference

word_count(text: str) -> int

Count the number of words in text.

slug(text: str) -> str

Convert text to a URL-friendly slug.

License

MIT

Keep it short. Show the install command. Show a working example. Link to full docs.


## Versioning — Semantic Versioning

Use **SemVer**: `MAJOR.MINOR.PATCH`

- **PATCH** (`1.0.0` -> `1.0.1`) — bug fix, no API change
- **MINOR** (`1.0.0` -> `1.1.0`) — new feature, backwards compatible
- **MAJOR** (`1.0.0` -> `2.0.0`) — breaking change

```toml
# pyproject.toml
[project]
version = "1.2.3"

Some projects automate versioning from git tags:

[build-system]
requires = ["hatchling", "hatch-vcs"]
build-backend = "hatchling.build"

[tool.hatch.version]
source = "vcs"   # version comes from git tags

# Then tag a release:
# git tag v1.2.3
# git push --tags

Building Distribution Files

pip install build
python -m build

This creates two files in dist/:

dist/
├── texttools-1.0.0-py3-none-any.whl    <- wheel (binary distribution)
└── texttools-1.0.0.tar.gz              <- sdist (source distribution)

• Wheel (.whl) — the preferred format. pip installs it directly without building.
• Source distribution (.tar.gz) — a tarball of the source. Used as a fallback, or when the wheel isn't available for a platform.

Publishing to PyPI

TestPyPI — a sandbox for practicing without polluting the real index:

pip install twine

# Upload to TestPyPI first
twine upload --repository testpypi dist/*

# Install from TestPyPI to verify
pip install --index-url https://test.pypi.org/simple/ texttools

Once you're confident, upload to the real PyPI:

twine upload dist/*

You need a PyPI account. Create one at pypi.org and use an API token (not your password):

# Configure once in ~/.pypirc
[pypi]
username = __token__
password = pypi-AgEIcH...your-token-here...

Or set environment variables:

export TWINE_USERNAME=__token__
export TWINE_PASSWORD=pypi-AgEIcH...
twine upload dist/*

After publishing, anyone can install your package:

pip install texttools

Automating Releases with GitHub Actions

A CI/CD pipeline that tests your code, builds the package, and publishes to PyPI automatically when you push a tag:

# .github/workflows/publish.yml
name: Publish to PyPI

on:
  push:
    tags:
      - "v*"   # trigger on any tag like v1.0.0

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      - run: pip install -e ".[dev]"
      - run: pytest

  publish:
    needs: test   # only publish if tests pass
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install build twine
      - run: python -m build
      - run: twine upload dist/*
        env:
          TWINE_USERNAME: __token__
          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}

Add your PyPI token to GitHub repository secrets as PYPI_API_TOKEN. Now every git tag v1.2.3 && git push --tags runs your tests and publishes automatically.

Code Quality Tools

A modern Python project uses these tools, all configured in pyproject.toml:

ruff — fast linter + formatter

pip install ruff

ruff check .        # lint
ruff check . --fix  # auto-fix what's possible
ruff format .       # format (like Black)

[tool.ruff]
line-length    = 88
target-version = "py310"

[tool.ruff.lint]
select = [
    "E",   # pycodestyle errors
    "F",   # pyflakes
    "I",   # isort (import sorting)
    "N",   # pep8-naming
    "UP",  # pyupgrade (modernize syntax)
]

mypy — static type checking

mypy src/

[tool.mypy]
python_version         = "3.12"
strict                 = true
ignore_missing_imports = true

pre-commit — run checks before every commit

pip install pre-commit

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.3.0
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.9.0
    hooks:
      - id: mypy
        additional_dependencies: [pydantic]

pre-commit install          # hook into git
pre-commit run --all-files  # run manually

Now git commit automatically runs ruff and mypy. Bad code can't accidentally be committed.

Dependency Management Best Practices

Pinning dependencies

Your pyproject.toml lists minimum versions for dependencies (requests>=2.28). This keeps the package flexible for users.

For applications (not libraries), pin exact versions to ensure reproducibility:

pip freeze > requirements.txt
# or
pip-compile pyproject.toml -o requirements.txt

pip-tools for reproducible installs

pip install pip-tools

# Generate pinned requirements from pyproject.toml
pip-compile pyproject.toml -o requirements.txt
pip-compile --extra dev -o requirements-dev.txt

# Install exactly these versions
pip-sync requirements.txt requirements-dev.txt

Using uv — the modern, fast alternative

uv is a new package manager written in Rust — 10-100x faster than pip:

pip install uv

uv venv                          # create virtual environment
uv pip install -e ".[dev]"       # install
uv pip compile pyproject.toml    # generate lockfile
uv run pytest                    # run in the venv

Complete Workflow Summary

Here is the complete workflow for a new Python package from zero to published:

# 1. Create the project
mkdir texttools && cd texttools
python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate

# 2. Create structure
mkdir -p src/texttools tests
touch src/texttools/__init__.py
touch src/texttools/core.py
touch tests/test_core.py
touch pyproject.toml README.md LICENSE

# 3. Write pyproject.toml (see above)

# 4. Install in editable mode
pip install -e ".[dev]"

# 5. Write code and tests
# ... (the fun part)

# 6. Run tests and quality checks
pytest
mypy src/
ruff check .

# 7. Commit and tag
git init && git add .
git commit -m "Initial release"
git tag v1.0.0

# 8. Build
python -m build

# 9. Publish
twine upload --repository testpypi dist/*   # test first
twine upload dist/*                          # then real PyPI

What You Learned in This Chapter

• A Python package is a directory with __init__.py described by pyproject.toml.
• Use the src/ layout to prevent importing local source during testing.
• pyproject.toml contains package metadata, dependencies, optional dependency groups, entry points (CLI commands), and tool configuration.
• Choose hatchling as your build backend for new projects.
• pip install -e . installs in editable mode — source changes are reflected immediately.
• Semantic versioning: MAJOR.MINOR.PATCH. Automate from git tags with hatch-vcs.
• python -m build creates a wheel and source distribution in dist/.
• twine upload dist/* publishes to PyPI. Use TestPyPI first.
• GitHub Actions automates testing and publishing on every git tag.
• ruff lints and formats. mypy type-checks. pre-commit enforces both before every commit.
• Pin exact versions in applications (pip freeze), use ranges in libraries (requests>=2.28).
• uv is the modern, fast alternative to pip.

What's Next?

Chapter 35 covers Professional Python Practices — code style, documentation with docstrings and Sphinx, git workflow, code review, and the habits that separate professional Python developers from hobbyists. This is the final chapter on professional practices before the advanced topics in Part 10.