Git Submodules & Monorepos

Difficulty: Advanced

Question

What are Git submodules? How do monorepos work, and what tools exist for managing them?

Answer

Git submodules allow you to embed one Git repository inside another as a subdirectory. The parent repo stores a reference to a specific commit in the submodule, not the actual files. This is useful for shared libraries, vendor code, or when different parts of a project have independent release cycles.

Submodule challenges: they add complexity to cloning, pulling, and switching branches. Developers must remember to initialize and update submodules. Commits in the parent only record which submodule commit to use, leading to frequent out-of-sync issues.

Monorepos take the opposite approach: put everything in one repository. Companies like Google, Facebook, and Microsoft use monorepos for massive codebases. Benefits include atomic cross-project changes, unified CI/CD, shared tooling, and simplified dependency management.

Monorepo tools: - Turborepo: Fast, uses caching and parallelism. Great for JS/TS projects. - Nx: Full-featured with dependency graph analysis and affected-only testing. - Lerna: Pioneered JS monorepos. Good for publishing multiple npm packages. - git subtree: Git-native alternative to submodules. Merges external repo history into a subdirectory.

Code examples

Git Submodule Operations

# Add a submodule
git submodule add https://github.com/lib/shared-utils.git libs/shared
# Creates .gitmodules file and libs/shared/ directory

# Clone a repo with submodules
git clone --recurse-submodules https://github.com/user/project.git
# Or after cloning:
git submodule init
git submodule update

# Update submodule to latest remote commit
cd libs/shared
git checkout main
git pull
cd ../..
git add libs/shared
git commit -m "chore: update shared-utils submodule"

# Update all submodules at once
git submodule update --remote --merge

# See submodule status
git submodule status
# +a1b2c3d libs/shared (v1.2.0)
# + means the submodule has a different commit than recorded

# Remove a submodule
git submodule deinit libs/shared
git rm libs/shared
rm -rf .git/modules/libs/shared
git commit -m "chore: remove shared-utils submodule"

Submodules are references to specific commits in external repos. The parent tracks which commit to use. Always clone with --recurse-submodules to get submodule contents.

Git Subtree Alternative

# Add a subtree (merges external repo into a directory)
git subtree add --prefix=libs/shared \
  https://github.com/lib/shared-utils.git main --squash

# Pull updates from the external repo
git subtree pull --prefix=libs/shared \
  https://github.com/lib/shared-utils.git main --squash

# Push changes back to the external repo
git subtree push --prefix=libs/shared \
  https://github.com/lib/shared-utils.git main

# Advantages over submodules:
# - No special clone commands needed
# - History is merged into the parent repo
# - Simpler workflow for contributors
# - No .gitmodules file to manage

# Disadvantages:
# - Larger repo size (history is duplicated)
# - Pushing changes back is more complex
# - Less clear boundary between repos

Subtrees are simpler than submodules for most use cases. The external code lives directly in your repo with no special initialization needed. Choose subtrees for simplicity, submodules for strict version pinning.

Monorepo with Turborepo

# Initialize a Turborepo monorepo
npx create-turbo@latest my-monorepo

# Directory structure:
# my-monorepo/
# +-- apps/
# |   +-- web/         (Next.js frontend)
# |   +-- api/         (Express backend)
# +-- packages/
# |   +-- ui/          (shared component library)
# |   +-- config/      (shared ESLint/TS config)
# |   +-- utils/       (shared utilities)
# +-- turbo.json
# +-- package.json

# turbo.json - define task dependencies
{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"],
      "outputs": ["dist/", ".next/"]
    },
    "test": {
      "dependsOn": ["build"]
    },
    "lint": {},
    "dev": {
      "cache": false
    }
  }
}

# Run tasks across all packages
npx turbo build    # builds all packages in dependency order
npx turbo test     # runs tests with caching
npx turbo lint     # lints all packages in parallel

# Run only affected packages
npx turbo build --filter=web...
# Only builds web and its dependencies

Turborepo analyzes the dependency graph and runs tasks in optimal order. Caching means unchanged packages are not rebuilt. --filter runs tasks only for affected packages.

Key points

Concepts covered

Git Submodules, Monorepos, git subtree, Turborepo, Nx, Lerna