Git Submodules Guide: Nested Repositories & Subtrees

Git submodules let you include one repository inside another while keeping their histories independent. This is essential for projects that depend on shared libraries, utility packages, or component repositories that are maintained separately. The parent repository stores the URL and a specific commit hash of each submodule, giving you precise control over which version of the dependency is used. According to GitHub’s repository analysis, approximately 12% of repositories with dependencies use submodules for dependency management — a smaller share than package managers but critical for projects that need tight coupling with specific commits of other repositories.

Understanding Submodules

A submodule is a reference to a specific commit in another repository. The parent repository stores the submodule’s URL and the exact commit hash it should be checked out at. This means the parent locks the dependency to an exact version — there is no “latest” or “semver range” with submodules. This precision is both the primary advantage and the main source of complexity for submodules.

parent-repo/
├── src/
├── libs/
│   └── shared-lib/   → github.com/org/shared-lib.git @ abc1234
└── README.md

The .gitmodules file at the root of the parent repository tracks all submodule configurations. This file should be committed to version control so all developers and CI systems know which submodules to fetch:

[submodule "libs/shared-lib"]
    path = libs/shared-lib
    url = https://github.com/org/shared-lib.git

Adding a Submodule

git submodule add https://github.com/org/shared-lib.git libs/shared-lib

This creates a .gitmodules file (or updates it), adds the submodule entry, and stages both .gitmodules and the submodule commit reference for commit. The submodule directory is created and populated with the repository contents at the upstream default branch’s HEAD.

Cloning a Repository with Submodules

# Clone and initialize all submodules in one step
git clone --recurse-submodules https://github.com/org/parent-repo.git

If you already cloned without --recurse-submodules:

# Initialize and fetch all submodules
git submodule update --init --recursive

The --recursive flag is necessary for submodules that themselves contain submodules (nested submodules). A common CI mistake is forgetting the --recursive flag, which leaves nested submodules empty.

Working with Submodules

Pulling Updates

Submodules do not update automatically when you pull the parent repository. The parent records a specific commit hash, and the submodule stays at that hash until you explicitly update it:

# Fetch latest remote changes for all submodules
git submodule update --remote --merge

# Update a specific submodule
git submodule update --remote libs/shared-lib

The --remote flag fetches the latest commit from the submodule’s remote tracking branch (usually origin/main). The --merge flag merges the fetched commit into the local submodule branch.

Committing in a Submodule

cd libs/shared-lib
git checkout main          # Switch from detached HEAD to a branch
git pull                   # Get latest changes
# Make changes, commit them
git push                   # Push submodule changes
cd ../..
git add libs/shared-lib    # Record the new commit hash in the parent
git commit -m "Update shared-lib to latest"

Always check out a branch in the submodule before making changes. By default, submodules are in detached HEAD state, which means any commits you make are not on a branch — they can be lost if you switch to a different commit. The detached HEAD state is one of the most common sources of confusion for developers new to submodules.

Removing a Submodule

git submodule deinit libs/shared-lib
git rm libs/shared-lib
rm -rf .git/modules/libs/shared-lib
git commit -m "Remove shared-lib submodule"

Removing a submodule requires three steps: deinitializing, removing from the working tree, and cleaning up the internal Git metadata. Missing any step can leave stale references in the repository.

Submodules vs Subtrees

Git subtrees merge external repositories into your repository without maintaining a separate reference:

Subtrees are simpler for consumers (no special clone flags) but bloat the parent repository’s history with the entire external repository’s history. Submodules are lighter but require more discipline from developers. The choice between them depends on whether your primary concern is consumer simplicity (choose subtrees) or repository size and history separation (choose submodules).

Common Pitfalls

• Detached HEAD — Submodules check out at specific commits, not branches. Always check out a branch before making changes inside a submodule
• Forgotten --recurse-submodules — Cloning without this flag leaves submodule directories empty. Add a .gitmodules check to your CI pipeline
• Outdated submodules — Teammates may not realize submodules need updating. Add a CI check: git submodule status should match the intended versions
• Submodule URL changes — If the submodule repository URL changes, developers with stale clones may have issues fetching updates
• Relative vs absolute URLs — Relative URLs (../shared-lib.git) work for forks and mirrors but require the same hosting provider

Best Practices

• Pin submodules to stable release tags, not main branch, for reproducible builds
• Document which submodules exist and their purpose in CONTRIBUTING.md or README
• Use relative URLs in .gitmodules for fork compatibility: ../shared-lib.git instead of full URLs
• Add a CI check that verifies submodule commits match expected versions
• Consider Git subtrees if your team finds submodule workflows confusing — subtrees are simpler for non-expert Git users
• Use git config --global submodule.recurse true to make Git commands recurse into submodules by default

Submodules in CI/CD

Integrating submodules with CI/CD pipelines requires special handling to ensure submodules are fetched and updated correctly.

GitHub Actions

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          submodules: true
          token: ${{ secrets.PAT_TOKEN }}

The submodules: true flag initializes and fetches all submodules recursively. For private submodules, use a personal access token (PAT) with repository access. GitHub Actions automatically uses the built-in GITHUB_TOKEN for the main repository but requires explicit tokens for cross-repository submodule access.

GitLab CI

variables:
  GIT_SUBMODULE_STRATEGY: normal

build:
  script:
    - git submodule update --init --recursive
    - make build

GitLab CI supports GIT_SUBMODULE_STRATEGY: normal which runs git submodule sync and git submodule update --init before the job starts. For recursive submodules, add GIT_SUBMODULE_DEPTH: 1 to limit clone depth.

Caching Submodules

Submodule contents can be cached between CI runs to reduce pipeline time:

- uses: actions/cache@v3
  with:
    path: .git/modules
    key: submodules-${{ hashFiles('.gitmodules') }}

Cache the .git/modules directory to avoid re-fetching submodule history on every pipeline run. Invalidate the cache when .gitmodules changes (new submodules added or URLs updated). This optimization can reduce CI setup time by 50-80% for repositories with large submodules.

Migration Guide: Submodules to Subtrees

Migrating from Submodules to Subtrees

If your team finds submodule management burdensome, migrate to subtrees:

# Check current submodule commit
git submodule status libs/shared-lib

# Remove the submodule
git submodule deinit libs/shared-lib
git rm libs/shared-lib
git commit -m "Remove shared-lib submodule"

# Add as a subtree (pins to the same commit)
git subtree add --prefix=libs/shared-lib \
  https://github.com/org/shared-lib.git \
  <commit-sha> --squash

Pulling Updates with Subtrees

# Pull latest upstream changes into subtree
git subtree pull --prefix=libs/shared-lib \
  https://github.com/org/shared-lib.git main --squash

The --squash flag collapses the external repository’s full history into a single commit, keeping the parent repository’s history manageable. Without --squash, every upstream commit is imported into the parent repository, potentially adding thousands of commits that are irrelevant to the parent project.

FAQ

When should I use submodules instead of a package manager?

Use submodules when you need tight coupling with a specific commit in another repository, when the dependency is a development tool or internal library that is not published to a package registry, or when you need to modify the dependency alongside the parent project. For most third-party dependencies, package managers (npm, pip, Maven) are simpler and more appropriate.

How do I handle CI/CD with submodules?

Ensure your CI system initializes submodules: git submodule update --init --recursive. For GitHub Actions, use actions/checkout@v4 with submodules: true. For GitLab CI, use GIT_SUBMODULE_STRATEGY: normal. Cache submodule contents to avoid fetching them on every pipeline run. For private submodules, configure authentication tokens in your CI environment.

Can I have nested submodules?

Yes, and they are common in large projects. Clone with --recurse-submodules to fetch all nested submodules. Update with git submodule update --init --recursive. Each level of nesting adds complexity, so consider whether a flat structure is possible. More than two levels of nesting is usually a sign that a different dependency management approach would be cleaner.

How do I see what version each submodule is at?

Run git submodule status to see the checked-out commit hash and brief status of each submodule. A leading - means the submodule is not initialized. A leading + means the submodule has uncommitted changes. A leading U means merge conflicts exist in the submodule. The commit hash shown is the exact version the parent repository has pinned.

What is the best way to update all submodules at once?

Run git submodule update --remote --merge to update each submodule to the latest commit on its remote tracking branch. Review the changes with git submodule status, then commit the updated references in the parent repository. For automatic updates in CI, use Dependabot or Renovate with submodule support to create automated PRs when submodule upstreams change.

Conclusion

Git submodules enable clean separation of concerns across repositories while maintaining precise dependency versioning. They require explicit management — pulling updates, checking out branches before making changes, and recording new commit hashes in the parent — but provide exact reproducibility that package managers cannot match. For teams that find submodules too complex, subtrees offer a simpler model at the cost of repository size. Choose the approach that matches your team’s workflow and dependency management needs.

For more Git workflows, see Git collaboration workflows and Git merge strategies.