Opscraft
Git · Lesson 7 of 8 · 3 min read

Repositories at scale and in CI

Monorepo vs polyrepo

Monorepo Polyrepo
Atomic cross-service changes Easy (one PR) Hard (coordinated PRs)
Code sharing & refactoring Easy Via published libraries/versions
CI Must build only what changed (path filters, Bazel/Nx/Turborepo) Simple per repo
Access control Coarser (CODEOWNERS helps) Per repo
Repo size/performance Can get large (sparse checkout, partial clone) Small repos
Tooling consistency One set of standards Drift between repos

Both work; the honest answer depends on team size, coupling between services and tooling investment.

Making Git fast in CI

git clone --depth 1 --branch main https://github.com/acme/app.git     # shallow clone
git clone --filter=blob:none https://github.com/acme/big.git           # partial clone: fetch blobs on demand
git sparse-checkout set services/orders libs/common                    # only some directories
git fetch --unshallow                                                  # if a step needs full history
  • Shallow clones speed up CI but break things that need history: git describe (versioning from tags), changed-file detection against the base branch, some changelog tools — fetch enough depth or tags (fetch-depth: 0 in GitHub Actions).
  • Cache the repository on persistent agents, or use reference clones.
Enlarge diagramHistoryFile contentsFolderseverythingeverythingeverythinglatest commit onlylatest onlyeverythingeverythingfetched on demandeverythingas clonedas clonedonly the ones you listFull cloneShallow --depth 1Partial --filter=blob:noneSparse checkoutShallow clones break git describe and changed-file detection: fetch more depth and tags in jobs that need them.
Figure 1. Ways to make clones fast in CI, and what each one downloads.
Swipe sideways to see all of it, or tap to open it full screen.

Large files, submodules and hooks

  • Git LFS stores large binaries (models, media) outside the normal object store, keeping pointers in Git. Don't commit build artifacts or big binaries to plain Git — use an artifact repository.
  • Submodules embed another repo at a pinned commit (clone with --recurse-submodules); powerful but easy to get wrong (detached HEADs, forgotten updates). Subtrees copy another repo's content into a subdirectory. Package managers are usually better for sharing code.
  • Hooks run scripts on events: client-side (pre-commit, commit-msg, pre-push) and server-side (pre-receive, on self-hosted servers). The pre-commit framework manages linters, formatters, secret scanners, terraform fmt, YAML lint across a team — but client hooks are optional, so enforce the same checks in CI.
  • .gitattributes controls line endings (* text=auto), diff drivers, LFS tracking and merge strategies per path. Windows CRLF vs Linux LF issues in shell scripts ("/bin/bash^M: bad interpreter") are fixed here.

Git hosting for DevOps

GitHub, GitLab, Bitbucket and AWS CodeCommit (closed to new customers in 2024, then reopened as generally available in November 2025) add PRs, protections, CI, webhooks and APIs. Webhooks from the Git host trigger CI (Jenkins, Actions) and GitOps controllers (Flux/Argo can also poll or receive webhooks for faster sync).