Pre-Commit Secret Scanner vs CI: What Each Place Missed

11 min · nov 2026

On a fresh clone of our test repository, ShipGuard 0.7.1 printed SAFE_TO_SHIP and exited 1 under --strict. Run on the same clone in changed-files mode, it scanned 0 files, said SAFE_TO_SHIP again, and exited 0. That clone was standing in for a CI job, and it was one of four places we ran the same check.

To compare where a pre-commit secret scanner should run, we built one small repository holding three fake secrets, each at a different stage of its life. Then we ran the same release gate from a plain git pre-commit hook, from the pre-commit framework, from a pre-push hook, and from a fresh git clone of what had been pushed. We did not run a real CI runner; the clone was our stand-in, and the limits section says what that leaves out.

What does a pre-commit secret scanner actually see?

Git runs the pre-commit hook before it creates the commit. The githooks manual states the stakes exactly: "Exiting with a non-zero status from this script causes the git commit command to abort before creating a commit." Which bytes the check reads is up to the tool.

We ran the published CLI, @agenticcli/shipguard@0.7.1 from npm, in a container with networking off. The three fake values were generated when the run started:

• A GitHub personal access token in config/github.ts, committed and then deleted in the next commit. It exists only in history.
• A Stripe test secret key in src/billing.ts, staged and not yet committed.
• A live-shaped Stripe key in .env.local, untracked and not covered by .gitignore.

This is a representative demo we wrote by hand, not code an assistant produced. Our Next.js and Supabase secrets test and the vibe-coded app test show the same gate on fuller app layouts. Every scan ran twice as separate invocations, with identical findings (capture pre-commit-placement-matrix, shipguard 0.7.1, 2026-10-05).

Does --changed read the staged version or the working tree?

The working tree. On shipguard 0.7.1, --changed (described in the ShipGuard docs) takes its file list from git, covering changed, staged and untracked files, and then reads each file's contents from disk. Two edits to the same repository showed it:

• Secret staged, then removed from the file on disk. The commit would contain the key. --changed did not report src/billing.ts at all.
• Clean version staged, secret only on disk. The commit would not contain the key. --changed reported it anyway, as a medium "Stripe Test Secret Key" finding on line 2.

Both runs still exited 1, because the untracked .env.local was flagged each time. A plain hook calling --changed therefore checks what sits on disk beside the commit, not the exact bytes going into it. When you stage part of a file and keep editing it, the bytes on disk and the bytes in the commit differ.

What does each placement catch on the same repository?

Both pre-commit rows used --changed. Every row ran scan --strict --json, where exit 1 means a blocking finding.

The pre-commit hook alone stopped the staged key before it had a commit hash, and one flag skipped it. The clone alone could not be skipped, yet never saw .env.local. Neither one saw the history token, so the check belongs in both, with a history scan behind them.

What did the hook print when it stopped the commit?

The plain hook's run below is copied from the capture's console log. The hook stopped the commit, the commit count stayed at 3, and --no-verify then created the commit without running the hook:

=== P1 plain .git/hooks/pre-commit
commit exit: 1
pre-commit hook: shipguard exit 1
commits: 3
commit --no-verify exit: 0
commits: 4

The commit that went through carried the staged Stripe test key. The pre-push hook later stopped that same history from leaving the machine, and git push --no-verify then moved all 4 commits to the remote.

The hook's 3 findings were the live-shaped key in .env.local (critical), the same file reported as ".env file tracked in git" (high), and the staged Stripe test key (medium). That second title is wrong: .env.local was never tracked, and we have logged it as a bug. The pre-push full scan read the same untracked file from disk, which is why it also reported three.

Why did the CI stand-in say SAFE_TO_SHIP and exit 1?

Because the two fields measure different things on 0.7.1. This is the full-scan report from the fresh clone, trimmed to the fields that matter (... marks removed lines):

{
  "findings": [
    {
      "id": "hardcoded_secret",
      "title": "Stripe Test Secret Key",
      "severity": "medium",
      "category": "secrets",
      "file": "src/billing.ts",
      "line": 2,
      ...
      "blocking": true
    }
  ],
  "riskScore": {
    "score": 15,
    "recommendation": "SAFE_TO_SHIP",
    "totalFindings": 1,
    ...
    "blockingCount": 1
  },
  "filesScanned": 5,
  ...
}

The recommendation comes from the risk score, and one medium finding left it at SAFE_TO_SHIP. The --strict exit counts blocking findings, and this one is marked "blocking": true, so the process exited 1. The report is consistent with itself; the headline field is what misleads, and that mismatch is on our bug list too. Until it changes, fail the CI step on the exit code and have a person read findings[].

The same clone scanned in changed-files mode returned an empty findings list with "filesScanned": 0 and exit 0. A CI job built on that mode, in a clean checkout like this one, would pass while reading nothing, which is why CI needs the full scan.

How do you set up a pre-commit hook for secrets?

The simplest version is a short shell script saved as .git/hooks/pre-commit in your own clone of the repository. This is the hook we ran, with the globally installed shipguard binary our container used (the install command is at the end):

#!/bin/sh
shipguard scan --changed --strict --json > .shipguard-report.json

Make it executable with chmod +x .git/hooks/pre-commit, and add .shipguard-report.json to .gitignore so the report never lands in a commit. Git reads the script's exit code: 0 lets the commit through and 1 stops it. The hook itself is not committed with the repository, so every clone needs it installed again. Prove the wiring once with a fake key, the way our run did: the commit has to fail with exit 1.

If your project uses Husky, the slot is a file: the Husky docs say "Adding a hook is as simple as creating a file." Put the same command in .husky/pre-commit; we did not test Husky, so prove that wiring with a fake key too.

How do you wire it into the pre-commit framework?

The pre-commit framework keeps the hook definition in a committed .pre-commit-config.yaml, so the setup travels with the repository and each developer runs pre-commit install once. This is the local hook from our fixture, with the installed binary in the entry; ours called it through bash -c and saved the report to a file:

repos:
  - repo: local
    hooks:
      - id: shipguard
        name: shipguard pre-ship check (changed files)
        entry: shipguard scan --changed --strict --json
        language: system
        pass_filenames: false
        always_run: true

The pre-commit configuration reference defines both keys: with pass_filenames: false no filenames are passed to the hook, and always_run: true runs the hook even when no file matches.

We then repeated the staged-then-edited case on the framework: key staged, then removed from the file on disk. The framework set the unstaged edit aside first, as its documentation says it does: "pre-commit only runs on the staged contents of files by temporarily stashing the unstaged changes while running hooks." Its output, copied from the capture log:

[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /tmp/.cache/pre-commit/patch1791215898-229.
shipguard pre-ship check (changed files).................................Failed
- hook id: shipguard
- exit code: 1
[INFO] Restored changes from /tmp/.cache/pre-commit/patch1791215898-229.

With the edit stashed, the file on disk matched the staged copy, and the key the plain --changed run had missed was flagged. The stash covers tracked files only, so the untracked .env.local was still read from disk.

Which other tools fit the same slot?

Any command that reads the files you care about and returns a reliable exit code can fill the slot. Does it read the staged copy or the working tree? Does it have a mode that reads old commits? Does it exit non-zero on the findings you care about?

Gitleaks answered the second one here: its git mode found the history token that every ShipGuard placement missed. Yelp's detect-secrets is built as a pre-commit hook, and its README is candid about what a hook can do, a line we also quoted in our guide to detecting hardcoded secrets:

This is not meant to be a sure-fire solution to prevent secrets from entering the codebase. Only proper developer education can truly do that. This pre-commit hook merely implements several heuristics to try and prevent obvious cases of committing secrets.

— Yelp · detect-secrets README

We did not test TruffleHog, another common choice. We compared local tools with the checks built into a git host in GitHub's secret scanner versus CLI tools.

Why run the same check in CI too?

The hook can be skipped, and git says so. The githooks manual notes that the pre-commit hook "can be bypassed with the --no-verify option."

After the plain hook stopped the commit, git commit --no-verify created it with exit 0, and git push --no-verify did the same to the pre-push hook. A fresh clone also starts with no hooks installed, as ours did.

The clone had no local changes, so --changed found 0 files and exited 0; in CI, run the full scan, shipguard scan --strict --json, as its own step in the job. And CI sees only what was pushed: the untracked .env.local never reached the clone, which makes that file the local hook's job.

Can your git host stop the push instead?

A git host can refuse the push on its own side, where no client flag reaches. GitHub describes push protection as a feature that "blocks pushes that contain secrets before they reach your repository" (GitHub Docs). That check sits on GitHub's side of the push, outside the local hooks that --no-verify bypasses.

GitLab's pipeline job has a narrower default: "By default, pipeline secret detection scans only the current state of the Git repository. Any secrets contained in the repository’s history are not detected." (GitLab Docs).

Our GitLab secret detection test measured how that scope changes the result. We pushed to neither host in this test, so the push protection and GitLab behaviour above is what their documentation says.

What about a key that is already in git history?

No check that reads files will find it. The token in our fixture was deleted one commit after it was added, and every ShipGuard placement missed it. GitHub says its secret scanning "scans your entire Git history on all branches of your repository for hardcoded credentials" (GitHub Docs).

Gitleaks 8.30.1 found it locally. We ran it on the pushed repository with its git command, which the gitleaks README says the command "lets you scan local git repos." It reported two leaks: the Stripe key in src/billing.ts and the token in config/github.ts from the earlier commit.

GitLab's documentation says "Removed secrets also persist in the Git history." (GitLab Docs). Rotate the key first. GitHub's guide on removing sensitive data notes that "Once the secret is revoked or rotated, it can no longer be used for access, and that may be sufficient to solve your problem." (GitHub Docs).

Where does an AI agent's own loop fit?

A coding agent edits files, runs commands and often makes the commit itself, so the cheapest point to check its work is before it reports "done". In the instructions your agent reads at the start of a session, have it run shipguard scan --strict --json and keep working until the exit code is 0 and the recommendation field reads SAFE_TO_SHIP.

It has to check both fields, because our CI stand-in returned SAFE_TO_SHIP with exit 1. Agent-written code does carry hardcoded credentials: our audit of Claude Code output found hardcoded AWS credentials among its findings.

We did not measure this placement on its own. It runs the same command on the same working tree, so the check reads what the agent just wrote to disk. An instruction in a file guides the agent but cannot force it, and an agent can pass --no-verify like anyone else, so the CI step stays.

What we did not test

We did not run a real GitHub Actions runner. Our CI stand-in was a fresh git clone of the pushed repository inside the same container, and we did not run act for this fixture. We did not push to GitHub or GitLab, so push protection, GitHub secret scanning and GitLab secret detection were not exercised here. Their behaviour above is what their documentation says.

We did not test Husky, detect-secrets or TruffleHog, and we ran gitleaks only in its git mode on the pushed repository. One repository with three planted secrets is a demonstration, not a detection rate, and two identical runs show the output was stable on this input, not on every input.

What are ShipGuard's own limits here?

On shipguard 0.7.1 (measured 2026-10-05), the free checks cover five areas: secrets, auth, destructive SQL in migration paths, payments, and deployment configuration. ShipGuard reads files on disk, so it does not see git history. In --changed mode it reads the working tree rather than the staged copy, and on a clean checkout it scans nothing and says SAFE_TO_SHIP.

Two output problems from this run are logged as bugs: the ".env file tracked in git" title on a file that was never tracked, and the SAFE_TO_SHIP headline over a blocking finding. Until both are fixed, read the findings and the exit code together. For the wider set of pre-ship checks these sit inside, see 12 checks before you ship.

To run the same check on your next commit:

npx @agenticcli/shipguard scan --strict --json

The ShipGuard page lists what each free check covers.

Published by AgenticCLI — developer tools for teams shipping AI-assisted code. ShipGuard is a deterministic, CLI-first release gate that runs locally, with no code leaving your machine. See all checks at agenticcli.dev/shipguard.

FAQ