Detect Hardcoded Secrets: 6 Shapes, 4 Scanners

14 min · jul 2026

Two lines in the same small repository hold the same forty-one characters. sha256 agrees, both times: eeb57e5d99ce4610. Three of four scanners report the first line and none of the four reports the second. What differs is the word to the left of the equals sign.

That is not a bug anyone shipped. It is the documented behaviour of every generic secrets rule we read, and it is worth sitting with before choosing a tool, because it changes which of your credentials you should expect to hear about.

What does it take for a scanner to see a hardcoded secret?

We built a small order service of the shape an agent writes in one sitting and put six hardcoded credentials in it. Every one is a hardcoded credential under CWE-798, MITRE's own entry for the class, which defines it in a sentence:

The product contains hard-coded credentials, such as a password or cryptographic key.

The six differ only in what a pattern matcher has something to grip on. One carries a Stripe test-key prefix. One is a high-entropy value assigned to a name containing SECRET. One is that same value under a plain name. One is a password inside a postgres:// connection string. One is the placeholder changeme-in-prod on a JWT signing key. The last is an admin password in a JSON config file.

Then we read the tree with ShipGuard, Gitleaks, Semgrep and Trivy. None of these credentials authenticates to anything, and the fixture generator is committed so the run can be repeated rather than believed. We have put this same set of tools against a different set of secrets before, which is part of why the shapes here were chosen to vary rather than the tooling.

What came back?

Real capture. @agenticcli/[email protected], Gitleaks 8.30.1, Semgrep 1.170.0, Trivy 0.72.0, 28 July 2026. Fixture and manifest: content-pipeline/captures/secret-detectability-four-scanners/.

Nobody was idle here and nobody was broken. Each tool returned something, several agreed with each other independently, and the credentials they all walked past are the ones whose lines offer a matcher nowhere to begin. Coverage, on this evidence, is a property of how a credential is written down rather than of how carefully it is guarded.

Why does the same secret disappear when you rename the variable?

The third row is the one worth isolating, because everything else about those two lines is held constant. The literal is 41 characters, written once in the fixture generator and interpolated into both files, so the two cannot drift apart into two different tests.

Real capture. @agenticcli/[email protected], Gitleaks 8.30.1, Semgrep 1.170.0, Trivy 0.72.0, 28 July 2026. Probe and manifest: content-pipeline/captures/secret-shape-flip/.

The mechanism is not hidden. Gitleaks builds its generic-api-key rule by calling GenerateSemiGenericRegex, which concatenates an identifier alternation, an assignment operator and a value pattern into a single regular expression. Its identifier list is fixed: access, auth, api, credential, creds, key, passw(or)d, secret, token. The rule describes itself as detecting "a Generic API Key, potentially exposing access to various services and sensitive operations."

Where in the pattern does the name sit?

Read the rule in the order the engine reads it, and the ordering does the explaining. The identifier alternation comes first in the compiled expression, the assignment operator second, the value last, and a regular expression that fails at its first alternation stops there.

So the name is not context the rule consults after finding something. The name is the part of the pattern that has to match first, and when it doesn't, the value to its right is never examined at all.

The two panels below trace one line through that ordering twice, with only the identifier changed.

Is rotationSalt an unusual thing to call it?

rotationSalt is not an exotic name. It is what you would call that variable, and a model asked to write a fingerprinting helper would call it the same thing. Naming is where a generator is at its most fluent and its least predictable, which is a property worth remembering when reviewing a fast-accepted diff.

Run the same thought over your own codebase for a moment. pepper, nonceSeed, internalToken, hmacMaterial, webhookSalt — every one is a credential and none of them contains a word on the list quoted above.

A keyword list can only hold names somebody thought of in advance, and the space of reasonable variable names is not a list.

Which secrets does every tool agree on?

The Stripe key, and its whole family. sk_test_, AKIA, ghp_, AIza — each announces its issuer from the characters alone, with no help from the surrounding line. Trivy's built-in secret rules read almost as a directory of companies: aws-access-key-id, github-pat, gitlab-pat, stripe-secret-token, slack-access-token, gcp-service-account, on through the file.

That coverage is real and it is worth having. It is also, and this is the uncomfortable part, the class of credential you are least likely to lose quietly — the one an audit of generated code finds first, because it is the one that looks like a secret. GitHub's own documentation describes the arrangement plainly:

GitHub partners with a large variety of service providers to validate detected secrets.

When a partner secret turns up in a public repository, the provider hears about it and can revoke the credential without waiting for you. The same page lists non-provider secrets — private keys, connection strings, generic API keys — as a separate surface you switch on rather than the default.

So which of your own credentials are left over?

Your database password has no partner programme. Neither does the signing key on your job queue, or the shared token between two of your own services, and those are the credentials that an AI-built app accumulates fastest because nothing about writing them involves visiting a dashboard.

A provider key is issued once, by somebody else, through an interface that records the issue. An internal secret is invented on the spot by whoever needed one, in the file where it was needed, with the same fluency that lets a model invent a dependency name nobody ever published. It has no registry, no expiry and no third party watching for it in public code.

That asymmetry is worth holding onto, because it inverts the intuition. The credentials with the most machinery behind them are the ones you are least likely to lose without noticing.

Why doesn't a stronger password help?

Because the floor cuts the wrong way. Entropy is there to separate a credential from ordinary prose, and it does that job well: a random-looking string in a source file is unusual, and treating it as suspicious is a reasonable default with a low false-positive cost.

What it cannot do is recognise a credential that looks like a word. Gitleaks sets Entropy: 3.5 on its generic rule and additionally skips any value matching ^[a-zA-Z_.-]+$, so a value made only of letters and punctuation is stood down on regardless of how long it is. changeme-in-prod fails both tests, which is why only one of our four tools said anything about it.

The weakness that makes a password bad is the same property that makes it invisible, and the two are not separable by any threshold.

Does anyone say this out loud?

Yelp's detect-secrets, which has run as a pre-commit hook in large organisations for years, is unusually direct about this in its own README:

This is not meant to be a sure-fire solution to prevent secrets from entering the codebase. Only proper developer education can truly do that. This pre-commit hook merely implements several heuristics to try and prevent obvious cases of committing secrets.

Its list of things that will not be prevented has two entries, and the second is default passwords that do not trigger its keyword detector, with login = "hunter2" as the worked example. A maintainer wrote that down years before our fixture reproduced it.

So the weak credential is protected by its own weakness. Raise the entropy and you cross the floor; keep it memorable and you stay under it, which is the choice a person makes when they intend to fix it after the demo. The same intention leaves a database open in test mode for thirty days, and it is generous rather than careless: nobody writes changeme believing it will still be there in March.

Is this only a repository problem?

CWE-798 splits the weakness into an outbound variant, where your code holds a credential in order to reach something else, and an inbound variant, where a credential is compiled into the product and every installation carries the same one.

A scanner reading your working tree can in principle see the first. The second is not in your repository at all — it travels in the artifact, the same direction that makes what you ship inherit what you installed worth a look of its own.

CVE-2026-5189, published on 15 April 2026 against Sonatype Nexus Repository Manager 3.0.0 through 3.70.5, is rated critical at CVSS 9.2:

CWE-798: Use of Hard-coded Credentials in Sonatype Nexus Repository Manager versions 3.0.0 through 3.70.5 allows an unauthenticated attacker with network access to gain unauthorized read/write access to the internal database and execute arbitrary OS commands as the Nexus process user.

How well does automated detection actually work?

MITRE's own Detection Methods table for the weakness is blunter than any vendor page on the subject:

Automated white box techniques have been published for detecting hard-coded credentials for incoming authentication, but there is some expert disagreement regarding their effectiveness and applicability to a broad range of methods.

The taxonomy that defines the weakness class records automated detection of it as contested, rates black-box detection of the inbound variant as only moderately effective, and notes that manual review may do better. That is not a small caveat buried in a footnote. It is the reference every one of those rule ids classifies against.

What can you check that a scanner cannot?

The useful move is to stop starting from the code. A scanner can only tell you about strings it recognised, so the gap is defined by what it did not recognise, and no amount of reading its output closes that.

Which of those rows earns its place?

The second row does most of the work in practice. Every createHmac call takes a key, every database client takes a connection string, and every outbound request that authenticates carries something — so the call sites enumerate the credentials whether or not any of them is named in a way a matcher likes. It is the same discipline that makes a checklist beat a tool run on the items automation genuinely does not cover.

The fourth row is the one people skip. A scan tells you a credential is present; it never tells you who has already read it. Anything that has been in a commit, a build log or a screenshot in a chat window is spent, and rotating it costs less than deciding whether it leaked.

What we didn't test, and what we can't tell you

Six credentials is a fixture, not a sample. This run establishes what these four tools do with these six shapes on this date, and it says nothing about a hit rate across real repositories — a number we would need a very different study to earn, and would not want to borrow from a vendor report.

Four tools is also not the field. GitHub secret scanning, TruffleHog, GitGuardian and SonarQube were not run here; GitHub's model is described from its own documentation rather than measured. The Semgrep run is the open-source CLI under --config=auto, which is a different product from Semgrep Secrets and its provider validators, and the distinction matters enough that we name which one every time.

We also never asked whether any credential was live, because none of ours is. Validity checking is the one question a provider partnership answers well and a local pattern match cannot answer at all.

And what our own tool doesn't do

ShipGuard reached four of the six, including the two nobody else did, and it missed the same two everybody missed. The rotationSalt line went unreported by our free-tier check for the same structural reason it went unreported by Gitleaks: our high-entropy detector also wants a recognisable name, and the admin password in config/admin.json cleared no threshold worth firing on.

We are not going to close that by adding names to a list. Any list is a list, and the credential your codebase happens to call something nobody put on ours is the one that gets through.

What a release gate can honestly offer is the well-shaped cases caught reliably, a verdict you can read before you ship, and a plain statement of the edge — which is also how we handle the checks that need a running system rather than a file to read.

The scan is worth running. It is not worth reading as an inventory.

FAQ