A leaked key isn’t a finding until you know it still works

Every secret scanner on the market can find a string that looks like an API key. Regexes for that have been in the public domain for a decade. The hard part was never detection, it’s that a detection on its own tells you almost nothing.

Consider two findings, both reported as CRITICAL. The first is a Stripe key that was rotated eight months ago and lingers in an old bundle nobody redeployed. The second is a Supabase service_role key that works right now and bypasses every row-level security policy you have. Pattern-matching cannot distinguish them, so most tools report both identically. The developer opens the report, checks the first one, finds it’s dead, and learns that this tool cries wolf.

That’s the real failure mode. Not a missed vulnerability, a lost reader.

The 64% problem

There’s a counter-argument worth taking seriously: if a key leaked, isn’t it compromised regardless? Shouldn’t you rotate it either way?

Yes. But GitGuardian’s State of Secrets Sprawl found that 64% of secrets leaked in 2022 were still valid years later. Not because anyone decided they were safe, but because nobody knew which ones mattered enough to act on. Undifferentiated urgency produces no urgency at all.

Asking the only authority that knows

There is exactly one system that can say whether a credential is live: the one that issued it. So when Flare.ai finds a key, it asks, with a single read-only identity request to that provider and nobody else.

• OpenAI and Anthropic: GET /v1/models
• Stripe: GET /v1/account
• GitHub: GET /user
• Slack: auth.test, which echoes identity rather than posting anything
• Supabase: the project ref is read straight out of the service_role JWT, so we know which host to ask without depending on bundle detection

None of these are billable. None of them write. They’re the cheapest “who am I” endpoint each provider offers, which is exactly the point: verification shouldn’t cost the victim anything.

What the answer changes

The verdict drives the grade, rather than the pattern driving it:

• Live, critical, CVSS 9.8, high confidence. Rotate now.
• Revoked, medium, CVSS 4.3. It still shouldn’t be in your bundle, but nobody needs to be paged.
• Unverified, graded on format alone, and labelled as unverified rather than dressed up as a maybe.

That middle row is the one that earns trust. A tool willing to tell you something is less serious than it looks is a tool you believe when it says something is worse.

The keys we refuse to verify

An AWS access key ID can’t be verified without the paired secret access key. We only ever recover the ID from a bundle, and we have no interest in holding the other half. So AWS findings stay unverified, permanently and on purpose.

Saying “we don’t know” is a feature. A scanner that guesses to fill a column is a scanner that will eventually guess wrong about something that matters.

You can turn the whole layer off with FLARE_DISABLE_SECRET_VERIFICATION=1. Everything degrades to unverified rather than failing.