Articles

A03:2021 is Injection. That's an address, not a severity. What the OWASP Top 10 ordering actually encodes, why 'we cover the Top 10' is a breadth claim and not a depth claim, and how the 2021 categories map onto CWE classes and what a source linter can and cannot see.

To benchmark a security linter or SAST tool, you need labeled examples: 'this code is vulnerable,' 'this code is safe.' That labeling decision IS the ground truth. It's made by a human, not an algorithm — and the benchmark inherits every assumption that human brought.

CWE-89 means SQL injection. That's a name, not a verdict. What the Common Weakness Enumeration actually encodes — views, abstraction levels, the Top 25 formula — and why a CWE id can never carry a severity.

A scanner can print CVSS 9.1 and the word LOW on the same line. What the score actually measures, what it doesn't, and why you should gate on the number — never the word.

An index looks like a measurement of 'the market.' It's a composite — a committee picks the members, a rule sets the weights, a calendar sets the rebalance. Editorial choices dressed as a number. Every overall score works this way; the honest ones publish the weights, the rationale, and what happens when the weights move.

A cancer screen can be 90% accurate and still wave through two-thirds of the tumors it exists to catch. The confusion matrix is the honest ledger behind that number — and accuracy is usually the last thing you should read.

Two radiologists agreed on 92 of 100 scans. Cohen's kappa scores that at 0.16 — barely above chance. When labels are judgment calls, raw percent agreement is the wrong check. Here is the right one.

Every benchmark on this site compares my own ESLint plugins to competitors — a real conflict of interest, not a disclaimer I can write my way out of. Here's the actual, checkable process I use instead of asking you to trust my verdict.

When a measure becomes a target, it ceases to be a good measure. Goodhart's Law is the failure mode where moving a number and improving the thing the number stood for split into two projects — and the cheaper one wins by default.

A blood panel, a fund's track record, and a security benchmark can all report a clean, confident number that is quietly wrong. Measurement bias doesn't announce itself — the five kinds, and how to catch each before you publish.

A test that scores 95% precision on a balanced sample can be right under 2% of the time when the thing it hunts is rare. That's not a bug in the arithmetic — it's the base rate problem, and it's why benchmark conditions rarely predict field behavior.

Entropy-based credential detection misses a surprising share of hardcoded secrets — because most secrets look like regular strings until you see their context. Our no-hardcoded-credentials rule fired 842 times on vercel/ai. The peer plugin fired 380. I assumed we had better recall — until I sampled. 807 of the 'extra' findings were TypeScript union-type literals, error class names, and the string 'test'. Here's what catches the rest.