Articles

Crypto failures are not one mistake repeated. They are four distinct layers of correctness, and each one looks fine from the layer above it.

Same prompts, both CLIs, scored by the ESLint security plugins I wrote. Gemini tied Claude on JWT and MongoDB, won the NestJS round, and never shipped a high-severity injection bug. The real finding: both frontier models miss the exact same hardening — the negative space static analysis was built for.

Same prompt. Claude Sonnet 4.6 got 6 security errors from eslint-plugin-nestjs-security. Gemini 2.5 Flash got 2. Both missed rate limiting on auth endpoints — and Gemini got guards, validators, and serialization right where Claude didn't.

I gave Claude one prompt and got 200 lines of correct NestJS. TypeScript compiled clean. Then I ran eslint-plugin-nestjs-security. 6 errors, 3 seconds. Here is what it found and why each one is an AI failure mode.

The 'most dangerous' AI model fixes 93% of the database vulnerabilities it writes; the 'safest' fixes 45%. Same 700 functions, opposite conclusion. When I broke a 5-model security benchmark down by domain — database, auth, file I/O, command execution — the rankings inverted. Aggregate numbers hide domain expertise. Here's the per-domain data and how to gate it in CI.

700 AI-generated functions. 5 models. The cheapest model leads the aggregate leaderboard — but one model writes clean-scanning JWT code where the flagship is flagged every time, and the 'worst' model fixes 93% of database bugs. The rankings don't tell you any of this.

When AI fixes a security bug, the original finding disappears — but 1 in 3 'fixes' quietly introduced a brand-new vulnerability in a different category. I tested this across 3 remediation rounds with Claude (opus alias, Feb 2026 run) using two approaches — ESLint-guided feedback vs. prompt engineering alone. I call it the Hydra Problem, and it exposes a fundamental limit of 'fix it again' workflows.

AI coding assistants are incredible—until they introduce security holes. I ran an experiment asking Claude (Haiku 3.5, Sonnet 4.5, Opus 4.5, Opus 4.6) to generate 80 Node.js functions — 20 real-world prompts across 4 models — with zero security context using my Claude Pro subscription. 65-75% had vulnerabilities. Then I tested if static analysis could help the models fix their own mistakes.

A reproducible performance benchmark: eslint-plugin-import vs eslint-plugin-import-next across 1K/5K/10K files. The no-cycle rule goes from 148.59s to 2.71s at 5,000 files (54.8x measured, n=3). Both plugins compute strongly-connected components and both re-run per-edge BFS path-finding — I measured that disabling the old plugin's SCC check changes nothing, so the gap isn't 'no SCC at all'; the exact remaining cause is honestly unresolved, and the numbers are reproducible from the linked result JSON.

Prompt injection via unvalidated user input, system prompt exfiltration, missing output validation, tool call injection — TypeScript types look correct in every case. Here are the 19 ESLint rules that catch them in CI.

I scanned 356 files across 10 public Vercel AI SDK apps for one prompt-injection bug and found it in an official Vercel template. The exploit, why string sanitization is a trap, and the CWE-74 ESLint rule that enforces a real validation boundary at write-time.

AI assistants leave demo keys, placeholder passwords, and bare config literals in source — CWE-798 at scale. One ESLint rule catches the literal and prints a CWE/CVSS/compliance/fix line an agent can parse, so the model that wrote the bug can close it. Walked against eslint-plugin-secure-coding v3.3.2.