Articles

Crypto failures are not one mistake repeated. They are four distinct layers of correctness, and each one looks fine from the layer above it.

A Serverless plugin with 142k weekly downloads last shipped a release in 2021. Downloads said taken; the registry said abandoned; the platform's roadmap said something else entirely. The incumbent map — weekly downloads × years since last publish × open-issue decay — is how I decide what to build next, including the pick I shipped against two months before the fact-check caught up with me.

Our loudest serverless claim — 'ghost billing after sls remove' — failed our own E2E run. The trap is real, but it lives on a different path entirely. The full story of re-scoping a claim in public, and why the narrower version came out stronger.

My serverless benchmark prints 0.88 vs 0.3025 — real measurements running on editorial weights. Every index has a committee; the only real question is whether it publishes its votes. Mine are here: seven dimensions, the null policy, and a ±50% attack on the weights.

The claims registry: one file where every public claim has a row, every row points at a versioned result file, and a 90-day staleness rule decides when a claim must re-earn its place. The contract, the lifecycle, and how any product adopts it in an afternoon.

A clock that runs ten minutes fast is perfectly reliable — the same wrong time every read — and completely invalid. Reliability and validity are independent properties, and confusing them is how confident dashboards stay green while the thing they measure quietly fails.

A linter runs at edit time. CodeQL runs on your PRs. SonarQube runs in CI. All three get called 'static analysis' — and vendors call the linter 'SAST.' The three are not the same thing, and the differences decide what each can and cannot catch.

A small test set is enough to prove a detector misses an entire category. It is not enough to support a confidence interval on a score. Both are true — here's how to know which claims your sample size actually supports.

GDP is not prosperity, price is not value, a test score is not learning. Every dashboard number stands in for a construct it can't fully measure — the discipline is knowing each gap and pairing every proxy with the counter-metric that catches its gaming.

A tool with 100% recall and 2% precision and a tool with 2% recall and 100% precision score the same F1: 3.9%. One number cannot tell you which failure mode you're buying. Definitions, the security asymmetry, F-beta, and a worked example with two static-analysis tools that share a precision score and behave nothing alike.

p > 0.05 — the line of output that separates a real finding from a lucky one. What a p-value actually means, the three ways it gets misread, and when running a significance test is the wrong move entirely.

A result is reproducible when the same team runs it twice and gets the same number. It is replicable when an independent team runs it and gets the same number. Most published results clear the first bar; far fewer clear the second — and that gap is where trustworthy numbers quietly break.