Hooks are hooks: compiling in your standards, in any language
Work processes and how to mentor a younger generation of engineer have been on my mind a lot lately. How do you teach a mindset or a way of examining problems? How do I share the experiences and history that help me know what to do, which tools to pick, which guardrails matter? I wrote very specifically about some options in Reducing the cost of the Nth and put the working example on GitHub at jasondostal/dotnet-api-platform, but it’s all .NET focused and that only helps if .NET is your thing. The concepts underneath it aren’t .NET concepts though, they aren’t even LLM assisted coding concepts. This post is a language-neutral version of my previous article factored out into concepts you can apply to most any project or framework you are working with.
“Good judgement is the result of experience, and experience the result of bad judgement,” goes a saying attributed to Mark Twain. How do I “know” what to do, how do I know what’s right? Decades of scars, lots of hard-won experience.
Two scenarios
You don’t have to take my word on the scars, consider these scenarios in the context of your own environment.
Scenario one. The business announces that account IDs are growing from 10 characters to 12. You have 20 APIs across 20 repos, built by different teams over eight years. There’s no shared model and no shared contract — each one has its own hand-written accountId field, validated (or not) its own way, documented (or not) in its own wiki page. How long does that change take? How do you even find all the places it lives? How confident are you in the list when you’re done? Now imagine instead that every one of those APIs was generated from contracts that referenced a shared type, a shared model.
Scenario two. It’s 2am and you’re paged. A vendor integration is rejecting records and the vendor swears they’re sending valid data. You go looking for the raw vendor responses so you can see what actually arrived — and there aren’t any. Somebody’s wrapper deserializes the response straight into a domain object and throws the original payload away. The evidence you need to settle the argument was in your process’s memory for about 40 milliseconds, six hours ago. Now you’re instrumenting a production system in the middle of the night and waiting for the failure to happen again. Instead of bringing this up at your next team retro (again), imagine the impact an automated architecture test would have, one that would fail your build whenever logging wasn’t handled to spec.
That’s really the whole idea. Intentions, good or bad, that live in meeting minutes, wikis or other text repositories decay quickly. Standards and intent that live as executable, deterministic checks are living, breathing standards.
On ratchets
There’s a term for this mentality, it’s a ratchet.
A ratchet only turns one way. Every surprise becomes a hard check, the check goes on the lowest rung it can run on (more on that later), and everything clicks one tooth tighter. That particular type of issue becomes something that’s very unlikely to ever happen again, and if you add the check to as low of a rung as you can, you will know when it does early enough to fix it. Twenty years of this and you have what looks from the outside like a slightly worrying amount of paranoia regarding the breadth of failure modes you want to prevent.
How do you know what to do, what to check when you are starting out? You don’t, and that is absolutely ok. Nobody does and anyone that tells you they do have all the answers doesn’t truly understand the situation. You learn, you grow and your judgement changes as you do and that is a wondrous journey and one of the pure joys of this profession.
This doesn’t just work on new codebases either. Did you inherit a monster app with 4,000 warnings in the build and who knows how many failures? That’s OK! Adjust your ratchet so that the 4,000 warnings are the high water mark and tighten it down from there as you incrementally work the debt down.
LLMs respond to ratchets very well, by the way. Numbers, metrics, hard verifiable checks are the type of thing they THRIVE on.
You said this wasn’t about LLMs?!
I lied, this is more than just old crufty tech guy talk. LLMs made writing code cheap, but it didn’t make any of the other parts of software development cheap. In order to code fast, in order to ship fast, you need to be able to validate, verify and test fast.
Everyone is asking the question “how do I trust AI-generated code”, and that is everyone coming to terms with the new bottlenecks in software development.
The question isn’t “how can I trust the AI’s code”, the question is “how can I trust the code being merged into my project regardless of author”. LLMs can crank out slop. Then again… so can humans. If you make quality something that can be deterministically checked, the author stops mattering. This has the side benefit of making your project self documenting, something that humans and LLMs both appreciate. Well written analyzer messages with references to decision documents means your project onboards developers to itself rather than you having to impart knowledge that only exists in your head. It teaches humans and guides LLMs forever, at zero marginal cost.
Who tests the tests?
This is where the paranoia kicks in. An LLM grading its own homework is a problem: models are famously good at writing passing but vacuous tests, and the model that wrote the code is the least skeptical reviewer it will ever meet.
Mutation testing is something that can “test the tests”. The tool mutates your code — flips a > to a >=, deletes a null check, inverts a conditional — and re-runs the suite. Stryker covers .NET and JS, mutmut covers Python. Meta deployed LLM-driven mutation testing across Facebook, Instagram and WhatsApp to harden compliance coverage at scale.
Mutation runs are slow and a good candidate for nightly runs or only on PRs.
The feedback latency ladder
In the agile, CI/CD centric world the concept of “shifting left” is king. I picture it as a ladder. Every check lives on some rung, and the lower the rung, the cheaper the feedback. Your job is to push every check down to the lowest rung it can possibly run on — as close to the person, or LLM, that can directly fix the problem as you can get it.
The bottom rung: live in the editor. Feedback in milliseconds, before anything is saved or committed. EditorConfig, IDE analyzers, format-on-save, and your coding harness’s hooks — Claude Code hooks, pre-edit and pre-commit hooks, whatever your agent tooling supports. A check here fires while the author, human or LLM, still has full context, and it costs essentially nothing.
The middle rung: quick local. Seconds to a couple of minutes, still on your machine. The compile, the linters, architecture tests, the unit suite. Anything deterministic that a developer or an agent can run in a loop without leaving the repo belongs here.
The top rung: the pipeline. Minutes to hours. CI builds, full integration tests, mutation runs, deployment gates. Necessary — this is your last deterministic word before code ships — but every check that only lives here is a check somebody waits on.
Off the ladder: production. Days, plus an incident report and a 2am page. Everything that slips past the rungs lands here, at maximum cost.
Checks should happen early and often is what I’m getting at. Use your coding harness hooks to prevent secrets from even being committed locally on disk so you don’t have to wait until a pipeline scanner picks it up on your agent’s PRs. Use Playwright so your agent can run interactive tests in your apps before calling features done so that you don’t have to spend 30 minutes “looking for the bugs”. Mocks, stubs and other unit testing techniques enable your agent to confirm functionality before a commit, much less a full deploy to a dev environment. Direct feedback as early and as verbose as possible is my ethos here.
One thing I want to call out is your agent instruction file (CLAUDE.md, AGENTS.md, whatever your harness reads). Keep this document as light as possible. Standards in prose are a failure waiting to happen. Use your instruction file to point your agent to the tests, the layout of the project, where to get more information and any environmental concerns. It should never be a control.
The map: concern → mechanism → your stack
So how do you know what to build, how do you know what tool to use? It will take time but here’s some of the things I’ve learned to rely on over the years.
| The scar | The mechanism | .NET | Python | JS / TS / frontend | Anywhere |
|---|---|---|---|---|---|
| The API drifted from its documentation | Contract-first + conformance testing | TypeSpec → OpenAPI, Schemathesis | Schemathesis, Hypothesis | Prism mocks, openapi-typescript | Spectral, Prism — these are contract-side and language-agnostic already |
| Someone used the forbidden thing again | Banned APIs / rules-as-code | Roslyn analyzers, BannedApiAnalyzers | Ruff custom rules | ESLint no-restricted-imports / no-restricted-syntax | Semgrep — write a rule once, enforce a pattern across every language in the org |
| The architecture eroded one shortcut at a time | Dependency boundaries as failing tests | NetArchTest, ArchUnitNET | import-linter | dependency-cruiser, eslint-plugin-boundaries | ArchUnit is the Java original; the idea ports everywhere |
| It worked last release | Snapshot & property-based testing | Verify | syrupy, Hypothesis | Vitest snapshots | — |
| The tests passed but tested nothing | Mutation testing | Stryker.NET | mutmut | Stryker | ratchet the kill rate |
| The app was compliant; the infrastructure wasn’t | Policy-as-code for IaC | PSRule for Azure | Checkov | — | OPA / Conftest — policy against any YAML, JSON, or Terraform |
| The UI broke and nobody noticed until a customer did | End-to-end, visual, and accessibility gates | — | — | Playwright, axe-core, screenshot diffs in CI | Playwright tests any web frontend regardless of backend language |
| The secret is in the git history forever | Scan on commit and in CI | gitleaks | gitleaks | gitleaks | gitleaks |
| Twelve repos, nine versions of the same package | Central version pinning | Central Package Management | uv lockfiles | pnpm catalogs | Renovate / Dependabot |
| The style argument that ate a sprint | One formatter, no configuration debates | EditorConfig, dotnet format | Ruff format | Prettier | pre-commit framework to bind it all |
A few notes on the map:
- Semgrep is a great cross language analyzer.
- PowerShell people, you’re not left out — PSScriptAnalyzer takes custom rules and Pester is a real test framework.
- Contract column is thin most of it is language agnostic.
- This applies to Infra as Code too. PSRule for Azure ships Microsoft’s Well-Architected guidance as executable checks against your Bicep and ARM templates; OPA/Conftest does the generic version for any structured config.
Watching it work
The dotnet-api-platform repo is one of my working examples of all of this stacked together. The contracts compile to code, analyzers enforce the audit path, architecture tests guard the boundaries, and the pipeline has hard blockers. The fun starts when you work with an LLM inside of a project with this structure.
Point a coding agent at a task in a repo set up like this and watch it write some “bad code”. It will trip analyzers, read errors, interpret policy from the message, fix or refactor its code and recheck it all without me in the loop. I can’t look at all of the code my agents write at the pace they can go. I’ve also always worked in larger organizations and have never had the luxury of enough time to become intimately familiar with every line of code my teams produce or use.
Does this fix everything, catch every failure? No, it couldn’t possibly. It does, however, provide an extensible framework for putting a permanent stop to the next class of bug that crops up.