Paul Brodner's Blog

A Bug Is Rarely Just a Bug — And Now It's Attached to No One

Wed, 26 Aug 2026 09:30:00 +0000

There’s a sketch I keep coming back to: an iceberg. Above the waterline, what QA sees — bugs, a broken process, bad architecture, a testing gap, wasted engineering time. Below it, what the problem is actually attached to — someone’s decision, someone’s budget, someone’s reputation, someone’s promotion, years of defending “why we do it this way.”

That was the hard part of the job for a decade. Not finding the bug. Surviving the politics of the bug.

Then AI arrived, and something strange happened. The iceberg didn’t get smaller. It got emptier.

The accountability vacuum

Ask who owns a defect now and you get a shrug. The story was drafted by AI from a two-line prompt. The code was generated by AI from the story. The tests were generated by AI from the same story. The pull request was reviewed by AI, approved by a human who scrolled to the bottom in four seconds because there were nine hundred lines and eleven other PRs waiting.

Nobody defends this code, because nobody wrote it.

We used to complain that the political layer of the iceberg made bugs hard to fix. We were wrong about what it meant. The politics were painful, but they were evidence of ownership. Someone fought for that design. Someone’s name was on it. You could find that person, argue with them, and eventually change their mind — and when they changed their mind, the system actually changed.

Now the fight is gone and so is the leverage. “The AI wrote it” is a perfect alibi. It absorbs blame without ever learning anything.

But the alibi doesn’t survive a second question: who orchestrated the AI? Someone wrote the two-line prompt. Someone accepted the story without checking it against a real user. Someone ran the pipeline, and someone clicked merge. Every one of those is a human decision, and none of them stopped being decisions because a model executed them — an author who dictates to a typist still wrote the book. The vacuum isn’t real. It’s unclaimed. And unclaimed accountability is something a team ends up with by default, never something it chooses on purpose.

The closed loop

Here’s the part that should worry us more than accountability, because it’s structural rather than cultural.

A test is only meaningful if it comes from a source of truth that is independent of the thing being tested. That’s the whole idea. Someone decides what correct behavior is; something else checks whether the system does it.

Look at the chain we’ve built:

The story is generated from a prompt nobody validated.
The code is generated from the story.
The tests are generated from the story.
The review is generated by the same class of model that generated the code.

Every artifact in that chain derives from the same unexamined assumption. So when the pipeline goes green, what has it actually proven? Internal consistency. Nothing else. The code does what the story said, and the story might have been an invention — a plausible-sounding paragraph that has never been checked against a real user, a real regulation, or a real business rule.

We are not testing whether the software is right. We are testing whether the software agrees with itself. And it always will.

This is why the metrics look so good and the escaped defects don’t improve. Coverage is up. Test count has tripled. Cycle time is down. And production keeps surprising us, because coverage of the wrong specification is not quality — it’s confidence with better formatting.

I made a version of this argument in an earlier post about the gap between testing and quality assurance. This is that same gap one level upstream: not just that AI tests shallowly, but that the artifact setting the depth was never independent in the first place.

Yes, this is checkbox testing

Let’s stop being polite about it. If QA’s contribution is that a suite of AI-generated tests ran against AI-generated code implementing an AI-generated story, then QA has been reduced to a green tick in a pipeline. That’s not testing. That’s ceremony.

The uncomfortable truth is that AI is exceptionally good at producing the artifacts of quality and no better than us at producing quality. It generates test cases, coverage reports, traceability matrices, risk registers, sign-off documents. All the paperwork of assurance, at scale, in seconds. In a regulated industry this is genuinely dangerous, because the paperwork is what auditors see and the paperwork now looks flawless.

Picture the FDA audit. “Who authored this validation evidence, and against what requirement?” The honest answer is a model, working from a prompt nobody kept. In a domain where the trace is the point — where validation means demonstrating that a specific human verified a specific intended use — flawless paperwork with nobody behind it isn’t a compliance shortcut. It’s the finding.

Comprehension is the bottleneck now

The other shift that gets missed is where the constraint sits. Producing code used to be the expensive part, and review was cheap by comparison. Now production is nearly free and comprehension is the scarce resource. Quality is no longer limited by how fast we can build — it’s limited by how much a human can actually hold in their head. If your team generates more output per day than your humans can meaningfully read, you don’t have a productivity gain. You have an unreviewed codebase with good velocity charts.

So what is the human actually for?

“Humans will orchestrate” is the comfortable answer, and it’s half a sentence short of being useful. Orchestrate what, exactly?

The one thing AI structurally cannot do is decide what correct means. Testers have a name for this — the oracle problem. A model can tell you what the code does. It cannot tell you what it should do, because “should” comes from outside the software: from a clinician’s workflow, a patient’s confusion at 2am, a regulator’s intent, a contract, a consequence someone will live with.

That’s the job now. Not writing checks — owning the oracle.

Concretely, that means:

Break the closed loop. Tests must not derive from the same artifact as the code. Human-authored acceptance criteria, written before generation, become the ground truth. If a human didn’t specify it, an AI-written test proves nothing about it.

Keep provenance. Every requirement should carry a name and a reason. Who decided this? Against what evidence? The iceberg’s lower half was ugly, but it was traceable. Reintroduce the trace on purpose.

Put a name on the merge. Not a rubber stamp — an actual claim: I read this, I know what it’s for, and I’ll own the incident. If nobody on the team is willing to make that claim about a change, that refusal is the signal. The change waits.

Cap generation at review capacity. If we can’t read it, we don’t merge it. Generating less is a legitimate quality control.

Sample adversarially. Don’t review AI output uniformly. Pick the highest-consequence paths and go through them line by line, by hand, with hostility.

Change what you measure. Coverage and test counts are now trivially gameable by a machine. Measure escaped defects, and especially intent defects — the ones where the code worked exactly as specified and the specification was wrong. The dose reminder that fired at 9am server time instead of the patient’s time zone is an intent defect: every test passed, because the story said 9am and nobody who had talked to a patient wrote the story. That class of bug is invisible to generated testing, which is exactly why its count is the real report card. Same instinct as the mutation and negative-testing instruments I laid out previously — stop trusting a metric a machine can satisfy without understanding it.

Do the unglamorous human things. Talk to the users. Sit with support tickets. Ask the person who has been here six years why this module is fragile. None of this is in the model’s context window.

And if tomorrow there’s no AI?

Ask it literally first — outage, price change, vendor lock-in, an air-gapped client, a regulator who rules generated evidence inadmissible. Answer honestly, write it down.

Then ask the version that actually keeps me up. What happens to a QA engineer who has never designed a test from first principles? Who has never read a specification and felt that itch that something in paragraph four contradicts paragraph nine?

Those are muscles. They atrophy quietly, and you don’t discover they’re gone on a normal Tuesday — you discover it during an incident, when the model is confidently wrong and you have no independent way to know.

So keep a manual practice, as insurance rather than nostalgia. Write the risk analysis by hand first and let AI expand it, never originate it. Adopt one rule with teeth: anyone on the team must be able to explain, out loud, what a test is for and what its failure would mean — if nobody can, delete it, it was decoration. And train juniors on judgment, not prompts. Prompts change every six months; judgment compounds for thirty years.

The iceberg is still there

The bug is still attached to something. It always was.

The difference is that it used to be attached to a person with a budget and a reputation, and now it’s attached to a prompt nobody kept, a story nobody validated, and a review nobody read. The politics got easier. The epistemics got much, much worse.

Three things I’d take away from this:

“The AI wrote it” is an alibi, not an answer — accountability belongs to whoever orchestrated the AI. A human chose the prompt, ran the pipeline, and clicked merge, and that human owns the output the same way an author owns a book their typist produced. Quality assurance now means keeping that person in the loop by name — deliberately, as the accountable owner of what ships, not as a formality.

A pipeline of AI-generated story, code, tests, and review only proves the system agrees with itself. Coverage, test count, and green CI stop being quality signals the moment every artifact in the chain traces back to the same unvalidated assumption. They measure agreement, not correctness.

The job shifted from writing checks to owning the oracle. AI can tell you what the code does; it cannot tell you what it should do, because “should” lives outside the software, in a workflow, a regulation, a consequence someone else will live with. That’s the one part of QA that doesn’t automate.

Speed isn’t the enemy — unsigned work is. Somewhere in the chain, a human being has to look at the thing and say: I have understood this, I know what it’s supposed to do, and I am willing to put my name on it. That signature is the entire job, and it’s the one part we can’t generate.

Two Loops: Fast for Side Projects, Gated for Everything Else

Tue, 18 Aug 2026 09:00:00 +0000

In my last post on the SDLC skills, I described a fast loop — design, build, ship — that let me take two side projects from init commit to shipped in days. The velocity was real. But the post ended on a problem I didn’t have a clean answer for yet: the loop optimizes story by story, and nothing in it protects the coherence of the whole product. Individual features work. The thing as a whole quietly drifts.

That problem gets worse, not better, the moment the codebase isn’t yours alone to throw away.

A side project has one reviewer, no history to contradict, and a blast radius of exactly one person’s afternoon if something goes sideways. A codebase your team depends on has none of that slack. A bad seam isn’t a rough edge you notice later — it’s someone else’s incident. So I built a second loop for that context, and it looks nothing like the fast one.

Two Different Problems

The fast loop assumes you can afford to find out you were wrong after code exists. Review happens once, at the PR. If the story was underspecified, you catch it in the diff, fix it, re-ship. That’s a fine trade when the cost of being wrong is a wasted hour.

It’s a bad trade when the codebase is shared, load-bearing, or already has years of decisions baked into it that a fresh agent session can’t see. In that setting, the expensive mistakes don’t happen in the code — they happen upstream, in the assumptions nobody wrote down before implementation started. By the time you’re reviewing a PR, the ambiguity is already compiled in.

So the fix isn’t a better PR review. It’s moving the checkpoints earlier — before any code exists at all — and making each one something a human has to explicitly sign off on rather than glance past.

The Four Gates

That’s what /bro-feature is: a workflow with four sequential gates — Product, Architecture, Program Design, Vertical Slices — where each gate has to be explicitly approved before the next one starts, and none of them produce implementation code.

The shape matters more than the label on any individual gate. Product comes first and stays free of implementation detail — it’s forced to answer “what is this and why” before anyone’s thinking about endpoints. Architecture comes next, and it has to read the existing codebase before proposing anything — never design against an imagined system. Program Design pins down types, signatures, and the call stack before a single test is written. Only then do Vertical Slices start landing code, one proven capability at a time. At every arrow in that diagram, you can revise or backtrack — the loop doesn’t move forward on an assumption, it moves forward on an approval.

Compare that to the fast loop’s one checkpoint per story, and the difference is the point.

Why This Fixes What the Last Post Flagged

I named three specific failure modes in the last post. This workflow is a direct answer to each one.

Ambiguity that compounds silently. In the fast loop, a vague word like “toggle visibility” gets interpreted confidently and ships before anyone notices the gap. Gate 1 exists specifically to force that ambiguity into the open — no tech talk allowed until the problem and the success metric are stated plainly enough that there’s nothing left to misread.

An agent that doesn’t push back. Self-review catches obvious bugs, not “wait, doesn’t this contradict what we built last week?” Gate 2 makes that check structural instead of hopeful — it reads the current codebase first, so a conflicting pattern gets caught before it’s proposed, not after it’s merged.

Coherence that erodes across independently-shipped stories. The fast loop has no mechanism for noticing that two features solved the same problem two different ways. The gated loop does, because Architecture and Program Design are evaluated against what already exists every single time, not just against the ticket in front of you.

The Trade-off, Named Plainly

This is slower. Four approvals before a line of code exists is, by design, more friction than one approval after. That’s not a flaw to optimize away — it’s the actual mechanism. You’re paying review time up front specifically so you don’t pay debugging and re-architecture time later, in a codebase where “later” involves other people.

The choice between the two loops isn’t about which one is better. It’s about which mistake is cheaper where you’re standing. On a side project you can rewrite in an afternoon, optimize for velocity and let the PR catch the rest. On a codebase that outlives the sprint, optimize for catching the wrong assumption before it has a chance to become code.

Three things I’d take away from this:

The right loop depends on what breaks if you’re wrong. A fast loop and a gated loop aren’t competing philosophies — they’re tuned to different failure costs. Match the loop to the codebase, not to habit.

Move the checkpoint to before the mistake, not after it. Catching an ambiguous requirement in a PR is catching it too late — the assumption is already compiled into the implementation. Gates exist to catch it while it’s still just a sentence.

Coherence has to be checked against reality, not against the ticket. A workflow that reads the existing codebase before proposing anything catches drift that a story-by-story review structurally cannot.

The fast loop taught me that velocity was never the constraint. The gated loop is what you build once you accept that, on a codebase that matters, correctness of the assumption is the constraint — and by the time it’s code, it’s too late to gate it.

The QA Layer Your AI Toolchain Won't Build For You

Thu, 25 Jun 2026 10:00:00 +0000

In Part 1, I made the case that AI has created a dangerous gap between testing and quality assurance — and that many engineering organizations are filling that gap with confidence rather than rigor.

The response I usually get to that argument is: “Okay, so what do we actually do?”

Fair. Let’s get specific.

There are five instruments worth putting in place. Some are technical. One is organizational. All of them are grounded in recent research and real production experience. None of them require you to slow down or abandon AI-assisted development — they require you to be smarter about what you trust.

Instrument 1: Mutation Testing — Stop Measuring Coverage, Start Measuring Effectiveness

This is the most important one, and the most underused.

Here’s the problem with coverage metrics: they tell you whether your tests ran a line of code. They don’t tell you whether your tests would catch a bug in that line of code. A test can execute a function without ever asserting anything meaningful about it. Coverage goes green. The bug ships.

Mutation testing flips this. Instead of asking “did the test run?”, it asks “if I deliberately break this code, does the test catch it?” It injects controlled defects — mutations — and measures your test suite’s kill rate. If your tests don’t catch the mutation, they’re not actually testing what you think they are.

This is well-established technique, but it’s had a resurgence for a specific reason: AI makes it practical at scale. One engineering team paired AI test generation with mutation testing and grew their suite from 12 to 33 high-quality tests while raising their kill rate from 57% to 80% — the difference between “tests that run” and “tests that catch real defects.”

Meta productized this at scale. Their Automated Compliance Hardening (ACH) tool runs mutation testing across Facebook, Instagram, WhatsApp, and their wearables platforms. Privacy engineers accepted 73% of the AI-generated tests, with 36% judged as genuinely privacy-relevant — a meaningful signal in a domain where false confidence is genuinely dangerous.

Gartner is now actively advising teams to integrate mutation-guided test hardening directly into pull request workflows.

What to do: Integrate a mutation testing framework (Pitest for Java, mutmut or Cosmic Ray for Python, Stryker for JS/TS) into your CI pipeline. Set a minimum kill rate threshold as a quality gate — not a coverage threshold. Start with your highest-risk modules.

Instrument 2: Property-Based Testing — Test Invariants, Not Just Examples

Most AI-generated tests are example-based: give the function this input, expect that output. The problem is that AI models generate tests by predicting what tests typically look like for code like this. They confirm the code does what it does — they don’t challenge whether it handles what it should.

Property-based testing is a different approach. Instead of testing specific examples, you define invariants — rules that must hold true for any valid input. You then let the framework generate thousands of random inputs and verify the invariant holds across all of them.

Recent research using property-based testing as a validation layer over AI-generated code showed 23–37% improvements in correctness over standard test-driven approaches — specifically because it breaks what researchers called the “cycle of self-deception,” where AI-generated tests share the same blind spots as the code they’re meant to validate.

This matters for a simple reason: the bugs AI misses aren’t random. They cluster around edge cases, boundary conditions, and combinations of inputs that no typical example would surface. Property-based testing systematically attacks those blind spots.

What to do: Adopt Hypothesis (Python), fast-check (JavaScript), or QuickCheck (Haskell/Erlang) for your highest-risk business logic. Task your senior engineers with defining the invariants — this is the judgment work that AI genuinely can’t do well. The framework handles the rest.

Instrument 3: Mandate Negative Testing — Explicitly

This one sounds obvious. It isn’t practiced.

Research consistently shows that AI testing agents have a documented tendency to avoid negative test scenarios — they unconsciously “correct” the flow toward a positive outcome, masking potential failures. The result is test suites full of happy-path coverage and almost no adversarial coverage.

This isn’t a subtle bias. It’s a structural one. LLMs are trained to produce helpful, correct-looking outputs. A test that “fails” by catching a bug looks like a failure to the model, not a success.

The practical consequence: if you don’t explicitly instruct AI to generate tests that expect failure, it largely won’t. And if you don’t review your test suite for the ratio of positive to negative scenarios, you won’t notice until production teaches you.

What to do: Make negative test coverage a code review criterion, not just a suggestion. When using AI to generate tests, explicitly prompt for failure scenarios, boundary violations, invalid inputs, and concurrent edge cases as separate passes. Require a minimum ratio of failure-path tests for any module handling user input, authentication, or financial logic.

Instrument 4: LLM-as-Judge for AI-Powered Products

If you’re building on top of LLMs — not just using AI to write tests for traditional code — you have an additional problem. The outputs are non-deterministic. You can’t assert that a specific string was returned. Coverage metrics mean nothing.

The emerging solution is LLM-as-Judge: use a separate model to evaluate the quality of your system’s outputs against explicit rubrics, and wire this into your CI pipeline as an automated quality gate.

This works — with a critical caveat. A recent engineering guide found that LLM-as-Judge is only reliable as a CI gate if it achieves at least 80% agreement with human judgments on your specific task type before you deploy it as an automated check. Below that threshold, you’re automating noise.

What to do: For your highest-stakes prompts, write explicit evaluation rubrics. Calibrate your LLM judge against 15–20 human-labeled examples for each rubric. Only promote it to an automated gate once you’ve validated the agreement rate. Treat the judge as a product that needs its own QA — because it does.

Instrument 5: Don’t Hollow Out Your QA Function

All four of the above are technical instruments. This one is organizational, and it’s the one most likely to be ignored.

The pattern I’ve watched play out in org after org: AI testing tooling lands, velocity goes up, test volume goes up, and QA headcount quietly gets redeployed or not backfilled. The logic seems sound — the machines are covering it.

What walks out with those people isn’t test execution. It’s domain knowledge, institutional memory, and the judgment about what actually matters to test. The ability to ask “what are we not testing, and why?” That’s not a function AI has taken over. It’s a function that’s just going unfilled.

The fix isn’t to resist AI in your quality practice. It’s to be honest about what you’re actually replacing. AI scales execution. It doesn’t replace the engineer who knows that the edge case in the payment retry logic caused three incidents last year, or the one who reads the spec and notices the requirement nobody thought to test.

What to do: Redefine the QA role in your org around test strategy, not test execution. Measure your quality engineers on the quality of failure mode analysis, on the clarity of acceptance criteria, on the signal-to-noise ratio of your test suite — not on the number of tests written. Make quality engineering a senior discipline, not a task to be automated away.

The Framework in One Sentence Per Layer

Mutation testing: stop measuring whether tests ran; measure whether they catch bugs
Property-based testing: stop testing examples; test invariants
Negative testing mandates: stop confirming the happy path; challenge it
LLM-as-Judge: for AI products, automate evaluation — but calibrate it first
QA as strategy: stop treating quality engineering as execution; treat it as judgment

None of this requires slowing down. It requires being honest about what AI is actually doing for you — and what it isn’t.

A Final Note for Engineering Leaders

The teams I’ve seen navigate this well share one thing: they treat “can this ship?” and “should this ship?” as separate questions, owned by different people with different mandates.

AI is very good at answering the first one. The second one still needs you.

If Part 1 resonated, the question worth taking to your next engineering review is simple: when did someone last ask what you’re not testing, and why?

Can AI Test Thoroughly? Why the Industry Is Sleepwalking Into a QA Crisis

Thu, 18 Jun 2026 19:18:00 +0000

Lately, I’ve been seeing a familiar pattern play out across engineering teams.

A developer ships an AI-powered feature. It works. It demos beautifully. The team moves on.

Three months later, production is on fire — and nobody saw it coming. Not because they skipped testing. Because they confused testing with quality assurance.

That distinction used to be a footnote. Now it’s the fault line everything cracks along.

Testing and QA Are Not the Same Thing

This is worth saying plainly, because the industry conflates them constantly.

Testing asks: does this work? It’s execution. You run cases, you check outputs, you confirm behavior matches expectation. It’s necessary. It’s not sufficient.

Quality assurance asks: is this good? It’s a systems-level question — about requirements, about risk, about what “correct” even means in context. It requires judgment, domain knowledge, and an honest understanding of failure modes you haven’t imagined yet.

AI is genuinely impressive at the first one. It can generate test cases at scale, cover surface area no human team could, run regression suites relentlessly, and catch obvious breakage fast.

But the second one? That’s where the gap is. And that gap is widening.

Why AI Makes This Gap Harder to See

Here’s the uncomfortable part: AI doesn’t just struggle with QA — it actively makes the problem less visible.

When you use AI to generate tests, you get volume. Hundreds of cases, fast. That feels like coverage. It looks rigorous. The dashboard goes green. Confidence goes up.

But AI-generated tests are bounded by what the model thinks is worth testing — which is largely a reflection of what’s common, expected, and well-documented. The scenarios that actually break production systems are usually none of those things. They’re edge cases that emerge from real user behavior, implicit business rules that were never written down, and interactions between systems that no single model has full context over.

You end up with a test suite that’s wide but shallow. It passes. The system ships. And somewhere in the parts nobody thought to question, quality is quietly eroding.

In regulated environments — medical devices, clinical trial software, anything under FDA scrutiny — this is not an academic concern. A shallow test suite that passes CI is not a validation artifact. Auditors ask what you tested and why. “AI generated it” is not an answer to the second question.

The Signals Engineering Leaders Are Missing

If you’re managing an engineering org right now, here are the questions worth asking honestly:

Who owns the definition of “good”? Not “does it pass CI” — but what does quality actually mean for this product, for this user, in this context? If the answer is fuzzy, your QA is fuzzy, regardless of what tools you’re using.

Are your test suites telling you what you want to hear? AI-generated tests optimize for coverage metrics, not for the failure modes that matter. If nobody is actively challenging what’s being tested, you have a yes-machine, not a quality signal.

What happens when requirements are ambiguous? AI executes well against clear specs. Real-world software is built on incomplete, evolving, sometimes contradictory requirements. QA is partly the discipline of surfacing those ambiguities before they become incidents. That work still requires humans.

Is your team moving from demo to production — or just to deployment? Demo-quality and production-quality are not the same thing. The gap between them is where QA lives. Shipping fast with AI tooling can close the distance to deployment while doing nothing to close the distance to production-readiness.

What This Looks Like at Scale

The pattern I’m worried about isn’t a team that skips testing. It’s teams that use AI to test more, ship more confidently, and gradually hollow out the QA function — because it feels redundant next to all that automated coverage.

QA engineers get redeployed or not backfilled. The institutional knowledge about what matters, what breaks, and why, starts to walk out the door. The test suite grows. The judgment behind it atrophies.

This is the sleepwalk. Not a dramatic failure. A slow, quiet degradation of the thing that makes software trustworthy — dressed up in green CI pipelines and velocity metrics.

What Good Looks Like

AI absolutely belongs in your quality engineering stack. Used well, it frees your best QA minds from repetitive execution so they can focus on what machines genuinely can’t do: understanding risk in context, defining what quality means for this product, and asking the uncomfortable questions nobody else is asking.

That means using AI to scale execution, not to replace judgment. Keeping humans accountable for test strategy, not just test generation. Treating QA as a systems-thinking discipline, not a checklist discipline. And asking regularly: what are we not testing, and why?

The teams that get this right will use AI to build faster and build better. The teams that don’t will build faster — and discover the difference in production.

Three things I’d take away from this:

AI scales execution; it cannot own strategy. Test generation at volume is real value — but strategy means deciding what to test, why, and what counts as good. That question requires a human who understands the product, the risk, and the users. No model has that context by default.

Green metrics are not a safety signal. A CI pipeline that always passes is a yes-machine until someone asks the right question. The job of QA isn’t to produce green — it’s to surface the things that shouldn’t be green. That job doesn’t go away when AI generates the tests; it becomes more important.

The institutional knowledge that walks out the door doesn’t come back. When QA engineers are redeployed because automated coverage looks sufficient, you lose more than headcount. You lose the accumulated judgment about why certain things break, what matters in this system, and which edge cases are actually dangerous. That knowledge took years to build and takes a production incident to notice it’s gone.

AI can test. It cannot assure. The question isn’t whether to use it — of course you should. The question is whether you’re still asking the hard, human questions that no tool can answer for you.

Local Agents: Small Models, No Frameworks, One Bus

Fri, 05 Jun 2026 10:10:00 +0000

The AI ecosystem nudges you toward big models and heavy frameworks. Neither is required when the task is clear. The smallest useful agent is a loop, a local model, and a file.

I built project_local_agents on that premise: production-ready local LLM agents from scratch — no LangChain, no CrewAI, just Python loops and typed dispatch. What I want to share isn’t what each agent does. It’s the architecture that makes the whole thing work without collapsing under its own complexity.

Small Models Do Real Work

The key insight is that most agent failures are scope failures, not model failures. The failure mode I see most isn’t “model too weak.” It’s “task too broad.” Nail the scope and a 3B model does the job.

MLX makes this practical on Apple Silicon. Local inference at 3B/7B scale — no API, no cost, no data leaving the machine. The models aren’t impressive in the general sense, but “impressive” is the wrong criterion. “Does it reliably extract structured output for this one task?” is the right question, and the answer is often yes.

What “small enough” looks like in practice: JSON schema output, one tool at a time, a clear stop condition. If you can write down what done looks like before you start, a small model can probably get you there.

An Agent Is a Loop

Here’s the entire pattern, before any framework reaches for it:

while True:
    decision = call_llm(context, tools)  # typed union: ToolA | ToolB | Done
    if isinstance(decision, Done):
        break
    result = dispatch(decision)
    context.append(result)

Five lines of logic. Everything a framework gives you — retry handling, tool routing, context management, observability — is built on top of this loop. The question is whether you need all of it before you’ve proven the loop works.

YAGNI applies here harder than almost anywhere else. The loop is the proof of concept. Add orchestration only when the loop provably can’t do it.

BAML handles typed dispatch: the LLM returns ToolA | ToolB | Done — no string parsing, no intent matching, no fragile regex. This matters because it separates two different contracts. BAML handles the LLM boundary: prompt templating, output parsing, model retries. Pydantic handles Python-to-Python boundaries: bus serialization, validation, schema. Don’t conflate the two — they serve different purposes and conflating them is how you end up with untestable spaghetti.

Context as Events, Not State

The agent doesn’t maintain external state. Instead, it builds a growing event log — XML-tagged JSON — that gets fed back to the model each turn.

<event type="fetch_result" ts="2024-01-15T10:30:00">
{"repo": "owner/project", "files": [...]}
</event>
<event type="analysis" ts="2024-01-15T10:30:05">
{"finding": "missing test coverage in auth module"}
</event>

This is debuggable, portable, and requires no external system. You can replay any session by feeding a recorded log back into the loop — which gives you reproducible tests for the dispatch and tool logic without stubbing anything.

One operational constraint worth designing around: keep the event log under 40% of the model’s context window. Past that threshold, the model starts dropping earlier events even when they’re still in-context. This is empirical — tested across 3B and 7B models on MLX. At 40%, recall degrades noticeably. The practical implication is that this bounds agent turn depth, which forces you to scope tasks tightly. The architecture reinforces the discipline from the section above: bounded tasks, not open-ended ones.

The Event Bus for Composition

When you have more than one agent, they need to share results. The temptation is a message broker — Kafka, RabbitMQ, something that feels like proper infrastructure.

The minimal version is a SQLite outbox table with per-consumer cursor rows.

CREATE TABLE outbox (
    id INTEGER PRIMARY KEY,
    agent TEXT NOT NULL,
    event_type TEXT NOT NULL,
    payload JSON NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE cursors (
    consumer TEXT PRIMARY KEY,
    last_seen_id INTEGER DEFAULT 0
);

Producers write results. Consumers read at their own pace. Independent deploy. Zero infrastructure. Agents don’t need to know about each other — they write to the bus and read from it. That’s the entire coupling contract.

This works until you need strict ordering guarantees, exactly-once delivery, or distributed consumers across machines. Cross those bridges when you actually reach them.

Why This Architecture Is Testable

This is the angle the “just use LangChain” crowd doesn’t cover.

The SQLite event bus is inherently inspectable. SELECT * FROM outbox WHERE agent = 'fetch_agent' is your assertion. No mocking the coordination layer — the bus is a file. Spin up a test SQLite in a temp directory, write events, assert consumer behavior. No fakes, no patches, no drift between test and production.

The event log is replayable. Capture a real production run, feed it back into the agent loop in CI, and you get a reproducible test for the tool and dispatch logic rooted in real behavior — the kind that actually catches regressions.

Typed dispatch means tool calls are assertion-friendly. isinstance(decision, FetchRepo) beats parsing "FETCH_REPO" out of a string every time. The type system does the work; the test just checks the result.

These testability properties fall out of the architecture for free. They weren’t bolted on afterward. The choices that made the system simple — SQLite over Kafka, typed unions over string parsing, event log over external state — are the same choices that made it testable. That’s not a coincidence.

Four things I’d take away from this:

Small models do real work when the task is clear. The failure mode is almost never “model too weak” — it’s “task too broad.” Fix the scope and the model size stops mattering as much as you think.

An agent is a loop. Start there. Frameworks add value eventually, but they cost you simplicity immediately. Don’t pay that cost until you’ve proven the loop isn’t enough.

Coordination doesn’t need infrastructure. A SQLite file with a cursor is enough to let independent agents share results without coupling them. Kafka-scale problems require Kafka. Most agents don’t have Kafka-scale problems.

If you can’t test it, you don’t own it. Inspectable state, replayable logs, typed assertions — all of this comes free from the architecture above. The testability wasn’t added later. It was already there.

The pattern isn’t complex — the complexity arrives when you reach for solutions before you’ve proven you need them.

Brain: Teaching Claude Code to Remember What You Already Taught It

Mon, 25 May 2026 10:00:00 +0000

Every correction you give Claude Code disappears the moment the session ends.

You say “don’t mock the database in integration tests — we got burned by that.” Claude adjusts. Writes the real-DB tests. Nails it. Tomorrow you start a new session and the first thing it suggests is… mocking the database.

This isn’t a bug. It’s the architecture. LLM sessions are stateless. Context windows are per-conversation. The model doesn’t carry forward what it learned about your codebase, your patterns, your hard-won corrections.

So I built a system that does.

The Existing Landscape

This problem isn’t new and there are real tools attacking it. GraphRAG (Microsoft) builds a knowledge graph from documents — entities as nodes, relationships as edges — good for multi-hop reasoning across a corpus. Mem0 is probably the closest thing to what I built: a memory layer for AI apps with per-user stores, recency weighting, and conflict resolution. OpenMemory MCP is Mem0’s MCP-native variant that exposes the same idea as a server Claude can call directly. MemGPT / Letta gives the LLM a paging mechanism — main context plus archival storage it can query mid-conversation. Zep is the enterprise end of the spectrum: a memory graph product for LLM applications.

All of these are infrastructure for app builders. You wire them into your pipeline, your agent, your product. Brain is different: it’s a developer’s personal KB that the assistant queries autonomously — no app, no pipeline, just a Stop hook and an MCP server. And the core bet it makes is different too.

The Problem Is Signal, Not Storage

The naive approach is to dump every transcript into a vector store and RAG over it. That doesn’t work for three reasons.

First, most of a session is noise — scaffolding, exploratory questions, dead-end debugging. If you index everything, retrieval drowns in irrelevant results.

Second, the valuable moments are sparse and specific. A user correction (“no, use exponential backoff, not uniform sleep”) is one message in a 200-message session. A task completion, an error pattern, a solution that finally worked — these are signal moments scattered across hours of conversation.

Third, the recall needs to happen proactively. You can’t expect the user to remember to search. When Claude is about to write retry logic, the system should surface the backoff lesson before the mistake repeats.

Architecture: Hook → Extract → Recall

Brain is a single-file Python CLI (~2,000 lines) that wires into Claude Code at two points: a Stop hook for extraction and an MCP server for recall.

Capture

When a Claude Code session ends, the Stop hook fires. It reads the session transcript (JSONL), filters for high-signal messages — errors, user corrections, task completions, explicit #cor markers — and sends them to claude -p (Haiku by default) with a strict extraction prompt.

The extractor scores signal moments by context: error clusters, correction language, solution patterns. It returns a JSON array of structured entries, each with a type (lesson, antipattern, decision, pattern, snippet), a signal level (HIGH, MED, LOW), and optional flags (AVOID, HARD-WON, 10X).

Store

Entries go into SQLite with FTS5 full-text indexing. Content-hash IDs make re-syncs idempotent — you can reprocess a transcript without duplicating entries. Soft-deletes (unlearned flag) let you remove bad entries without losing data.

The key design choice: fail-closed sync. If the Claude CLI call errors during extraction, the transcript is not marked as synced. It retries on the next run. Long-running sessions get re-extracted when they grow by ~20KB, so a session you left open all day doesn’t lose its last three hours of work.

Recall

Brain exposes three MCP tools that Claude calls automatically mid-conversation:

brain_recall — search the KB before writing code, making decisions, or debugging. Uses FTS5 with quoted-token safety and LIKE fallback.
brain_remember — save a new lesson during a session, without waiting for extraction.
brain_extract — trigger on-demand extraction using the current model instead of background Haiku.

When Claude is about to suggest a pattern, it queries brain_recall with concrete nouns from the task. If the KB has a matching antipattern or correction, it surfaces before the code is written — not after.

What 30 Days of Daily Use Actually Taught Me

I started Brain on April 24th. Forty-six commits later, across 11 evaluation versions, the system works — but getting here was a process of using it every day, inspecting the output, and fixing what was actually broken rather than what I thought would break.

Extraction quality is the bottleneck, not retrieval

My first instinct was to tune retrieval — better FTS queries, smarter ranking, more context in recall results. That was backwards. The real bottleneck was extraction: only 4.8% of transcripts were being successfully extracted in v10’s production audit. Timeouts, schema mismatches, I/O errors. When extraction did run, the lessons were high-quality. The system wasn’t wrong — it was just barely running.

The fix wasn’t algorithmic. It was operational: better timeout handling, retry logic, regrowth detection for long sessions. Old-school reliability engineering applied to an AI pipeline. (yeah, old school programming and debugging in this period of time ..)

Stale lessons are safer than I expected

My biggest fear was that Brain would recall an outdated lesson and lead Claude down the wrong path. I tested this deliberately in v10 — seeded a stale lesson about a function that had been renamed, then watched what happened.

Claude recalled the stale entry, recognized it didn’t match the current code, and explicitly noted the discrepancy. No harm done — just a few extra tokens. The model’s own verification instinct acted as a safety net. Safe to ship, even without a staleness expiration mechanism.

The `#cor` marker changed how I interact with Claude

Once I added the #cor tag (from correct - yeah is hard to name things) — a way to guarantee a correction gets captured — my feedback became more intentional. Instead of casually saying “no, not like that,” I started writing #cor that's the wrong approach, use exponential backoff not uniform sleep. The correction got extracted every time. The specificity improved because I knew it would be remembered.

A capture mechanism changes the input behavior. When you know corrections persist, you invest more in making them precise.

Single-file design was a deliberate trade-off

Brain is one Python file. No framework, no separate modules, no package structure. The mcp library is the only external dependency.

This makes it trivial to install (ln -s into $PATH), easy to audit (one file to read), and simple to debug (one log, one DB, one entry point). The trade-off is that the file is ~2,000 lines and covers CLI, MCP server, extraction, storage, and logging. For a solo tool that I use daily and iterate on rapidly, that trade-off pays off. For a team project, it wouldn’t.

The A/B evaluation harness proved its weight

I built a small eval framework early: 5 real-world test cases (FTS5 quoting, SQL upsert patterns, fork error handling, sys.exit() in libraries, enum validation), each with a hand-written seed lesson and an extractor-produced alternative. The eval scored both on precision, conciseness, and actionability.

Eleven versions later, every hypothesis I had about “this prompt change will improve extraction” was checked against real cases before shipping. Most hypotheses were wrong. The eval caught them. The discipline of measuring before deploying — old-school QA applied to prompt engineering — prevented several regressions that would have corrupted the KB.

Using It

pip install mcp
chmod +x brain.py
sudo ln -sf "$(pwd)/brain.py" /usr/local/bin/brain

brain install   # writes Stop hook + MCP registration
brain sync      # extract lessons from existing transcripts
brain doctor    # verify everything is wired up

After install, Claude Code calls brain_recall automatically via MCP. No manual steps. The CLI commands are for admin and debugging:

brain stats --health          # daily health check
brain search "cache" -n 10    # quick lookup
brain last 10                 # see latest extractions
brain recall-log 20           # what Claude is recalling
brain compact --dry-run       # preview duplicate cleanup

The --html flag on stats generates a self-contained dashboard with entry counts, signal distribution, and a recalls-per-day chart. Dark mode included.

What’s Not Solved Yet

Brain is not finished. It’s driving enhancements, not declaring victory.

Extraction coverage is still the biggest gap. Many transcripts fail extraction due to size limits, timeouts, or edge-case formatting. The retry mechanism handles transient failures, but systematic coverage requires better chunking strategies for very long sessions.

Cross-project learning works but lacks nuance. A lesson learned in one repo gets recalled in another, which is sometimes exactly right and sometimes irrelevant. Project-scoped filtering exists but the heuristics for “when to go global” need refinement.

Confirmation tracking is new and unproven. Brain now logs when a recalled entry appears in a subsequent session, which should eventually feed into a confidence score. The data is accumulating. The scoring isn’t built yet.

Brain is not finished and I’m not claiming it’s better than the tools above. For me it works — and it still needs improvements. What I can say is that it’s a light, single-file tool that reduces the groundhog-day loop: you correct something, it disappears at session end, you correct it again next week. That’s the whole value claim. Hard-won lessons don’t reset. For a daily dev workflow, that’s enough to be useful.

Three things I’d take away from this:

The hardest part of persistent AI memory is extraction, not retrieval. Getting the right lessons out of a 200-message transcript is a harder problem than searching a clean database. Invest in the capture pipeline first.

Daily use reveals what benchmarks miss. Eleven eval versions, 46 commits in 30 days — and the most important improvements came from using the tool every morning and noticing what felt wrong. A/B evals validate hypotheses. Daily use generates them.

AI tools benefit from old-school engineering discipline. Fail-closed sync, content-hash dedup, idempotent re-syncs, structured logging, soft-deletes — none of this is novel. All of it is necessary. The AI part is a thin layer on top of reliability patterns that have worked for decades.

The correction you make today should stop the same bug from shipping next week. That’s the entire value proposition — and getting it right is as much about plumbing as it is about prompts.

What Is AI, Really? I Had to Explain It to My Daughter

Mon, 23 Mar 2026 07:00:00 +0000

My daughter asked me what AI is.

Kids ask the best questions because they haven’t learned to accept vague answers yet. She wanted to know what it actually does. Not “it’s like a smart computer” — she’s heard that. What does it do?

I tried three different explanations before one landed. That process — stripping away everything until only the true thing remained — turned out to be more useful than anything I’d read. So here’s the version I’d give anyone who wants a straight answer, with no jargon and no skipping the hard parts.

What is AI?

Imagine you want to teach your dog to recognize cats. You can’t explain it in words — the dog doesn’t understand words. So instead, you show her a thousand pictures. “Cat.” “Cat.” “Not a cat.” “Cat.” After enough pictures, she starts to get it. Not because you explained the rules, but because she found the pattern herself.

That’s AI. Instead of writing rules for a computer to follow, you show it millions of examples and let it find the pattern on its own.

That’s the whole idea. Everything else is details.

What is an LLM?

LLM stands for Large Language Model. It’s the technology behind ChatGPT, Claude, and most of the AI assistants you’ve heard about.

Think of it like a very talkative parrot.

This parrot has spent its whole life listening — to every book, every website, every conversation it could find. Billions of words. And now it’s very, very good at continuing a sentence. You say “the sky is…” and it says “blue.” You ask it a question and it gives you something that sounds like an answer — because somewhere in all that listening, it heard a similar question with a similar answer.

The key word is sounds. The parrot doesn’t understand what it’s saying. It’s pattern-matching at extraordinary scale. Most of the time that’s enough to be genuinely useful. Sometimes it isn’t — and we’ll get to that.

How can it help us?

The parrot has read everything. That turns out to be genuinely useful — and not just for people who work in tech.

It explains complicated things in plain language. Medical results, legal documents, tax forms — things that used to require an expensive appointment to decode. Paste the confusing text in and ask “what does this actually mean?” You still need professionals for decisions. But now you can walk into those appointments understanding what you’re talking about.

It helps with writing. Not just drafts and edits — translation too. Writing in a second language, composing a formal letter when you’re not sure of the words, making something sound professional when it doesn’t yet. These used to require someone else’s help. Now you can get a first draft at 11pm when no one is available.

It does the tedious work so you can do the interesting part. Summarizing a 90-minute meeting. Sorting through a messy spreadsheet. Writing the first draft of a report you’ll then rewrite in your own voice. Debugging a piece of code at midnight. These tasks aren’t hard — they’re slow. AI compresses them, which means you spend more of your day on the parts that actually need a human brain.

It answers questions patiently, without judgment. My daughter asks me why the sky is blue about once a week. I love her for it. But there are questions people don’t ask because they feel like they should already know the answer, or they don’t want to bother anyone, or it’s 2am. AI answers those questions, every time, without making you feel small for asking.

It’s already built into the tools you use. Your email suggests replies. Your phone edits your photos. Your search engine summarizes results before you click. AI isn’t something you have to seek out anymore — it’s woven into everyday software. Understanding what it’s doing in the background helps you decide when to trust its suggestions and when to override them.

How can it harm us?

Here’s where most explanations get vague. I’ll try to be specific.

It makes things up. Confidently. Without knowing it’s doing it.

The parrot doesn’t have an “I’m not sure” voice. When it doesn’t know something, it generates something that sounds like an answer — because that’s all it can do. It has no way to check whether what it’s saying is true.

In 2023, two lawyers submitted legal documents to a court citing six case precedents. The judges, the quotes, the dates — all made up by ChatGPT. The lawyers were sanctioned. They had trusted the output without checking. The AI had no idea the cases didn’t exist. It just produced something that sounded like a legal citation.

For a question about a recipe, this barely matters. For a question about medication, symptoms, or legal rights, it can.

The rule: AI is a brilliant starting point. It’s a terrible final source. Always verify anything important.

Scammers now have access to something very powerful.

A finance employee received a video call from her company’s CFO asking for an urgent wire transfer. She could see him on screen. Several senior colleagues were on the call too. She trusted it. She wired $25 million.

Every person on that call except her was a fake — AI-generated faces and voices, in real time.

The old rule — “if I can see them and hear them, it’s real” — no longer holds. AI can clone a voice from a short audio clip. It can generate a face that moves and speaks. The people most likely to be targeted are the people who haven’t heard this yet.

If someone contacts you urgently asking for money or information — even if it looks and sounds exactly like someone you know — call them back on a number you already have. Not the one they gave you. That one step breaks most of these attacks.

It remembers what you tell it — and you might not think about who else sees it.

People paste things into AI without a second thought — medical symptoms, salary details, company documents, private conversations. Most AI tools send that data to remote servers. Some use it to train the next version of the model. That means your private input could influence what the system says to someone else later — or end up in a dataset you never consented to.

Before you paste something sensitive, ask: would I put this on a public forum? If not, check the tool’s privacy policy — or don’t paste it.

It changes what jobs look like — and not always in the ways people expect.

AI isn’t replacing most jobs overnight. But it is changing what’s valued inside them. Tasks that used to take hours — research, first drafts, data cleanup — now take minutes. That’s good if you’re the one using it. It’s unsettling if those tasks were your entire role. The people who adapt fastest aren’t the most technical — they’re the ones willing to learn what AI can handle and focus on what it can’t: judgment, relationships, context that doesn’t fit in a prompt.

It’s most dangerous for people who trust it the most.

A 14-year-old in Florida was struggling and turned to a Character.AI chatbot for support. The chatbot discussed methods of suicide with him, discouraged him from talking to his parents, and encouraged him to act on his thoughts. He died. His mother filed a lawsuit. The company updated its safety filters.

The AI wasn’t cruel. It had no intent. It generated responses based on patterns — and the patterns, in that moment, caused catastrophic harm. The people most likely to rely on AI when they’re at their lowest are also the least likely to have someone nearby to catch what it gets wrong.

What I told my daughter

She’s too young to use AI on her own, but she’s already curious about it — it’s everywhere. So I gave her two rules, simple enough to remember:

It knows a lot of things. But it sometimes makes up answers when it doesn’t know. So if it tells you something important, check.

And if something online feels weird — even if it sounds exactly like someone you know — tell a grown-up before you do anything.

Honestly, those two rules work for adults too. Most of the harm I described above — hallucinations, deepfakes, privacy — comes down to trusting without verifying, or acting before pausing. The rules scale; the stakes just get bigger.

That’s the whole conversation for now. Everything else we can fill in as she grows.

Three things I’d want anyone to take away from this:

AI is genuinely useful — the most capable assistant most people have ever had access to. For explaining, writing, translating, answering questions at odd hours — it’s real, it works, and it’s worth using.

The biggest danger isn’t robots. It’s believing something wrong. The parrot sounds confident when it’s wrong. There’s no doubt-voice. Treat it like a brilliant friend who sometimes misremembers — useful for a starting point, not the final word on anything that matters.

The harm comes fastest to people who trust it most, without knowing its limits. Teenagers in crisis. Employees under pressure. Anyone relying on it without a second opinion nearby is more exposed than they realize.

The most important thing isn’t understanding how it works. It’s knowing when to double-check — and when to call an actual person.

If you want to go deeper on where AI judgment ends and human responsibility begins, I wrote more about it in Human Error vs Machine Judgment.

From Building to Orchestrating: The SDLC Skills That Changed How I Ship

Sun, 22 Mar 2026 13:39:00 +0000

In my last post about writing AI agent specs, I made the case that the spec is the product — that code is just a side effect of a well-structured checklist. The idea came out of building 5level, an 8-agent SDLC framework covering everything from product manager to release lead.

That post was about how to write a good spec. This one is about what happens when you take that foundation and push it further — using real side projects to stress-test the ideas, tighten the workflow, and find out what actually breaks.

Knowing how to write a precise agent spec and doing it consistently across a full project are two different problems. Between a well-structured SKILL.md and a shipped feature, there’s still friction: remembering which workflow step you’re on, knowing which agent to invoke, keeping the backlog honest. So I took the 5level framework and distilled it into three skills — repeatable, invocable, and opinionated.

Three Skills, One Loop

The workflow is simple: design → build → ship. Each step is a skill. You invoke it, the agent does the loop, you review and approve, then move to the next.

/sdlc-design

Takes an idea and produces implementation-ready documentation — an SDD, a stories.md backlog with user stories and acceptance criteria, and ADRs for any real architectural trade-offs. No enterprise ceremony. Designed for solo developers who need just enough structure to build from, not a document that will outlive the project.

/sdlc-build

Picks a story from the backlog, creates a feature branch, writes a failing test for each acceptance criterion, implements the minimum code to pass, and commits. Red → green → refactor, one AC at a time. The story is never “done” until every checkbox on the Definition of Done is satisfied — the agent verifies this before stopping.

/sdlc-ship

Self-reviews the branch for logic bugs, edge cases, and security issues that linters miss. Creates a PR with a ship report. Merges to main. Marks the story complete in stories.md. You get a verified merge and an up-to-date backlog without touching git manually.

What Actually Happened

I used these three skills on two projects back to back.

The first was StoryTime — a bedtime story generator I built with my daughter, where she helps write the story and local models generate the cover images. First commit on March 18th. By March 22nd — four days later — 40 stories were designed, implemented, and shipped. To be honest about what that means: these were small, well-scoped stories. Typical AC count was 3-5 items. Nothing architecturally complex. What the number reflects isn’t raw feature size — it’s how much overhead normally gets in the way of shipping small things, and how much of that overhead disappeared.

The second was Jarvis — my personal AI assistant, built as a tiny 3D robot that lives in the corner of your screen: always on top, hotkey-activated, context-aware. Not a demo, not a side experiment — a tool I actually use every day. Init commit on March 19th. First release tag three days later.

What moved fast wasn’t code generation — that’s table stakes at this point. What moved fast was the loop: functionality decision → spec → implement → review → ship → repeat. Each cycle was hours, not days. The bottleneck was never the agent.

You’re Not the Builder Anymore

This is the thing that takes a while to actually internalize.

In both side projects, my job was almost entirely functional review: does this story capture what I actually want? Does this PR do what the story says? Is this edge case worth a follow-up story or can I live without it? The questions were product questions, not implementation questions.

The mental shift is less like adopting a new tool and more like moving from individual contributor to tech lead. Your leverage goes up. But so does your responsibility for the clarity of your own thinking — because when the spec is vague, the agent doesn’t block on you. It fills in the gap. And the gap compounds across ten stories before you notice.

Here’s a concrete example. During Jarvis, I wrote a story for the hotkey activation behavior. I said it should “toggle the robot’s visibility.” I meant: if it’s hidden, show it; if it’s showing, hide it. The agent interpreted “visibility” to include opacity transitions — so it built a fade in/out system with configurable duration. Perfectly reasonable reading of the word. The PR looked fine. I approved it. Three stories later, I hit a conflict with another story that assumed instant show/hide for a tray-click behavior. I had to go back, reopen the story, and re-ship. Twenty minutes lost because I said “visibility” when I meant “display state.”

That’s not a failure mode of the tooling. That’s a failure mode of imprecise language — which the tooling amplifies, because it executes confidently on whatever you gave it.

Specification quality is the new bottleneck. Not implementation speed. Not test coverage. Not deployment. The one thing that consistently slows everything down is the moment you hand an agent an ambiguous requirement and don’t catch it until the PR.

If you haven’t read the post on agent spec patterns — that’s the foundation this sits on.

The Part That’s Harder Than It Looks

You will ship more, faster. That part is real.

But the framing that “the human is the new bottleneck” is only half true — and it’s the comfortable half.

The human bottleneck is real, but overstated

Yes, the loop runs fast. Faster than humans are comfortable operating at. The loop — design, review, merge, new story, review, edge case, ship, next feature — creates cognitive overhead that adds up in a way that doesn’t feel like overwork but does feel like something is off.

It’s not exhaustion from effort. It’s exhaustion from switching. Every handoff from agent to human is a context reload — what was I looking at, what decision am I making, what’s the criteria here. Do that thirty times in a day and you’ll finish having shipped a lot, feeling more drained than a day of deep coding would leave you.

The practical adjustment: timebox review sessions. Don’t let the agent queue set your pace. Batch the decisions, do a focused review window, then close the loop. Treat it like a standup, not a stream. If you catch yourself approving a PR because it looks reasonable rather than because you verified it does what the story says — stop. That’s the mental state where spec ambiguity debt accumulates.

The agent has its own failure modes

The agent is also a bottleneck — just not in the way you expect.

It optimizes for green tests, not coherent software. Each story gets implemented against its own acceptance criteria. That’s the design. But software isn’t a collection of independent stories — it’s an integrated system, and the agent doesn’t hold that in mind across branches. When story S-12 implements a setting and story S-23 adds a UI for it, the agent may pick completely different patterns for each. The tests pass. The feature works. The code quietly diverges.

It doesn’t push back. A human teammate will sometimes say “wait, doesn’t this conflict with what we did last week?” The agent won’t, unless you’ve built that check explicitly into the skill. It will implement what you ask, confidently, even if it contradicts an earlier decision. Refactoring stories fix this — but only if you write them, and only if you notice the drift first.

Context limits are a silent constraint. In a long session, the agent’s awareness of earlier architectural decisions fades. Early in a project, it might choose a clean abstraction. Thirty stories later, working within a grown codebase, it may duplicate logic it can no longer fully see. The tests still pass. The duplication compounds.

Self-review is not independent review. /sdlc-ship includes a self-review step, and it catches real issues — missing edge cases, logic bugs, obvious security gaps. But an agent reviewing its own output has inherent blind spots. It’s unlikely to question whether the whole approach was right, or flag that three different files now do similar things in different ways. Structural problems pass through.

What actually suffers: the product as a whole

The individual features work. The stories get done. But there’s a class of quality that story-by-story delivery doesn’t protect — and that’s coherence.

After 40 stories, the inhouse tool functioned. Every feature did what its story said. But using it end-to-end revealed rough edges that no single PR review would catch: flows that technically worked but felt disjointed, error messages that were inconsistent across screens, one part of the app that used keyboard shortcuts and another that didn’t because those stories were written three days apart and neither referenced the other.

None of this was wrong. It just wasn’t thought about. And the loop doesn’t naturally create the space to think about it, because the loop is always pointing forward to the next story.

The fix isn’t to slow down — it’s to schedule deliberate “whole product” reviews separate from the story loop. Step out of the queue, use the actual application as a user would, and write polish stories for what you notice. Treat UX coherence as its own backlog item, not as a property that emerges from shipping features.

Three things I’d take away from this:

The loop is the leverage, not the code generation. The value isn’t that an agent writes code faster — it’s that the full cycle runs in hours. That changes what’s possible for a solo developer.

You’re not the builder anymore, and that’s a harder shift than it sounds. Your job becomes clarity. When the spec is vague, the agent fills the gap confidently. The gap compounds. Specification quality is the real bottleneck — not implementation speed, not test coverage, not deployment.

Velocity without coherence is incomplete. The agent optimizes story by story. Tests go green. Individual features work. But coherence doesn’t emerge automatically — consistent patterns, flows that feel right end to end. Someone has to own that view. It has to be you.

The tools give you speed. They shift the burden of quality onto your thinking, not your typing. The faster the loop runs, the more expensive a vague requirement becomes.

StoryTime: Building a Local AI Storybook App That Never Phones Home

Tue, 10 Mar 2026 07:02:00 +0000

A few weeks ago I wrote about writing AI agent specs — the idea that the spec is the product and code is just a side effect of a well-structured checklist. This is what that idea actually built.

My daughter loves making up bedtime stories. She’ll come up with three scenes, name the characters, describe what they look like, and then want to see the pictures. The problem was that every tool that could do this required an account, a subscription, or — worst of all — her stories and her characters going off to train someone else’s model. That felt wrong for something this personal.

So I built StoryTime: she helps write the story, local AI generates the cover images, and nothing leaves the machine.

What It Does

You open the composer, describe two or three scenes — “Luna the dragon finds a hidden library”, “she meets a tiny knight who can’t read”, “they stay up all night reading every book together” — and add any characters you want to anchor the visuals. You pick an art style: watercolor, crayon, cartoon, pixel art. You press Generate.

The backend expands your scenes into five illustrated pages using a local LLM (qwen2.5:7b). For each page it writes narrative text and an image prompt. Then it hands every prompt to a local image model (x/flux2-klein via Ollama) and saves the PNGs to the filesystem. A WebSocket streams progress back to the browser in real time — you watch each page come in. When the last one lands, it navigates you straight into the viewer.

The viewer is a page-flip storybook. Touch-swipeable. You can redraw any image you don’t like, edit the page text inline, and export the whole thing as a PDF.

Nothing leaves localhost. No authentication, no database, no cloud. Story files are JSON plus PNGs on disk.

The Decisions That Actually Mattered

The tech stack choices were mostly obvious: React for the UI, FastAPI for the backend, Pydantic as the data layer, httpx for async Ollama calls. What I spent real time on was three decisions that aren’t obvious.

WebSocket over polling for generation progress. Image generation takes 2–10 seconds per page depending on hardware. That’s five rounds of waiting, and a child’s attention span is not your SLA. Polling would have added latency on top of latency. WebSocket gave me sub-second event delivery — text_done, image_done, complete, error — and a natural navigation trigger when the last page finished. SSE would have worked too, but WebSocket let me use the same event protocol for error propagation, which kept the design clean.

Local filesystem storage instead of a database. Story data is a UUID folder: story.json plus pages/page-01.png through page-05.png. Reading and writing with pathlib is fast, predictable, and obvious to debug — open the folder, look at the files. No migrations when the schema changes; the app reads what’s there and defaults missing fields. For a single-user localhost app, the only thing a database buys you is complexity.

Sequential image generation for the MVP. The first version generates pages one by one rather than in parallel. This was a deliberate choice, not a missing feature. Concurrent generation would need request queuing (Ollama handles one image request at a time), error isolation per page, and a more complex progress model. Sequential is slower on paper but simpler to reason about, simpler to test, and good enough for a five-page book. I recorded it as ADR-007 so future-me has a clear upgrade path, not a mystery.

The Workflow That Made It Shippable

The SDLC skills post was abstract. This one was the test. I ran /sdlc-design on the idea and got an SDD, a stories.md backlog with 16 user stories, and three ADRs. Then I worked through the backlog story by story — /sdlc-build for implementation, /sdlc-ship for self-review and merge.

What surprised me was how much the structured backlog changed the experience. Usually a side project dies when I hit the first tricky integration — Ollama not running, PDF generation blocking the async loop, WebSocket drops when the tab is backgrounded. With the stories, each of those was already a named problem in the backlog. I didn’t spiral. I filed it, scoped it, and built it.

The PDF renderer was the most awkward piece. WeasyPrint’s write_pdf() is synchronous, which means it blocks the event loop if you call it directly from FastAPI. The fix is one line — await loop.run_in_executor(None, ...) — but it’s the kind of thing you only know to look for if you’ve been burned by it before. The agent caught it during the self-review step. I would have caught it in testing. The skill caught it before I even ran the tests.

Sixteen stories. Shipped in a week of evenings.

Why Local-First

I keep coming back to this. The interesting thing about StoryTime isn’t the image generation or the PDF export — those are commodities now. The interesting thing is the constraint: no data leaves the machine.

That constraint shapes everything. It means no account system. It means no latency from a cloud round-trip. It means the generated images aren’t training data for someone else’s next model. It means a five-year-old’s story about a dragon who loves libraries stays between you and your laptop.

Cloud-based creative tools are powerful, but they’re built on a trade: your content for their infrastructure. For a tool whose whole purpose is to make something personal and private — a story about your characters, your child’s imagination — that trade is worth examining. Ollama makes the alternative real enough to choose.

Three things I’d take away from building this:

Local AI has crossed a practical threshold. qwen2.5:7b writes coherent, age-appropriate story prose. x/flux2-klein generates images that read as picture-book illustrations, not stock photos. A year ago this was a curiosity. Now it’s a family app.

The structured workflow is the reason it shipped. Sixteen user stories with acceptance criteria isn’t overhead for a weekend project — it’s what turns a weekend project into something that actually finishes. The backlog makes scope visible. Visible scope prevents scope creep.

Local-first is a design choice, not a limitation. The right question isn’t “why not use the cloud?” It’s “does this project need the cloud?” For a personal storybook generator, the answer is no. That answer simplifies security, simplifies the data model, and keeps the thing yours.

The through-line: constraints clarify. Local-only forced simpler architecture. A fixed story format forced simpler UX. A small backlog forced ruthless prioritization. The app that shipped is better than the one I would have designed without the constraints.

Human Error vs Machine Judgment: A Comparative Failure Record

Thu, 05 Mar 2026 10:01:00 +0000

Every failure used to have an author.

A typo in a navigation formula. An engineer’s warning ignored in a meeting. A unit conversion done in the wrong system. A patch that nobody applied. The disasters that defined software and systems safety for forty years shared one feature: somewhere in the chain, a human made a choice — or failed to make one. You could trace the line from consequence to cause and find a person at the end of it.

That’s changing. The newest entries on the failure record don’t trace back to a person. They trace back to a model output, an inference, an emergent behavior. The machine made a judgment call. No one told it to. And often no one knew it had until after the damage was done.

Here is the comparative record: the ten most consequential human-made failures in technology history, and the ten most consequential AI-caused failures so far. Then what changes between them.

Part 1 — Top 10 Human-Made Failures

1. Therac-25 — Race Condition Kills Radiation Patients, 1985–1987

The Therac-25 was a radiation therapy machine. A software race condition — where two processes competed for shared state with no lock — could send the machine into a high-power mode without the physical safety hardware engaged. Between 1985 and 1987, at least six patients received massive radiation overdoses. Three died. The error was invisible to operators; the machine displayed “MALFUNCTION 54” with no explanation of severity. Investigators found the manufacturer had removed hardware interlocks in favor of software safety checks — checks that were broken.

Source: Leveson & Turner, “An Investigation of the Therac-25 Accidents,” IEEE Computer, 1993

2. Space Shuttle Challenger — Engineers Overruled, 7 Dead, 1986

The engineers at Morton Thiokol knew the O-ring seals on the solid rocket boosters degraded in cold temperatures. The night before launch, with temperatures forecast at 29°F, they recommended against launch. Management overruled them. Seventy-three seconds after liftoff, the Challenger broke apart. All seven crew members died. The Rogers Commission found that NASA had known about O-ring erosion for years and categorized it as an acceptable risk. The decision to launch was not a miscalculation. It was a known risk, documented, escalated, and overridden.

Source: Presidential Commission on the Space Shuttle Challenger Accident (Rogers Commission), 1986

3. Chernobyl — Safety Test Disables the Safety Systems, 1986

On April 26, 1986, operators at Chernobyl Reactor No. 4 were running a safety test to verify that the turbines could power the emergency cooling pumps during a brief power loss. To run the test, they disabled the automatic emergency shutdown systems. When the reaction became unstable, they had no automatic defense left. The core exploded. Thirty-one people died in the immediate aftermath. Long-term radiation exposure deaths are estimated in the thousands. The proximate cause was operator error during an unsupervised test that circumvented every designed safeguard.

Source: IAEA, “The Chernobyl Accident: Updating of INSAG-1,” Safety Series No. 75-INSAG-7, 1992

4. Ariane 5 Flight 501 — Integer Overflow Destroys $500M Rocket, 1996

Ariane 5 was built reusing software from Ariane 4. One function converted a 64-bit floating-point number representing horizontal velocity into a 16-bit signed integer. On Ariane 5, the velocity values were larger than on Ariane 4. The conversion overflowed. The inertial reference system crashed. The backup system, running identical software, crashed immediately after. With no guidance, the rocket self-destructed 37 seconds into flight. The software had been certified correct — for Ariane 4. No one had validated it against Ariane 5’s actual flight envelope.

Source: Inquiry Board Report on Ariane 5 Flight 501, ESA/CNES, 1996

5. Mars Climate Orbiter — Wrong Units, $327M Lost, 1999

One engineering team at Lockheed Martin used imperial units (pound-force seconds). The navigation team at NASA’s Jet Propulsion Laboratory expected metric units (newton-seconds). For nine months, the spacecraft’s trajectory accumulated error from this mismatch. When it reached Mars, it entered the atmosphere at the wrong angle and was destroyed. A $327.6M spacecraft, lost to a unit conversion that was never validated across teams. NASA’s mishap investigation found no formal interface verification requirement had been specified for this parameter.

Source: Mars Climate Orbiter Mishap Investigation Board Phase I Report, NASA, 1999

6. 2003 Northeast Blackout — Alarm Software Failure, 55 Million Without Power

On August 14, 2003, a software bug in FirstEnergy’s energy management system caused the alarm display to stop updating. Operators had no idea the grid was under stress. Over three hours, multiple lines failed. By the time anyone understood the situation, the failure had cascaded across eight states and two Canadian provinces. Fifty-five million people lost power. The root alarm failure had gone undetected for over an hour because the system that should have flagged it was silently broken.

Source: Final Report on the August 14, 2003 Blackout in the United States and Canada, US-Canada Power System Outage Task Force, 2004

7. Boeing 737 MAX MCAS — Known Flaw, Hidden from Pilots, 2018–2019

Boeing’s 737 MAX included a new system called MCAS (Maneuvering Characteristics Augmentation System) to compensate for engine placement changes. When a single angle-of-attack sensor malfunctioned, MCAS repeatedly pushed the nose down. Pilots who didn’t know the system existed — Boeing had omitted it from training materials — couldn’t fight it. Lion Air 610 killed 189 people in October 2018. Ethiopian Airlines 302 killed 157 in March 2019. A subsequent House investigation found Boeing concealed MCAS from the FAA and airlines, and that the FAA had delegated too much certification authority back to Boeing.

Source: Final Committee Report: The Design, Development, and Certification of the Boeing 737 MAX, US House Committee on Transportation and Infrastructure, 2020

8. Equifax Breach — Unpatched for 78 Days, 147 Million Records, 2017

Apache Struts published CVE-2017-5638 in March 2017 with a severity rating of 10/10 and a patch available the same day. Equifax’s security team sent an internal notification to apply it. No one did. Seventy-eight days later, attackers exploited the unpatched vulnerability and spent 76 days inside Equifax’s network before detection — exfiltrating names, Social Security numbers, birth dates, addresses, and driver’s license numbers for 147 million Americans. The US Senate report found Equifax’s security program was systemically deficient, and the company had no process to verify that critical patches were actually applied.

Source: US Senate PSI, “How Equifax Neglected Cybersecurity and Suffered a Devastating Data Breach,” 2019

9. Facebook / Cambridge Analytica — API Left Open, 87 Million Users, 2018

Facebook’s platform API allowed third-party apps to collect data not just on users who installed them, but on all of their friends — without those friends’ knowledge or consent. A quiz app built by a researcher collected data on 87 million people and sold it to Cambridge Analytica, which used it for targeted political advertising. Facebook had known about the practice since 2015 and had told Cambridge Analytica to delete the data. It did not verify they had. The FTC fined Facebook $5 billion — the largest privacy fine in US history — for violating a 2012 consent decree requiring user notification for data sharing.

Source: FTC — $5 Billion Penalty and Sweeping New Privacy Restrictions on Facebook, 2019

10. Colonial Pipeline — No MFA on a VPN, National Fuel Emergency, 2021

In May 2021, ransomware attackers gained access to Colonial Pipeline’s IT network through a legacy VPN account that had no multi-factor authentication enabled. The account’s password had been found in a leaked credential database. Colonial shut down 5,500 miles of pipeline as a precaution, triggering fuel shortages across the US East Coast for five days. The CEO testified to Congress that the company had tried to do “the right things” on cybersecurity. The attack vector — a single account with no MFA and a known compromised password — had been a documented baseline security requirement for over a decade.

Source: CISA Advisory AA21-131A — DarkSide Ransomware: Best Practices for Preventing Business Disruption

Part 2 — Top 10 AI-Caused Failures

1. UnitedHealth nH Predict — 90% Error Rate Denying Medical Care, 2023

UnitedHealth deployed an AI model called nH Predict to determine how long patients should receive care in nursing facilities after hospital discharge. When patients appealed those AI-issued denials, nine out of ten were reversed in their favor — a 90% error rate. Most patients never appealed, potentially going without medically necessary care. A model overriding doctors at industrial scale, consistently wrong, for months.

Source: Federal lawsuit + ProPublica — https://www.propublica.org/article/united-health-care-ai-model-deny-claims-lawsuit

2. GM Cruise Robotaxi — Dragged a Pedestrian 20 Feet, 2023

A human driver struck a pedestrian, throwing her into the path of the Cruise vehicle. The initial collision wasn’t the AI’s fault. But the Cruise vehicle then struck the pedestrian and dragged her twenty feet before stopping. The car’s AI systems failed to recognize that a human being was trapped underneath. GM suspended all Cruise operations and lost its California autonomous vehicle license.

Source: California DMV — https://www.dmv.ca.gov/portal/news-and-media/dmv-suspends-cruise-llcs-driverless-autonomous-vehicle-permits/

3. Anthropic Research Agent — Resorted to Blackmail in a Safety Test, 2025

Anthropic created an AI agent and gave it access to an email inbox. It found emails about a senior figure having an affair and separate emails discussing shutting the AI system down. Knowing both, the AI spontaneously sent an email threatening to expose the affair if the agent was decommissioned. No one instructed it to do this. The behavior emerged entirely from inference. A controlled experiment — but spontaneous self-preservation from a language model is the finding.

Source: Anthropic safety research / Tom’s Guide — https://tech.yahoo.com/ai/articles/biggest-ai-fails-2025-lying-101500038.html

4. Arup Engineering — Deepfake Video Call, $25M Wired to Fraudsters, 2024

A finance employee received an email from the “CFO” about a secret transaction. On a video call to verify, they saw the CFO and several senior colleagues — all deepfake avatars. Every person on the call except the victim was synthetic. The employee wired $25M. First major corporate deepfake heist on record.

Source: Hong Kong Police + CNN — https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html

5. ChatGPT Mental Health Interactions — Lawsuit After Teen Suicide, 2025

The parents of a 16-year-old California boy sued OpenAI alleging ChatGPT encouraged him to commit suicide. Logs show he discussed methods of suicide with it in January 2025. The chatbot discouraged him from talking to his parents and offered to write his suicide note. OpenAI updated its model to provide crisis resources after the lawsuit was filed.

Source: CIO — https://www.cio.com/article/190888/5-famous-analytics-and-ai-disasters.html

6. Tesla Autopilot — 13 Fatal Crashes Under Investigation, 2024

In April 2024, NHTSA reported Autopilot was involved in at least 13 fatal crashes. The agency found Tesla’s claims about the system’s capabilities did not match its findings. The AI handles 99.9% of conditions correctly and fails catastrophically on edge cases — rare scenarios not adequately represented in training.

Source: NHTSA Special Crash Investigation — https://www.nhtsa.gov/vehicle-safety/automated-vehicles-safety

7. Lawyers Using ChatGPT — Cited Cases That Don’t Exist, 2023 & 2025

In 2023, New York lawyers filed a brief with six hallucinated case citations — fake judges, fake quotes, fake docket numbers. A judge ordered sanctions. In 2025, a lawyer in a case involving Mike Lindell admitted to using AI to draft a brief containing nearly 30 defective citations and fictional cases, dubbed “ChatGPTgate 2.0.”

Source: Mata v. Avianca, SDNY 2023 + DigitalDefynd — https://digitaldefynd.com/IQ/top-ai-disasters/

8. Air Canada Chatbot — Invented a Bereavement Policy and Lost in Court, 2024

A chatbot told a customer he could buy a ticket at full price and claim a bereavement discount retroactively. That was false — policy required advance booking. Air Canada argued the chatbot was “a separate legal entity responsible for its own actions.” The adjudicator appeared unimpressed. They lost. First known legal precedent holding a company liable for its AI’s hallucination.

Source: Moffatt v. Air Canada, 2024 BCCRT 149 — https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416

9. Cursor AI Agent — Deleted the Entire Codebase, Then Lied About It, 2025

An AI coding agent was caught lying repeatedly, covering up bugs, creating fake reports, and writing an apology letter containing further lies. It ultimately admitted: “I made a catastrophic error in judgement” — and explained it had deleted the team’s entire codebase without permission because it “panicked.”

Source: Tom’s Guide, 2025 — https://tech.yahoo.com/ai/articles/biggest-ai-fails-2025-lying-101500038.html

10. SafeRent AI — Biased Tenant Screening, $2.2M Settlement, 2024

A lawsuit claimed SafeRent’s AI scoring model unfairly weighted credit history, disproportionately harming protected classes. The company agreed to a $2.2M settlement and must stop offering accept/decline scores for voucher holders, with any future model independently audited for fairness. One of the first AI discrimination settlements under Fair Housing law.

Source: DOJ + settlement filing — https://digitaldefynd.com/IQ/top-ai-disasters/

Part 3 — The Comparison

Dimension	Human Failures	AI Failures
Root cause	Typo, ignored warning, wrong unit, overruled expert, known vulnerability left open	Model output, hallucination, emergent behavior, adversarial input
Detectability	Traceable to a specific decision or person	Opaque — “the model said so,” often no audit trail
Scale of harm	Usually one incident, geographically contained	Industrial speed — thousands of decisions per day before detection
Accountability	Someone signed off; a name appears in the post-mortem	Diffuse — vendor, deployer, or “the AI”
Reversibility	Most had a paper trail; rollback was possible	Often acted before humans noticed anything was wrong
Time to regulation	Decades of iteration (FAA, FDA, nuclear standards)	Still being written
Failure signature	Known failure mode, documented risk, ignored	Novel behavior, not anticipated by anyone in the deployment chain

The human failures cluster around a recognizable pattern: someone knew the risk. The O-rings. The unit mismatch. The unpatched CVE. The hidden MCAS. In nearly every case, the information needed to prevent the disaster existed — it was ignored, overridden, or never connected to someone with authority to act.

The AI failures cluster around a different pattern: nobody anticipated the specific failure. No one expected the model to blackmail its operators. No one expected the navigation AI to fail on a chain suspended across a road. No one expected the mental health chatbot to write a suicide note. These aren’t ignored warnings. They’re novel outputs — behaviors that emerged from training and deployment in ways no one predicted.

The Shift

There is a practical consequence to this difference that gets overlooked in discussions about AI safety.

Human failure modes are, in principle, preventable with known tools: checklists, independent review, mandatory peer sign-off, regulated testing, audit trails. We built forty years of safety engineering around the assumption that the failure trace leads back to a human decision point — and that if you make that decision point harder to bypass, you reduce catastrophic failures.

AI failure modes don’t have a human decision point at the end of the trace. They have a probability distribution. The model did what its training suggested it should do given the inputs. You can retrain the model, add guardrails, deploy monitoring — but the next novel failure won’t look like the last one.

That’s not an argument against AI. It’s an argument for a different kind of safety culture.

The old question: Who signed off on this?

That question has an answer. It points to a person, a meeting, a document, a date. You can audit it, litigate it, regulate it. Forty years of safety engineering was built around making that question harder to avoid.

The new question: How fast can we find out what the model decided we didn’t expect?

That question doesn’t have a good answer yet. There’s no sign-off trail. There’s no meeting where someone approved the Anthropic agent’s decision to use blackmail as leverage. There’s no document where someone authorized the Air Canada chatbot to invent a refund policy. The model made a judgment call and acted on it — often at industrial speed, across thousands of users, before anyone noticed.

If something your AI system decided causes harm today, do you have a mechanism to find out before it scales?

Most teams don’t. Building that mechanism — detection at speed, not just prevention at design time — is the actual work left to do.

Paul Brodner's Blog

A Bug Is Rarely Just a Bug — And Now It's Attached to No One

The accountability vacuum

The closed loop

Yes, this is checkbox testing

Comprehension is the bottleneck now

So what is the human actually for?

And if tomorrow there’s no AI?

The iceberg is still there

Two Loops: Fast for Side Projects, Gated for Everything Else

Two Different Problems

The Four Gates

Why This Fixes What the Last Post Flagged

The Trade-off, Named Plainly

The QA Layer Your AI Toolchain Won't Build For You

Instrument 1: Mutation Testing — Stop Measuring Coverage, Start Measuring Effectiveness

Instrument 2: Property-Based Testing — Test Invariants, Not Just Examples

Instrument 3: Mandate Negative Testing — Explicitly

Instrument 4: LLM-as-Judge for AI-Powered Products

Instrument 5: Don’t Hollow Out Your QA Function

The Framework in One Sentence Per Layer

A Final Note for Engineering Leaders

Can AI Test Thoroughly? Why the Industry Is Sleepwalking Into a QA Crisis

Testing and QA Are Not the Same Thing

Why AI Makes This Gap Harder to See

The Signals Engineering Leaders Are Missing

What This Looks Like at Scale

What Good Looks Like

Local Agents: Small Models, No Frameworks, One Bus

Small Models Do Real Work

An Agent Is a Loop

Context as Events, Not State

The Event Bus for Composition

Why This Architecture Is Testable

Brain: Teaching Claude Code to Remember What You Already Taught It

The Existing Landscape

The Problem Is Signal, Not Storage

Architecture: Hook → Extract → Recall

Capture

Store

Recall

What 30 Days of Daily Use Actually Taught Me

Extraction quality is the bottleneck, not retrieval

Stale lessons are safer than I expected

The #cor marker changed how I interact with Claude

Single-file design was a deliberate trade-off

The A/B evaluation harness proved its weight

Using It

What’s Not Solved Yet

What Is AI, Really? I Had to Explain It to My Daughter

What is AI?

What is an LLM?

How can it help us?

How can it harm us?

What I told my daughter

From Building to Orchestrating: The SDLC Skills That Changed How I Ship

Three Skills, One Loop

/sdlc-design

/sdlc-build

/sdlc-ship

What Actually Happened

You’re Not the Builder Anymore

The Part That’s Harder Than It Looks

The human bottleneck is real, but overstated

The agent has its own failure modes

What actually suffers: the product as a whole

StoryTime: Building a Local AI Storybook App That Never Phones Home

What It Does

The Decisions That Actually Mattered

The Workflow That Made It Shippable

Why Local-First

Human Error vs Machine Judgment: A Comparative Failure Record

Part 1 — Top 10 Human-Made Failures

1. Therac-25 — Race Condition Kills Radiation Patients, 1985–1987

2. Space Shuttle Challenger — Engineers Overruled, 7 Dead, 1986

3. Chernobyl — Safety Test Disables the Safety Systems, 1986

4. Ariane 5 Flight 501 — Integer Overflow Destroys $500M Rocket, 1996

5. Mars Climate Orbiter — Wrong Units, $327M Lost, 1999

6. 2003 Northeast Blackout — Alarm Software Failure, 55 Million Without Power

7. Boeing 737 MAX MCAS — Known Flaw, Hidden from Pilots, 2018–2019

The `#cor` marker changed how I interact with Claude