The tech who only watched the green light is the person automation creates: great throughput while the fixture works, stuck the day a pogo pin wears out, because they never learned what normal looks like from the inside. Lisanne Bainbridge named that irony in 1983. Coding agents are rerunning it on anyone who accepts the diff without thinking.
Every test floor I ever worked on had one. The tech who loads the board, hits start, watches the light go green, and loads the next board. Great throughput. Never a complaint.
Then the fixture fails. A pogo pin wears out, or a relay starts sticking, and the light goes red on a board that’s fine. That tech is stuck. They’ve never had to read the test log, never probed the fixture, never figured out what normal looks like from the inside, because the machine always told them. The person who can fix it is the one who spent a year doing the boring boards by hand.
I keep seeing that tech in how people use coding agents. Sometimes I see them in the mirror.
What is cognitive surrender with coding agents?
knlb’s essay Resisting Cognitive Surrender is the version of this I keep sending people. He defines cognitive surrender as “uncritically accepting whatever agents output without actually thinking, exploring or considering alternatives.” Working that way is fast, not particularly taxing, and easy to slide into.
He is not anti-agent. He is against taking the easy path without noticing. He writes that he finds himself making suboptimal decisions, accruing technical debt, and reducing his familiarity with the system instead of improving it. The agents’ expressed confidence makes that easier. So does peer pressure: if everyone else is shipping more PRs, spending time to understand the change looks like you are slow.
The part that landed for me is about debugging. When he built things by hand he could diagnose a bug from familiarity. With heavy agent use, his only move becomes spinning up one more session and hoping, and the automated fix makes his understanding a little worse each time. That’s the green light tech, in a terminal.
He also names a Gell-Mann amnesia effect. When he has expertise, he can simplify the output. When he doesn’t, the agent looks like the smartest engineer in the room, until it gets stuck creating a directory and the assumption resets. Accountability without ownership is the job that leaves you with.
He quotes Andy Grove: “delegation without follow-through is abdication.” Put that on the wall next to the tester.
What did Bainbridge’s Ironies of Automation actually say?
Lisanne Bainbridge published “Ironies of Automation” in Automatica, volume 19, number 6, in 1983. It’s a short brief paper. knlb tells you to read it directly. Do that.
The core irony is in the designer’s attitude. The designer who tries to eliminate the operator still leaves the operator to do the tasks the designer cannot think how to automate. Those leftover tasks are the hard, rare, weird ones. Because the machine handles everything routine, the human never gets the practice that would have made them good at those hard ones. You lose the skill exactly when you need it most.
Bainbridge splits what is left into two jobs: monitor that the automatics are working, and take over if they are not. Takeover needs manual control skill and cognitive skill. Physical skills deteriorate when they are not used, particularly the refinements of gain and timing. A formerly experienced operator who has been watching the plant may now be an inexperienced one. If they take over they may set the process into oscillation.
Cognitive skill has the same trap. Efficient retrieval of long-term knowledge depends on frequency of use. That knowledge develops through use and feedback, not through a classroom lecture with no practice. Working storage, the feel for what the process is doing right now, takes time to build. An operator asked to take over quickly from automation has only minimum information.
Monitoring looks simpler. It isn’t. Bainbridge cites Mackworth’s vigilance studies: even a highly motivated person cannot maintain effective visual attention toward a source of information on which very little happens, for more than about half an hour. A classic method of forcing attention is to make the operator keep a log. People can write down numbers without noticing what they are.
The sharper irony: the automatic system was put in because it can do the job better than the operator, yet the operator is asked to monitor that it is working. If the computer is making decisions the human could not make in real time, the human cannot check those decisions in real time either. Bainbridge’s line is that the human monitor has been given an impossible task.
She also writes the sentence that maps onto agents without stretching: by taking away the easy parts of the task, automation can make the difficult parts more difficult. And the final irony: it is the most successful automated systems, with rare need for manual intervention, which may need the greatest investment in human operator training.
Her practical suggestions are not “ban the machine.” Hands-on control for a short period each shift, or simulator practice if that suggestion is laughable. Training for general strategies, not only procedures. Displays that fail obviously instead of camouflaging a drift.
None of that is about large language models. All of it is about a person who is still responsible after the easy work went away.
How does knlb apply that to agent work?
His fixes are practical, which is why I like them. They map onto Bainbridge without forcing it.
- Pick a vibe limit. Decide how complex a change you’re willing to make with an agent at your current level of confidence, and set it by risk. Sandboxed toy project, go wild. Fragile production system that’s hard to sanity check, you drive. Recalibrate when the model drops.
- Watch for “yes, continue.” If you’re repeatedly tapping it, either delegate the whole problem properly (better sandbox, better feedback loop) or admit you stopped thinking and go check the decisions.
- Force yourself into the loop on work you care about. He writes this post with a Codex session to tighten sentences, but he makes the edits by hand. In his notebook’s AGENTS.md: “Never edit files directly, always wait for the author to request or confirm changes, or let him make the edits directly.” That’s a person deliberately keeping their hands on the boards.
- Force comprehensibility. Custom review views. Stacks of small diffs. “Make the change easy, then make the easy change.” A debugger until you see the real path, not the agent’s story of the path.
- Keep your own handle on the metrics. Black box and white box observability, triangulated, and you read it yourself instead of only reading the agent’s account of what it did. Watch resource use too. Unexpected CPU is a canary.
- Deterministic validation. Held-out code and tools the model can’t argue with.
- Shadow traffic. Run the candidate against production-like traffic with no side effects before it ships.
When he does decide to delegate, the point is to flatten the vibe-limit curve: make bigger changes safer, not to stop looking. Style guides from his own code. Fast deploys so a miss can come back out. Build to throw on a prototype, then evaluate, then repeat. Good design is still the multiplier. He quotes Christopher Alexander: no complex adaptive system succeeds in a reasonable time unless adaptation can proceed subsystem by subsystem.
That last point is the fixture lesson in another dialect. A loosely coupled system is one you can still understand when the agent is wrong.
What habits keep you from becoming the green light tech?
The fixture lesson is that the green light is a claim, not a measurement. The tester is saying “I checked.” Whether it checked the right thing is a separate question, and you can only answer it if you know what the right thing looks like. I wrote about the version of this where every call passed and nothing got done. Same failure, different floor: a loop that looks healthy because every step returned OK.
A related miss is trusting a single clean summary. One clean summary is an anecdote, not a measurement. The house agent writes a perfectly nice summary of its night. I still read the raw log. If you only read the agent’s account, you are the person writing numbers in the log without noticing them.
Your tests are lying to your agent is the other side of the same gauge problem. An agent that believes every red test will chase ghosts. An operator who believes every green light will ship a bad board. Calibrate the instrument, then keep a person who can tell when the instrument is lying.
So a few habits:
- I do some of the boring changes by hand on systems I own. Not because it’s faster. It isn’t. Because it’s the practice that keeps me able to debug the day the agent can’t. Bainbridge’s “short period of hands-on each shift” is the same move.
- I read the logs myself on anything that matters. The house agent still rereads its rulebook every wake. I still read what it did, not only what it said it did.
- When I notice I’ve approved five steps in a row without reading them, I stop. That’s the moment the light went green and I stopped looking at the board. knlb’s “yes, continue” warning, with a step count.
- I keep a check the model cannot argue with. A test, a held-out script, a fixture measurement. If the only evidence is the agent’s narrative, you don’t have evidence.
A traveler I’d actually use:
- What is the vibe limit for this repo, written down, by risk?
- What is the no-progress signal that means I stopped thinking?
- Which logs do I read myself, not the summary?
- What deterministic check can fail this change even if the agent is confident?
- Did I do any of the boring work by hand this week on a system I own?
None of this is anti-agent. I run a lot of them. It’s the same deal the test floor always had: automate the routine, and keep somebody on staff who still knows how the fixture works. Make sure that somebody is you.