Test the Whole Truth Table

Nobody signs off a logic board after checking one row of the truth table. A coding agent that checks the one account it was logged in as is doing exactly that. Render every state before you merge.

Nobody signs off a logic board after checking one row of the truth table.

Four inputs, two values each, sixteen rows. You walk every one. If the board passes fifteen and fails one, it fails. That’s not perfectionism. That’s what a truth table is for.

In my sketch of this post, the truth table is printed in ink, except one cell, which is filled in with pencil.

Coding agents are skipping rows.

One account, one row

Matt Rothenberg wrote Pure UI in the agentic era, and his example is worth walking through. He says up front that it’s contrived. He also says he has seen versions of it in nearly every React codebase he has worked in.

He asks an agent to add a delete action to each row of a functions table. He forgets to say that people with the viewer role shouldn’t see it. The agent edits two files, runs the typecheck, which passes, opens the page in a browser, and reports back: done, and it renders correctly.

It did render correctly. For the owner account the dev server happened to be signed in as.

Same page as a viewer: the header says “View only,” and every row still has a Delete button. The header knew the role. The table never had a role check, and the prompt only asked the agent to change the table.

Sixteen rows

His dashboard takes four inputs: role, plan, billing status and a beta flag for live logs. Two values each. That’s sixteen screens.

He renders all sixteen in a grid with the agent’s version of the page. The agent checked one of them. Eight of them are viewer screens, and all eight show Delete to people who can only view.

That’s the truth table. The agent tested row one and called the board good.

His line on why this is worse with agents is the one I’d put on the wall: an agent can report “done” with the same confidence whether it checked one state or sixteen, while each generated change gets less review time.

Make the inputs explicit

The fix starts with an old idea. Guillermo Rauch wrote Pure UI in 2015, and the core of it is that a UI should be a pure function of application state. Same inputs, same screen. His test for done was blunt:

If you think up and hand out 20 distinct states in which your application can behave, then you should be able to clearly see them rendered on a screen as HTML. If only 10 are done, then the implementation is only 50% done.

He also called out the trap: it’s very easy to underestimate how many states there are.

The problem in a real app is that the state is scattered. In Matt’s example the role comes from a session hook, the plan from a context at the app root, billing from a layout wrapper four layers up, and the flag from a global client. His diagram of the page counts seven declared props and 15 reads it doesn’t declare. Nobody holds that in their head, person or agent.

So the refactor is to move the state that changes the screen to the page boundary and pass it in. Once the inputs are explicit, the input list is the truth table. You can turn rows into stories, tests or a grid, and for a screen like this one, Matt points out that rendering sixteen states is cheaper than setting up sixteen accounts.

None of this replaces Storybook or Playwright. Matt is clear that they work alongside it. And Pure UI doesn’t decide which rows matter. It gives you a clean place to write that decision down.

What I’d ask before merging

This is the same habit as calibrating the tests your agent trusts and the same split as humans approve intent, machines prove conformance. The agent can grind through the rows. A person still writes the table.

Before I merge an agent’s UI change, I want answers to four questions:

  1. What are the inputs? Role, plan, billing, flags, region, whatever changes the screen.
  2. How many rows is that? Multiply it out. If nobody knows, that’s the first finding.
  3. Which rows did the agent actually see? One logged-in account is one row.
  4. Which rows does this change touch? Those get rendered, every time.

It’s the same reason a tuning loop needs a golden unit. You need a known reference to compare against, and you need to know what you didn’t measure.

A green typecheck and one screenshot is row one. Walk the table.

Sources

Frequently asked questions

What does ui = fn(state) mean?

It is the Pure UI idea from Guillermo Rauch’s 2015 essay: a screen should be a pure function of application state. Give it the same inputs and you get the same screen. That makes every screen a row you can render and check on purpose.

Why do coding agents miss UI bugs in other user roles?

They usually check the state they started in. In Matt Rothenberg’s example, the agent added a Delete action, ran the typecheck, opened the page as the owner account the dev server was signed in as, and reported done. The same page as a viewer showed Delete on every row.

How do you check every state a screen can be in?

Move the state that changes the screen to explicit inputs at the page boundary, list the values each input can take, and render the combinations side by side. With four inputs of two values each, that is sixteen screens, which is cheaper to render than setting up sixteen accounts.

Disagree or have a war story? Reply on X