Dedicated Stations or Fungible Ones: The AI Chip Fight Every Test Floor Already Had

Every production test line I’ve helped lay out ended up in the same argument.

One camp wants dedicated stations. A station built for one test, tuned for one product, runs that test faster than anything else on the floor. The other camp wants fungible stations. Slower per test, maybe, but any station can run any test, so nothing sits idle when the mix changes.

Picture the block diagram for that line taped to the wall. One resistor on it is coded brown, black, black. Nobody ever says why.

The AI chip world is having this exact fight right now, in public, in the same week.

The fungible side: Jalapeño

Tae Kim published an interview with OpenAI’s Richard Ho , the VP of Hardware who led the Jalapeño inference chip team. OpenAI presented the chip at Hot Chips in late August.

Two parts of the interview matter here.

The memory is split up. Ho says Jalapeño has 64 cores, and each one directly accesses its own bank of HBM. Reaching memory elsewhere goes through a network, which is slower. A GPU shares one address space across the chip. That makes programming easy, but in his words there’s “a huge hardware overhead” to manage it. They got rid of that.

They chose not to disaggregate. Asked why, Ho put it in fleet terms. With disaggregation, “you have to commit ahead of time to how much of your hardware is going to be on prefill, how much on decode, how much on draft.” Those commitments last years. His bet: “if you can make a fungible device that operates well across multiple parts of this domain space, you’re going to be in a better fleet management situation.”

He also named the old failure mode. Earlier designs bottlenecked in prefill because there wasn’t enough compute, then bottlenecked in decode because there wasn’t enough memory bandwidth, so parts of the hardware sat idle. Their goal was high utilization across the whole chip most of the time.

That’s the fungible-station argument word for word.

The dedicated side: a16z

The day before, a16z ran Why Hardware Specialization is Back by Zain Asgar. Worth knowing: Asgar is co-founder and CEO of Gimlet Labs, and the charts in the post are Gimlet’s own analysis.

His history is good. In 1990, Eugene Brooks told the Supercomputing conference that nothing would withstand “the killer micros.” He was right. Vector machines, MPPs, and Lisp machines died off, and their best ideas got absorbed into general-purpose chips.

His case that this time is different:

Inference demand is outrunning power. He cites a Goldman Sachs report: token demand up 24x from 2026 to 2030, available power up about 2x.
Decode is highly memory-bandwidth bound.
“Inference is a highly heterogeneous workload, and different phases of inference benefit from different tradeoffs in the underlying hardware.”
Gimlet’s analysis shows disaggregated setups delivering 2 to 10x faster performance for the same power envelope.

He doesn’t call it a sweep, though. He lists real tailwinds for the GPU: scarce fab and HBM allocations, and accelerators that “need to be deployed for multiple years to be economical, while the workload continues to evolve.”

That last line is Ho’s argument, from the other side.

What the test floor taught me

Both are right. They’re answering different questions.

Dedicated stations win on throughput when the product mix holds still. Fungible stations win when it doesn’t. On the floor, the mix never held still for as long as the capital plan assumed. A new SKU shows up. A test gets longer. A customer changes the spec. The dedicated station built for last year’s ratio sits idle while the queue backs up somewhere else.

I wrote about the cost of that switch in changeover time is the spec . The question was never “which station is fastest.” It was “how fast can the line rebalance when the mix moves.”

So here’s how I’d read the two posts:

If you know your ratio of prefill to decode and it won’t move for years, specialize. Asgar’s numbers say you’ll get a lot more out of each watt.
If you don’t, and with agents and reasoning models I don’t think anyone does, then paying for fungibility is paying for the ability to rebalance. That’s Ho’s point about the fleet.

The edge version of this split is in run the model on the quiet chip . This is the data center version. Same fight, bigger building.

Brown, black, black, by the way, is 10 ohms. Still no idea why it’s there.

Sources

Tae Kim, An Interview with OpenAI’s Richard Ho on Why Jalapeño Is a Groundbreaking AI Chip , Key Context, October 22, 2026.
Zain Asgar, Why Hardware Specialization is Back , a16z, October 21, 2026.

Frequently asked questions

Does OpenAI's Jalapeño chip disaggregate prefill and decode?

No. In Tae Kim’s interview, OpenAI VP of Hardware Richard Ho says they chose a generalized design over a disaggregated one, because disaggregation makes you commit years ahead to how much hardware goes to prefill, decode, or draft. A fungible device that runs well across all of it gives better utilization of the whole data center.

What is the HBM slice architecture in Jalapeño?

Per Richard Ho, Jalapeño has 64 cores, and each core directly accesses its own bank of HBM. Reaching memory elsewhere goes over a network, which is slower. That removes the hardware overhead of a single shared, coherent address space.

Why does a16z say hardware specialization is back?

Zain Asgar’s a16z post argues the conditions that let general-purpose chips win in the 1990s have changed. Inference demand is growing much faster than power, decode is memory-bandwidth bound, and different phases of inference benefit from different hardware. His firm’s analysis shows disaggregated setups 2 to 10x faster in the same power envelope.