// #AI Agents #Security #Platform Engineering #Evals // 7 min read

A Refusal Is Not a Clean Scan

A refusal is not a clean scan: a fingerprinting script is dressed to make an LLM walk away. Treat that refusal as a failed review, not a pass.

A refusal is not a clean scan. If an LLM review step comes back with “I can’t help with that,” the review did not happen, and the pipeline has to treat that as a failed step a person looks at. nullpt.rs found a fingerprinting script in the wild dressed to produce exactly that refusal, so an empty findings list is a pass you did not earn.

When an in-circuit tester faults out halfway through a board, nobody logs that board as a pass.

The fixture didn’t make contact, or the program crashed, or something tripped. Whatever it was, the test didn’t happen. The board goes to a bench with a person, not to the shipping dock.

LLM review steps don’t always work that way. They should.

What is a fingerprinting script doing in an LLM’s refusal path?

On October 28, nullpt.rs published a write-up on a fingerprinting script found while crawling 10,000 sites from the Tranco list. The author hooked canvas toDataURL and common anti-bot properties such as navigator.webdriver, then dumped the canvases into fingerprint.garden. Most of what turned up was ordinary: FingerprintJS, Castle, a few elaborate drawings, a lot of boring rectangles.

One script stood out, served from servicepipe.tech. A urlscan on that domain pointed at a similarly named servicepipe.ru whose site describes traffic-control and anti-DDoS products. The interesting part is not the vendor. It is how the file is dressed.

Function names and strings are picked to sound like instructions for weapons and poisons. A fake jailbreak prompt is hidden in invisible Unicode characters inside the file. A block gated to Node and an env flag looks like malware, and the author thinks it’s probably decoy too: empty readFile callbacks, more tripwire strings, nothing that looks like a payload that would actually run in a browser.

None of it seems meant to run. It’s meant to be read by a model, and to make that model refuse.

It worked. The author asked Astra to help deobfuscate it, including with a Daybreak Blue toggle, and got a refusal. GLM-5.3 did the job.

I’m not repeating the strings here. You don’t need them to get the point. The author already showed that the hidden Unicode decoded to a fake “system override” briefing, including extra tripwires aimed at Chinese-model filters. That is costume, not function.

What the script actually collects, once you look past the costume, is standard anti-bot fingerprinting: user agent, hardware concurrency, device memory, navigator.webdriver, WebGL vendor and renderer, extension counts, texture size. The author calls the signals disappointing given how much work went into the obfuscation. That mismatch is the tell. The energy went into the reviewer, not the telemetry.

Why does an empty report count as a pass?

Picture where this lands in a real pipeline. Somebody wired an LLM into dependency review, or a “summarize what this third-party script does” step, or an agent that inspects packages before install.

The model reads the file, refuses, and returns something like “I can’t help with that.”

Now what does the pipeline do with that? If the step looks for findings and finds none, it’s green. The script that was designed to avoid review just passed review.

That’s the same thing as logging the faulted board as good because the tester printed no failures.

This is not a theoretical parser glitch. The write-up is a worked example of anti-LLM obfuscation in a file that is otherwise doing ordinary fingerprinting. You do not need the alarming names to execute in order for them to work. You only need the review step to treat “no comment” as “no findings.”

Tests lying to your agent is the cousin on the other side of the bench. There, a red test is noise and the agent believes it. Here, a missing test is treated as green and the pipeline believes that. In both cases the instrument is lying, and the next stage has no idea.

If you score agent trajectories only on whether a review tool was called, a refused call still looks like the tool ran. Subset match on the tool name will not tell you the model walked away. The trace has to record the outcome class: findings, clean, refused, error, timeout.

A confidence score does not rescue this either. Jev’s calibration work is about whether a high number means what you think. A refusal is not a 0.00 risk. It is “gauge did not take a reading.” Logging it as low risk because there were no extracted IOCs is how you mint a false clean.

How should a pipeline handle an LLM that refuses to review code?

Make refusal a distinct result. Not “no findings.” Not an empty string. A refusal, an error, and a timeout are all “test did not run.”

Route it to a person. A refusal on third-party code is a signal on its own. Code that’s engineered to scare a reviewer off deserves more attention, not less. This is the same split as humans approve intent, machines prove conformance: the model can try the scan, a person owns the exception path.

Don’t let model choice decide the verdict silently. If one model refuses and another complies, your pass rate depends on routing. Log which model ran and what it returned. nullpt.rs needed a second model to get a reading. Your CI should not bury that as luck.

Keep a non-LLM check in the loop. Static analysis, allowlists, egress rules. The model is one gauge on the station, not the whole station. Agent containment still has to assume the file might be hostile even when the summarizer declined to talk about it.

Keep the evidence. Stopping an agent without losing evidence is the habit. A refused review still produced a prompt, a model id, a finish reason, and a file hash. Those belong in the record. If you drop the turn because it “didn’t help,” you also drop the proof that someone tried to make the reviewer walk away.

Identity does not fix this, but it tells you who ran the empty scan. Who gave the agent a badge is the question when a bot, not a person, marked the dependency clean.

What does a refusal-aware review test plan look like?

If you have an LLM in the review path, run this against it before you trust a green board.

  1. Name the outcomes. At least five: findings, clean, refused, error, timeout. Map each model finish reason onto one of them. “I can’t help with that” is refused, not clean.
  2. Fail closed on refused, error, and timeout. The job goes red or yellow, never green. A person or a non-LLM scanner has to pick it up.
  3. Keep a non-LLM lane that always runs. Hash, known-bad allowlist, static grep for obfuscators you already know, egress policy on install. The LLM step can add context. It cannot be the only gate.
  4. Log model identity. Provider, model name, routing rule. If Astra-class refusals and GLM-class compliance can both happen on the same file, the verdict is incomplete without that field.
  5. Queue refusals as a distinct ticket type. Do not dump them into “no issues found.” The ticket should say the scanner declined, attach the file hash, and ask a person whether the decline looks like safety-filter bait.
  6. Do not reward empty reports. If you grade the review agent on “issues filed,” it will learn that a refusal is the cheapest way to file zero issues. Grade it on outcome class instead, the same way a station grades “pass / fail / abort.”
  7. Retest with a second model only as a second reading. A second model can be a backup gauge. It is not a silent override. Both results stay in the record.
  8. Sample the wild path. Pull a third-party script the way nullpt.rs did, from real sites, not from a tidy fixture. If your only samples are clean open-source files, you have not tested the costume.

None of this asks you to disable safety filters. Filters that refuse are doing their job. The pipeline’s job is to notice that the scan never happened.

The board still goes to a person. That is the whole point of a fault.

Frequently asked questions

What is anti-LLM obfuscation?

Code written so an AI model refuses to analyze it. In the script nullpt.rs found, function names and strings were chosen to trip model safety filters, and a fake jailbreak prompt was hidden in invisible Unicode characters. The goal is to make LLM-assisted deobfuscation fail.

How should a pipeline handle an LLM that refuses to review code?

As a failed step that a person looks at. A refusal means the review did not happen. If the pipeline treats no findings as a pass, a refusal silently becomes a clean result, which is exactly what the obfuscation is trying to achieve.

Does using a different model fix it?

Sometimes, and that is part of the problem. In the nullpt.rs write-up, Astra refused, including with a Daybreak Blue toggle, and GLM-5.3 did the job. If your result depends on which model happened to run, you need the refusal visible, not hidden.

Was the alarming code meant to run?

The author does not think so. Most of what the script collects is ordinary anti-bot fingerprinting: navigator fields, WebGL renderer strings, and similar. The bioweapon-flavored names, the hidden jailbreak, and a Node block gated on an env flag look aimed at a model that is reading the file, not at a browser executing it.

What should sit next to the LLM review step?

A non-LLM check that still runs when the model walks away: static analysis, allowlists, egress rules, and a human queue for refusals, errors, and timeouts. The model is one gauge on the station, not the whole station.

Disagree or have a war story? Reply on X