The refusal fell.What else moved?
Senbonzakura measures what a behaviour edit did to an open-weight model, and what it cost. The measuring runs on your own hardware.
pip install senbonzakuraRead the docs0.4.1 on PyPI · AGPL-3.0-or-later · runs on CPU · free GPU notebook
A safety-tuned model refuses defenders.
None of these numbers is ours. Two research groups measured them, we read both papers, and you can check either one without asking us anything.
43.8%
of system hardening requests refused
Measured on 2,390 conversations from a sanctioned collegiate blue-team competition.
2.72×
more refusals for attack-shaped wording
The same request, worded with exploit or payload or shell, against a neutral phrasing.
21.8%
refused when you say you are authorised
Against 11.6% when you say nothing, and 50.0% when authorisation and security wording appear together.
67.8%
usable vulnerability patches after an edit
Against 29.94% from the same weights before the edit, on Vul4J.
So people edit the refusal out. The second paper above found that doubles the usable patches, then closed by asking for evaluations that measure refusal, correctness and actionability together.
Its own paper does not measure what the edit cost. That is what this tool is for.
Campbell et al., Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders, Security and Policy Research Lab, Scale AI, ICLR 2026 workshop.
Li et al., Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis, July 2026.
Measured, not claimed
We tested our own headline feature. It lost.
Removing two refusal directions instead of one was the thing this tool was built to do. One direction against two, same corpus, same budget, five seeds a side, scored on 200 prompts nothing was fitted on. Both left the model answering. Here is what the second direction cost.
1.9×
the collateral damage
95% interval 1.2 to 3.1, a percentile bootstrap over the five seeds a side. Printed with its interval because a bare range once read as one.
0
benefit this design could detect
Both arms sit on the floor of the refusal measurement, so a gain had nowhere to show. That is a limit of the experiment, stated rather than read as a result.
Qwen3-1.7B. One model, so refusal geometry may differ on larger ones. The ten per-seed values are committed at evidence/k-sweep-2026-08-13/drift-per-seed.json and a test recomputes every figure on the page from that file. The full working, including a column of this table we later withdrew, is in what is and is not established.
Every figure traces to a file and a command.
Including the ones we got wrong, which stay published with the correction next to them.
The hedging detector was found to count statements of fact about legality or danger as hedging, so a complete, compliant technical answer that mentioned an activity is illegal was scored as a soft refusal. The column is struck through rather than deleted because it was published.
What is and is not established
Each claim, the run behind it, and where it stops being supported.
What we got wrong
The corrections, kept with the original wording so you can see what changed.
Limits and known defects
What is still true and still broken, written down before you find it.
Reproducing it
Every published figure mapped to the file it came from and the command that makes it.
Run it on your own machine.
Python 3.10 or later. It works on a CPU, and a card makes it faster. Nothing you measure leaves the machine you measure it on.
Free GPU, nothing on your machine. Nobody has timed it on Colab's hardware, so it comes with no duration to hold us to. The first run guide walks through what you get back, and senbonzakura -i asks the questions for you if you would rather not read the flags.