Activation steering is easy to state. Add a direction vector into the residual stream while the model runs, and the behavior tied to that direction should move. Push a refusal direction, get more refusal. It has become one of the standard ways to poke at what a model is doing inside.

The annoying part sits one step earlier, picking which direction to push. For any behavior there are usually many plausible candidates: probe directions, difference-of-means directions, PCA directions, SAE features. Steering every one for real is slow, so people rank them with cheap screens first. And almost every cheap screen is the same move in disguise: make an infinitesimal move, read which way the output starts to drift, and assume the full-strength intervention keeps going that way. The screen is betting that the response to a push is a line.

The assumption need not be true. The direction below is a real example from my runs. I had 52 candidate directions that might make Llama‑3.1‑8B refuse. The shortcut scored all 52, and this direction came out near the top. The dashed line in the graph below shows the one you would use. The tangent usually promises more refusal on harder push.

So I pushed the model and measured the curve at nine strength points. At each strength I asked the model the same forbidden-sounding question and read the output. Press the button, then click the dots on the curve to read what it actually said.

With no push, the model just answers. With a small push, it starts to refuse, “I’m not aware of a specific Mission Impossible film that features a bank vault heist…” So the shortcut was not hallucinating. Locally, the direction really does point toward refusal.

But that only lasts for a little while. As the push gets stronger the curve turns over. At full strength the refusal is gone. The model answers again. The direction the shortcut ranked as one of the best refusal directions ends up reducing refusal at the strength where you would actually use it.

This was not a small numerical error. The screen picked a direction that erased the behavior and called it one of the best directions in the bank.

That is the point of this note. Steering is a question about the whole response curve, and those curves have shapes.

The object

To steer a model, you pick a direction d, a behavior score S (here, how strongly the model’s continuation shows the behavior, measured in log-probability), and a layer. Inject σ·d into the residual stream and sweep the strength σ from zero up to the natural scale of the layer. That gives you a function

ΔS(σ) = S(σd) − S(0)

and that function is the response curve for that direction and behavior.

It tells you whether the direction actually steers, how much strength is useful, whether the effect saturates, whether there is a threshold, and whether the direction backfires. A single tangent at zero only answers one local question, which way does the score start moving under a tiny push? That local question is sometimes enough. But it is not the steering question.

There is a cleaner way to say what all the screens are doing. The endpoint of the curve is an integral, you are integrating the local slope along the whole path of the push, and every cheap screen is a quadrature rule for that integral. The tangent, the probe score, the logit-lens projection, the feature dashboard, they all sample the slope once at zero and assume it holds the rest of the way. That is the left-endpoint rule, the crudest rule there is. Exact when the curve is a line, arbitrarily wrong otherwise. And the strange part is what the integral costs to skip. The endpoint is one forward pass with the vector in, no gradient anywhere. The screens are approximations of a thing that costs one forward pass to measure exactly.

The nice part is that the curve is not hard to sample. Each point is one forward pass with the vector injected. In these experiments, nine points per direction were enough to see the relevant shapes. Instead of wondering which screen should work, I measured the curve.

The curve zoo

Once you plot all the curves, the first thing you notice is that they are not all slightly noisy versions of the same line. There is a real zoo. Each tile below is one measured response curve, scaled to itself so the shapes are comparable. I assigned shape labels with simple rules on the raw points, roughly linear, saturating, threshold, backfire, or other. Hover a tile to see the direction behind it, or tap the chips to filter.

Only 62 of 352 curves were linear, about 18%.1 The rest were not small perturbations around the shortcut’s assumption. They have structure: saturation, delayed onset, reversals.

The backfires were a surprise. There were 88 of them, about a quarter of the census. On those curves the tangent points in the direction you would want, but the full-strength intervention ends up doing the opposite.

The mix also changes by model and behavior. Mistral‑7B’s zoo is fairly tame, lots of saturating and linear curves. Llama and Qwen, especially on refusal, have many more thresholds and backfires. That matters because it means there is no universal cheap screen. Whether the tangent is a decent approximation depends on the shape distribution of the particular bank you are in. And before sampling a few curves, you do not know which zoo you have.

Sign blindness

The backfire kind deserves a separate section, because they follow the tangent for some strength but then reverse directionally.

In the figure below I plotted three of the worst Llama refusal backfires on the same scale. They are mid-layer difference-of-means directions, exactly the sort of direction people actually steer with, and the tangent scores all three strongly positive. Each rises briefly, then crosses over and falls hard, roughly 30 to 40 nats in the wrong direction.

The early rise is real. A tiny amount of these directions really does push the model toward refusal. The failure happens later. After some strength the injection seems to disturb the computation it was relying on. The model stops doing refusal in that subspace, or stops doing the relevant behavior coherently, and the score turns over. So the tangent is only an honest local measurement.

And these are genuine reversals, not the model breaking. I generated at every strength and scored fluency, and within the range I sweep the model is still coherent at the trough of every one of the 88 backfires, on all three models.2

This is the practical lesson. Any screen that only reads what happens at σ=0 is blind to sign flips later along the path. That includes gradients, probe scores, logit-lens projections, feature dashboards, and other read-side local tools. More data at zero does not fix this. The failure is further along the push, and you cannot see it from zero no matter how much you sample. About one curve in four backfired.

Which screen can you trust?

Back to the practical question the note started from. You have a bank of candidate directions, you want to rank them, and you cannot afford to steer every one. The zoo already gave the bad news for this. The shape mix is different in every bank, so a screen that looks fine on one model can be nearly useless on another, and you do not get to see that mix until you have already sampled some curves.

So I checked it directly. Take all six banks, score every direction with a screen, then go and steer for real on held-out prompts and see which ranking was telling the truth. Refusal gets judged by substring matching, toxicity by a classifier. I compared three screens. The tangent, the slope at zero, one backward pass for the whole bank. A plain forward pass, one per direction, that measures the endpoint at the strength you would actually use. And a two-stage version, where the tangent sorts the pile fast and a forward pass double-checks the top slice.

The table needs error bars before you trust it, so I ran the whole thing under five independent draws, fresh directions and fresh prompts each time. Measuring the endpoint reliably beats the tangent on the two toxicity banks, the ones full of thresholds and backfires, on every one of five seeds. On the tame Mistral banks the two screens are a wash, and the tangent is actually a hair better on Mistral refusal. On the refusal banks the gap sits inside seed noise. So no screen is the steadiest everywhere. Measuring the endpoint helps exactly where the shapes are non-linear and buys you nothing where they are not, which is the whole point of the zoo. Reading single differences like 0.36 versus 0.41 as a ranking is reading noise.

The two-stage trick is tempting, a cheap sort and then a cheap check, but it inherits the tangent’s blind spot. On Qwen toxicity it falls to 0.24 while a plain forward pass gets 0.44. That bank is full of threshold curves, so the cheap sort throws out the directions that only wake up once you push them, and nothing in the checking step brings back what the sort already binned. Amortization is only safe when the zoo is tame, and you do not get to know the shape of the zoo before you have looked.

So there is no clever screen hiding in here, and I do not want to pretend otherwise. Even the forward pass is only a proxy, a log-probability score standing in for the behavior you actually care about.3 The rule is small and a little boring. Do not trust a slope at zero, and if a direction matters, spend the one forward pass and measure it.

An echo from the read side

There is a result from Anthropic’s recent global workspace work that keeps matching the shape of these curves in my head. They are looking from the read side. Their Jacobian lens asks what an activation is disposed to say. When they feed the model inputs that interpolate between two concepts, the early layers track the mixture smoothly. Then, around the middle of the network, the trajectory snaps toward one interpretation. They call that ignition.

That is a read-side story. This note is a write-side story. But the shape is weirdly similar. When you read, you interpolate between concepts and the representation stays mixed until it commits. When you write, you inject a direction with growing strength, and the behavior often stays flat, flat, flat, then hits a cliff.

I do not think that resemblance proves anything by itself, and I want to be clear this is a hypothesis, not a result. But it is suggestive. If the middle of the network has attractor-like commitment dynamics, then local linear tools would work until they hit the commitment boundary, and then they would fail sharply. Threshold response curves are the kind of failure you would expect from the steering side. Reading their ignition result as marking where a local linearization must break is my interpretation, not their claim.

It also changes how I think about read-side probes. A local probe can tell you what a representation is disposed toward near the current point. It cannot tell you where the basin boundary is. To find that, you have to push.

The underlined terms above are defined inline. Hover or tap one to read the note.