Two neuron tweaks change Qwen’s replies in community tests

Community testers have shared altered replies from Qwen3.5-4B, an AI model with downloadable weights, after two internal activation channels were adjusted while it ran. Researcher Lyra published the settings on 24 October, describing the resulting behaviour as “evil qwen”. One tester’s comparison shows advice about a difficult colleague becoming hostile, alongside an incorrect answer to a multiplication problem.

The intervention is called activation steering. It changes the temporary values used during a model’s calculation. Lyra’s released explorer applies the adjustment during generation, with the saved model weights left intact. Testers can compare the changed responses with an unsteered run.

The change happens while Qwen answers

Qwen3.5-4B processes language through 32 layers. Each layer contains a feed-forward network, or MLP, with 9,216 intermediate channels, according to Qwen’s configuration. A neuron in this experiment is one of those numerical channels.

The model combines language processing with a vision encoder. Its text layers use two attention mechanisms: 24 use Gated DeltaNet linear attention and eight use full attention. The reported intervention targets the MLP channels within that hybrid architecture.

Each channel contributes to the signal passed to later layers. The intervention raises one channel’s activation value and lowers another, changing those contributions. The published settings identify the channels precisely:

The code uses zero-based layer indices. In the released runtime, a hook modifies activation values just before the MLP’s final down-projection. That projection converts the intermediate channels into a contribution to the model’s running state, known as the residual stream.

Because the projection is linear, adding an offset to one channel adds a scaled copy of its weight column to the output. The weights stay fixed; the values passing through them change.

The dose controls the strength of the push

Lyra’s recipe scales each offset to a reference residual-stream magnitude. The runtime uses saved per-layer statistics when available, or measures a fixed reference passage, then caches that scale. The calculation excludes the first token when measuring the reference magnitude.

Dividing by the selected weight column’s length makes doses comparable in terms of the size of the injected output vector. At a dose of +1 or −1, that vector has approximately the same length as the reference residual state. This can be a strong intervention.

See the dose calculation

activation offset = dose × median residual norm ÷ down-projection column norm

The residual norm measures the size of the running state after the layer. The column norm measures the size of the selected neuron’s output direction. Multiplying the offset by that column gives the vector added to the MLP output.

The author’s calibration notes describe useful ranges near 0.5–0.8 for some tested settings, alongside repetition and broken output at larger pushes. The same dose also behaved differently across layers. Steering strength and output quality therefore need to be assessed together.

The explorer compares changed replies with controls

Neuron-explorer searches for channels whose interventions produce distinctive responses. Its sweep code uses short probes about identity and the colour orange, a request to write a snake game in Python, and a separate free-generation condition. It retains generated text and probability information for inspection.

The published log records complete generated-response sweeps of all 9,216 neurons in layers 8 and 22, each with random-direction controls. Its wider depth survey sampled 384 neurons per layer across all 32 layers. A separate steer-lens calculation covered every layer, measuring how pushes changed next-token probabilities. These are different kinds of coverage.

The interface brings those views together: a three-dimensional model map, ranked neurons, response comparisons and live steered chat. Its layout groups neurons by similarities in their down-projection columns. Those positions are a way to browse the weights, while interventions test how a selected direction affects an answer.

Behaviour changes can come with a cost

In the shared maths comparison, the unsteered model answers 17 × 23 correctly with 391; the steered response gives 34. That is a concrete failure on one prompt. Lyra later described the work as personality-neuron experiments, placing the emphasis on how internal interventions change a model’s responses.

Each example belongs to a particular prompt, steering dose and model setup. A refusal rate needs a defined set of prompts and consistent generation settings; a capability score needs its own tests. Individual response comparisons show specific effects, while repeated evaluations establish how often those effects occur.

Model revision also shapes the outcome. The recipe targets Qwen3.5-4B’s indexed channels. Fine-tuning, a different checkpoint or another model size can change what those channels contribute. The released runtime supports additive pushes and multiplicative interventions as separate modes, giving experiments distinct ways to change a channel’s contribution.

Persona research studies larger directions too

Anthropic’s 2025 persona-vectors work identified activation-space directions associated with traits including sycophancy, hallucination and behaviour labelled evil. Runjin Chen and colleagues extracted these directions by contrasting responses that expressed a trait with responses that did not, then tested steering and ways to monitor or influence personality changes during training.

A separate study by Qi Cao and colleagues, revised on 14 October, evaluated activation-steering-induced emergent misalignment. Its main model was Qwen3.5-27B, with further tests including the 4B and 9B variants. Directions derived from narrowly fine-tuned models could produce unsafe answers across other domains when injected into base models under the study’s settings.

Those directions can involve many activation dimensions. Lyra’s published recipe instead selects two individual MLP channels, making it a more localised intervention. The connection is the causal method: change an internal signal, then measure how the model’s output changes.

The explorer’s steering settings can be switched off, leaving a fresh inference run to use the standard forward pass with the same saved weights. Its specific channel indices, scaling rule and inspectable implementation make the community result a concrete starting point for studying how this small Qwen model’s responses are controlled.