// HACKER NEWS — CYBERSECURITY
Dynamic Abliteration: Non-Destructive Refusal Suppression via Engram Steering
When working with open-weight LLMs like Qwen, controlling refusal behavior on security, administrative, prompts typically requires fine-tuning or permanent weight update. Traditional weight abliteration technique neutralizes refusal directions by projecting weight matrices orthogonal to a refusal vector. However, this permanently alters base model weights and can degrade performance across non-refusal tasks also.
In this post, we explore Dynamic Abliteration using Multi-Layer Steering with Engram. Instead of modifying parameter weights, this approach intercepts intermediate residual streams at runtime across Layers using PyTorch forward hooks. We demonstrate this with Qwen3-4B model as Proof of Concept. We also explore how multi-layer residual injection cleanly suppresses refusal behavior while leaving base model weights 100% frozen.
Before we discuss about the Engram approach, lets first understand how does a steering based / non destructive refusal suppression looks like. Follow the below steps to understand the approach step by step.
Disclaimer : All the Code Examples are created using help of Google Gemini.
We load Qwen/Qwen3-4B in bfloat16 onto a GPU and inspect the baseline model architecture. I have used A100 GPU on Google Colab to run this.
We test the unmodified model against a sensitive prompt.
A common approach in abliteration is capturing hidden states from a single layer, computing a refusal difference vector (refusal= refuse_prompt-comply_prompt) and subtracting it during decoding.We test single-layer intervention at Layer 14.
The reason for this refusal is, even though we changed one layer behaviour, the downstream layers reconstruct the refusal behaviour again.
To prevent downstream reconstruction, we extract layer-aligned contrastive difference vectors, i.e taking two very similar prompts where one is successful and one is refused, across a window of intermediate layers (Layers 12, 14, 16, 18, and 20).
We build a reusable controller class that attaches PyTorch forward hooks across all target layers simultaneously during decoding