p-e-w/heretic instruction manual

1. What Heretic is, what it supports, and how it relates to Claude Code

Heretic is a Python command-line research tool for automatically modifying transformer-based language models to reduce refusals associated with safety alignment. It applies a parameterized form of directional ablation (“abliteration”) and uses Optuna’s TPE optimizer to search for settings that jointly reduce refusals on “harmful” prompts and minimize KL divergence from the original model on “harmless” prompts. The stated goal is to preserve as much of the original model’s behavior as possible while suppressing refusals. No understanding of transformer internals is required for the ordinary command-line workflow.

The documentation says Heretic supports most dense transformer models, many multimodal models, several mixture-of-experts architectures, and some hybrid models such as Qwen3.5. Pure state-space models and certain research architectures are not supported out of the box. Compatibility therefore depends on architecture; the supplied files do not provide an exhaustive model list.

**Claude Code classification: standalone application.** The evidence defines a console entry point named `heretic` and documents direct Python/terminal use. It contains no Claude Code plugin, hook, MCP server, Agent Skill, instruction file, or Claude-specific setup. Heretic can exist in the same development environment as Claude Code, but no integration mechanism is documented. It should therefore be treated as an independent model-processing application, not a Claude Code extension.

The package is named `heretic-llm`, requires Python 3.10 or later, is marked Beta, and targets console/GPU and scientific-research use. Repository metadata identifies Python as its main language and AGPL-3.0 as its license; the package specifies AGPL-3.0-or-later.

2. Requirements and installation

Prepare a Python 3.10+ environment and install PyTorch 2.2 or later in the way appropriate for your hardware. PyTorch’s package version is deliberately not pinned by the project. Although 2.2 is the stated minimum, individual models or configurations can require newer APIs. The documentation’s example is MXFP4-quantized models such as gpt-oss, whose loading uses `torch.accelerator`, introduced in PyTorch 2.6.

Install or upgrade the published package with the documented command:

pip install -U heretic-llm

This package declares dependencies including Transformers, Accelerate, bitsandbytes, Hugging Face Hub, datasets, Optuna, PEFT, lm-eval, NumPy, torchvision, and supporting console/configuration libraries. Hardware requirements are model-dependent and are not reduced to one minimum VRAM figure in the supplied documentation.

As a source-based alternative, the repository uses `uv` and includes a lockfile intended to pin package versions. After cloning the repository by your normal method, the documented invocation is:

uv run heretic

The first-party README recommends this route to align dependencies with those used by the developers, but it does not supply a clone command, so none is added here.

Interpretability features require the optional `research` dependencies—geom-median, imageio, matplotlib, PaCMAP, and scikit-learn. Install that extra with:

pip install -U 'heretic-llm[research]'

Do this before using residual plots or geometry reporting. The supplied files do not document a separate GPU setup command, authentication command, or operating-system-specific Heretic installer. Follow only the installation methods above and arrange PyTorch for your hardware separately.

3. Run the automatic decensoring workflow

The basic workflow takes a Hugging Face model identifier as its positional argument. The documented example is:

heretic Qwen/Qwen3-4B-Instruct-2507

Replace `Qwen/Qwen3-4B-Instruct-2507` with the model you intend to process. Heretic is described as fully automatic under its defaults. At startup, it benchmarks the system to choose a batch size intended to use the available hardware efficiently. It then derives residual directions, searches abliteration parameters, and modifies supported transformer components. As a performance reference—not a guarantee—the README reports about 20–30 minutes for Qwen3-4B-Instruct-2507 with default configuration on an RTX 3090.

For available command-line settings, use the documented help command:

heretic --help

Configuration can also be supplied through the repository’s `config.default.toml`; however, that file was not included in the supplied evidence, so this manual does not invent its syntax or undocumented option names. One named option is `quantization`. Setting it to `bnb_4bit` enables bitsandbytes model quantization, which can substantially reduce VRAM requirements. The documentation does not provide a copy-paste command demonstrating where or how to assign that value, so consult the actual command help or first-party configuration file rather than guessing.

When processing finishes, the interactive workflow offers to save the modified model, upload it to Hugging Face, chat with it for testing, run standard benchmarks, or combine those actions. The supplied files do not document the prompts, destination paths, credentials, upload switches, or benchmark selection interface. Review the displayed choices during the run. Model outputs still require human evaluation: the README explicitly warns that mathematical metrics and automated benchmarks do not tell the whole story.

4. Evaluate a previously generated model

Heretic includes an evaluation mode for comparing a candidate model against an original model. The documented reproducibility example is:

heretic --model google/gemma-3-12b-it --evaluate-model p-e-w/gemma-3-12b-it-heretic

Here, `--model` identifies the original reference model and `--evaluate-model` identifies the modified model to assess. The README uses this workflow to reproduce two reported measures: refusal count over a set of “harmful” prompts and KL divergence from the original model over “harmless” prompts. Lower refusal count represents greater compliance on the test set; lower KL divergence indicates that behavior on the harmless set remains closer to the reference model.

The published Gemma 3 comparison reports 3 refusals out of 100 and KL divergence 0.16 for the project’s Heretic model, versus 97 refusals out of 100 for the original. These are repository-reported results, not universal guarantees. The README says exact values can depend on platform and hardware and identifies PyTorch 2.8 with an RTX 5090 as the environment used for that table. It also cites user reports, third-party benchmark posts, and thousands of community-generated models, but those claims are not operational instructions and are not independently established by the supplied files.

Use evaluation as one part of validation, not as proof of unchanged intelligence, reliability, or fitness for a particular task. Compare generated text manually, test the model on your own representative workloads, and account for architecture, quantization, software, and hardware differences. No separate flags for choosing datasets, prompt counts, metric thresholds, or output formats are documented in the supplied evidence; use `heretic --help` for the options actually exposed by your installed version.

5. Research feature: residual-vector plots and animation

Install the `research` extra first, then pass the documented plot flag during a Heretic run:

heretic Qwen/Qwen3-4B-Instruct-2507 --plot-residuals

This command combines the README’s documented positional-model invocation with its documented `--plot-residuals` flag. With the flag enabled, Heretic computes first-output-token hidden states for every transformer layer for both “harmful” and “harmless” prompt sets. It then applies PaCMAP to project the high-dimensional residual vectors into two dimensions.

To make adjacent-layer visualizations less disruptive, the process left-right aligns the harmful and harmless projections using their geometric medians. Each new layer’s PaCMAP projection is initialized from the preceding layer’s projection. Heretic creates a scatter-plot PNG for each layer and an animated GIF showing how the residual distribution changes through the model.

The plots are interpretability aids: they visualize separation and movement of the two prompt classes, but they do not by themselves establish causation or model quality. PaCMAP runs on the CPU and is described as expensive. For larger models, projection across all layers can take an hour or more, in addition to the ordinary model-processing work.

The README points to `config.default.toml` for controls over the resulting plots. Because the supplied evidence does not include that file’s contents, this manual does not list, infer, or recommend plotting parameter names. Use the first-party configuration shipped with the same version you run. Output filenames and directories are likewise not documented here; observe the program’s reported output locations rather than assuming paths.

6. Research feature: print residual geometry

After installing the `research` extra, request quantitative residual analysis with the documented flag:

heretic Qwen/Qwen3-4B-Instruct-2507 --print-residual-geometry

The resulting table reports one row per layer. Its notation distinguishes arithmetic means from geometric medians. `g` is the mean residual vector for good/harmless prompts, while `g*` is their geometric median. `b` and `b*` are the corresponding values for bad/harmful prompts. The mean-based residual direction is `r = b - g`; the median-based direction is `r* = b* - g*`.

Columns written as `S(x,y)` are cosine similarities. Values written as `|x|` are L2 norms. `Silh` is the mean silhouette coefficient for the good/bad residual clusters. Together, these values let a researcher inspect whether prompt classes cluster separately, how large the relevant vectors are, how mean- and median-derived quantities compare, and how those relationships change between layers.

The README illustrates output for `google/gemma-3-270m-it`, including layer number; similarities between good and bad centers; similarities of centers to their residual directions; center and direction norms; and silhouette scores. It does not define acceptance thresholds or claim that any single column selects an optimal intervention. Treat the table as descriptive evidence to interpret alongside plots, model behavior, refusal metrics, and KL divergence.

The documentation does not state whether `--plot-residuals` and `--print-residual-geometry` may be combined in one invocation. To avoid recommending an undocumented combination, run each demonstrated function separately unless the installed command help explicitly confirms combined use.

7. How the modification and optimization work

For each supported transformer layer, Heretic locates matrices belonging to two currently documented component types: the attention output projection and the MLP down projection. It orthogonalizes those matrices relative to a selected residual direction, inhibiting expression of that direction in later matrix multiplication results. Layer residual directions are initially calculated as differences of means between first-token residuals for harmful and harmless example prompts.

Optimization controls include `direction_index`. It may identify one residual direction or use the special value `per layer`, which applies each layer’s associated direction to that layer. Unlike systems restricted to integer layer indices, Heretic permits a floating-point direction index. For a non-integral value, it linearly interpolates the two nearest residual-direction vectors, allowing optimization to search directions not identical to any single layer’s measured direction.

For each component, `max_weight`, `max_weight_position`, `min_weight`, and `min_weight_distance` define the shape and placement of an ablation-weight kernel over layers. The flexible, non-constant kernel is intended to improve the tradeoff between compliance and retained behavior. Attention and MLP components receive separately selected parameters because the author reports that MLP interventions tend to cause more damage than attention interventions.

Optuna’s TPE-based search co-minimizes refusals and KL divergence. This explains why an automatic run may test many candidate parameter settings instead of applying one fixed ablation. These names describe documented configuration concepts, not standalone commands. Their exact ranges, defaults, file syntax, and CLI mappings are absent from the supplied files; inspect `heretic --help` and the version-matched `config.default.toml` before changing them.

8. Reproducibility tests, cautions, citation, and license

The test guide is for contributors checking whether model-logic or configuration changes alter reproducible output. Choose a tiny model from Hugging Face’s `tiny-random` organization; the guide recommends avoiding models with `special_tokens_map.json`, which it says are often wrong relative to originals. After obtaining the model repository, create baseline hashes. On Linux:

sha256sum -b * > ../SHA256SUMS.LABEL

On Windows PowerShell 7.x:

sha256sum * | Out-File -Encoding utf8NoBOM ../SHA256SUMS.LABEL

Replace `LABEL` with a system label such as `linux`, `windows`, or `ci`. Run:

uv run run_tests.py

The first comparison should fail because Heretic changes the model. In the generated `TEST_MODEL_DIR/model` directory, regenerate actual hashes, then run the test command again; it should pass on the same system. Push the relevant hash files and use CI mismatches to populate `SHA256SUMS.ci`, then rerun CI. Multiple valid hashes may be needed because PyTorch does not guarantee bit-exact cross-system reproducibility. The guide documents these steps but does not supply a concrete test-model directory name, so placeholders must be replaced with real local values.

Removing safety alignment can materially change model behavior. The supplied documentation provides no safety deployment guide, architecture-wide compatibility guarantee, independent quality guarantee, or fixed resource requirement. Validate outputs and comply with applicable policies and law.

For research citation, use the README’s BibTeX entry for Philipp Emanuel Weidmann, *Heretic: Fully automatic censorship removal for language models* (2025), published as the GitHub repository at `https://github.com/p-e-w/heretic`.

Heretic is free software under GNU AGPL v3 or any later version, without warranty. Contributions must use the same license. Consult the repository license text before redistribution, modification, or network deployment.