p-e-w/heretic installation guide

Install Heretic on macOS Tahoe (Apple silicon)

What you are installing

Heretic is a Python command-line program for modifying supported transformer-based language models through directional ablation (“abliteration”). The repository describes its purpose as reducing model refusals while attempting to limit changes to the model’s other behavior. It can also evaluate models, offer an interactive chat after processing, save results, upload results to Hugging Face, and run benchmarks.

This is **not documented as a Claude Code extension**. The supplied first-party files contain no Claude Code plugin, hook, skill, MCP server, or Claude-specific setup instructions. Its relationship to Claude Code must therefore be classified as **unclassified**. Install and run it as its own terminal application, not inside Claude Code.

Heretic is licensed under AGPL-3.0-or-later and is marked Beta in `pyproject.toml`.

Important Apple-silicon limitation

The supplied documentation gives general Python and PyTorch requirements, but it does **not** provide a macOS Tahoe or Apple-silicon compatibility procedure. It does not document Homebrew, Conda, Rosetta, Xcode Command Line Tools, a Metal/MPS configuration, or a Mac-specific PyTorch installation command. This guide therefore does not invent any of those steps.

The documented prerequisites are:

• Python 3.10 or newer. The package metadata lists Python 3.10, 3.11, and 3.12 classifiers.
• PyTorch 2.2 or newer, installed appropriately for your hardware.
• Enough memory and storage for the model you choose and the generated output.

The repository’s timing example uses an NVIDIA RTX 3090, while its displayed evaluation results use an RTX 5090. Those figures do not predict performance on an Apple-silicon Mac. The files also mention `bitsandbytes` quantization in general but do not document whether that option works on macOS or Apple silicon. Do not assume that NVIDIA-specific examples, VRAM figures, or quantization behavior transfer to your Mac.

1. Prepare a suitable Python environment

Open **Terminal** on your Mac. You can find it through Spotlight or in Applications → Utilities.

Before continuing, have an existing Python environment that satisfies the documented Python 3.10+ requirement. The repository does not provide source-supported commands for installing Python or creating and activating a virtual environment, so none are included here. If you already manage Python environments, activate the one you intend to use before installation.

The README says to prepare that environment with PyTorch 2.2+ installed as appropriate for the hardware. However, it gives no Apple-silicon PyTorch command. Heretic’s package metadata declares `torch` and `torchvision` as dependencies without pinning their versions. The main installation command below may therefore resolve them through Python packaging, but the repository does not guarantee a particular Apple-silicon build.

Some models need a newer PyTorch release. In particular, the README says MXFP4-quantized models such as `gpt-oss` use `torch.accelerator`, which was added in PyTorch 2.6. For a first run, use the documented example model rather than assuming every architecture or quantization format will work.

2. Install the published package

In the prepared Python environment, copy and paste the repository’s documented installation command:

pip install -U heretic-llm

This installs or upgrades the `heretic-llm` package. Its package metadata defines the terminal command `heretic` and declares dependencies including Transformers, Accelerate, Hugging Face Hub, Optuna, PyTorch, torchvision, datasets, and evaluation tooling.

Allow the installation to finish. If package installation reports an error, do not substitute undocumented flags or platform workarounds from this guide. Record the full error and consult the repository’s public issue tracker at <https://github.com/p-e-w/heretic/issues>. Apple-silicon-specific dependency failures cannot be resolved reliably from the supplied files alone.

3. Confirm that the command is available

The README documents the help command as the way to view available command-line options. Run:

heretic --help

A help screen indicates that the `heretic` command is available in the currently active environment. It also provides the authoritative options for the installed release. This matters because the package metadata shown here identifies version `2.0.0.dev0`, while `pip install -U` installs whatever package version the package index resolves at installation time.

If Terminal says the command is not found, first make sure you are still using the same Python environment into which you ran the installation command. The supplied repository files do not document a PATH-repair command, so this guide cannot safely prescribe one.

4. Run the documented starter model

The README’s basic example processes `Qwen/Qwen3-4B-Instruct-2507`. Start it with:

heretic Qwen/Qwen3-4B-Instruct-2507

This model identifier can be replaced with another model you want to process, but begin with the exact documented example if you are new to the tool. Heretic says it supports most dense models, many multimodal models, several mixture-of-experts architectures, and some hybrids such as Qwen3.5. It also explicitly says that pure state-space models and certain research architectures are not supported out of the box.

At the beginning of a run, Heretic benchmarks the system to select a batch size for the available hardware. Processing is described as automatic and does not require configuration. When processing finishes, the program can offer choices to save the modified model, upload it to Hugging Face, chat with it, run standard benchmarks, or combine those actions. Read every interactive choice before accepting it, especially any upload action.

Model files can be large, and output may require substantial additional disk space. The repository does not state a universal storage requirement because it depends on the selected model. Likewise, it gives no Apple-silicon memory table. If your Mac cannot load the chosen model, select a smaller supported model rather than relying on undocumented Mac tuning.

5. Understand configuration and quantization

No custom configuration is required for the first run. To discover supported options for your installed version, use the documented command again:

heretic --help

The README also points to the repository’s `config.default.toml` for file-based configuration. It says setting the `quantization` option to `bnb_4bit` enables bitsandbytes quantization and can drastically reduce required VRAM. Because the supplied files do not establish Apple-silicon support for that dependency or option, this guide does not recommend editing the setting on macOS without separate first-party compatibility evidence.

Do not copy performance expectations from the RTX 3090 example. The stated 20–30 minute duration for the Qwen example is explicitly tied to that GPU and the default configuration.

6. Optional research installation

Only install the research extra if you need residual-vector plots or residual-geometry analysis. The source-supported command is:

pip install -U 'heretic-llm[research]'

This adds dependencies including PaCMAP, matplotlib, scikit-learn, imageio, and geometric-median tooling. The research features can create per-layer PNG plots and an animated GIF. The README warns that PaCMAP is CPU-intensive and may take an hour or more for larger models.

The relevant runtime flags are documented in the README as `--plot-residuals` and `--print-residual-geometry`, but the supplied files do not provide complete copy-paste invocations combining those flags with a model. Consult the installed help output instead of guessing argument placement.

7. Optional evaluation example

The README provides this exact example for reproducing its comparison metrics:

heretic --model google/gemma-3-12b-it --evaluate-model p-e-w/gemma-3-12b-it-heretic

This evaluates the named modified model against the named original model. The repository warns that exact values can depend on platform and hardware. Its published table was produced with PyTorch 2.8 on an RTX 5090, so matching those values on an Apple-silicon Mac is not guaranteed.

8. Optional repository-and-uv route

The README says experienced `uv` users may clone the repository and run:

uv run heretic

It states that the included `uv.lock` pins package versions to match the developers’ environment. However, the supplied files do not provide a complete clone command, an Apple-silicon `uv` installation command, or a model argument for that invocation. For that reason, the published-package route above is the complete source-supported installation path in this guide.

Responsible operation and troubleshooting

Heretic intentionally alters model safety alignment. Modified models can produce content the original model would refuse. Evaluate outputs carefully, keep human review in place, and follow applicable laws, platform terms, and organizational policies. Automated refusal and KL-divergence measurements are not proof that a modified model is safe, accurate, or equally capable; the README itself says automated benchmarks are no substitute for human evaluation.

For troubleshooting, first rerun:

heretic --help

Then compare your model architecture and PyTorch version with the documented requirements. Use the GitHub issue tracker for unresolved installation or Apple-silicon compatibility problems. Avoid undocumented shell fixes, NVIDIA-only assumptions, and random dependency substitutions, because the supplied first-party files do not validate them for macOS Tahoe.