QWE AI Academy Local 4B chat models still leak em dashes and neat “not X, but Y” turns after you ban them in the system message. You fix one reply. The next draft grows a fresh tic. A Show HN drop – Qwen3-4B with those habits trained out of the weights – tries a different fix: change the model, not the prompt.
Erik Kannike’s qwen3-4b-deslop-v4 (merged + ~2.5 GB Q4_K_M GGUF) was trained on an M2 MacBook Air from Qwen3-4B-Instruct-2507. As of the author’s held-out set of 40 prompts, a custom slop score moved 121 → 12.5 per 1,000 words, em dashes 20.5 → 0, about 68% of answers landed in the author’s “human range” (<15), task fit sat near 80%, and factual checks held 30/30. No system prompt in train or eval.
We notice the tics because they arrive in packs – dash, triad, tidy contrast – like a drummer who only knows one fill. Once you hear the fill, every kit sounds the same. That irritation is why people chase bans and steering hooks. It is also why a weight-level patch feels different when it works.
Why prompt bans and runtime steering keep slipping
The ban list is familiar: no em dashes, no lists of three, don’t say “look into.” One shot looks clean. Next generation, the assistant register sneaks the pattern back in another costume.
Earlier runtime work from the same author subtracted an “assistant-register” direction while generating and paired it with a plain-style user prompt. Score got near ~15. Cost: hooks on every token, and ordinary human-prose predictions shifted more than the final trained checkpoint does (deslop reports KL drift 0.061 on human prose for the weight-level path).
A tiny circuit edit helped first. Sign-flip four MLP neurons (layer 32 n4852, 33 n348, 28 n1121, 32 n5418) – four down_proj columns, a 175-byte patch – cut “not X, but Y” by roughly 70% with low drift. Still not the whole bundle. Rank-16 LoRA on attention+MLP, two epochs, bare user prompts, then hard filters on the teacher answers, is what baked the cleaner habits into the weights. Final round kept 161 of 288 prompts after those filters.
How to run it
Fast path if Ollama is already installed:
ollama run hf.co/erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_MThe pull ships params the HF card recommends: temperature 0.7, top-p 0.9, repeat penalty ~1.1. Leave system empty. Training used bare user messages only.
llama.cpp instead?
llama-cli -hf erikkannike/qwen3-4b-deslop-v4-GGUF:Q4_K_M --repeat-penalty 1.1 --temp 0.7 --top-p 0.9Full bf16 merged weights: erikkannike/qwen3-4b-deslop-v4. Load like any Qwen3-4B instruct checkpoint:
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "erikkannike/qwen3-4b-deslop-v4"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, torch_dtype="auto")
msgs = [{"role": "user", "content": "Write a short internal note on killing weekly status meetings."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=300, do_sample=True, temperature=0.7)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))Grab the separate 126 MB LoRA? You also need the four-neuron edit. The adapter was trained with that sign flip active; adapter alone will not match the published behavior. Merged HF weights and the GGUF already include both.
Pro tip: keep generations under ~250 words or keep the ~1.1 repetition penalty on. Every training answer was short. Past a few hundred tokens without the penalty, the model can lock onto one sentence shape and repeat it.
Side-by-side: internal note on killing status meetings
Same bare user prompt, temp 0.7, no system message – base Instruct-2507 vs deslop-v4.
Base draft: em dashes, a clean triad of “benefits,” and a closing contrast. Readable. Also unmistakably template-shaped.
Deslop draft: shorter sentences, a stance in the first screen, almost no formula contrasts, zero em dashes. Trade-off hits fast: the voice sits closer to a careful junior writer than a polished columnist. Personal tone can feel diary-flat. Concrete names and places still get invented more than you want – detail was rewarded in the filter rounds – even though invented studies/percentages dropped to roughly 1 in 30 answers on the author’s checks.
Task-fit filters were not optional polish. An intermediate round scored even lower on slop (9.9) and still failed in use: opinion pieces with no stance, fabricated studies, and “, which” joins amplified ~28× from the teacher’s own style prompt. Filters that demanded an early stance word, first-person on personal prompts, email greetings when relevant, a length band, and a ban on invented studies are what made the final checkpoint usable (~80% task fit on blog, LinkedIn-style, explainer, and opinion genres in that eval).
When this model is the wrong tool
Skip code, long multi-turn chat, non-English, and tool use. Those genres were not in the test set; expect parity with base or worse.
Length requests drift. “About 150 words” often lands 180-230.
The slop number is one author’s 26-trait regex style guide. No blind human reader study backs the “68% human range” claim yet – the model card and blog both say so. Treat it as a relative signal against the same scorer, not a community verdict. HN chatter (including notes that the task-fit filter was the practical takeaway) is still anecdotal at scale; see the Show HN thread.
Need lively voice or strong stylistic personality? This is not it. Pair with a human rewrite pass, or wait for the stronger human-essay teacher the author has flagged as a next step.
Is a regex “human range” the right north star, or just the only score a solo MacBook run can afford? Open question. For shipping drafts this week, the better test is still: would you send the raw output after a two-minute skim?
Prompting habits that match how it was trained
- Genre + approximate length in the first user sentence.
- Opinion work: demand a clear stance up front.
- Only add “vary sentence length; avoid joining every clause with which or so” if that tic returns on a long run.
- Repetition penalty ~1.1 for anything past a couple hundred tokens – the Ollama params file already sets this.
- Fact-check names, places, and leftover numbers. Confabulated detail is the remaining tax.
Full training log, neuron notes, and earlier steering posts live in the author write-up if you want to redo the pipeline on a larger base.
FAQ
Does it need a special system prompt?
No. Train and eval used bare user prompts. A heavy style system message usually makes output worse.
Can I fine-tune this further on my own writing?
Yes in principle – standard Qwen3-4B checkpoint plus a small LoRA. Start from the merged weights, keep rank modest, and filter your demos the way the author did: does the answer actually do the job the prompt named? Weak teachers cap weak students. A 7-8B or 30B pass of the same recipe is already on the author’s roadmap.
How does the four-neuron edit actually work?
Four MLP neurons form a small circuit that pushes “not X, but Y” constructions. Flipping their sign negates four columns of the matching down_proj weights. That edit alone removed about 70% of those contrasts with almost no KL movement on human prose. The LoRA then learned the rest of the cleaner style on the patched model. Load adapter and edit together; GGUF and merged HF repos already have both applied.
Pull the GGUF. Generate three short pieces you need this week. Keep whatever passes a quick “would I send this without a heavy rewrite?” check. That beat any leaderboard for real workflow fit.