Deploy Lawma 8B: Legal LLM Install Guide (2026)

Most “legal LLM” articles are surveys – long lists of Harvey, Legal-BERT, SaulLM, plus a warning about hallucinations. Useful once. Then what? If you actually want to run one locally on your own case files, you need install commands, not another comparison table.

This tutorial deploys Lawma, an open-weights legal LLM from the Social Foundations of Computation group. It’s a fine-tuned Llama 3 model that, per the ICLR 2025 paper, beats GPT-4 on 95% of 260 legal classification tasks by an average of 17 accuracy points. That’s a real, reproducible number – the weights are on Hugging Face and the eval methodology is in the paper.

What Lawma actually is (and isn’t)

Lawma comes in two sizes: Lawma 8B (fine-tune of Llama 3 8B Instruct) and Lawma 70B (fine-tune of Llama 3 70B Instruct). Both were trained on 500k task examples totalling 2B tokens, drawn from the Supreme Court and Songer Court of Appeals databases.

Here’s the important part nobody in the “top 10 legal LLMs” listicles will tell you: the model was fine-tuned only on multiple-choice questions, not on general instructions. Paste in “draft me a motion to dismiss” and Lawma will produce text – but it wasn’t trained for that. It was trained to answer classification prompts in MMLU format. Feed it the right prompt shape and it’s excellent. Feed it the wrong shape and it fails silently, no error, just unreliable output.

If your use case isn’t multiple-choice legal annotation, the authors recommend fine-tuning Lawma further on your specific task. The paper buried an interesting finding: just 50 labeled examples matched GPT-4 zero-shot on 6 of 10 highlighted tasks. 250 examples covered 8 of 10. At 1,000 examples, it matched or beat GPT-4 on all 10.

System requirements

These are practical numbers for inference, not the 7×H100 training rig (the paper reports 600 H100 hours for 8B, around 1,600 for 70B – cloud budget territory). The table below uses estimated figures based on weight size plus EleutherAI’s transformer math rule of up to 20% inference overhead; CUDA and Python versions reflect community recommendations as of mid-2026 and may shift.

The VRAM math: an 8B model in fp16 is ~16 GB of weights. Add up to 20% for KV cache and activations, round up, and a 24 GB consumer card handles it comfortably. The 70B estimate of ~140 GB is derived from the same formula – two A100 80GB cards get you to 160 GB total, which covers it.

There’s a quieter reason open-weights legal LLMs like Lawma matter that rarely comes up in the commercial-tool discussions: your case files don’t leave your infrastructure. Sending privileged documents to a third-party API is a conflict-of-interest question as much as a technical one. Running Lawma locally sidesteps it entirely. Whether that trade-off is worth the hardware cost is a question every team answers differently – but at least it’s the team’s question to answer, not the vendor’s.

Pick your serving engine first

Before touching the download, decide how you’ll serve it.

• vLLM – production-grade. Turns out a 2024 comparative analysis found up to 24× higher throughput than TGI under high-concurrency workloads, driven by PagedAttention. Recommended default.
• Transformers (raw) – fine for one-off scripts and evaluation. Slow at scale.
• TGI – skip it. The repo is archived as of March 2026 and in maintenance mode. Older Lawma tutorials still recommend it; they’re stale.
• Ollama / llama.cpp – works if someone has published a GGUF quant. Community quants of Lawma exist as of this writing, but none are official.

Install: the actual commands

Start with a clean Python 3.11 environment. Mixing CUDA versions across projects is where most “CUDA out of memory” errors that aren’t really out-of-memory errors come from.

# 1. Clean environment
conda create -n lawma python=3.11 -y
conda activate lawma

# 2. Install PyTorch matching your CUDA version
pip install torch --index-url https://download.pytorch.org/whl/cu121

# 3. Install vLLM (pulls in transformers, tokenizers, etc.)
pip install vllm

# 4. Log in to Hugging Face (Lawma is public but caching helps)
pip install huggingface_hub
huggingface-cli login

# 5. Serve Lawma 8B
vllm serve ricdomolm/lawma-8b 
 --dtype float16 
 --max-model-len 4096 
 --port 8000

First launch downloads about 16 GB from Hugging Face. Grab coffee. Once cached under ~/.cache/huggingface, subsequent starts take under a minute.

Verify it works – with the RIGHT prompt shape

Tutorials that copy-paste from generic Llama 3 guides get this wrong. Testing Lawma with a chatty prompt like “summarize this contract” makes it look broken. Test it the way it was trained – a multiple-choice legal question:

curl http://localhost:8000/v1/completions 
 -H "Content-Type: application/json" 
 -d '{
 "model": "ricdomolm/lawma-8b",
 "prompt": "Question: In the Supreme Court case, which party won?nA. PetitionernB. RespondentnC. NeithernAnswer:",
 "max_tokens": 5,
 "temperature": 0.0
 }'

A single letter back (A, B, or C) – that’s what “fine-tuned on multiple-choice” means in practice. Get a paragraph of prose instead? Either your prompt format is off or you accidentally loaded base Llama 3 without the instruct adapter.

Common errors and what actually fixes them

“CUDA out of memory” on a 24 GB card. vLLM pre-allocates KV cache aggressively by default. Add --gpu-memory-utilization 0.85 to the serve command, and drop --max-model-len to 2048. Most legal classification prompts are short – you won’t miss the extra context.

The gibberish problem. Generic English output when you expected a letter answer almost always means you’re hitting the wrong endpoint. Lawma expects raw MMLU-style prompts. Switch from /chat/completions to /completions and strip any chat template wrapping. That single endpoint change fixes it in the majority of cases.

Axolotl install fails during fine-tuning setup. A well-known community pain point – Superteams’ guide notes axolotl’s install steps aren’t clearly defined and an A100/Ampere GPU is required. On a T4 or older card, use LLaMA-Factory instead.

Slow Hugging Face download. Set HF_HUB_ENABLE_HF_TRANSFER=1 and install hf_transfer. Doubles or triples throughput on fast connections.

Fine-tune on your own tasks

Specialization wins – that’s the paper’s central argument. Lawma 8B is a starting point. The real gains come from fine-tuning on your classification task with your labeled data. The few-shot data point above (50 examples → GPT-4-level on 6 tasks) comes from the arXiv paper’s few-shot analysis; the fine-tuning setup uses axolotl via the official repo.

git clone https://github.com/socialfoundations/lawma.git
cd lawma/fine-tune
# Edit config.yml to point at your dataset
# datasets:
# - path: your-username/your-legal-dataset
# type: alpaca
axolotl train config.yml

One heads-up: as of Aug 2026, the socialfoundations/lawma repo has no formal releases – no version tags, no release artifacts. You’re pulling from main. Pin the commit SHA in your Dockerfile now, not after something breaks in production.

How many labeled examples are actually enough for a genuinely novel legal classification task – one that shares no terminology or case-type with the Supreme Court training data? The paper doesn’t fully answer that. It’s an honest gap. If you’re working in a specialized area like patent litigation or environmental enforcement, budget time for your own eval before committing to a fine-tuning run.

Upgrade and uninstall

No version number to track. You’re following a research repo – when new commits land, git pull and rerun. For the weights, Hugging Face auto-caches by revision; delete ~/.cache/huggingface/hub/models--ricdomolm--lawma-8b to free the ~16 GB.

Full cleanup:

conda deactivate
conda env remove -n lawma
rm -rf ~/.cache/huggingface/hub/models--ricdomolm--lawma-*
rm -rf ~/lawma # the cloned repo

FAQ

Can I use Lawma for legal research outside the US?

Probably not well. Training data is US Supreme Court and Court of Appeals cases – civil-law terminology and procedural categories won’t transfer cleanly. Fine-tune on your jurisdiction’s data first.

Why not just use Lawma 70B if I have the hardware?

You can, but weigh it carefully. The 70B needs roughly 140 GB VRAM just to load (estimated: two A100 80GB cards), and per the README, the accuracy gap over 8B is small on average. A concrete example: a batch job over 100k legal documents will finish faster and much cheaper on 8B running a single H100 than on 70B spread across a multi-GPU node. Reach for 70B only when your eval set shows a meaningful gap on a specific task – not by default.

Is Lawma safe to use for actual legal work?

The ICLR 2025 paper explicitly argues that zero-shot GPT-4 is not sufficient for real legal work – and by extension, no LLM replaces a lawyer’s judgment. Lawma is a research tool: legal annotation, empirical legal studies, document classification, coding schemes. Don’t generate advice for clients with it.

Next step: spin up the vLLM server with the exact command above, hit the /completions endpoint with one of your own legal classification prompts formatted MMLU-style, and check the answer against your ground-truth label. If it’s right on the first few, start collecting a fine-tuning set of 200-500 examples for your specific task – that’s where Lawma actually pays off.