Deploy PMC-LLaMA 13B Clinical LLM [Install Guide]

General-purpose LLMs invent drug doses and skate past specialty jargon. PMC-LLaMA 13B was trained the other way: knowledge injection on 4.8M biomedical papers plus ~30K textbooks (MedC-K), then instruction tuning on MedC-I (~202M tokens). Open weights. Your GPU. No PHI leaving the building.

This guide targets the instruction-tuned checkpoint the repo still points at: axiong/PMC_LLaMA_13B (HF release dated 2023-09-01). No fancy Git tags – just that model id.

Think of the card like a clinic cabinet. If the cabinet is 24 GB, full FP16 13B weights (~26 GB by ordinary LLaMA-13B math) simply do not fit. You either quantize, offload, or pick a smaller sibling. That single constraint drives every install choice below.

System Requirements for PMC-LLaMA

Training (paper) burned racks of A100s. Inference is lighter – but not “laptop light.” Context was trained at 2048 tokens; longer threads get chopped. 7B checkpoints exist when silicon is tight.

Official Download Source

Scripts and the version table live on the official GitHub repo. Weights:

• Primary (instruction-tuned): axiong/PMC_LLaMA_13B – HF model card
• Pretrain-only: chaoyi-wu/MedLLaMA_13B
• Smaller: chaoyi-wu/PMC_LLAMA_7B (and a 10-epoch variant)

Background paper: arXiv:2304.14454 (USMLE 56.36 / MedMCQA 56.04 / PubMedQA 77.9 on the reported 13B setup). HF lists openrail; base is original LLaMA-1, so treat use as research/non-commercial until counsel signs off. No official Docker. No versioned GitHub releases beyond the model names – as of the repo README.

Step-by-Step Installation

git clone https://github.com/chaoyi-wu/PMC-LLaMA.git
cd PMC-LLaMA

conda create -n pmc-llama python=3.9 -y
conda activate pmc-llama

conda install pytorch==1.13.0 torchvision==0.14.0 torchaudio==0.13.0 pytorch-cuda=11.6 -c pytorch -c nvidia
pip install transformers==4.28.1 sentencepiece datasets
# Optional modern loaders:
pip install accelerate bitsandbytes

Those pins are what the authors published (README Environment). Newer torch + CUDA 12 stacks often load LlamaTokenizer / LlamaForCausalLM fine – until a shape or tokenizer mismatch shows up. Start pinned; loosen only after a clean generate.

First pull is automatic on from_pretrained, or: pip install huggingface_hub then huggingface-cli download axiong/PMC_LLaMA_13B.

Community GGUF (llama.cpp / Ollama) and AWQ/GPTQ builds cut VRAM a lot. Official path stays Transformers.

First-Time Configuration & Prompt

Use the Alpaca-style scaffold from the README. Save as run_pmc.py:

import transformers
import torch

tokenizer = transformers.LlamaTokenizer.from_pretrained('axiong/PMC_LLaMA_13B')
model = transformers.LlamaForCausalLM.from_pretrained(
 'axiong/PMC_LLaMA_13B',
 torch_dtype=torch.float16,
 device_map="auto" # or model.cuda() on one large GPU
)

prompt_input = (
 'Below is an instruction that describes a task, paired with an input that provides further context.'
 'Write a response that appropriately completes the request.nn'
 '### Instruction:n{instruction}nn### Input:n{input}nn### Response:'
)

example = {
 "instruction": "You're a doctor, kindly address the medical queries according to the patient's account. Answer with the best option directly.",
 "input": (
 "###Question: A 23-year-old pregnant woman at 22 weeks gestation presents with burning upon urination. "
 "She states it started 1 day ago and has been worsening despite drinking more water and taking cranberry extract. "
 "She otherwise feels well and is followed by a doctor for her pregnancy. "
 "Her temperature is 97.7°F (36.5°C), blood pressure is 122/77 mmHg, pulse is 80/min, respirations are 19/min, and oxygen saturation is 98% on room air."
 "Physical exam is notable for an absence of costovertebral angle tenderness and a gravid uterus. "
 "Which of the following is the best treatment for this patient?"
 "###Options: A. Ampicillin B. Ceftriaxone C. Doxycycline D. Nitrofurantoin"
 )
}
input_str = [prompt_input.format_map(example)]
model_inputs = tokenizer(input_str, return_tensors='pt', padding=True)

with torch.no_grad():
 topk_output = model.generate(
 model_inputs.input_ids.to(model.device),
 max_new_tokens=200,
 top_k=50
 )
print(tokenizer.batch_decode(topk_output)[0])

Keep the full ### Instruction / ### Input / ### Response shell. Strip it and the model freestyles – or echoes the question. That echo pattern is what GitHub #16 described on the 7B when special_tokens_map looked empty.

Tight VRAM? load_in_8bit=True (bitsandbytes) or a community 4-bit AWQ/GGUF build. Q4-class ports commonly land near ~7-10 GB.

Verify the Install Works

Run the script. You want a lettered choice plus a short rationale – not a Xerox of the prompt. Watch nvidia-smi during load; if the process dies at peak resident memory, you are in OOM territory (fix16 path on a 24 GB card without offload is the usual culprit).

Sanity one-liner: python -c "from transformers import LlamaTokenizer; print(LlamaTokenizer.from_pretrained('axiong/PMC_LLaMA_13B'))" – should print without tokenizer-map drama. Repo simple_test.py works the same checkpoint if present.

Honest question while it loads: do you actually trust a local 2023 LLaMA-1 derivative more than a managed API – or do you just need the bits on-prem for policy? That answer decides whether you stop at a CLI script or wrap FastAPI later.

Common Install Errors and Fixes

• OOM / CUDA out of memory – Expected on ≤24 GB at FP16. Fix: device_map="auto" + load_in_8bit=True, or GGUF + llama.cpp. Lower max_new_tokens.
• Generation repeats the prompt / empty output – Seen on 7B (issue #16) with empty special-token maps. Prefer the instruction-tuned 13B, match README add_special_tokens behavior, set pad/eos explicitly if the tokenizer is blank.
• Torch/transformers clash or HF download hangs – Revert to 1.13 + 4.28.1 first. Open issue #38 still questions whether every 13B HF tree is complete; retry via huggingface-cli (token if rate-limited) and confirm files under ~/.cache/huggingface.
• Import / config surprises on new stacks – Architecture is classic LLaMA. Newer transformers usually load it; pin if config.json fights you.

The catch is length: blow past the 2048 training window and you get truncated junk. Cap inputs there.

Upgrade, Alternatives, Uninstall

No changelog conveyor belt – only the model table on GitHub. Same lab’s later lines (MMedLM / MMed-Llama-3-8B, 2024) live in linked repos if you want multilingual Llama-3 bases. Swapping 7B → 13B is an id change plus enough free VRAM.

conda deactivate
conda env remove -n pmc-llama
rm -rf ~/.cache/huggingface/hub/models--axiong--PMC_LLaMA_13B
# optional: rm -rf PMC-LLaMA

That drops the multi-GB weight cache. Delete community quant copies the same way.

Next: run the README medical-QA fixture on your GPU, check the letter against clinical judgment, then – if you need a service – wrap generate in a tiny FastAPI or Gradio app. Human in the loop stays non-negotiable.

FAQ

Does PMC-LLaMA need internet after the first download?

No. Cache on disk → fully offline.

Can I run it on a 12 GB consumer card?

Yes – if you accept 4-bit community ports (AWQ/GPTQ/GGUF) or heavy CPU offload. Tokens/sec drop; quality can soften. Try the 7B checkpoint first on limited silicon; the official FP16 13B path wants more headroom than 12 GB.

Is this safe for real patient decisions?

No – and the authors never sold it that way. It is a literature-trained research model. It can hallucinate, miss contraindications, and mirror dataset bias. Fine for lit-style QA, draft summaries, or teaching under clinician review. On-prem only, redact PHI, log prompts/outputs. Production diagnosis is out of scope.