Deploy FinGPT (Open Source Finance LLM): A 2025 Install Guide

The number-one mistake people make with FinGPT? Treating it like a normal app you install. It isn’t. There’s no fingpt binary, no Docker image on Docker Hub, no version like v2.1.0 to pin. The GitHub releases page is empty – just auto-generated source snapshots, no semver tags at all.

So when you read “install the latest FinGPT version,” what that actually means is: clone the master branch of the research repo, then pull a specific LoRA adapter from HuggingFace and merge it onto a base model you download separately. That’s the real deployment path for this open source finance LLM, and once you reverse-engineer it that way, everything else clicks.

What you’re actually deploying

Several mostly-independent components live in the AI4Finance Foundation repo – sentiment analysis, a forecaster, RAG, a benchmark suite – each in its own subdirectory with its own scripts. The v3 sentiment models sit in fingpt/FinGPT_Sentiment_Analysis_v3 and use Llama-2 as base with LoRA fine-tuning; the forecaster lives in fingpt/FinGPT_Forecaster and predicts stock price movements from market news. The original research is arXiv:2306.06031 by Yang, Liu, Wang – worth skimming to understand what the fine-tuning is actually optimizing for.

For this guide I’m targeting the most common deployment: running FinGPT v3 for financial sentiment inference on a single GPU. v3.3 uses Llama2-13B as the base, v3.2 uses Llama2-7B, and v3.1 uses ChatGLM2-6B – all explicitly built to be trainable and inferable on a single RTX 3090. Pick v3.2 if you want headroom; pick v3.3 if you want maximum quality and have 24 GB of VRAM exactly available.

System requirements (honest version)

The repo doesn’t publish official minimum specs, so these come from cross-referencing the base-model footprints (as of 2024 benchmarks) with what users report actually works.

Why those GPU numbers? LLaMA-7B in FP16 occupies about 12.3 GB of VRAM for inference, and LLaMA-13B in FP16 needs roughly 24 GB – right at the limit of a 24 GB card (per the bacloud GPU requirements guide, 2024). The LoRA adapter itself adds almost nothing, but the KV cache and Python overhead eat the remaining margin. Load Llama2-13B at FP16 on a 24 GB card with device_map="auto" and PEFT will silently offload some layers to CPU – and that’s where the first big error comes from.

That silent offload is worth pausing on. The model appears to load fine. No exception fires. Then inference crashes. It’s the kind of failure that reads like a bug in your code when it’s actually a resource-planning problem – and it’s bitten enough people that it has its own GitHub issue thread. More on that in the errors section.

Step-by-step install (the v3.2 forecaster path)

Using v3.2 here because it fits cleanly on a single 24 GB card without quantization tricks. Same recipe works for v3.3, just swap the adapter name.

1. Clone the repo and set up the environment

# clone master - there are no release tags to choose from
git clone https://github.com/AI4Finance-Foundation/FinGPT.git
cd FinGPT

# isolated env
conda create -n fingpt python=3.10 -y
conda activate fingpt

2. Install dependencies manually

The top-level setup.py is mostly a placeholder – it declares package metadata but not a working CLI. So pip install . from root won’t get you anywhere useful. Install the actual ML stack yourself:

pip install torch==2.1.2 --index-url https://download.pytorch.org/whl/cu121
pip install transformers==4.36.2 peft==0.7.1 accelerate==0.25.0
pip install datasets sentencepiece bitsandbytes scipy

Pinning peft==0.7.1 matters. Newer PEFT versions changed how adapter offloading works, which interacts badly with the v3-era adapter checkpoints. Blindly run pip install peft and you’ll likely hit the meta-tensor warning from issue #186.

3. Pull the base model and the LoRA adapter

Llama-2 weights require a gated HuggingFace access request. Once approved:

huggingface-cli login
# then in Python:
from huggingface_hub import snapshot_download
snapshot_download("meta-llama/Llama-2-7b-chat-hf")
snapshot_download("FinGPT/fingpt-forecaster_dow30_llama2-7b_lora")

Turns out all the trained adapters live under the FinGPT organization on HuggingFace – that’s the project’s official home for checkpoints and datasets, not the GitHub repo itself.

First-time configuration and verification

Here’s a minimal inference script. Save as test_fingpt.py at the repo root:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

BASE = "meta-llama/Llama-2-7b-chat-hf"
ADAPTER = "FinGPT/fingpt-forecaster_dow30_llama2-7b_lora"

tokenizer = AutoTokenizer.from_pretrained(BASE)
base = AutoModelForCausalLM.from_pretrained(
 BASE,
 torch_dtype=torch.float16,
 device_map="cuda:0", # NOT "auto" - see Common errors
)
model = PeftModel.from_pretrained(base, ADAPTER)
model.eval()

prompt = "Summarize the sentiment of: 'Apple beat Q3 earnings estimates on iPhone strength.'"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda:0")
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Run it: python test_fingpt.py. If you see a coherent sentiment paragraph – not gibberish, not a stack trace – the deployment works. Latency varies by hardware and quantization settings; expect it to be slower than a hosted API and faster than you’d get from a CPU-only setup.

Common errors and real fixes

These two errors account for most of the open issues on the repo. Both come from real bug reports, not guesses.

Error 1: PeftModel meta-tensor offload crash

Symptom: a warning about “copying from a non-meta parameter in the checkpoint to a meta parameter”, followed by a traceback in peft_model.py at _update_offload. Docs say use device_map="auto" for convenience, but GitHub issue #186 (July 17, 2024) shows exactly why that backfires: loading fingpt-forecaster_dow30_llama2-7b_lora via PeftModel.from_pretrained crashes inside _update_offload at line 908 when VRAM is tight enough to trigger CPU offload.

Fix: if your model fits in one GPU, use device_map="cuda:0" explicitly. If it doesn’t fit, load the base in 4-bit with BitsAndBytesConfig(load_in_4bit=True) before attaching the adapter – that avoids the offload path entirely.

Error 2: train_lora.py AttributeError on strip()

Issue #160 (Feb 8, 2024) nails it: run prepare_data.ipynb then train_lora.py and you get AttributeError: 'NoneType' object has no attribute 'strip'. The script assumes every row has a non-null output field. The data prep notebook doesn’t enforce that.

Fix: before training, filter the JSON dataset:

import json
rows = [json.loads(l) for l in open("data/dataset_new/train.jsonl")]
clean = [r for r in rows if r.get("output") and r["output"].strip()]
with open("data/dataset_new/train.jsonl", "w") as f:
 for r in clean:
 f.write(json.dumps(r) + "n")

Upgrade and uninstall

“Upgrade” here is a strange concept. There’s no version to upgrade to. What you actually do is:

• Repo updates:git pull in the FinGPT directory. Check the README’s news section before pulling – sometimes a component subdirectory gets refactored.
• Adapter updates: the FinGPT HuggingFace organization occasionally publishes new LoRA checkpoints. Pull the new repo ID – adapters are small, so keeping multiple versions side by side is cheap.
• Uninstall:conda env remove -n fingpt nukes the env. Delete the FinGPT/ directory and the HuggingFace cache at ~/.cache/huggingface/hub/models--FinGPT--* and ~/.cache/huggingface/hub/models--meta-llama--* to reclaim the ~30 GB of weights.

One question I keep coming back to: is FinGPT a product, or a portfolio of research experiments wearing the same label? After deploying it, it’s the second. That’s not a bad thing – it just means treating it like one instead of expecting a polished install flow.

Which raises a broader question that the repo doesn’t answer: what’s the right lifecycle for a research codebase that people are running in production? The GitHub issues show real users hitting real errors on models the authors probably consider “demo quality.” There’s no roadmap, no deprecation policy, no SLA. That’s fine for a research tool – but worth knowing before you build something on top of it.

FAQ

Is FinGPT free for commercial use?

Yes – MIT license, which means commercial use, modification, and redistribution are all permitted. The catch: the base models (Llama-2, ChatGLM2) have their own licenses you must comply with on top of FinGPT’s MIT terms.

Can I run FinGPT without a GPU?

Load the base model in 4-bit with bitsandbytes and CPU offload, then attach the LoRA – it works. One token per second on a modern CPU, though, which makes interactive use painful. If you’re experimenting and don’t have a GPU, Google Colab’s free T4 (16 GB VRAM) with the v3.1 ChatGLM2-6B adapter is the path of least resistance.

How does FinGPT compare to BloombergGPT?

Different philosophies entirely. BloombergGPT is closed, trained on privileged Bloomberg data, and not publicly deployable – there is no install guide for it because there’s nothing to install. FinGPT is the opposite: open, retrain-able on your own corpus, with fine-tuning costs reported at under $300 per run (as of the 2023 paper) versus BloombergGPT’s ~$3M training bill. If reproducibility matters to you, FinGPT wins by default.

Next step: clone the repo, pull FinGPT/fingpt-forecaster_dow30_llama2-7b_lora, and run the verification script above against five real news headlines you care about. If the outputs look reasonable, you have a working baseline – then go look at fingpt/FinGPT_Benchmark/benchmarks/fpb.py to measure it against the FPB sentiment dataset before you trust it on anything live.