How to Install PrivateGPT 1.0: Private ChatGPT for Documents

The most common question people ask before deploying a private ChatGPT for documents: Do I really need to clone a giant repo, install Poetry, download models, and pray the wheel builds? As of the PrivateGPT 1.0 release, the answer is finally no. There’s now a package install path that skips most of that pain – if you know which one to pick.

This tutorial walks through deploying PrivateGPT against a local Ollama server so you can query private documents with no data leaving your machine. We’ll use the newer package method as the primary path and cover the source-clone Docker route as a fallback.

What PrivateGPT 1.0 actually is (and isn’t)

Every older tutorial describes PrivateGPT as “a local chatbot for your PDFs.” That was true in 2023. It’s not accurate anymore. PrivateGPT 1.0 was rebuilt from the ground up as a proper API layer for private AI applications – the question it answers is how do you build a useful application on top of a local model, not how do you run the model itself.

Think of it this way: Ollama is the engine. PrivateGPT is the car. Per the official introduction, it sits above any OpenAI-compatible model server and adds file ingestion, retrieval with citations, tool use, database querying, tabular analysis, MCP support, and code execution on top. You still need something to serve the model – PrivateGPT is the layer that makes that model useful for documents.

If you came here looking for a Gradio chat UI with a “source_documents” folder, that’s the legacy 0.6.x flow. Still works, still supported, but a different mental model entirely.

System requirements before you start

PrivateGPT is picky about its Python version – this is the #1 install killer.

As of mid-2026, the official package install docs pin Python 3.11 exactly – 3.10 and 3.12+ are not supported. This isn’t a soft recommendation. Try 3.12 and Poetry produces a wall of dependency resolution errors that never mention Python at all. Use pyenv or conda to pin the version before anything else.

Install PrivateGPT via the package method (recommended)

This is the path most tutorials still miss. Turns out the wheel index at wheels.privategpt.dev ships pre-built binaries – meaning you skip the C++ compilation that trips up most source installs. The package method is designed for developers running PrivateGPT against an existing LLM server (Ollama, LM Studio, etc.) without cloning the repository.

First, install Ollama and pull a model. Assuming you already have Ollama running:

ollama pull llama3.1
ollama pull nomic-embed-text
ollama serve

Then install PrivateGPT itself. Use uv – it handles the private wheel index cleanly:

uv venv --python 3.11
source .venv/bin/activate

uv pip install private-gpt 
 --index-url https://wheels.privategpt.dev/packages/ 
 --extra-index-url https://pypi.org/simple/

export OPENAI_API_BASE=http://localhost:11434/v1
private-gpt

Set OPENAI_API_BASE to your LLM server’s endpoint and start – PrivateGPT binds to port 8080 by default (per the package install docs). Open http://localhost:8080 and you should see the workbench.

Port gotcha: If you’re mixing tutorials, watch the port number. The package install serves on 8080. A source install started with make run serves on 8001. Guides written for one method will silently send you to the wrong URL.

Alternative: Docker Compose from source

Full Gradio UI plus file ingestion pipeline in one shot – that’s the Docker path. The 0.6.2 release focused specifically on Docker enhancements, and the compose file ships three profiles: ollama-cpu, ollama-cuda, and ollama-api.

git clone https://github.com/zylon-ai/private-gpt
cd private-gpt

# Pick a profile that matches your hardware
docker compose --profile ollama-cpu up
# or
docker compose --profile ollama-cuda up

Use ollama-api if Ollama is already running on your host – Docker won’t spin up its own instance. The compose file pulls pre-built images from a remote registry by default; local builds are optional.

Verify it works

Two quick checks. The health endpoint first:

curl http://localhost:8080/health

Then hit the completions API to confirm the model is reachable:

curl http://localhost:8080/v1/completions 
 -H "Content-Type: application/json" 
 -d '{"prompt": "Say hello", "max_tokens": 20}'

Both responding means the stack is wired. The real test is ingesting an actual document – upload something via the workbench UI and ask a question about its contents. The response should cite the source file by name. No citation means RAG isn’t working, which usually points to the embeddings model not being pulled (the nomic-embed-text step above is easy to skip).

The install errors that actually happen

Most “PrivateGPT won’t install” threads on GitHub trace back to three problems. Skip ahead to whichever matches your error message.

Failed building wheel for llama-cpp-python

The most reported blocker, showing up across GitHub issues #1488, #1511, and #1584 on WSL, Ubuntu, and Windows alike. The root cause is almost always a missing or mismatched C++ toolchain – not PrivateGPT itself. The package install method avoids this entirely because the wheels arrive pre-compiled.

For source installs: Linux and WSL need build essentials first (sudo apt-get install -y build-essential cmake python3.11-dev), then retry. Windows without WSL needs Visual Studio Build Tools with the C++ workload selected. macOS needs xcode-select --install. Get the compiler sorted before touching Poetry.

Poetry resolver hangs or errors on Python version

If Poetry spends 10 minutes solving dependencies and then errors out, check python --version. Anything other than 3.11.x will fail here. The fix:

pyenv install 3.11.9
pyenv local 3.11.9
poetry env use 3.11.9
poetry install --extras "ui llms-ollama embeddings-ollama vector-stores-qdrant"

Ollama container can’t be reached

Running Ollama in a separate container is a documented pain point. The symptom: setting api_base to http://ollama:11434 returns nothing, with an [Errno 99] Cannot assign requested address warning in the logs. As of this writing (July 2026), no official fix exists in the docs. The reliable workaround: use the ollama-api Docker profile and point it at Ollama on the host via host.docker.internal:11434.

When PrivateGPT is the wrong tool

Every tutorial pitches this as the obvious choice for private document AI. It’s often not.

• Single-user, single-file chat – Open WebUI with Ollama is simpler and gives you a chat UI in five minutes. PrivateGPT’s API layer is overkill.
• You just want a chatbot – LM Studio has one built in. No FastAPI backend needed.
• You need enterprise SSO and audit logs – PrivateGPT is open source; the commercial Zylon product exists precisely because those features aren’t in the OSS codebase.

PrivateGPT makes sense when you’re building an application on top of local models – a company knowledge base, a document Q&A service for a regulated industry, a custom RAG pipeline you’ll extend with your own tools. If “chat with my PDFs” is the whole use case, you can do it with less setup.

Upgrading from 0.6.x and clean uninstall

If you’re on 0.6.x, the upgrade to 1.0 isn’t drop-in. The API surface changed. Configuration moved. The safest path is a fresh install into a new venv, then migrate your settings.yaml profile by hand – most keys still exist but some paths shifted.

To uninstall the package version:

uv pip uninstall private-gpt
rm -rf .venv local_data/

For a Docker install, docker compose down -v removes the containers and volumes. Ollama models sit in ~/.ollama and survive – delete them separately if you want a truly clean slate.

FAQ

Can PrivateGPT run fully offline?

Yes, once installed. The install itself needs internet to pull wheels and models. After that, disconnect and it keeps working.

Do I have to use Ollama, or can I plug in something else?

Anything OpenAI-compatible works – LM Studio, LocalAI, vLLM, llama.cpp’s server mode. Set OPENAI_API_BASE to the endpoint and PrivateGPT treats it identically. This flexibility is the whole point of the 1.0 rewrite: the inference layer is swappable, so you can swap Ollama for a hosted OpenAI-compatible endpoint and the rest of the stack (ingestion, retrieval, citations) stays untouched.

Why does the community keep linking to the imartinez/privateGPT repo?

Older tutorials cite the original author’s URL. GitHub forwards it automatically now – the project moved under the zylon-ai organization when the founders formed a company around it. Clone from github.com/zylon-ai/private-gpt. The redirect works, but some forks still carry a stale README.

Run curl http://localhost:8080/health and, once you get the OK, upload your first real document via the workbench. That’s the moment the whole stack proves itself.