QWE AI Academy Two paths. Call a closed API – ChatGPT, Claude, Gemini – pay per token, prompts leave your machine. Or download open weights, run on hardware you control, keep data local. Private docs, regulated text, steady high volume? Local usually wins on control and long-run cost. One-off questions? The API is still less hassle.
You’re a developer or small team watching API invoices climb and privacy reviews pile up. You want a solid LLM for internal summaries or coding help, and customer text cannot leave the building. That tension is exactly why open weights exist: the trained parameters, on your disk, under your process.
Practical Setup: First Local Model in Minutes
Skip the philosophy for a minute. A modern laptop with 16 GB RAM – or a consumer NVIDIA card – already runs useful 7B-20B quantized models.
- Install Ollama (one-line script on macOS/Linux, installer on Windows). It pulls GGUF quantizations and serves an OpenAI-compatible API on localhost:11434.
- Pull a starter that fits your RAM: a 7B-8B or 20B-class tag. Library names change; check the page before you script anything.
- Chat with
ollama run <model>, or point any OpenAI SDK athttp://localhost:11434/v1.
curl -X POST http://localhost:11434/v1/chat/completions
-H "Content-Type: application/json"
-d '{"model":"your-model-tag","messages":[{"role":"user","content":"Summarize this in one sentence."}]}'
Want a GUI? LM Studio browses Hugging Face, shows estimated VRAM per quantization, and can flip on a local server. Same weight files, different shell.
Pro tip: Before any commercial deploy, open the model’s Hugging Face page and read the LICENSE file – not the marketing blurb. Apache 2.0 and MIT are usually clean. Custom “community” licenses often aren’t.
What You Actually Downloaded
In a neural net, weights and biases are the learned numbers that map inputs to outputs. An open-weights drop publishes those final numbers (plus, usually, architecture config and enough inference code to load them). Fine-tune and redistribute rights sit in the license, not in the file format.
Open weights stop at the parameters. Full Open Source AI – OSAID v1.0 (October 2024) from the OSI – also wants training code and enough data information that a skilled person could rebuild a substantially equivalent system (OSI open-weights note). Llama, many Qwen and DeepSeek releases, and OpenAI’s gpt-oss sit in the open-weight-only bucket.
| What you get | Closed API | Open weights | Open Source AI (OSAID) |
|---|---|---|---|
| Download & run locally | No | Yes | Yes |
| Fine-tune on private data | Limited | Yes | Yes |
| Training code + data detail | No | Usually no | Yes |
| Typical license | Vendor ToS | Apache/MIT or custom | OSI-approved |
Hugging Face is still where almost every release lands within hours. Model cards list intended use, evals, and the license file that actually binds you.
License Filters, Format Lock-In, Serving
Base model runs. Next fork in the road: product constraints, not model quality.
Apache 2.0 / MIT (many Qwen builds, recent DeepSeek flagships, gpt-oss): commercial use, modification, redistribution with almost no paperwork. Build on those first if you sell software.
Llama Community License is the trap teams miss. Free commercial use for most orgs – as of the terms commonly attached to current Llama checkpoints – only while you stay under roughly 700 million monthly active users. Cross that line and you need a separate Meta agreement Meta can refuse. Some other drops add revenue caps, field-of-use bans, or distillation clauses that forbid training new models on the outputs. Read the file for the exact tag you pulled; versions differ.
OpenAI shipped gpt-oss-20b and gpt-oss-120b as open weights under Apache 2.0 (2025 release notes / OpenAI Help Center). Not on the OpenAI API. The 20B class is aimed at ~16GB-class boxes; 120B is a single-80GB-GPU story per their claims. The model card pushes a preferred “harmony” response format. Wrong chat template → quality falls off a cliff. These are not drop-in Llama-template clones.
Fine-tunes: LoRA/QLoRA on your domain data stays on your side (Unsloth, Hugging Face PEFT, same checkpoints). Multi-user serving: vLLM or llama.cpp server usually beat default Ollama throughput once you leave single-dev chat.
As of mid-to-late 2026, multi-trillion-parameter open-weight frontier models have mostly come from Chinese labs (e.g. Kimi-class releases called out on public trackers); Western open-weight names still in heavy use include Mistral, Nvidia Nemotron, and gpt-oss. Parameter counts and licenses move – re-check the card the week you ship.
Honest Limitations
You own the ops. GPU hours, quant choice, safety filters, monitoring. Closed APIs hide that mess behind a price tag.
You still cannot fully audit how the model was built. No training-data composition, no training code → bias and contamination questions stay soft. That gap is the core OSI complaint, not a blog nitpick.
Weights, once out, stay out. An API can deprecate a checkpoint overnight. Your downloaded copy cannot be remotely revoked. Community forks strip refusals in a weekend. If misuse of a capable base model is in your threat model, that permanence is the feature and the bug.
Quantization is not free quality. Q4_K_M is the default people actually ship for size versus quality. Code and exact structured extraction often want Q5/Q8 when VRAM allows. And the “fits in 24 GB” line on a model card is almost always weights-only. Runtime = weights + KV cache (grows with context) + overhead. Long context on a card that only fit the weights → OOM. Measure with your real context length, not the marketing footprint.
FAQ
Are open weights the same as open source AI?
No. Parameters only versus parameters plus the preferred form to rebuild (code + data information under OSAID). Headline “open” LLMs are usually the first kind. See the comparison table above.
Can I use Llama commercially?
Most teams: yes, under the Llama Community License, while monthly active users stay under the cap (commonly cited around 700M on current terms). Picture a consumer app that suddenly crosses that MAU line after a viral year – you do not get a grace button in the weights file; you need Meta’s separate deal. Always read the LICENSE on the exact checkpoint. Not legal advice.
How much hardware do I need to start?
16 GB RAM laptop or 8-12 GB VRAM: many 7B-8B Q4 models and some 20B-class quants for chat and light coding. 70B-class dense at usable quality: plan on ~40 GB+ VRAM, multi-GPU, or heavy CPU offload. Start tiny. Measure tokens/second and quality on your real prompts, then scale. MoE models activate fewer parameters per token – they still need the full weight set resident, so disk and load RAM do not magically shrink.
Next step is boring and decisive: install Ollama, pull one small open-weight model that fits, run the same prompt you currently send to a closed API. Latency, quality, data location. That single run beats another definition page.