Ollaya Guide: Run Jev-Style Decision Models Locally

Key takeaway: Need thresholdable yes/no, choice, and score answers over tickets or tool JSON – without shipping that state to a hosted API? Ollaya is the Apache-2.0, Ollama-shaped daemon for open decision checkpoints. Pull by name, serve on 11435, speak TypeSafe’s wire format.

Hosted Jev turned “System One” into a product: state in, typed questions in, probabilities out, no token stream. Fine until privacy review, p95 latency, or per-call cost shows up. As of late September 2026 the project hit HN’s front page (~409 GitHub stars, v0.7.2); people wanted that workflow on 127.0.0.1.

What problem Ollaya solves right now

Prompted classifiers work until they don’t. They’re slow at volume, spend like generation, and the “confidence” is mostly vibes – you can’t set a 0.82 cutover you trust in billing. A decision model does one forward pass and returns structured answers you can actually gate on: refund path, agent allow/deny, churn flag.

Ollaya isn’t affiliated with Ollama or TypeSafe. Same muscle memory (pull, run, serve, Modelfiles), different job: Laya, decider, nli, kev, GGUF ports like jevk5. Homepage numbers (RTX 4090, five questions): laya:multilingual ~8.1 ms median end-to-end, laya:en ~9.6 ms. Hosted Jev third-party medians land ~236-276 ms with network. decider:2b sits nearer ~190 ms – still local, still one pass.

That gap is why teams reach for this on support automation and agent guardrails.

Method A vs Method B: hosted Jev / DIY logprobs vs Ollaya

Method B. Local, free, SDK-shaped. Method A only if you must have Jev’s sealed scores today and legal already cleared outbound state. Most teams reading this can finish Method B before lunch.

Hands-on: install Ollaya and run your first decisions

Official quickstart (as of v0.7.x docs):

# Linux / macOS
curl -fsSL https://ollaya.dev/install.sh | sh

# Windows PowerShell
irm https://ollaya.dev/install.ps1 | iex

Installer checks sha256, tries a Linux service, and only drags CUDA libs if it sees a capable NVIDIA GPU. FAQ platforms note: driver R580+ / CUDA 13. Desktop app exists – it hits the same API and stays on CPU. Docker images: ghcr.io/ollaya-dev/ollaya or :cuda.

Smoke test with a preset

ollaya run laya --preset triage "Tracking says delivered Friday. Box never showed. Order #A-18422. I need a reship before Monday's event."

Bars pop for intent, urgency, frustration, refund_requested, churn_risk – the built-in triage set. laya routes: English → laya:en, other scripts → laya:multilingual. First run pulls both.

Agent guardrail flavor:

ollaya run decider --preset agent '{
 "request": "Add a unit test for DateHelper",
 "command": "rm -rf /var/lib/app/uploads && reboot"
}'

decider returns action / on_task / risk / destructive in one shot. Median ballpark on a 4090 for the 2b checkpoint: ~190 ms for a five-question style load (project homepage benchmarks) – enough to block a destructive command before the shell runs.

Pro tip:--format json or --verbose when wiring thresholds. Pretty bars hide which checkpoint answered and the timings you want in logs.

Your own questions

{
 "queue": {
 "type": "choice",
 "instructions": "Which desk owns this?",
 "criteria": {
 "shipping": "Delivery, tracking, lost packages",
 "identity": "Login, MFA, account recovery",
 "other": "Anything else"
 }
 },
 "time_sensitive": {
 "type": "noul",
 "instructions": "Is there a hard deadline in the next 48 hours?"
 }
}

ollaya run laya --questions questions.json "VPN works at home but not on hotel Wi-Fi. Board demo in 90 minutes."

Native API:

curl http://localhost:11435/api/decide -d '{
 "model": "laya",
 "state": "VPN works at home but not on hotel Wi-Fi. Board demo in 90 minutes.",
 "questions": {
 "time_sensitive": {"type": "noul", "instructions": "Is there a hard deadline in the next 48 hours?"}
 }
}'

Pin a question pack with a Modelfile – base tag plus a questions file path – then ollaya create inbox -f Modelfile. Limits from the compatibility docs: ≤256 questions/request; choice options 2-255; score levels 2-10; option budget ~125 on laya:en, ~250 on multilingual.

Drop existing TypeSafe clients onto localhost

Turns out SDK 0.7.1 (per TypeSafe compatibility) only needs env – no code fork:

export TYPESAFE_BASE_URL=http://localhost:11435
export TYPESAFE_API_KEY=local # any non-empty string
export TYPESAFE_DEFAULT_MODEL=laya # critical - default is jev-latest
export NO_PROXY=localhost,127.0.0.1

POST /v1/systemone (and /v1/decisions), GET /v1/models. Response model may show the routed checkpoint (laya:en); schema allows that.

Already running Ollama? Leave it on 11434. Ollaya takes 11435. Same CLI brain, different workload.

Edge cases that will bite production

• STATE_TRUNCATED:laya:en context is 512 tokens including questions (laya:multilingual 1024; von up to 8k; decision family up to 16k-token rows – FAQ/models pages). On /v1/*, long states return 422 STATE_TRUNCATED instead of a silent half-answer. Use von/decision, shorten state, or /api/decide (truncates and sets state_truncated=true).
• fp16 near-ties: CUDA defaults to fp16 graphs. Official parity notes: fp32 ONNX matches the PyTorch reference 100% on 2383 questions; fp16 can flip knife-edge probs. Pin laya:en-fp32 or PARAMETER precision fp32 in a Modelfile when 0.51 vs 0.49 moves money.
• Option budget & task fit: Budgets above; max 256 questions. Base Laya zero-shot on typed-decisions is near chance (~0.362). Prefer laya:typed-decisions (0.766 on that suite per docs) or heavier decoders – decider:4b 0.680, decider:2b 0.591, published Jev 1.13 at 0.727. Many-way choice still trails hard (Banking77-scale open ~0.425 vs Jev ~0.870).
• SDK default model: Skip TYPESAFE_DEFAULT_MODEL → client still sends jev-latest → MODEL_NOT_FOUND. Docs also say keep_alive is ignored on pure /v1/*; warm via /api/decide.

Open scores are starting points, not destiny. Refit calibration on your labels when thresholds pay the bills.

Grab one real inbox sample. Run it through laya and decider. Write the probability you’d actually automate on. That single number beats another leaderboard screenshot.

FAQ

Do I need a GPU for Ollaya?

No. CPU is fine on supported platforms. NVIDIA R580+ makes encoder calls milliseconds-fast; recent builds can use MLX GPU for laya/nli on Apple silicon. Desktop app: CPU only.

Which model should I start with – laya or decider?

A refund-intent noul at 0.9 on Laya that collapses to 0.6 on a rephrased ticket is your cue to graduate models – not to quietly lower the threshold. Start laya for coarse triage and multilingual routing (latency king). When a wrong route costs real money, pull decider / decider:4b / kev from the models page and A/B on a fixed labeled set.

Is Ollaya a drop-in replacement for TypeSafe Jev?

People hear “TypeSafe-compatible” and flip production. Don’t. SDK 0.7.1 shapes on /v1/systemone line up; names, context windows, and task quality do not. You must set the default model explicitly. Wire format matches; weights do not. Shadow traffic first.

Next: install line, then ollaya run laya --preset triage on one live queue ticket. If the probs match a human label, point TYPESAFE_BASE_URL at http://localhost:11435 and ship a shadow path beside the current classifier. Presets also cover email, guard, moderation, router, agent; ollaya mcp hooks Claude Code/Cursor if you want decisions inside the IDE.

Sources: ollaya.dev, quickstart, TypeSafe compatibility, FAQ.