Skip to content
ChatGPT & LLMs

Qwen 3.8 27B on Cerebras: 1500 tok/s Guide

Qwen 3.8 27B on Cerebras at ~1500 tok/s. Call it in minutes, pin reasoning_effort, and dodge the three rate-limit traps that burn credits fast.

5 min readBeginner

Key takeaway: Need Qwen 3.8 27B at ~1500 tok/s today? Skip the GPU path. Hit Cerebras’ hosted API – and pin reasoning_effort plus the dual TPM buckets first, or a fat cache will 429 you before lunch.

Two routes chase this drop. Method A: Apache-2.0 weights local (GGUF / vLLM). Method B: Cerebras public endpoint, model id qwen-3.8-27b. Local keeps privacy and the open card’s ~262K native window. Cerebras wins on single-stream decode – the official model page lists ~1500 tokens/sec, which no single consumer card matches for that shape of work. Speed-first afternoon? B.

Method A vs Method B

Local (A) Cerebras API (B)
Speed (single stream) ~50-200 tok/s on strong consumer GPUs (community reports) ~1500 tok/s listed
Context Up to ~262K native 64K free trial / 128K paid
Cost shape Hardware + power + your time $0.99 / $1.49 per M in/out (Developer, as of model page)
Setup Quants, VRAM, serving stack API key + ~10 lines of Python
Fit Private data, offline, max window Burst coding, demos, agent experiments

I started down path A, stared at disk and VRAM math, then flipped to B when I just needed answers before lunch. Same model family. Different bottleneck.

Why people are yelling about it

The HN thread (~579 points, 175+ comments) split in hours: raw decode speed on one side, quota shock on the other. Cerebras serves the original unpruned weights with selective weight-only quantization for storage – catalog notes, not a mystery prune. Output flies. Defaults and buckets decide whether you smile or open a billing tab.

Walkthrough: call it

Sign up at the Cerebras Cloud Console, create a key under API Keys, export it. Billing docs: new accounts get $5 free credits after a verified payment method (card add itself is free). Credits expire 30 days after grant – short fuse, treat them as a spike test budget.

export CEREBRAS_API_KEY="your-api-key-here"
pip install --upgrade cerebras_cloud_sdk

Minimal chat – pass reasoning_effort="none" on purpose:

import os
from cerebras.cloud.sdk import Cerebras

client = Cerebras(api_key=os.environ.get("CEREBRAS_API_KEY"))

resp = client.chat.completions.create(
 model="qwen-3.8-27b",
 reasoning_effort="none",
 messages=[{"role": "user", "content": "Refactor this one-liner safely: xs.sort()"}],
)
print(resp.choices[0].message.content)
# Optional: print(resp.time_info) # queue / prompt / completion timings

Already on OpenAI SDKs? Same key, model id, base URL https://api.cerebras.ai/v1 – wiring is in Cerebras’ quickstart / OpenAI compatibility path.

Pro tip: Hard bugs → reasoning_effort="high" (maps to Qwen xhigh per the reasoning docs). Autocomplete-style turns → always "none". Effort picks a mode, not a fixed token budget.

Zero install first? Playground at cloud.cerebras.ai → pick qwen-3.8-27b → View Code.

Edge cases that actually bite

  • Cached tokens count toward Total TPM. Developer tier (as of rate-limits docs): 150K uncached TPM and 450K total TPM (total default 3× uncached). Fat prompt cache + a few multi-turn agent steps can 429 you while “new” generation looks tiny. 429 text names the bucket. HN users describe exhausting total TPM on the order of ~90 seconds in hot loops.
  • Default reasoning is high. Unset → simple prompts still think hard. You pay completion tokens and wait longer for the final answer even when decode is blazing. Set none when you want cheap/fast.
  • Hosted context ≠ open weights. Free trial 65,536 / paid 131,072 here – not the Hugging Face card’s 262,144 native (YaRN toward 1M on weights you run yourself). Size RAG and repo dumps to the API ceiling or the request dies at the boundary.
  • Images: base64 PNG/JPEG only. Data URIs in user messages. Max 2 images/request on Free Trial, 10 on Developer; 10 MiB total payload. No remote image URLs.
  • Coding-agent tool loops. Community runs: tokens arrive fast, then tool retries and shell wait eat the wall-clock gain. Cap max_completion_tokens, backoff on 429. Oversized max values get estimated up front for quota checks – they can trip limits before work starts.

What ~1500 tok/s is (and isn’t)

1M output tokens at a sustained 1500 tok/s ≈ 11 minutes of pure generation. Throughput math. Not an SLA. Prefill, queue, tools, and reasoning still own the clock you feel.

One HN agent session clocked p50 around 890 tok/s with brief 429s. Treat ~1500 as peak decode under good conditions.

Is peak speed the wrong scoreboard sometimes? When the shell and the retrier are the bottleneck, a slower model with cleaner tool behavior can feel the same in human minutes – annoying, but that’s the trade.

Developer pricing on the model page: $0.99 input / $1.49 output per million. Free Trial: 5 RPM, 30K uncached TPM, 90K total TPM, 1M tokens/day. Figures change – recheck the official page before you budget a launch.

FAQ

Is Qwen 3.8 27B free on Cerebras?

No unlimited free tier. Trial caps + $5 credits (30-day expiry) after a verified payment method, then listed per-million rates.

Why did I hit rate limits so fast on a “1500 tok/s” model?

tok/s is not tok/min forever. Developer total TPM is 450K. Picture a ~50K cached context reused across a handful of calls: the total bucket empties with almost no fresh generation. Shrink context, back off, or talk higher capacity if you’re in a tight agent loop.

Should I use high reasoning for every coding task?

No – and leaving the default glued on is the usual foot-gun. High fits architecture reviews and nasty concurrency bugs. Tight edit loops, structured JSON, tool-only turns? none (or low/medium) usually cuts cost and latency without wrecking quality. Keep high in the toolbox; don’t spray it.

Next action: Create a key, run the snippet with reasoning_effort="none", then the same prompt at high. Compare usage + time_info. That A/B beats another benchmark screenshot.

Related Tutorials