QWE AI Academy Pick this path if you care more about on-device speed and privacy on Apple Silicon than about squeezing the last point of word error rate. Skip it (for now) if you need multilingual output, proven café-noise robustness, or legal/medical-grade transcripts.
Embedding Gemma Transcribe does not fine-tune Google’s EmbeddingGemma 2 into a classic ASR network. A small reader learns to pull words from the model’s shared audio-text embedding space. On the project’s base M3 MacBook Air (8 GB) figures, that lands near 12.6 seconds per hour of audio (~288× real-time) on the Neural Engine, with word error rate around 5% on the numbers they publish. Free Mac app; English-only at launch.
Think of the frozen audio encoder as a library that already shelves sound near spelling. The reader is a quick clerk: it takes a short slice, walks to the right shelf in the 768-d space, and grabs the closest card from a fixed English word list. That is retrieval energy, not a full acoustic decoder stack – which is why a tiny head trained in under two hours on 8×H100 can move so much audio on a thin laptop.
What EmbeddingGemma 2 is doing here
EmbeddingGemma 2 is Google DeepMind’s open multimodal embedding model (~740M: about 270M text, optional ~170M vision, ~300M audio), unified 768-d space, Apache 2.0, 8K context, released around early October 2026 per Google’s docs/blog/HF listing. Spoken “cat” sits near the text token “cat.”
Project write-up on eg2.haku.sh: freeze that audio encoder. Train a small reader that maps short audio slices into the text side of the space. Nearest neighbor among roughly 229,000 English words becomes the token. Another tiny model fixes punctuation and capitals. Openly licensed English speech only. Base checkpoint was never trained as speech-to-text. Creator notes the spark was the shared-space behavior people saw with Gemini Embedding 2, then tried again after EmbeddingGemma 2 went open.
Method A vs Method B
| Aspect | Method A: Dedicated STT (Whisper-class, Parakeet, Granite Speech) | Method B: EmbeddingGemma Transcribe |
|---|---|---|
| Core approach | End-to-end ASR trained for transcription | Frozen multimodal embedder + small reader + nearest-word lookup |
| Speed (base M3 Air, project table) | Granite Speech 5.0 TurboCTC ~150× (25.7 s/hour via MLX GPU); other stacks vary | ~288× (12.6 s/hour) on Neural Engine |
| WER (published) | Often ~2-3% on clean English for leaders | ~5% band (5.18% LibriSpeech test-clean; ~11.5-11.8% harder set) |
| Languages | Many multilingual options | English only as of launch (multilingual training planned) |
| Privacy / hardware | Varies; some GPU-heavy | On-device Neural Engine; audio stays on the Mac |
| Setup | Often Python / MLX / llama.cpp | One free native app (v1.6 in project materials) |
Method A still owns lowest error, language coverage, and messy audio when the stack is built for that. Method B owns throughput, install friction, and “never upload the file.” The project page already admits the WER gap and talks about a stronger reader later – so you are not guessing about the trade.
Batch podcasts and personal dictation on a MacBook? Method B is the one worth timing on your own files first.
Edge cases before you rebuild a workflow
- Fixed English vocabulary (~229k). OOV names, jargon, and code-switch map to a nearest neighbor. Wrong word can look oddly confident.
- LibriSpeech-flavored WER may read optimistic. Creator figures: 5.18% test-clean, ~11.5-11.8% on the harder split. Community pushback: LibriSpeech is often in-distribution for systems like this, so spontaneous speech will likely score worse.
- Overlap, heavy noise, accents: no solid public numbers yet. Threads asking about two-speaker pileups did not get hard benchmarks in what’s available. Conference calls and cafés stay experimental.
- Apple Silicon Mac only (macOS 14+). First launch pulls ~715 MB of models. No Windows/Linux app from the project at launch; speed story is Neural Engine / Mac-centric.
Is ~5% “good enough”? For notes and first-pass show drafts, often yes. For anything you’d swear to in court or a chart, keep a dedicated ASR path and a human pass.
Mac walkthrough (beginner path)
Requirements match the download page: macOS 14+, Apple Silicon. Grab the free build from eg2.haku.sh.
- Install and open the app.
- First launch downloads models (~715 MB). Let it finish before judging speed.
- Batch: drop an audio file.
- Live: hold Right Option, speak, release – text goes into the focused app.
- Path stays on-device via the Neural Engine per the project.
Pro tip: use a clean 1-2 minute mono clip you already typed out. Count substitutions yourself. That single file beats any blog table for “will I tolerate this?”
No Python env. No weight conversion. If you still need embeddings for search or RAG, load the base EmbeddingGemma 2 weights separately – the STT reader sits on top; it does not rewrite Google’s encoder.
FAQ
Is this actually faster than Parakeet or Whisper on a MacBook Air?
On the project’s same-machine M3 Air row: yes versus their Granite TurboCTC ~150× figure. Parakeet-style numbers they cite via FluidAudio come from other Macs, so don’t treat those as a head-to-head on your Air.
Can I use it for languages other than English?
Not with this reader head today. Picture a bilingual interview dump – you’ll still want Method A. Creator has said multilingual training is planned; until that ships, treat non-English as out of scope.
Does this modify the base EmbeddingGemma 2 release?
No. A common mix-up is assuming any fast STT demo must have full fine-tuned the backbone. Here the audio encoder stays as Google released it. Only the small reader and punctuation model get trained. That’s also why the build could land after a short 8×H100 run, and why original embedding use cases still work if you load the stock weights on their own.
Concrete next step: download the app, drop one short recording you know cold, and time wall-clock seconds. Your file, your accent, your noise floor – those three beat another screenshot of 288×.