QWE AI Academy The #1 mistake with the essay everyone’s sharing right now? Reading it as an anti-AI rant and moving on. It isn’t. It’s a diagnostic tool – and if you use LLMs daily, it hands you a checklist for recognizing when the model just burned two hours of your life.
Journalist Yael Grauer published a short essay called “Stop Telling Me To Ask An LLM” (as of this writing, recently posted). It hit Hacker News, blew up, and the comments split into two camps: people who think it’s a takedown of AI, and people who think it’s really about a communication problem. Both camps are missing the useful part.
The one-line takeaway
Before you ask a human for help, prove your question survived the model. Before you tell someone to ask an LLM, ask them what they already tried. Everything else in this piece is mechanics.
What actually happened (30-second version)
Grauer describes calling a senior expert about a hard research question – one with no industry consensus. She wanted the pattern-matching that only comes from decades of watching decisions blow up. She asked him where he’d look, personally, for the answer – not what the textbook says. His answer, per the essay: “Honestly? Ask Claude.”
She’d already asked Claude. That wasn’t the step she’d skipped. She’d spent hours going back and forth with the model and still had a question that survived all of that. That phrase – “a question that survived the model” – is why this piece is worth reading as a tutorial and not a vent.
Two ways to read this – one actually helps
Reading A: etiquette complaint. Don’t tell people to “ask Claude” – it’s dismissive. You feel mildly judged, change nothing, move on.
Reading B: a signal about question types. Some questions belong to a model. Some don’t. Learning to tell the difference saves hours. That’s the workflow reading, and it’s the only one worth keeping.
The playbook: what to do when a question survives the LLM
Five steps. Each one either closes the question or produces cleaner evidence that a human is genuinely the next step.
1. Confirm the question actually survived
“The LLM couldn’t answer this” usually means “I asked once, in one phrasing.” That’s not a data point. Turns out, LLMs generate meaningfully different answers for the same question depending on how it’s expressed – a finding from a 2024 Knowledge Mechanisms survey (arXiv 2407.15017). Try three phrasings minimum:
- Direct: “Why does X happen?”
- Reversed lookup: “What causes Y outcomes like X?”
- Comparative: “How does X differ from Z, and when does each apply?”
The reversal matters more than most people realize. There’s a documented failure mode – the reversal curse, described by Berglund et al. and cited in the same survey – where models trained on facts in one direction reliably fail when the same fact is queried in reverse. “Who is Tom Cruise’s mother?” works fine. “Who is Mary Lee Pfeiffer’s son?” breaks things. If you only asked in the forward direction, you didn’t fully ask.
2. Make the model audit its own gaps
Add this to your prompt: “List the specific pieces of information you’d need to answer this confidently. Mark each as: known from training / uncertain / would require real-world data you don’t have.”
Flip the default. The model wants to answer; you’re asking it to inventory its blind spots first. The gaps it lists are almost always exactly what an expert would know and the model wouldn’t. Skip this step and you’ll get a confident-sounding non-answer instead.
3. Decompose with Self-Ask
Complex question? Use Self-Ask prompting. The model breaks the question into sub-questions, answers each in sequence, then combines. Research on multi-hop reasoning shows this approach improves accuracy on questions with interdependent parts (Learn Prompting documentation, based on the original Self-Ask paper). Prompt: “Before answering, list the sub-questions this depends on. Answer each one. Then combine.”
When Self-Ask fails – and it will on certain question types – the sub-questions are the actual prize. Copy them into your message to a human expert. Now you’re not sending “help me with X.” You’re sending “the model stalled on sub-questions 3 and 5 – do you have a read on those?” That’s a question worth someone’s Tuesday afternoon.
4. Know the three shapes the model can’t fill
Some questions aren’t hard prompts – they’re the wrong shape for an LLM entirely. A 2024 paper on the epistemology of LLMs puts it plainly: these models don’t have access to the basis of their knowledge – the reasons why their knowledge is well-founded. They’re reliable transmission mechanisms of already-reasoned information. Which means three flavors of question will almost always survive:
- “Which of these conflicting sources do you trust?” – requires evaluating evidence, not summarizing it.
- “What would you personally do here?” – requires lived experience, not aggregation. It’s the difference between asking a review site for the best late-night bar and asking a friend who has the same taste as you. The model can do the former. It cannot do the latter.
- “What’s not documented anywhere?” – undocumented decisions, pattern recognition from watching things fail, insider gotchas that never made it into a write-up.
Here’s the uncomfortable question buried in all of this: if models are trained on what humans already reasoned and wrote down, what happens to the knowledge that never gets written down? The expert Grauer called had decades of pattern-matching that exists nowhere in a training corpus. That gap isn’t closing on its own.
5. Escalate with your work visible
When you message a human, lead with what you already ran. Three sentences: (a) the question, (b) what the model gave you and why it’s not enough, (c) the specific thing you think a human would know. The person you’re asking can now respond in 30 seconds instead of 30 minutes – or say “I don’t know either” without guilt.
Edge cases most people miss
The benchmark trap. Don’t assume the model is close-but-not-quite. A 2025 clinical study (PMC11722524) put OpenAI o1 at 66.0% accuracy on medical calculator selection – two human annotators averaged 79.5% on the same 100 questions. That’s a 13-point gap on a defined task with clear right answers. On open-ended questions like the ones in Grauer’s essay, the gap is worse and there’s no scoreboard.
The politeness dodge cuts both ways. The HN thread surfaced something the essay doesn’t quite say explicitly: sometimes “ask Claude” is a soft no, not laziness. The other party either doesn’t know how much work you’ve already put in, or they’re giving a gentle let-down instead of saying they don’t have bandwidth. If you’re on the receiving end, don’t push. If you’re giving it, just say you don’t have time – it’s more respectful than the redirect.
The reversal curse changes your search strategy. If you’re trying to identify something by its properties (“what’s the name of the thing that does X?”) and the model shrugs, try the forward direction in a different tool – or rephrase as “given that Thing does X, what is it called?” The model may know Thing→Property perfectly and fail Property→Thing entirely.
What to actually do this week
Pick one question that’s been sitting unresolved – something you half-asked ChatGPT and gave up on. Run it through steps 1-3 above. Still stuck? Message one person with the sub-question list from step 3. Not “hey, thoughts on X?” but “the model stalled on sub-question 4 – do you have a read?”
That version of the ask respects everyone’s time, including yours.
FAQ
Is this essay saying we shouldn’t use LLMs?
No. Grauer explicitly used Claude first, for hours. The essay is about one specific failure mode – questions that survive the model – not about skipping models entirely.
How do I tell someone “I already asked the LLM” without sounding defensive?
Show, don’t claim. Paste the two most useful things the model gave you, then write: “Here’s where it stopped being useful – I’m looking for what your experience knows that this doesn’t.” That framing turns a defensive move into a compliment. Most experts respond fast when a question arrives that narrow, because it costs them almost nothing to answer a specific gap rather than explain a whole domain from scratch.
Which model is best at admitting it doesn’t know?
Honest-uncertainty behavior isn’t something the major labs benchmark publicly, and results shift week to week as models update – so any ranking here would be stale within a month. What consistently helps more than model choice is the prompt itself: explicitly ask the model to separate what it’s confident about from what it’s inferring. Don’t ask? Most models will guess and sound certain. That’s the default failure mode across the whole category, not a quirk of any one system. The “audit your gaps” prompt in step 2 above is the closest thing to a universal fix.