LLM Capability Gap Widening: Climb the Funnel Guide

Why does everyone keep talking past each other about what LLMs can actually do?

You’ve seen the screenshots. One friend laughs at a free chatbot that can’t count letters. Another just shipped a full app rewrite overnight with an agent. The gap in shared understanding of LLM capability is widening – and Andrej Karpathy’s recent post made it explode again.

Key takeaway: Stop treating ChatGPT like a smarter Google. Install an agentic coding tool (Claude Code or Codex), give it a real multi-file project, and watch weeks of work compress into hours. That’s the only reliable way to climb from the casual tier into the professional 0.2%.

The funnel just got sharper

Karpathy dusted off an April observation because the split is no longer two camps arguing. It’s a steep funnel. Rough napkin math (as of that recent post): ~6 billion people (~75%) have barely touched LLMs. Around 1-2 billion (~20%) poke free-tier chatbots and see derpy assistants or writing helpers. Roughly 20 million (0.2%) use frontier models professionally in math and code and watch agents finish projects that used to take weeks. At the tip, maybe ~5,000 people see internal swarms doing multi-week mega-projects, science, and zero-days – work that would have taken top humans years.

Why the slope gets this steep: impact scales with problem size, ambition, and time horizon. Paragraph answers barely stress the system. Capability is also jagged – it spikes in digital, verifiable, high-value domains (code, math) because clear rewards make reinforcement learning pay off. Access tiers and last year’s free models still shape most public opinions, so the camps keep talking past each other.

Method A vs Method B: stay chatting or go agentic

Method A is what almost everyone does. Open the free (or even paid) chat window. Ask one-off questions. Paste a paragraph of code. Treat it like autocomplete or a search box. You stay in the 20% casual band. The model looks clever until it doesn’t. You walk away thinking “nice toy, not life-changing.”

Method B hands the model a terminal, your full repo, and permission to edit, run, and verify. Claude Code (Anthropic’s agentic coding docs) and OpenAI Codex turn the same underlying models into agents that plan, act, and check their own work – read codebases, edit files, run commands, keep persistent instructions. Turns out the jump Karpathy describes is mostly this loop: less than a year ago many of us still typed every character; now a prompt can restructure an entire codebase.

Method B is how you feel the real slope. Method A is fine for quick answers. It just never closes the understanding gap.

Detailed walkthrough: climb with one concrete project

I hit the wall myself last month. Free chat kept giving me half-working snippets. Then I dropped a messy side-project folder into Claude Code and the difference was physical – like switching from a bicycle to a motorcycle mid-ride.

Here’s the exact path that worked (same idea with Codex; pick the stack you already pay for).

1. Get on a paid plan that includes the agent (pricing as of recent plan checks; this may have changed). Claude Pro (~$20/mo) gets you into Claude Code. ChatGPT Plus does the same for Codex on supported surfaces. Higher Max/Pro tiers buy more runway if you run long sessions.
2. Install per the current official docs. Claude Code on macOS/Linux typically uses their curl install script, then cd your-project && claude and log in with the same account. Codex has its own CLI/IDE/cloud install path on the OpenAI Codex docs.
3. Drop a CLAUDE.md (or AGENTS.md) in the root. Tell it your stack, style rules, and “always run tests after edits.” Persistent project memory is what free chat never keeps.
4. Give it a real job, not a toy. Example prompt I used: “This repo is a half-finished expense tracker with broken auth and no tests. Explore the structure, fix the login flow, add basic unit tests, run them, and open a PR-style summary of changes. Do not ask me every step – propose the plan then execute.”
5. Watch the loop: gather context → edit files → run commands → verify. Intervene only when it stalls. In my case it spent ~40 minutes, fixed three modules, wrote tests that passed, and left a clean git status.

Pro tip: Start the session with “You have full permission to edit and run anything in this directory. Prefer small verifiable steps and report after each major change.” It cuts the over-defensive try/catch spam Karpathy has complained about.

That single afternoon moved me from “LLMs are okay writing aids” to “I will never hand-type boilerplate again.” The professional tier is not magic. It’s frontier model + agent loop + a big enough problem.

When the climb stalls (edge cases that keep people stuck)

Small one-shot prompts never reveal the gap – even on frontier models. Impact scales with ambition, problem size, and horizon; you need a reservoir of multi-hour or multi-day work you actually care about (refactor, migration, full feature).

The catch is tooling, not the subscription badge. Paying ~$20/mo for chat but never installing Claude Code or the Codex CLI/IDE agent leaves file and terminal access locked. You stay casual-tier even on a paid model.

Heavy agent runs on base Pro/Plus tiers often hit usage walls mid-project – community and plan comparisons (as of recent checks; this may have changed) call out windows around 5 hours plus weekly caps. No official promise of unbroken multi-hour autonomy on entry paid plans. Split the work or step up tier if you slam into the cap.

Exact size of the internal ~5,000 cohort and what “frontier-grade systems” include is still napkin math. Karpathy labels it that way; labs have not published a headcount or full stack spec.

Non-code domains stay jagged. Soft writing and open-ended advice improve slower because verification is fuzzy. Code and math light up first for a reason.

Honest question worth sitting with: if your week has zero multi-hour digital tasks with a clear pass/fail, which tier can you even sample?

FAQ

Do I need to be a full-time engineer to feel this?

No. A personal project, a script folder, or a messy export turned into markdown is enough. Digital and checkable beats job title.

Claude Code or Codex – which one first?

Use the subscription you already pay for. Switching stacks just to sample agents wastes the week. Claude Code is terminal-first with CLAUDE.md-style persistent instructions and deep repo reads (Anthropic’s docs). Codex rides ChatGPT Plus/Pro/Business plans and leans on CLI, IDE, and cloud sandboxes for restructures, tests, and PR-style output (OpenAI’s Codex pages). If you have neither yet, pick one $20 plan and run the same messy repo through it for a few nights before you compare.

Isn’t this just hype that will fade?

Free chat got better. That is not the professional jump. Karpathy’s point is the manual-to-agent shift landed in under a year – agents building, copying, translating, decompiling apps that used to burn weeks. Community reports of full app builds and vulnerability hunts keep stacking while most people still judge models from last year’s free surface. The understanding gap widens because the tip keeps accelerating.

Pick one real project tonight. Install the agent that matches your account. Give it the messy folder and a clear multi-step goal. Come back in an hour and decide for yourself which tier you actually live in.