QWE AI Academy Two Paths to Self Improving AI – Only One You Can Use Today
I spent a weekend chasing the shiny version of self improving AI: systems that rewrite their own code and keep getting smarter without me. Then I tried the boring version – a plain ChatGPT chat that criticizes its own answers and remembers the lessons. The boring one won. It actually improved my daily outputs. The shiny one stayed a research demo that costs real money and hardware.
For most of us the useful definition is narrow: a lightweight reflection loop around an existing LLM. Compounding quality, no training runs, no GPU rack, no lab badge. Full recursive self-improvement is real research progress. It is still not something you spin up after coffee.
Quick Background: What “Self Improving” Actually Means
Superhuman chess from zero human games sounds like science fiction until you remember AlphaZero. From random play plus the rules alone – and a pile of self-play – it passed Stockfish in roughly four hours on the chess run, with the same self-play + MCTS recipe hitting shogi and Go hard (Silver et al., 2017).
Today the field splits. Camp one touches weights or rewrites agent source. Camp two only edits the scaffold: prompts, memory buffers, tools, control logic. A Stony Brook survey (Yang et al., TMLR; as framed in that 2026 overview) treats the work as a closed loop – acquire data, select it, optimize, refine inference – under an evaluation watchdog. Open-ended RSI that improves the improver without bound still is not here; labs automate slices of the pipeline.
Method A vs Method B
| Aspect | Method A: Full / Near-RSI (weights or code rewrite) | Method B: Scaffold Reflection Loops |
|---|---|---|
| What changes | Model parameters or entire agent codebase | Prompts, episodic memory, tools, rubrics |
| Examples | Darwin Gödel Machine, AutoResearch, lab internal training | Reflexion-style critique + retry, DSPy-style prompt search, plain ChatGPT loops |
| Barrier | High – GPUs, large API spend, research setup | Low – any chat UI + a checkable success signal |
| Speed of iteration | Hours to days per meaningful gain | Minutes per cycle |
| Risk of runaway | Higher (harder to inspect) | Lower (you read every reflection) |
| Real gains shown | SWE-bench 20%→50%; ~11% training speedup | HumanEval ~80%→91%; everyday writing/coding quality jumps |
SWE-bench at 20% is a rough day. After archive-and-mutate cycles on its own Python agent code, Sakana’s Darwin Gödel Machine pushed that to 50.0% (Polyglot 14.2%→30.7% in the same line of work). One reported 80-iteration run landed near $22,000 in API fees – figure as of the DGM reports, and cloud pricing moves. Karpathy’s AutoResearch pointed an agent at a tiny train.py, capped each try at five minutes, kept only real wins: on the order of 700 runs across two days, about 20 additive gains, time-to-GPT-2 style training cut by ~11%. Cool demos. Not “open ChatGPT and go.”
Method B is what I keep open on weekdays. Verbal reflection plus a short episodic memory – no gradients – is exactly the pattern Reflexion (Shinn et al.) used to move HumanEval pass@1 from a GPT-4-class baseline around 80% up to 91%.
Detailed Walkthrough: Build a Working Self-Improving Loop in ChatGPT or Claude
Here’s the pattern I kept after watching loops collapse into vague “try harder” sludge. Three pieces only: a task with a checkable signal, a reflection prompt that forces specificity, and a memory block that travels forward.
- Define the task + success signal. Example: “Write a Python function that flattens nested lists and returns None for empty input. I’ll run these three unit tests.” The tests are the external judge. Pure self-talk without that signal plateaus fast – often after two or three rounds – because the model grades its own homework.
- First attempt prompt. Just the task. Save the output.
- Reflection prompt (the part that actually moves the needle). Paste the attempt + the failure signal and say: “In 2-3 sentences name the exact mistake or wrong assumption, then state one concrete change for the next try. Do not rewrite the code yet.”
- Retry prompt. Original task + “Previous lessons:” + the reflection text. Hide the broken code itself so the model doesn’t copy-paste the bug.
- Repeat a few times or until the signal passes. Keep the growing lesson list short – prune older lines when context gets tight.
Task: Implement flatten(seq) that yields non-iterable leaves. Empty → None.
Attempt 1 output: [your first code]
Tests failed: empty case returned -1 instead of None; nested tuples not handled.
Reflection: I treated empty as a sentinel value instead of None and only recursed on lists. Next: check type(None) early and use collections.abc.Iterable.
Now rewrite using only those lessons.That’s the whole machine. I ran it on real coding chores and messy first drafts. Quality moved faster than me hand-tuning adjectives in the system prompt. Same shape as the Reflexion HumanEval jump: memory of specific failures beats one-shot hope.
Pro tip: Force the reflection first-person and concrete (“I assumed X”). Generic advice burns tokens and adds zero lift.
Want it hands-off? A short Python script against the OpenAI or Anthropic API works. Or let a prompt optimizer (DSPy-style) search instructions when you already have a tiny labeled set. I only automated after the manual loop proved itself twice.
Edge Cases That Break the Loop (and How I Hit Them)
Same-model critique is fragile. If the model cannot see its own blind spot, the “lesson” cements the error. I watched a writing loop keep stuffing the same flowery filler because the critic scored it as elegant. Fix when stakes are high: inject an external rubric, or park a second, different model as judge.
Context dies quietly. Four rich reflections in, my ChatGPT window mashed the memory into mushy summaries. I now keep only the last two lessons – or dump the full buffer into a note and re-inject a short digest. Session timeouts wipe the slate; save the buffer yourself.
Method A burns cash and silicon. That ~$22k DGM-style run is the warning sticker. AutoResearch wants a real GPU overnight. Stay on Method B unless the research goal (and budget) say otherwise.
Honest gap in the literature: how far can pure scaffold/prompt improvement compound before you need weight updates? Short loops look great. Multi-week autonomous agents with zero parameter change are still sparsely measured – no widely shared long-horizon benchmark for that slice – so cross-tool bragging stays slippery.
FAQ
Is full recursive self-improving AI available to regular users right now?
No. Bounded loops yes; open-ended self-rewriting of the improvement process itself, no.
Do I need code or can I do this inside the ChatGPT web UI?
Web UI is enough for most writing and light coding. Copy reflections and lessons between turns by hand. When unit tests matter, I switch to a tiny notebook that calls the API and runs pytest – loop ticks while I grab coffee. I stayed pure-chat until the pattern was boringly reliable.
How many iterations before it stops helping?
People assume “more reflections = smarter forever.” That is the trap. With a sharp external signal you usually see the win by attempt two, three, or four. Past that, returns shrink and you start overfitting the exact tests or rubric. Fifth failure? Stop looping. Fix the task definition or the metric. If you only have the model talking to itself with no tests, expect an even earlier plateau – the blind-spot problem above shows up faster.
Open a fresh chat. Pick one annoying recurring LLM task. Add a clear success check. Run the three-prompt loop once. Save the final lessons as a reusable snippet. That is a self-improving AI system you can use before lunch – not a lab slide.