The AI assistant makes you faster and, in the same motion, stops you getting better
You watch the terminal fill. The change is done, tests pass, feature shipped. You told the assistant what you wanted, it wrote the implementation, and you merged it. It feels like progress. It even feels like learning — you saw every line it wrote, after all.
It is not learning, and the controlled evidence is blunt about it: AI assistance raises output and, in the same motion, suppresses skill formation — and the skill it suppresses first is the one you need most to trust the tool at all.
This is not an anti-AI piece. It is a piece about a specific discipline for anyone who ships on an assistant, and — if you are building an AI mentor, tutor, or assistant product — a design rule that decides whether your product is durable value or a dependency you are selling. And it applies with special force to the beginners you are building for: the kid in a coding tutor, the agent in a travel planner, the solo builder who has the least prior knowledge against which to check the model's output.
---
The demo that feels like progress
Run a session with Claude Code or any capable coding assistant and the loop is addictive for the right reasons: describe, get a working implementation, test, merge. Do it forty times a week and the output compounds. The quiet assumption underneath is: finishing the task and reading the code is how skill builds.
That assumption is where the trouble starts. Reading a result is not producing it. And which part of you grows depends entirely on which of those two you actually do.
The category — AI mentors, tutors, assistants, "learn by building with AI" — has a second, louder assumption on top: that the tool both finishes the work and teaches you. A kids' coding tutor, a travel planner that does the itinerary, a mentor that gives you the implementation. All of them, one shared pitch: the more the assistant does, the better you get.
The evidence says the opposite. The tool that finishes the work is the same tool that does — for a measurable slice of the learner — the work that builds skill, instead of the learner.
---
What the controlled evidence says
Two primary studies carry this, and neither is an opinion essay.
Anthropic's own randomized controlled trial. Anthropic took 52 mostly-junior engineers, all regular Python users unfamiliar with the Trio library, and asked them to learn Trio with or without AI assistance over roughly 70 minutes. The AI-assisted group successfully completed the tasks. They also scored lower on the learning assessment — and the largest gap between the two groups was on debugging questions: identifying when code was wrong, diagnosing the error, reading code. The researchers' reading: the non-AI group got better at debugging by resolving Trio errors themselves, and developers leaning on AI may "rely on AI to complete tasks as fast as possible at the cost of skill development." Anthropic — "How AI assistance impacts the formation of coding skills"
The shape of the result matters more than the numbers: task completed, skill not formed, and the widest gap in the exact ability — catching when output is wrong — that you need to safely use the tool in the first place.
Bastani et al., PNAS 2025. A field experiment with nearly 1,000 high-school math students. Students using a ChatGPT-style tutor (GPT Base) scored 48% better during AI-assisted practice — and then 17% worse on an exam once the AI was removed, versus a control group that never had access. The researchers read the pattern as students using GPT as a "crutch" during practice and performing worse on their own. Bastani et al. — "Generative AI: Can AI help or harm learning?" (SCALE/PNAS)
Note what the safeguard group proved. A second tutor arm that gave hints, not answers (GPT Tutor) raised practice performance 127% while scoring about the same as the control on the exam. The harm tracked answer-giving, not AI itself. You can build an assistant that raises both performance and learning — but only by withholding the answers.
Together, the two studies reverse the "AI makes you better" take with randomized evidence: output goes up while measured skill does not — and the fallback is severe the moment the tool is removed.
---
Why the obvious fix does not hold
The obvious response to "your assistant is doing the learning work" is: great, delegate everything, ship faster, and I'll absorb it through exposure. That is exactly the response that fails, and the cognitive mechanism is well-established enough to name.
Durable skill is built by generation — producing, attempting, retrieving, forming the answer yourself, including producing a wrong answer and then correcting it. The generation effect is one of the most replicated findings in memory research (Slamecka & Graf 1978; meta-analyzed in Bertsch et al. 2007): actively producing an answer encodes materially stronger than passively receiving the same information, and the benefit survives even when the generated answer is wrong but corrected. Bjork's "desirable difficulties" work extends it: conditions that feel harder — generation, retrieval, spacing — build durable skill, while fluent passive reception creates an illusion of competence. Generation effect / desirable difficulties — synthesis
Here is the mechanism in one line: an assistant that completes the task converts every one of your generation events into a reception event. You read its output instead of forming your own. Performance can rise — you absorb the output — while the underlying ability does not transfer. That is not a hunch about AI; it is the generation effect applied to who the model does the work for.
This is also why just shipping faster with the model doing the hard parts does not hold as a learning strategy. You are optimizing the fluent-reception track, which is exactly the track that feels good and teaches the least.
---
The verification skill is the first casualty
Here is the part that should worry an operator most, because it is a trap with a one-way door.
To safely use AI output, you must already be able to detect when it is wrong. That verification ability is itself a skill — and it is learned the way debugging is learned: by resolving errors yourself. Which is precisely the work the assistant now does for you.
So Anthropic's debugging-gap result is not an accident of the study design. It is the mechanism surfacing where it always would: debugging is internalized only by doing it, and the tool that removes the doing removes the learning. The first skill to erode is the one you need to catch the tool's mistakes. The gap between what the AI produces and what you can verify is your exposure, and it widens silently.
This is also the explanation for a claim every experienced builder has felt: "AI is like having a very eager junior developer," as Addy Osmani (engineering lead, Google Chrome) puts it — "they write code quickly but need constant supervision." InfoWorld / Osmani — why AI assists experienced developers most The reason is the verification precondition: you cannot reliably offload work you cannot check. The same tool is a multiplier for the expert who can verify, and a skill-suppressor for the novice who cannot.
A self-reported survey from Microsoft Research (CHI 2025) corroborates the direction at scale: knowledge workers report lower perceived effort of critical thinking when using GenAI, and continued use correlates with more reported over-reliance. Microsoft Research / CHI 2025 We treat this as supporting color, not proof — it measures reported effort, not capability — but it points the same way as the two randomized trials.
---
Why this hits the beginner hardest
The same dynamic — assistant does the work, generation becomes reception — lands hardest on the person with the least prior knowledge, because the verification skill has nothing to stand on yet.
Think about who you are building for. A kids' coding tutor: the child has no way to know the model's scaffolded solution is right, and no baggage of prior errors to correct. A travel planner taking over itinerary reasoning: the traveler delegates the decision-making that the agent was supposed to be learning. A solo founder shipping with an assistant on day one: they check the model's output against experience they do not yet have.
The beginner does not get a smaller loss — they get the whole loss, because generation was all they had. The experienced operator still holds a bank of debugging instincts from a decade of doing the work; the beginner holds a terminal full of the assistant's output. Anthropic's own study used learners (junior engineers on an unfamiliar library), not veterans, and the gap was widest for exactly that population.
---
The reusable object: the keep-the-skill rule
You do not have to choose between shipping fast and building skill. You have to stop outsourcing the load-bearing share of the work you are claiming to learn, and you have to verify what you do delegate. Four bars, about fifteen minutes per cycle, no model required.
Bar one — hand-built share. Each cycle, deliberately hand-build or re-derive one load-bearing piece the AI would otherwise do for you. Not the boilerplate — the piece that actually carries the risk. Keep generation in you by producing that piece yourself, even if it is rough and you fix it with the AI afterward. That generation event is the skill deposit.
Bar two — verify by re-explaining. Before you trust or merge any AI key block, re-state it in your own words or write a test that encodes the expected behavior. The test is the mechanical form of verification: you cannot write "this function must do X" unless you know X. That act is the debugging-adjacent skill the study showed employees losing.
Bar three — measure skill, not output. For any learning claim about yourself or your team, check: can you still do it with the AI off? Bastani's +48%/−17% is the price of skipping this. Output volume measures the model, not you. The only true learning signal is performance after the tool is removed.
Bar four — withhold, don't give (if you build an AI mentor). Give answers and your product maximizes today's metric and pays tomorrow's withdrawal penalty. Withhold with hints and you keep both the performance and the skill. Design for the user's performance after removal, not during.
The discipline is a deliberate share of hand-built, self-generated work plus an explicit verify-by-re-explaining/test loop. That is the difference between the machine finishing the whole job and the machine finishing the job while you guard the skill.
---
The product fork, if you are building an AI mentor
If you are the builder — and this whole batch of briefs is AI-assistant products — one decision rule settles most of your design:
Does the user still perform after the AI is switched off?
Answer-giving (GPT Base) looks great in the demo and collapses in the field: +48% during practice, −17% after removal. Hints-but-not-answers (GPT Tutor) looks slightly less smooth in the demo and preserves both performance and learning: +127% during practice, no harm on the exam. The two are not trading performance against learning; they are trading what your metric measures. If your north-star is task completion with the tool on, you are optimizing the crutch. If it is performance with the tool off, you are optimizing skill.
Design implication: every "show me the answer / do it for me" surface in your product is a withdrawal-penalty surface. Every "hint / here's the concept / try again" surface is a skill surface. This is the difference between a tool your users need and a dependency your users are trapped in — and, for parent-tracked kids'-tutor briefs especially, the difference between teaching and selling fluency.
---
One concrete first step this week
This week, on one cycle you would normally hand entirely to the assistant: hand-build one load-bearing piece yourself, write the test that proves the AI's key block does what you think, and — if you are building for anyone — ask the person learning from your product "can you do it without the AI?" and take the honest answer.
The machine can finish the whole job. The question is only where you keep the skill. Keep the one skill it cannot hand you — the ability to catch when it is wrong — and the assistant stays a multiplier. Give that away, and "faster" is all you were ever buying.
The keep-the-skill rule in one line: output works for you; skill only builds in the work you do yourself. Hand-build one piece each cycle, verify by re-explaining or testing, measure with the AI off — and if you build the mentor, withhold the answers.
---
Sources
- Anthropic — "How AI assistance impacts the formation of coding skills" (RCT). https://www.anthropic.com/research/AI-assistance-coding-skills
- Bastani et al., "Generative AI: Can AI help or harm learning?" (SCALE/PNAS 2025). https://scale.stanford.edu/publications/generative-ai-can-harm-learning — SSRN preprint: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4895486
- UNESCO — "Guidance for Generative AI in Education and Research" (2023). https://unesdoc.unesco.org/ark:/48223/pf0000386693
- Generation effect / desirable difficulties (Slamecka & Graf 1978; Bertsch et al. 2007; Bjork) — synthesis. https://edtechdev.github.io/aied/concepts/desirable-difficulties
- Addy Osmani / InfoWorld — AI assistants help experienced developers most. https://www.infoworld.com/article/3619505/why-ai-is-best-for-experienced-developers.html
- Microsoft Research (CHI 2025) — GenAI and perceived critical-thinking effort / over-reliance (self-report; supporting only). https://dl.acm.org/doi/abs/10.1145/3706598.3713778
- Anchor for the originating brief. https://github.com/anthropics/claude-code
---
X entryway list (from the approved angle)
- Hook: sharp claim.** Claim: "Your AI coding assistant makes you faster and, in the same motion, stops you getting better. It finishes the work you were supposed to do to build the skill — and the skill it erodes first is the one you need most: catching when it's wrong." Links to the article — the link is the entry to the performance-vs-learning tension and the rule, not the whole point.
- Hook: surprising proof point.** Claim: Anthropic's own study split engineers on a new library with/without AI; AI users completed the tasks but learned less, biggest gap on debugging — faster at the work, worse at catching the errors. Links to the article — the study is the entry; the article explains the mechanism and the debugging-first erosion.
- Hook: surprising proof point.** Claim: In a ~1,000-student trial, ChatGPT-style tutor students scored 48% better during practice and 17% worse on the exam after removal vs never having it — the surest sign a product teaches you is you fall apart when it's switched off. Links to the article — the +48%/−17% pair is the load-bearing number; the article is why answer-giving causes it.
- Hook: mechanism.** Claim: Skill is built by generation — producing, even a wrong answer you correct — not by reading results; the assistant converts your generation events into reception events, which is why output rises as skill stalls. Links to the article — the mechanism stands alone; the article is its observable behavior and the fix.
- Hook: workflow / build sequence.** Claim: The keep-the-skill build — hand-build one load-bearing piece each cycle, re-explain the AI's key blocks, write the test that encodes expected behavior, measure "can I/they still do it with the AI off?" Links to the article — the build is the actionable unit; the article is why it holds plus the worked examples.
- Hook: result.** Claim: The truth test of any AI mentor you build: does the user still perform after the tool is removed? Answers look great in the demo and collapse in the field — Bastani's +48%/−17% is the bill for answer-giving; hints keep both performance and skill. Links to the article — the product decision rule (GPT Base vs GPT Tutor) stands alone; the article is the full evidence.
These entryways are named but not written as posts — distribution happens in a later stage after the draft is approved.