Soumitra SahaJune 1, 2026

What Actually Happens When You Use AI to Build an App From Scratch

I look at documented public experiments — gpt-engineer, smol-dev, and real developer workflows — to show where AI-generated apps shine, where they fall apart, and what the job shift means in practice.

A full-stack app — React front end, Node back end, MongoDB — scaffolded in about 40 minutes for roughly $9 in API costs. The smol-ai/developer demo called it “surprisingly complex.” The video went viral in 2023. What it didn’t mention: the developer spent another two hours cleaning up the output before it ran cleanly in a browser, and by the time it was production-ready, most of the interesting logic had been rewritten by hand.

That gap — between the demo and the deployable — is the thing worth talking about. Not because AI app generation is a fraud, but because the honest picture is more interesting than either the hype or the backlash.

There’s a pattern across the documented public experiments: AI gets you to about 80% of a working app remarkably fast. The remaining 20% — architecture decisions, tests, coherent cross-file dependencies, long-term maintainability — is mostly still human work. And the better you understand where that line sits, the more value you get out of the tools.

What documented experiments actually shipped

The two most-watched public AI code generation experiments from 2023 were gpt-engineer (55k stars, archived April 2026, and the direct precursor to lovable.dev) and smol-ai/developer (12k stars). Both let you describe an application in plain text and watch an AI scaffold an entire codebase.

What they actually produced: Chrome extensions, Tic Tac Toe games, basic Python CLIs, rough full-stack scaffolds. The smol-dev project was explicit about what the human’s role was: “Notice that AI is only used as long as it is adding value — once it gets in your way, just take over the codebase from your smol junior developer with no fuss and no hurt feelings.” That’s not a warning label. It was the intended operating model.

The smol-dev team documented exactly where coherence broke down: “Shared_dependencies.md is sometimes not comprehensive in understanding what are hard dependencies between files. So we just solved it by specifying a specific name in the prompt.” Their fix was to add an intermediate planning step — have the AI write a dependency spec before writing any code. That design insight came from the humans working on the project, not the model itself. The model then executed that plan.

This matters because it shows the actual shape of the work: humans provide the architecture constraints, AI fills in the implementation details. The generated code was real and often ran. The structure that made it coherent was designed by people.

Where AI-generated code genuinely helps

Simon Willison — creator of Datasette, Django co-creator — has documented his AI-assisted development workflow more thoroughly than almost anyone. His clearest insight: AI didn’t just make him faster, it changed which projects were worth starting at all.

“In the past I’ve had plenty of ideas for projects which I’ve ruled out because they would take a day—or days—of work to get to a point where they’re useful… But if ChatGPT can drop that down to an hour or less, those projects can suddenly become viable.”

His example: a CORS proxy server. Three prompts, one bug fix iteration, a working deployed service in under an hour. The code he would have “painstakingly pieced together from documentation” got produced in seconds. That’s the real gain — not automation, but compression. The time between having an idea and having working code to test that idea dropped dramatically for a certain class of problem.

The smol-dev team documented the same pattern for unfamiliar API territory: “We have never really learned CSS animations, but now can just say we want a ‘juicy css animated red and white candy stripe loading indicator’ and it does the thing.” That lowered activation energy for unfamiliar code is genuinely useful, especially when you’re building something adjacent to your usual stack.

GitHub’s 2023 survey of 500 enterprise developers reported that 92% were already using AI coding tools, with 70% citing better code quality and faster completion times as top benefits. Those developers aren’t wrong. The benefits are real — they just apply unevenly across the full development lifecycle.

The bug you can’t see until you look

Here’s the thing that doesn’t show up in productivity statistics.

Willison documented a subtle error in an AI-generated benchmarking script. He’d asked the AI to compare two approaches to schema detection in SQLite. The code ran cleanly. The chart looked reasonable. He published the post. Then he noticed: the timer started before table creation, not after. The numbers were measuring the wrong thing.

“You have to closely review EVERYTHING they do. I’m embarrassed I missed this.”

This isn’t a traditional bug. The logic was sound. The structure was reasonable. The code satisfied the stated specification exactly. The problem was that the code didn’t match the intent — and the AI had no way to know the difference. It generated what was asked for; what was asked for wasn’t precise enough.

This class of problem is the tricky one. It doesn’t crash. It doesn’t fail tests (because there are no tests). It looks correct on first read. Only someone who understands what the code was supposed to accomplish — and checks that it actually does — catches it. That person is you.

Where the scaffold falls apart

Three categories show up consistently across documented experiments:

Tests. Neither gpt-engineer nor smol-dev generated test suites for the apps they scaffolded. The Chrome extension has no tests. The full-stack app has no coverage. Writing tests requires understanding what the code should do when it’s wrong — which requires reasoning about failure scenarios that the prompt never described. AI can generate tests when you ask for them specifically, but it can’t decide what to test without knowing the acceptance criteria deeply.

Architecture. The smol-dev solution to cross-file coherence — writing a shared dependency spec first — is itself an architecture document. The AI generates the implementation; the human defines the structure. That’s not a workaround, it’s the pattern. Decisions about where state lives, how services communicate, what gets extracted into a shared utility — these require context that doesn’t fit in a prompt. They require knowing what will change six months from now, which the model fundamentally doesn’t have access to.

Maintenance. gpt-engineer was archived in April 2026. The generated projects it produced are snapshots — working code without the reasoning about why things are the way they are. When requirements change, or a dependency updates, or a security issue surfaces, the person maintaining the codebase needs to understand it from the inside. A generated app has no institutional memory. The original prompt is gone. That’s not the AI’s failure; it’s an inherent property of generating code from a description.

What the developer actually does now

The description from Willison that rings most true: AI code assistance is “a really weird kind of intern — fast, unbelievably well-read, compliant, but with zero actual understanding.” The intern analogy carries the important caveat: “They won’t argue with you. That’s not necessarily a good thing.” An intern who writes exactly what you ask for, whether or not it was the right thing to ask.

The tasks that shift to AI, based on documented practice: generating first implementations, handling boilerplate, writing familiar patterns for unfamiliar APIs, first drafts of tests once you’ve defined the cases, running benchmarks. The tasks that stay human: deciding the architecture before you write the spec, reviewing output critically, knowing when code is right-looking but wrong-thinking, defining the acceptance criteria that the AI uses as its target.

This maps to what the GitHub survey found. Developers expected AI to help them “focus on solution design” and “design features and products with AI instead of writing boilerplate.” That prediction is accurate. The software development lifecycle — define the problem, build the solution, validate it, ship it, maintain it — is unchanged. What changes is how long the “build” step takes, and who writes the first draft.

If you’re reading this as a newer developer and wondering whether to be concerned: the fullstack development lessons from 2025 cover this in more depth. The short version is that the gap between “developer who can prompt AI” and “developer who can evaluate, architect, and maintain what AI produces” is where the value lives. The latter requires genuine technical depth. If anything, that depth matters more now, because the first draft arrives faster and the review work becomes the bottleneck.

And if you’re wondering about the TypeScript angle — TypeScript becomes notably more important in AI-assisted workflows, because it’s one of the most effective mechanisms for making the AI’s output verifiably correct before you run it.

The job shifts — it doesn’t disappear

The most useful mental model isn’t “AI builds apps” or “AI can’t build apps.” It’s that AI changes where the hard work happens, not how much of it there is.

The hard work used to concentrate at “getting the first working version.” Now it concentrates at “making sure the working version is actually correct.” That’s a different skill set emphasis, but it’s not a reduction in the amount of judgment required. It might be an increase.

The developers who get the most out of AI-generated code are the ones who stopped asking “can it build the whole app?” and started asking “can it get me to a testable first draft faster, so I can spend my time on the decisions that actually require judgment?” That question gets a clear answer: yes, for the right class of problems, it can.

The smol-dev repo put it well in its README: the tool is for “engineering with prompts, rather than prompt engineering.” The prompts don’t replace the engineering. They become part of it.

A node graph showing four rows of connected circles — the top row fully solid and linked, the middle row partially hollow, the bottom row mostly absent — representing the pattern of AI-generated code completing the obvious parts and missing the structural ones. — The completion gradient: top-row modules are solid and connected, bottom-row modules are hollow or absent. AI-generated apps follow the same pattern — the scaffolding is real, the structural layer is yours to build.