The AI Cost of JavaScript Frameworks — Real Benchmark Numbers
By Arash Latifi
Nolan Lawson built the same app 5 times per framework with Claude agents and measured cost, tokens, time, and Lighthouse results. Here's what actually matters when an agent writes your code.
TL;DR — When AI agents write the code, lightweight and well-known frameworks (Astro, Hono+htmx) build cheaper and faster than Angular or SvelteKit — but the biggest real-world gap is client-side JS and LCP, not build cost.
When an agent writes your code, "developer experience" stops being a good reason to pick a framework. Nolan Lawson took this seriously: he had Claude Opus 5.5 build the same Pokémon app (PokéAPI) 5 times per framework, then scored outputs with a conformance suite and Lighthouse. Median of 5 runs:
| Metric | Best | Worst | Gap | |---|---|---|---| | Cost per build | Hono+htmx $2.67 | Angular $4.97 | ~86% more | | Time per build | Astro 8.2 min | Angular 17.2 min | >2x | | Tokens per build | Nuxt 3.85M | Angular 9.76M | ~2.5x | | JS sent to client | Astro 0 KiB | Next.js 142 KiB | 142x | | Mobile LCP | Astro 810ms | Next.js 2090ms | 2.5x slower |
Why Angular was the most expensive
Three concrete reasons from the writeup:
- Code size: 2,009 lines vs 1,370 for Hono+htmx — roughly 70% more code. And more code means more tokens on every future edit, not just the initial build.
- Agent failure: in 2 of 5 runs, the agent built an SSR app that rejected non-localhost requests, producing no measurable app at all.
- MPAs (Astro, Hono+htmx) ship plain HTML, so they build faster and cheaper.
Quoting Rich Harris: "Frameworks are not tools for organizing your code; they are tools for organizing your mind." In the agentic era, code size literally maps to token cost over the life of the codebase.
One big caveat: the test app was content-driven
The app was a Pokémon list with search and filtering — closer to Wikipedia than Figma. As skrebbel pointed out on HN, Astro scores well here because it's built for content-driven sites. A highly interactive, realtime app could flip results. Lighthouse and Core Web Vitals are also inherently biased against SPAs, which favors MPAs on TBT and JS bytes. Several commenters noted agents sometimes struggle with the Astro + React islands boundary — so "Astro for everything" isn't the answer either.
Practical checklist when an agent writes your code
- [ ] If your site is content-driven (blog, landing page, shop), seriously consider an MPA or low-JS framework — the LCP difference is real, not marketing.
- [ ] After every agent session, check JS bytes and LCP in Lighthouse; one Next.js app in the benchmark skipped lazy-loading images and LCP spiked.
- [ ] Prefer frameworks well represented in training data — SvelteKit got expensive because the agent re-read the v3 migration guide every single run.
- [ ] Treat code size as a cost: every extra line costs tokens on each future edit.
- [ ] Install the official MCP/skills, but don't expect magic: Nuxt's MCP was never used, and Next.js skill files were read in only 1 of 5 runs.
Read the numbers honestly
Cost figures are quoted API prices from Claude Code, not what the author actually spent (he's on a Max plan). Everything here is stochastic — LLMs sometimes go off on tangents and burn tokens regardless of framework, and Lighthouse itself gives inconsistent LCP readings. Lawson ran 5 iterations and reported medians with ranges, so read these as trends, not laws.
The encouraging part: Performance, Accessibility, and CLS scores were nearly saturated across all frameworks (97–100). Whichever you pick, a competent agent can produce a fast, accessible result — the real differences are cost and JavaScript weight.
Q: Should I drop Next.js for Astro?
Not necessarily. The benchmark app was content-heavy. For apps with heavy interactivity and state, Next.js or SvelteKit remain reasonable. But for mostly-content sites (blogs, shops), the LCP and JS-bytes gaps are genuinely meaningful.
Q: Will these costs match mine?
It depends on your model's pricing. The figures are direct Claude Opus 5.5 API pricing at high effort. Absolute numbers change with cheaper models or subscriptions, but the relative ranking between frameworks roughly holds — Angular stays at the top.
Q: What about vanilla JS — no framework?
Surprisingly it underperformed several frameworks: the agent built an SPA all 5 times, with LCP reaching 1,667ms. A good framework beats "nothing" even for agents.