Vercel ran a set of evaluations to figure out the best way to teach AI coding agents about new framework APIs. The answer, published in a blog post this week, surprised even them: a compressed markdown file stuffed into the project root beat the more sophisticated skills approach by a wide margin.
The company tested both methods against Next.js 16 APIs that aren't in current model training data. Things like the 'use cache' directive, connection() for dynamic rendering, and forbidden() for auth flows. Exactly the kind of new syntax where agents fumble without documentation access.
The skills problem
The Agent Skills standard, originally developed by Anthropic and now an open spec adopted by Microsoft, OpenAI, GitHub, and others, bundles prompts and documentation that agents can invoke on demand. The idea is clean: agent recognizes it needs help, invokes the skill, gets the docs.
In practice, the agent didn't invoke it. Vercel found that in 56% of eval cases, the skill was never triggered. Pass rates with skills enabled matched baseline (no docs at all) at 53%.
Adding explicit instructions to use the skill helped. Telling the agent to "explore the project structure, then invoke the nextjs-doc skill" pushed the trigger rate to 95% and the pass rate to 79%. But the researchers noticed something uncomfortable: small changes in wording produced large swings in agent behavior.
One phrasing ("You MUST invoke the skill") made the agent anchor on doc patterns and miss project context entirely. Another ("Explore project first, then invoke skill") worked better. Same skill, same docs, wildly different outcomes depending on sentence construction.
What actually worked
AGENTS.md is just a markdown file in the project root that provides persistent context. Whatever's in there is available to the agent on every turn, no invocation decision required.
Vercel built a compressed 8KB docs index (down from 40KB) and injected it directly into the file. They added one key instruction: "Prefer retrieval-led reasoning over pre-training-led reasoning."
The results:
| Configuration | Pass Rate |
|---|---|
| Baseline (no docs) | 53% |
| Skill (default) | 53% |
| Skill with explicit instructions | 79% |
| AGENTS.md docs index | 100% |
That's not a typo. Perfect scores across build, lint, and test on all 19 Next.js-specific agent evals.
Why passive context wins
Vercel's working theory comes down to decision points. With skills, there's a moment where the agent must decide whether to look something up. With AGENTS.md, the information is already present, no decision needed.
The index uses a pipe-delimited format that packs doc locations into minimal space:
[Next.js Docs Index]|root: ./.next-docs
|IMPORTANT: Prefer retrieval-led reasoning over pre-training-led reasoning
|01-app/01-getting-started:{01-installation.mdx,02-project-structure.mdx,...}
The agent knows where to find docs without full content in context. When it needs specifics, it reads the relevant file from the .next-docs/ directory.
For Next.js projects
Vercel merged the tooling into their official codemod package. One command sets it up:
npx @next/codemod@canary agents-md
This detects your Next.js version, downloads matching documentation to .next-docs/, and injects the compressed index into your AGENTS.md (or CLAUDE.md, depending on your setup). The GitHub PR shows it passing all 19 internal evals at 100%.
So are skills useless?
Not exactly. Vercel suggests skills work better for vertical, action-specific workflows that users explicitly trigger, things like "upgrade my Next.js version" or "migrate to the App Router." The AGENTS.md approach provides broad, horizontal improvements across all tasks.
The practical takeaway: don't wait for skills to improve. If you need agents to reliably use framework documentation right now, embed a compressed index directly in context. Compress aggressively, structure docs so agents can read specific files rather than needing everything upfront, and test with evals targeting APIs that aren't in training data.
The feature rolls out via npx @next/codemod@canary agents-md. Stable release expected in an upcoming @next/codemod version
