Anthropic published its AI Fluency Index on Sunday, a study built on 9,830 anonymized Claude conversations from a single week in January 2026. The headline finding is uncomfortable for a company selling AI tools: when Claude produces something that looks finished, people stop questioning whether it's actually correct.
In conversations where Claude generated artifacts (code, documents, interactive tools, which accounted for 12.3% of the sample), users were 5.2 percentage points less likely to flag missing context, 3.7 points less likely to check facts, and 3.1 points less likely to question the model's reasoning. Users in those same conversations were more directive upfront, specifying goals and formats with greater precision. They just didn't bother verifying what came back.
The iteration gap
The study's other core finding cuts the opposite direction. Users who treated Claude's first response as a draft rather than a deliverable performed dramatically better across every fluency metric Anthropic tracked. These iterative conversations, which made up 85.7% of the sample, showed roughly double the number of competent AI behaviors compared to one-and-done exchanges.
The numbers get sharper when you look at critical evaluation. Iterative users questioned Claude's reasoning 5.6 times more often and spotted missing context at 4 times the rate. That's a massive gap, though it's worth asking how much of it reflects user skill versus task complexity. Someone debugging code across ten messages is probably more engaged than someone asking Claude to write a birthday card, and the study doesn't fully disentangle the two.
Anthropic measured these behaviors using its Clio tool, a privacy-preserving analysis system that uses Claude itself to classify conversation patterns without human reviewers reading raw chats. The underlying framework, the 4D AI Fluency Framework, was developed by Professors Rick Dakan and Joe Feller and defines 24 behavioral indicators. Only 11 of those are observable inside a chat interface, which Anthropic openly acknowledges limits the picture.
What the study can't see
The 13 behaviors Anthropic couldn't measure are, by their own admission, "some of the most consequential dimensions of AI fluency." These include things like whether users are honest about AI's role when sharing outputs with colleagues, or whether they consider downstream consequences of generated content. The behaviors that matter most for responsible AI use happen after the chat window closes.
There's also a plausible alternative explanation for the artifact finding that Anthropic raises but can't resolve. When someone asks Claude to build a UI or write a script, maybe factual precision matters less than whether the thing works. Users might be testing code in a separate environment or sharing drafts for peer review. The drop in conversational scrutiny might not reflect complacency so much as a shift in where evaluation happens.
Still, the pattern echoes findings from Anthropic's own coding skills study published in late January, which found that developers using AI assistance scored 17% lower on comprehension quizzes than those who coded by hand. There's a throughline here: the easier AI makes things look, the less people engage their own judgment.
Only 30% set the rules
One quieter data point stood out. In just 30% of conversations, users told Claude how they wanted it to behave. Instructions like "push back if my assumptions are wrong" or "explain your reasoning before answering" were rare. Anthropic frames this as an opportunity, suggesting that setting explicit collaboration terms can change the dynamic of an entire exchange. "Tell me what you're uncertain about," the report suggests as a prompt prefix, which is the kind of advice that sounds obvious but apparently almost nobody follows.
The study connects to Anthropic's broader Economic Index research, which found that augmentative use of AI (treating it as a thinking partner rather than a task-completing machine) now accounts for 52% of Claude.ai conversations, overtaking pure automation. The Fluency Index adds texture to that finding: most people iterate, but far fewer evaluate what that iteration produces.
What comes next
Anthropic says it plans cohort analyses comparing new users to experienced ones, qualitative research to capture the unobservable behaviors, and causal studies to test whether encouraging iteration actually leads to better critical evaluation or just correlates with it. The sample skews toward early adopters who opted into multi-turn conversations during a single week, so these numbers describe a specific, likely above-average population of AI users. How the general public fares is anyone's guess.
Kristen Swanson led the research and wrote the report. The study data was collected January 20 through 26, 2026.
