In many review set-ups, approving an AI-assisted draft takes one click, while rejecting it means writing an explanation or fixing the text. A busy reviewer can feel that difference. One experiment looked at what a version of this does to checking.
The experiment
In Beck, Eckman, Kern and Kreuter, Bias in the Loop: How Humans Evaluate AI-Generated Suggestions, Harvard Data Science Review, April 2026, 2,784 participants recruited through Prolific in the US checked greenhouse gas emissions figures that an "AI" had pulled from corporate tables. The researchers prepared the suggestions themselves, so they could control which ones were wrong. For each figure, participants accepted the suggestion or flagged it as incorrect.
The researchers varied three things, in eight combinations: whether flagging an error also required typing the correct value, whether the first three suggestions were all right or all wrong, and whether a bonus was offered to the most accurate tenth of participants (Beck et al.).
What they found:
Extra effort reduced corrections. When flagging an error meant also typing the correct value, participants made fewer corrections and more often accepted incorrect suggestions. They also made fewer unnecessary corrections. Overall accuracy was 68% against 66% in the simple comparison, and unchanged in the authors' regression models. The authors call the tendency to under-report errors weak.
Attitudes towards AI were the strongest predictor. Participants who viewed AI more favourably were less accurate and corrected less. Those with more negative views of AI and automation corrected more, including more corrections of suggestions that were right (Beck et al.). The authors describe the effects in the study as modest.
A bonus did not help. Offering a payment for accuracy did not meaningfully change performance, and neither did whether the first three suggestions were right or wrong (Beck et al.).
Some errors were much harder to catch. Spelling and hallucination errors were caught 82% of the time. Values from the wrong year or scope, where the right figure sat in another cell, were caught 77% of the time. Errors that needed knowledge of the task's rules were caught 31% of the time.
The participants were crowdworkers on one kind of task, without a group working without suggestions for comparison. The effects were small. We read the study as a signal about review design that is worth acting on. It does not tell us how much any one change will help.
How to design the review step
The authors make several recommendations for their kind of task (Beck et al.). These are the ones we think carry over to a documentation, L&D or marketing team, with our adaptations.
Make flagging as quick as approving. The authors recommend keeping workload comparable across decision options (Beck et al.). In a content team, a reviewer can mark a passage as wrong in one step, with an optional note, and the writer handles the fix.
Separate judging from fixing. The authors suggest one group judges whether a suggestion is correct and a separate group makes the corrections (Beck et al.). The reviewer flags and the writer fixes.
Put rule-based checks on the checklist. In the study, errors that needed knowledge of the rules were caught far less often than spelling errors (Beck et al.). The participants had no special training, and a trained reviewer may do better, but style rules, regulatory wording and terminology still deserve their own named checks.
Watch for reviewers who approve everything. The authors suggest monitoring for people who accept or reject nearly all suggestions (Beck et al.). A team can look at approval rates by reviewer and use them to start a conversation about how review is going.
Test review now and then. The authors list deliberately altered items and attention checks as something future studies could include (Beck et al.). A team can try the same idea: occasionally seed a draft with a known error and see whether review catches it.
Know your reviewers' leanings. Reviewers who were favourable to AI missed more errors, and sceptical ones corrected more text that was right (Beck et al.). Pairing the two, or reviewing the same sample together and comparing results, may help each see their own pattern.
AI Workflow Transformation (Programme 03) has a module on AI evaluation and review systems, covering human review structures, validation workflows and trust calibration. To talk about your team's review process, book a discovery call.
