Workflows

Test AI one workflow step at a time before you change how the team works

Picture a team lead who hears that AI saves time, buys the licences and asks everyone to use it for "the whole process". Some writers may soon report that they are faster, while others report that fixing drafts takes longer than writing them did. Both can be right if the answer depends on the step.

What the research shows

In 2023, a team of researchers working with Boston Consulting Group ran a field experiment (Dell'Acqua et al., Navigating the Jagged Technological Frontier, Harvard Business School Working Paper 24-013). With 758 knowledge workers, they found that on tasks within what AI could do well, people using AI completed 12.2% more tasks, 25.1% more quickly, with significantly better quality. On a task chosen to sit outside that frontier, people using AI were 19% less likely to reach a correct solution. The authors call this boundary a jagged frontier.

A newer experiment found uneven time savings across three related tasks done by the same people. Researchers at Figma, the design software company, ran a randomised trial with 50 product designers and 50 product managers, testing Figma's own AI tool (Stewart, Anise, Hogan and Griffin, Does AI Save Time on Product Design?, arXiv, 2026). Each person did three design tasks, with or without the tool. Over the three tasks combined, the tool group took about 15 minutes less, a 27% cut in total time (the abstract reports an average of about 20% per task). The savings were not spread evenly:

  • On a task to change a design to dark mode, there was no significant difference in time (Stewart et al.). For the designers alone, the tool group was slower on this task, though the difference was not significant.

  • On a task to add an option to a settings menu, the tool group took about a quarter less time (27%) (Stewart et al.).

  • On a task to build an interactive comment panel, the tool group saved 32% (Stewart et al.).

Participants rated the quality of their work themselves, and said it was about the same or somewhat worse than what they would make outside the experiment (Stewart et al.). No expert judged the designs. The study is small, in a lab, on one vendor's tool, and its authors work for that vendor. The study measured whole tasks, and applying it to the steps of a workflow is our own inference. We read it as a sign that an average time saving can hide parts of the work where the tool helps little or not at all.

How to test a workflow step by step

This is how we suggest a documentation, L&D or marketing team checks a workflow before changing it. It can run alongside normal work and needs no special tools.

  1. Write the workflow down as steps. For release notes, that might be collecting the changes, grouping them, drafting, checking against the product, editing for style and getting sign-off. Keep each step small enough that one person does it in one sitting.

  2. Pick a few real pieces of work. Use recent jobs the team knows well, so people can judge the output quickly.

  3. Try each step with and without AI. Record the time for the whole step, including writing the prompt, reading the output and fixing it.

  4. Have the usual reviewer judge the result. Ask the reviewer whether the output from each step would pass review as it stands, and what had to change. The Figma study relied on self-ratings; a team can use its real reviewer.

  5. Decide step by step. Mark each step as "use AI", "use AI with a named check" or "do it by hand for now". Write the reason next to each one.

  6. Look again when the tool or the work changes. A step that was slow with AI last quarter may be quicker now, and the reverse can happen too.

The result is a short table the whole team can see, showing where AI is used, where it is not, and why. New team members learn the workflow from it, and the team lead has a record to show when someone asks what AI is doing for the team.

We broke one workflow into steps in our post on release notes with AI.

[ANANT: add a real example of a step in a writing-team workflow where AI turned out slower or less reliable than doing it by hand, if we have one.]

AI Workflow Transformation (Programme 03) includes workflow redesign and a workshop where teams build AI-assisted workflow models from their own work. To talk about a workflow your team wants to change, book a discovery call.

Sources: Dell'Acqua et al., Navigating the Jagged Technological Frontier, Harvard Business School Working Paper 24-013 (2023); Stewart, Anise, Hogan and Griffin, Does AI Save Time on Product Design? A Randomized Controlled Experiment of AI Prompt-to-Design Workflows, arXiv (2026).

Send feedback

Goes to our inbox. Nothing is published.

All posts