Inside AI's comfort zone, quality soared. One step outside it, consultants using AI got the answer wrong more often.

A study by Dell'Acqua and colleagues, run with Boston Consulting Group in 2023, gave 758 consultants eighteen realistic tasks — some with GPT-4, some without. On tasks the AI handled well, users finished 12 percent more work, 25 percent faster, at more than 40 percent higher quality. Weaker performers gained most: 43 percent. Then came a task designed to sit just beyond the AI's abilities. There, consultants using it were 19 percentage points less likely to get the right answer. The authors call this invisible line the jagged technological frontier.¹

Talent Takeaway

A useful AI session goes beyond prompting. We can sort a team's tasks into two piles — where AI is reliable and where it still struggles — and agree on the extra checks the second pile needs.

¹ Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2023). Navigating the jagged technological frontier: Field experimental evidence of the effects of artificial intelligence on knowledge worker productivity and quality. https://doi.org/10.2139/ssrn.4573321

Further reading: Brynjolfsson, E., Li, D., & Raymond, L. (2023). Generative AI at work. https://doi.org/10.3386/w31161