Researchers at the University of Pennsylvania gave nearly 1,000 high school students an AI math tutor. One group used a standard chatbot, another a version built to give hints instead of full answers.
While the AI was there, both groups did much better on practice problems: 48 percent better with the standard chatbot, 127 percent better with the hint version. Then came the real test, without AI. Students who had used the standard chatbot scored 17 percent lower than classmates who never had a tutor. Students who had worked with the hint version did about as well as the classmates.¹
Talent Takeaway
It reminds us of what good teachers and coaches have always done: ask, nudge, and let people find the answer themselves. We can set up AI tools the same way, with a simple instruction such as "Give me hints, and let me try first." And it helps to keep some tasks where people practice on their own.
¹ Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26). https://doi.org/10.1073/pnas.2422633122
Further reading: Lee, H., Sarkar, A., Tankelevitch, L., Drosos, I., Rintel, S., Banks, R., & Wilson, N. (2025). The impact of generative AI on critical thinking. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 1–22. https://doi.org/10.1145/3706598.3713778