A Rare Randomized Test of the AI Teachers Actually Use
Most of what we read about AI in the classroom is anecdote dressed as evidence. A new working paper from Alp Sungu of the Wharton School, with Benjamin Lira Luttges and Angela Duckworth, is a different animal. The team ran a randomized controlled trial across 14 middle and high schools in Turkey during the spring 2025 semester, covering 193 teachers, 2,816 students and 14,198 student course observations. Teachers were randomly assigned to keep teaching as usual, to receive a custom generative AI teaching assistant built on GPT-4o, or to receive that same tool plus weekly reminders and usage feedback.
This matters because it tests the exact product category that vendors are selling hardest right now. The tool in the study was not a student chatbot or a homework helper. It was a teacher facing assistant for lesson planning, materials and instructional support, the same pitch that MagicSchool, SchoolAI, Brisk and a dozen others make to districts every week. When a study isolates that intervention and measures downstream student outcomes with random assignment, education leaders should read the result carefully rather than reach for the marketing deck they were shown last quarter.
What the Numbers Say About Motivation and Grades
The headline finding is uncomfortable. Students whose teachers received the AI assistant reported lower intrinsic motivation, a drop of about 0.11 standard deviations. They rated their courses as less important, less enjoyable and less interesting than students in the control group did. There was also a marginal hit to student confidence. This is not a rounding error in a small pilot. It is a consistent, negative signal on the outcome that predicts whether students keep showing up and keep trying.
On the metric vendors love to promise, academic performance, the study found no statistically significant average effect at all. The AI did not lift grades across the population. Worse, the effect was unevenly distributed in a way that should trouble anyone deploying at scale. Students taught by lower performing teachers scored 0.129 standard deviations worse when their teacher had the tool, while students of stronger teachers showed a small, directionally positive but non significant improvement. The technology widened a gap rather than closing one.
The Uneven Effect: Weaker Teachers, Heavier Users
The most counterintuitive result is about experience. We tend to assume that the teachers most comfortable with AI will get the most value from it. The data says the opposite. The negative effect on students was roughly three times larger for teachers who were already heavy AI users before the trial began. In the authors' framing, familiarity did not breed acceptance, and experience bred concern. The people best positioned to lean on the tool were the ones whose students paid the highest price.
That pattern points at a mechanism rather than a fluke. Heavy adopters appear to have offloaded more of the creative and judgment heavy parts of teaching to the model, and their students noticed. The usage data supports this reading. Around 66 percent of teacher conversations with the tool were about creating teaching materials, only 16 percent involved deeper instructional support, and the median conversation ran to just two prompts. This was fast, shallow content generation, not a sustained collaboration that sharpened a lesson. Volume of use was not a proxy for quality of use.
Why Generic Output Erodes the Classroom
The authors offer a plain explanation that will resonate with anyone who has sat through a slide deck someone clearly did not write. AI generated materials may be competent but generic, and students notice when their teacher's own voice disappears. The connection between a teacher and a class is partly built on idiosyncrasy, the specific example, the local reference, the argument the teacher clearly cares about. Smooth, averaged output can quietly remove exactly the friction and personality that made a subject feel worth caring about.
For edtech buyers, this reframes the risk. The danger is not that teacher AI produces wrong answers, though it can. The subtler danger is that it produces plausible, forgettable ones at scale, and that the aggregate effect on a student's relationship with the material is corrosive rather than neutral. A tool can pass every accuracy benchmark a procurement team writes and still make a class less engaging. Motivation is a leading indicator, and this study caught it moving in the wrong direction before grades ever showed the damage.
What This Means for Procurement and Rollout
We would not read this as a reason to ban teacher AI. We would read it as a reason to stop measuring success by adoption. Seat counts, weekly active teachers and prompts sent are the metrics vendors report because they are easy and flattering. This trial shows that a program can look like a runaway success on those numbers while quietly depressing the student outcomes the district actually cares about. The reminder and feedback arm of the study, which nudged teachers to use the tool more, did not rescue the result.
The practical implication is that deployment needs guardrails and training aimed squarely at the heaviest adopters, not just the reluctant ones. Districts should pair any teacher AI rollout with explicit norms about where human authorship stays non negotiable, and should instrument for student motivation and engagement, not only for tool usage and grades. The authors themselves conclude that successful deployment might need guardrails and training, especially for the teachers most eager to adopt. That is a design and change management problem, and it lands on the CIO's desk as much as the curriculum office's.
The Caveats, and the Signal Underneath Them
The researchers are careful, and so should we be. The intervention lasted a single semester and used one specific tool, so results could differ with longer exposure, a different product design, stronger guardrails or more discriminating assessments. A one semester window may catch a novelty dip that fades, or it may miss a slower erosion that compounds. This is one rigorous study in a young field, not a verdict, and the sample sits in one country's school system rather than across many.
Even with those limits, the direction of the finding is the story. When a category as hyped as teacher AI is finally subjected to a large randomized trial, the average student came out slightly worse on motivation and no better on grades, with the harm concentrated where support was already thinnest. Buyers have spent two years being told these tools obviously help. The honest position now is that they might, under conditions we have not yet pinned down, and that the burden of proof belongs with the vendor and the deployment, not with the skeptic.
