AI in the classroom takes off with no guardrails

Politico

26-06-2025

The push to prepare kids for the AI future is raising tricky ethical questions about how to bring the technology into the classroom.
At stake: AI can make teaching more effective and prepare the next generation's workers, but there are mounting concerns over its impact on kids' emotional health and development.
Getting AI into schools is such an imperative that President Donald Trump issued an executive order in April to promote AI literacy in schools, focusing particularly on K-12 students.
The issue of AI in schools even became a meme after Education Secretary Linda McMahon mistakenly called the tech 'A1' – the steak sauce brand – in a gaffe.
The conversation over the merits of AI in education is happening as the technology rapidly makes its way into school districts. A Gallup poll released Wednesday reports that more than a quarter of K-12 teachers already have their students using AI learning systems. The breakneck speed of adoption is even concerning proponents of the technology.
'Are we essentially condoning and accelerating our path towards this world where students are just starved of human interaction?' said Alex Kotran, who has introduced AI into hundreds of schools as CEO of The AI Education Project, a nonprofit funded by the likes of OpenAI and Nvidia.
'By foisting more AI companions on them in the form of, let's say, AI tutors, do we normalize it?'
Early research shows that AI has a lot to offer for schooling. A 2023 study found the technology can provide students with more tailored instruction and feedback, and the United Nations suggested in 2021 that AI chatbots could be a 24/7 personalized educational resource for students when teachers are off the clock.
It was in that positive spirit that Trump issued an April executive order to increase AI in the classroom.
The order created an Artificial Intelligence Education Task Force assigned to establish plans for a nationwide 'challenge' highlighting applications of AI in schools by July, and to identify federal AI resources to support state educational agencies. The order also sets July deadlines for the Department of Education to set grant guidelines for providing AI resources to students.
The White House said that with the order, 'we can ensure that every American has the opportunity to learn about AI from the earliest stages of their educational journey.'
It came as some of the country's largest public school districts – including those in Miami, Los Angeles, and Philadelphia – have moved to introduce AI chatbots to their K-12 students.
At the same time, evidence is emerging that AI could wreak havoc on a young person's mental health. The American Psychological Association warned in a health advisory that AI may impair adolescents' ability to distinguish between human and simulated empathy, and affect their real-world relationships.
Several high-profile lawsuits also accuse AI chatbots of leading kids to harm themselves, including a case alleging that a 14-year-old boy died by suicide as a result of his relationship with a Character.AI bot.
Sam Hiner, who is developing an autonomous math tutor as the founder of the AI company EdEngage, told DFD it's important that AI educational programs don't say they're proud of a student or otherwise emulate humans.
'You're attributing an emotional state to the AI that the user is going to naturally want to build a connection with, especially if you're a child and your brain is still developing,' said Hiner, who also co-founded the kids advocacy group Young People's Alliance that has pushed for restrictions on minors' access to AI. He said he worried that connections with AI may supplant those with teachers, a foundational relationship for students' well-being.
Robbie Torney, a director of Common Sense Media's AI education programs, adds that the technology's mental health risks can vary depending on the age of the student. 'The risk of parasocial relationships with technology … do tend to increase as kids get younger,' said Torney, pointing to a study showing that children between the ages of 5 and 7 are more likely to form emotional attachments to smart speakers.
Neither the Department of Education nor the White House's Artificial Intelligence Education Task Force responded to DFD's inquiries about their progress on carrying out the executive order and whether they were seeking to mitigate safety risks.
Torney notes that many schools may be unprepared to safely manage federally supported AI adoption. Eighty percent of educators say their districts do not have policies for responsible use of the technology. 'That means the districts or schools haven't necessarily conducted AI assessments for thinking about how it's actually being used,' he said. Such policies could be crucial, given the AI education sector itself is largely unregulated. Forbes reported that some of the most popular and highly valued AI tutors generated unhealthy diet advice for teens and pick-up artist tips.
The AI Education Project's Kotran said that, absent regulation, there's no 'top-down' way to ensure that schools are minimizing safety risks when procuring AI tools. 'It's still the Wild West,' he said. 'Each district is figuring this out for themselves.'
Government workers are slowly warming to AI
American government employees are notably more bearish on AI than the global average for all professionals, according to results from a 2,275-person survey Thomson Reuters published Thursday. In an interview with DFD, Reuters CEO Steve Hasker highlighted the disparity as being particularly noteworthy: While 80 percent of respondents overall thought AI would have a transformative impact on their work over the next five years, only 64 percent of U.S. government employees agreed.
Hasker partly attributed the gap to private companies having more obvious metrics for measuring the success of their AI applications, such as revenue. 'The impact of AI on commercial enterprises, in terms of efficiency and effectiveness, is probably better understood than the impact on mission-driven organizations,' he told DFD.
However, Hasker asserted that AI's impact on processing backlogs of court cases and immigration applications could be a good starting place for governments to measure the effectiveness of the technology. 'There is a push for greater tech adoption to drive efficiency and to get a better return from taxpayers' dollars,' he said.
To be sure, these employees aren't completely resistant to AI. Reuters provided additional data to DFD showing that only 1 percent of U.S. government workers believe their department is adopting AI too quickly – 63 percent are satisfied with the current speed, while 36 percent think it is too slow.
Labor's fights with big tech are heating up
Three bills in California are setting the stage for a showdown between tech companies and workers this summer, our colleagues at POLITICO's California Decoded report.
Major labor groups like the Service Employees International Union squared off this week with Silicon Valley lobbyists in making their case on the issues to the state's labor committees.
Two of the bills concern the use of technology in the workplace. Introduced in February, SB442 sets staffing levels for stores that have self-checkout. The kiosks have been promoted as a way to cut down on labor costs, but unions argue this hasn't led to lower prices for consumer products. Another bill, SB7, limits employers' ability to use tracking systems to monitor their workers.
The third bill, AB1340, would give gig economy drivers collective bargaining rights.
The Decoded team notes that labor groups have generally been successful in pushing for legislation this year, but that Big Tech still has plenty of opportunities to shut such efforts down. The gig economy bill is a reminder of a fight over a law California passed in 2019, which partly classified rideshare and delivery drivers as employees rather than independent contractors. Tech industry groups poured more than $200 million to pass a proposition that ultimately eroded it a year later.
post of the day
THE FUTURE IN 5 LINKS
Stay in touch with the whole team: Aaron Mak ([e-mail]); Mohar Chatterjee ([e-mail]); Steve Heuser ([e-mail]); Nate Robson ([e-mail]); and Daniella Cheslow ([e-mail]).