What the new AI report could mean for health care

Politico

19-03-2025

We're extending the free trial through April 25 of POLITICO Pro's Technology: California Decoded newsletter, exploring how the Golden State is defining tech policy within its borders and beyond.
QUICK FIX
— Top health lawmaker Mia Bonta exclusively tells us her views on Newsom's expert report.
— Meanwhile, one of the report's authors explains why they avoided politics where possible.
— Budget watchdog asks lawmakers to rethink a $25 million CHIPS investment.
Welcome to California Decoded! Happy Wednesday, there are no controversies to report. JK here we go. Send feedback, tips and story ideas to [e-mail] and [e-mail].
Driving the day
INTERVIEW: RUBBER MEETS ROAD — Assembly Health Chair Mia Bonta sees an opportunity to make California healthier with the new AI report commissioned by Gov. Gavin Newsom's office.
The Bay Area Democrat (and potential 2026 state superintendent candidate, as our Playbook colleagues report), told us the expert panel's call for greater AI model transparency highlights one of her biggest concerns: that companies are pushing AI as a replacement for health workers and stand-ins for kids seeking therapy.
Bonta introduced a bill last month, AB 489, that would ban companies from marketing AI chatbots as licensed health professionals like nurses and psychologists. Her committee may soon take up another bill that would outlaw chatbots from luring in kids with addictive reward structures.
And there could be more legislation to come in future sessions, she told us, aimed at wielding transparency requirements to ensure equitable care and lower hospital bills.
'The industry still has major steps to take to achieve transparency,' Bonta, whose husband Rob Bonta has also been taking on tech industry challenges as attorney general, told us.
California Decoded sat down with Bonta to examine her vision for AI policy in health care.
This interview has been edited for length and clarity.
Are you pursuing a broad approach or a more pointed approach to regulating AI in health care?
No pun intended, but I think our regulatory framework needs to be pretty surgical … I'm particularly focused right now on making sure that health care professionals are not misrepresented to vulnerable communities.
Given the mental health care crisis that is happening for children right now, having them think that they are getting counsel and advice from a human being … and having, in actuality, that be an AI-generated avatar, that's a deep concern to me.
Do you think the state needs to create clearer rules for how AI chatbots are allowed to market themselves, particularly to children?
I do, and I think it's very complicated … We have to get very skilled in the Legislature to be able to make sure that we're providing very clear language that doesn't have unintended consequences around what we're trying to regulate.
I think we were very close in the U.S. to adopting a regulatory framework that would have a robust application [to kids' safety], and now we are not in a position to be able to rely on the federal government. That context is causing California to need to step up.
Is that because of President Donald Trump and the Republican-controlled Congress?
It is definitely because of an attitude that does not protect humanity, doesn't protect data and privacy and doesn't protect the basis of allowing us to use science and data and research to drive our decisions.
Do you see any gaps or missing perspectives in pieces of legislation dealing with kids' safety and AI?
I think we always run the risk of not taking the time to hear the voices that don't have the ability to be in the room.
If you take these broad conversations around AI regulation, and you are somehow not acting [on] the human components of how we need to shape this and making sure that we're focused on traditionally disenfranchised communities — like youth and low-income people and people with disabilities and people of color and BIPOC communities — you are always going to come up with the wrong answer.
Are you worried about how insurance companies use AI to evaluate claims?
I think it's an area of concern. We just had a briefing from the insurance commissioner, Ricardo Lara, and he raised this as an area of inquiry himself.
Anytime that you're using large language models, you need to make sure that we're testing for bias, and certainly when your health insurance claims could be denied based on some kind of generated AI model.
Is there anything under the radar that you're watching to see if it's worth tackling with future legislation, possibly next year or next session?
I could imagine a world in which we are using the crazy amount of data that we have around why health care costs are so high to be able to generate some solutions that would be cost-saving to the individual. That is my pie-in-the-sky hope for how we'll proceed with legislation moving forward.
HAPPENING TODAY
ALL DAY — Nvidia's annual GTC conference continues in San Jose.
1:00 p.m. — The Assembly Communications and Conveyance committee holds an informational hearing on AT&T's bid to phase out its 'carrier-of-last-resort' obligation, which requires the telecom giant to provide landline phone service in parts of California where no other coverage is available.
Happening tomorrow
EVENT: DECODED GOES LIVE — Got a burning question to ask us? Now's your chance! We're going live tomorrow at 12 p.m. PT for POLITICO Pro subscribers in a bicoastal briefing on how Silicon Valley leaders are shaping policy debates in Washington and Sacramento — and how government officials are either advancing or obstructing their agenda.
We'll touch on issues including AI regulation, data centers, privacy laws and how tariffs are impacting California's tech sector in conversation with our Washington-based colleagues Brendan Bordelon, Steven Overly and Luiza Savage.
Sign up for tomorrow's briefing here.
Artificial Intelligence
INTERVIEW: AUTHOR'S NOTE — While Sacramento and Silicon Valley are fervently parsing the AI report commissioned by Newsom, we went to the source and asked one of the three main authors about what it is and isn't intended to do, and its potential political impacts.
Jennifer Chayes is the dean UC Berkeley's College of Computing, Data Science and Society, part of the trio tasked by Newsom with putting together the report. She told California Decoded their goal was to come up with a common set of principles that could form the starting point for policy, while avoiding third rail issues like AI's impact on the labor force or its massive and growing energy use.
'I think it's a wonderful starting point now for legislation,' Chayes said. But she also said she and her coauthors were concerned about the report being politicized and misconstrued. 'It is just the nature of important conversations like this that pieces may be taken out of context and used in ways that detract from our goal,' Chayes said.
While Chayes studiously avoided commenting on any pending legislation, one section of the report did sync up with state Sen. Scott Wiener's current AI safety bill. Both contain sections emphasizing the need to expand whistleblower protections for those inside AI companies to ring the alarm should a program become dangerous.
AI companies may not be thrilled about the idea and the greater potential for employees to send up flares that could give away their secret sauce. But Chayes said it was a section that all the authors agreed on and which was grounded in existing scholarly work.
That is an example of the authors basing their carefully-worded report on existing research to get around their findings becoming a political lightning rod. And while so far their findings have been mostly uncontroversial, not everyone is pleased.
The report 'primarily urges that California wait and see — leaving lawmakers with little direction on best policies to pursue,' wrote Jonathan Mehta Stein, chair of the advocacy group California Initiative for Technology and Democracy, which last year supported Wiener's vetoed bill on AI safety, SB 1047, that ultimately prompted Newsom's creation of the expert group.
But Chayes said it was probably too early to say if sweeping regulation of frontier AI model's like Wiener's prior effort is the final destination on the path laid out by her report.
'I think AI is evolving too quickly,' she said. 'You don't want to create policies that cannot adapt.'
Silicon Valley
IN THIS ECONOMY? — California's in no state to pig out on (computer) chips that rely on Washington's support, thanks to Trump's tariffs and 'uncertain' federal spending, the nonpartisan Legislative Analyst's Office said yesterday.
As we reported for Pro subscribers yesterday, the LAO recommended that state lawmakers reject Newsom's budget request to spend $25 million on a computer chip design center proposed under the federal CHIPS Act.
The money is better spent on 'more promising' development opportunities like income tax credits amid the state's 'precarious' budget outlook and Trump's federal spending clampdown, the office said.
It's a dramatic announcement as the future of the CHIPS Act hangs in the balance, with Trump calling on House Speaker Mike Johnson to 'get rid' of the law during a joint address to Congress earlier this month. Congress passed the bipartisan law in 2022 to keep U.S. semiconductor manufacturing ahead of overseas competitors like China and Taiwan.
But there's no guarantee lawmakers will listen to the budget wonks. Assemblymember Patrick Ahrens, a Silicon Valley Democrat, told us yesterday that California should keep its $25 million commitment to the chip design facility unless federal funding is 'foolishly but formally withdrawn.'
'Backpedaling on these critical investments in technology manufacturing only serves our economic competitors and political adversaries like China,' Ahrens said in a statement.
Privacy
NO, SERIOUSLY, IN THIS ECONOMY?? — Assemblymember Sharon Quirk-Silva warned California's privacy watchdogs yesterday to play nice with businesses as Trump's tariffs weigh on the state's economy.
The Orange County Democrat's warning came after business groups protested the California Privacy Protection Agency's consideration of potentially sweeping new AI rules at a budget subcommittee hearing yesterday. Big Tech and state business leaders fear the rules will trigger job cuts and billions of dollars in lost profits. Advocates argue the rules could grant users more control over how their information is used online.
Her concerns show lawmakers are carefully considering how slumping tech stocks caused by Trump's tariffs could hit California's bottom line as they navigate the state budget process.
'We are in a very precarious time economically when we weigh what's happening with the federal government with tariffs,' Quirk-Silva told CPPA staff at the hearing. 'We need to listen to our business owners because these compounded impacts are, in fact, going to drive people out of the state unless we can do whatever we need to do to protect them.'
CPPA staff told Quirk-Silva the agency is considering criticism from business groups alongside support from labor unions and data privacy advocates as it finalizes the proposed rules. The draft regulations would require businesses to scale back their use of automated tools in everything from hiring to advertising if people ask to opt out.
Byte Sized
— Vice President JD Vance is trying to position himself as a member of both the MAGA populists and right-leaning Silicon Valley tech elites, saying the two tribes can live in peace (POLITICO)
— The Federal Trade Commission removed content critical of Amazon, Microsoft and AI companies from its website (Wired)
— Employees of a federal tech unit allege they were targeted by Elon Musk in part because of their role in preventing overspending (The Intercept)
Compiled by Nicole Norman
Have a tip, event or creepy glimmer of empathy from an AI nurse to share? Do reach out: Emma Anderson, California tech editor; Chase DiFeliciantonio, AI and automation reporter; and Tyler Katzenberger, Sacramento tech reporter.