FDA's AI tool for medical devices struggles with simple tasks
The tool — which is still in beta testing — is buggy, doesn't yet connect to the FDA's internal systems and has issues when it comes to uploading documents or allowing users to submit questions, the people say. It's also not currently connected to the internet and can't access new content, such as recently published studies or anything behind a paywall.
The artificial intelligence, dubbed internally CDRH-GPT, is intended to help staffers at the agency's Center for Devices and Radiological Health, a division responsible for ensuring the safety of devices implanted in the body as well as essential tools like X-rays and CT scanners.
The division was among those affected by the sweeping mass layoffs at the Department for Health and Human Services earlier this year. While many of the device reviewers were spared, the agency eliminated much of the backend support that enables them to issue approval decisions on time.
The work of reviewers includes sifting through large amounts of data from animal studies and clinical trials. Depending on the applicant, it can take months or even over a year — which an AI tool could feasibly help shorten.
Experts, however, are concerned that the FDA's push toward AI could outpace what the technology is actually ready for.
Since taking over the agency on April 1, Commissioner Dr. Marty Makary has pushed to integrate artificial intelligence across the FDA's divisions. How this move into AI could affect the safety and effectiveness of drugs or medical devices hasn't been determined.
Last month, Makary set a June 30 deadline for the AI rollout. On Monday, he said the agency was ahead of schedule.
But the two people familiar with CDRH-GPT say that it still needs significant work and that FDA staff were already concerned about meeting the June deadline, at least in its original form.
'I worry that they may be moving toward AI too quickly out of desperation, before it's ready to perform,' said Arthur Caplan, the head of the medical ethics division at NYU Langone Medical Center in New York City. He stressed that reviewing medical devices accurately is essential, since people's lives depend on it.
'It still needs human supplementation,' Caplan said. AI 'is really just not intelligent enough yet to really probe the applicant or challenge or interact.'
The FDA directs all media inquiries to the Department of Health and Human Services. A spokesperson for HHS did not respond to a request for comment.
On Monday, Makary announced that a separate AI tool, called Elsa, had been rolled out to all FDA employees. Elsa is now intended for basic tasks agency-wide, such as summarizing data from adverse event reports.
'The first reviewer who used this AI assistant tool actually said that the AI did in six minutes what it would normally take him two to three days to do,' Makary said in an interview last week. 'And we're hoping that those increased efficiencies help. So I think we've got a bright future.'
The reality inside the agency is quite different, the same two sources said.
While the concept is solid and a step in the right direction, they said, some staff feel it's being rushed and not yet ready for prime time.
'AI tools to help with certain tasks for reviewers and scientists seems reasonable given the potential utility of AI,' one of the people said. However, the person said they disagree with the 'aggressive roll out' and claims that it could reduce work 'by hours and days.'
To be sure, experts say, it's not uncommon for a company or government agency to launch a new product and then refine it through iterative updates over time.
Staff have worked hard to get Elsa up and running, the people said, but it still can't handle some core functions and needs more development before it can support the agency's complex regulatory work.
When staff tested the tool Monday with questions about FDA-approved products or other public information, it provided summaries that were either incorrect or only partially accurate, one of the people said.
It's unclear, the people said, whether CDRH-GPT will eventually be integrated into Elsa or remain a standalone system.
Richard Painter, a law professor at the University of Minnesota and a former government ethics lawyer, said there are also concerns about potential conflicts of interest. He wondered whether there is a protocol in place to prevent any government official — such as an FDA reviewer using the technology — from having financial ties with companies that could benefit from AI. While the technology has existed for years, he said, it's still a new venture for the FDA.
'We need to make sure that the people involved in these decisions do not have a financial interest in the artificial intelligence companies that would get the contracts,' Painter said. 'A conflict of interest can greatly compromise the integrity and the reputation of a federal agency.'
Some at the FDA don't see AI as a solution to their overwhelming workloads — they see it as a sign that they may eventually be replaced.
The FDA is 'already spread thin from the RIF [layoffs] and the steady loss of individuals while in a hiring freeze and no capacity to backfill,' one of the people said.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Daily Mail
17 hours ago
- Daily Mail
Report: White House meltdown over return of RFK Jr. aide
A staffer for HHS Secretary Robert F Kennedy Jr - who drew the ire of President Donald Trump's MAGA base and was fired - returned to his job last week after Chief of Staff Susie Wiles stepped in. Dr Vinay Prasad's unexpected return to the Food and Drug Administration (FDA) marks one of the more dramatic personnel reversals of the Trump administration. And some of Trump's MAGA allies, both in Washington and across the country, still have concerns about him and doubts about his loyalty. 'It seems to be amateur hour at the FDA, and Prasad is an unnecessary distraction,' a Trump official told the Daily Mail. A hematologist-oncologist and outspoken critic of FDA bureaucracy, Prasad has long argued that the agency is too slow to adapt to scientific innovation. In July, Makary told Politico that Prasad doesn't have a 'political bone to his body.' According to senior administration officials who spoke with Politico, Wiles worked closely with Kennedy and Makary to convince the president that Prasad was not anti-Trump and should be reinstated. 'I think it really is something good about the president that he's willing to change his mind when persuaded,' one of the senior administration officials told Politico. Still, Prasad's rehiring not only signals Trump's willingness to overlook past criticisms but also represents a quiet victory for Kennedy, who now has one of his most trusted allies back inside the agency. It also underscores the limits of Loomer's influence in shaping personnel decisions despite her past successes. She has reportedly been behind nearly a dozen ousts of so-called 'deep state' officials she says have been harmful to the administration. Trump reportedly fired multiple officials on the White House National Security Council after an April meeting with Loomer. But after Prasad's rehiring was announced, Loomer called the move 'demoralizing' and an 'open disrespect of the MAGA base.' Prasad's reinstatement places him at the center of one of the administration's most ambitious policy fights: reshaping how the US regulates vaccines and biotechnology. But in a column for Newsmax, Texas-based conservative strategist Charlie Kolean warned that Prasad's regulatory approach could slow the pace of medical innovation and delay life-saving treatments. He argued that Prasad's emphasis on excessive caution would create bottlenecks for biotech development, giving foreign competitors an advantage. And he told the Daily Mail that the FDA's direction on drug approvals and biotech regulation could shift under Prasad's leadership. During his initial stint at the FDA, Prasad blocked the approval of a gene therapy for Duchenne muscular dystrophy, citing concerns over the strength of the clinical evidence. The move sparked backlash from patient advocacy groups and families affected by the disease, who argued that the therapy represented a rare and urgent lifeline.
Reuters
17 hours ago
- Reuters
Rocket Pharma's shares soar after US FDA lets gene therapy trial resume
Aug 20 (Reuters) - The U.S. health regulator has allowed a study of Rocket Pharmaceuticals' (RCKT.O), opens new tab gene therapy to resume, less than three months after halting it following a patient's death. The drugmaker's shares soared over 30% in morning trading on Wednesday. Rocket's mid-stage trial, testing its experimental therapy, RP-A501, for a genetic disorder called Danon disease, was halted in May by the Food and Drug Administration after a patient died from a rare complication. The company said on Wednesday that the FDA has lifted its clinical hold on the trial, to be resumed with a revised pre-treatment regime and a lower dose of the gene therapy. Rocket said it will discontinue prophylactic use of drugs that inhibit the C3 protein, which is part of the immune system, as part of the pre-treatment regime. The patient who died in the May trial was one of two who were administered the C3 inhibitors. The patient had suffered from capillary leak syndrome, which can cause organ failure. In the new trial, three patients, at least four weeks apart, are expected to receive a lower dose of its therapy. Rocket said the adjusted dose was proposed based on data from an early-stage study, which showed a better safety profile. Danon disease, which affects males more severely than females, causes heart muscle damage and progressive muscle weakness. Fewer than 1,000 people in the United States are currently known to have the condition, according to National Institutes of Health.
The Guardian
19 hours ago
- The Guardian
Walmart recalls potentially radioactive shrimp sold in 13 US states
Walmart has recalled frozen, raw shrimp sold in 13 states because federal health officials say it could have potential radioactive contamination. The US Food and Drug Administration asked Walmart to pull three lots of Great Value brand frozen shrimp from stores after federal officials detected Cesium-137, a radioactive isotope, in shipping containers and a sample of breaded shrimp imported from Indonesia. The products could pose a 'potential health concern' for people exposed to low levels of Cesium-137 over time, FDA officials said. 'If you have recently purchased raw frozen shrimp from Walmart that matches this description, throw it away,' FDA officials said in a statement. The risk from the recalled shrimp is 'quite low', said Donald Schaffner, a food safety expert at Rutgers University. Cesium-137 is a byproduct of nuclear reactions, including nuclear bombs, testing, reactor operations and accidents. It's widespread around the world, with trace amounts found in the environment, including soil, food and air. The level detected in the frozen breaded shrimp was far lower than FDA intervention levels. However, the agency said that avoiding potentially contaminated products could reduce exposure to low-level radiation that could lead to health problems over time. The FDA is investigating reports of Cesium-137 contamination in shipping containers and products processed by PT Bahari Makmur Sejati, doing business as BMS Foods of Indonesia. US Customs and Border Protection officials alerted FDA that they found Cesium-137 in shipping containers sent to US ports in Los Angeles, Houston, Miami and Savannah, Georgia. FDA officials collected several product samples and detected contamination in one sample of frozen breaded shrimp. The shipping containers and products were denied entry into the US However, the FDA then learned that Walmart had received potentially affected products imported after the first detection, from shipments that did not trigger contamination alerts. Walmart immediately recalled the products, a company spokesperson said. They include Great Value brand frozen raw shrimp with lot codes 8005540-1, 8005538-1 and 8005539-1, all with best-by dates of 15 March 2027. The shrimp was sold in Alabama, Arkansas, Florida, Georgia, Kentucky, Louisiana, Missouri, Mississippi, Ohio, Oklahoma, Pennsylvania, Texas and West Virginia. Consumers should discard the products or return them to any Walmart store for refund.
