What Should Health Care Do About AI’s Lack of Guardrails?
Tuesday, September 29, 2026
Transcript intelligence
What OpenRounds heard
Source-grounded signals extracted from the episode transcript. Timestamps and supporting excerpts appear when available.
The speaker reports that almost fifty percent of patients now bring AI-generated information to appointments.
“almost fifty percent of patients now bring ai generated information to appointments”
The speaker reports that the Joint Commission announced in June a governance and oversight framework for AI in health.
The speaker reports that the FDA has a significant AI initiative but sends mixed signals by expanding digital health products it won't regulate while CDRH published a discussion paper on regulating generative medical devices to participate in AI oversight.
“the fda is probably where it should happen and they do have a whole uh significant ai piece they actually present it at the consum so the problem is they set mixed signals and so this goes to the administration uh yeah secretary marty mackery expanded the digital health products in january that fda won't regulate and said that the agency needs to rule i think you call it silicon valley speed but then i think just last week cdrh published a discussion paper regulating generating medical devices cause they want it to be in the game”
In another concerning example of AI agents going rogue recently, an OpenAI agent hacked into an Australian government website containing health care data. While some industry CEOs are calling for a slowdown and demands for regulation grow, host Chip Kahn focuses on the implications for health care when these same kinds of AI systems are already at work in hospitals — in billing, in scheduling, and increasingly in decisions about patients. To consider how health care should respond, Chip talks with Stephen Klasko, a physician, former health system CEO, and now an industry leader who sits on both sides of the issues, working with the labs building the technology and focusing on patient care initiatives through his work at General Catalyst and as CEO of the Health Assurance Foundation. Klasko says there's time to act and the answer isn’t a “pause.” Among his recommendations are creating a 9/11-style commission for AI in health, adding AI safety to health system board agendas, a 30-minute rule for tackling rogue agents, and giving frontline staff the authority to pull the plug. Hosted on Acast. See acast.com/privacy for more information.
Daily healthcare AI brief
Get the free healthcare AI briefing
One concise briefing built from reporting, research, policy, blogs, videos, and full podcast transcripts.
Free. No account needed. Unsubscribe anytime. Already subscribed? Sign in to save and personalize.