The FDA has published a discussion paper on whether generative AI-enabled medical devices need their own regulatory framework, opening a public comment window that closes October 19.
Regulators say these devices behave differently from the software and AI tools they already oversee. They produce variable outputs that shift as underlying models learn, accept open-ended inputs that premarket testing cannot exhaustively cover, and often run on third-party foundation models whose training data and evaluation methods are largely opaque. That makes specific errors hard to trace back to a model.
The paper, from the FDA’s Center for Devices and Radiological Health, flags confabulation and hallucination as core risks. A device that fills gaps with plausible but invented information, or associates a symptom with the wrong condition, can mislead clinicians who trust confident-sounding output.
Among the ideas floated is competency-based testing modeled on medical training, licensure exams, supervised practice, and periodic re-evaluation, combined with device benchmarking. The agency expects any eventual rules to be risk-based, with action-directing or agentic functions treated as higher risk than tools that only supply non-directive information.
Device makers, clinicians, and researchers are invited to weigh in by the deadline. For hospitals already piloting generative tools outside formal device pathways, the paper signals where scrutiny is heading.
