美国医院与医生团体呼吁FDA将生成式AI医疗器械责任归于开发者
Developers, not doctors, should be held accountable for generative AI medical devices, providers say
美国医院协会、美国医学会和美国远程医疗协会等团体向FDA提交意见,主张生成式AI医疗器械的安全与性能责任应由开发者而非医疗机构承担。这些团体认为监测此类器械会给小型和农村诊所带来过重负担,并建议FDA建立问题上报渠道。FDA数字健康卓越中心主任Rick Abramson表示,该机构计划在未来几个月发布针对生成式AI医疗器械的指南草案。
This audio is auto-generated. Please let us know if you have feedback.
National healthcare groups representing hospitals, health systems, physicians and telehealth companies are telling the government they don't want to be held responsible for the safety and performance of medical devices that use generative AI, arguing that developers should shoulder that burden.
In comments submitted to the Food and Drug Administration, the American Hospital Association, American Medical Association and American Telemedicine Association said that monitoring devices that use generative AI — including tools like chatbots and those that synthesize medical information — would place too heavy a load on providers, particularly small and rural practices.
Rather, developers should carry the responsibility of monitoring AI devices used in healthcare, and the FDA should create ways for providers to report problems with devices.
“Manufacturers should remain responsible for the safety and performance of the devices they sell,” according to comments submitted by ATA Action, the lobbying arm of the American Telemedicine Association.
As AI tools advance at warp speed, the healthcare industry is grappling with who’s responsible for the tools’ mistakes. For providers, taking on responsibility could increase their exposure to malpractice lawsuits, while tech companies could face their own negligence claims. Neither party is eager to take on responsibility for the tools.
One problem, doctors say, is a lack of transparency into the AI models themselves. Hospitals told the FDA that, while health systems usually have established governance to review AI-backed devices, a lack of understanding into how the models actually work would make it hard for providers to identify flaws in the models.
“While hospitals and health systems support evaluation and monitoring activities, vendors and developers must be responsible for the ongoing performance, maintenance and integrity of the tools they market,” the AHA said.
The comments came in response to the FDA’s request for feedback from the industry on its proposed approach to regulating generative AI-enabled medical devices, which have unique characteristics compared to other types of AI. For instance, they accept open-ended inputs and produce different outputs to similar inputs, the FDA noted. Various trade associations, including the AHA, AMA, ATA and Workgroup for Electronic Data Interchange, submitted comments to the FDA.
Providers orgs question human supervision safeguard, ask for AI labels
Beyond the question of responsibility, provider groups generally agreed with other elements of the FDA's framework, like its proposal for assessing risk for generative AI-backed devices.
The agency is proposing to assess risk based on the actions a device performs. For example, a device that provides information that doesn't require immediate action, such as a prediction of a future cardiovascular event, has limited consequences and is low-risk, whereas a device that makes decisions entirely on its own could have severe consequences.
Still, some groups had suggestions on how to tweak the framework.
The AHA noted that activity or action alone may not be enough to predict risk. Some activities carry several risks, such as AI tools that recommend emergency care, which could cause patients with non-emergency conditions to seek care, or fail to recommend emergency care to a patient who needs it.
The framework should include more variables to account for these possibilities, such as the likelihood of error or negative outcomes from using a specific device, rather than only assessing the tool’s activity level: whether it is purely informational, results in an action or the action occurs under the supervision of a human clinician.
The AMA had recommendations on the human-supervision aspect of the framework, noting that having a human-in-the-loop – a commonly employed safeguard for AI tools – does not necessarily mitigate risk. Clinicians may make errors when reviewing AI suggestions, or they may become too reliant on the tools and may not be able to effectively oversee them, so evaluating the humans too should be part of the risk assessment, according to the AMA.
The discussion paper also describes approaches to evaluating the devices before they hit the market and monitoring them afterward.
WEDI further suggested that the FDA create a process for reporting adverse events and develop “nutrition” labels to give patients and clinicians insights into a tools’ functions and its potential risks.
This is not the first time a nutrition label-type card has been suggested for health AI tools. The Biden administration had issued a rule for developers to create these types of cards, but the Trump administration rescinded it last year.
Trust in health AI is critical to encourage adoption. Physicians say they want AI that is validated by a trusted entity and is continuously monitored. They also want a clear liability framework and more oversight of AI-enabled medical devices by governing bodies, like the FDA.
Physicians may get what they want soon enough. Rick Abramson, director of the FDA's Digital Health Center of Excellence, said at a conference earlier this week that the agency plans to post draft guidance on its approach to regulating generative AI-enabled medical devices in the coming months.
来源:Healthcare Dive · healthcaredive.com