跳到正文
原文
MedTech Dive 器械媒体· Anuja Vaidya·· 27 天前AI 评分40

美国医院协会等机构建议将AI医疗设备监管责任归于开发者

Developers, not doctors, should be held accountable for generative AI medical devices, providers say

AI 导读

美国医院协会(AHA)、美国医师协会(AMA)和远程医疗协会(ATA)在向FDA提交的评论中建议,应要求开发者而非医疗机构承担基于生成式AI的医疗设备安全与性能监管责任。FDA计划于未来几个月发布针对AI医疗设备的监管草案,目前各方对风险评估框架存在分歧:AHA认为需纳入设备潜在错误概率等更多变量,AMA强调需评估人类监督的有效性,WEDI则提议建立医疗AI的警示标签制度和不良事件报告流程。

正文

National healthcare groups representing hospitals, health systems, physicians and telehealth companies are telling the government they don't want to be held responsible for the safety and performance of medical devices that use generative artificial intelligence, arguing that developers should shoulder that burden.  

In comments submitted to the Food and Drug Administration, the American Hospital Association, American Medical Association and American Telemedicine Association said that monitoring devices that use generative AI — including tools like chatbots and those that synthesize medical information — would place too heavy a load on providers, particularly small and rural practices.

Rather, developers should carry the responsibility of monitoring AI devices used in healthcare, and the FDA should create ways for providers to report problems with devices.

“Manufacturers should remain responsible for the safety and performance of the devices they sell,” according to comments submitted by ATA Action, the lobbying arm of the American Telemedicine Association.

As AI tools advance at warp speed, the healthcare industry is grappling with who’s responsible for the tools’ mistakes. For providers, taking on responsibility could increase their exposure to malpractice lawsuits, while tech companies could face their own negligence claims. Neither party is eager to take on responsibility for the tools. 

One problem, doctors say, is a lack of transparency into the AI models themselves. Hospitals told the FDA that, while health systems usually have established governance to review AI-backed devices, a lack of understanding into how the models actually work would make it hard for providers to identify flaws in the models.

“While hospitals and health systems support evaluation and monitoring activities, vendors and developers must be responsible for the ongoing performance, maintenance and integrity of the tools they market,” the AHA said.

The comments came in response to the FDA’s request for feedback from the industry on its proposed approach to regulating generative AI-enabled medical devices, which have unique characteristics compared with other types of AI. For instance, they accept open-ended inputs and produce different outputs to similar inputs, the FDA noted. Various trade associations, including the AHA, AMA, ATA and Workgroup for Electronic Data Interchange, submitted comments to the FDA.

Providers orgs question human supervision safeguard, ask for AI labels

Beyond the question of responsibility, provider groups generally agreed with other elements of the FDA's framework, like its proposal for assessing risk for generative AI-backed devices.

The agency is proposing to assess risk based on the actions a device performs.  For example, a device that provides information that doesn't require immediate action, such as a prediction of a future cardiovascular event, has limited consequences and is low-risk, whereas a device that makes decisions entirely on its own could have severe consequences.

Still, some groups had suggestions on how to tweak the framework.

The AHA noted that activity or action alone may not be enough to predict risk. Some activities carry several risks, such as AI tools that recommend emergency care, which could cause patients with non-emergency conditions to seek care, or fail to recommend emergency care to a patient who needs it. 

The framework should include more variables to account for these possibilities, such as the likelihood of error or negative outcomes from using a specific device, rather than only assessing the tool’s activity level: whether it is purely informational, results in an action or the action occurs under the supervision of a human clinician.

The AMA had recommendations on the human-supervision aspect of the framework, noting that having a human-in-the-loop — a commonly employed safeguard for AI tools — does not necessarily mitigate risk. Clinicians may make errors when reviewing AI suggestions, or they may become too reliant on the tools and may not be able to effectively oversee them, so evaluating the humans too should be part of the risk assessment, according to the AMA.

The discussion paper also describes approaches to evaluating the devices before they hit the market and monitoring them afterward.

WEDI further suggested that the FDA create a process for reporting adverse events and develop “nutrition” labels to give patients and clinicians insights into a tools’ functions and its potential risks. 

This is not the first time a nutrition label-type card has been suggested for health AI tools. The Biden administration had issued a rule for developers to create these types of cards, but the Trump administration rescinded it last year. 

Trust in health AI is critical to encourage adoption. Physicians say they want AI that is validated by a trusted entity and is continuously monitored. They also want a clear liability framework and more oversight of AI-enabled medical devices by governing bodies, like the FDA.

Physicians may get what they want soon enough. Rick Abramson, director of the FDA's Digital Health Center of Excellence, said at a conference earlier this week that the agency plans to post draft guidance on its approach to regulating generative AI-enabled medical devices in the coming months.

来源:MedTech Dive 器械媒体 · medtechdive.com