五款大语言模型药物推荐集中于知名品牌,模拟显示处方采购成本或增1019%
AI's preference for drugs with a strong online presence may cost millions
日本大阪公立大学联合MD Anderson癌症中心测试五款大语言模型,发现其模拟选药时首选推荐平均97.5%集中于同一款产品,且多为品牌药。该研究发表于Nature Health,模型还倾向于不推荐较晚获FDA批准或临床更常用的产品,并出现含已停产产品的与监管信息不符的推荐。按美国数据模拟,若医生依模型推荐开药,年度采购成本将增加约5560万美元,相当于1019%。
Large language models (LLMs), such as ChatGPT and Claude, are increasingly being used to provide health information to the public. Although this is beneficial for improving access to treatment information, their limitations were revealed by a new study that found LLMs were biased in their treatment choices and that using these outputs in practice could drive up treatment costs.
A research team led by professor Daiju Ueda at Osaka Metropolitan University's Graduate School of Medicine, in collaboration with the University of Texas MD Anderson Cancer Center, asked five LLMs (GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Flash, Llama 4 and MedGemma) to recommend up to three FDA-approved drug products for a range of simulated scenarios.
Although an average of 5.5 FDA-approved products were available for each specified active ingredient, a single product accounted for an average of 97.5% of first-choice recommendations across the tested scenarios and models. The study is published in Nature Health.
"The recommendations were mostly for brand-name drugs, those highly visible in Google searches and treatments with dedicated Wikipedia pages," Ueda said.
The LLMs also tended not to recommend products with later FDA approval and those more commonly prescribed in actual physician practice. The study also identified recommendations that were inconsistent with regulatory information, including discontinued products.
The researchers then evaluated how much this could potentially cost patients using U.S. data. They looked at a scenario in which physicians prescribed drugs in accordance with the LLMs' recommendations.
"We found that annual acquisition costs would increase by approximately $55.6 million," Ueda summarized. "This represents a 1,019% increase in the cost of purchasing these prescriptions."
When instructed to prioritize lower-cost or generic products, the models shifted their recommendations accordingly; however, recommendations remained concentrated on a limited set of products.
"Our findings highlight the need to evaluate health care AI for biases in product selection and the handling of cost and regulatory information," Ueda said. "Combining AI systems with trusted drug information sources and human oversight may help support more appropriate medication choices."
Publication details
Junya Sato et al., Large language models show product name convergence in drug recommendations, Nature Health (2026). DOI: 10.1038/s44360-026-00205-y
Journal information: Nature Health
Key medical concepts
Who's behind this story?
Lisa Lock
BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021. Full profile →
Andrew Zinin
Master's in physics with research experience. Long-time science news enthusiast. Plays key role in Science X's editorial success. Full profile →
Citation: AI's preference for drugs with a strong online presence may cost millions (2026, October 27) retrieved 27 October 2026 from https://medicalxpress.com/news/2026-10-ai-drugs-strong-online-presence.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
来源:Medical Xpress · medicalxpress.com