跳到正文
原文
Medical Xpress·· 1 小时前AI 评分30

医学研究代码共享存在不足:审查揭示文档缺口

Public code is not always reusable: Review uncovers gaps in medical study documentation

AI 导读

一项国际研究团队的审查发现,尽管医学研究代码共享不常见,公开代码并不意味着提供足够的文档和信息供他人重用。研究人员使用大型语言模型分析近4000篇论文,揭示代码共享和文档完整性存在显著差异,提出TRIPOD-Code指南以规范代码共享。

正文
AI helps researchers investigate how thousands of medical studies share code
There are two gaps in code repositories for medical studies: Code sharing remains uncommon, and making code public does not consistently mean providing the documentation and supporting information needed for reuse. Credit: Wing Ngan

The data and code that underpin a research study can tell us what was actually done, and how. In a medical study, for example, how were patients selected for inclusion? How were missing values handled? How were the results evaluated? Access to a study's data and code helps researchers to answer these questions, to examine the methods and to build on the work.

"Studies involve many choices about how to prepare data, develop models and evaluate performance. These are sometimes called 'researcher degrees of freedom,'" says Tom Pollard, a research scientist at MIT's Laboratory for Computational Physiology and Computer Science and Artificial Intelligence Laboratory. "The choices we make affect the results, and a methods section in a paper can only capture so much detail."

While sharing code can be a first step, in order for code to be understood and reused, it must be sufficiently structured and described. The problem is that, right now, there is little guidance for researchers on what constitutes a well-organized, well-described collection of code.

This is where an international team led by researchers at MIT and Dana-Farber Cancer Institute sought to help by examining how code is currently shared in clinical prediction studies. Such studies develop or evaluate models to estimate patient outcomes, such as the risk of developing a disease. Because these models inform decisions about patient care, researchers need to be able to scrutinize how they were built and tested.

Using large language models (LLMs), the researchers examined nearly 4,000 papers to identify statements about shared code. They then followed the statements to assess hundreds of associated code repositories.

Their review revealed two key gaps: Code sharing remains uncommon, and making code public does not consistently mean providing the documentation and supporting information needed for reuse. The open-access findings, recently published in Nature Medicine, are informing TRIPOD-Code, a reporting guideline being developed to clarify what researchers should share.

Following the code

TRIPOD-Code builds on TRIPOD and TRIPOD+AI, checklists that help researchers report how clinical prediction models were developed and evaluated. To understand current code-sharing practices, the team identified studies citing either guideline. The review included almost 4,000 studies that developed, updated or validated prediction models, and whose full texts were available through PubMed Central's open-access collection.

"The challenge was that the information was highly inconsistent. First, we had to identify whether or not a paper shared code," says Thomas Sounack, the study's first author. He adds that authors "discuss and link to code from a paper in many different ways."

The team used an LLM to help screen papers, identify statements about code availability and extract links. Software then retrieved the linked repositories—online collections of code and related files—for further assessment.

The workflow went beyond checking whether a link existed. It also assessed repositories against characteristics such as completeness of documentation, software requirements and terms of reuse.

"The real benefit of LLMs is their ability to process unstructured information such as text. These models helped us work with information that was scattered across papers and software repositories, rather than neatly organized in a table," says Pollard, who is a senior author of the study. "That allowed us to examine how code was being shared in much more detail across a large collection of studies."

Human reviewers evaluated the work of the LLMs. Two researchers independently checked 500 papers and a separate sample of repositories, resolving disagreements before comparing their assessments with the automated results.

Agreement was stronger for concrete characteristics, such as whether a license file was present, than for judgments about the adequacy of documentation. Because classification errors remain possible, the authors emphasize the overall patterns rather than treating every count as exact.

Inconsistent code sharing

Only about 1 in 8 studies reported sharing code. Where code was shared, the contents of repositories varied widely.

Some repositories were "extremely well organized and documented, while others gave almost no guidance at all," Sounack says. Good code sharing, he explains, means "giving enough information to understand what the code does and be able to run it to reproduce the results reported in the paper."

Practices also varied substantially across journals and countries. Among journals, the proportion of studies sharing code ranged from zero to more than 70%. Some journals with low sharing rates nevertheless had written code-sharing policies, suggesting that policies alone may not be enough to motivate code sharing.

The review also helped to identify examples of effective practice. Some researchers, for example, provided synthetic datasets to assist with code reuse where the original patient records could not be shared publicly.

Setting a standard for code sharing

Researchers who share code are already making an effort to make their work transparent. But clinical prediction research brings together people from medicine, statistics and computer science, whose expectations for documenting and sharing code may differ.

TRIPOD-Code aims to establish a consensus. Through an expert review process, the team is developing guidance on which elements of shared analytical code should be considered essential.

"We want to give researchers practical guidance on how to share code, and give journals clearer expectations to communicate to authors," Pollard says. "The review gives us evidence to inform those discussions."

Publication details

Thomas Sounack et al., AI-assisted scoping review of code sharing in clinical prediction model research, Nature Medicine (2026). DOI: 10.1038/s41591-026-04691-1

Journal information: Nature Medicine

Who's behind this story?

Lisa Lock

Lisa Lock

BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021. Full profile →

Robert Egan

Robert Egan

Bachelor's in mathematical biology, Master's in creative writing. Well-traveled with unique perspectives on science and language. Full profile →

This story is republished courtesy of MIT News (web.mit.edu/newsoffice/), a popular site that covers news about MIT research, innovation and teaching.

Citation: Public code is not always reusable: Review uncovers gaps in medical study documentation (2026, November 20) retrieved 20 November 2026 from https://medicalxpress.com/news/2026-11-code-reusable-uncovers-gaps-medical.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

来源:Medical Xpress · medicalxpress.com