算法辅助个性化风险沟通促进美国流感疫苗接种的三项随机现场试验
Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials
Geisinger团队在Nature Human Behaviour发表三项随机现场试验结果,探讨算法辅助个性化风险沟通对流感疫苗接种的影响。试验中高风险患者通过短信接收个性化风险等级及医疗记录信息,数据显示算法辅助沟通显著提升了特定风险组别的疫苗接种率,且效果在控制实际风险水平后依然稳健。
三项随机现场试验提供算法辅助风险沟通提升流感疫苗接种率的一手证据,读者可据此判断人工智能在公共卫生干预中的实际效果。
Data availability
De-identified data necessary to reproduce all results reported here are posted on OSF at https://osf.io/mqbux/. All data are posted at the individual level, except vaccination timing data (that is, vaccination dates) which are aggregated by experimental condition because dates of service are Protected Health Information under 45 CFR §164.514(b). Related documents available include study protocols and statistical analysis plans (available on ClinicalTrials.gov registrations, study 1: https://clinicaltrials.gov/ct2/show/NCT04323137; study 2: https://clinicaltrials.gov/ct2/show/NCT05009251; study 3: https://clinicaltrials.gov/ct2/show/NCT05509283; study 4: https://clinicaltrials.gov/ct2/show/NCT05509270).
Code availability
Code necessary to reproduce all results reported here is available via OSF at https://osf.io/mqbux/.
References
Bronsoler, A., Doyle, J. J. & Van Reenen, J. M. The impact of healthcare it on clinical quality, productivity and workers. SSRN Electron. J. 14, 23–46 (2021).
Rajpurkar, P., Chen, E., Banerjee, O. & Topol, E. J. AI in health and medicine. Nat. Med. 28, 31–38 (2022).
Goh, K. H. et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat. Commun. 12, 711 (2021).
Tomašev, N. et al. A clinically applicable approach to continuous prediction of future acute kidney injury. Nature 572, 116–119 (2019).
Young, A. T., Amara, D., Bhattacharya, A. & Wei, M. L. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review. Lancet Digit. Health 3, e599–e611 (2021).
Logg, J. M., Minson, J. A. & Moore, D. A. Algorithm appreciation: people prefer algorithmic to human judgment. Organ. Behav. Hum. Decis. Process. 151, 90–103 (2019).
Dietvorst, B. J., Simmons, J. P. & Massey, C. Algorithm aversion: people erroneously avoid algorithms after seeing them err. J. Exp. Psychol. Gen. 144, 114–126 (2015).
Longoni, C., Bonezzi, A. & Morewedge, C. K. Resistance to medical artificial intelligence. J. Consum. Res. 46, 629–650 (2019).
Fridman, A., Gershon, R. & Gneezy, A. COVID-19 and vaccine hesitancy: a longitudinal study. PLoS ONE 16, e0250123 (2021).
MacIntyre, C. R., Mahimbo, A., Moa, A. M. & Barnes, M. Influenza vaccine as a coronary intervention for prevention of myocardial infarction. Heart 102, 1953–1956 (2016).
Warren-Gash, C., Smeeth, L. & Hayward, A. C. Influenza as a trigger for acute myocardial infarction or death from cardiovascular disease: a systematic review. Lancet Infect. Dis. 9, 601–610 (2009).
People at increased risk for flu complications. Centers for Disease Control and Prevention https://www.cdc.gov/flu/highrisk/index.htm (2022).
Goeijenbier, M. et al. Benefits of flu vaccination for persons with diabetes mellitus: a review. Vaccine 35, 5095–5101 (2017).
Nichol, K. L. Efficacy and effectiveness of influenza vaccination. Vaccine 26, D17–D22 (2008).
Troeger, C. E. et al. Mortality, morbidity, and hospitalisations due to influenza lower respiratory tract infections, 2017: an analysis for the Global Burden of Disease Study 2017. Lancet Respir. Med. 7, 69–89 (2019).
Bonezzi, A., Ostinelli, M. & Melzner, J. The human black-box: the illusion of understanding human better than algorithmic decision-making. J. Exp. Psychol. Gen. 151, 2250–2258 (2022).
Cadario, R., Longoni, C. & Morewedge, C. K. Understanding, explaining, and utilizing medical artificial intelligence. Nat. Hum. Behav. 5, 1636–1642 (2021).
Pezzo, M. V. & Beckstead, J. W. Patients prefer artificial intelligence to a human provider, provided the AI is better than the human: a commentary on Longoni, Bonezzi and Morewedge (2019). Judgm. Decis. Mak. 15, 443–445 (2020).
Adadi, A. & Berrada, M. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018).
Miller, T. Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1–38 (2019).
Mercado, J. E. et al. Intelligent agent transparency in human-agent teaming for multi-UxV management. Hum. Factors 58, 401–415 (2016).
Stubbs, K., Wettergreen, D. & Hinds, P. J. Autonomy and common ground in human–robot interaction: a field study. IEEE Intell. Syst. 22, 42–50 (2007).
Benartzi, S. et al. Should governments invest more in nudging?. Psychol. Sci. 28, 1041–1055 (2017).
Patel, M. S., Volpp, K. G. & Asch, D. A. Nudge units to improve the delivery of health care. New Engl. J. Med. 378, 214–216 (2018).
Dai, H. et al. Behavioural nudges increase COVID-19 vaccinations. Nature 597, 404–409 (2021).
Milkman, K. L. et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc. Natl Acad. Sci. USA 118, 10–12 (2021).
Milkman, K. L. et al. A 680,000-person megastudy of nudges to encourage vaccination in pharmacies. Proc. Natl Acad. Sci. USA 119, e2115126119 (2022).
Santos, H. C., Goren, A., Chabris, C. F. & Meyer, M. N. Effect of targeted behavioral science messages on COVID-19 vaccination registration among employees of a large health system: a randomized trial. JAMA Netw. Open 4, 5–8 (2021).
Schneider, F. H. et al. Financial incentives for vaccination do not have negative unintended consequences. Nature 613, 526–533 (2023).
Yokum, D., Lauffenburger, J. C., Ghazinouri, R. & Choudhry, N. K. Letters designed with behavioural science increase influenza vaccination in Medicare beneficiaries. Nat. Hum. Behav. 2, 743–749 (2018).
Chen, N., Trump, K.-S., Hall, S. & Le, Q. The effect of postcard reminders on vaccinations among the elderly: a block-randomized experiment. Behav. Public Policy 7, 240–265 (2020).
Rabb, N. et al. Evidence from a statewide vaccination RCT shows the limits of nudges. Nature 604, E1–E7 (2022).
Szilagyi, P. G. et al. Effect of personalized messages sent by a health system’s patient portal on influenza vaccination rates: a randomized clinical trial. J. Gen. Intern. Med. 37, 615–623 (2022).
Modin, D. et al. Effect of electronic nudges on influenza vaccination rate in older adults with cardiovascular disease: prespecified analysis of the NUDGE-FLU trial. Circulation 147, 1345–1354 (2023).
Johansen, N. D. et al. Electronic nudges to increase influenza vaccination uptake among patients with heart failure: a prespecified analysis of the NUDGE-FLU trial. Eur. J. Heart Fail. 25, 1450–1458 (2023).
Brewer, N. T., Chapman, G. B., Rothman, A. J., Leask, J. & Kempe, A. Increasing vaccination: putting psychological science into action. Psychol. Sci. Public Interest 18, 149–207 (2017).
Sheeran, P., Harris, P. R. & Epton, T. Does heightening risk appraisals change people’s intentions and behavior? A meta-analysis of experimental studies. Psychol. Bull. 140, 511–543 (2014).
Motta, M., Sylvester, S., Callaghan, T. & Lunz-Trujillo, K. Encouraging COVID-19 vaccine uptake through effective health communication. Front. Polit. Sci. 3, 630133 (2021).
Reñosa, M. D. C. et al. Nudging toward vaccination: a systematic review. BMJ Glob. Health 6, e006237 (2021).
French, D. P., Cameron, E., Benton, J. S., Deaton, C. & Harvie, M. Can communicating personalised disease risk promote healthy behaviour change? A systematic review of systematic reviews. Ann. Behav. Med. 51, 718–729 (2017).
Schulberg, S. D. et al. Cardiovascular risk communication strategies in primary prevention. A systematic review with narrative synthesis. J. Adv. Nurs. 78, 3116–3140 (2022).
Brewer, N. T. What works to increase vaccination uptake. Acad. Pediatr. 21, S9–S16 (2021).
Tuckerman, J. et al. Short message service reminder nudge for parents and influenza vaccination uptake in children and adolescents with special risk medical conditions. JAMA Pediatr. 177, 337–344 (2023).
Johansen, N. D. et al. Electronic nudges to increase influenza vaccination uptake in Denmark: a nationwide, pragmatic, registry-based, randomised implementation trial. Lancet 401, 1103–1114 (2023).
Sääksvuori, L. et al. Information nudges for influenza vaccination: evidence from a large-scale cluster-randomized controlled trial in Finland. PLoS Med. 19, e1003919 (2022).
Wolk, D. M. et al. Prediction of influenza complications: development and validation of a machine learning prediction model to improve and expand the identification of vaccine-hesitant patients at risk of severe influenza complications. J. Clin. Med. 11, 4342 (2022).
La Macchia, S. T., Louis, W. R., Hornsey, M. J. & Leonardelli, G. J. In small we trust: lay theories about small and large groups. Pers. Soc. Psychol. Bull. 42, 1321–1334 (2016).
Rudert, S. C., Reutner, L., Greifeneder, R. & Walker, M. Faced with exclusion: perceived facial warmth and competence influence moral judgments of social exclusion. J. Exp. Soc. Psychol. 68, 101–112 (2017).
Clark, J. K., Thiemb, K. C., Hoover, A. E. & Habashi, M. M. Gender stereotypes and intellectual performance: stigma consciousness as a buffer against stereotype validation. J. Exp. Soc. Psychol. 68, 185–191 (2017).
Mann, T. C. & Ferguson, M. J. Reversing implicit first impressions through reinterpretation after a two-day delay. J. Exp. Soc. Psychol. 68, 122–127 (2017).
Filiz, I., Judek, J. R., Lorenz, M. & Spiwoks, M. The extent of algorithm aversion in decision-making situations with varying gravity. PLoS ONE 18, e0278751 (2023).
Waters, E. A. et al. Translating cancer risk prediction models into personalized cancer risk assessment tools: stumbling blocks and strategies for success. Cancer Epidemiol. Biomarkers Prev. 29, 2389–2394 (2020).
Hasselblad, V. & Hedges, L. V. Meta-analysis of screening and diagnostic tests. Psychol. Bull. 117, 167–178 (1995).
Castelo, N., Bos, M. W. & Lehmann, D. R. Task-dependent algorithm aversion. J. Mark. Res. 56, 809–825 (2019).
Saccardo, S. et al. Field testing the transferability of behavioural science knowledge on promoting vaccinations. Nat. Hum. Behav. 8, 878–890 (2024).
DellaVigna, S. & Linos, E. RCTs to scale: comprehensive evidence from two nudge units. Econometrica 90, 81–116 (2022).
Service, O. et al. EAST: four simple ways to apply behavioural insights. Annu. Rev. of Policy Des. 5, 1–53 (2017).
Jacobson, M., Chang, T. Y., Shah, M., Pramanik, R. & Shah, S. B. Can financial incentives and other nudges increase COVID-19 vaccinations among the vaccine hesitant? A randomized trial. Vaccine 40, 6235–6242 (2022).
Campos-Mercade, P. et al. Monetary incentives increase COVID-19 vaccinations. Science 374, 879–882 (2021).
Bryan, C. J., Tipton, E. & Yeager, D. S. Behavioural science is unlikely to change the world without a heterogeneity revolution. Nat. Hum. Behav. 5, 980–989 (2021).
Brody, I. et al. Targeting behavioral interventions based on past behavior: evidence from vaccine uptake. Organ. Behav. Hum. Decis. Process. 192, 104465 (2026).
de Ridder, D., Kroese, F. & van Gestel, L. Nudgeability: mapping conditions of susceptibility to nudge influence. Perspect. Psychol. Sci. 17, 346–359 (2022).
Chater, N. & Loewenstein, G. The i-frame and the s-frame: how focusing on individual-level solutions has led behavioral public policy astray. Behav. Brain Sci. 46, e147 (2022).
Funk, C., Hefferon, M., Kennedy, B. & Johnson, C. Trust and Mistrust in Americans’ Views of Scientific Experts. https://www.pewresearch.org/science/2019/08/02/trust-and-mistrust-in-americans-views-of-scientific-experts/ (Pew Research Center, 2019).
Pasquini, G., Stocking, G., Kikuchi, E., Pula, I. & Kirzinger, A. Where Do Americans Get Health Information, and What Do They Trust? https://www.pewresearch.org/science/2026/04/07/where-do-americans-get-health-information-and-what-do-they-trust/ (Pew Research Center, 2026).
Saccardo, S. et al. Scaling Nudges: Who Moves and How. https://doi.org/10.2139/ssrn.3971192 (SSRN, 2025).
R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing https://www.r-project.org (2021).
Coppock, A. randomizr: easy-to-use tools for common forms of random assignment and sampling. Comprehensive R Archive Network (CRAN) https://cran.r-project.org/web/packages/randomizr/index.html (2024).
Colling, L. J. bayesplay: the bayes factor playground. R package version 0.9.3. GitHub https://github.com/bayesplay/bayesplay (2023).
Rich, B. table1: tables of descriptive statistics in HTML. CRAN: Contributed Packages https://doi.org/10.32614/CRAN.package.table1 (2025).
Wickham, H. stringr: simple, consistent wrappers for common string operations. Comprehensive R Archive Network (CRAN) https://cran.r-project.org/web/packages/stringr/index.html (2025).
Lüdecke, D. sjPlot: data visualization for statistics in social science. Comprehensive R Archive Network (CRAN) https://CRAN.R-project.org/package=sjPlot (2025).
Coppock, A. randomizr: easy-to-use tools for common forms of random assignment and sampling. Comprehensive R Archive Network (CRAN) https://cran.r-project.org/web/packages/randomizr/index.html (2025).
Champely, S. pwr: basic functions for power analysis. GitHub https://github.com/heliosdrm/pwr (2023).
Wickham, H. The split-apply-combine strategy for data analysis. J. Stat. Softw. 40, 1–29 (2011).
Sievert, C. Interactive Web-Based Data Visualization with R, Plotly, and Shiny (CRC, 2020).
Rizopoulos, D. ltm: an R package for latent variable modeling and item response theory analyses. J. Stat. Softw. 17, 1–25 (2006).
Grolemund, G. & Wickham, H. Dates and times made easy with {lubridate}. J. Stat. Softw. 40, 1–25 (2011).
Long, J. A. jtools: analysis and presentation of social scientific data. J. Open Source Softw. 9, 6610 (2024).
Hugh-Jones, D. ggmagnify: create a magnified inset of part of a ‘Ggplot’ object. GitHub https://github.com/hughjonesd/ggmagnify (2026).
Wickham, H. et al. Welcome to the Tidyverse. J. Open Source Softw. 4, 1686 (2019).
Lenth, R. V. & Piaskowski, J. emmeans: estimated marginal means, aka least-squares means. Comprehensive R Archive Network (CRAN) https://cran.r-project.org/web/packages/emmeans/index.html (2026).
Gomila, R. Logistic or linear? Estimating causal effects of experimental treatments on binary outcomes using regression analysis. J. Exp. Psychol. Gen. 150, 700–709 (2021).
Hellevik, O. Linear versus logistic regression when the dependent variable is a dichotomy. Qual. Quant. 43, 59–74 (2009).
Wetzels, R. et al. Statistical evidence in experimental psychology: an empirical comparison using 855 t tests. Perspect. Psychol. Sci. 6, 291–298 (2011).
Dienes, Z. Using Bayes to get the most out of non-significant results. Front. Psychol. 5, 781 (2014).
Acknowledgements
We thank A. Lanyado, R. Yesharim, E. Hertzman, E. Choman, C. Metzger, E. Cohen and T. Sagiv (originally at Medial EarlySign) for discussions about study design and data analysis in relation to the AI algorithm, and for their assistance in generating reasons that patients were considered to be high risk. We thank S. Martin and K. Murphy for their invaluable support of this work, C. Walter and P. Juthani for research assistance, and S. Brietzke for help de-identifying survey data. We also thank Geisinger Marketing for help preparing and sending the messages. Finally, we thank members of Geisinger Business Intelligence and Advanced Analytics, especially C. Hartranft, G. Strevig, C. Cauthorn and E. Reich, for data assistance, and the Geisinger Data Core and R. Maff for their assistance liaising between Geisinger and Medial EarlySign.
Funding
Studies 1 and 2 were supported by the National Institute on Aging of the National Institutes of Health under award number P30AG034532 (which provided salary support to G.M.R., A.G., M.S., J.J.D., M.N.M. and C.F.C.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the paper, beyond approval of the initially proposed designs for the field trials.
Author information
Author notes
These authors contributed equally: Michelle N. Meyer, Christopher F. Chabris.
These authors jointly supervised this work: Michelle N. Meyer, Christopher F. Chabris.
Authors and Affiliations
Behavioral Insights Team, Geisinger, Danville, PA, USA
Gail M. Rosenbaum, Amir Goren, Maheen Shermohammed, Michelle N. Meyer & Christopher F. Chabris
Department of Laboratory Medicine, Diagnostic Medicine Institute, Geisinger, Danville, PA, USA
Donna M. Wolk & Ann Marie Tice
Geisinger Commonwealth School of Medicine, Danville, PA, USA
Donna M. Wolk
Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA
Joseph J. Doyle Jr.
National Bureau of Economic Research, Cambridge, MA, USA
Joseph J. Doyle Jr.
Department of Bioethics and Decision Sciences, Geisinger, Danville, PA, USA
Michelle N. Meyer & Christopher F. Chabris
Authors
- Gail M. Rosenbaum
- Amir Goren
- Maheen Shermohammed
- Donna M. Wolk
- Ann Marie Tice
- Joseph J. Doyle Jr.
- Michelle N. Meyer
- Christopher F. Chabris
Contributions
G.M.R., A.G., M.S., J.J.D., M.N.M. and C.F.C. conceptualized the project; J.J.D., M.N.M. and C.F.C. acquired funding; G.M.R., M.S., A.G., A.M.T., D.M.W., J.J.D., M.N.M. and C.F.C. designed methodology; G.M.R., M.S. and A.G. performed the investigations and administrated the project; G.M.R. and M.S. curated the data, undertook formal analysis and visualized the data; A.G., J.J.D., M.N.M. and C.F.C. supervised the work; G.M.R., A.G. and M.S. wrote the original draft; G.M.R., M.S., A.G., A.M.T., D.M.W., J.J.D., M.N.M. and C.F.C. contributed to the review and editing of the paper.
Corresponding author
Correspondence to Christopher F. Chabris.
Ethics declarations
Competing interests
Geisinger paid Medial EarlySign to generate the risk scores and reasons used in the studies. The authors declare no other competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Mark Pezzo and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Vaccination rate differences in Study 1 by actual and communicated risk levels.
Differences in the Study 1 primary outcome are shown separately by actual risk level, communicated risk level, and study arm. The vertical dashed line at 0% represents the overall passive control vaccination rate across risk level of 50.8% shown in Fig. 1b. All patients who were sent messages in the top 3% of risk were told they were “in the top 3% of risk.” Patients in the top 4–10% of risk were randomized to be told they were “in the top 10% of risk” or that they were “at high risk.” Vaccination rates in the passive control group by risk level are shown for comparison. Sample sizes for the groups from top to bottom are 2,906, 2,867, 2,855, 2,863, 6,734, 3,378, 3,452, 3,377, 3,389, and 3,380. Error bars indicate 95% confidence intervals.
Extended Data Fig. 2 Study 2 vaccination timing.
The number of patients vaccinated by study arm for each business day relative to the intervention start date overlayed with local-linear regression lines estimated separately before (8/1/2021–9/8/2021) and on or after (9/9/2021–11/5/2021) the intervention start date. Send dates for each modality are indicated by the vertical, dotted lines. Patients in experimental arms who were eligible for SMS received either an early SMS or a late SMS (see Methods for details). Sample sizes in each arm: passive control n = 8,466, active control n = 8,522, high risk only n = 8,498, high risk medical records n = 8,470, and high risk algorithm n = 8,525.
Extended Data Fig. 3 Vaccination rate differences in Study 2 by actual and communicated risk levels.
Differences in the Study 2 primary outcome separately by actual risk level, communicated risk level, and study arm. The vertical dashed line at 0% represents the overall passive control vaccination rate across risk level of 31.5% shown in Fig. 2. Passive control vaccination rates by risk level are displayed for comparison with active arms. Sample sizes in each group, from top to bottom: 2,565, 2,581, 2,551, 2,555, 2,561, 5,901, 5,941, 5,947, 5,915, and 5,964. Error bars indicate 95% confidence intervals.
Extended Data Fig. 4 Study 3 vaccination timing.
The number of patients vaccinated by study arm for each business day relative to the intervention start date. The intervention ran from 9/13/2022 through 10/25/2022 for patients in the active control and algorithm arms and 9/14/2022 through 10/26/2022 for patients in the high risk only and high risk medical records arms. Those in the passive control arm were randomized to one of those two timeframes (see Methods for more details). The plot includes local-linear regression lines estimated separately before (8/1/2022–9/12/2022 or 8/1/2022–9/13/2022) and on or after (9/13/2022–10/25/2022 or 9/14/2022–10/26/2022) the intervention start date. Send dates for each modality are indicated by the vertical, dotted lines. Sample sizes in each arm: passive control n = 7,823, active control n = 7,840, high risk only n = 7,809, high risk medical records n = 7,797, and high risk algorithm n = 7,860.
Extended Data Fig. 5 Vaccination rate differences in Study 3 by actual and communicated risk levels.
Study 3 primary outcome separately by actual risk level, communicated risk level, and study arm. The vertical dashed line at 0% represents the overall passive control vaccination rate across risk level of 23.8%. All patients in the high-risk arms were told they were in the top 20% of risk. Vaccination rates in the passive control group are shown for comparison with active arms, stratified by risk level. Sample sizes in each group from top to bottom are 3,917, 3,930, 3,954, 3,848, 3,862, 3,906, 3,910, 3,855, 3,949, and 3,998. Error bars indicate 95% confidence intervals.
Extended Data Fig. 6 Power sensitivity analysis.
Power for a range of effect sizes for each study, given a priori assumptions. Studies were powered to detect 2.0 percentage-point minimum detectable effects (MDEs), consistent with previous nudge literature. In study 1, our preregistered power analysis revealed 92% power to detect this effect size, but 80% power is also shown for consistency with studies 2 and 3, where we had 80% power to detect a 2 percentage-point effect.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rosenbaum, G.M., Goren, A., Shermohammed, M. et al. Algorithm-assisted personalized risk communication to encourage flu vaccination in the USA: three randomized field trials. Nat Hum Behav (2026). https://doi.org/10.1038/s41562-026-02603-4
Received: 08 September 2023
Accepted: 03 September 2026
Published: 05 October 2026
Version of record: 05 October 2026
DOI: https://doi.org/10.1038/s41562-026-02603-4
来源:Nature 健康护理主题 · nature.com