Journal of Science Policy & Governance
|
Volume 21, Issue 03 | January 23, 2022
|
Policy Brief: Promoting Health Equity through Improved Regulation of Artificial Intelligence Medical Devices
Kristina Dortche (1,2), Grace McCarthy (3,4), Sara Banbury (1), Isabel Yannatos (5)
Corresponding author: [email protected] |
Keywords: artificial intelligence; healthcare; equity; FDA; SaMD regulation; medical device
Executive Summary
Existing health disparities in the United States are partially driven by the way healthcare is delivered. There is interest in using Artificial Intelligence (AI)-driven software as medical devices (SaMD) to aid in healthcare delivery and reduce health disparities. However, AI-driven tools have the potential to codify bias in healthcare settings. Some AI-driven SaMDs have displayed substandard performance among racial and ethnic minorities. Auditing these tools for biased output can help produce more equitable outcomes across populations. However, there are currently no explicit Food and Drug Administration (FDA) regulations that examine bias in AI software in healthcare. Therefore, we propose the FDA develop a distinct regulatory process for AI-driven SaMDs that includes assessing equitable output across populations and avoiding potential health disparity exacerbation. This change could help prevent AI-driven health disparities nationwide.
-Read the full article through download.-
Background header image courtesy of The Sunday Guardian
Acknowledgements
We thank Victor Acero, University of Pennsylvania, for their assistance drafting and editing this manuscript. We also thank the Penn Science Policy and Diplomacy Group for providing the opportunity and instruction to work together.
Kristina Dortche Kristina Dortche is completing residency in Urology at the Cleveland Clinic Foundation in Cleveland, OH. She obtained her MD at the Perelman School of Medicine where she was involved in developing coursework and national conferences highlighting disparities in health and healthcare. She also earned her MPH at Columbia Mailman School of Public Health in Health Policy and Management. She hopes to use her training to identify and address health disparities that plague pediatric patients in Urology.
Grace McCarthy Dr. Grace McCarthy is a postdoctoral scholar at Oregon Health & Science University, where she obtained her Ph.D. in Cancer Biology. She works with groups such as Alliance for Visible Diversity in Science, the National Postdoctoral Association, and her department’s DEI Committee to advocate for diverse, equitable, and safe research environments. She also serves as the Mission Chair for the Portland Affiliate of the Pancreatic Cancer Action Network, overseeing pancreatic cancer education, outreach, and advocacy. Dr. McCarthy is dedicated to increasing access and equity in healthcare.
Sara Banbury Sara Banbury is a third year medical student at the Perelman School of Medicine at the University of Pennsylvania. She has previous experience in community organizing and education, and has worked with Students Opposing Racism in Medicine during her time at medical school. She is interested in the intersection of medicine and advocacy, specifically increasing physician advocacy regarding issues that affect their patient populations. She is currently pursuing her MD, and hopes to integrate advocacy and health equity work into her future career.
Isabel Yannatos Isabel Yannatos is a Ph.D. candidate in biomedical studies at the University of Pennsylvania. She works in Dr. Corey McMillan’s lab on racial disparities in aging. Her work focuses on how environmental exposures, such as neighborhood resources and air pollution, contribute to disparities in epigenetic aging and cognitive outcomes. Isabel is passionate about the intersection of research and policy in addressing health disparities. She is pursuing a certificate in public health and cognitive aging and other opportunities in health policy and advocacy. In her career she hopes to translate research into action to promote health equity.
We thank Victor Acero, University of Pennsylvania, for their assistance drafting and editing this manuscript. We also thank the Penn Science Policy and Diplomacy Group for providing the opportunity and instruction to work together.
Kristina Dortche Kristina Dortche is completing residency in Urology at the Cleveland Clinic Foundation in Cleveland, OH. She obtained her MD at the Perelman School of Medicine where she was involved in developing coursework and national conferences highlighting disparities in health and healthcare. She also earned her MPH at Columbia Mailman School of Public Health in Health Policy and Management. She hopes to use her training to identify and address health disparities that plague pediatric patients in Urology.
Grace McCarthy Dr. Grace McCarthy is a postdoctoral scholar at Oregon Health & Science University, where she obtained her Ph.D. in Cancer Biology. She works with groups such as Alliance for Visible Diversity in Science, the National Postdoctoral Association, and her department’s DEI Committee to advocate for diverse, equitable, and safe research environments. She also serves as the Mission Chair for the Portland Affiliate of the Pancreatic Cancer Action Network, overseeing pancreatic cancer education, outreach, and advocacy. Dr. McCarthy is dedicated to increasing access and equity in healthcare.
Sara Banbury Sara Banbury is a third year medical student at the Perelman School of Medicine at the University of Pennsylvania. She has previous experience in community organizing and education, and has worked with Students Opposing Racism in Medicine during her time at medical school. She is interested in the intersection of medicine and advocacy, specifically increasing physician advocacy regarding issues that affect their patient populations. She is currently pursuing her MD, and hopes to integrate advocacy and health equity work into her future career.
Isabel Yannatos Isabel Yannatos is a Ph.D. candidate in biomedical studies at the University of Pennsylvania. She works in Dr. Corey McMillan’s lab on racial disparities in aging. Her work focuses on how environmental exposures, such as neighborhood resources and air pollution, contribute to disparities in epigenetic aging and cognitive outcomes. Isabel is passionate about the intersection of research and policy in addressing health disparities. She is pursuing a certificate in public health and cognitive aging and other opportunities in health policy and advocacy. In her career she hopes to translate research into action to promote health equity.
References
- Adamson, Adewole S., and Avery Smith. 2018. “Machine Learning and Health Care Disparities in Dermatology.” JAMA Dermatology 154 (11): 1247–48. https://doi.org/10.1001/jamadermatol.2018.2348.
- “Aiming for Truth, Fairness, and Equity in Your Company’s Use of AI.” 2021. Federal Trade Commission. April 19, 2021. http://www.ftc.gov/business-guidance/blog/2021/04/aiming-truth-fairness-equity-your-companys-use-ai.
- Amisha, Paras Malik, Monika Pathania, and Vyas Kumar Rathaur. 2019. “Overview of Artificial Intelligence in Medicine.” Journal of Family Medicine and Primary Care 8 (7): 2328–31. https://doi.org/10.4103/jfmpc.jfmpc_440_19.
- “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”. 2021. Accessed May 3, 2022. PDF file. https://www.fda.gov/media/145022/download
- Aydın, Mustafa, Fırat Hardalaç, Berkan Ural, and Serhat Karap. “Neonatal Jaundice Detection System.” Journal of Medical Systems 40, no. 7 (July 2016): 166. https://doi.org/10.1007/s10916-016-0523-4.
- Berry, Jay G., Bloom Sheila, Foley Susan, Palfrey Judith S. 2010. “Health Inequity in Children and Youth With Chronic Health Conditions”. Pediatrics 126, no. 3 (December 2010): S111-S119. https://publications.aap.org/pediatrics/article-abstract/126/Supplement_3/S111/29999/Health-Inequity-in-Children-and-Youth-With-Chronic?redirectedFrom=fulltext
- Briganti, Giovanni, and Olivier Le Moine. 2020. “Artificial Intelligence in Medicine: Today and Tomorrow.” Frontiers in Medicine 7. https://www.frontiersin.org/article/10.3389/fmed.2020.00027.
- Chen, Mei, and Michel Decary. “Artificial Intelligence in Healthcare: An Essential Guide for Health Leaders.” Healthcare Management Forum 33, no. 1 (January 2020): 10–18. https://doi.org/10.1177/0840470419873123.
- ———. “Embedding Health Literacy Tools in Patient EHR Portals to Facilitate Productive Patient Engagement.” Studies in Health Technology and Informatics 257 (2019): 59–63.
- Commissioner, Office of the. 2022. “Robert Califf.” U.S. Food and Drug Administration. March 10, 2022. https://www.fda.gov/about-fda/fda-organization/robert-califf.
- European Commission. “Proposal For a Regulation of The European Parliament and of The Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts”. (2021). https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206
- FDA. "Artificial Intelligence/Machine Learning (Ai/Ml)-Based Software as a Medical Device (Samd) Action Plan." (2021). https://www.fda.gov/media/145022/download
- FDA. "Developing the Software Precertification Program: Summary of Learnings and Ongoing Activities." (2020). https://www.fda.gov/media/142107/download
- Flores, Laura E., Walter R. Frontera, Michele P. Andrasik, Carlos Del Rio, Antonio Mondríguez-González, Stephanie A. Price, Elizabeth M. Krantz, Steven A. Pergam, and Julie K. Silver. 2021. “Assessment of the Inclusion of Racial/Ethnic Minority, Female, and Older Individuals in Vaccine Clinical Trials.” JAMA Network Open 4 (2): e2037640. https://doi.org/10.1001/jamanetworkopen.2020.37640.
- Gianfrancesco, Milena A., Suzanne Tamang, Jinoos Yazdany, and Gabriela Schmajuk. “Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data.” JAMA Internal Medicine 178, no. 11 (November 1, 2018): 1544–47. https://doi.org/10.1001/jamainternmed.2018.3763.
- Health, Center for Devices and Radiological. “Premarket Notification 510(k).” FDA. FDA, October 3, 2022. https://www.fda.gov/medical-devices/premarket-submissions-selecting-and-preparing-correct-submission/premarket-notification-510k..
- ———. “De Novo Classification Request.” FDA, October 3, 2022. https://www.fda.gov/medical-devices/premarket-submissions-selecting-and-preparing-correct-submission/de-novo-classification-request.
- ———. “Premarket Approval (PMA).” FDA. FDA, October 3, 2022.
- https://www.fda.gov/medical-devices/premarket-submissions-selecting-and-preparing-correct-submission/premarket-approval-pma.
- ———. “Digital Health Software Precertification (Pre-Cert) Pilot Program.” FDA. FDA, March 8, 2022. https://www.fda.gov/medical-devices/digital-health-center-excellence/digital-health-software-precertification-pre-cert-pilot-program.
- Hoffman, Sharona, and Andy Podgurski. "Artificial Intelligence and Discrimination in Health Care." Yale J. Health Pol'y L. & Ethics 19 (2019): 1. https://openyls.law.yale.edu/bitstream/handle/20.500.13051/5964/Hoffman_v19n3_1_49.pdf?sequence=2&isAllowed=y#page=13
- Hoodbhoy, Zahra, Sarah Masroor Jeelani, Abeer Aziz, Muhammad Ibrahim Habib, Bilal Iqbal, Waqaas Akmal, Khan Siddiqui, Babar Hasan, Mariska Leeflang, and Jai K. Das. “Machine Learning for Child and Adolescent Health: A Systematic Review.” Pediatrics 147, no. 1 (January 1, 2021): e2020011833. https://doi.org/10.1542/peds.2020-011833.
- “How FDA Regulates Artificial Intelligence in Medical Products.” n.d. Accessed May 14, 2022. https://pew.org/3yglbCS.
- LaVeist, Thomas A., Darrell Gaskin, and Patrick Richard. 2011. “Estimating the Economic Burden of Racial Health Inequalities in the United States.” International Journal of Health Services: Planning, Administration, Evaluation 41 (2): 231–38. https://doi.org/10.2190/HS.41.2.c.
- Leslie, David, Anjali Mazumder, Aidan Peppin, Maria K. Wolters, and Alexa Hagerty. “Does ‘AI’ Stand for Augmenting Inequality in the Era of Covid-19 Healthcare?” BMJ 372 (March 16, 2021): n304. https://doi.org/10.1136/bmj.n304.
- “Minority Population Profiles - The Office of Minority Health.” 2021. https://minorityhealth.hhs.gov/omh/browse.aspx?lvl=2&lvlID=26.
- Muehlematter, Urs J., Paola Daniore, and Kerstin N. Vokinger. 2021. “Approval of Artificial Intelligence and Machine Learning-Based Medical Devices in the USA and Europe (2015–20): A Comparative Analysis.” The Lancet Digital Health 3 (3): e195–203. doi.org/10.1016/S2589-7500(20)30292-2
- Murphy, Sherry L, Kochanek, Kenneth, Xu, Jiaquan, Arias, Elizabeth. 2021. “Mortality in the United States, 2020”. NCHS Data Brief 427: 2. https://www.cdc.gov/nchs/data/databriefs/db427.pdf
- Obermeyer, Ziad, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.” Science (New York, N.Y.) 366 (6464): 447–53. https://doi.org/10.1126/science.aax2342.
- Officer (OCIO), Office of the Chief Information. 2021. “HHS Artificial Intelligence (AI) Strategy.” Text. HHS.Gov. December 22, 2021. https://www.hhs.gov/about/agencies/asa/ocio/ai/strategy/index.html.
- Ong, Mei-Sing, Mary P. Mullen, Eric D. Austin, Peter Szolovits, Marc D. Natter, Alon Geva, Tianxi Cai, Sek Won Kong, and Kenneth D. Mandl. “Learning a Comorbidity-Driven Taxonomy of Pediatric Pulmonary Hypertension.” Circulation Research 121, no. 4 (August 4, 2017): 341–53. https://doi.org/10.1161/CIRCRESAHA.117.310804.
- Price, James H., Jagdish Khubchandani, Molly McKinney, and Robert Braun. 2013. “Racial/Ethnic Disparities in Chronic Diseases of Youths and Access to Health Care in the United States.” BioMed Research International 2013 (September): e787616. https://doi.org/10.1155/2013/787616.
- “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD): Discussion Paper and Request for Feedback”. n.d. Accessed May 3, 2022. PDF file. https://www.fda.gov/media/122535/download
- Richardson, Jordan P., Cambray Smith, Susan Curtis, Sara Watson, Xuan Zhu, Barbara Barry, and Richard R. Sharp. 2021. “Patient Apprehensions about the Use of Artificial Intelligence in Healthcare.” Npj Digital Medicine 4 (1): 1–6. https://doi.org/10.1038/s41746-021-00509-1.
- Services, U.S. Department of Health and Human. "Artificial Intelligence (AI) Strategy." (2021). https://www.hhs.gov/sites/default/files/hhs-ai-strategy.pdf
- Seyyed-Kalantari, Laleh, Haoran Zhang, Matthew B. A. McDermott, Irene Y. Chen, and Marzyeh Ghassemi. 2021. “Underdiagnosis Bias of Artificial Intelligence Algorithms Applied to Chest Radiographs in Under-Served Patient Populations.” Nature Medicine 27 (12): 2176–82. https://doi.org/10.1038/s41591-021-01595-0.
- Singh, Devin, Sujay Nagaraj, Pouria Mashouri, Erik Drysdale, Jason Fischer, Anna Goldenberg, and Michael Brudno. “Assessment of Machine Learning–Based Medical Directives to Expedite Care in Pediatric Emergency Medicine.” JAMA Network Open 5, no. 3 (March 16, 2022): e222599. https://doi.org/10.1001/jamanetworkopen.2022.2599.
- “Software as a Medical Device (SAMD): Clinical Evaluation : Guidance for Industry and Food and Drug Administration Staff - Digital Collections - National Library of Medicine.” n.d. Accessed May 3, 2022. https://collections.nlm.nih.gov/catalog/nlm:nlmuid-101720008-pdf.
- “Software as a Medical Device (SaMD): Key Definitions.” 2013, 9. https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf
- Tat, Emily, Deepak L. Bhatt, and Mark G. Rabbat. 2020. “Addressing Bias: Artificial Intelligence in Cardiovascular Medicine.” The Lancet Digital Health 2 (12): e635–36. https://doi.org/10.1016/S2589-7500(20)30249-1.
- “Using Artificial Intelligence and Algorithms.” 2020. Federal Trade Commission. April 8, 2020. http://www.ftc.gov/business-guidance/blog/2020/04/using-artificial-intelligence-algorithms.
- Vokinger, Kerstin N., Stefan Feuerriegel, and Aaron S. Kesselheim. 2021. “Mitigating Bias in Machine Learning for Medicine.” Communications Medicine 1 (1): 1–3. https://doi.org/10.1038/s43856-021-00028-w.
- Vyas, Darshali A., Leo G. Eisenstein, and David S. Jones. 2020. “Hidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms.” New England Journal of Medicine 383 (9): 874–82. https://doi.org/10.1056/NEJMms2004740.
- “WHO Issues First Global Report on Artificial Intelligence (AI) in Health and Six Guiding Principles for Its Design and Use.” n.d. Accessed May 14, 2022. https://www.who.int/news/item/28-06-2021-who-issues-first-global-report-on-ai-in-health-and-six-guiding-principles-for-its-design-and-use.
- Yala, Adam, Constance Lehman, Tal Schuster, Tally Portnoi, and Regina Barzilay. “A Deep Learning Mammography-Based Model for Improved Breast Cancer Risk Prediction.” Radiology 292, no. 1 (July 2019): 60–66. https://doi.org/10.1148/radiol.2019182716.
DISCLAIMER: The findings and conclusions published herein are solely attributed to the author and not necessarily endorsed or adopted by the Journal of Science Policy and Governance. Articles are distributed in compliance with copyright and trademark agreements.
ISSN 2372-2193
ISSN 2372-2193