Collaborative of Prominent Academic Institutions Launches Groundbreaking Healthcare AI Challenge

The Healthcare AI Challenge is a first-of-its-kind interactive virtual environment that enables healthcare professionals to experience and assess the world’s most advanced AI healthcare solutions.

Mass General Brigham AI is hosting the Healthcare AI Challenge, a multi-institutional virtual, interactive series of events where healthcare professionals can explore and assess the latest AI healthcare technologies in real-world healthcare scenarios.

The Healthcare AI Challenge Collaborative is launching with a diverse set of healthcare institutions and their healthcare professionals, including Mass General Brigham; Emory Healthcare; the Department of Radiology at the University of Wisconsin School of Medicine and Public Health; and the Department of Radiology at the University of Washington School of Medicine. The American College of Radiology (ACR), a professional medical society representing radiologists, has also joined the Healthcare AI Challenge Collaborative as a founding member to ensure its 42,000-member community has access to the Healthcare AI Challenge. Additional member institutions will be announced and onboarded in subsequent phases.

“The velocity of AI innovations and breadth of their healthcare applications continues to increase. This unprecedented growth leaves clinicians struggling to determine the effectiveness of these innovations in safely delivering value to healthcare providers and our patients,” explained Keith Dreyer, DO, PhD, Chief Data Science Officer at Mass General Brigham, and leader of Mass General Brigham AI, the healthcare system’s AI business. “The Healthcare AI Challenge is a collective response to the complexities involved in advancing the responsible development and use of AI in healthcare. This new approach strives to put clinicians in the driver’s seat, allowing them to evaluate the utility of different AI technologies and ultimately, determine which solutions have the greatest promise to advance patient care.”

Participating healthcare professionals will be granted access to the Healthcare AI Challenge that features late breaking AI solutions they can assess for effectiveness on specific medical tasks, such as providing medical image interpretation, in a simulated environment. Participants with relevant healthcare credentials can then provide their feedback on the solutions’ performance and utility, which will generate publicly available insights and analytics. By crowdsourcing input from healthcare professionals, the Healthcare AI Challenge seeks to create continuous, consistent and reliable expert evaluations of AI solutions in medicine. Importantly, scaling the evaluation of these technologies and sharing the insights broadly and transparently can result in societal benefit for healthcare stakeholders and patients globally.

“We need to go beyond collaboratives that come to consensus on how to think about AI,” said Alistair Erskine, MD, chief information and digital officer at Emory Healthcare and Emory University. “We need healthcare delivery communities to provide real-world experience of the application of AI at the point of care. That is what the Healthcare AI Challenge is designed to do.”

An interactive data-driven solution

The Healthcare AI Challenge runs on a robust and diverse data-rich environment representative of the founding academic members.

Radiology was chosen as the first in a series of healthcare AI events given the historic use of AI solutions to make rapid and meaningful impact in the field against the wave of generative AI model proliferation.

“Medical imaging provides many types of data, and up to 95% of healthcare data collected is unstructured, non-text data,” said Richard Bruce, MD, associate professor of radiology and the vice chair of informatics at the University of Wisconsin School of Medicine and Public Health. “AI has the potential to interpret and distill that data at a new scale and speed, but what we need is the ability to quickly test and compare different AI solutions. The Healthcare AI Challenge will offer a platform to evaluate and compare tools across various clinical situations.”

Health care professionals at institutions that are a part of the Healthcare AI Challenge Collaborative who register can log onto the Healthcare AI Challenge, select one of several events—such as image interpretation—and choose from a series of challenges to assess any one of the multiple foundation models available on the platform. The image interpretation challenges include questions focused on draft report generation, key findings, differential diagnosis, among others. The expert then rates the clinical skill level of the foundation models’ responses, which contributes to the insights and analytics rankings. Only verified healthcare professionals can participate in challenges that contribute to the rankings. The results of the Healthcare AI Challenge can be followed by the general public at HealthcareAIChallenge.org.

“We are facing an overwhelming influx of FDA-approved AI tools in healthcare, especially in radiology. Forming an academic collaborative could play a crucial role in validating and selecting these tools, ensuring they adhere to the highest standards of efficacy and safety,” said Dushyant Sahani, MD, chair and professor of the Department of Radiology at the University of Washington School of Medicine.

The rankings generated by the Healthcare AI Challenge can serve to provide industry, healthcare stakeholders, and the public with a transparent analysis of AI solutions’ performance across a wide range of healthcare data and clinical scenarios. User activity and expert feedback are expected to provide valuable insights to help AI developers better understand how healthcare professionals consider AI healthcare solutions in the context of providing clinical value. In turn, developers can enhance these technologies so that they are fit-for-purpose, commercially viable and clinically relevant. This may foster collaborations and knowledge-sharing with industry to help ensure responsible adoption of AI in healthcare settings.

“The American College of Radiology® is leading safe and effective radiological artificial intelligence implementation by offering real-world solutions that address the challenges radiologists face today,” said Christoph Wald, MD, PhD, MBA, FACR, vice chair of the ACR Board of Chancellors and chair of the ACR Commission on Informatics. “The ACR is strengthening care by promoting AI use to identify efficiencies, improve outcomes, build safeguards and optimize patient care. Participating in the Healthcare AI Challenge is another important step in ACR’s efforts to facilitate the safe, responsible and transparent evaluation of AI, especially as the use of newer foundation models is poised to unfold in healthcare.”

The Healthcare AI Challenge will continually add new AI solutions, events, multimodal diverse data, domain experts and specialties to its interactive environment, including pathology, genomics, waveform, as well as public and specialized text-based foundation models.

About Emory Healthcare

Emory Healthcare, with nearly 24,000 employees and 10 hospitals, is the most comprehensive academic health system in Georgia. System-wide, it has 2,796 licensed patient beds, more than 3,450 physicians practicing in more than 70 specialties, serving metro Atlanta and Georgia. It also provides services to greater Georgia through a joint venture at St. Francis–Emory Healthcare Hospital in Columbus, six regional affiliate hospitals, and its clinically integrated physician network. For more, visit Emory Healthcare.

About University of Wisconsin School of Medicine and Public Health

The University of Wisconsin School of Medicine and Public Health is recognized as one of the nation’s leading institutions in health sciences education, research, and service. Founded in 1907 as the medical school of the University of Wisconsin–Madison, in 2005 it became the nation’s first school to integrate the disciplines of medicine and public health. With a deep commitment to a vision of healthy people and healthy communities, we translate discovery into application and interconnect clinical care, education and research. The school employs more than 5,400 faculty and staff and provides educational opportunities for nearly 2,400 students and postgraduate trainees. Grants awarded to School of Medicine and Public Health principal investigators in fiscal year 2023 totaled $641 million in federal and non-federal awards. Some of the nation’s leading researchers, educators, and clinicians are among the faculty, including several National Medal of Science recipients and National Academy of Science honorees. 

About UW Medicine

As the only comprehensive clinical, research and learning health system in the five-state WWAMI (Washington, Wyoming, Alaska, Montana, Idaho) region, UW Medicine provides a higher degree of healthcare, ranging from primary and preventive care to the most highly specialized care for the most complex medical conditions. Nearly 29,000 healthcare professionals, researchers and educators work together at UW Medicine with a single mission: to improve the health of the public. 

About the American College of Radiology

The American College of Radiology (ACR), founded in 1924, is a 42,000-member medical association that advances patient care, medical practice and collaborative results through advocacy, quality standards, research and education. www.acr.org