1. NAM Digital Learning Collaborative AI and the Future of Continuous Health Learning & Improvement Workgroup – Publication Introduction
Michael E. Matheny, MD, MS, MPH
Director, Center for Population Health Informatics Departments of Biomedical Informatics, Medicine, and Biostatistics
Vanderbilt University Medical Center

2. DLC AI & Future of Continuous Health Learning and Improvement Workgroup
Original Charter: to explore the fields of AI and their applications in health and health care
strategies to enhance data integration to advance healthcare AI
practical challenges to AI model development and implementation
opportunities for accelerating progress

3. Initial Workgroup Membership
Sonoo Thadaney, Stanford (Workgroup Co-chair)
Michael Matheny, Vanderbilt (Workgroup co-chair)
John Burch, JLB Associates
Wendy Chapman, University of Utah
Jonathan Chen, Stanford University
Len D’Avolio, Cyft
Sharam Ebadollahi, IBM Watson Health Group
Hossien Estiri, Harvard Medical School
Steve Fihn, University of Washington
Jim Fackler, John Hopkins School of Medicine
Seth Hain, Epic
Brigham Hyde, Precision Health Intelligence
Edmund Jackson, HCA
Hongfang Liu, Mayo Clinic
Doug McNair, Cerner
Eneida Mendonca, University of Wisconsin Madison
Sean Khozin, FDA
Matthew Quinn, HRSA
Robert E. Samuel, Aetna
Bob Tavares, Emmi Solutions
Howard Underwood, Anthem)
Daniel Yang, Moore Foundation
Jonathan Perlin, CMO HCA, DLC Co-Chair
Reed Tuckson, Tuckson Health Con., DLC Co-Chair
Wendy Nilsen, NSF
Joachim Roski, Booz Allen Hamilton
Howard Underwood, Anthem
Daniel Yang, Moore Foundation
Doug Badzik, Department of Defense)
Carlos Blanco, National Institute on Drug Abuse
Paul Bleicher, OptumLabs
Carla Brodley, Northeastern University
Tim Estes, Digital Reasoning
Daniel Fabbri, Vanderbilt University Medical Center
Kenneth R. Gersing, NIH
Michael Howell, Google
Brigham Hyde , Precision Health Intelligence
Javier Jimenez, Sanofi
Jennifer MacDonald, VA
Nigam H. Shah, Stanford)
David Sontag, MIT
Noel Southall, NIH
Shawn Wang, Anthem
Maryan Zirkle, PCORI
The workgroup formed and met early 12/2017 initially at NAM with representative experts from government, academia, industry, and non-profit foundations

4. NAM Workgroup Publication Objectives & Scope
Develop a reference document for model developers, clinical implementers, clinical users, and regulatory and policy makers to:
understand strengths and limitations of AI/ML
promote use of these methods and technologies within the healthcare system
Highlight areas of future work needed in research, implementation science, and regulatory bodies to facilitate broader use of AI/ML in healthcare

5. NAM DLC AI Publication: Organization
TOPIC Leads
NAM DLC Jonathan Perlin, Reed Tuckson
NAM Program Office Danielle Whicher, Mahnoor Ahmed
Publication Editors Sonoo Thadaney, Michael Matheny
Chapter 1: Introduction Sonoo Thadaney, Michael Matheny
Chapter 2: History of AI Edmund Jackson, Jim Fackler
Chapter 3: Promise/Opportunities for AI Joachim Roski, Wendy Chapman
Chapter 4: Pitfalls/Challenges for AI Eneida Mendonca, Jonathan Chen
Chapter 5: AI Development & Validation Hongfang Liu, Nigam Shah
Chapter 6: AI Deployment in Clinical Settings Steve Fihn, Andy Auerbach
Chapter 7: Regulatory & Policy Issues Doug McNair, Nicholson Price
Chapter 8: Conclusions & Key Needs Sonoo Thadaney, Michael Matheny

6. AI: What Do We Mean?
There are a family of technologies within AI, throughout the day you will hear some chapters focus more on certain elements, such as machine learning, but it is important to know that the scope of the publication includes all of these sub-domains.

7. “Health & Healthcare” Settings
Direct Encounter-Based Care
“Non-Traditional” Settings: CVS, Home
Population Health Management
“Back Office” Healthcare Administration
Patient/Consumer Facing Technologies

8. Target Audiences Direct Care Providers Patients and their Caregivers
Healthcare System Leadership & Admin
Data Scientists (Developers)
Clinical Informatics (Implementers)
Legislative & Regulatory Bodies
Third Party Payors
Chapters will have different audiences to address different facets of the domain, some chapters may be targeting multiple audience segments

9. Chapter 2: History & Current State of AI
Discusses history of AI with examples from other industries
Summarize the growth, maturity, and adoption in healthcare as compared to other industries.
Target general audience

10. Chapter 3: Promise & Potential Impact of AI
Focus on the utility of AI for improving healthcare delivery
Discuss near-future opportunities and potential gains from the use of AI
Target General Audiences

11. Chapter 4: Potential Unintended Consequences of AI
Focus on the potential unintended consequences of AI on:
work processes
culture
equity / fairness
patient-provider relationship
workforce composition & skills
Target General Audiences

12. Chapter 5: AI Modeling Development & Validation
Most Technical Chapter
Topics
process for developing and validating models
choice of data, variables, model complexity
performance metrics, validation
Target Model Developers

13. Chapter 6: Deploying AI in Clinical Settings
Focus on implementing and maintaining AI within ‘production’ healthcare domains
Address issues of:
Software development
Integration into a Learning Healthcare System
Applications of Implementation Science
Model Maintenance & Surveillance over Time
Target Healthcare System Leaders & Implementers

14. Chapter 7: Regulatory & Policy Considerations
Summarize key legislative and regulatory considerations for the use of AI in health care
Identify strengths and weaknesses in current framework
Discuss legal liability concerns
Make recommendations to address gaps

15. Chapter 8: Conclusions & Key Needs
build on and summarize key & cross-cutting themes from previous chapters
Recommend key areas for:
Moving the field forward
Highlight over-arcing these from chapters

16. Publication Timeline NAM Meeting 11/2017
Publication Workgroup Kick-Off 02/2018
Content Scope Established 05/2018
Chapter Outlines Completed 07/2018
Chapter Draft Versions /2018
NAM Meeting /2019
Publication Revisions /2019
NAM/External Reviews 03/2019
Tentative Release 04/2019

17. Mental Framework for This Meeting
Out of Scope: Discussion of Major Content Additions/Subtractions
In Scope: Changes to Framing / Addressing Imbalance / Voice of Chapters
In Scope: Focus on Recommendations
Identify and discuss modifications, additions, and subtractions as each chapter is discussed
Be mindful of a desired balance between stakeholder groups (patients, providers, administrators, regulatory bodies, etc.)
If you felt like the opportunity to discuss a point passed and major themes, please send it to us in an , or write it down and give it to us during a break

18. Thank You NAM Leadership DLC Leadership NAM Staff Leads
Victor Zhau
Michael McGinnis
DLC Leadership
Jonathan Perlin
Reed Tuckson
NAM Staff Leads
Danielle Whicher
Mahnoor Ahmed
DLC Clinical AI Workgroup Members