MSC Perspectives on Medicine - Harlan Krumholz - 9-28-23

September 28, 2023

ID10749

To CiteDCA Citation Guide

00:00It's my pleasure to introduce our first
00:02speaker today for our first lecture, Dr.
00:05Harlan Krumholtz. Dr.
00:07Krumholtz is the Harold Hines Junior
00:10Professor of Medicine and the Founder
00:13and Director of the Center for
00:15Outcomes Research and Evaluation
00:17Corps at the Yale School of Medicine.
00:20He is one of the most influential
00:22investigators in outcomes research and
00:24healthcare delivery and has made significant
00:27contributions to valuebased care.
00:29Among his many contributions, Dr.
00:31Krumholz played a seminal role
00:33in nationwide efforts to reduce
00:34the door to balloon Time,
00:36a hospital performance metric
00:37focused on the timely initiation of
00:40percutaneous coronary intervention
00:42for myocardial infarctions.
00:43He has also played an influential
00:45role in bringing attention to the
00:47high rates of hospital readmissions
00:48for conditions such as heart failure,
00:50which led to the creation of the
00:53Hospital Readmissions Reduction Program,
00:54where hospitals with higher than
00:56expected readmission rates can face
00:59reductions in Medicare payments.
01:01Doctor Krumholz was also the
01:02founding governor of the Patient
01:04Centered Outcomes Research Institute,
01:06established as part of the Affordable
01:07Care Act,
01:08aim to help patients clinicians make
01:10more informed decisions and has also
01:12led efforts to develop publicly
01:14available outcome measures for the
01:16Centers for Medicare and Medicaid Services.
01:18Currently,
01:19he also hosts a podcast alongside
01:21his college like Doctor Howard
01:22Foreman called Health and Veritas,
01:24which features the most important thing.
01:26The most important thing right
01:28podcast is called Health and Veritas,
01:30which focuses on commentary and
01:32guest speakers focus on the latest
01:34news and ideas in healthcare.
01:36As a practicing cardiologist, Dr.
01:37Krumholtz completed his bachelor's at Yale
01:39College and D at Harvard Medical School,
01:41residency in internal medicine at
01:44UCSF and fellowship in cardiology
01:46at the Beth Israel Hospital,
01:47where he also earned a master's in
01:49health policy and management from
01:51the Harvard School of Public Health.
01:53So very fortunate to have Doctor
01:55Kromholtz join us today for his talk
01:57titled Medicine in an Information Age,
01:59Opportunities to Improve Health
02:01and Healthcare.
02:02Doctor Kromholtz,
02:02thank you so much for taking
02:03the time to be with us today,
02:04and I will turn it over to you
02:06when you're ready.
02:06Thanks for inviting me.
02:07What a terrific introduction.
02:08I really appreciate it.
02:11Thank you and hello everybody.
02:13So it's it's lunchtime.
02:14So I thought this could be just
02:15a high level talk and share some
02:17ideas and see what you think
02:19and well there's like a question
02:20answer thing people can put in
02:22comments and questions, right,
02:24Yes. And I'll maybe towards the end if we
02:27get questions accumulated all you know feel
02:30that's great, but but even through
02:31if people put up something and it
02:33seems relevant, I can address it.
02:35So don't don't want people to think
02:37that this has to be so formal
02:39that you can't put in a question
02:41or we can't talk about whatever
02:42it is that you're interested in.
02:43So go ahead. Sorry.
02:49So the I titled this first
02:51of all honored to be invited.
02:54It was really appreciated.
02:55The invitation I always love talking to,
02:58I guess I think this is medical
03:01student invitation and largely
03:03medical student audience.
03:04And so I really love that in
03:06the title of the Medicine and
03:08Information age Opportunities to
03:10Improve Health and healthcare.
03:11Just to start with disclosures for a second.
03:17Always good practice.
03:18And so let me just say my goals so surprised.
03:21A high level overview of medicine in
03:24an information age to discuss what
03:26clinicians need to know about a I and
03:29then to discuss some current research
03:31directions maybe to inspire folks
03:33to think about now they might play
03:35role in what's going to be a very
03:37different era of medicine going forward.
03:39First thing I want to do is
03:40just introduce you.
03:40I'm I'm the founder and director of
03:42the Center for Outcomes Research
03:43and Evaluation and I'm really
03:45fortunate to be surrounded by an
03:47extraordinary group of very talented
03:49individuals that cross disciplines,
03:54doctors, nurses, data scientists,
03:58project managers.
04:00Actually, I couldn't even begin
04:01to list all the different skill
04:03sets among the people in core,
04:04about 130 people and and the the
04:07focus is on outcomes research.
04:09I just want to take a second to say
04:10like you know what is that exactly?
04:12People sometimes get confused about
04:14outcomes research all states of
04:15outcomes what's outcomes research and
04:17outcomes research which actually frames
04:19my thinking about this issue about
04:21medicine and information ages is sort
04:23of a field that's focused on the end result.
04:26And what we say is at the end of the day,
04:28what have we accomplished to to for people,
04:31for patients,
04:32for society and and how can we do better.
04:35So if it really is saying net,
04:36net, what's what's the output,
04:39not how many programs have been produced or
04:41how many cool a I innovations have been made.
04:44But but really when you look at it,
04:46at the end of the day,
04:47what's happened and have we
04:49actually made things any better.
04:50And of course when you look at the
04:52United States healthcare system,
04:53all you see is that we spend more
04:55money every year and if anything life
04:56expectancy in the last couple years,
04:57not even counting.
04:58The pandemic has has that the improvements
05:01have stagnated and even reversed.
05:03And if you look at areas like health status,
05:05functional status,
05:06multimorbidity,
05:06a whole range of outcomes,
05:08we're not actually doing better.
05:09So in some ways an indictment of
05:11the work that I've done because
05:13I've been unable to actually
05:15translate this into improvements.
05:17But this is our aspiration to say
05:19that we've got to be applying our
05:20resources in ways and applying our
05:22our energy in ways that actually
05:24produce tangible benefits for people.
05:27And our goal is really not just to
05:30describe issues within healthcare,
05:32but it really is to to be solution oriented.
05:34And so the goal is to create knowledge
05:36that produces health outcomes for discovery,
05:39accountability and improvement
05:39with a focus on on maybe if you
05:42talk about healthcare at large,
05:43talk about effectiveness and efficiency,
05:45certainly equity,
05:46patient centeredness or putting
05:48our patients and and and people
05:50in stronger positions or tipping
05:52the balance of power toward them,
05:54safety,
05:55timeliness and then just population health.
05:57How are we helping people to avoid the
05:59need for healthcare and to be able to
06:01stay healthy for longer periods of time,
06:03compress morbidity.
06:04And these are the kind of
06:06themes that that pervade this.
06:09And in our thesis,
06:10I think over the last decade has
06:12been that medicine is primarily
06:14now in information science.
06:16And I think this is really important for
06:18medical students as you go through to
06:20sort of recognize that you're sitting
06:21at this cusp of immense change in medicine,
06:23maybe we we sort of always talk about,
06:26yeah,
06:26that's exciting that changes,
06:27breakthrough is shifting.
06:30But but this time I really think it's true.
06:33I mean,
06:33you know,
06:34it's as true as the moment
06:36in history when we, you know,
06:38when we started understanding
06:40microbes is causing disease.
06:41I mean there, there've been junctures
06:42where all of a sudden new understanding
06:44and new capabilities fundamentally
06:45transformed our ability to to get
06:48things done and to to help people.
06:50And I think we stand at that that moment now.
06:53I mean when I look at the Lascar
06:55awards that just came out, you know,
06:57Lascar sort of American nobles and
06:59very important awards that that you
07:01know sort of highlight the sort of
07:03very best discovery and basic science.
07:05What they gave it to this year was alpha
07:07fold which is about how proteins fold.
07:09Basically being able to take
07:11one-dimensional information about a protein,
07:12the amino acid sequence and predict how
07:14it looks in three-dimensional space.
07:16Which turns out to be a pretty
07:18difficult problem because you know
07:19you've got this long string of amino
07:21acids But the way they can interact
07:23with each other which in bonding and
07:25and you know the the folding can be
07:28quite a challenge to figure out.
07:30And we've done it historically by
07:32crystallography or you know what,
07:33a wide range of really labor intensive,
07:36difficult areas and maybe we've
07:38solved 200,000 of what would be
07:408,000,000 proteins and alpha fold
07:41with a I figures this out.
07:43But the cool thing too
07:44was one of the winners.
07:45So this really prestigious award
07:47which really represents kind of
07:48a landmark in people's careers,
07:49he just graduated from his PhD in 2017.
07:53I mean this is what this world now
07:56has enabled which is marked advances
07:58by people at your stage of career.
08:00It's at stages of people who are
08:02quite younger.
08:03It's you don't need Gray hair to
08:05be able to be making immense
08:07contributions anymore.
08:08And and what we've done is in on
08:10the clinical side is we've digitized
08:12all the data but we've still now
08:14failed to actually bridge the
08:16potential of all that digital data
08:19into tangible benefit for patients.
08:21And and that's the problem now
08:23I wrote this piece in 2014,
08:26I think and you know I'd sort of hope
08:30that we would move a little faster
08:32maybe that's almost a decade ago that
08:33big data and new knowledge in medicine,
08:36the thinking,
08:36training tools needed for
08:38learning healthcare system.
08:39And it was sort of like a laying
08:40out of this issue about when
08:42I I've talked about big data,
08:43it's really about data science,
08:45information science.
08:46It's not just about the data
08:48itself but it's how are you going
08:50to parlay this new capabilities
08:52that we have with digital data.
08:53And and I wrote that a big data in
08:57medicine massive quantities of of
08:59healthcare data accumulating from
09:01patients and populations and advanced
09:03analytics that can give those data
09:05meaning hold the prospect of becoming
09:07an engine for knowledge generation
09:09that's necessary to address the
09:11unmet information needs of patients,
09:12clinicians,
09:13administrative researchers and
09:14health policymakers.
09:15And and what I further went on to
09:18conclude was that and I'll just read this,
09:22but I still think it stands. You know,
09:24this is a historic moment in medicine.
09:27There's a remarkable opportunity to
09:28promote medicine as an information
09:30science and strengthen the foundations
09:31of a learning health system.
09:32To find by the Institute of Medicine
09:35is designed to generate and apply
09:37the best evidence for collaborative
09:39healthcare choices of each patient
09:41provided to drive the process of
09:42discovery as a natural outgrowth of
09:44patient care and to ensure innovation,
09:46quality, safety and value in healthcare.
09:48So the idea is that it's not like
09:50research sits over here and and
09:52clinical care sits over here,
09:54but there becomes an integration
09:56of the act of providing care with
09:58the generation of new insights
10:00and knowledge and that there's an
10:02actionable insights everywhere.
10:04And and sort of one way that I have been
10:06known to talk about this is to say look,
10:08if you look at Amazon,
10:10Google and Tesla as prototypes,
10:12you say with every purchase on Amazon,
10:14with every search on Google,
10:15with every mile driven on Tesla,
10:17they get smarter.
10:18But but actually with every patient we see,
10:20we don't get smarter.
10:22You know, the the,
10:23the knowledge, the experience,
10:24it gets sequestered within the people
10:26that were involved in that care.
10:28It's not systematically analyzed
10:31and leveraged in ways that produces
10:33a better system.
10:34Now then we conduct sort of slower,
10:37more cumbersome labor intensive
10:38research way over here and when that
10:41produces some output, you know,
10:42we try to get it translated into the
10:46clinical ecosystem and usually you know,
10:49it takes a long time and and
10:50there's distrust.
10:51You know the,
10:52the,
10:52the clinicians think the researchers
10:53don't really understand the problems on
10:55the clinical side and they're they're
10:58studying very special populations.
11:00You know,
11:00the inclusion exclusion criteria make
11:02it so that the studies are about
11:04people that aren't like the people
11:05that you're seeing in everyday practice.
11:07So it's sort of ends up being
11:09hard to translate but but you know
11:11Google that that algorithm is being
11:12used all the time.
11:13So with every use that, you know,
11:15it's getting better and the
11:17the idea is that that could,
11:19that could be what we could have.
11:21And then I do want to say,
11:22you know,
11:23there are always are these
11:25concerns about how technology
11:26will find its way into medicine.
11:28And I'm going to talk about this in
11:30a minute about some of the untoward
11:32unintended adverse consequences.
11:33But there still is this aspiration
11:35that wrote this paper with Rob Kales,
11:37now Commissioner of FDA and Haider Warwick,
11:39who's actually now in special
11:42advisor to the Commissioner.
11:44But we,
11:44we were sort of trying to brainstorm
11:46about this and saying you know
11:47this digital transformation has
11:48the potential to make healthcare
11:50more humane and personalized as we
11:51understand more about our patients.
11:53Look,
11:54a lot of industries are using technology,
11:57you know,
11:58customer based software like
11:59you know the CRM
12:01stuff that Salesforce is using.
12:03I mean helps us, you know,
12:05sales people know how to use this,
12:07make people feel comfortable.
12:07I know about you, I can help you.
12:10It's very specific to you.
12:11But you know medicine we have
12:12yet to get there, but it's,
12:14it could be on the horizon.
12:16So here's a question that I
12:18just wanted to pose also.
12:20I mean especially if you're in training now,
12:21you sort of think, well,
12:23what do I need to know about a I to be
12:26a really good clinician in the future.
12:27Now some of you I hope will actually be
12:30interested in becoming data scientists
12:32and and actually I hope outcomes
12:34research slash data scientists.
12:35And the reason I say the outcomes
12:37research side is because you care about
12:39consequence that you you actually
12:40want to become data scientists.
12:42We're going to produce knowledge that
12:43you can be able to track to how it's
12:45actually improving people's health,
12:47how people are better off because of it,
12:48how you relieve suffering.
12:49I mean that that you sort of see that
12:51it's not just about the data science,
12:53but it's about what that data
12:55science is going to do for people.
12:57But but you, you know,
12:58there's also is a fair question
12:59to say what are clinicians if you
13:02aren't interested in becoming,
13:03you know, deeply involved in in
13:04that in this revolution,
13:06which is exciting.
13:06I hope to you know,
13:08continue to excite people about
13:09this as being a career path.
13:11But it won't be for everybody and
13:13and in fact most people will choose
13:15you know to to take advantage of
13:18what it's going to produce rather
13:20than be part of the generators
13:22of what it will produce.
13:23But but you know what do you need
13:25to know And and so well there's
13:26a whole range of things we could
13:28talk about that would be worth
13:29knowing about what are the types
13:30of a I and how are they applied.
13:31What are the benefits and limitations of a I.
13:34What about a I ethics A I regulation
13:36There's a whole field that's emerging
13:37about how should we regulate this
13:39what's how should the FDA be involved
13:41what how do we protect ourselves.
13:43Of course this is even short of all
13:45the talk about the LLM's destroying
13:46the world and what do we do you know
13:49to protect ourselves from the new
13:52Terminator type of you know scenarios
13:53that would come out of sentience.
13:58I know beings out of the LOM's and a I
14:00but but I'm just saying you know within
14:03medicine what what should be regulated
14:05How about evidence based practice using a
14:07I tools and integration into workflows.
14:09I mean, we could be doing long
14:12courses on this and and trying to
14:14make sure that people know about,
14:16you know, natural language processing,
14:18machine learning, robotics,
14:19expert systems, neural networks.
14:21You know what, what are the differences
14:23and similarities between all this?
14:24What are people coming out with?
14:27You know, what is it that people
14:28don't even know yet?
14:29Like take some of the LLM's large language
14:31models like ChatGPT where honestly
14:32the people built it still don't quite
14:35understand what's going on under the hood.
14:37So you know, how is that possible?
14:38You know, we could be talking
14:40about all the types of a I.
14:41We could be talking about benefits
14:43and limitations of a I in medicine,
14:45about how clinicians should understand
14:48the potential benefits such as
14:50improved accuracy and efficiency and
14:51the risks such as bias and concerns.
14:53But, you know,
14:54we could spend a lot of time on that.
14:55We could spend time on on ethics and their
14:57whole fields of ethics that are growing up.
14:59Jennifer Miller at our place,
15:01one of the world's experts,
15:02she's turning her attention to
15:04some of these issues as well.
15:06And like I said, you know,
15:08there should be courses on regulation
15:10because we still haven't figured this out.
15:12And no matter how good the A I is,
15:14if we don't figure out the
15:16implementation science side of it,
15:18which is sort of how do we actually
15:20integrate this into workflows in
15:22ways that that delight people as they use it,
15:24You know,
15:25it's just not going to work.
15:26And you know,
15:27how do you even evaluate these A I algorithms
15:30when they're put into clinical practice?
15:32What represents evidence?
15:33Can you use them to generate
15:35evidence when it's not statistical
15:37inference but it's something else?
15:38One could you use predictive models?
15:41So all of this stuff,
15:43you know,
15:44we could be teaching people and and let
15:46me just pause on the data part for a minute.
15:48You know,
15:49so even in the data we could
15:51be spending a lot of time,
15:52you know that this new world
15:55is highly dependent on on the
15:57acquisition and refinement of data.
16:00There are new sources of data,
16:02but there's lack of standards
16:04and immense fragmentation.
16:05So I've seen a lot of these
16:08large scale aggregators of data,
16:09Komodo or Verdigm trinetics.
16:12I mean, there's a whole,
16:14you know,
16:15even Epic is now trying to develop
16:17like repositories of data.
16:19But the problem is,
16:21because people move between
16:22healthcare systems and their
16:24records sit in different areas,
16:26it becomes hard to actually be
16:27able to follow the course of a
16:28single patient over the course of
16:29time and to know what people have.
16:31And when you go to many of these data sets,
16:33you you can't tell what they don't have.
16:37So you know, they're they're telling
16:40you what's within their purview,
16:41but it's not a comprehensive set
16:43of data that spans all systems.
16:45And so you end up having that
16:48problem plus there are a lot of
16:50errors in the system as well.
16:52So it, you know,
16:53becomes a a bit of a problem.
16:55Now there was a paper that came
16:56out that just I thought was
16:58emblematic of this neurologic and
17:00psychiatric risk trajectories after
17:01SARS CL V2 infection analysis to
17:04your retrospective of one point
17:06to almost 1.3 million patients.
17:09So you know,
17:09I mean the journal editors get
17:11impressed once you present you know
17:12you submit something that's got this
17:14many patients and and it's it's got
17:16a lot of fancy analysis but but
17:18here's what what the appendix said,
17:20the supplemental appendix said this
17:22network this the network that's
17:23based on this paper contains data
17:25provided by participating healthcare
17:27organizations so long as their name
17:29remains anonymous as a data source.
17:31So this was a company that that
17:35collaborated with a bunch of researchers.
17:37But but by the way,
17:38the researchers have no idea
17:40where did the data come from,
17:41how complete was it,
17:42what it was just like a data set.
17:44And and I'm concerned that this is
17:46CNN was just reporting yesterday,
17:49I was talking,
17:49going back and forth to them about
17:51some of the antiobesity drugs.
17:52EPIC had produced a report.
17:54They were reporting it nationally.
17:55I said to them,
17:56so how complete are these data?
17:58How what?
17:58What happens if people were using
18:01a telemedicine thing like Weight
18:03Watchers to fill their an anti obesity
18:05medicine or you know was being done
18:06outside of the healthcare system?
18:08How do you know?
18:09How does EPIC know what they captured
18:11and what they didn't capture?
18:13And what's their denominator?
18:14And the answer was they didn't know.
18:16They didn't know.
18:17But you know, I said well then you can.
18:18You should write articles that say epic says,
18:21but you shouldn't present as fact
18:23analyses that are based on on
18:25data sets that can't be clearly
18:27defined in methods that aren't
18:29clearly inexplicitly stated.
18:31And yet, you know, this is a state.
18:33Those are reporters.
18:33But I'm just saying,
18:34even in in the journal articles,
18:36you're seeing that.
18:37So the journal accepted this paper
18:39even though that no one can tell
18:40them where the data came from.
18:42And you know, is that a problem?
18:43Maybe. Maybe. So let me keep going.
18:48So another issue about the data,
18:50just to to stay with data for a minute,
18:53is of course bias. You got.
18:54I'm sure most all of you have
18:57heard about these issues about
18:59bias within data system.
19:00If you train on on our own current
19:03habits and if they have embedded in
19:06bias or structural racism has has
19:08influenced the patterns of care.
19:10Like even who dies.
19:11You know that because some people
19:13die earlier than maybe
19:14they necessarily needed to because
19:15they didn't have access to the right
19:17care or or because of other factors.
19:19When you start creating predictive models,
19:21it becomes a self fulfilling prophecy that
19:24embeds within those models problems that
19:27we currently have within our care system.
19:30So we have to be thinking about like you
19:31know, So what is it we're trying to do?
19:32This happened with the Brigham.
19:34There's a study that was done where
19:36they were trying to predict healthcare
19:38utilization and it turned out that that
19:40that white patients were more likely
19:42to have higher healthcare utilization
19:44after discharge in black patients.
19:46And so then they use that to say, well,
19:49these people need more attention because
19:51they're consuming more resources as a
19:53proxy for their health outcomes were worse,
19:55but it actually wasn't clear in the end
19:56that their health outcomes were worse.
19:58And the reason that black patients may not
20:00have been utilizing as much may not have
20:01been because they had better recovery,
20:03but because they had less access.
20:05And so if you embed within the system,
20:08hey, we want to identify the highest
20:10risk patients based on healthcare
20:11utilization that that may have been
20:13biased against people who had barriers
20:15to access and actually the the lack of
20:18healthcare utilization was actually
20:19a signal of a problem.
20:21Rather than actually that they
20:23had been doing better.
20:24So these are just examples
20:26of where you've got to be,
20:28got to be careful and there are
20:29other kinds of bias too where you've
20:31got a I systems that are king off
20:33of information that that may not
20:35be intrinsic to the patient.
20:37And and that's not what I'm talking about.
20:39I mean,
20:39that's an example where people
20:41were trying to determine images
20:42and whether it was cancer or not.
20:44And if there was a marker in the
20:46image that was suggesting they were
20:48getting radiation therapy, you know,
20:49was king off of that instead of
20:51actually what the lesion look like.
20:53And that that's another kind of bias
20:55where you're kind of so the interpretability,
20:57what's driving it?
20:58Does it make sense?
20:59Is it aligned with with Justice?
21:01I mean,
21:02is it treating people the right proper
21:04ways And these are all the all issues
21:06and then you know that they're issues
21:08with regard to performance of these,
21:09the many of these things that
21:12we're using in a I.
21:13And again,
21:14there's a lunchtime conversation with you.
21:16I'm just sort of throwing out things
21:17that may give you different ideas,
21:19but some some of you may have seen this.
21:22But you know,
21:23there's this whole thing about
21:24sepsis and whether or not we can
21:26be identifying sepsis earlier,
21:27intervening faster and saving lives as
21:30people come into the emergency department.
21:33And so then there are all these
21:35national campaigns acting quickly
21:36can save lives from sepsis.
21:37It's about temperature,
21:39infection, mental decline,
21:40being extremely ill.
21:41But there are people with subtle
21:43changes that are missed.
21:44And the question is sort of
21:46like is there are
21:46there ways to create clinical
21:48decision support tools that might
21:50be able to help us to more rapidly
21:52identify with the right people,
21:53triage them appropriately and and save lives.
21:56And so EPIC, you know,
21:57embarked on a development of an
21:59algorithm that would be embedded
22:01within the electronic medical
22:03record and and they instituted it.
22:06But you know when people studied it
22:08a hospital algorithm designed to
22:10predict the deadly condition misses
22:12most cases and a new study founds
22:14it also had many false alarms.
22:16So it it lacked both sensitive and
22:18specimen see when being tested independently.
22:21But I think the most important thing
22:22about this is that EPIC did this.
22:24So that meant that it was being
22:27disseminated broadly within
22:28the electronic medical record.
22:30And people may have assumed that if
22:32that's true then the performance
22:34must be good enough.
22:35They may not realize that the FDA
22:36didn't have to prove it and and that
22:38this was something that was just,
22:40you know,
22:41it didn't have independent at
22:42that time evaluation and it might
22:44have given false sense of security
22:46when people did have sepsis and it
22:48might have been wasted resources
22:50for people as it had false alarms.
22:52This is the article that appeared
22:54in JAMA Internal Medicine external
22:56validation of a widely implemented
22:58proprietary sepsis prediction
23:00model hospitalized patients.
23:01I think it's important widely
23:04implemented proprietary from a private
23:06company sepsis prediction model.
23:08You know that many people ended
23:10up probably using and when they
23:12were looking at its performance
23:14it was found to be quite lacking.
23:16And so it both missed people who
23:20who likely had sepsis and it it
23:22found people that that did.
23:24So we've been writing a lot about this.
23:26This is Makoto Mori,
23:28a surgical resident at Yale led a
23:31paper while he was getting his PhD.
23:33Well,
23:34this was opinion piece that we
23:35put together about what would be
23:38sensible regulation and clinical
23:40implementation have clinical decision
23:41support software as a medical device.
23:43I think to really be into this space.
23:46You know what we're at least our group
23:48is we're interested in generating new
23:50tools but we're also interested in
23:52how those tools can be regulated apply,
23:54how we can make sure that they're
23:55fair and how they ultimately
23:56improve outcomes for patients.
23:57So.
23:57So we feel like we've got to be
23:59involved in a stretch from you know
24:02beginning to end about how this is
24:04implemented because we feel that the
24:07technology itself and the tools are
24:09important but but singularly they
24:11won't change medicine unless it the
24:14entire ecosystem is primed for that.
24:17There's was another paper that
24:19Chinchi Huang in our group led
24:22with many folks at core and some
24:25external advisors that where we
24:27were trying to say like well,
24:29so if you're evaluating a lot
24:30of these new models,
24:31what kind of metrics should we be using?
24:33Can we standardize those metrics
24:36so people get used to you know
24:38looking at whether or not this is
24:40good enough, can I trust it,
24:41I mean just like you might
24:43look at any any other results.
24:44So this is performance metrics
24:46for their comparative analysis
24:47of clinical risk prediction model
24:49employing machine learning.
24:51And so there's a lot of just like
24:54we're comparing drugs head to head,
24:56you know when somebody comes
24:57out with something new that
24:59we're we may use as a tool.
25:00The question is, you know, So what?
25:02What is it being compared against,
25:03what's the performance of
25:05it and how does it work.
25:06So again another dimension to
25:09this is that we've got to figure
25:10out what we what we can trust.
25:11So getting back to this issue
25:13to the what do you need to know
25:15about a I do you need to know
25:17about recurrent neural Nets as a
25:19type of deep learning process,
25:20sequences of data do you need to know
25:22about convolutional neural Nets,
25:24which you know are are whole different
25:26thing people often using within images.
25:28You know what what?
25:29What do you really need to know
25:30and how deep do you need to go?
25:32As someone who wants to be an
25:35expert clinician and to what degree
25:37does this have to be integrated
25:39into the curriculum?
25:40I I will maybe ask the same
25:42question about basic science.
25:44You know,
25:44how deep do you need to go
25:47into genomics or biochemistry?
25:49I I know this is heresy,
25:50especially at this medical school.
25:52This suggests that there are some of
25:54these very basic elemental pieces
25:56of medical education that are essential.
25:58But but then why is it that if you
26:01quiz clinicians, expert clinicians,
26:03amazing clinicians, awards,
26:04they can no longer remember this education?
26:07So it it's not something they
26:08use in everyday life.
26:09It's actually not something
26:11that's contributing to their
26:13performance as clinicians.
26:14But we insist on it in in ways that
26:16are historical and traditional.
26:18Now I'm not saying that you shouldn't
26:20know anything about a I and I'm
26:21also not saying you shouldn't know
26:22anything about genomic survival chemistry.
26:23The question is what would be good
26:26to know that would contribute to
26:28that for people thinking about
26:29becoming expert clinicians,
26:30that's different from people who
26:32are going to actually become deeper
26:34content experts in these areas.
26:35And and I will say as an addition,
26:37I mean,
26:38I think medical school should have
26:39a lot more social science in it.
26:41I mean people,
26:41we should be knowing more anthropology
26:43and sociology.
26:44And honestly,
26:44you need to know some economics
26:46because you know,
26:47that's what medicine is today.
26:49It's about understanding how we
26:51bring psychology.
26:52You know,
26:52how do we bring to bear as expert clinicians.
26:54That's what that's what we're
26:55employing every day, psychology.
26:56We're trying to help people navigate
26:58difficult economic circumstances.
26:59We're we're trying to understand
27:01cultural differences and trying
27:02to figure out how we can best.
27:04If precision medicine isn't
27:05simply about your genomics,
27:07it's about who you are.
27:09And that that needs to be.
27:10I think that's balancing basic
27:11science and social science.
27:13And and I'm saying data science
27:14can be the piece that brings all
27:16of this together in the future.
27:18So you know that there's this
27:20I'm kind of flipping this.
27:22You know, also like the clinicians
27:23need to know about this.
27:25I really think that one of the major
27:28points is what are those working in
27:30a I need to know about clinicians.
27:32And what I say about this is that
27:34there's good basic foundational
27:35information for you to know and also to
27:37understand how you can know what to rely on.
27:40But because that in the end
27:41becomes the most important thing.
27:42But but really the very best
27:44a I is almost invisible.
27:46I mean it,
27:47it makes your life easy without
27:49even you knowing it.
27:51It it sort of enables connectivity and
27:54communication and and inform choices
27:56and decision support in ways that
27:59that require no effort on your side.
28:02And what what you've got to know,
28:03if your clinician saying I'm not going
28:05to go into this field is all you need
28:07to know is whether it works or it doesn't.
28:09So you know,
28:10and this is an example
28:12of the selfdriving cars,
28:14I mean for you to to get into
28:17a car that's selfdriving,
28:19like you don't need to know all the
28:21algorithms behind how does that car work.
28:24I mean you don't need to take courses
28:25in data science to figure out like
28:27how do they create those algorithms.
28:29What you need to know is, is this safe?
28:32I mean, you get on a plane,
28:34you don't need to know how to fly the plane.
28:35You don't even need to know
28:37Bernoulli's principle.
28:37You just need to know it.
28:39Just have a good safety record.
28:40If I get on this plane,
28:41am I likely to safely get to my destination?
28:43You know it.
28:44You need to know like what can you
28:46trust and so what's the performance
28:49What doesn't really actually
28:50advance my ability to do what what
28:52I want to do and and by the way
28:55clinicians are are not Luddites.
28:56Well not all of us.
28:57I mean you know every one of you is
29:00deep into a I systems every day and the
29:02very best ones you don't even know about.
29:05So you know,
29:06I,
29:06I,
29:06I should say anyone using a phone is using
29:09advanced facial recognition software
29:11every single moment of every single day.
29:14But you know my mom didn't have
29:17to take a course, you know,
29:19in facial recognition.
29:20You know software development.
29:21Like,
29:22you know,
29:22she all she needs to know is she holds
29:24this up and they can open her phone.
29:26And and and honestly because
29:28that's a quick closed loop,
29:31she's going to know rapidly
29:32whether that's like worthwhile
29:34like because does it work or not.
29:36And you know if it works for her,
29:37that's great and she doesn't even have
29:39to know that that's an A I system.
29:40But but it actually makes your life easier.
29:42It makes all of her lives easier and it
29:45and it works seamlessly. So you know,
29:48this is just examples like you know,
29:50you take weather maps,
29:52you know they're intuitive, you can
29:55immediately understand what they mean.
29:56But they're high.
29:57They're taking high dimensional data.
29:58Supercomputers are are sifting through
30:00remarkable volumes of complex data
30:02and applying high level algorithms
30:04to produce predictive models that
30:06help us understand whether it's
30:08going to rain tomorrow or not.
30:10And then it's,
30:11it's summarized in ways that you know,
30:14I mean I have to take a whole day course on
30:16Epic to figure out how to use that software.
30:18You know,
30:18these things are like just intuitive,
30:20like nobody.
30:20I didn't take a course on weather maps.
30:22Neither did my mom or anyone else.
30:24I know, you know,
30:25they were built to take complex
30:27data and be able to portray it in
30:28a way that I immediately understand
30:30what it means within a a second
30:32And and yet it is taking high,
30:36you know, highly high,
30:38very high dimensional data.
30:39I mean an example of this was
30:42and I keep bringing up my mom
30:43because she's not you know,
30:44a tech sophisticated and and she's,
30:48you can imagine I'm older.
30:49She's older.
30:50She's an older individual.
30:51And you know one day she was she lives
30:53in Florida and and we were talking
30:56about the hurricane season and she says,
30:59you know,
30:59I saw the map and did you see the
31:01European model suggesting that the
31:03hurricane may go this way and the US
31:05model suggesting it goes that way.
31:06And I'm just thinking to myself like
31:08she's internalized that these are
31:10like predictive models and like it's
31:12it's actionable information and and
31:14and there's a level of uncertainty
31:15because there's a cone of uncertainty.
31:17She's got the confidence
31:19intervals around like what?
31:20What's going on?
31:21I mean,
31:21all of this stuff without her
31:22ever having to take a statistics
31:24class or understanding it.
31:25This is where you need to get to.
31:27I mean,
31:28I thought this was one of
31:29the best examples of it.
31:30So and by the way,
31:31this is a system where everybody's data
31:33is contributing to everyone's benefit.
31:35So the people on the road a mile
31:37ahead of me are helping me know what's
31:39what's going on on the road ahead.
31:40And I'm contributing data to
31:42the people a mile behind me.
31:43By the way, I love that for medicine.
31:45It's like the person who's six
31:47months ahead of me with a disease
31:48is contributing data that's helping
31:50me with my disease.
31:51Today,
31:51in my experience today is contributing
31:53to the person a mile behind me
31:55who's going to be like the only
31:57thing was no one really explicitly
31:58asked my permission about it.
32:00So that's the only thing.
32:01Maybe that's a little different,
32:02but but I it is a is a system of
32:05generosity where we're saying how
32:07fast I'm going and what's going on.
32:09And occasionally you know even like
32:11inputting data in that's helping to
32:14make the system smarter and better.
32:16But look how complex this is, right?
32:18Without ever having to take a course
32:20in ways or in any of the systems,
32:22you know, the way they use colors,
32:23the way you know that they've got icons.
32:26The the telling me times.
32:28I mean again,
32:30conveying very complex data that's
32:32coming from high dimensional information
32:34that's dynamic and changing minute by minute,
32:37you know is is become very useful.
32:38And this isn't even like talking
32:40about what's going on in in you know
32:42planes these days as planes become much
32:45safer and and you know there's no pilot.
32:48I mean flying is an information science.
32:51I mean you know you you've got,
32:52you've got to be able to provide the
32:55key information and and and improve
32:57performance of people in the cockpit and
32:59it's made such a difference in aviation.
33:02We're yet to really embrace this you
33:04know that to the ways in which you
33:06can improve our performance and and
33:07again this isn't just about the pilot.
33:09It's about the pilot and the teams and
33:11and the you know every single person
33:13hundreds of people contribute to each flight.
33:16So it it really is enhancing the
33:18communication of the coordination and
33:20the insurance that assurance that this
33:21is going to be safe and get successfully
33:24complete each of the missions.
33:26And again within medicine by the way,
33:27there's a ton of a I I mean every
33:30single radio graphic study that's you
33:32know complex and high dimensional is
33:34is leveraging you know a I in ways
33:36and and we're just at the beginning.
33:38I mean we're not even at the point
33:39of routine interpretation.
33:40But but all this is going on at the
33:43same time and in a connected world,
33:45you know this is going to be all this.
33:47So so anyway part of my my thing about
33:50this point is if we're really great with a I,
33:53which I hope some of you will
33:55be participating in,
33:55then actually clinicians won't need
33:57to know a lot about a I like what
34:00what they'll need to know is what
34:01they can trust and somehow we're
34:03going to have to be able to convey
34:04that trust with the regulations.
34:06But it shouldn't be like they're
34:07going to have to take courses in
34:09a I they're going to be experts
34:10in a I like that would be amiss.
34:12You know what we want them to be is be
34:14experts in interacting with patients
34:16and and in clinical decision making
34:17and and we want to assist them in
34:19pattern recognition and and we want
34:22to provide help to make sure nothing
34:24falls through the cracks and that their
34:26days can be as impactful as possible.
34:29So you know but what about
34:31medical miracles what what it,
34:32what is on the horizon and I just
34:34say you know we've been talked
34:35a lot about bench to bedside
34:36but I really think that the this future
34:39is calculation to clinic and you
34:41know this is going to be a quite an
34:44exciting moment I think as we get this.
34:47But the problem is, you know, today,
34:50you know, this is this famous painting,
34:53the Doctor, you know,
34:54the time when the doctors didn't
34:55have much that they could provide.
34:56But it it gave this sense of,
34:58you know, really caring doctor
35:00focused on the individual and.
35:01And what has technology done for us so far?
35:03Sort of this, which is, you know,
35:06turned our attention, if anything,
35:07away from our patients and distracted us.
35:11And and Gwande wrote this piece not
35:14too long ago, which was basically
35:16why doctors hate the computer.
35:17I mean, this is so ironic.
35:18You know,
35:19we're at this moment of immense potential.
35:21And the truth is most people
35:23on the front lines are like,
35:25I don't even want anything that's
35:28technological because everything you've
35:30given me so far has only made my life worse.
35:33And I have no sense that it's largely
35:35put me in a position to do a better job
35:38for my patients outside of things like,
35:40you know, the algorithms that are
35:41used to enhance imaging and so forth.
35:42But I mean, but largely on the front lines,
35:45I don't think that that people
35:47feel like they're being, you know,
35:48deluged with information.
35:49Now emails are coming every day.
35:51There's no sifting organization of
35:53those emails that they're working into
35:55late hours just documenting in ways
35:57that no one has ever even takes advantage of.
36:00And no one even trusts a lot of
36:02the unstructured data within the HR
36:03because so much basting and cutting.
36:04So it it's like you know people get
36:06in this position and go on to say I've
36:08come to feel a system that promised
36:10to increase my mastery over my work
36:12instead increase my works mastery over me.
36:15So I I don't blame people on the front
36:17lines for being skeptical that this
36:20can actually produce benefit And and
36:22there's also just a history of of a you know,
36:25things taking time.
36:26I mean when when Linneke, you know,
36:29developed the first stethoscope,
36:30but believe it or not,
36:31this is like the first stethoscope
36:32wasn't like a stethoscope.
36:33You see today.
36:34It was just a long tube and said that,
36:35you know, you could avoid it and it was
36:37to avoid putting your ear on people's chest.
36:38So in some ways it was for patient
36:41dignity and and comfort,
36:42but it also amplified this.
36:45But it it took a while before
36:46people would adopt this.
36:47They said no,
36:48you no tool is going to be better
36:49than me putting my ear on the,
36:51you know, someone's back to listen
36:53to the sounds and it it,
36:54you know, it took a while.
36:55But but in our era, by the way,
36:57I think this whole thing of
36:59a stethoscope is antiquated.
37:01I mean, I like it.
37:02It's a nice accouture.
37:03It's nice thing to wear.
37:04It makes me look like I'm a doctor.
37:07But asking medical students to pattern
37:12recognize sounds instead of using
37:15advanced acoustic devices to take
37:17those devices that take that raw data,
37:18process it and not put out.
37:21Like whether you heard Ronchi or Rawls,
37:23or whether you heard S1 or a split
37:26S2 or murmurs or GALLOPS or rubs
37:29interpreting it into an actual diagnosis.
37:3260% chance this person's got
37:34severe microbird station,
37:3530% chance this person's got this or that.
37:37I mean it can give me the interpretability
37:39of what that was based on.
37:40But the truth is,
37:42you know people's hearing changes
37:43over the course of their career.
37:45We have no closed loop and you know
37:48no one's telling us here's the gold
37:49standard we should continue to train on.
37:51And and the skills of physicians largely
37:53every time it's tested on auscultation
37:56or or or any sort of mostly pattern
37:59recognition is is not very good and
38:02yet you know we persist in in old ways.
38:05I wrote a piece in Wall Street Journal
38:07suggesting that that we would move
38:08from an error of the stethoscope into
38:10a new digital error and I got a whole
38:12bunch of emails from people calling me
38:14crazy and and you know that that this
38:16was so essential to the profession
38:18that people learn auscultation.
38:19I I'm not saying it's not but but
38:22by the way every time we tested
38:24people don't learn it anymore and
38:26and and maybe they shouldn't and I'll
38:28take this to electrocardiograms.
38:29I mean you know what we're teaching
38:30people to know the raw data.
38:32You know what what's the inflections
38:33of the waves?
38:34What's the P/E, QRST,
38:36waves and be able to get the patterns,
38:38but,
38:39but is this really what we should be doing?
38:41My daughter's a medical student and she
38:42just finished her cardiology rotation
38:44and I said that I think that's great.
38:46But I really think in five years you're not,
38:48you know,
38:48the the,
38:49the expert systems are going to be
38:51such that instead of asking you to
38:53describe the morphology of the ECG,
38:56we're going to be so much better
38:58being able to infer what does it mean,
39:00what does it mean?
39:01And so in at our place, Rohan Keira,
39:04it's just doing, you know,
39:06remarkable work being able to take
39:08not only taking the ECG and saying
39:11what would an expert cardiologist say,
39:13but being able to take the ECG
39:15and say what is it that even the
39:18expert cardiologist can't quite
39:19proceed because these are a cluster
39:22of changes that are nuanced and
39:24difficult to to recognize.
39:26He's not just saying,
39:28here's the textbook of electrocardiography.
39:30Now how can I automate that?
39:32That's 100 years or 150 years of
39:35120 years of of knowledge based on
39:40patterns correlating with certain,
39:42you know more whether it would
39:44be in a disease states or or
39:46abnormalities or rhythm disorders.
39:48And the truth is we can go beyond that,
39:51far beyond that.
39:52And that's what he's doing.
39:53You know, this is an article and it's
39:54just an example what he's done in
39:56detection of left ventricular systolic
39:59dysfunction from electrocardiographic
40:00images where we can actually infer.
40:02What you would find on an echocardiogram
40:05from the electrocardiogram in
40:07ways that that no prior ECG based
40:09textbook could ever describe.
40:10So it's not a matter of saying we can provide
40:13an expert cardiologist on your shoulder,
40:15but it's like we can actually go beyond that.
40:17And so you know he's got really a I
40:19enhanced screening of heart disease
40:21from ECG images as examples and can
40:23show you where it's leveraging that
40:25information within the electrocardiogram
40:27and and he's doing it on images,
40:28others are doing it on raw data.
40:31But you know this is this is going to
40:35change how we manage all of a medicine.
40:40Sanjay Niedia in radiation oncology,
40:43I think that we call it therapeutic
40:45radiology here that is also working
40:46with our group that in in what he's
40:49doing is it's same thing for Rohan
40:51is really finding digital biomarkers.
40:54So what is it that our eyes can't perceive?
40:56I mean the the radio graphic in this case,
40:58the radio graphic images contain such a
41:01depth of information that actually go
41:03beyond what we can proceed with our senses,
41:05with our,
41:06with our eyesight and and and and
41:09analyze with our brains.
41:11And so the idea is that we can be using
41:13these advanced analytics to be able to,
41:15for example,
41:16find clues that predict recurrences
41:18that we would not have been able
41:20to detect just by looking at it.
41:22Or you know,
41:23what happens still even today
41:24when we're comparing you know,
41:26prior images with current images,
41:29you know, you're kind of looking
41:30across or sometimes you know,
41:32maybe even trying to overlay,
41:33but we could be doing so much better.
41:34Is it really bigger or is not.
41:36You know,
41:36I hear radiologists go,
41:37I'm looking at this image,
41:38I'm looking at this and it looks
41:39like it's slightly bigger or they're
41:41they're putting calipers on it.
41:42I mean this is going to be gone
41:44because these kind of analytics
41:45can really take advantage of the
41:47three-dimensional nature of these images.
41:50I mean, you know,
41:52because we're looking at slices and
41:53then going down slices and trying to
41:55integrate in our brain like what it
41:57looks like and and all of this stuff
41:59is going to improve tremendously
42:00because of what we're going to be able to do.
42:02And then yeah,
42:03what's going on in the on the wards,
42:04you know,
42:05like when we're trying to decide
42:06about anticoagulation for someone
42:08with atrial fibrillation,
42:08isn't this just kills me.
42:10You know,
42:11we use things like Chad's vast score,
42:12which is, you know, Yes.
42:13No congestive heart failure. Yes.
42:16No hypertension. Yes. No diabetes.
42:18Yes. No vascular disease.
42:20I mean this is insane. I mean.
42:23Because, yes,
42:24no hypertension.
42:25I mean there are people who've got
42:27severe uncontrolled hypertension or
42:28people have died controlled hypertension.
42:29We've had a history of hypertension.
42:30They don't even have it anymore.
42:32But it's, you know,
42:33still a legacy comorbidity
42:34that they carry you giving all
42:36those people the same weight.
42:37And and this isn't very precise and
42:39it doesn't also take into account
42:41some populations have higher
42:43risks than other populations.
42:44So a score of two means something
42:46different in the stroke belt then
42:48it might mean within the Northeast.
42:50And and none of this is taken
42:52into account in these things and
42:53there's a need for a whole new
42:55range of predictive models that
42:56I call it say are digital native.
42:59They're not cross walking
43:00from registry data over,
43:02but they're actually being developed
43:03straight from the EHR And then we're
43:06doing a lot of work in blood pressure.
43:08You and Lou wrote this paper and
43:10she's doing a lot really I think
43:12novel work leveraging the electronic
43:13health records for population
43:14health where we can identify people
43:16who fall into the cracks,
43:17people who are being ignored.
43:19People have signatures of of
43:21hypertension that suggests they
43:22have secondary hypertension but no
43:24one's actually looking into that and
43:26and putting people into categories
43:28based on what's impeding them from
43:30achieving their hypertension.
43:31We we stop calling.
43:33There's this idea of secondary
43:35hypertension resistant hypertension
43:36which is largely from medical causes.
43:38But we start talking about persistent
43:40hypertension which can be from social
43:42determinants of health as well.
43:43And and of course if we can do this
43:46kind of systematic evaluation,
43:48the entire medical record,
43:49we can be doing this for HealthEquity
43:50too because people are left behind.
43:52People fall through the cracks
43:54are often minoritized populations.
43:56Populations that are disadvantaged
43:57populations have been subject to
43:59structural racism.
44:00We can actually automate ways to
44:01to make sure that people don't fall
44:03through the cracks because of the
44:05way we've set this up.
44:06Our group Lisa Souter in rheumatology
44:09and care endorsing Pediatrics and
44:11Q Lynn who's been with our group
44:13is with the school public health.
44:14You know I've been working on
44:16measure management.
44:16How can we begin to develop quality
44:19measures that come straight from the
44:21EHR and take into account the care
44:23all the characteristics of the individuals.
44:25Wade Schultz who was working
44:26with CORE for a long time,
44:28is now working with BIDS.
44:30He's in laboratory medicine,
44:31was developing A foundational system
44:33that the computational health
44:34platform or CHIP at Yale that was
44:36creating the possibilities of being
44:38able to do all this.
44:39You can't do this without sitting
44:41on a on a platform that enables
44:44that data to be secure, private,
44:46but but be able to be leveraged in
44:50this highly advanced analytic way.
44:52And and Wade contributed importantly
44:53to this work or like I said was very
44:56much involved with him as he was
44:58developing the computational health platform.
45:00So you know,
45:01this is Daphne Kohler who is in
45:04the life sciences and and has
45:06made tremendous contributions.
45:07But but you know it's been working
45:09in this bioinformatics
45:09space. But I think this is
45:12still generalizable to people
45:13interested in the clinical side.
45:15Machine learning is capable of making
45:17sense of immense amounts of high
45:19content biological data most of which
45:20is too high dimensional for humans to
45:23interpret and and I believe that that's
45:25true within the clinical domain too.
45:26It's it's not our failure it's just
45:28that as more and more data become
45:31available more high dimensional
45:33data become available multi
45:35what we call multimodal data.
45:37So I consider that social
45:39determined health exposures,
45:40biological data, clinical data,
45:42a whole range of information and all
45:44this is to put patients in a stronger
45:46position because ultimately their values,
45:48preferences and goals should dictate
45:50the decisions that they make.
45:51But you want it to be made based
45:54on really good information,
45:55information that's as highly
45:56specific to them as possible.
45:58And so the options are as clear
46:00as possible and that we we make
46:02it so that the system works more
46:06more seamlessly.
46:06So you know the as I get to the end here,
46:09it's just basically that what
46:12clinicians need are solutions,
46:14what they need are tools to
46:15improve what they in partnership
46:16with their patients can achieve.
46:18So that's where we have to start
46:20building you know solution based
46:22products that that are both
46:25helping us with accountability,
46:27improvement and discovery and and put
46:29us in a stronger position to to help
46:32people than we've been in the past.
46:34So, you know that's what I
46:36think clinicians need to know.
46:37They can trust you're going to see a
46:39whole new way of happening in and I
46:41think these large language models with,
46:43you know chat GB T,
46:44it's going to make all this
46:46much more accessible.
46:46And one of the ironies of this
46:48is that you're not going to have
46:49to be an expert coder anymore.
46:51I mean that that may not be exactly
46:53true today, but these these systems,
46:54the large language models are
46:56going to help with coding.
46:57They're going to help with application.
46:59I I I think they're going
47:00to be very important.
47:01It's just another dimension of A I
47:03where I'm going to be able to take the
47:05discharge summary and say to chat GB,
47:06T translate this into a a fourth grade level,
47:10translate this into an eighth grade level,
47:11translate this into a health
47:13literacy level of A or B.
47:14And we're it's going to help us
47:16communicate with our patients.
47:17It's going to help us to be
47:18able to ask questions,
47:19to get answers in ways that we
47:21haven't before and it's going to
47:23put us in a much stronger position.
47:24So the the future's there for us to take it.
47:26It's it that of course has
47:29potential for good or bad.
47:30So it's up to us to steer it towards good
47:34and then never to forget the bottom line,
47:36which is as we use this technology,
47:39it's a very important that we retain
47:41the humanity within our profession.
47:43Recognize the,
47:44the,
47:44the deep importance of connecting with
47:46our patients and ensuring that net net
47:48at the end of the day we've helped them in,
47:51in ways that they perceive
47:52as aligned with their values,
47:54preferences and goals.
47:55But you know,
47:57I envy the medical students because
47:59you're entering one of the most
48:01exciting periods ever in medicine.
48:02And over the course of your career,
48:04it will be amazing what you see.
48:06And I hope many of you will be
48:08principal participants in the steering
48:10of medicine towards a better day.
48:12So thank you very much.
48:14Appreciate being invited and and
48:16having this opportunity to talk to
48:21you. Awesome.
48:21Thank you so much, Doctor Krumholz.
48:23That was a very captivating talk.
48:24And I know you can't hear them,
48:25but I'm sure they're all clapping
48:27and oh, I'm sure, I'm sure
48:30awesome. So we do have a couple
48:31of questions that I'll just feel to you.
48:33One comment was about your
48:36talk about Google and how they,
48:38you know, each click and how
48:40they update the algorithms,
48:42you know, every single day.
48:45So this question,
48:46you know it's about on the comment
48:48about Google updating algorithms
48:49perhaps daily they don't need to
48:51go through an FDA approval.
48:52But is there a way, you know,
48:54once a healthcare algorithm is approved by
48:57the FDA and they and the base information,
48:59you know, that they can approve
49:01and update in an expedited fashion?
49:04Yeah, I think it's really good question.
49:05And you know I saw this with Apple,
49:07you know, I was talking to Apple
49:09about their AFIB algorithm and you
49:11know they actually made it better,
49:12they make it better all the time,
49:14but they actually can't implement it till
49:15they go back through an approval process.
49:17So this gets to the AI
49:19regulation piece, which is,
49:22you know, how how could this work
49:24where there was constant evaluation
49:26of the performance of an algorithm.
49:29I mean, should this be punctuated
49:31where like every year you put in a new
49:34algorithm for approval or could should
49:36it be continuous where the FDA can
49:38continually evaluate the performance
49:39of a of a model and see what's
49:41being leveraged within the company.
49:43But I think this is an area, right,
49:45for, for innovation in creative
49:47thinking because just as you
49:49say that it's higher stakes.
49:50I mean, if Google screws up a search,
49:51it's not, you know, not the big of deal.
49:53But, you know, if we're in medicine,
49:55we have to have higher standards and we
49:57have to derisk these algorithms to make
49:59sure they're doing what they're doing,
50:01You know, at the very least,
50:04even if we have to go through
50:06periodic upgrades,
50:07the the algorithm itself in the
50:09background should be capable of
50:11learning all the time and it it
50:13should also be capable of determining
50:15whether or not it's gotten worse.
50:17I mean in some ways that you may,
50:18many of you may know chats if you teach math.
50:20Some people have suggested and published,
50:23you know was worse than ChatGPT 3.5.
50:26How'd that happen?
50:27You know and you know so there needs
50:28to be ways to determine over time
50:30whether performance is degrading.
50:32It could be the patients are changing.
50:33It could be the information
50:34is changing that's coming in.
50:35It could be whole range of things
50:37that could perturb a certain model.
50:38I mean that's not unlikely it happened
50:40on the A-fib algorithm because it's
50:41just using a sensor from an Apple Watch,
50:43but but it could be happening
50:44in a lot of other ways.
50:45So I think we need to develop
50:47dynamic approaches,
50:50Yeah. And and you kind of
50:52already touched on this,
50:53this relates to kind of what
50:54you talked about in your answer.
50:55But next question,
50:56also wanted to kind of ask,
50:58you know, do you feel that a,
50:59I should be held to a higher
51:01standard than just being better
51:02than our current standard of care?
51:05I don't know. Would you take better,
51:06I mean, you know, it's like I'm I'm
51:09not sure any new innovation needs to
51:12be evaluated based on what benefit it
51:14provides and at what risk and at what cost.
51:18And so, you know, if it if it's better
51:23then you know, I I don't know what it means
51:26and maybe the questioner could clarify that.
51:29What would a higher standard mean?
51:31I mean to me it's there's so many
51:32things that we integrate into medicine.
51:34I mean robotic surgery,
51:35I mean it's never really been
51:36shown to improve outcomes.
51:37It costs millions of dollars.
51:39It's mostly used for advertising.
51:41You know, it's like, yeah,
51:42I don't think that was good,
51:44you know, to do that.
51:45You know, it it it maybe it's more
51:47fun for the surgeons, I don't know.
51:49But it's like I don't know,
51:50I'm not sure the benefit it's driving.
51:51We should be,
51:52we should be applying high
51:53standards to everything that
51:54gets implemented within medicine.
51:55We should be valuing it in those dimensions,
51:58safety,
51:58effectiveness and cost and
51:59figuring out how do we better
52:01serve the patient's needs.
52:07All right. So the next question you know
52:10ask you know considering you know certain
52:12specialties that rely very heavily on
52:14image analysis and pattern recognition.
52:17Do you think that a I would eventually
52:18replace you know specialties such as you
52:21know pathology or radiology in the future.
52:24You know look if you're heavily
52:25reliant on on pattern recognition
52:27then it's up to you as a specialty
52:29to show you what your value added
52:31is beyond pattern recognition.
52:33I I do believe there will be a
52:36day where the pattern recognition
52:38software will be able to provide
52:42comprehensive information about
52:45images and and and signals, right.
52:48And so I think that the
52:50essence of medicine will be
52:54for individual patients and and
52:56the human touch. How do you bring
52:58that information into a system?
53:00I mean what role does it play?
53:02I mean in the same way you could say
53:04will we have selfdriving cars with
53:06you always need to have a driver.
53:07Will you have do you need
53:09pilots in the cockpit.
53:10You know, I, I,
53:11you know the safe thing to say
53:12about this is in the way the AM a
53:14loves to talk about it so it doesn't
53:15offend anyone is to say you know
53:17this is augmented intelligence.
53:18We help humans perform better.
53:21But I think there may be a day where
53:23there at least is a requirement
53:26for fewer people to be able to
53:28provide the kind of oversight.
53:29I mean I'm not sure as humans
53:31that we will ever be completely
53:33comfortable with not having a human,
53:35you know, in in in the loop.
53:37You know it it it at least
53:39provides a degree of comfort,
53:41someone you can talk to interact
53:43with on a human basis and someone
53:45who has expertise that can
53:47help manage complex systems.
53:49But the question is will it
53:51ultimately enable us to say,
53:52you know one radiologist can actually
53:55oversee a larger number of a larger
53:57workload because the way in which
53:59this is going and and again this
54:01is you'd have to validate this.
54:03But but by the way our current system
54:05depending on humans is rife with errors too.
54:07So you know we we have to sort
54:09of manage this.
54:09I I don't have a position on it
54:12as much as think I want to be open
54:15minded about what's best for patients.
54:17And so then I don't want to there
54:19never should be a goal to say my
54:20goal is to ensure the persistence
54:22of my specialty or to ensure the
54:24persistence of my institutional
54:25organization or the way it exists.
54:28Because of the way it is we should
54:30always be thinking what is the ideal
54:32optimal configuration that provides
54:34the very best outcomes for our patients.
54:36And and by the way outcomes also
54:38includes their experience how
54:40they feel about it not just did
54:42they live or die or get get,
54:44you know are they healthier.
54:45But it will also be about their
54:48experience of the care because there,
54:50you know This is why I don't think
54:51that you know people aren't going
54:53to go away because kind of comfort
54:55you provide the the guidance
54:57the the help through difficult
54:59situations the holding vans.
55:01You know that there's an
55:02essential feature of medicine
55:03that's about the human aspects.
55:05But but the question is can this
55:07technology help us perform so much
55:09better and be maybe leverage you
55:11know force multiply our efforts in
55:14ways that that allows us to cover
55:16more people more efficiently in ways
55:18without losing the human part of it.
55:23Awesome. We have one more question.
55:26Do you feel that there
55:27are issues in applying AI,
55:28you know that it's derive from a
55:30large population based model to
55:32a more local healthcare system.
55:33So now it was like you know,
55:34other ways to kind of adjust you know,
55:36a large population data into a more,
55:38you know, local area.
55:40Yeah, I think that's a really,
55:41really good question.
55:42And by the way we don't do
55:44a good job of that mess.
55:45I mean that Chads Vasco is an example,
55:47we have one Chads Vasco for the entire
55:49world and yet you know you have a
55:50stroke belt where the risk of stroke
55:52has been 50% higher than is in New Haven.
55:54But we don't do any calibration
55:56of that model. I you know,
55:57I believe that in the future probably
55:59there's going to be fine tuning of
56:01models that you know where appropriate,
56:02when there are specifics to to populations
56:05that need to be taken into account.
56:08And if there's any question about that,
56:09if we get to precision medicine world,
56:11we want to be able to have our
56:13information be as specific to that
56:15individual if we're using models
56:17or maybe maybe corrections or fine
56:19tuning that needs to be done in
56:21order to to make sure that it's
56:24performance is best for that group.
56:25And and by the way,
56:26this is an issue that comes up where
56:27maybe if you've got for example white
56:29populations but you haven't included
56:31black populations or you know you
56:32don't have an included diversity of
56:34people within the training models,
56:36you can potentially be getting output
56:39that's not as relevant to the entire group.
56:41So it really is incumbent upon us to
56:42make sure that it is equally applicable.
56:46Awesome. So we are just about at 1:00.
56:51I don't see any more. Awesome.
56:53Yeah, I don't see any more questions anymore.
56:56Once again, thank you so much,
56:57Doctor Krumholtz,
56:58for joining us on our perspectives
57:01lecture and for giving a very great
57:03talk on AI and health and information age.
57:06And thank you everyone for coming in today.
57:08I just want to, you know,
57:10put on one more vouch for Doctor Krumholtz.
57:14And Doctor Foreman's podcast can
57:17attest that it's, it's very good.
57:20I'll just say that.
57:21Thank you so much.
57:22Thank you so much for that and.