2-22 MEDG: Master Adaptive Learner

February 22, 2024

ID11339

To CiteDCA Citation Guide

00:00On the Master Adaptive Learner model,
00:03an innovative approach to lifelong learning
00:06which is so critical for our students,
00:10residents and faculty.
00:11He currently is the Senior Associate
00:14Dean for Undergraduate Medical Education
00:16and the Associate Professor in PEDs at
00:19Critical Care Medicine at Vanderbilt,
00:21where you all know our Dean was
00:23Chair of Medicine there for a while.
00:26He's been engaged with us
00:28because Bill and our team here,
00:30it was John and Michael Green and
00:32Mike Schwartz and a few others,
00:34Dana on our EPA project.
00:36So we've known Bill for a number of years.
00:39What you're talking about today, Bill,
00:41you've done so much scholarship with some
00:43of our other medical education colleagues.
00:45It's so important to really
00:47think about lifelong learning.
00:49You've written the books,
00:51you've written articles and when I
00:53look back and I'm so proud that you
00:56know you you originally he went to
00:59medical school and and then you ended
01:02up as a in PEDs and you went to Baylor,
01:05then you moved to Vanderbilt and you
01:08also had you did a master's in education.
01:12So you're really very well versed in
01:16how does teaching promote learning.
01:18You've really participated in a number
01:21of professional organizations and you've
01:23been a real leader at the double AMC.
01:25When you think of initially it was also
01:27you were part of learning communities
01:30and then you were involved in pediatric
01:33training across the continuum early
01:35on And again when you're even I
01:37think you might still be doing this,
01:39but at least until 22 you were
01:41part of the accelerating change
01:43in medical education and that was
01:45part of the double AMC.
01:47And again such an important group
01:50when you look back of where we are
01:53with our core EP as you were very
01:56instrumental in leading your Vanderbilt
01:58group and you did that from 18 to 21.
02:01So I really appreciate you taking
02:03the time to work with us today
02:05and share your expertise.
02:06I'll pass it over to you
02:08Bill and thank you very much.
02:11Thank you for those kind words.
02:12I really am excited to spend
02:14some time with you all today.
02:15Hopefully you can see some slides.
02:17I'm going to just get situated so
02:18I can see your little zoom faces,
02:20which I appreciate.
02:20Those that are able to have the camera on.
02:22I know that everybody
02:24can't and that's OK too.
02:25I'm really excited to spend this time
02:28together today and I I hope that it
02:30will be useful and meaningful to you.
02:32Again, I I love thinking about this
02:34and talking about these topics.
02:35They're some of my absolute favorite.
02:37But I do hope that there will be at
02:39least one thing that is memorable useful
02:41for you practically and what you do.
02:44I will say at front I don't have
02:46any disclosures but as she alluded
02:47to this work did come from the AMA
02:49consortium accelerating change
02:50in medical education and I had
02:52some salary support during that.
02:53I was an editor on the master
02:54active learner book.
02:55But all of those royalties go to the AMA,
02:57not to me.
02:57I think it's a great resource,
02:59but I don't get anything out of
03:01it if you happen to to use it.
03:02And then the other disclosure and
03:03this was in the middle of the
03:05bio but I am a paediatrician.
03:06I'm a paediatric ICU doc.
03:07So I do see the world.
03:09My worldview is the as the through
03:10the lens of an intensivist.
03:12I get now in my my Dean role to work
03:14with a lot of other specialists and
03:15specialties and I really enjoy that
03:17and I've learned a ton from that.
03:18But at the core I'm still a paediatrician
03:21and so I just I I note that up front,
03:24All right.
03:24If we were physically together today,
03:25I would actually hand out note cards.
03:28Since we're not I will ask you to If
03:29you have a note card, great, grab it.
03:31If not grab a scrap of paper.
03:32Grab something that you can write on.
03:34And my hope is that by the end
03:36of our time together today,
03:37and I'll be watching the clock,
03:39I will leave time for question
03:41and answer at the end.
03:42I really hope we can have some some
03:44lively questions in in that time.
03:45I will also make my slides
03:46available so you can have those.
03:48But my hope is by the end of our time,
03:50there'll be at least one idea,
03:51one reference, one book, one topic,
03:54something that you want to track
03:54down after this 'cause you're like,
03:56oh,
03:56I want to know a little bit more about that,
03:58that would be helpful to me.
03:59So that's my hope.
04:00By the end of it,
04:00there's at least one hour and I can't tell
04:02if you have anything written on the card.
04:03It's just for you.
04:05But I do hope it is something that
04:07will be helpful for you, all right.
04:09So as I talked with your leadership
04:11group around what we really
04:12want you to be able to
04:13accomplish by the end of our time,
04:15you can see those three here to
04:16to really think about the basics
04:17of the master active learner model
04:19and why it might be helpful with
04:20relation to lifelong learning.
04:21To be able to discuss how faculty
04:23educators can acquire the skills
04:25to teach learners around this.
04:26And then to be able to explore how these,
04:29how explore strategies that might
04:30allow you to implement this in your
04:32own teaching and your own learning.
04:33And so we'll we'll think through
04:35some of that together.
04:36I will ask you to hold the tension of
04:39two separate although very much related
04:41lenses as we think through today's topic.
04:43The 1st is you are all educators
04:44and so I do want you thinking
04:46about your your learners,
04:47the learners that you work with or
04:49those are students or residents
04:50or fellows or interprofessional or
04:52across the board whatever learners
04:53you might work with really important.
04:55But I want to challenge you to
04:57actually think also about this lens
04:59about yourself as a lifelong learner
05:01and what ideas or concepts within
05:03this model could help you become even
05:05a more effective lifelong learner.
05:07And so I'll ask you to hold attention of
05:09both of those, thinking about your learners,
05:11but also thinking about yourself as
05:13a learner.
05:14All right.
05:15So the original paper came out in 2017.
05:17Again,
05:17you see the many great co-authors
05:19here that that helped us work
05:22through developing this model.
05:23And it really it started as a group
05:25that was looking at lifelong learning.
05:27And as we dove deeply into
05:28the health professions,
05:29education literature,
05:29the general education literature,
05:31There's been a lot written about that
05:33and there are a lot of separate theories,
05:35ideas, context.
05:35And what we wanted to do was to pull
05:39together a lot of that literature in a
05:41way that would create a shared language,
05:43a shared mental model for
05:45educators and trainees alike to
05:46really facilitate conversation.
05:48And so we hope that that's what came
05:50out of this work and we'll dive into
05:52the details of what the model actually
05:54looks like before or in a minute.
05:56But before that,
05:57I want to pause and ask two questions
06:00because if if I can get these right,
06:02I think the rest of it makes
06:03a lot more sense.
06:04So first of all,
06:05what is a master adaptive learner?
06:06And then, but even more importantly,
06:07why are they needed?
06:08If they are, if they're not needed,
06:10I'm about to waste an hour of your time.
06:11So hopefully I can convince
06:12you that they are needed,
06:13but that's for you to judge.
06:16So at the core,
06:17what we think a master adaptive learner is,
06:20is an individual individual who
06:23utilizes A metacognitive approach to
06:26self regulated learning that really
06:27is at the core of this that leads
06:30to adaptive expertise development.
06:31So there's a lot buried in just
06:33that kind of definition and I
06:34want to unpack just a little bit
06:36of it. I want to start with the
06:38the end goal of this type of
06:40learner being adaptive expertise
06:41and there's a lot of literature.
06:43I included some of these here and I
06:44in a minute I'm going to drop into
06:46the chat a handout and on the back
06:48of it will have a lot of references
06:50for those that want to take an even
06:51deeper dive into some of these.
06:53But the literature talks about an
06:55adaptive expert being able to do 2
06:57things or two big picture things.
06:59The first being to function very
07:01efficiently on everyday tasks
07:02that have known solutions.
07:04The literature describes that as routine
07:05expertise and it is very needed of
07:07health professionals across the board.
07:09When there is a known solution
07:10in the literature,
07:11you correctly diagnose the patient
07:12and you take that known solution
07:13from literature and apply it
07:15effectively and efficiently.
07:16Really, really important.
07:18And clinicians need the expertise to
07:22create solutions for workplace challenges
07:24that are novel that we haven't faced before.
07:26And that is where often times the
07:28medical training isn't setting
07:30our trainees up for success.
07:31So that as you think about
07:33it a little bit more,
07:34kind of in the weeds,
07:36the skills of an adaptive expert,
07:38first of all,
07:39starts with recognizing that
07:41routine approach won't work here.
07:43And there's a group of emergency
07:44medicine physicians that have done some
07:46really cool work building on this idea
07:48and they they've created this great
07:50academic medicine last page graphic
07:52that talks about being able to flip
07:54the switch from routine practice to
07:56adaptive practice based on the situation.
07:58So if it is a new and novel challenge
08:00where the routine approach won't work,
08:02the adaptive expert is able to flip the
08:04switch and take a different approach.
08:06And that different approach really allows
08:08the individual to reframe the problem,
08:10to explore new concepts,
08:11IE learning, woo Hoo,
08:12lifelong learning and to win
08:14appropriate invent new solutions,
08:15IE innovate.
08:16And so that those skills
08:19are incredibly important.
08:20The literature goes on to talk about that.
08:23This adoptive expertise is a
08:24product of a learned skill set.
08:26It is characterized by habits of mind and
08:28then develop over time and with practice.
08:31And so I'll tell you that we start talking
08:32about these ideas actually during the
08:34application cycle with our Med students.
08:36And then at orientation,
08:37they hear from me again.
08:38And then throughout the four
08:39years that they get with me,
08:40we are talking about these ideas with the
08:42goal of helping them develop this skill
08:45set and develop these habits of mind.
08:47All right, So four quick slides.
08:50Does the current system produce clinicians
08:52like this or framed a little bit differently,
08:55do practicing physicians learn and
08:57develop this type of expertise?
08:59So if you are a practicing physician,
09:01you're on the hook for the next 4 slides.
09:02If you are not,
09:03take a deep breath.
09:04You're off the hook for the next 4 slides.
09:06But I would make the argument
09:07that this is probably
09:08true of us as humans and not just physicians,
09:09but the literature. This,
09:11this paper specially is just about
09:13physicians and so this is a paper from
09:16Glenn Reguera and Maria Monopolos and it's,
09:17it's a little bit old now,
09:18but it it is so valuable in helping to
09:20frame what these clinicians are doing that I,
09:23I I still find it incredibly valuable.
09:25So they talk about four assumptions
09:27for practicing physicians.
09:28The first assumption is that we will
09:31naturally reflect on our own performance for
09:34the sake of highlighting weaknesses and gaps,
09:37which would be lovely if it's true.
09:38But in general,
09:39we discount evidence of poor performance.
09:41We discount feedback that is
09:43inconsistent with who we think
09:45we are as a great physician.
09:46I'm sure that's not true
09:47of you or your learners,
09:49but it is of me and it is of my learners,
09:50and it's what the literature would say.
09:52But let's say,
09:53for the sake of argument,
09:54we do try to identify our weaknesses.
09:56The second assumption is that we will
09:58actually be able to identify our
10:00weaknesses when we look for them,
10:01which unfortunately we're not
10:03good at either because in general
10:04we are horrible self assessors.
10:06There's a lot of literature around
10:08human ability to self assess,
10:10especially when it is uninformed
10:11by data and other things,
10:13especially in areas of weakness.
10:14We are really poor at that.
10:16So maybe we think we're funny
10:17when everybody else is like,
10:18no, you're not funny at all.
10:20But let's say for the sake of argument
10:21we try to find our weaknesses,
10:23we're actually able to identify them.
10:24The third is that we will then try
10:26to learn in a way that addresses
10:28those weaknesses very specifically.
10:29Surely we get this one right.
10:32No, we don't,
10:32because learning in areas of
10:34weakness is really hard,
10:35that we will in general gravitate
10:37towards things we already do.
10:38OK Like think about first
10:40year students for example.
10:41Are they going to learn a topic?
10:42Want to spend time learning a topic
10:44that they know something about or
10:46a topic they know nothing about?
10:47If your learners are like mine,
10:48they're going to spend more time
10:49on the things they already know a
10:51little bit about because it's easier.
10:52Which in my mind is the medical
10:54education equivalent of skipping leg day.
10:56No offence to this gentleman,
10:57I'd stole him from the Internet,
10:58but clearly he has done some work
11:00up here enhancing that area,
11:02but maybe to the exclusion of
11:03those little legs.
11:04I don't mean to leg shame him.
11:05Again, if you if you know him,
11:07I apologize.
11:07He was he was freely available
11:09on the Internet.
11:10Lastly, let's say for the sake of argument,
11:12we try to find our weaknesses.
11:13We're actually able to identify them.
11:15We then try to learn in a
11:16way that addresses them.
11:17The 4th assumption is that we will
11:19actually then change our practice
11:21based on what we have just learned.
11:23Unfortunately, we are not good at this
11:25either and even though this is from 2008,
11:27the data has continued to show,
11:29which is really depressing for those of
11:31us that do CME events that that think
11:33about teaching other faculty that faculty
11:35don't change their practice based on in
11:38general large scale CME type events,
11:40which is really disheartening, right?
11:42And so we need something different.
11:45And so I would love for you,
11:46if you're willing to participate in the chat,
11:49just thinking back on what we've
11:50just talked about and as it sets us
11:52up for the conversation to come.
11:53What are one or two ideas that stand
11:55out to you about lifelong learning and
11:57the need for master adaptive learners?
12:00If you can just drop those in the chat,
12:01that would be great.
12:13Give people just a minute or so.
12:20Yeah. The benefits of
12:21continuous self reflection.
12:22Agree completely.
12:25Ah, openness to new things.
12:26Yes, yes. self-awareness.
12:31You guys are great.
12:31Keep them coming.
12:34All
12:41right, so you can keep them coming.
12:44I'm going to keep.
12:45I will keep looking at them as I can.
12:47Oh, being able to pivot.
12:47I really like that one.
12:50So all of the things you just noted
12:52in addition to what I talked about,
12:53I think makes the argument that we
12:55do need this type of learner that
12:57approaches learning differently.
12:58And so let's dive into
12:59this shared mental model,
13:01the shared language that we
13:02tried to create with this model.
13:04And so I'm going to drop a handout
13:06as I mentioned into the chat,
13:12you feel free, I'll have it on the slides.
13:13But this is just if you want to
13:15also have it for reference or
13:16if you want to look at it later.
13:20There's a lot going on here.
13:21And so we'll actually walk through
13:22different parts of this model and
13:23spend a little bit of time in each.
13:24But I wanted you to get to see the overview
13:27first before we dive into the components,
13:30because each of the components
13:31has its own body of literature.
13:33You could do our talks on any
13:35component of this.
13:36And. And so again,
13:37this is going to be the high level overview,
13:39but hopefully to show you how it can
13:41connect and how it might be useful for
13:43you as a learner yourself and with
13:45those trainees that you work with.
13:47All right.
13:47So as you can see in the upper
13:49left around the little gears,
13:50there are four things and we're
13:52not going to belabour these,
13:54but these are the processes
13:55that we think should be going
13:57on throughout the learning,
13:58self monitoring,
13:59so being able to notice our own
14:00actions and being willing to kind of
14:02use those to improve our behaviour.
14:03The metacognition setting goals,
14:05planning an approach,
14:07monitoring our progress,
14:08making adjustments along the way,
14:09really important for learning in general.
14:12And most of us probably skip these steps.
14:15Reflection,
14:15one of you mentioned this in the in
14:18the chat which I absolutely love
14:19seeing that whether it's before during
14:21an A learning encounter or after
14:23reflection is incredibly impactful.
14:25And again there's a lot of great
14:27literature about reflection and how
14:28it really does help us gain greater
14:30understanding through its process.
14:31And then lastly,
14:32critical thinking,
14:33some of those higher order cognitive
14:35skills being a little bit more deliberate
14:38about our thinking at the core both of
14:40the definition and then of the model is,
14:42is this,
14:42it is at the core,
14:43it's self regulated learning
14:45strong literature based from the
14:48general education literature as
14:49well as the adaptation from Larry
14:52Grepin and Casey White and others
14:54to the Med Ed version planning,
14:56learning, assessing and adjusting.
14:57And for any of you that do quality
14:59improvement in the clinical space,
15:00this should also look really,
15:01really familiar.
15:02It very much mimics the PDSA
15:04cycle or plan do study,
15:06active quality improvement and
15:07we love the idea of our learners
15:10doing PDSA cycles on themselves.
15:12How can they improve add as learners,
15:14how can we as faculty improve as learners.
15:19So I want to just walk through
15:20each of these four gears.
15:21We'll start with the planning phase
15:22and I will continue to circle back and
15:24allow you to add things in the chat
15:25so that we can engage a little bit that way.
15:28Within the planning phase,
15:29there are three kind of core steps,
15:31the first being identifying a gap between
15:33what is and what could be or should be.
15:35So a gap in our own practice,
15:36our own knowledge,
15:37our own skills or attitude,
15:38Selecting one of those for learning
15:41and then searching out resources.
15:43And if you haven't seen this book,
15:44I highly recommend it.
15:45It's called Being Wrong by Catherine Schultz.
15:47She's also got a great Ted Talk if
15:49you prefer the Ted Talk version.
15:50Her premise is that in general,
15:52as humans, we hate being wrong.
15:53Which I'm like, Yep, Yep,
15:54that totally resonates.
15:55And her argument is that instead
15:58of running away from it,
15:59which most of us do,
16:00we should be running towards what?
16:02Where we're wrong.
16:03Because that's where the learning happens.
16:04And that's just such a different frame.
16:06And again, think about our students.
16:09Are they far more likely to
16:10tell you what they don't know?
16:12Or they're going to try to stand there
16:13on rounds and just be quiet because they
16:14don't want you to know what they don't know,
16:16right.
16:16One is really good for learning, one is not.
16:19And so we, again,
16:20we're not perfect by any means,
16:21but we are trying to create a
16:23culture where it's not only OK,
16:25but encouraged to say, oh,
16:26I don't really understand that.
16:28I I can't feel the liver.
16:29Nope, I didn't hear that murmur.
16:31I don't have any idea what's
16:32going on with this patient.
16:33Can you help me?
16:34Because that learning is far better
16:35than the student who just is.
16:37Like if I stand here quietly,
16:38maybe they won't know what I don't
16:40know and I will get a good mark.
16:41We want to embrace being wrong because
16:44that allows whether it's a knowledge,
16:46skill or attitude gap,
16:47it allows us to target that for improvement
16:50and ultimately provide better care.
16:53So I want you to think in the chat with me,
16:55how do trainees identify potential
16:57gaps in their practice And it's
16:59probably different based on if
17:00they're pre clerkship in the
17:02clerkship post clerkship a resident.
17:04It probably varies but I'll
17:06start with just trainees broadly.
17:08How might trainees identify their own gaps?
17:12And then don't worry.
17:13The follow up question is how
17:14do you identify gaps?
17:15Ha ha.
17:15It's a little bit easier though to
17:17think about trainees because that's
17:18them and we can talk about them
17:20and what they don't do perfectly.
17:21It's a little harder when we
17:22think about ourselves,
17:23but I would like you to think about both.
17:29Oh yeah. It. I agree, Michael,
17:31that it's an often times
17:32it's an affective response
17:36that we can have in a situation
17:38where we're recognizing a gap.
17:40They're asked to do something by a
17:41supervisor and don't know the answer.
17:42Yes. And then there's that moment of
17:45panic of I don't know how to do this.
17:46Should I try to fake it?
17:48Should I admit what I don't know?
17:50And that that is true of us too.
17:51At least it is of me.
17:52I can't speak for you,
17:53but I will tell you,
17:54especially if I've got a really
17:55good resident or a really good
17:57student who asked me a question
17:58on rounds and I'm like I have a
17:59moment to try to use the old tactic,
18:01well, that's a great question.
18:02Why don't you look that up
18:03and tell us tomorrow,
18:03which is a total stall tactic or
18:06for me to say I don't actually know,
18:08I don't know if I'm current with
18:09the literature on that topic.
18:10Let's let's look that up together
18:12and I will tell you the latter is far
18:14better role modeling and setting the
18:16stage and the tone that that is okay.
18:18And so I have had to get very
18:20comfortable saying I don't know and yes,
18:22we should try to figure this out together.
18:24And I love that comparing us ourselves
18:26to colleagues that is one of the ways
18:27that I know where my gaps are is,
18:29is I'm getting sign out from my
18:30colleagues or giving sign out and
18:32they ask questions and you're like
18:34I didn't think about that at all.
18:35Thank you for raising that.
18:36We should absolutely investigate
18:37that for this patient. OK.
18:39You guys are great. This is wonderful.
18:41All right. You can keep them coming.
18:43If you've got other ideas,
18:44I'm going to move to the learning gear.
18:46So I want us to think there for
18:48a minute where this is,
18:49I would argue most of my learners
18:51and and probably me as well,
18:52if I've got a little bit of time,
18:54I'm going to skip the planning.
18:55I'm going to dive right into the learning,
18:56just going to do the learning
18:58and that it that works.
19:00But I would argue it works far
19:01more effectively and probably more
19:03efficiently to actually spend that
19:04time upfront planning and then to use
19:06targeted learning as opposed to I'm
19:07just going to sit and read for a little bit.
19:10So this is the phase where we
19:11actually do engage in the learning.
19:13And so with that,
19:14I actually want to pause before
19:16I go on and and and ask you
19:18all what learning strategies do
19:20your trainees typically use?
19:21And it should probably be no shock at
19:23this time at all that the follow up
19:25is what typical strategies do you use.
19:27But let's start with trainees.
19:28What what strategies are your
19:30trainees often using when they are
19:32trying to learn a new concept or
19:33learn to address something that
19:34they feel like they don't know?
19:46And while you're typing,
19:47if you're memorizing, yes, 100%.
19:50If your students are anything like mine,
19:52I expect to see Anki or question banks.
19:54Oh, there's Anki. Yes,
19:55I'm glad it made it in there,
19:58which can be a good
20:00tool. But for the students that that
20:01is the only tool, or question banks,
20:03which is the for some students,
20:04that's the only tool that they use. I'm like,
20:07no, it it is a very helpful dipstick.
20:10It tells you what you know
20:11and what you don't know.
20:12That's really good for gap identification.
20:14But it is not a great strategy to really
20:17understand ideas and understand concepts.
20:19And so we try to encourage our
20:20students to use multiple strategies.
20:22You guys are great producing
20:23some of these strategies.
20:24And so then I'm going to switch it a
20:26little bit to you think about how you
20:28learn and what strategies you use.
20:30And I love this last comment
20:31about they do like podcasts.
20:33I think that's true of our learners as well.
20:34They hate textbooks.
20:35I love textbooks.
20:37It makes me so happy to have a textbook.
20:38And my learners, you're absolutely right,
20:40They absolutely don't like that.
20:41They want they want PDFs,
20:43they want PowerPoints,
20:44they want something that has been digested.
20:46And and so I think part of that
20:47is learning about our learners
20:48and how can we best support them.
20:50How do we provide great content,
20:52great ideas,
20:52but not things that can be easily memorized,
20:55easily forgotten,
20:56that don't actually lead to one of
20:57the things we'll talk about later,
20:59IE deep conceptual understanding.
21:02All right.
21:05So let's talk a little bit about
21:06some of these strategies,
21:07which I didn't see all of them pop up.
21:09I usually do,
21:09but I'm not going to ask for a show of hands.
21:12But rereading is one that I use highlighting
21:15and underlining absolutely also do it.
21:16And unfortunately,
21:17there is consistent literature
21:19that that is a horrible strategy.
21:21It does not lead to durable memory.
21:23It is very time consuming and
21:24it gives us this false sense
21:26of familiarity with the topic.
21:28So we think we know more than we actually do.
21:31Do I still default to rereading
21:33and highlighting under lining?
21:34I often do if I'm not being very intentional.
21:35So I'll just I will out myself there.
21:38I will not ask for a show of your hands,
21:39but I really this is a great book
21:41here in the that in the bottom
21:42corner Make it Stick which is
21:44written by educational psychologists.
21:46They're college professors and
21:47it really walks through a lot of
21:49the science but describes it in
21:51a very accessible way,
21:52written for more of a lay public,
21:54really great book.
21:54I've even talked with other schools that
21:56that use this book for all of their students.
21:58We haven't gotten that far,
21:59but I really we use a lot of it,
22:00little pieces of it.
22:01And in that book they make the
22:03quote that learning is deeper and
22:05more durable when it's effortful.
22:07So if it's not hard,
22:08we're probably not learning,
22:09which is a little bit depressing,
22:10right?
22:11It's like if it's easy it's probably
22:14not leading to long term understanding.
22:16And I love,
22:17I saw Retrieval practice show up in
22:18here. Absolutely.
22:19Knowledge retrieval strategy.
22:20So quizzing yourself can be
22:22a really effective strategy.
22:24Again, I just don't think
22:24it can be the only one.
22:25But knowledge retrieval practice
22:26is a really well proven strategy
22:29with great literature as is
22:30space repetitious learning,
22:32especially if you combine those two.
22:34Concept mapping is another,
22:36which is a fairly labour intensive way,
22:39but it actually forces the learners
22:40to put on paper or on computer
22:42how they think about things and
22:43how they see concepts organized
22:44and how they see them connected.
22:46And it's that process of making
22:48decisions that is really active
22:50and gets them thinking.
22:52This is a paper that I was part of and
22:53it's one of my favorite papers that
22:55I've ever been able to be part of.
22:56One, because I loved all the
22:57people I got to do it with.
22:58But two, it was really fun to wrestle with.
23:01These are great theories,
23:02these are great concepts,
23:04but practically what does that mean?
23:06And so these are 4 strategies
23:08in the literature,
23:09Deep Conceptual Understanding,
23:11Meaningful Variation,
23:12Productive Struggle and Discovery and
23:15Metacognitive Strategies that have been
23:17shown to lead to this type of expertise,
23:19learning when used well.
23:20And what we did in this paper was
23:22try to walk through at the UME level,
23:24talking about my Vanderbilt program.
23:26At the GME level,
23:27the four emergency medicine physicians
23:29talking about their different programs.
23:31And then Alyssa Hollow and Heather
23:32Billings at the Mayo Clinic talking
23:34about at a CPDCME level how
23:36these concepts could be applied.
23:38And so it's not exhaustive,
23:40but it is really how we tried to
23:43apply these concepts locally.
23:45To build on this idea that mastery
23:47requires both possession of ready knowledge,
23:49important, but the conceptual
23:51understanding of how to use it.
23:54And I'll be honest,
23:54this is what we hear from the students
23:56when we talk about these things.
23:57Like, yes,
23:57but that just takes too long.
23:59I can get through an AKI deck far faster
24:01than I can to sit and do a concept map.
24:04Yes, that is absolutely true.
24:06So we try to make the argument
24:07to them that much like compound
24:09interest in the banking world,
24:11they're making an initial investment
24:12in learning this and laying the
24:14foundation for these ideas and these
24:16concepts that will over time allow
24:17them to learn far more effectively and
24:19ultimately more efficiently as well.
24:22So I want us to think and in
24:24just a minute I'm going to give
24:26you opportunity to to brainstorm
24:28some specific ideas,
24:29but so we're going to walk
24:30through each of these.
24:30So just be thinking ahead.
24:33Deep conceptual understanding.
24:34Again,
24:34this moves our learners away from
24:37rote memorization,
24:38membring factoids so that you can
24:39get a question right on a test
24:41to really trying to understand
24:43concepts and ideas.
24:44And so there's a lot of different
24:45strategies that can be used to do that.
24:47We do some things in our curriculum
24:48where we try to circle back to
24:49the foundational science in their
24:51clerkship
24:51and in their post clerkship phase.
24:52To really try to accomplish that
24:55meaningful variation is a is a
24:57strategy that helps them see different
24:59presentations of the same concept.
25:01For example, how might
25:03appendicitis present differently?
25:04How might we give you multiple
25:06versions of somebody with a cough?
25:08They each have different things and so
25:10that you can start to compare and contrast.
25:12And I think that this is really
25:13important as we help them think through
25:16case based learning is a great way
25:18to do this clinically in the ICU,
25:19one of my favorite ways to do meaningful
25:21variation is we're talking about a given
25:23patient and then I change one variable.
25:24I say what if instead of being
25:27a one year old female,
25:29this was a 12 year old male.
25:30What would change about this
25:32presentation if or what if their
25:34renal function was poor because I
25:35can play the what if game forever and
25:38that's creating variation within the
25:39case that we're talking about next
25:42productive struggle and discovery.
25:44Again, this gets back to that idea
25:46that if it's easy,
25:47we're probably not learning.
25:48And one of the places that that
25:50I think this type of productive
25:52struggle and discovery is incredibly
25:54impactful is the simulation lab.
25:56You can create a lot of variation
25:57to connect back with the last one,
25:58but also allow the learners to struggle
26:01a little bit and to not be perfect
26:03and to allow no patients to be harmed
26:04in the middle of that struggle and that.
26:06And so we can think about that.
26:09And I just want to just make the
26:11connection between this idea of productive
26:12struggle and discovery and a huge body
26:14of literature from the Bjorks around
26:16this idea of desirable difficulties.
26:18That these really are not the undesirable
26:21difficulties of unnecessary stressors
26:23and unnecessarily hard things.
26:25But learning that is challenging
26:27because it offers opportunity to for
26:30encoding and retrieval processes,
26:32as already was alluded to with
26:33the retrieval practice,
26:35because ultimately that supports learning,
26:37comprehension and remembering.
26:39And so with that,
26:41I actually I'm going to drop a link
26:43to a Google Sheet in the chat.
26:45And for those that are able,
26:46I would love for you to pull up the
26:48Google Sheet and help me brainstorm
26:50a little bit.
26:51I'm going to also put it on the screen
26:53for those that aren't able to open it up.
26:55But I would love for you to just pick one
26:57of these 3 deep conceptual understanding,
26:59meaningful variation,
27:00productive struggle and discovery
27:02and just type in the box.
27:04Pick one of these boxes and just
27:06OfferUp a potential strategy that
27:07would allow that to work well with a
27:09student or a trainee or even yourself,
27:11and give you just a few minutes on that.
27:46One of the challenges in the
27:47Google Sheet will be if you're
27:49trying to overwrite somebody else.
27:50So if you have to pick one lower
27:51in the list, that's also good.
27:52But because look at these coming in.
27:54I love it. Yeah, concept mapping is a,
27:57in my mind, a great way to focus
28:00on deep conceptual understanding
28:05high context applications
28:06for questions. Yeah,
28:10Yep, extending a case can absolutely
28:12create some meaningful variation.
29:06Give people maybe one more
29:08minute and if you have,
29:09if as you're thinking about your ideas,
29:11you can also move over to
29:12the far right column of what
29:14barriers might get in the way of
29:15using one of these strategies.
29:21Again, I'm happy to to correlate these
29:23and save them and share them after
29:24the fact because you guys are putting
29:26up some great ideas of how you might
29:28actually do this in your context.
29:30And I can tell you what works in
29:31my context but I don't always know
29:32what will work in other contexts.
29:33So I I love that you are able to
29:35create this. All
29:41right, feel free to keep
29:43adding as the thoughts come.
29:44I will move to the 4th of the strategies
29:46which is metacognitive strategies.
29:48And this really is again what are the
29:50the different aids or tools we can
29:52use to help our our learners think
29:53about their thinking or how to help
29:56ourselves think about our thinking.
29:57And I would argue that the
29:59monster adaptive learner model in
30:00and of itself is one of those.
30:01If they're just feeling stuck,
30:03learning, walking through and helping
30:05them think about where they might be
30:07getting stuck can be really powerful.
30:09And I'm going to have a follow
30:10up conversation with some of the
30:12coaches around again how we might
30:13use this type of tool to facilitate
30:15coaching conversations because I
30:16think it can be very important,
30:18but to just to dive a little
30:19into the content,
30:20the assessing phase really is trying them,
30:22trying out what they've just learned.
30:25And it is a combination of
30:27informed self-assessment.
30:27So again,
30:28not just regular plain old
30:29navel gazing self-assessment,
30:30but informed usually by data
30:32or conversation or experience
30:34and external feedback.
30:35And I'll be honest,
30:36I don't know who writes
30:37the Schoolies cartoons,
30:38but I feel like they
30:39understand medical education,
30:40that I was shooting for a great
30:41dog and I only got a good dog.
30:42That's the equivalent of read more,
30:45which I'm sure nobody puts
30:46in your student assessments,
30:47but that happens a fair amount in hours.
30:50But really being able to partner some data
30:53that's coming in and an external perspective.
30:56And then lastly,
30:57thinking about adjusting,
30:58how do we actually change our practice?
30:59And again,
31:00I would argue, well,
31:01learners when they have some time are
31:02going to dive in and do the learning.
31:04They probably skip the planning.
31:05They usually skip the assessing.
31:07And almost all of us skip the adjusting
31:09of actually changing our practice
31:11based on what we've just learned.
31:13And so this is where we need to
31:14learn how to incorporate into
31:16practice what we've just learned.
31:17And so at the core,
31:19this is really change management.
31:20We get,
31:21we there's a lot we can learn
31:22from the business world who has
31:24thought and studied a lot over the
31:26years around change management.
31:27Part of it is how big of a
31:28change is actually required.
31:30And there's some great work in
31:32the late 80s from Fox and others
31:34around physicians in their
31:36own practice change around.
31:37The bigger the change,
31:38the harder it's going to be.
31:39So a small change like me remembering
31:41I need to use antibiotic Y instead
31:43of antibiotic X, that's on me.
31:45I just need to do that and we
31:46can even provide some decision
31:48support to try to prompt that.
31:49But if I was a surgeon in the
31:5090s that always took gallbladders
31:52out through a giant incision
31:53and this newfangled laparoscopic
31:55technology came along,
31:56that's a much bigger change and it
31:58impacts my identity as a surgeon.
32:00Not only that,
32:01I have to
32:02worry about training a whole staff,
32:04getting new equipment.
32:05So it's a much bigger change and
32:06so it requires far more deliberate
32:08change management. All right.
32:11So to pull out from the four gears,
32:12the planning, learning,
32:13assessing and adjusting,
32:14you can see that this all happens
32:16in a learning environment.
32:17I think that is just
32:18really to be cognizant of.
32:20There are things in our learning
32:22environment we often create
32:23and craft in our classroom.
32:24I can control things like the
32:26lighting and the temperature.
32:27Of note, it is always too cold.
32:28My students bring Snuggies and they
32:30think it's too cold in our building.
32:32But when I send them over to the hospital,
32:34there's a lot less control we have.
32:35And I would imagine this is
32:37true for your learners as well,
32:38as you think about some of the
32:39things that are happening in that
32:41clinical learning environment,
32:43lack of time, intra team dynamics.
32:45Again,
32:45I would never intentionally have
32:47my students see the internists
32:50making fun of the ER docs,
32:51making fun of the surgeons, MIT, right?
32:53But that happens in our system.
32:54We're working on it,
32:55We're trying to get better,
32:56but that happens and that's
32:57not good for learning.
32:58I would never want my students
32:59to see an obese patient being
33:01made fun of for their weight.
33:03And unfortunately that has
33:04happened in our system.
33:06And so we're working on that also.
33:07But those are things that are
33:08very much in the environment
33:09that our students are learning in
33:11that will impact how they learn.
33:13So we need to just be thoughtful about this.
33:16And then I not to get too controversial,
33:20but I think this is also one of the
33:22places where we set up certain expectations,
33:24right.
33:24So if if I have a graded clerkship
33:27where I'm asking students to compete
33:29versus each other for a grade that
33:32sends one message and that's one
33:34environment versus if I have a pass
33:36fail construct that allows them to
33:37to meet a certain bar and then get that.
33:40Now Full disclosure,
33:40I do have a pass fail clerkship
33:42year but my post clerkship 2 years
33:44are are graded honours high pass.
33:45So we have a a hybrid system
33:47that has some of both and we try
33:49to talk about why that is.
33:51But we are sending messages with the
33:53way we design policies and grading
33:55structures and across the board.
33:57And so that also is part of
33:59the learning environment.
34:00And so I'm going to drop a different
34:03Google Sheet in the chat and I would
34:05love for you to just think with me
34:06for probably two or three minutes.
34:08What are some of the factors
34:09in the learning environment,
34:10some that are positively impactful
34:11for this type of learning for
34:13both us and our trainees and
34:15maybe some that are negative.
34:16And so I will pull that back up over
34:18here as well so that we can see it.
34:22You should see two columns,
34:22a positive column and a negative column.
34:24And I would again two or three minutes.
34:25I would love to just think through
34:28with you some of the factors in the
34:31learning environment, broadly defined,
34:32that are either positive or
34:34negative impacts on our trainees.
34:36Learning and on our own learning.
34:41Oh my goodness, the contagion,
34:43effective FOMO, whoever put that in there,
34:45thank you so much.
34:46It is amazing how once two students
34:49start using something like Anki,
34:51then everybody's like, oh,
34:51I'm going to fail if I don't use
34:53it because they're using it.
34:54And it's amazing how quickly
34:56those circulate around.
34:57Yeah, I think Ed overcrowding is
35:00absolutely an impactor on learning,
35:02usually in the negative sense.
35:04I I it's very hard to come up with
35:05a positive sense where where that
35:07would actually improve learning.
35:08More patience maybe.
35:09But especially when they're in the hallway,
35:10at least they are in our
35:12situation when they're,
35:12you know they're boarding in the
35:13hall and think it's it's not good.
35:15It's not productive for learning.
35:16We would never design the
35:18learning environment that way.
35:19Said a different
35:23way. Ah, humble teachers.
35:25Yes, a really good skilled humble
35:27teacher can make such a difference.
35:29I love the role modeling.
35:30Absolutely. Teachers who are
35:33actually motivated to teach.
35:34Yes, yes, the USMLE obsession.
35:37I'm going to just,
35:38I'll retweet that one and
35:39and just leave it there.
35:40Strong inclusion there.
35:45All right. You can keep adding and again,
35:47I will make those available
35:49to the group afterwards.
35:50I want to wrap things up in the
35:52last few minutes so that we can
35:53take some time for questions.
35:55But I I will also just highlight
35:57as we're moving to the end,
35:58there are batteries that
35:59you see there at the bottom,
36:01curiosity, motivation, mindset,
36:02resilience that we think really
36:05power this type of learning.
36:07And it's it's come up several
36:08times in the chat already that
36:10idea of curiosity and mindset.
36:11And each of these could be
36:13their own hour talk or more
36:15great literature bases for each,
36:16but they are really important.
36:18So in the last couple of slides
36:19I want you to think through a
36:21couple of cases with me because
36:22I hope again that all of this
36:24has been helpful in some way.
36:25And so I want you to think
36:27through these cases with me.
36:28The 1st is Susie.
36:30She's a clinical student in your program.
36:31She is overwhelmed at the end of each
36:34day with all of the things she didn't know.
36:36She dutifully keeps a list of all of her
36:38unanswered questions in her little notebook,
36:39but can't ever seem to find time
36:42to investigate her answers.
36:43So if you were working with her,
36:45if you were her coach for example,
36:47thinking through where she
36:48might be getting stuck,
36:48I would argue she is not getting
36:50stuck at at gap identification.
36:52She's actually quite good at that.
36:53She's got a notebook full of them.
36:55It's the being able to select one
36:56of us where she's getting stuck.
36:57And so if you try to help her identify gaps,
36:59you're missing, you're missing her.
37:00She's done that and it's overwhelming.
37:02So if you can help look
37:03through her list and say,
37:04you know,
37:04right now you have a patient with diabetes,
37:06why don't we focus on your question
37:08here about electrolyte derangements
37:09in patients with diabetic ketoacidosis
37:11and learn on that And then because
37:13you've got a patient with that.
37:14And so I will be honest,
37:15I was this type of learner as a trainee.
37:17I had notebooks and papers,
37:18scraps and all kinds of
37:20things with questions.
37:20And it was just overwhelming to me at the
37:23end of the day of how to really learn that.
37:25All right.
37:26The next one is Tommy.
37:27He's a trainee in your program.
37:28He really wants to do well.
37:30He tries to spend time each
37:31night reading and rereading
37:33systematically through his textbook.
37:34He underlines, he highlights as he goes,
37:36but he never seems to remember
37:38what he's read.
37:39So where might Tommy's struggle be?
37:42I think you could make the argument
37:45that he hasn't effectively planned.
37:46So yes, probably some of that.
37:47But more importantly,
37:48he's not using effective learning strategies.
37:51And so I will also be honest
37:53that I was Tommy.
37:54I showed up to pediatric residency
37:55and was given an Oski's textbook
37:57and said read this cover to cover.
37:58You will be a great pediatrician,
38:00which turns out was horrible advice.
38:03I tried, I would fall asleep.
38:04I could never remember what I read.
38:06I really, I was trying to follow that
38:08instruction And for me that was, it was
38:10absolutely not an effective way to do it.
38:11All right. I want you to think with
38:13me on this last training, Sandra.
38:15She's an advanced trainee planning to
38:17enter a highly competitive subspecialty.
38:20I will allow you to determine which of those
38:21in your mind you're already thinking about.
38:23She's growing frustrated that her
38:25clinical assessments rate her
38:26lower than a perfect score.
38:28She believes that she is consistently
38:30demonstrating top performance
38:31despite what these assessments show.
38:33She makes comments like that.
38:35Attending just doesn't like me.
38:37I'd like to see that faculty member
38:39develop rapport with a rude patient
38:41to describe why her scores are lower.
38:43So where might Sandra's challenge be?
38:48So I would argue it's over here in
38:49the assessing. So she has some data.
38:51She is just currently choosing
38:53not to believe that data.
38:54And So what she really needs is an external
38:56perspective to help her think through,
38:58well, where might the truth be in that data
39:00or at times unfortunately to just say no.
39:03I have also seen that or
39:05the data is consistent.
39:06This is how you are being received.
39:07And so how, what are the strategies
39:10you could use to address that?
39:12All right, so we've covered a lot.
39:15I just want to quickly summarize and
39:17then we'll take some question and answer
39:19in just a minute before two things.
39:21So we started with what is expertise,
39:22we talked about routine expertise,
39:24that efficient application of known
39:26solutions and adaptive expertise
39:27that new learning and innovation
39:29to solve those novel challenges.
39:31We talked a lot about the how really
39:33focusing on the different gears,
39:35the planning, learning, assessing,
39:36adjusting of the model.
39:37We talked about four different
39:39potential strategies,
39:39focusing on deep conceptual understanding,
39:42meaningful variation,
39:43productive struggle and discovery,
39:46many cognitive strategies.
39:47We talked about the batteries just a
39:48little bit and then ultimately the
39:49learning environment and the coaching.
39:51And so I want you to pull back out your
39:53note card or your scrap of paper and
39:55I want you to think what was one thing,
39:57one ID or strategy that you would like to
40:00learn more about in the next two weeks.
40:02And I'm going to ask you to physically
40:03write it down or in just a minute
40:05they're going to drop a link in
40:07the chat for this evaluation of
40:08our our time together today.
40:10And you could also include
40:11it in the comments there.
40:13In thinking about what are you
40:14going to commit to do in the next
40:16two weeks to learn more about?
40:17And so I would encourage you either on
40:19your paper or in the survey or both,
40:21to think through what is the most important,
40:23what's the most meaningful thing you
40:25heard in the last hour and what's that
40:27one idea that you would like to learn
40:29more about in the next two weeks?
40:30I'm going to pause.
40:31We're going to have a little
40:32bit of zoom silence.
40:33Don't worry, don't get nervous.
40:36I will circle back in about 60 seconds.
41:13right, Feel free to keep writing
41:15or keep filling out your survey.
41:16With that, I'm about to pause for questions.
41:19I will leave you with the two pictures
41:21that I have on my wall in my office.
41:23The first is remember why you started
41:25and this to me is just that reminder
41:28of my original WHY was really to be an
41:30outstanding clinician to provide great,
41:32great care to my patients and their
41:34families and I hope I still do that.
41:37I still spend part of my time doing that.
41:38But my WHY has very much more become
41:40about my trainees now and helping
41:42them learn and develop and even more
41:44recently about my team and how do I
41:46support them to grow and develop.
41:48And then the the other is also just
41:50a challenge I hope is is true of
41:51all of us to always be learning.
41:53And with that,
41:54I'm I'm going to stop sharing
41:55and hopeful that we can have a
41:57little bit of conversation.
41:58I'm happy to try to answer some questions.
41:59Thank you for your time and attention.
42:01I really appreciate it.
42:02Oh, Bill, this was so wonderful.
42:05Thank you. We really appreciate it.
42:08So we have our Deputy Dean for Education,
42:11Jessica, Lucy put an educational
42:13strategic plan together.
42:14She's actually on the call today.
42:16And one of the very important aspects
42:18as we move forward is how do we really
42:20develop learners who are lifelong learners.
42:23And we do want to really
42:24look at the pedagogy.
42:25And we have hired a director of pedagogy.
42:28Bill Rando is with us,
42:30which we're very excited about that
42:32and really be thinking through.
42:34I know that Dana's heading the coaches
42:36program and you're familiar with that.
42:38But I would love to hear some of
42:40your advice in some of the aspects
42:42of what changes have you seen
42:44at Vanderbilt or other schools.
42:46When you look at whether it's
42:48Larry Gruppen or Martin,
42:50to really think about what kind
42:52of pedagogy could you we implement
42:54to make some of these changes.
42:57Yeah. Oh, I love it.
42:59I think there's a lot that can be done.
43:00And I will tell tell you
43:01some of what we've done.
43:02Not saying that it is
43:03perfect or the only way.
43:04I think we have tried to be very
43:06intentional in the way we talk about these
43:08ideas and these skills with our students.
43:10So instead of just hoping,
43:11which we used to do that they will just
43:13pick up this type of learning expertise,
43:15we're far more explicit now about
43:16talking about it and giving them words,
43:18giving them connections to go learn more,
43:20to give them opportunities to practice it.
43:22We've also learned a lot around
43:25better targeting and tailoring when
43:26we talk about different topics.
43:28So we don't actually start
43:30with self-assessment.
43:30We wait until our clerkship year,
43:32which is year two to do a
43:33little bit more about that.
43:34We start with learning strategies
43:36with the idea that they come in and
43:38they are starting to recognize that
43:39what got them to medical school might
43:41not be working as well as they liked.
43:43And so we use that as an opportunity
43:45to talk through some of the
43:47learning strategies ideas.
43:48I think that some of the other
43:49structural things we have done,
43:51we use a lot of case based learning in
43:53our first year of small group learning
43:55CBLT and and that has us has for us
43:57been really impactful and allowing
43:59them to get in there and wrestle.
44:01So some of that productive struggle and
44:03discovery seeing some variation within
44:05cases learning with and from each other.
44:07Pedagogically for us that
44:09has been really impactful.
44:12We did make AI alluded to this earlier,
44:14a change to our core clerkships
44:15when we moved them from year
44:17three to year 2 back in 2013.
44:18We did make it pass fail.
44:20I think that changes.
44:22We wanted it to truly be a foundational
44:24experience in the clerkships and it
44:27changes the ability to admit what you
44:29don't know when you're not trying
44:31to compete with the person next to
44:33you or trying to look perfect now.
44:34So Full disclosure,
44:35I also said we have graded immersion phase,
44:38so years 3:00 and 4:00 but even with that
44:42we we changed the way we did grades,
44:45we set here is an honours performance,
44:48here's the criteria for that
44:49and any student that met those
44:50criteria could get an honour.
44:51So they're no longer competing
44:52with each other which they were
44:54in our old system that they are,
44:55they know what the standard is and then
44:57they're competing with that or they're
44:58they're trying to perform to that.
45:00And so structurally for us
45:01that felt like it was a a,
45:03an approach that we that was more
45:05consistent with who we wanted
45:06to be in a CBME frame.
45:07We've we've determined these outcomes and
45:10now we're helping you work towards them.
45:12So those are some of the things
45:13that come to mind right away.
45:15I could happy to continue to talk,
45:16but I see two other hands.
45:17So let me try to answer those and
45:19we can circle back as as needed to
45:21some of the other examples here.
45:22Perfect. Thanks Vijay.
45:27Hi, thank you so much.
45:29This was so informative and engaging talk.
45:33I'm Vijay Karp.
45:34I'm an anesthesiologist and the Vice
45:37chair for Education in my department.
45:38And one of the things I've noticed in my
45:41interaction with the trainees is that a
45:44lot of them are using ChatGPT to get,
45:48you know, most of their information.
45:51So are you using any sort of,
45:55you know, training, incorporating,
45:57training on using these large language
45:59models for our trainees so that they
46:03can use them responsibly and ethically?
46:06And the second question is when do
46:09you see AI in terms of you know,
46:13helping in our journey towards being these,
46:16you know, towards this master
46:18adaptive learning.
46:21Thank you for the question.
46:22I really, I love it. I will.
46:25I'll start with the easy answer.
46:26No, we are not currently doing that.
46:27But we very much like your question have
46:30recognized students are using this.
46:32There is absolutely a set of competencies
46:34needed to use tools like that effectively.
46:37And so I just have on January 1st was
46:40able to add a person to my team with
46:42some FTE support to think about how do
46:44we incorporate this into our curriculum.
46:46So not just how do you use the tool,
46:48but what are the competencies to
46:51think about asking really effective
46:52questions because it turns out there's
46:54really a skill set around what
46:55prompts you enter into the the tools,
46:57whichever of them they are.
46:58And so we will,
47:00we are going to start incorporating
47:01that this fall in our first year
47:03curriculum around what are the core
47:06competencies related to informatics and
47:07AI that all of our graduates will need.
47:10And so I'm really excited.
47:11We're partnering with our Department
47:13of Biomedical Informatics to do that.
47:15And there's been a lot of concern
47:18about the ethical pieces that you've
47:20mentioned that you alluded to as well.
47:22And so we are working on having a Vanderbilt,
47:26a Vanderbilt instance of the the
47:27chat tools so that we can put in
47:29data to start to look at it.
47:30And so that the students are not
47:33necessarily putting in either
47:35copyrighted materials or faculty's
47:36materials or patient materials for sure
47:39into something that would would cut.
47:41So we're wrestling with that.
47:43And then to the last part,
47:44I think this is one of the areas
47:47where I'm learning so much from
47:48my colleagues that do this well.
47:50And the way that they have helped
47:52me think about it and frame it is
47:53these tools are that they are,
47:55they're just they're tools and if we
47:57learn to use them really effectively,
47:59there's a lot of the consolidation,
48:02scanning for different articles,
48:03organizing it can be done,
48:06but we're really early in that.
48:07So there's a lot of that, you know,
48:08you've heard about the,
48:09there's a fancy name for it,
48:10but basically the fake citations that
48:12they that they will come up with.
48:14And so there's a lot that we need
48:15to learn about. Hallucinations.
48:17Yes. Hallucinations, Yes.
48:17Like there's a word for it and I
48:18couldn't come up with it. Thank you.
48:20So I think there's so much that
48:22and that that.
48:23Right.
48:23Even if also if we're really honest,
48:24our learners will be far
48:25better at this than we are.
48:27And so we need to figure out how
48:29to rebuild the the skill set and
48:31the tools to keep up with them or
48:33at least to be able to have the
48:35conversation to then help prepare them
48:37for what could be a very different
48:38potentially practice of medicine
48:40or lifelong learning set of habits.
48:42At the core,
48:43I still think there is a lot of ideas,
48:45content, skills,
48:46attitudes that have to be learned,
48:49some of which large language models could
48:51help with, many of which they won't.
48:53And so then really trying to wrestle
48:55with great, where can these tools help
48:57so that we can put focus elsewhere
48:59for some of those other things.
49:00But thank you for the question.
49:02I love it. Great, Michael.
49:04Thanks again.
49:06Bill, I'm very interested in test
49:09enhanced learning which as you know
49:11is a form of retrieval practice,
49:13but the the potency of it really lives
49:17and dies by the quality of the question.
49:20And you know you have to remove the
49:22irrelevant difficulty and make it
49:24kind of a high context question that
49:27requires application of knowledge.
49:30And the thing that that really
49:32convinced me of this,
49:34which is this study that blew me away.
49:37I want if you don't know what
49:39I just wanted to do.
49:40They actually put students in F
49:44MRI machines and had them answer
49:48recall questions and then the
49:51other half was in the Mr.
49:53FM RI machine answering high context
49:56application questions and the FM RIS
49:59are completely different anatomically
50:03the ones with the high context
50:05question the parts of the cortex
50:07associated with deeper learning
50:09were lighting up But that just that
50:12you can't get better than that.
50:14No I oh thank you I have not seen that paper.
50:16I I remember vividly having a
50:18conversation around 2011 or 2012
50:20with some faculty here and saying
50:22wouldn't it be amazing if do
50:24exactly what you just described
50:26and we didn't but I wish we hadn't.
50:27I'll send it to you.
50:29Please do.
50:29I think it's by Dang DENGI think
50:33he's the first author I I was.
50:35I would also be very interested in
50:37getting that paper because it it
50:39relates very closely to work I'm doing.
50:41Thank you for bringing that up Michael.
50:43Sure.
50:43We'll send it to everybody. Just Michael
50:46just send it to Dorothy. Perfect.
50:49OK, I'll do that. Thanks Dana. Can I make.
50:53I just want to make one other comment
50:54on Michael's I I love the idea of of
50:56test enhanced learning and I will
50:58say we have struggled at figuring out
50:59how to implement that practically.
51:00Well, we have tried different things,
51:02one of which that I really thought
51:03was going to be awesome was in the
51:05middle of a a simulated patient
51:07case or standardized patient case.
51:08We gave the student the opportunity
51:10to come out to enter what they,
51:11what their differential diagnosis was to do,
51:14to have free access to the Internet to
51:17use resources and then to kind of hone
51:19their their differential and to go back in.
51:21And to your point,
51:22I think it was still helpful and it did give
51:25us an ability to assess how they were doing.
51:26But our assessment metrics just weren't
51:28weren't sensitive enough to really
51:30capture what they were doing and how
51:32well they were using different tools.
51:34And were they just going to Google
51:35or were they trying to use primary
51:36literature or were they using?
51:37So I think there's a lot of work that
51:39could be done to really better assess some
51:42of those skills in a test enhanced way.
51:44Yeah, absolutely.
51:44Michael has a fabulous systematic
51:46review and beamy that he put
51:48together on test Enhanced learning.
51:50I actually think you need real
51:52estate in the curriculum for it.
51:53So that's our big plug as we
51:56move forward. Janet always
51:57wants more real estate.
52:02Can I go ahead?
52:04Oh, thank you very much, Bill.
52:05And as you know, we,
52:06I was so taken by this model for our
52:09new coaching program that we are all
52:11in on the Master Adaptive Learner.
52:14And the thing I really liked about the book,
52:15not that the AMA is giving me
52:17any credit for plugging the
52:19Master Adaptive Learner book.
52:20But there's a chapter in that
52:21book for all the people on
52:23the call right now.
52:24People who are lecturing people who
52:26are teaching in the clinical setting.
52:27People are teaching in the GME setting.
52:30Because as individual
52:30faculty kind of the more
52:32we have a institutional framework
52:35around the things that we can do to plan
52:37our teaching because we can be so
52:39directive read that chapter do
52:41this thing and and we can't find
52:43the little coals that are already
52:45burning inside them. And our
52:47coaches are working on those batteries and
52:49charging them as matter type of learning.
52:51But they're so used to being controlled,
52:54what to do and why to do everything else.
52:56System I think helps more than potentially
52:58some other places that might be more
53:01restrictive or pro strict post post frictive.
53:05You know what I mean proscriptive.
53:08But I don't know if you could talk a
53:10little bit more about individual teaching
53:13on the fly, teaching in a clinical
53:14scenario that I don't want to sound
53:15like it's too big, but an example
53:18of you as an individual teacher being
53:21an Mal attending. Yeah. And and Bill,
53:24just before you make a plug, please,
53:27everyone start filling in your eval.
53:29We really appreciate all your comments
53:31while Bill answers Dana's question.
53:36Yeah, Dana, I, I appreciate the question.
53:38And I think there are,
53:41there's so many layers to what
53:43happened in a clinical environment.
53:44So my my time in the PEDs ICU is often
53:48the morning is focused on rounds and
53:49going patient by patient and all of
53:51that into that environment is one thing.
53:52The if we pull learners away to
53:54talk a little bit about a few
53:55things or it's embedded within.
53:56And so I think the strategies vary a little
53:59bit based on the context even within the
54:01context of on time in the in the ICU.
54:04One of the my learners would laugh at
54:05this because it's absolutely what I do.
54:07I almost never answer a question
54:09with with what they want to hear.
54:11They asked me what should we do here.
54:12I'm like well let's talk about what
54:13do you think we should do here.
54:14And so my main strategy one,
54:16because I think it engages their
54:17curiosity a little bit better,
54:19but two,
54:19it actually helps me assess
54:20their reasoning skills because I
54:22even if they're totally wrong,
54:23I want them to think through and
54:25ideally think through out loud what
54:27they think we should do because
54:28it allows me to really gauge their
54:30understanding of the cut context
54:31of the situation with the patient
54:33of the Physiology below that.
54:35And so I do a lot of question answering,
54:36no question asking and.
54:40And for me that is the strategy that
54:42helps me better tailor if I am going
54:44to drop in then little minutes of
54:46teaching to target it better so that
54:48I'm not teaching them teaching them
54:50something that they learned already
54:52or they already know something that I
54:53did a while ago and I haven't used it
54:55recently because it is fairly time intensive.
54:57But if there's a little bit more time,
54:59is asking them to do a concept
55:01map about a given patient,
55:04'cause it's really informative
55:05to quickly see, well,
55:06wow,
55:06you've got a great understanding
55:08about the microbes that might
55:09be causing this infection and
55:11the antibiotics and all of that.
55:12But you really,
55:13this part over here where you're
55:15talking about the symptoms,
55:16those aren't connected at all.
55:17So let's let's spend my time teaching there.
55:20Again, it requires a little bit more time,
55:21'cause you're asking the learner
55:22or learners to put on paper how
55:24they're thinking about something.
55:25But it's a really quick way to
55:27assess where are they strong,
55:28where might we spend some teaching
55:30time so that I'm not in my mind
55:32wasting their time.
55:33I think the other,
55:34and I alluded to this earlier,
55:35that's as important or maybe more
55:37important is the role modeling when
55:39I don't know something and to be
55:41willing to just upfront admit that
55:42they hear me tell families often,
55:44I don't know for sure what's
55:45going on with your child.
55:46Here's what we're doing to evaluate it.
55:48Here's what we think it could be,
55:49but I we don't know for sure.
55:52And so I hope that that builds in a little
55:54bit more comfort with uncertainty which
55:56I think again is embedded in medicine.
55:58But for learners that's hard, right.
56:00If you think about especially
56:02our pre clerkship learners,
56:03they want black and white,
56:05they want the ABCD.
56:06There should be always a right choice and
56:08it should be very clear which one it is.
56:10And most of the patients we take
56:12care of aren't that black and white
56:14and they so there's uncertainty and
56:16and so I also think if when there
56:17are times where I can highlight
56:19some of that uncertainty,
56:20I think it serves them well to
56:22have some conversation around it.
56:23That's especially difficult for
56:26oncologists and and you're you're
56:30acknowledging that who don't
56:32like to admit the uncertainty of
56:34treatments and it often leads to
56:36very devastating side effects.
56:38So I kudos
56:39for modeling that.
56:41Wonderful. So Bill, thank you so
56:43much for getting us excited about
56:45this and I'm sure we'll all walk away
56:48reading your papers in your book.
56:51So thank, thank you very much and
56:53thanks for everyone to participate.
56:54It was a great session.
56:56Thanks everybody. Really enjoy.