# 2-22 MEDG: Master Adaptive Learner

February 22, 2024ID11339

To CiteDCA Citation Guide

- 00:00On the Master Adaptive Learner model,
- 00:03an innovative approach to lifelong learning
- 00:06which is so critical for our students,
- 00:10residents and faculty.
- 00:11He currently is the Senior Associate
- 00:14Dean for Undergraduate Medical Education
- 00:16and the Associate Professor in PEDs at
- 00:19Critical Care Medicine at Vanderbilt,
- 00:21where you all know our Dean was
- 00:23Chair of Medicine there for a while.
- 00:26He's been engaged with us
- 00:28because Bill and our team here,
- 00:30it was John and Michael Green and
- 00:32Mike Schwartz and a few others,
- 00:34Dana on our EPA project.
- 00:36So we've known Bill for a number of years.
- 00:39What you're talking about today, Bill,
- 00:41you've done so much scholarship with some
- 00:43of our other medical education colleagues.
- 00:45It's so important to really
- 00:47think about lifelong learning.
- 00:49You've written the books,
- 00:51you've written articles and when I
- 00:53look back and I'm so proud that you
- 00:56know you you originally he went to
- 00:59medical school and and then you ended
- 01:02up as a in PEDs and you went to Baylor,
- 01:05then you moved to Vanderbilt and you
- 01:08also had you did a master's in education.
- 01:12So you're really very well versed in
- 01:16how does teaching promote learning.
- 01:18You've really participated in a number
- 01:21of professional organizations and you've
- 01:23been a real leader at the double AMC.
- 01:25When you think of initially it was also
- 01:27you were part of learning communities
- 01:30and then you were involved in pediatric
- 01:33training across the continuum early
- 01:35on And again when you're even I
- 01:37think you might still be doing this,
- 01:39but at least until 22 you were
- 01:41part of the accelerating change
- 01:43in medical education and that was
- 01:45part of the double AMC.
- 01:47And again such an important group
- 01:50when you look back of where we are
- 01:53with our core EP as you were very
- 01:56instrumental in leading your Vanderbilt
- 01:58group and you did that from 18 to 21.
- 02:01So I really appreciate you taking
- 02:03the time to work with us today
- 02:05and share your expertise.
- 02:06I'll pass it over to you
- 02:08Bill and thank you very much.
- 02:11Thank you for those kind words.
- 02:12I really am excited to spend
- 02:14some time with you all today.
- 02:15Hopefully you can see some slides.
- 02:17I'm going to just get situated so
- 02:18I can see your little zoom faces,
- 02:20which I appreciate.
- 02:20Those that are able to have the camera on.
- 02:22I know that everybody
- 02:24can't and that's OK too.
- 02:25I'm really excited to spend this time
- 02:28together today and I I hope that it
- 02:30will be useful and meaningful to you.
- 02:32Again, I I love thinking about this
- 02:34and talking about these topics.
- 02:35They're some of my absolute favorite.
- 02:37But I do hope that there will be at
- 02:39least one thing that is memorable useful
- 02:41for you practically and what you do.
- 02:44I will say at front I don't have
- 02:46any disclosures but as she alluded
- 02:47to this work did come from the AMA
- 02:49consortium accelerating change
- 02:50in medical education and I had
- 02:52some salary support during that.
- 02:53I was an editor on the master
- 02:54active learner book.
- 02:55But all of those royalties go to the AMA,
- 02:57not to me.
- 02:57I think it's a great resource,
- 02:59but I don't get anything out of
- 03:01it if you happen to to use it.
- 03:02And then the other disclosure and
- 03:03this was in the middle of the
- 03:05bio but I am a paediatrician.
- 03:06I'm a paediatric ICU doc.
- 03:07So I do see the world.
- 03:09My worldview is the as the through
- 03:10the lens of an intensivist.
- 03:12I get now in my my Dean role to work
- 03:14with a lot of other specialists and
- 03:15specialties and I really enjoy that
- 03:17and I've learned a ton from that.
- 03:18But at the core I'm still a paediatrician
- 03:21and so I just I I note that up front,
- 03:24All right.
- 03:24If we were physically together today,
- 03:25I would actually hand out note cards.
- 03:28Since we're not I will ask you to If
- 03:29you have a note card, great, grab it.
- 03:31If not grab a scrap of paper.
- 03:32Grab something that you can write on.
- 03:34And my hope is that by the end
- 03:36of our time together today,
- 03:37and I'll be watching the clock,
- 03:39I will leave time for question
- 03:41and answer at the end.
- 03:42I really hope we can have some some
- 03:44lively questions in in that time.
- 03:45I will also make my slides
- 03:46available so you can have those.
- 03:48But my hope is by the end of our time,
- 03:50there'll be at least one idea,
- 03:51one reference, one book, one topic,
- 03:54something that you want to track
- 03:54down after this 'cause you're like,
- 03:56oh,
- 03:56I want to know a little bit more about that,
- 03:58that would be helpful to me.
- 03:59So that's my hope.
- 04:00By the end of it,
- 04:00there's at least one hour and I can't tell
- 04:02if you have anything written on the card.
- 04:03It's just for you.
- 04:05But I do hope it is something that
- 04:07will be helpful for you, all right.
- 04:09So as I talked with your leadership
- 04:11group around what we really
- 04:12want you to be able to
- 04:13accomplish by the end of our time,
- 04:15you can see those three here to
- 04:16to really think about the basics
- 04:17of the master active learner model
- 04:19and why it might be helpful with
- 04:20relation to lifelong learning.
- 04:21To be able to discuss how faculty
- 04:23educators can acquire the skills
- 04:25to teach learners around this.
- 04:26And then to be able to explore how these,
- 04:29how explore strategies that might
- 04:30allow you to implement this in your
- 04:32own teaching and your own learning.
- 04:33And so we'll we'll think through
- 04:35some of that together.
- 04:36I will ask you to hold the tension of
- 04:39two separate although very much related
- 04:41lenses as we think through today's topic.
- 04:43The 1st is you are all educators
- 04:44and so I do want you thinking
- 04:46about your your learners,
- 04:47the learners that you work with or
- 04:49those are students or residents
- 04:50or fellows or interprofessional or
- 04:52across the board whatever learners
- 04:53you might work with really important.
- 04:55But I want to challenge you to
- 04:57actually think also about this lens
- 04:59about yourself as a lifelong learner
- 05:01and what ideas or concepts within
- 05:03this model could help you become even
- 05:05a more effective lifelong learner.
- 05:07And so I'll ask you to hold attention of
- 05:09both of those, thinking about your learners,
- 05:11but also thinking about yourself as
- 05:13a learner.
- 05:14All right.
- 05:15So the original paper came out in 2017.
- 05:17Again,
- 05:17you see the many great co-authors
- 05:19here that that helped us work
- 05:22through developing this model.
- 05:23And it really it started as a group
- 05:25that was looking at lifelong learning.
- 05:27And as we dove deeply into
- 05:28the health professions,
- 05:29education literature,
- 05:29the general education literature,
- 05:31There's been a lot written about that
- 05:33and there are a lot of separate theories,
- 05:35ideas, context.
- 05:35And what we wanted to do was to pull
- 05:39together a lot of that literature in a
- 05:41way that would create a shared language,
- 05:43a shared mental model for
- 05:45educators and trainees alike to
- 05:46really facilitate conversation.
- 05:48And so we hope that that's what came
- 05:50out of this work and we'll dive into
- 05:52the details of what the model actually
- 05:54looks like before or in a minute.
- 05:56But before that,
- 05:57I want to pause and ask two questions
- 06:00because if if I can get these right,
- 06:02I think the rest of it makes
- 06:03a lot more sense.
- 06:04So first of all,
- 06:05what is a master adaptive learner?
- 06:06And then, but even more importantly,
- 06:07why are they needed?
- 06:08If they are, if they're not needed,
- 06:10I'm about to waste an hour of your time.
- 06:11So hopefully I can convince
- 06:12you that they are needed,
- 06:13but that's for you to judge.
- 06:16So at the core,
- 06:17what we think a master adaptive learner is,
- 06:20is an individual individual who
- 06:23utilizes A metacognitive approach to
- 06:26self regulated learning that really
- 06:27is at the core of this that leads
- 06:30to adaptive expertise development.
- 06:31So there's a lot buried in just
- 06:33that kind of definition and I
- 06:34want to unpack just a little bit
- 06:36of it. I want to start with the
- 06:38the end goal of this type of
- 06:40learner being adaptive expertise
- 06:41and there's a lot of literature.
- 06:43I included some of these here and I
- 06:44in a minute I'm going to drop into
- 06:46the chat a handout and on the back
- 06:48of it will have a lot of references
- 06:50for those that want to take an even
- 06:51deeper dive into some of these.
- 06:53But the literature talks about an
- 06:55adaptive expert being able to do 2
- 06:57things or two big picture things.
- 06:59The first being to function very
- 07:01efficiently on everyday tasks
- 07:02that have known solutions.
- 07:04The literature describes that as routine
- 07:05expertise and it is very needed of
- 07:07health professionals across the board.
- 07:09When there is a known solution
- 07:10in the literature,
- 07:11you correctly diagnose the patient
- 07:12and you take that known solution
- 07:13from literature and apply it
- 07:15effectively and efficiently.
- 07:16Really, really important.
- 07:18And clinicians need the expertise to
- 07:22create solutions for workplace challenges
- 07:24that are novel that we haven't faced before.
- 07:26And that is where often times the
- 07:28medical training isn't setting
- 07:30our trainees up for success.
- 07:31So that as you think about
- 07:33it a little bit more,
- 07:34kind of in the weeds,
- 07:36the skills of an adaptive expert,
- 07:38first of all,
- 07:39starts with recognizing that
- 07:41routine approach won't work here.
- 07:43And there's a group of emergency
- 07:44medicine physicians that have done some
- 07:46really cool work building on this idea
- 07:48and they they've created this great
- 07:50academic medicine last page graphic
- 07:52that talks about being able to flip
- 07:54the switch from routine practice to
- 07:56adaptive practice based on the situation.
- 07:58So if it is a new and novel challenge
- 08:00where the routine approach won't work,
- 08:02the adaptive expert is able to flip the
- 08:04switch and take a different approach.
- 08:06And that different approach really allows
- 08:08the individual to reframe the problem,
- 08:10to explore new concepts,
- 08:11IE learning, woo Hoo,
- 08:12lifelong learning and to win
- 08:14appropriate invent new solutions,
- 08:15IE innovate.
- 08:16And so that those skills
- 08:19are incredibly important.
- 08:20The literature goes on to talk about that.
- 08:23This adoptive expertise is a
- 08:24product of a learned skill set.
- 08:26It is characterized by habits of mind and
- 08:28then develop over time and with practice.
- 08:31And so I'll tell you that we start talking
- 08:32about these ideas actually during the
- 08:34application cycle with our Med students.
- 08:36And then at orientation,
- 08:37they hear from me again.
- 08:38And then throughout the four
- 08:39years that they get with me,
- 08:40we are talking about these ideas with the
- 08:42goal of helping them develop this skill
- 08:45set and develop these habits of mind.
- 08:47All right, So four quick slides.
- 08:50Does the current system produce clinicians
- 08:52like this or framed a little bit differently,
- 08:55do practicing physicians learn and
- 08:57develop this type of expertise?
- 08:59So if you are a practicing physician,
- 09:01you're on the hook for the next 4 slides.
- 09:02If you are not,
- 09:03take a deep breath.
- 09:04You're off the hook for the next 4 slides.
- 09:06But I would make the argument
- 09:07that this is probably
- 09:08true of us as humans and not just physicians,
- 09:09but the literature. This,
- 09:11this paper specially is just about
- 09:13physicians and so this is a paper from
- 09:16Glenn Reguera and Maria Monopolos and it's,
- 09:17it's a little bit old now,
- 09:18but it it is so valuable in helping to
- 09:20frame what these clinicians are doing that I,
- 09:23I I still find it incredibly valuable.
- 09:25So they talk about four assumptions
- 09:27for practicing physicians.
- 09:28The first assumption is that we will
- 09:31naturally reflect on our own performance for
- 09:34the sake of highlighting weaknesses and gaps,
- 09:37which would be lovely if it's true.
- 09:38But in general,
- 09:39we discount evidence of poor performance.
- 09:41We discount feedback that is
- 09:43inconsistent with who we think
- 09:45we are as a great physician.
- 09:46I'm sure that's not true
- 09:47of you or your learners,
- 09:49but it is of me and it is of my learners,
- 09:50and it's what the literature would say.
- 09:52But let's say,
- 09:53for the sake of argument,
- 09:54we do try to identify our weaknesses.
- 09:56The second assumption is that we will
- 09:58actually be able to identify our
- 10:00weaknesses when we look for them,
- 10:01which unfortunately we're not
- 10:03good at either because in general
- 10:04we are horrible self assessors.
- 10:06There's a lot of literature around
- 10:08human ability to self assess,
- 10:10especially when it is uninformed
- 10:11by data and other things,
- 10:13especially in areas of weakness.
- 10:14We are really poor at that.
- 10:16So maybe we think we're funny
- 10:17when everybody else is like,
- 10:18no, you're not funny at all.
- 10:20But let's say for the sake of argument
- 10:21we try to find our weaknesses,
- 10:23we're actually able to identify them.
- 10:24The third is that we will then try
- 10:26to learn in a way that addresses
- 10:28those weaknesses very specifically.
- 10:29Surely we get this one right.
- 10:32No, we don't,
- 10:32because learning in areas of
- 10:34weakness is really hard,
- 10:35that we will in general gravitate
- 10:37towards things we already do.
- 10:38OK Like think about first
- 10:40year students for example.
- 10:41Are they going to learn a topic?
- 10:42Want to spend time learning a topic
- 10:44that they know something about or
- 10:46a topic they know nothing about?
- 10:47If your learners are like mine,
- 10:48they're going to spend more time
- 10:49on the things they already know a
- 10:51little bit about because it's easier.
- 10:52Which in my mind is the medical
- 10:54education equivalent of skipping leg day.
- 10:56No offence to this gentleman,
- 10:57I'd stole him from the Internet,
- 10:58but clearly he has done some work
- 11:00up here enhancing that area,
- 11:02but maybe to the exclusion of
- 11:03those little legs.
- 11:04I don't mean to leg shame him.
- 11:05Again, if you if you know him,
- 11:07I apologize.
- 11:07He was he was freely available
- 11:09on the Internet.
- 11:10Lastly, let's say for the sake of argument,
- 11:12we try to find our weaknesses.
- 11:13We're actually able to identify them.
- 11:15We then try to learn in a
- 11:16way that addresses them.
- 11:17The 4th assumption is that we will
- 11:19actually then change our practice
- 11:21based on what we have just learned.
- 11:23Unfortunately, we are not good at this
- 11:25either and even though this is from 2008,
- 11:27the data has continued to show,
- 11:29which is really depressing for those of
- 11:31us that do CME events that that think
- 11:33about teaching other faculty that faculty
- 11:35don't change their practice based on in
- 11:38general large scale CME type events,
- 11:40which is really disheartening, right?
- 11:42And so we need something different.
- 11:45And so I would love for you,
- 11:46if you're willing to participate in the chat,
- 11:49just thinking back on what we've
- 11:50just talked about and as it sets us
- 11:52up for the conversation to come.
- 11:53What are one or two ideas that stand
- 11:55out to you about lifelong learning and
- 11:57the need for master adaptive learners?
- 12:00If you can just drop those in the chat,
- 12:01that would be great.
- 12:13Give people just a minute or so.
- 12:20Yeah. The benefits of
- 12:21continuous self reflection.
- 12:22Agree completely.
- 12:25Ah, openness to new things.
- 12:26Yes, yes. self-awareness.
- 12:31You guys are great.
- 12:31Keep them coming.
- 12:34All
- 12:41right, so you can keep them coming.
- 12:44I'm going to keep.
- 12:45I will keep looking at them as I can.
- 12:47Oh, being able to pivot.
- 12:47I really like that one.
- 12:50So all of the things you just noted
- 12:52in addition to what I talked about,
- 12:53I think makes the argument that we
- 12:55do need this type of learner that
- 12:57approaches learning differently.
- 12:58And so let's dive into
- 12:59this shared mental model,
- 13:01the shared language that we
- 13:02tried to create with this model.
- 13:04And so I'm going to drop a handout
- 13:06as I mentioned into the chat,
- 13:12you feel free, I'll have it on the slides.
- 13:13But this is just if you want to
- 13:15also have it for reference or
- 13:16if you want to look at it later.
- 13:20There's a lot going on here.
- 13:21And so we'll actually walk through
- 13:22different parts of this model and
- 13:23spend a little bit of time in each.
- 13:24But I wanted you to get to see the overview
- 13:27first before we dive into the components,
- 13:30because each of the components
- 13:31has its own body of literature.
- 13:33You could do our talks on any
- 13:35component of this.
- 13:36And. And so again,
- 13:37this is going to be the high level overview,
- 13:39but hopefully to show you how it can
- 13:41connect and how it might be useful for
- 13:43you as a learner yourself and with
- 13:45those trainees that you work with.
- 13:47All right.
- 13:47So as you can see in the upper
- 13:49left around the little gears,
- 13:50there are four things and we're
- 13:52not going to belabour these,
- 13:54but these are the processes
- 13:55that we think should be going
- 13:57on throughout the learning,
- 13:58self monitoring,
- 13:59so being able to notice our own
- 14:00actions and being willing to kind of
- 14:02use those to improve our behaviour.
- 14:03The metacognition setting goals,
- 14:05planning an approach,
- 14:07monitoring our progress,
- 14:08making adjustments along the way,
- 14:09really important for learning in general.
- 14:12And most of us probably skip these steps.
- 14:15Reflection,
- 14:15one of you mentioned this in the in
- 14:18the chat which I absolutely love
- 14:19seeing that whether it's before during
- 14:21an A learning encounter or after
- 14:23reflection is incredibly impactful.
- 14:25And again there's a lot of great
- 14:27literature about reflection and how
- 14:28it really does help us gain greater
- 14:30understanding through its process.
- 14:31And then lastly,
- 14:32critical thinking,
- 14:33some of those higher order cognitive
- 14:35skills being a little bit more deliberate
- 14:38about our thinking at the core both of
- 14:40the definition and then of the model is,
- 14:42is this,
- 14:42it is at the core,
- 14:43it's self regulated learning
- 14:45strong literature based from the
- 14:48general education literature as
- 14:49well as the adaptation from Larry
- 14:52Grepin and Casey White and others
- 14:54to the Med Ed version planning,
- 14:56learning, assessing and adjusting.
- 14:57And for any of you that do quality
- 14:59improvement in the clinical space,
- 15:00this should also look really,
- 15:01really familiar.
- 15:02It very much mimics the PDSA
- 15:04cycle or plan do study,
- 15:06active quality improvement and
- 15:07we love the idea of our learners
- 15:10doing PDSA cycles on themselves.
- 15:12How can they improve add as learners,
- 15:14how can we as faculty improve as learners.
- 15:19So I want to just walk through
- 15:20each of these four gears.
- 15:21We'll start with the planning phase
- 15:22and I will continue to circle back and
- 15:24allow you to add things in the chat
- 15:25so that we can engage a little bit that way.
- 15:28Within the planning phase,
- 15:29there are three kind of core steps,
- 15:31the first being identifying a gap between
- 15:33what is and what could be or should be.
- 15:35So a gap in our own practice,
- 15:36our own knowledge,
- 15:37our own skills or attitude,
- 15:38Selecting one of those for learning
- 15:41and then searching out resources.
- 15:43And if you haven't seen this book,
- 15:44I highly recommend it.
- 15:45It's called Being Wrong by Catherine Schultz.
- 15:47She's also got a great Ted Talk if
- 15:49you prefer the Ted Talk version.
- 15:50Her premise is that in general,
- 15:52as humans, we hate being wrong.
- 15:53Which I'm like, Yep, Yep,
- 15:54that totally resonates.
- 15:55And her argument is that instead
- 15:58of running away from it,
- 15:59which most of us do,
- 16:00we should be running towards what?
- 16:02Where we're wrong.
- 16:03Because that's where the learning happens.
- 16:04And that's just such a different frame.
- 16:06And again, think about our students.
- 16:09Are they far more likely to
- 16:10tell you what they don't know?
- 16:12Or they're going to try to stand there
- 16:13on rounds and just be quiet because they
- 16:14don't want you to know what they don't know,
- 16:16right.
- 16:16One is really good for learning, one is not.
- 16:19And so we, again,
- 16:20we're not perfect by any means,
- 16:21but we are trying to create a
- 16:23culture where it's not only OK,
- 16:25but encouraged to say, oh,
- 16:26I don't really understand that.
- 16:28I I can't feel the liver.
- 16:29Nope, I didn't hear that murmur.
- 16:31I don't have any idea what's
- 16:32going on with this patient.
- 16:33Can you help me?
- 16:34Because that learning is far better
- 16:35than the student who just is.
- 16:37Like if I stand here quietly,
- 16:38maybe they won't know what I don't
- 16:40know and I will get a good mark.
- 16:41We want to embrace being wrong because
- 16:44that allows whether it's a knowledge,
- 16:46skill or attitude gap,
- 16:47it allows us to target that for improvement
- 16:50and ultimately provide better care.
- 16:53So I want you to think in the chat with me,
- 16:55how do trainees identify potential
- 16:57gaps in their practice And it's
- 16:59probably different based on if
- 17:00they're pre clerkship in the
- 17:02clerkship post clerkship a resident.
- 17:04It probably varies but I'll
- 17:06start with just trainees broadly.
- 17:08How might trainees identify their own gaps?
- 17:12And then don't worry.
- 17:13The follow up question is how
- 17:14do you identify gaps?
- 17:15Ha ha.
- 17:15It's a little bit easier though to
- 17:17think about trainees because that's
- 17:18them and we can talk about them
- 17:20and what they don't do perfectly.
- 17:21It's a little harder when we
- 17:22think about ourselves,
- 17:23but I would like you to think about both.
- 17:29Oh yeah. It. I agree, Michael,
- 17:31that it's an often times
- 17:32it's an affective response
- 17:36that we can have in a situation
- 17:38where we're recognizing a gap.
- 17:40They're asked to do something by a
- 17:41supervisor and don't know the answer.
- 17:42Yes. And then there's that moment of
- 17:45panic of I don't know how to do this.
- 17:46Should I try to fake it?
- 17:48Should I admit what I don't know?
- 17:50And that that is true of us too.
- 17:51At least it is of me.
- 17:52I can't speak for you,
- 17:53but I will tell you,
- 17:54especially if I've got a really
- 17:55good resident or a really good
- 17:57student who asked me a question
- 17:58on rounds and I'm like I have a
- 17:59moment to try to use the old tactic,
- 18:01well, that's a great question.
- 18:02Why don't you look that up
- 18:03and tell us tomorrow,
- 18:03which is a total stall tactic or
- 18:06for me to say I don't actually know,
- 18:08I don't know if I'm current with
- 18:09the literature on that topic.
- 18:10Let's let's look that up together
- 18:12and I will tell you the latter is far
- 18:14better role modeling and setting the
- 18:16stage and the tone that that is okay.
- 18:18And so I have had to get very
- 18:20comfortable saying I don't know and yes,
- 18:22we should try to figure this out together.
- 18:24And I love that comparing us ourselves
- 18:26to colleagues that is one of the ways
- 18:27that I know where my gaps are is,
- 18:29is I'm getting sign out from my
- 18:30colleagues or giving sign out and
- 18:32they ask questions and you're like
- 18:34I didn't think about that at all.
- 18:35Thank you for raising that.
- 18:36We should absolutely investigate
- 18:37that for this patient. OK.
- 18:39You guys are great. This is wonderful.
- 18:41All right. You can keep them coming.
- 18:43If you've got other ideas,
- 18:44I'm going to move to the learning gear.
- 18:46So I want us to think there for
- 18:48a minute where this is,
- 18:49I would argue most of my learners
- 18:51and and probably me as well,
- 18:52if I've got a little bit of time,
- 18:54I'm going to skip the planning.
- 18:55I'm going to dive right into the learning,
- 18:56just going to do the learning
- 18:58and that it that works.
- 19:00But I would argue it works far
- 19:01more effectively and probably more
- 19:03efficiently to actually spend that
- 19:04time upfront planning and then to use
- 19:06targeted learning as opposed to I'm
- 19:07just going to sit and read for a little bit.
- 19:10So this is the phase where we
- 19:11actually do engage in the learning.
- 19:13And so with that,
- 19:14I actually want to pause before
- 19:16I go on and and and ask you
- 19:18all what learning strategies do
- 19:20your trainees typically use?
- 19:21And it should probably be no shock at
- 19:23this time at all that the follow up
- 19:25is what typical strategies do you use.
- 19:27But let's start with trainees.
- 19:28What what strategies are your
- 19:30trainees often using when they are
- 19:32trying to learn a new concept or
- 19:33learn to address something that
- 19:34they feel like they don't know?
- 19:46And while you're typing,
- 19:47if you're memorizing, yes, 100%.
- 19:50If your students are anything like mine,
- 19:52I expect to see Anki or question banks.
- 19:54Oh, there's Anki. Yes,
- 19:55I'm glad it made it in there,
- 19:58which can be a good
- 20:00tool. But for the students that that
- 20:01is the only tool, or question banks,
- 20:03which is the for some students,
- 20:04that's the only tool that they use. I'm like,
- 20:07no, it it is a very helpful dipstick.
- 20:10It tells you what you know
- 20:11and what you don't know.
- 20:12That's really good for gap identification.
- 20:14But it is not a great strategy to really
- 20:17understand ideas and understand concepts.
- 20:19And so we try to encourage our
- 20:20students to use multiple strategies.
- 20:22You guys are great producing
- 20:23some of these strategies.
- 20:24And so then I'm going to switch it a
- 20:26little bit to you think about how you
- 20:28learn and what strategies you use.
- 20:30And I love this last comment
- 20:31about they do like podcasts.
- 20:33I think that's true of our learners as well.
- 20:34They hate textbooks.
- 20:35I love textbooks.
- 20:37It makes me so happy to have a textbook.
- 20:38And my learners, you're absolutely right,
- 20:40They absolutely don't like that.
- 20:41They want they want PDFs,
- 20:43they want PowerPoints,
- 20:44they want something that has been digested.
- 20:46And and so I think part of that
- 20:47is learning about our learners
- 20:48and how can we best support them.
- 20:50How do we provide great content,
- 20:52great ideas,
- 20:52but not things that can be easily memorized,
- 20:55easily forgotten,
- 20:56that don't actually lead to one of
- 20:57the things we'll talk about later,
- 20:59IE deep conceptual understanding.
- 21:02All right.
- 21:05So let's talk a little bit about
- 21:06some of these strategies,
- 21:07which I didn't see all of them pop up.
- 21:09I usually do,
- 21:09but I'm not going to ask for a show of hands.
- 21:12But rereading is one that I use highlighting
- 21:15and underlining absolutely also do it.
- 21:16And unfortunately,
- 21:17there is consistent literature
- 21:19that that is a horrible strategy.
- 21:21It does not lead to durable memory.
- 21:23It is very time consuming and
- 21:24it gives us this false sense
- 21:26of familiarity with the topic.
- 21:28So we think we know more than we actually do.
- 21:31Do I still default to rereading
- 21:33and highlighting under lining?
- 21:34I often do if I'm not being very intentional.
- 21:35So I'll just I will out myself there.
- 21:38I will not ask for a show of your hands,
- 21:39but I really this is a great book
- 21:41here in the that in the bottom
- 21:42corner Make it Stick which is
- 21:44written by educational psychologists.
- 21:46They're college professors and
- 21:47it really walks through a lot of
- 21:49the science but describes it in
- 21:51a very accessible way,
- 21:52written for more of a lay public,
- 21:54really great book.
- 21:54I've even talked with other schools that
- 21:56that use this book for all of their students.
- 21:58We haven't gotten that far,
- 21:59but I really we use a lot of it,
- 22:00little pieces of it.
- 22:01And in that book they make the
- 22:03quote that learning is deeper and
- 22:05more durable when it's effortful.
- 22:07So if it's not hard,
- 22:08we're probably not learning,
- 22:09which is a little bit depressing,
- 22:10right?
- 22:11It's like if it's easy it's probably
- 22:14not leading to long term understanding.
- 22:16And I love,
- 22:17I saw Retrieval practice show up in
- 22:18here. Absolutely.
- 22:19Knowledge retrieval strategy.
- 22:20So quizzing yourself can be
- 22:22a really effective strategy.
- 22:24Again, I just don't think
- 22:24it can be the only one.
- 22:25But knowledge retrieval practice
- 22:26is a really well proven strategy
- 22:29with great literature as is
- 22:30space repetitious learning,
- 22:32especially if you combine those two.
- 22:34Concept mapping is another,
- 22:36which is a fairly labour intensive way,
- 22:39but it actually forces the learners
- 22:40to put on paper or on computer
- 22:42how they think about things and
- 22:43how they see concepts organized
- 22:44and how they see them connected.
- 22:46And it's that process of making
- 22:48decisions that is really active
- 22:50and gets them thinking.
- 22:52This is a paper that I was part of and
- 22:53it's one of my favorite papers that
- 22:55I've ever been able to be part of.
- 22:56One, because I loved all the
- 22:57people I got to do it with.
- 22:58But two, it was really fun to wrestle with.
- 23:01These are great theories,
- 23:02these are great concepts,
- 23:04but practically what does that mean?
- 23:06And so these are 4 strategies
- 23:08in the literature,
- 23:09Deep Conceptual Understanding,
- 23:11Meaningful Variation,
- 23:12Productive Struggle and Discovery and
- 23:15Metacognitive Strategies that have been
- 23:17shown to lead to this type of expertise,
- 23:19learning when used well.
- 23:20And what we did in this paper was
- 23:22try to walk through at the UME level,
- 23:24talking about my Vanderbilt program.
- 23:26At the GME level,
- 23:27the four emergency medicine physicians
- 23:29talking about their different programs.
- 23:31And then Alyssa Hollow and Heather
- 23:32Billings at the Mayo Clinic talking
- 23:34about at a CPDCME level how
- 23:36these concepts could be applied.
- 23:38And so it's not exhaustive,
- 23:40but it is really how we tried to
- 23:43apply these concepts locally.
- 23:45To build on this idea that mastery
- 23:47requires both possession of ready knowledge,
- 23:49important, but the conceptual
- 23:51understanding of how to use it.
- 23:54And I'll be honest,
- 23:54this is what we hear from the students
- 23:56when we talk about these things.
- 23:57Like, yes,
- 23:57but that just takes too long.
- 23:59I can get through an AKI deck far faster
- 24:01than I can to sit and do a concept map.
- 24:04Yes, that is absolutely true.
- 24:06So we try to make the argument
- 24:07to them that much like compound
- 24:09interest in the banking world,
- 24:11they're making an initial investment
- 24:12in learning this and laying the
- 24:14foundation for these ideas and these
- 24:16concepts that will over time allow
- 24:17them to learn far more effectively and
- 24:19ultimately more efficiently as well.
- 24:22So I want us to think and in
- 24:24just a minute I'm going to give
- 24:26you opportunity to to brainstorm
- 24:28some specific ideas,
- 24:29but so we're going to walk
- 24:30through each of these.
- 24:30So just be thinking ahead.
- 24:33Deep conceptual understanding.
- 24:34Again,
- 24:34this moves our learners away from
- 24:37rote memorization,
- 24:38membring factoids so that you can
- 24:39get a question right on a test
- 24:41to really trying to understand
- 24:43concepts and ideas.
- 24:44And so there's a lot of different
- 24:45strategies that can be used to do that.
- 24:47We do some things in our curriculum
- 24:48where we try to circle back to
- 24:49the foundational science in their
- 24:51clerkship
- 24:51and in their post clerkship phase.
- 24:52To really try to accomplish that
- 24:55meaningful variation is a is a
- 24:57strategy that helps them see different
- 24:59presentations of the same concept.
- 25:01For example, how might
- 25:03appendicitis present differently?
- 25:04How might we give you multiple
- 25:06versions of somebody with a cough?
- 25:08They each have different things and so
- 25:10that you can start to compare and contrast.
- 25:12And I think that this is really
- 25:13important as we help them think through
- 25:16case based learning is a great way
- 25:18to do this clinically in the ICU,
- 25:19one of my favorite ways to do meaningful
- 25:21variation is we're talking about a given
- 25:23patient and then I change one variable.
- 25:24I say what if instead of being
- 25:27a one year old female,
- 25:29this was a 12 year old male.
- 25:30What would change about this
- 25:32presentation if or what if their
- 25:34renal function was poor because I
- 25:35can play the what if game forever and
- 25:38that's creating variation within the
- 25:39case that we're talking about next
- 25:42productive struggle and discovery.
- 25:44Again, this gets back to that idea
- 25:46that if it's easy,
- 25:47we're probably not learning.
- 25:48And one of the places that that
- 25:50I think this type of productive
- 25:52struggle and discovery is incredibly
- 25:54impactful is the simulation lab.
- 25:56You can create a lot of variation
- 25:57to connect back with the last one,
- 25:58but also allow the learners to struggle
- 26:01a little bit and to not be perfect
- 26:03and to allow no patients to be harmed
- 26:04in the middle of that struggle and that.
- 26:06And so we can think about that.
- 26:09And I just want to just make the
- 26:11connection between this idea of productive
- 26:12struggle and discovery and a huge body
- 26:14of literature from the Bjorks around
- 26:16this idea of desirable difficulties.
- 26:18That these really are not the undesirable
- 26:21difficulties of unnecessary stressors
- 26:23and unnecessarily hard things.
- 26:25But learning that is challenging
- 26:27because it offers opportunity to for
- 26:30encoding and retrieval processes,
- 26:32as already was alluded to with
- 26:33the retrieval practice,
- 26:35because ultimately that supports learning,
- 26:37comprehension and remembering.
- 26:39And so with that,
- 26:41I actually I'm going to drop a link
- 26:43to a Google Sheet in the chat.
- 26:45And for those that are able,
- 26:46I would love for you to pull up the
- 26:48Google Sheet and help me brainstorm
- 26:50a little bit.
- 26:51I'm going to also put it on the screen
- 26:53for those that aren't able to open it up.
- 26:55But I would love for you to just pick one
- 26:57of these 3 deep conceptual understanding,
- 26:59meaningful variation,
- 27:00productive struggle and discovery
- 27:02and just type in the box.
- 27:04Pick one of these boxes and just
- 27:06OfferUp a potential strategy that
- 27:07would allow that to work well with a
- 27:09student or a trainee or even yourself,
- 27:11and give you just a few minutes on that.
- 27:46One of the challenges in the
- 27:47Google Sheet will be if you're
- 27:49trying to overwrite somebody else.
- 27:50So if you have to pick one lower
- 27:51in the list, that's also good.
- 27:52But because look at these coming in.
- 27:54I love it. Yeah, concept mapping is a,
- 27:57in my mind, a great way to focus
- 28:00on deep conceptual understanding
- 28:05high context applications
- 28:06for questions. Yeah,
- 28:10Yep, extending a case can absolutely
- 28:12create some meaningful variation.
- 29:06Give people maybe one more
- 29:08minute and if you have,
- 29:09if as you're thinking about your ideas,
- 29:11you can also move over to
- 29:12the far right column of what
- 29:14barriers might get in the way of
- 29:15using one of these strategies.
- 29:21Again, I'm happy to to correlate these
- 29:23and save them and share them after
- 29:24the fact because you guys are putting
- 29:26up some great ideas of how you might
- 29:28actually do this in your context.
- 29:30And I can tell you what works in
- 29:31my context but I don't always know
- 29:32what will work in other contexts.
- 29:33So I I love that you are able to
- 29:35create this. All
- 29:41right, feel free to keep
- 29:43adding as the thoughts come.
- 29:44I will move to the 4th of the strategies
- 29:46which is metacognitive strategies.
- 29:48And this really is again what are the
- 29:50the different aids or tools we can
- 29:52use to help our our learners think
- 29:53about their thinking or how to help
- 29:56ourselves think about our thinking.
- 29:57And I would argue that the
- 29:59monster adaptive learner model in
- 30:00and of itself is one of those.
- 30:01If they're just feeling stuck,
- 30:03learning, walking through and helping
- 30:05them think about where they might be
- 30:07getting stuck can be really powerful.
- 30:09And I'm going to have a follow
- 30:10up conversation with some of the
- 30:12coaches around again how we might
- 30:13use this type of tool to facilitate
- 30:15coaching conversations because I
- 30:16think it can be very important,
- 30:18but to just to dive a little
- 30:19into the content,
- 30:20the assessing phase really is trying them,
- 30:22trying out what they've just learned.
- 30:25And it is a combination of
- 30:27informed self-assessment.
- 30:27So again,
- 30:28not just regular plain old
- 30:29navel gazing self-assessment,
- 30:30but informed usually by data
- 30:32or conversation or experience
- 30:34and external feedback.
- 30:35And I'll be honest,
- 30:36I don't know who writes
- 30:37the Schoolies cartoons,
- 30:38but I feel like they
- 30:39understand medical education,
- 30:40that I was shooting for a great
- 30:41dog and I only got a good dog.
- 30:42That's the equivalent of read more,
- 30:45which I'm sure nobody puts
- 30:46in your student assessments,
- 30:47but that happens a fair amount in hours.
- 30:50But really being able to partner some data
- 30:53that's coming in and an external perspective.
- 30:56And then lastly,
- 30:57thinking about adjusting,
- 30:58how do we actually change our practice?
- 30:59And again,
- 31:00I would argue, well,
- 31:01learners when they have some time are
- 31:02going to dive in and do the learning.
- 31:04They probably skip the planning.
- 31:05They usually skip the assessing.
- 31:07And almost all of us skip the adjusting
- 31:09of actually changing our practice
- 31:11based on what we've just learned.
- 31:13And so this is where we need to
- 31:14learn how to incorporate into
- 31:16practice what we've just learned.
- 31:17And so at the core,
- 31:19this is really change management.
- 31:20We get,
- 31:21we there's a lot we can learn
- 31:22from the business world who has
- 31:24thought and studied a lot over the
- 31:26years around change management.
- 31:27Part of it is how big of a
- 31:28change is actually required.
- 31:30And there's some great work in
- 31:32the late 80s from Fox and others
- 31:34around physicians in their
- 31:36own practice change around.
- 31:37The bigger the change,
- 31:38the harder it's going to be.
- 31:39So a small change like me remembering
- 31:41I need to use antibiotic Y instead
- 31:43of antibiotic X, that's on me.
- 31:45I just need to do that and we
- 31:46can even provide some decision
- 31:48support to try to prompt that.
- 31:49But if I was a surgeon in the
- 31:5090s that always took gallbladders
- 31:52out through a giant incision
- 31:53and this newfangled laparoscopic
- 31:55technology came along,
- 31:56that's a much bigger change and it
- 31:58impacts my identity as a surgeon.
- 32:00Not only that,
- 32:01I have to
- 32:02worry about training a whole staff,
- 32:04getting new equipment.
- 32:05So it's a much bigger change and
- 32:06so it requires far more deliberate
- 32:08change management. All right.
- 32:11So to pull out from the four gears,
- 32:12the planning, learning,
- 32:13assessing and adjusting,
- 32:14you can see that this all happens
- 32:16in a learning environment.
- 32:17I think that is just
- 32:18really to be cognizant of.
- 32:20There are things in our learning
- 32:22environment we often create
- 32:23and craft in our classroom.
- 32:24I can control things like the
- 32:26lighting and the temperature.
- 32:27Of note, it is always too cold.
- 32:28My students bring Snuggies and they
- 32:30think it's too cold in our building.
- 32:32But when I send them over to the hospital,
- 32:34there's a lot less control we have.
- 32:35And I would imagine this is
- 32:37true for your learners as well,
- 32:38as you think about some of the
- 32:39things that are happening in that
- 32:41clinical learning environment,
- 32:43lack of time, intra team dynamics.
- 32:45Again,
- 32:45I would never intentionally have
- 32:47my students see the internists
- 32:50making fun of the ER docs,
- 32:51making fun of the surgeons, MIT, right?
- 32:53But that happens in our system.
- 32:54We're working on it,
- 32:55We're trying to get better,
- 32:56but that happens and that's
- 32:57not good for learning.
- 32:58I would never want my students
- 32:59to see an obese patient being
- 33:01made fun of for their weight.
- 33:03And unfortunately that has
- 33:04happened in our system.
- 33:06And so we're working on that also.
- 33:07But those are things that are
- 33:08very much in the environment
- 33:09that our students are learning in
- 33:11that will impact how they learn.
- 33:13So we need to just be thoughtful about this.
- 33:16And then I not to get too controversial,
- 33:20but I think this is also one of the
- 33:22places where we set up certain expectations,
- 33:24right.
- 33:24So if if I have a graded clerkship
- 33:27where I'm asking students to compete
- 33:29versus each other for a grade that
- 33:32sends one message and that's one
- 33:34environment versus if I have a pass
- 33:36fail construct that allows them to
- 33:37to meet a certain bar and then get that.
- 33:40Now Full disclosure,
- 33:40I do have a pass fail clerkship
- 33:42year but my post clerkship 2 years
- 33:44are are graded honours high pass.
- 33:45So we have a a hybrid system
- 33:47that has some of both and we try
- 33:49to talk about why that is.
- 33:51But we are sending messages with the
- 33:53way we design policies and grading
- 33:55structures and across the board.
- 33:57And so that also is part of
- 33:59the learning environment.
- 34:00And so I'm going to drop a different
- 34:03Google Sheet in the chat and I would
- 34:05love for you to just think with me
- 34:06for probably two or three minutes.
- 34:08What are some of the factors
- 34:09in the learning environment,
- 34:10some that are positively impactful
- 34:11for this type of learning for
- 34:13both us and our trainees and
- 34:15maybe some that are negative.
- 34:16And so I will pull that back up over
- 34:18here as well so that we can see it.
- 34:22You should see two columns,
- 34:22a positive column and a negative column.
- 34:24And I would again two or three minutes.
- 34:25I would love to just think through
- 34:28with you some of the factors in the
- 34:31learning environment, broadly defined,
- 34:32that are either positive or
- 34:34negative impacts on our trainees.
- 34:36Learning and on our own learning.
- 34:41Oh my goodness, the contagion,
- 34:43effective FOMO, whoever put that in there,
- 34:45thank you so much.
- 34:46It is amazing how once two students
- 34:49start using something like Anki,
- 34:51then everybody's like, oh,
- 34:51I'm going to fail if I don't use
- 34:53it because they're using it.
- 34:54And it's amazing how quickly
- 34:56those circulate around.
- 34:57Yeah, I think Ed overcrowding is
- 35:00absolutely an impactor on learning,
- 35:02usually in the negative sense.
- 35:04I I it's very hard to come up with
- 35:05a positive sense where where that
- 35:07would actually improve learning.
- 35:08More patience maybe.
- 35:09But especially when they're in the hallway,
- 35:10at least they are in our
- 35:12situation when they're,
- 35:12you know they're boarding in the
- 35:13hall and think it's it's not good.
- 35:15It's not productive for learning.
- 35:16We would never design the
- 35:18learning environment that way.
- 35:19Said a different
- 35:23way. Ah, humble teachers.
- 35:25Yes, a really good skilled humble
- 35:27teacher can make such a difference.
- 35:29I love the role modeling.
- 35:30Absolutely. Teachers who are
- 35:33actually motivated to teach.
- 35:34Yes, yes, the USMLE obsession.
- 35:37I'm going to just,
- 35:38I'll retweet that one and
- 35:39and just leave it there.
- 35:40Strong inclusion there.
- 35:45All right. You can keep adding and again,
- 35:47I will make those available
- 35:49to the group afterwards.
- 35:50I want to wrap things up in the
- 35:52last few minutes so that we can
- 35:53take some time for questions.
- 35:55But I I will also just highlight
- 35:57as we're moving to the end,
- 35:58there are batteries that
- 35:59you see there at the bottom,
- 36:01curiosity, motivation, mindset,
- 36:02resilience that we think really
- 36:05power this type of learning.
- 36:07And it's it's come up several
- 36:08times in the chat already that
- 36:10idea of curiosity and mindset.
- 36:11And each of these could be
- 36:13their own hour talk or more
- 36:15great literature bases for each,
- 36:16but they are really important.
- 36:18So in the last couple of slides
- 36:19I want you to think through a
- 36:21couple of cases with me because
- 36:22I hope again that all of this
- 36:24has been helpful in some way.
- 36:25And so I want you to think
- 36:27through these cases with me.
- 36:28The 1st is Susie.
- 36:30She's a clinical student in your program.
- 36:31She is overwhelmed at the end of each
- 36:34day with all of the things she didn't know.
- 36:36She dutifully keeps a list of all of her
- 36:38unanswered questions in her little notebook,
- 36:39but can't ever seem to find time
- 36:42to investigate her answers.
- 36:43So if you were working with her,
- 36:45if you were her coach for example,
- 36:47thinking through where she
- 36:48might be getting stuck,
- 36:48I would argue she is not getting
- 36:50stuck at at gap identification.
- 36:52She's actually quite good at that.
- 36:53She's got a notebook full of them.
- 36:55It's the being able to select one
- 36:56of us where she's getting stuck.
- 36:57And so if you try to help her identify gaps,
- 36:59you're missing, you're missing her.
- 37:00She's done that and it's overwhelming.
- 37:02So if you can help look
- 37:03through her list and say,
- 37:04you know,
- 37:04right now you have a patient with diabetes,
- 37:06why don't we focus on your question
- 37:08here about electrolyte derangements
- 37:09in patients with diabetic ketoacidosis
- 37:11and learn on that And then because
- 37:13you've got a patient with that.
- 37:14And so I will be honest,
- 37:15I was this type of learner as a trainee.
- 37:17I had notebooks and papers,
- 37:18scraps and all kinds of
- 37:20things with questions.
- 37:20And it was just overwhelming to me at the
- 37:23end of the day of how to really learn that.
- 37:25All right.
- 37:26The next one is Tommy.
- 37:27He's a trainee in your program.
- 37:28He really wants to do well.
- 37:30He tries to spend time each
- 37:31night reading and rereading
- 37:33systematically through his textbook.
- 37:34He underlines, he highlights as he goes,
- 37:36but he never seems to remember
- 37:38what he's read.
- 37:39So where might Tommy's struggle be?
- 37:42I think you could make the argument
- 37:45that he hasn't effectively planned.
- 37:46So yes, probably some of that.
- 37:47But more importantly,
- 37:48he's not using effective learning strategies.
- 37:51And so I will also be honest
- 37:53that I was Tommy.
- 37:54I showed up to pediatric residency
- 37:55and was given an Oski's textbook
- 37:57and said read this cover to cover.
- 37:58You will be a great pediatrician,
- 38:00which turns out was horrible advice.
- 38:03I tried, I would fall asleep.
- 38:04I could never remember what I read.
- 38:06I really, I was trying to follow that
- 38:08instruction And for me that was, it was
- 38:10absolutely not an effective way to do it.
- 38:11All right. I want you to think with
- 38:13me on this last training, Sandra.
- 38:15She's an advanced trainee planning to
- 38:17enter a highly competitive subspecialty.
- 38:20I will allow you to determine which of those
- 38:21in your mind you're already thinking about.
- 38:23She's growing frustrated that her
- 38:25clinical assessments rate her
- 38:26lower than a perfect score.
- 38:28She believes that she is consistently
- 38:30demonstrating top performance
- 38:31despite what these assessments show.
- 38:33She makes comments like that.
- 38:35Attending just doesn't like me.
- 38:37I'd like to see that faculty member
- 38:39develop rapport with a rude patient
- 38:41to describe why her scores are lower.
- 38:43So where might Sandra's challenge be?
- 38:48So I would argue it's over here in
- 38:49the assessing. So she has some data.
- 38:51She is just currently choosing
- 38:53not to believe that data.
- 38:54And So what she really needs is an external
- 38:56perspective to help her think through,
- 38:58well, where might the truth be in that data
- 39:00or at times unfortunately to just say no.
- 39:03I have also seen that or
- 39:05the data is consistent.
- 39:06This is how you are being received.
- 39:07And so how, what are the strategies
- 39:10you could use to address that?
- 39:12All right, so we've covered a lot.
- 39:15I just want to quickly summarize and
- 39:17then we'll take some question and answer
- 39:19in just a minute before two things.
- 39:21So we started with what is expertise,
- 39:22we talked about routine expertise,
- 39:24that efficient application of known
- 39:26solutions and adaptive expertise
- 39:27that new learning and innovation
- 39:29to solve those novel challenges.
- 39:31We talked a lot about the how really
- 39:33focusing on the different gears,
- 39:35the planning, learning, assessing,
- 39:36adjusting of the model.
- 39:37We talked about four different
- 39:39potential strategies,
- 39:39focusing on deep conceptual understanding,
- 39:42meaningful variation,
- 39:43productive struggle and discovery,
- 39:46many cognitive strategies.
- 39:47We talked about the batteries just a
- 39:48little bit and then ultimately the
- 39:49learning environment and the coaching.
- 39:51And so I want you to pull back out your
- 39:53note card or your scrap of paper and
- 39:55I want you to think what was one thing,
- 39:57one ID or strategy that you would like to
- 40:00learn more about in the next two weeks.
- 40:02And I'm going to ask you to physically
- 40:03write it down or in just a minute
- 40:05they're going to drop a link in
- 40:07the chat for this evaluation of
- 40:08our our time together today.
- 40:10And you could also include
- 40:11it in the comments there.
- 40:13In thinking about what are you
- 40:14going to commit to do in the next
- 40:16two weeks to learn more about?
- 40:17And so I would encourage you either on
- 40:19your paper or in the survey or both,
- 40:21to think through what is the most important,
- 40:23what's the most meaningful thing you
- 40:25heard in the last hour and what's that
- 40:27one idea that you would like to learn
- 40:29more about in the next two weeks?
- 40:30I'm going to pause.
- 40:31We're going to have a little
- 40:32bit of zoom silence.
- 40:33Don't worry, don't get nervous.
- 40:36I will circle back in about 60 seconds.
- 41:13right, Feel free to keep writing
- 41:15or keep filling out your survey.
- 41:16With that, I'm about to pause for questions.
- 41:19I will leave you with the two pictures
- 41:21that I have on my wall in my office.
- 41:23The first is remember why you started
- 41:25and this to me is just that reminder
- 41:28of my original WHY was really to be an
- 41:30outstanding clinician to provide great,
- 41:32great care to my patients and their
- 41:34families and I hope I still do that.
- 41:37I still spend part of my time doing that.
- 41:38But my WHY has very much more become
- 41:40about my trainees now and helping
- 41:42them learn and develop and even more
- 41:44recently about my team and how do I
- 41:46support them to grow and develop.
- 41:48And then the the other is also just
- 41:50a challenge I hope is is true of
- 41:51all of us to always be learning.
- 41:53And with that,
- 41:54I'm I'm going to stop sharing
- 41:55and hopeful that we can have a
- 41:57little bit of conversation.
- 41:58I'm happy to try to answer some questions.
- 41:59Thank you for your time and attention.
- 42:01I really appreciate it.
- 42:02Oh, Bill, this was so wonderful.
- 42:05Thank you. We really appreciate it.
- 42:08So we have our Deputy Dean for Education,
- 42:11Jessica, Lucy put an educational
- 42:13strategic plan together.
- 42:14She's actually on the call today.
- 42:16And one of the very important aspects
- 42:18as we move forward is how do we really
- 42:20develop learners who are lifelong learners.
- 42:23And we do want to really
- 42:24look at the pedagogy.
- 42:25And we have hired a director of pedagogy.
- 42:28Bill Rando is with us,
- 42:30which we're very excited about that
- 42:32and really be thinking through.
- 42:34I know that Dana's heading the coaches
- 42:36program and you're familiar with that.
- 42:38But I would love to hear some of
- 42:40your advice in some of the aspects
- 42:42of what changes have you seen
- 42:44at Vanderbilt or other schools.
- 42:46When you look at whether it's
- 42:48Larry Gruppen or Martin,
- 42:50to really think about what kind
- 42:52of pedagogy could you we implement
- 42:54to make some of these changes.
- 42:57Yeah. Oh, I love it.
- 42:59I think there's a lot that can be done.
- 43:00And I will tell tell you
- 43:01some of what we've done.
- 43:02Not saying that it is
- 43:03perfect or the only way.
- 43:04I think we have tried to be very
- 43:06intentional in the way we talk about these
- 43:08ideas and these skills with our students.
- 43:10So instead of just hoping,
- 43:11which we used to do that they will just
- 43:13pick up this type of learning expertise,
- 43:15we're far more explicit now about
- 43:16talking about it and giving them words,
- 43:18giving them connections to go learn more,
- 43:20to give them opportunities to practice it.
- 43:22We've also learned a lot around
- 43:25better targeting and tailoring when
- 43:26we talk about different topics.
- 43:28So we don't actually start
- 43:30with self-assessment.
- 43:30We wait until our clerkship year,
- 43:32which is year two to do a
- 43:33little bit more about that.
- 43:34We start with learning strategies
- 43:36with the idea that they come in and
- 43:38they are starting to recognize that
- 43:39what got them to medical school might
- 43:41not be working as well as they liked.
- 43:43And so we use that as an opportunity
- 43:45to talk through some of the
- 43:47learning strategies ideas.
- 43:48I think that some of the other
- 43:49structural things we have done,
- 43:51we use a lot of case based learning in
- 43:53our first year of small group learning
- 43:55CBLT and and that has us has for us
- 43:57been really impactful and allowing
- 43:59them to get in there and wrestle.
- 44:01So some of that productive struggle and
- 44:03discovery seeing some variation within
- 44:05cases learning with and from each other.
- 44:07Pedagogically for us that
- 44:09has been really impactful.
- 44:12We did make AI alluded to this earlier,
- 44:14a change to our core clerkships
- 44:15when we moved them from year
- 44:17three to year 2 back in 2013.
- 44:18We did make it pass fail.
- 44:20I think that changes.
- 44:22We wanted it to truly be a foundational
- 44:24experience in the clerkships and it
- 44:27changes the ability to admit what you
- 44:29don't know when you're not trying
- 44:31to compete with the person next to
- 44:33you or trying to look perfect now.
- 44:34So Full disclosure,
- 44:35I also said we have graded immersion phase,
- 44:38so years 3:00 and 4:00 but even with that
- 44:42we we changed the way we did grades,
- 44:45we set here is an honours performance,
- 44:48here's the criteria for that
- 44:49and any student that met those
- 44:50criteria could get an honour.
- 44:51So they're no longer competing
- 44:52with each other which they were
- 44:54in our old system that they are,
- 44:55they know what the standard is and then
- 44:57they're competing with that or they're
- 44:58they're trying to perform to that.
- 45:00And so structurally for us
- 45:01that felt like it was a a,
- 45:03an approach that we that was more
- 45:05consistent with who we wanted
- 45:06to be in a CBME frame.
- 45:07We've we've determined these outcomes and
- 45:10now we're helping you work towards them.
- 45:12So those are some of the things
- 45:13that come to mind right away.
- 45:15I could happy to continue to talk,
- 45:16but I see two other hands.
- 45:17So let me try to answer those and
- 45:19we can circle back as as needed to
- 45:21some of the other examples here.
- 45:22Perfect. Thanks Vijay.
- 45:27Hi, thank you so much.
- 45:29This was so informative and engaging talk.
- 45:33I'm Vijay Karp.
- 45:34I'm an anesthesiologist and the Vice
- 45:37chair for Education in my department.
- 45:38And one of the things I've noticed in my
- 45:41interaction with the trainees is that a
- 45:44lot of them are using ChatGPT to get,
- 45:48you know, most of their information.
- 45:51So are you using any sort of,
- 45:55you know, training, incorporating,
- 45:57training on using these large language
- 45:59models for our trainees so that they
- 46:03can use them responsibly and ethically?
- 46:06And the second question is when do
- 46:09you see AI in terms of you know,
- 46:13helping in our journey towards being these,
- 46:16you know, towards this master
- 46:18adaptive learning.
- 46:21Thank you for the question.
- 46:22I really, I love it. I will.
- 46:25I'll start with the easy answer.
- 46:26No, we are not currently doing that.
- 46:27But we very much like your question have
- 46:30recognized students are using this.
- 46:32There is absolutely a set of competencies
- 46:34needed to use tools like that effectively.
- 46:37And so I just have on January 1st was
- 46:40able to add a person to my team with
- 46:42some FTE support to think about how do
- 46:44we incorporate this into our curriculum.
- 46:46So not just how do you use the tool,
- 46:48but what are the competencies to
- 46:51think about asking really effective
- 46:52questions because it turns out there's
- 46:54really a skill set around what
- 46:55prompts you enter into the the tools,
- 46:57whichever of them they are.
- 46:58And so we will,
- 47:00we are going to start incorporating
- 47:01that this fall in our first year
- 47:03curriculum around what are the core
- 47:06competencies related to informatics and
- 47:07AI that all of our graduates will need.
- 47:10And so I'm really excited.
- 47:11We're partnering with our Department
- 47:13of Biomedical Informatics to do that.
- 47:15And there's been a lot of concern
- 47:18about the ethical pieces that you've
- 47:20mentioned that you alluded to as well.
- 47:22And so we are working on having a Vanderbilt,
- 47:26a Vanderbilt instance of the the
- 47:27chat tools so that we can put in
- 47:29data to start to look at it.
- 47:30And so that the students are not
- 47:33necessarily putting in either
- 47:35copyrighted materials or faculty's
- 47:36materials or patient materials for sure
- 47:39into something that would would cut.
- 47:41So we're wrestling with that.
- 47:43And then to the last part,
- 47:44I think this is one of the areas
- 47:47where I'm learning so much from
- 47:48my colleagues that do this well.
- 47:50And the way that they have helped
- 47:52me think about it and frame it is
- 47:53these tools are that they are,
- 47:55they're just they're tools and if we
- 47:57learn to use them really effectively,
- 47:59there's a lot of the consolidation,
- 48:02scanning for different articles,
- 48:03organizing it can be done,
- 48:06but we're really early in that.
- 48:07So there's a lot of that, you know,
- 48:08you've heard about the,
- 48:09there's a fancy name for it,
- 48:10but basically the fake citations that
- 48:12they that they will come up with.
- 48:14And so there's a lot that we need
- 48:15to learn about. Hallucinations.
- 48:17Yes. Hallucinations, Yes.
- 48:17Like there's a word for it and I
- 48:18couldn't come up with it. Thank you.
- 48:20So I think there's so much that
- 48:22and that that.
- 48:23Right.
- 48:23Even if also if we're really honest,
- 48:24our learners will be far
- 48:25better at this than we are.
- 48:27And so we need to figure out how
- 48:29to rebuild the the skill set and
- 48:31the tools to keep up with them or
- 48:33at least to be able to have the
- 48:35conversation to then help prepare them
- 48:37for what could be a very different
- 48:38potentially practice of medicine
- 48:40or lifelong learning set of habits.
- 48:42At the core,
- 48:43I still think there is a lot of ideas,
- 48:45content, skills,
- 48:46attitudes that have to be learned,
- 48:49some of which large language models could
- 48:51help with, many of which they won't.
- 48:53And so then really trying to wrestle
- 48:55with great, where can these tools help
- 48:57so that we can put focus elsewhere
- 48:59for some of those other things.
- 49:00But thank you for the question.
- 49:02I love it. Great, Michael.
- 49:04Thanks again.
- 49:06Bill, I'm very interested in test
- 49:09enhanced learning which as you know
- 49:11is a form of retrieval practice,
- 49:13but the the potency of it really lives
- 49:17and dies by the quality of the question.
- 49:20And you know you have to remove the
- 49:22irrelevant difficulty and make it
- 49:24kind of a high context question that
- 49:27requires application of knowledge.
- 49:30And the thing that that really
- 49:32convinced me of this,
- 49:34which is this study that blew me away.
- 49:37I want if you don't know what
- 49:39I just wanted to do.
- 49:40They actually put students in F
- 49:44MRI machines and had them answer
- 49:48recall questions and then the
- 49:51other half was in the Mr.
- 49:53FM RI machine answering high context
- 49:56application questions and the FM RIS
- 49:59are completely different anatomically
- 50:03the ones with the high context
- 50:05question the parts of the cortex
- 50:07associated with deeper learning
- 50:09were lighting up But that just that
- 50:12you can't get better than that.
- 50:14No I oh thank you I have not seen that paper.
- 50:16I I remember vividly having a
- 50:18conversation around 2011 or 2012
- 50:20with some faculty here and saying
- 50:22wouldn't it be amazing if do
- 50:24exactly what you just described
- 50:26and we didn't but I wish we hadn't.
- 50:27I'll send it to you.
- 50:29Please do.
- 50:29I think it's by Dang DENGI think
- 50:33he's the first author I I was.
- 50:35I would also be very interested in
- 50:37getting that paper because it it
- 50:39relates very closely to work I'm doing.
- 50:41Thank you for bringing that up Michael.
- 50:43Sure.
- 50:43We'll send it to everybody. Just Michael
- 50:46just send it to Dorothy. Perfect.
- 50:49OK, I'll do that. Thanks Dana. Can I make.
- 50:53I just want to make one other comment
- 50:54on Michael's I I love the idea of of
- 50:56test enhanced learning and I will
- 50:58say we have struggled at figuring out
- 50:59how to implement that practically.
- 51:00Well, we have tried different things,
- 51:02one of which that I really thought
- 51:03was going to be awesome was in the
- 51:05middle of a a simulated patient
- 51:07case or standardized patient case.
- 51:08We gave the student the opportunity
- 51:10to come out to enter what they,
- 51:11what their differential diagnosis was to do,
- 51:14to have free access to the Internet to
- 51:17use resources and then to kind of hone
- 51:19their their differential and to go back in.
- 51:21And to your point,
- 51:22I think it was still helpful and it did give
- 51:25us an ability to assess how they were doing.
- 51:26But our assessment metrics just weren't
- 51:28weren't sensitive enough to really
- 51:30capture what they were doing and how
- 51:32well they were using different tools.
- 51:34And were they just going to Google
- 51:35or were they trying to use primary
- 51:36literature or were they using?
- 51:37So I think there's a lot of work that
- 51:39could be done to really better assess some
- 51:42of those skills in a test enhanced way.
- 51:44Yeah, absolutely.
- 51:44Michael has a fabulous systematic
- 51:46review and beamy that he put
- 51:48together on test Enhanced learning.
- 51:50I actually think you need real
- 51:52estate in the curriculum for it.
- 51:53So that's our big plug as we
- 51:56move forward. Janet always
- 51:57wants more real estate.
- 52:02Can I go ahead?
- 52:04Oh, thank you very much, Bill.
- 52:05And as you know, we,
- 52:06I was so taken by this model for our
- 52:09new coaching program that we are all
- 52:11in on the Master Adaptive Learner.
- 52:14And the thing I really liked about the book,
- 52:15not that the AMA is giving me
- 52:17any credit for plugging the
- 52:19Master Adaptive Learner book.
- 52:20But there's a chapter in that
- 52:21book for all the people on
- 52:23the call right now.
- 52:24People who are lecturing people who
- 52:26are teaching in the clinical setting.
- 52:27People are teaching in the GME setting.
- 52:30Because as individual
- 52:30faculty kind of the more
- 52:32we have a institutional framework
- 52:35around the things that we can do to plan
- 52:37our teaching because we can be so
- 52:39directive read that chapter do
- 52:41this thing and and we can't find
- 52:43the little coals that are already
- 52:45burning inside them. And our
- 52:47coaches are working on those batteries and
- 52:49charging them as matter type of learning.
- 52:51But they're so used to being controlled,
- 52:54what to do and why to do everything else.
- 52:56System I think helps more than potentially
- 52:58some other places that might be more
- 53:01restrictive or pro strict post post frictive.
- 53:05You know what I mean proscriptive.
- 53:08But I don't know if you could talk a
- 53:10little bit more about individual teaching
- 53:13on the fly, teaching in a clinical
- 53:14scenario that I don't want to sound
- 53:15like it's too big, but an example
- 53:18of you as an individual teacher being
- 53:21an Mal attending. Yeah. And and Bill,
- 53:24just before you make a plug, please,
- 53:27everyone start filling in your eval.
- 53:29We really appreciate all your comments
- 53:31while Bill answers Dana's question.
- 53:36Yeah, Dana, I, I appreciate the question.
- 53:38And I think there are,
- 53:41there's so many layers to what
- 53:43happened in a clinical environment.
- 53:44So my my time in the PEDs ICU is often
- 53:48the morning is focused on rounds and
- 53:49going patient by patient and all of
- 53:51that into that environment is one thing.
- 53:52The if we pull learners away to
- 53:54talk a little bit about a few
- 53:55things or it's embedded within.
- 53:56And so I think the strategies vary a little
- 53:59bit based on the context even within the
- 54:01context of on time in the in the ICU.
- 54:04One of the my learners would laugh at
- 54:05this because it's absolutely what I do.
- 54:07I almost never answer a question
- 54:09with with what they want to hear.
- 54:11They asked me what should we do here.
- 54:12I'm like well let's talk about what
- 54:13do you think we should do here.
- 54:14And so my main strategy one,
- 54:16because I think it engages their
- 54:17curiosity a little bit better,
- 54:19but two,
- 54:19it actually helps me assess
- 54:20their reasoning skills because I
- 54:22even if they're totally wrong,
- 54:23I want them to think through and
- 54:25ideally think through out loud what
- 54:27they think we should do because
- 54:28it allows me to really gauge their
- 54:30understanding of the cut context
- 54:31of the situation with the patient
- 54:33of the Physiology below that.
- 54:35And so I do a lot of question answering,
- 54:36no question asking and.
- 54:40And for me that is the strategy that
- 54:42helps me better tailor if I am going
- 54:44to drop in then little minutes of
- 54:46teaching to target it better so that
- 54:48I'm not teaching them teaching them
- 54:50something that they learned already
- 54:52or they already know something that I
- 54:53did a while ago and I haven't used it
- 54:55recently because it is fairly time intensive.
- 54:57But if there's a little bit more time,
- 54:59is asking them to do a concept
- 55:01map about a given patient,
- 55:04'cause it's really informative
- 55:05to quickly see, well,
- 55:06wow,
- 55:06you've got a great understanding
- 55:08about the microbes that might
- 55:09be causing this infection and
- 55:11the antibiotics and all of that.
- 55:12But you really,
- 55:13this part over here where you're
- 55:15talking about the symptoms,
- 55:16those aren't connected at all.
- 55:17So let's let's spend my time teaching there.
- 55:20Again, it requires a little bit more time,
- 55:21'cause you're asking the learner
- 55:22or learners to put on paper how
- 55:24they're thinking about something.
- 55:25But it's a really quick way to
- 55:27assess where are they strong,
- 55:28where might we spend some teaching
- 55:30time so that I'm not in my mind
- 55:32wasting their time.
- 55:33I think the other,
- 55:34and I alluded to this earlier,
- 55:35that's as important or maybe more
- 55:37important is the role modeling when
- 55:39I don't know something and to be
- 55:41willing to just upfront admit that
- 55:42they hear me tell families often,
- 55:44I don't know for sure what's
- 55:45going on with your child.
- 55:46Here's what we're doing to evaluate it.
- 55:48Here's what we think it could be,
- 55:49but I we don't know for sure.
- 55:52And so I hope that that builds in a little
- 55:54bit more comfort with uncertainty which
- 55:56I think again is embedded in medicine.
- 55:58But for learners that's hard, right.
- 56:00If you think about especially
- 56:02our pre clerkship learners,
- 56:03they want black and white,
- 56:05they want the ABCD.
- 56:06There should be always a right choice and
- 56:08it should be very clear which one it is.
- 56:10And most of the patients we take
- 56:12care of aren't that black and white
- 56:14and they so there's uncertainty and
- 56:16and so I also think if when there
- 56:17are times where I can highlight
- 56:19some of that uncertainty,
- 56:20I think it serves them well to
- 56:22have some conversation around it.
- 56:23That's especially difficult for
- 56:26oncologists and and you're you're
- 56:30acknowledging that who don't
- 56:32like to admit the uncertainty of
- 56:34treatments and it often leads to
- 56:36very devastating side effects.
- 56:38So I kudos
- 56:39for modeling that.
- 56:41Wonderful. So Bill, thank you so
- 56:43much for getting us excited about
- 56:45this and I'm sure we'll all walk away
- 56:48reading your papers in your book.
- 56:51So thank, thank you very much and
- 56:53thanks for everyone to participate.
- 56:54It was a great session.
- 56:56Thanks everybody. Really enjoy.