4-25 MEDG: Student Engagement in Attaining Competency

April 25, 2024

ID11607

To CiteDCA Citation Guide

00:00And we'll have the recording
00:02available on our website tomorrow.
00:04So I'm thrilled.
00:05Welcome to our Med Ed discussion group.
00:08And I am thrilled we know Jessica very well,
00:11but I'll give you just a brief introduction.
00:13She is our Deputy Dean for Education
00:16and the Harold W Jockers Professor
00:18of Medical Education and Professor
00:20of OBGYN and Reproductive Sciences.
00:23She graduated Harvard Medical School
00:25and then fortunately came to Yale,
00:28starting her residency in 98.
00:30She then went on to receive a master's
00:33degree in epidemiology here at Yale in O 6,
00:37and she was always
00:39passionate about education.
00:40She was a clerkship director
00:42in OBGYN until 2013,
00:43and then she was the director of
00:46our Electives and Sub internships
00:47for the School of Medicine,
00:49and she knew she wanted to have education
00:52as an important part of her life.
00:55So we're so fortunate to have
00:56her as the Deputy Dean.
00:58She's received a number of teaching awards.
01:00In 2006,
01:01she received the annual teaching award
01:04from professors of Gynecology and OB,
01:07and then she also received our Leah
01:10Loenstein Award for the promotion of
01:12humane and egalitarian medical education.
01:15And she certainly brings that to her role
01:17in what she's doing as she leads us.
01:20She also received the Bonfock Teaching Award,
01:23and she recently received the Helmand
01:25Award from the American College of
01:28Nurse Midwives in recognition of her
01:30work and promoting collaboration
01:32between physicians and midwives.
01:34And I think that's an important part
01:36when you think of our interprofessional
01:38aspects that what you bring to your role,
01:40Jessica.
01:41And when I think back of what you've done
01:44since 2020 in your role as Deputy Dean,
01:47the strategic plan,
01:48the implementation.
01:50And now we we have Bill Rando who is a
01:52new recruit because of your strategic
01:54plan and both Jessica and Bill on Med
01:57Ed Day are going to be doing a workshop.
01:59So please mark your calendars
02:01and sign up for it.
02:02That's June 6th.
02:04So,
02:04Jessica,
02:05thank you so much for coming
02:07to give this talk today.
02:10Thanks so much, Janet.
02:12I'm so happy to be here and to
02:15see all of you and engage in a
02:17very important conversation today.
02:20And that's regarding student
02:22engagement in our curriculum
02:24and in attaining competency.
02:26So I'll bring my slides up.
02:39Hey, is that good, Dorothy
02:43looks good. Yeah.
02:44OK, great. Great. OK.
02:47So, so today I'm focusing on student
02:51engagement and attaining competency.
02:53It is one of the core domains of our
02:58medical education strategic plan,
03:00which as Janet mentioned,
03:02we developed in the first couple of
03:05years that I was here and we're very
03:07excited to continue work in this domain.
03:10So today, we're going to be thinking
03:14about how we think about teaching
03:17and learning in 2024 and what is the
03:20impetus to potentially change how
03:22we're doing this and approaching it.
03:25Consider some of the innovations
03:27that we are currently using to
03:30improve engagement in our curriculum.
03:32And then also relate that to the
03:36milestones that we have developed
03:38related to the competencies that we
03:42hope all students reach by the time
03:45they graduate in preparation for
03:47the next phase of their training.
03:50And then lastly,
03:51to explore the role of the longitudinal
03:54coach in helping our students track
03:56their progress in achieving these milestones.
03:59So before I go further,
04:01we wanted to do a brief poll to
04:03learn about who's here today from
04:06the different schools and and
04:08then follow that with a poll about
04:10your roles within those schools.
04:12So if you can just fill that out quickly,
04:14that would be very helpful to us.
04:39I think we've reached saturation, Jessica.
04:41I'll close the poll and
04:43share the results. OK.
04:49OK, great. And can you also send the other
04:53poll about everyone's roles? Definitely.
05:06So as I'm speaking today,
05:08you know I will be focusing a lot
05:13on the MD program for which we have
05:15developed competencies and milestones.
05:18But in speaking with the
05:19leadership of the PA program,
05:21they have noted that the competencies
05:23that we've developed for the MD
05:25program are very well aligned with
05:27the competencies for the PA program.
05:29And therefore, you know,
05:31kind of the creative approaches we're
05:33taking to help students engage in the
05:36curriculum and reach these competencies,
05:38I hope would be relevant to both
05:41of our programs and probably even
05:44translates over to the School of
05:47Nursing and some of the other
05:49health professions as well.
05:51I'm
05:51going to close the poll,
05:52Jessica, and share the results.
05:54OK. Thank you. Sure. All right.
05:56Excellent. So I'm so glad to see
05:58that we have people from different
06:00backgrounds and and professions here.
06:12OK. So as Janet mentioned,
06:14we did work together across the
06:17School of Medicine with multiple
06:19stakeholders to develop our
06:21strategic plan for medical education,
06:24which was published and disseminated in 2022.
06:28And it really is guiding a lot of our work.
06:31It's helping us to prioritize our
06:33resources and efforts in helping the
06:36school move forward over the next decade.
06:39So as you can imagine,
06:41student engagement and attaining competency
06:45rose to the top as one of the highest
06:48priorities for us to address as a school.
06:50And so when you think about
06:52why this rose to this top,
06:54some of you have experienced this first hand,
06:58right?
06:58So I'm talking about the impetus for changing
07:01how we approach teaching and learning.
07:04And so the slide, you know,
07:07is a little distressing to look at,
07:09but it's not very different from how many
07:13of us feel when we come to a setting to
07:17give a lecture to our medical students.
07:21I know that it's not necessarily the same in
07:24some PA program or in other other programs,
07:27but this is not uncommon.
07:29We've been hearing that
07:31attendance in our lecture,
07:32our lectures can be as low as 10
07:35students out of the 104 in our class.
07:37And so I think it's really important that
07:40we really reflect on why that's happening.
07:43Is it something to do with how
07:46students of of this era are engaging
07:49with information and how they learn?
07:52Is it have something to do
07:54with our lecture styles?
07:56And so I really challenge us to think
07:59about is are the methods that we're using
08:02in these types of setting evidence based.
08:05Because we know that just showing
08:07up to give a lecture without
08:10really engaging in some sort of
08:12faculty development and you know,
08:14kind of learning what the evidence
08:16based approaches might be can be
08:18less effective than we would hope.
08:21We also have heard from students
08:23that there are many other resources
08:25that are accessible to them.
08:27There are online question banks,
08:29there are online modules that
08:31have been created,
08:33you know,
08:34thinking about evidence based pedagogy
08:37and really incorporating and personalizing
08:41platforms in which students may engage.
08:44And so in in many circumstances
08:47students are finding alternative
08:49resources better than the the
08:52pedagogies and the curriculum that
08:54we're presenting to them on a daily basis.
08:58So it's something we really need
09:00to think about.
09:01The other thing that we can all
09:04acknowledge is that a traditional lecture,
09:06you know,
09:07as we remember from maybe our
09:10own days in education is not
09:13necessarily engaging interactive.
09:16It may not stimulate critical thinking,
09:18clinical reasoning,
09:20collaboration,
09:21professional identity formation,
09:23all things that we know are important
09:26in the foundational informative
09:29experiences in educating our students.
09:35What we do know from the evidence is
09:39that small group pedagogies are effective
09:42and supported by evidence and there's
09:44a number of different approaches.
09:46And so I'm not here today to go over those.
09:48That is, that is some work
09:50that we have for the future.
09:52But there's a lot of evidence showing
09:54that these types of these types of
09:57approaches to learning more effectively
09:59engage students in critical thinking.
10:02We know that this requires attendance, right?
10:05And that is a challenge that we are
10:08facing participation and contribution
10:10by all team members to be effective.
10:14We also know that in order to have
10:16a a small group that functions well,
10:19we need to have faculty development for
10:22the faculty, but also for the students.
10:25The students need to have some type
10:28of preparation to introduce them
10:30to the pedagogy that we're going to
10:32be using in the classroom in order
10:34for it to actually work for them.
10:37A small group is not going to be effective
10:40if we simply divide up the class,
10:42find 8 faculty members and then just
10:45put them in the room, right, and say,
10:48OK, now cover this material, right.
10:50That's not going to be an effective use
10:54of the faculty's time or the students time.
11:00And we have some evidence of that
11:02already when you look at the course
11:05evaluations that students give us.
11:06And also in the advisory groups
11:09that are held after every course,
11:11as well as in the recent sessions that I
11:13and the other associate Deans have been
11:16holding with small groups of students.
11:18So we've been holding lunches in small
11:21groups with every class and we're
11:23currently going through the first
11:25year class and meeting with them.
11:27They have been in in our curriculum
11:30since last August and kind of are
11:33coming to the end of their first year.
11:36And I've captured some of the
11:39comments that we've been receiving
11:42in these small group settings,
11:44you know just to to look at a few of them.
11:47Some of the small groups are amazing,
11:49but many have been dysfunctional.
11:51Sometimes the faculty have no idea what
11:54we have already covered and have no idea
11:57how to facilitate interactive learning.
11:59Early on, I realized I could learn
12:01the material better on my own,
12:02so I just stopped going.
12:05It was 5050 on the small group,
12:07so why take a chance?
12:09At least on my own,
12:09I can make use of the hour and a half that.
12:12That's a theme that I've heard many times.
12:14When we ask students,
12:16why are you not attending your small groups?
12:19What made you stop going,
12:21right?
12:21And so I'm hearing that you know they
12:24it's a 5050 chance whether or not
12:25it's going to be a positive learning
12:27experience and a good use of their time.
12:30So some of them are choosing not to attend.
12:34It's not fair when some students
12:35have a great small group and others
12:37get stuck in an ineffective one
12:38and we're not allowed to switch.
12:40And then lastly,
12:42content covered in the small groups yields
12:45only a couple questions on the qualifier.
12:48So it doesn't seem like
12:50a good use of my time.
12:51We can just read the summaries afterwards.
12:54So that's also that last comment is
12:57reflecting on what value students
12:59are placing on these groups.
13:01I don't think they're necessarily thinking.
13:04Many of them are not necessarily thinking
13:06of the small groups as an opportunity
13:08to learn the skills of critical thinking,
13:11clinical reasoning,
13:13collaboration,
13:13professional identity formation.
13:16Like they're they're thinking of
13:18the small groups as a place to
13:20go maybe extract information and
13:22they're they have determined that
13:25there are other higher higher yield
13:27ways to get that information.
13:29So what can we do about this And
13:31and you know I just pausing just
13:34one more moment on this slide.
13:36I think that over the past you know
13:38couple years you know we were we were
13:41kind of told by the Dean that we
13:43needed to adhere to the Yale system
13:45and that we could not make curriculum
13:47mandatory especially even our small
13:48groups which you know I think was
13:51frustrating for many faculty members.
13:53But when you see these comments,
13:55you start to understand why
13:57students are objecting to these
13:59these sessions being mandatory.
14:01Because if we're not delivering A consistent,
14:04effective product of an opportunity
14:07for interaction with faculty
14:10that is enhancing their learning,
14:13it's kind of hard for us
14:14to say you must be there.
14:16And I'm not saying that all of our small
14:18groups are fraught with these difficulties.
14:21We have many, many examples of effective
14:25small groups and I think what we're
14:28recognizing is we have to capture
14:29what's going on in those groups,
14:31determine what's effective and try
14:33to spread that amongst the other
14:36courses and across all faculty.
14:40So, well, I've mentioned several times
14:43the competencies that we are seeking
14:46for our students to achieve throughout
14:48their time as a medical student.
14:50And so as you can see,
14:52we have 9 competencies and I would
14:56point out that mechanisms and treatment
14:58of disease is just one of them, right.
15:00And that is probably the one
15:03that is most knowledge based.
15:05And our qualifiers are designed to
15:09probably test that one competency.
15:12But we are expecting students
15:15to be developing all of the
15:17other competencies you see here,
15:20even in their first year prior to
15:22entering the clinical curriculum.
15:26We also need to be able to then support
15:29students to monitor their progress in the
15:32curriculum through formative assessment.
15:34And so like we that is something that
15:38we're recognizing is critical to helping
15:41students further engage in the curriculum.
15:46So I mentioned that we have some
15:48things that are working and we are
15:50so proud of those things and I
15:52think we've all recognized that we
15:54need to do more of these things.
15:57So we introduced simulation into the pre
16:02clinical curriculum a number of years ago.
16:05It was normally used more
16:07in the clerkship period.
16:09And what we have found is that
16:12simulation can be a very effective
16:15tool to not just teach clinical skills
16:18and procedural approaches to care.
16:20It is also very effective in engaging
16:23students in thinking about Physiology,
16:26learning pharmacology.
16:27And so you know this is something that
16:32is not mandatory and yet we have very,
16:35very high participation levels in
16:37these sessions when we hold them.
16:40So it's a signal to us that this
16:42is an approach that students are
16:45finding effective for their learning.
16:48The other,
16:49the other things that have been found
16:51to be very effective are frequent
16:53engagement with real patients,
16:54bringing real patients into the classroom
16:57and with our simulated participants.
16:59You know in in learning clinical
17:02skills we've I think there's been a
17:04great expansion of the use of the
17:06simulated participants not just in
17:08communication skills but getting into
17:10deeper levels of of things that we hope
17:13our students will learn how to do.
17:15You know,
17:17talking about about death and dying and
17:21giving bad news and engaging in consent,
17:24There's so many things that can be simulated.
17:27Patients can help us in guiding our students.
17:32Next, the effective pedagogies
17:34that we're seeing in small groups,
17:37Bob Kim,
17:38I'm going to call him out because this,
17:41his approach to teaching in the
17:44pathology lab has been lauded by the
17:47medical students year after year.
17:49And I give a lot of credit to Rob
17:51Homer and his team to like help spread
17:54that pedagogy because they've taken
17:56all of the leaders of the pathology
17:58labs now and kind of exposed them and
18:01had them receive some development from
18:03Bob and really tried to adopt that pedagogy.
18:07And that's an example of how when we
18:09determine that something is effective
18:12and useful to students and increasing
18:15participation and engagement,
18:16we need to provide the other faculty
18:19with the opportunity to adapt
18:21and adopt these this,
18:23these skills in the classroom.
18:26We also have a large number of case
18:29based learning sessions that are getting
18:31very high reviews across the courses,
18:34small group instruction in
18:36the clinical skills program.
18:38And then you know moving on to technology,
18:41when we bring point of care
18:43ultrasound into teaching of anatomy,
18:45Physiology and physical exam,
18:47students find that extremely helpful.
18:50It enhances their learning.
18:52It's not something that they can do
18:54sitting at home in front of a screen.
18:57It's it's something that's engaging
18:59them and pushing them to for
19:02to greater understanding.
19:04We've been integrating virtual
19:06reality and biomedical imaging
19:08into anatomy and in other forums.
19:11And then lastly,
19:12we know that students like to
19:16interact with question banks
19:18and assessment for learning,
19:20which I know Michael Green is a
19:22big champion of.
19:23It's it's something that we need to
19:26integrate more across all of our courses.
19:28So having opportunities for for students
19:31to engage in thinking about questions,
19:34going over kind of the thought processes
19:38that help them in hoping they're thinking
19:41and consolidating their knowledge.
19:47So this is the famous slide of Bill Rando
19:49that many people have been showing and
19:52there's slide decks now really highlighting
19:55the hope and gratitude that we have
19:59in having Bill Rando join our team.
20:02So Bill, you know I'll let him
20:04speak for himself as well,
20:06but he comes here with.
20:09Expertise in pedagogy and also I
20:11would say a really fantastic approach
20:14in engaging all of us in thinking
20:17about what we're currently doing,
20:19considering what's working.
20:21He's been meeting with students.
20:23He's been attending lectures and small
20:26groups and really trying to get a
20:28sense of of what's going on here and
20:31what is working and what is not and
20:33and what is it going to take to help
20:35us get to a curriculum and a pedagogy
20:38that is more effective for our learners.
20:44So just kind of laying out,
20:45you know, kind of their approach.
20:47I think that Phil is taking,
20:49he's currently engaged in an,
20:51you know, ongoing comprehensive
20:53review of what we're doing,
20:56helping us to identify the things that are
20:59working consistent with the Yale system.
21:02And the plan is,
21:03is that we will begin to pilot some
21:06of these in as early as September,
21:09I actually should say August of 2024.
21:12And even now he may be working with
21:15course directors even before then.
21:16But kind of like the take off period
21:19we're hoping is going to be in
21:22August and also embedding formative
21:25assessments in the sessions that we do.
21:28So I'll come back to that in a moment.
21:32So once we are able to really identify
21:35what can and we think will work,
21:38the plan will be to engage in widespread
21:41faculty development to help all of
21:44the courses adopt these pedagogies.
21:45And it is going to require us to
21:48remove some of the outdated pedagogies.
21:50So like we have many lectures that
21:53are still in our curriculum that are
21:56not very well attended and when we
21:59look at the data about how many people
22:01are actually watching the podcasts of
22:03these lectures, even those are low.
22:06So we really must then question,
22:08you know,
22:09why are we taking up space in our
22:11curriculum with these things.
22:13So, so we are,
22:14we are going to need to really be
22:17introspective and be willing to move
22:20things or remove things or change things.
22:23So it's it's going to be
22:25an exercise for all of us.
22:31So I'm going to move on to how we hope to
22:34use the milestones to increase engagement.
22:37So the milestones I'll show in a moment,
22:40but they are, they are the things
22:44that help guide us in, in reaching
22:47those competencies for our students.
22:50So some of the other things
22:53that I've heard from students,
22:55actually as recent as yesterday,
22:59is that if it's not on a qualifier,
23:01it must not be important.
23:03So why are we teaching it?
23:06Or if it's not on step one or two,
23:08it must not be very important either, right?
23:10And so, you know,
23:12engaging in conversations
23:13with students think, you know,
23:15kind of really trying to impress
23:17upon them that you know the
23:19factual based knowledge and even,
23:20you know, kind of the things that
23:22make their ways into qualifiers,
23:23that's just one small piece of the pie.
23:26There's so much more that we're hoping
23:28to engage them in the things that
23:30are not on the qualifiers, right?
23:32The ability to explain their
23:34understanding and reasoning to others,
23:36to formulate their own questions
23:39and seek information and even,
23:41you know,
23:42consider questions that don't
23:43have answers right,
23:45to stimulate their curiosity and imagination.
23:48Maybe that's a stepping off point
23:50to a research project and a thesis.
23:53We also want to make sure that students
23:55are developing the ability to listen,
23:57collaborate and communicate effectively.
23:59We know that the the first day they
24:02step onto their clerkships that those
24:05are skills that people are expecting
24:07them to have begun to develop.
24:09And we need to make sure that we're
24:12providing opportunities for them to
24:15actually make progress in developing
24:17those skills in the pre clinical curriculum.
24:20We also don't assess a professional
24:24identity of formation in the
24:27qualifier and I would say probably
24:29when when I showed you on the screen
24:31just before most of the milestones.
24:33So what are the milestones?
24:34So we engaged in a project about two
24:38years ago looking at our competencies,
24:41which I have on the previous slide
24:44as a reminder and then went a
24:47step further and subdivided them
24:49because these are very broad topics.
24:51What do we mean by that?
24:53And so I've put AQR code up here
24:55that you can access to look on
24:58online at our website where we have
25:01all of the competencies listed.
25:03Dana Dunn and Michael Green LED
25:06this effort with other members
25:08of of the medical education team
25:11and and then took these to the
25:14educational policy and curriculum
25:16committee for review and approval.
25:19And we now have 25 sub competencies
25:21or as we call them educational
25:24program objectives and each of
25:27those educational program objectives
25:30has its own set of milestones.
25:33So what are milestones?
25:34So I put up mechanisms and
25:36treatment of disease as an example,
25:39competency here and mechanisms,
25:42mechanisms and treatment of disease
25:45has an educational program objective
25:48kind of outlined here in more detail.
25:51And then you can see that we went
25:54through the effort of saying OK,
25:56mechanisms and treatment of disease in that.
26:00In that competency,
26:01educational program objective,
26:02what do we hope to see students doing by
26:06the end of their pre clerkship phase,
26:08then at the end of their clerkship and
26:11then right right before graduation.
26:14And so a lot of effort went into just to
26:17coming up with the language to describe this.
26:21Many stakeholders were involved in in
26:23vetting it and tweaking it and getting
26:26it to where we think it needed to be.
26:29And so we can use this tool
26:31to help guide us in,
26:33in creating the assessments that are
26:35needed so that students can monitor
26:38their progress in these domains.
26:40I've now put up clinical reasoning so
26:42you can see we've done this for each and
26:45every competency and sub competency.
26:49When you see,
26:50you know the pre clerkship
26:53success described here.
26:54One of the first things that
26:55we should be thinking about,
26:57well how do we know a student
26:59has reached that point?
27:00How do we know?
27:02And we know that the qualifiers
27:04are probably not giving us enough
27:08information about how students
27:10are meeting this milestone.
27:13Here's another one.
27:14And so this really behooves us and and
27:19pushes us to begin to create formative
27:22assessments for our students to be
27:24able to gather data on how they're doing.
27:27And you know,
27:28I'd like us to be as creative as possible
27:32in developing those types of assessments.
27:36They would probably involve a lot of
27:42observation and interaction with students,
27:45observing them with each other,
27:48engaging in conversations and critical
27:51thinking about different cases and scenarios.
27:54So these are things that are challenging.
27:58They cannot be put on a multiple
28:02choice exam and yet we are
28:05expecting students to reach these.
28:08So I hope that you'll spend some
28:12time kind of reviewing the the,
28:14the rest of the confidences
28:16and milestones on the website.
28:20So kind of getting at what do
28:21we do with these.
28:22So every milestone must
28:25be formatively assessed.
28:27This is not just coming from us like
28:30thinking that it's a good thing to do,
28:32it's it's one of the citations
28:34we received from the LCME.
28:36So that's our accrediting body and they,
28:39you know,
28:40looked at what we had in place to
28:43provide formative assessment for our
28:45students in monitoring their progress.
28:48They looked at our student survey data
28:51with regard to that and noted that we
28:56not not only do we not have a lot there,
28:59but students are saying that they want
29:02more opportunities for formative assessment.
29:04They want more quantity as well as
29:07quality of formative assessment.
29:09They want to know how they're doing and
29:11if they're ready to enter the clerkships,
29:13they want to know if they're ready to
29:15enter the advanced training period.
29:17And so by really outlining the
29:20milestones for students now,
29:22you know,
29:23it's on us to help develop ways and
29:26tools for us to help measure their progress.
29:29And so you know, some have asked,
29:31well,
29:31is that moving away from the El system.
29:33And and I would say the way that
29:36we're approaching this I think is a
29:38fantastic way to be consistent with
29:40the Yale system because we're trying
29:42to provide tools that our students
29:44can use to monitor their own progress.
29:46And as you know and how do
29:48we know they're doing OK,
29:50we're going to be having them meet with their
29:53coaches to go over their progress here.
29:55So we need we need formative assessments
29:58in every phase of the curriculum
30:01for every educational program
30:03objective with their milestones.
30:04We need this to be occurring in
30:07our small group settings within our
30:11simulations and clinical skills.
30:13The assessments need to be meaningful
30:15and provide information so students
30:18can be aware of their progress,
30:20know what they need to work on.
30:23And there I'd also eventually like to
30:26see alternative forms of assessment
30:28for a given milestone.
30:30And why is that important?
30:33Because if students know that they
30:35can receive this feedback within
30:37the small group settings and that
30:39their participation is going to
30:41provide the data that they need to
30:43monitor their progress,
30:45it is an incentive for participation.
30:47It gives new meaning to
30:50participation in workshops.
30:53If they are not attending the workshops,
30:57then how do they demonstrate it?
31:00So one could say they just won't have data.
31:03And so they're going up then meeting
31:05with their coach and they're like,
31:06well, how show me how you, you know,
31:09you're demonstrating that you're
31:11meeting the milestone for communication
31:13and the student doesn't have
31:14data to go over with their coach.
31:17One conversation could be, well,
31:19then I think you, you know,
31:20maybe you should consider going to
31:23your small groups because that's
31:25where you can demonstrate this.
31:27Some could argue while I'm developing
31:29that skill in another setting.
31:31So it would be kind of nice to
31:33have an alternative way for their
31:36skill to be assessed.
31:37And I'm curious myself,
31:39if students find better ways to
31:42demonstrate that they're developing
31:44clinical reasoning skills when they're
31:47not participating in workshops that
31:50are geared to clinical reasoning,
31:53Are they, are they finding other
31:54effective ways to learn that?
31:56I mean,
31:56that would be consistent with
31:58the Yale system,
31:59but right now we don't know because
32:01we're not engaging in assessment
32:04of those skills consistently.
32:09So I'm putting up a draft of the pilot
32:13that Michael Green has helped to develop.
32:17So this is a very simple approach
32:20to providing formative assessment
32:22on some of the competencies that
32:25we haven't traditionally measured.
32:28And so clinical reasoning as
32:29you can see is one of them.
32:31It is. It's it's a very,
32:32very important skill to develop and
32:36we've provided just a description
32:39of what we mean by it here using
32:42case scenarios etcetera.
32:44The student identifies information needed,
32:47forms questions,
32:48assimilates new information and
32:50articulates A cognitive process used to
32:53develop a basic differential diagnosis.
32:55That's the milestone for
32:57pre clerkship students.
32:59So we want all students to get to that
33:01before they enter their clerkships.
33:03Are we providing them with feedback
33:06and information for them to know that
33:08they've gotten to where they need,
33:10where they need to be?
33:14This is one way that we hope that
33:16we can provide that information.
33:17By engaging the preceptors who are
33:20working with them in small groups,
33:22you can see this is a very simple
33:25and straightforward approach and it
33:28is our hope that this type of data
33:30that's going to be collected not just
33:33on this competency and milestone but
33:36multiple others in our small groups.
33:39So this one gets at participation
33:43in the small group setting
33:47being an accountable team member.
33:50We have another one on
33:53interprofessional communication,
33:56all pre clerkship milestones and
34:00another one here on appraising and
34:04assimilating applying evidence to
34:06their thinking and critical thinking.
34:08So those are examples and right now
34:13I believe Michael Green is working
34:15with some of the course directors
34:17to embed these in the current small
34:19group sessions that are happening
34:21so we can pilot their utility
34:23and helping us reach the goal of
34:27gathering data to provide to students.
34:29So what do we do with students
34:31once they have all this data?
34:33So you know, as many of you know we are,
34:37we have rolled out a new coaching
34:39model where each first year student is
34:42assigned to a coach at the beginning
34:45and that coach will follow them
34:47through to the end of medical school.
34:49And the goal is to have them meet
34:51quarterly to review their progress
34:53in reaching these milestones.
34:55So the data that we're gathering
34:57from the sample formative assessments
34:59that I just showed you,
35:00and that's just one example of
35:02the type of data we're gathering,
35:05will be collated together on
35:07a student dashboard.
35:09And students will be able to review
35:11that data with their coach and really
35:15reflect and have important conversations
35:19about whether or not they feel
35:20they've reached the goal or they are,
35:22where they are,
35:23they happy with,
35:24where they're at,
35:25what are the challenges they're
35:27facing in developing the skill set
35:29and what are different ways that
35:32the students can improve themselves
35:34and grow in this particular domain.
35:39And then as I mentioned earlier,
35:41the absence of data on a particular
35:44milestone should prompt a conversation
35:46about how a student can demonstrate
35:49their progress in other ways.
35:51Or perhaps one of the ways is to
35:53re engage with a curriculum and
35:56a pedagogy that as I noted in the
35:59beginning is improved and effective
36:01and now working for our students.
36:10Just want to reflect and and
36:12provide gratitude on the 1st 18
36:15coaches who have embarked on this,
36:17this project with us and very
36:20pleased to hear And well,
36:22I guess we'll be announcing soon that we
36:25have recruited another group of 18 coaches.
36:28And so each of the coaches I think
36:30has about six students at this time
36:32that they'll be following through
36:34the curriculum and this will go in
36:37through second year, third year,
36:394th year all the way up to graduation.
36:42So it's our hope that by really kind
36:44of rolling out a model like this that
36:47we will increase student engagement
36:49in our curriculum by providing them
36:52opportunities to gather data on how
36:54they're doing and I hope create new
36:58meaning for students and how they can
37:02like what the curriculum here at the
37:05Yale School of Medicine is to them.
37:07You know,
37:08not something that is overly
37:10variable and then somewhat optional,
37:13but actually integral to their learning.
37:15And you know,
37:16they regard it as a mechanism and
37:19a tool to develop the competencies
37:22that we've outlined.
37:24And I can't assess and I can't
37:26emphasize enough how important I
37:28think it is to embed these types
37:30of formative assessments in almost
37:32every learning activity that we have.
37:35So that by participating in the curriculum,
37:38students are gaining information
37:39about how they're doing.
37:41Without that, you know,
37:43engaging in the curriculum is
37:45almost just kind of like a drop in,
37:47kind of like maybe it'll help me,
37:49maybe it won't.
37:50What am I going to get from it?
37:52Maybe I'll get something from it,
37:54maybe I won't.
37:55We have to be much more consistent
37:57in what we're providing to students
37:59and really be willing to kind of
38:02evaluate ourselves and how we're
38:05doing and and think about how
38:09we can improve at the same time.
38:11So I'll stop there and and open
38:14it up for a greater discussion.
38:17Thanks, Jessica.
38:18That was wonderful and we have a
38:20number of excellent questions.
38:22What I'm going to start with,
38:23there was a question about lectures
38:25and you did mention that you were
38:27going to sort of review the lectures
38:28and and really make some space
38:30for more engagement activities
38:31because I think the question was
38:33we all agreed that traditional
38:35lecture format is not that great,
38:37but we're still using many lectures.
38:40And what are the plans to make changes?
38:43For example, I think even HMS
38:45now doesn't have any lectures.
38:47If you could comment on that.
38:49Yeah, well, I think that, you know, this
38:53is going to be a process that takes time.
38:57I don't think that the wise thing to do
39:00would be just to end all lectures in our
39:04curriculum because we have to have valuable
39:07things to replace them with, right?
39:09So it's going to take a lot of time
39:12to go through and look through each of
39:15the courses with the course directors
39:17and say what are the things that
39:19you think are the most high yield?
39:21Which things do you think should
39:24remain a large group format?
39:26And if it's going to be a large group format,
39:29how can we make it more
39:31interactive and stimulating?
39:33How can we make sure that it
39:35will engage students in critical
39:37thinking and not just kind of as
39:40passive recipients to knowledge.
39:42We know that presenting information
39:44that way has a very low retention rate.
39:47So you know,
39:48I think we're going to need to
39:50engage the course directors in that
39:52exercise and be very thoughtful
39:54about what remains large group,
39:56what becomes small group and then if
39:58things are going to become small group.
40:01I know that we need to have student
40:04participation in order for that
40:05to be meaningful.
40:06And so we really need to be thinking
40:09about what we have to do to make that
40:12high value added for students once
40:14we move something to a small group.
40:17And that was also building on another
40:19comment saying that attendance has
40:20been low in the small groups as well.
40:22So just building on that,
40:23that yes, we don't want to
40:25just blame the lectures alone.
40:27One question, Are there other medical
40:29schools that have effectively motivated
40:31students to attend any of the learning
40:34events that we're familiar with?
40:37That's a great question, Janet,
40:39and you might know more about that than me.
40:42I do know that at many medical schools,
40:45small group activities are
40:48mandatory and attendance is taken.
40:51Now we have tried that in the past
40:53here by kind of defining which
40:56small groups should be mandatory,
40:58stating that we are committing to
41:01providing high quality instruction
41:03in those small groups that are
41:05mandatory and have consistent
41:06membership over a certain amount of
41:09time with a consistent faculty member.
41:11But I think one of the things we
41:13have to acknowledge is we if we
41:16want to make things mandatory,
41:18we have to do a lot more work to
41:20make sure they're high quality.
41:22Because forcing students to attend
41:24something that is very variable in quality,
41:27in nature and not necessarily effective
41:29for learning is not really creating
41:32trust with our students in the curriculum.
41:35So it, you know,
41:36it maybe in the future we'd get to
41:38a point where even students say,
41:40yes, we agree this series of small
41:42groups should be mandatory because
41:44we all agree that they are highly
41:47effective methods for teaching.
41:49And we know that we're benefiting from it.
41:51We have data that's showing us
41:53that we're benefiting from it.
41:55But I don't think we quite were
41:57there when we tried to make things
41:59mandatory in the past.
42:00And for those of you who are around
42:03when when kind of like things
42:05started to boil over,
42:07many students were coming forward
42:09saying that they were being essentially
42:11forced to attend things that were
42:14not effective for their learning.
42:16I I know though that many of you
42:19hold high quality small group
42:21sessions and you do provide effective
42:24learning settings and it's very
42:26frustrating when you see your roster,
42:28you're supposed to have 12
42:30students there and two show up.
42:32That is extremely frustrating.
42:33And so I guess what I would say when I've
42:37talked to students about why that happened,
42:39it's because,
42:40let's put it, you know,
42:43to use colloquial terms,
42:44they've been burned right?
42:46So before in the past,
42:48like months before they got to your course,
42:50they've tried to attend small group sessions
42:53and there's been so much variability in
42:56the quality that it's now taking a chance.
42:59Like if I go to this,
43:00is it going to be high value?
43:01Maybe.
43:02Like we saw the quote,
43:03it's a 5050 chance.
43:05So they then the attendant attendance
43:08declined significantly over time.
43:10So like the very last course of the year
43:13now is seeing very low attendance rates.
43:16So this is something that we're hoping
43:19to work with Bill to really address
43:21at the core root of the issues.
43:24And I think you're raising
43:25the important point that yes,
43:26there are some schools like
43:28National University of Singapore,
43:29Duke and US and it's not
43:32about being mandatory,
43:33it's about having a very
43:35consistent expectations of what
43:37And just like you said Jessica,
43:39having a faculty with the
43:40same group of students.
43:41So they really work together and understand
43:44the goal of clinical reasoning and
43:48critical thinking and how that works.
43:51There was a really interesting
43:52question here though,
43:53asking how do we balance sort
43:55of different goals.
43:56Stimulating the students curiosity
43:58and problem solving is crucial,
44:01but it's true they only have a
44:03finite bandwidth and so how do we
44:06balance problem solving in this
44:07curiosity with more pragmatic goals?
44:09Are we just teaching so they will gather
44:12knowledge for the steps or qualifiers?
44:16That's a great question.
44:18Yeah, I think there's,
44:20there's an important balance
44:21to reach there because the
44:23students are coming to us to
44:27be exposed to content, right.
44:30And you know, they they are expecting on
44:33some level some delivery of knowledge.
44:35But I think more and more what we're
44:38recognizing is that the knowledge
44:41and the content is available in
44:43so many different forums now.
44:46And that's not necessarily the
44:48purpose of coming together, right?
44:51So the purpose of coming together is,
44:54like I've mentioned many times,
44:55to learn how to think critically
44:57and engage with this information
44:59and apply this information.
45:01So we're going to have to
45:03try to strike a balance.
45:04You know where when small groups happen,
45:07students know what they need to do
45:10ahead of time in in coming prepared,
45:12whether it's watching one of
45:14our podcasts or engaging with
45:17some material ahead of time.
45:19They need to know what that is.
45:20And then when they get to their small group,
45:22they need to know what the
45:24expectation is from of themselves
45:26in participating in that group.
45:28Right now,
45:29there's so much variability in our
45:32pedagogy that students don't actually
45:34know what it is they should be doing as
45:37they move course to course to prepare
45:39for the different types of groups.
45:42Because the groups are kind
45:43of structured differently.
45:45They have different pedagogies,
45:47different formats.
45:48You know,
45:49some of our small groups right now
45:51are kind of run as many lectures and
45:54others are more case based than are
45:56expecting student participation and
45:58engagement and critical thinking.
46:01And so if students don't know what
46:03it is they're expected to do in
46:06participating in these small groups,
46:08I think it really effects their
46:10ability to participate effectively,
46:12which
46:13really answers this other question
46:14of how do you maintain the value
46:16for the few students who do attend,
46:18But by having a consistent approach
46:21that the students understand,
46:23working collaborative collaboratively
46:25with the same faculty.
46:27And then another person says,
46:29well, are we going to be able to
46:30get enough small group leaders
46:31if this is going to be a change?
46:34Yeah, that's that's an excellent question.
46:39You know, I that's I think,
46:40I think I'm sure Bill thinks
46:42about this all the time as well.
46:43So you know here here he is trying
46:45to gather data on what's working well
46:48and beginning to envision potentially
46:50what we can share with all of the
46:52other course course directors and
46:54all the faculty you know as a method
46:57to integrate into their courses.
46:59But that's going to take time to roll out.
47:01So what do we do in the interim, right.
47:03We can't like halt the curriculum
47:05and then make changes and then
47:08have everything, you know,
47:09kind of be different come August.
47:11You know, it's it's can't work that way.
47:13It's gonna have to be more iterative
47:16and change is going to happen I
47:19think more gradually over time.
47:21And so we will be challenged.
47:23And I hope that you know even in
47:26interacting in these exercises where we
47:28talk about it here and with Bill and
47:31some of your Friday seminars and other
47:34faculty development sessions we hold.
47:35I hope that people are starting to
47:37like take in you know what is evidence
47:39based pedagogy and what are the
47:41things that I can begin to incorporate
47:43into my small groups now even before
47:45we're kind of like rolling out a more
47:48consistent approach in our curriculum.
47:51Great these wonderful notes even
47:53from our librarians and we have such
47:56a rich resource and and please for
47:58everyone please chat because there are
48:00a number of resources that Courtney's
48:02offered and there's also identifying
48:05A librarian for each of the colleges.
48:08So again we we do see the value even
48:11James said that a librarian at his
48:13medical school was an unofficial
48:15coach then called counselors and
48:18her input helped this person in
48:21professional development.
48:22So again these are wonderful
48:24comments let's just open it up.
48:26I just I love all these and we'll save
48:28all the the chat the so let's just
48:33yes Dana that is a great overview
48:36Jessica and you've done so much great
48:39work and I'm very excited for all
48:41the possibilities and I might have
48:43gotten this idea from Bill Rando but
48:46to to pilot something where the the
48:49students who seem to learn best or
48:52value coming together to have them
48:54like track and opt into like a track
48:56to to to commit to each other in that way.
48:59And it's almost kind of like you
49:00could have your
49:00own natural experiment about
49:02learning goals and what works best for people
49:05you know while while this is
49:07getting going. I'm pretty sure
49:08that was your idea Bill but it's
49:11it's it's an idea to think about not
49:14leaving not alienating the ones who
49:15are really getting disappointed by the
49:17others that don't show up to help.
49:23Great point, Dana.
49:26And often pilots work.
49:27I don't know if you want to say
49:29anything more to that, Bill.
49:33Yes, I do. Hi. Thank you for bringing
49:35that up, Dana. Hello, everyone.
49:40Yeah, I mean, I think what, what? What?
49:43What? Jessica was just saying at the end,
49:45it's that this isn't there's
49:48no way this can happen as oh,
49:50we're a whole new school as of September 1st.
49:54But we can do experiments.
49:57We can, we can create new
50:01opportunities for faculty.
50:02I mean, one thing is maybe there
50:07won't be as many faculty lecturing,
50:11but we've still got a lot to do.
50:13We've got a lot to do with students
50:17and we only have 105 students which
50:19I'm always amazed it's not just
50:22small number relatively we we can
50:27do formative assessment with them
50:30through coaches and through faculty
50:33who who now may be having new
50:37roles as as the curriculum changes.
50:42And all of these things can be piloted
50:45on a small scale and then we talk about
50:49it and then things over time change.
50:51But thank you for bringing that up,
50:52Dana. Great.
50:54And and we're also Scott,
50:56I think your question,
50:58Scott about that you would like
51:00to have more opportunities to
51:02learn what are the effective
51:03strategies in providing educational
51:05opportunities for our students.
51:06I think everyone feels that.
51:08So it's an important piece
51:10that we'll be working on. Yeah,
51:12I anticipate that, you know,
51:14as Bill is gathering more and
51:16more data from seeing what it is
51:18that's working in our curriculum,
51:20I anticipate we would be wanting
51:22to hold sessions to share with more
51:24faculty what is working, right.
51:26And then there's probably a lot out
51:28there that we haven't even tried
51:30at this institution that other
51:32schools have found is working and
51:34they've written about and published.
51:35So those are also things that we'll want
51:38to bring in and begin to incorporate
51:41and either pilot or expand to other faculty.
51:44I see a bunch of questions about
51:46the need for more faculty.
51:48And yes, that may be true, right?
51:51You could look at it both ways.
51:53It might mean that we need more
51:55faculty to hold more small groups.
51:56Or in some situations it might be that
52:00it's not more faculty but fewer faculty,
52:03but just over longer periods of time.
52:06Some schools have moved to that
52:08where they have a core group of
52:11workshop leaders that follows the
52:13students through the curriculum
52:15or team based learning leaders.
52:16So these are things we're really going
52:17to have to think about. What would work?
52:19I see Franz has her hand up.
52:22Thank you. And I'm sorry that
52:23I was in and out because I was
52:25still seeing patients. But
52:29I just want to acknowledge,
52:31you know that I think, yes,
52:34I think we all know that we
52:36need to make some changes.
52:38And I'm extremely grateful for everything
52:40that you and others have done that we
52:45have done with you and for Bill's help.
52:49I've already had wonderful
52:50discussions with Bill.
52:52I love talking to Bill,
52:53but I also want to acknowledge
52:56that when we worked on the new,
52:58no longer new curriculum, we also
53:01were aiming to introduce new pedagogy.
53:05And you know, I also,
53:09I don't want faculty to be
53:11discouraged feeling that,
53:12oh, we've tried and we've,
53:14we've, you know,
53:16increased the number of small group sessions
53:19to be at least 50% of all sessions.
53:21We switched a lot of sessions to TBL,
53:24So I was glad to hear that you said
53:27you also want to identify what
53:28works so that we can keep what works
53:30that we're already doing well.
53:32But maybe we need to clearly,
53:35I think if we want to
53:39to to to to for the students
53:41to be more engaged,
53:42we need to do something different.
53:44And so it's not that we didn't
53:47try and I think that small to
53:51have enough faculty is an issue,
53:53but right now we have too many
53:55faculty for the small groups because
53:57we don't have enough students that
53:59join those small group sessions.
54:01So, you know, this is a work in progress.
54:04And I just want to give us a little bit of,
54:07you know, kudos for what we did in
54:09the past and for continuing to be
54:11engaged in being agents of change.
54:13And thank you all.
54:14Yeah,
54:15Thank you, Franz. Yeah.
54:16I mean, you have been an
54:18amazing leader in this respect,
54:20trying every year to tweak the pedagogy and
54:23across the lifespan in the small groups.
54:25And like we've all been right there
54:27with you like trying different things.
54:29And you know, I think you know
54:31that has been a very important
54:33process in learning like what is
54:35what works and what hasn't worked.
54:37And I think that's also something
54:39we all have to acknowledge is
54:41that it's not like we're going to
54:44identify the pedagogy that will
54:45work and just roll it out, you know,
54:48through the whole curriculum.
54:49I think we're going to need to think
54:52about each course's needs and you know
54:54what are the goals of each course.
54:57You know,
54:58maybe some are more focused on
55:00clinical reasoning and some are more
55:02focused on acquiring a knowledge base,
55:04right, in certain areas.
55:06So we just really need to work with
55:08Bill and the course directors to
55:10kind of determine what can work
55:12best in each setting,
55:13but have some type of consistent
55:15approach that we are able to
55:17present to our students when they
55:19come here and explain to them what
55:22the expectations are and and why
55:25like why are we teaching in this
55:27way and what's the purpose of it.
55:30We know that just kind of like
55:32throwing them into the small groups
55:34with faculty without explaining
55:35that ahead of time.
55:36It has not been effective.
55:38So we have a lot of work to do.
55:40We're even we're working right
55:41now on developing some sessions
55:43for intro to the profession so we
55:44can kind of get students right
55:46at the beginning and
55:47just before. I would love for everyone
55:49to just as we have the last one or
55:52two minutes if you'd go into the
55:54chat and fill out the evaluation.
55:56We really cherish your input and the
55:58evaluations are important for all
56:00of us and then we can still have I
56:02know Bill has had his hand up and and
56:05Farzana had her hand up but Bill go
56:07ahead and but please go into the chat
56:10if you wouldn't mind while we finish
56:11up and and fill out the eval bill
56:15Thanks. I I related in some ways to
56:20what Franz was saying and and you know
56:22I think everyone agrees with this.
56:23One thing that I notice
56:25can't miss it and that we're.
56:27I think we're gonna really try to
56:30address is we need to to change the
56:33culture of the students as they
56:36relate to this institution more,
56:41more more definitively more clearly
56:45more enthusiastically get them
56:48engaged in this process so that
56:52they're not always just looking for
56:54things that are high yield for them.
56:56I mean I hate to say ask not
56:58what you can do for your school,
57:00what your school can do for you.
57:01Ask what you can do for your
57:03school in a what.
57:04What that is to say is that
57:08learning happens when we set
57:10goals and we take make efforts.
57:13So that's another big part of the change.
57:16I, you know, it's it's that that,
57:20yeah, that's all I want to say.
57:21Thanks, Dylan. And it's One O 1.
57:23So Jessica, thank you so much.
57:25It was stimulating and your
57:27leadership and helping us work
57:29together and move all of these
57:31great ideas forward is so important.
57:33So thank you very much and thanks everybody.
57:35It's our last session of this year,
57:38but we have Med Ed Day.
57:39So see you on Med Ed Day in person.
57:42Bye bye. Thanks
57:43everyone. Thanks, Janet.