1/5 YES!: Improving Written Feedback

January 05, 2024

ID11156

To CiteDCA Citation Guide

00:00But thank you for joining us and the
00:03tag team that we have is that I'll
00:06say a few words about the series.
00:08Yes, the all medical educator series that
00:12I've had the great fun and privilege of,
00:14of putting together with with Dana.
00:17And we have put together roughly a
00:19dozen or so talks over the course
00:22of the year where we're hoping to
00:25give very practical materials to
00:28educators ranging from how to well,
00:30how to do everything that as clinical
00:33physician educators we need to do.
00:36And we will be talking about feedback
00:37today and we certainly welcome your
00:39feedback about the series as a
00:41whole in addition to the feedback
00:43to the session itself.
00:45So with that,
00:46I will pass it on to my compadre,
00:50Dana to introduce our speakers.
00:51Dana,
00:54thank you, Andreas.
00:56Happy Friday everybody.
00:58Thanks for joining.
01:00Dana Dunn, Infectious Disease Co
01:02Director of this series with Andreas.
01:05And I'm happy to welcome two of my
01:08colleagues who have given this talk.
01:09When we did this in the Department of
01:12Medicine only for the last few years and
01:14it's been really well received as helping us.
01:18For so many of us that need to
01:21write evaluations on learners,
01:22regardless of the level
01:24that they're they're in,
01:25they're going to introduce
01:26you to a helpful framework.
01:28So I'm excited to introduce
01:30Matt Grant to you both,
01:31to all of you today.
01:33He's an infectious disease colleague
01:34of mine who's an Associate program
01:36Director for the Internal medicine
01:38Residency program and Laura Morrison,
01:40who is in the section of
01:42Geriatrics who directs hospital
01:44and palliative care education,
01:46as well as is the director of the Hospice
01:49and Palliative Care Medicine Fellowship.
01:51So they're going to chat with
01:53us for a while and at the end,
01:55we're going to remind you
01:56about some upcoming sessions.
01:57So over to you,
01:59Matt and Laura.
02:01Thanks for that introduction, Dana.
02:03It's really great to see everybody's faces.
02:06Happy New Year.
02:07I'm in day three of COVID Isolation
02:10after a trip to New York City.
02:12Essentially, I'll timed.
02:13So you're relieving my loneliness
02:15and you're boosting my mood,
02:17so I thank you for that.
02:19Our CME code for today,
02:21as is referenced in the chat, is 38938.
02:27All right, So we're really
02:28excited to be talking to you
02:29today about written feedback.
02:31We have three big goals set
02:33up for your objectives.
02:34One is to talk about best
02:37practices for delivering and
02:38actually writing written feedback.
02:41We want to talk about some barriers
02:42that are in place that are real and
02:44they do hinder us and we actually
02:47want to move through the theoretical
02:49and lecture based piece of the
02:52talk rather swiftly because we
02:53actually want to do some practical
02:55writing with you guys today.
02:56So we want to have you put keys to
03:00key fingers to keyboard and we're
03:03going to be using the chat to
03:06actually write down some written
03:07feedback based on some clinical
03:09scenarios or student learner
03:11scenarios and we're going to give you
03:13some kind and supportive feedback.
03:17All right. So three parts of the talk today.
03:19We're going to be giving a little
03:21bit of background and then we're
03:23going to be introducing a new
03:24model for you to practice today.
03:26And if you like it,
03:27we hope that you will use it a little bit
03:29or a lot moving forward into the future.
03:31And then we're going to spend
03:32the about half the session,
03:33we hope actually doing some writing
03:35because it is a written feedback talk.
03:41All right, so as I mentioned before,
03:44there are legitimate barriers
03:46to written feedback.
03:48Good read, good written feedback
03:49does not occur in a vacuum.
03:51It is part of a admittedly time
03:54intensive process which typically
03:56begins with when you meet a learner
03:59doing goal and expectation setting
04:01and then putting yourself in the same
04:04space as the learner so you can observe
04:06their skill set and take note of it.
04:08And then if the if the observation
04:11period is long enough,
04:12doing some mid rotational verbal feedback.
04:15And then we always like to a
04:17tenant of written feedback is that
04:19it should be coupled to kind of
04:22synchronous written feedback.
04:24So the written feedback and
04:25the verbal feedback that the
04:27learner receives should match,
04:29should dovetail very, very well,
04:31be very, very close to each other.
04:33Another obvious barrier is a
04:35limited observation window.
04:37For example with the throughput
04:38issues involving the emergency
04:40room and the respiratory viruses
04:42going on when when I attend,
04:48when I attend on the Donaldson
04:50firm in East Pavilion.
04:51We don't get a lot of fresh admissions
04:53because a lot of patients need to
04:54get admitted by hospitalists while
04:56they sit in the ER for 24 hours.
04:57Though I don't get to see unfortunately
04:59a lot of of my medical students you know
05:02sub is and interns do fresh work up.
05:05So you know we we can only see
05:06what we can see and we can try
05:08to be there as much as possible
05:10but we do have limitations there.
05:12Faculty skills is a barrier.
05:15I applaud all of you for being here
05:17today being proactive about trying to
05:19develop your skills and written feedback.
05:21It's been a process for myself,
05:23and I'm sure it has for Laura
05:24and Dana and all these people who
05:27are in a teaching position.
05:28And sadly,
05:29fear is a really important
05:31barrier to written feedback.
05:32There's a lot of misinformation out
05:34there about what the implications of
05:37written feedback can be for a learner.
05:40Some people mistakenly think that
05:42what you write in Med Hub or in
05:44other scenarios will follow the
05:46learner in some sort of negative way.
05:48People are scared about their
05:51own faculty positions,
05:52are scared sometimes about their own
05:54promotion and how if giving constructive
05:55feedback to the learner may kind of
05:57bounce back and hurt them when it
05:59comes to the time of their own promotion.
06:02And lastly,
06:03you know,
06:04if you reach out to somebody's
06:07mentor and and tell them you think
06:09someone's really skill deficient,
06:11people feel that they could be at risk
06:12for getting pulled into a real time sink.
06:16So I've been interested in this written
06:19feedback issue for several years.
06:20I had the pleasure of in 2019 doing
06:23a Med Ed fellowship working with
06:25Janet Hafler and John Encandela,
06:27and I chose an audit of the internal
06:31medicine residency's attending to
06:34resident written Med hub evaluations.
06:38I did a qualitative and quantitative
06:41analysis and I coded all the free
06:44form written feedback that went from
06:47attendings to both interns and residents.
06:50And the most important take away point
06:52of that is that we were giving at
06:54that time and we still are giving an
06:57incredibly small amount of actionable,
06:59constructive feedback to our learner.
07:01So on average,
07:01you'd have to work with two
07:03attendings for enough time to get
07:04them both to fill out an evaluation,
07:06which was not inevitable to get
07:08one piece of behavioral change
07:10that was recommended to you.
07:11So that was a little sad to me
07:14and that drive my interest in
07:16this realm for the last five,
07:18and it will continue for many more years.
07:21We don't have time to do a deep dive on this,
07:23but both gender bias and racial bias are
07:27prevalent and well described in this realm.
07:31This particular study that I'm referencing
07:33here involved emergency medicine.
07:34Residents and residents identifying as
07:38female less consistently got feedback
07:41about how to improve their skills,
07:43and they also got descriptors that
07:45more prevalently described their
07:47personality rather than things that
07:48were associated with their behaviors
07:50and their team leadership skills
07:52and kind of the things that we
07:54would think would lend people to be
07:55selected for leadership positions.
08:00Likewise not this is in the
08:04undergraduate medical domain.
08:05This is about clerkship students.
08:09So this is like a a map of what
08:12attributes were related to people
08:14based on the grade they got on the
08:16rotation and whether they were under
08:18represented or not in medicine.
08:20And you can see here highlighted
08:22in the lower right quadrant that
08:25under represented in medicine.
08:27Clerkship students again received
08:29more personal attribute descriptors
08:32and they were less likely
08:34to receive competence based.
08:36And you can see there were often, you know,
08:39appreciated for their bilinguality,
08:42but that we're missing a lot of
08:46these specific leadership traits.
08:51So when we think about written feedback,
08:53we don't think about it
08:55just as an evaluation.
08:56There's really three parts or like
08:58a tripod that good written feedback
09:01sits on and that is appreciation.
09:03So appreciation is really important.
09:05It reinforces the effective and the
09:08effective things that the learner's doing,
09:12the strong things.
09:13And it helps build that learning
09:15climate so that you can safely deliver
09:18constructive feedback and that that
09:20constructive feedback can be taken in
09:22a in a kind and respectful and that
09:25the the learner wants to implement
09:27change and isn't taken aback by what
09:29you're telling them to do better.
09:30So that's the coaching leg of the
09:32tripod that I was just talking about.
09:38Everybody wants to improve.
09:41Ideally, they want to do so in
09:43a nurturing environment where
09:44they're supported and they feel
09:46valued and that the that their
09:48preceptor or mentor in them cares
09:51about them both as a person and as
09:53a developing physician or learner.
09:58And so all these things are important.
09:59And lastly, the evaluative part is
10:02I'm going to get into why evaluation
10:04is so important and that's not
10:06the part that people who are not
10:09in leadership positions within
10:11educational leadership tend to focus on.
10:13But it is really important.
10:17So what makes written feedback so important?
10:19Well, it does tails as I said or
10:22reinforces the verbal feedback.
10:23So when you meet with someone at the
10:26end of a rotation, for instance,
10:27and then you tell them one thing,
10:29and then ideally you know a few days or
10:32a week later they read the same thing,
10:34they're more likely to kind of
10:37internalize that feedback, act on it,
10:39cannot keep it in their mental
10:42register as they move forward.
10:44It's also a really important
10:46source for a variety of, you know,
10:50committees and things such
10:52as letters of recommendation,
10:53Mspes or Dean's letters.
10:56That so especially that positive
10:58reinforcing feedback that you're
10:59giving in the written feedback
11:01realm is helping our learners
11:04achieve their professional goals and
11:11really helping us create milestones
11:13and observe a trajectory of somebody
11:15as they grow from someone who
11:18requires a lot of assistance in the
11:20patient care realm to eventually
11:22ideally becoming fully independent.
11:31So in general this applies to both written
11:33and verbal feedback. But what does good
11:35feedback look like? Well,
11:38it should be sorry.
11:41It should be well intended
11:42and based on shared goals.
11:44It should be observed as a direct behavior.
11:49We like to get away from describing
11:52someone's personality in an evaluation and
11:55really describing their actions and their,
11:57you know, if we're going to say how you know,
12:01kind they are or how much they
12:02advocate for their patients or
12:04the rapport they're building.
12:05I think specific behaviors or anecdotes
12:07are always so much more powerful to me.
12:10When I'm reading, let's say,
12:12a letter of recommendation in my
12:14position as like selecting candidates
12:16for residency or fellowship,
12:18I feel like that's so much more meaningful.
12:20Good feedback should be directly observed,
12:23should be a very specific thing,
12:26should be as timely as possible,
12:28and it should be manageable.
12:29So when I have somebody on the Donaldson
12:32firm doing HIV medicine with me,
12:34you know it's not reasonable
12:35for me to give the feedback.
12:36You know,
12:37like you should learn and be
12:39comfortable prescribing antiretrovirals.
12:41That is a really complicated
12:43goal and someone needs a really
12:45well defined path that needs to
12:46be broken down into steps.
12:48So you can't just,
12:49you can't just throw something that
12:52is that complex out and ask someone
12:55to perform to gain progress or gain
12:57skills that are such a multi step process.
13:04Feedback comes in a couple different forms.
13:06There's both formative which is kind
13:08of in the process of learning, whereas
13:10summative feedback is more evaluative.
13:13Formative feedback can come in the form of,
13:15you know, texts, emails either to
13:19the learner or to their mentors
13:21to let them know about, you know,
13:24both reinforcing and constructive.
13:26Sometimes we want to hear about
13:27and we do hear about really amazing
13:29things our residents and interns
13:31and students do on the floors.
13:33And at other times we hear that
13:36someone is kind of not on trajectory,
13:39not on target with their peers and
13:40they're going to need a little
13:41bit of help in catching up.
13:43It's really important that
13:44we know both things.
13:48Many of you who are here today in the
13:50audience may work with students or
13:52residents in workshops or in like a
13:55standardized patient encounters or Epas.
13:57And those are another example
13:59of feedback that's important.
14:00Whereas summative or as I said kind of
14:03like the at the end where you're kind
14:05of making an evaluative like is this
14:08person ready to graduate, you know,
14:09how is this person ranked as a candidate.
14:11Moving on, graduating from medical school,
14:14you know these are no longer
14:16serving a coaching role,
14:18but this kind of written feedback is
14:20serving more of a an evaluative role.
14:26So a couple pros about written feedback.
14:28Both the learner and the mentor
14:30can refer back to it over time.
14:32Whereas you know with verbal feedback,
14:33you hear what you hear and you know
14:36human memory is not infallible.
14:37We can kind of twist things
14:39in our minds over time,
14:40but the written word stays the same.
14:42It can be helpful for documentation to make
14:45sure if somebody needs some help catching up.
14:49As I mentioned that that's their kind
14:52of begins a process of identifying
14:55what's going on with the learner
14:57and taking steps to come up with
14:59a plan to help them catch up.
15:01Cons of RIN feedback is that if it's not
15:03done without concomitant verbal feedback,
15:05you're missing an important opportunity for
15:08the learner to self assess about their skill,
15:11to watch them physically react
15:13to the feedback you're getting,
15:14and to engage in like an open
15:17dialogue about it.
15:18And if the written feedback doesn't contain
15:20or imply a specific plan for improvement,
15:23it kind of tends to just make the
15:25person feel bad about their skill set.
15:28And if they don't have a way that's obvious
15:30or told to them about how to get better,
15:33then it just seems,
15:35you know, derogatory.
15:37And it's can be frustrating to not know
15:40what where to take negative feedback
15:42that doesn't have a plan of where to go.
15:47All right.
15:47So at this point,
15:47I'm going to turn it over to Laura Morrison,
15:50and she's going to tell you about a new
15:52framework for doing written feedback
15:53that we hope you're going to practice today.
15:56Thanks, Matt. I want to take this
15:59opportunity to just kind of end
16:01our stage setting by giving us a
16:03longitudinal look at all a lot of
16:06the concepts that Matt just laid out.
16:08So just realizing that whether we're
16:11interacting with a learner in a course
16:14or a workshop or a clinical rotation,
16:17there really is a beginning,
16:18a middle and an end.
16:20And each of those have a a possibility
16:23of an important contribution to feedback.
16:26Again, referring to the importance
16:28of setting goals and expectations,
16:31making them smart,
16:32kind of referring to that specificity
16:34and all the other important
16:37elements of goal setting.
16:38And remembering that Dana Dunn gave
16:41us a talk on that last month in
16:44this series that you can refer to.
16:46But the beginning is about that growth
16:49mindset that Matt mentioned and really
16:52establishing the opportunity for bi
16:55directional feedback and those expectations.
16:57So in my setting,
16:59I typically ask the learner what
17:01their goals are and then I actually
17:03share what mine are and what I'm
17:05actually working on and what I'm
17:07going to ask them for feedback about
17:09ahead of time for my own learning,
17:14right.
17:14And then we have this the mid middle
17:17period which is helpful because
17:19it's it's actually invaluable,
17:21it's the it's the time that we can
17:24actually provide some earlier feedback
17:25and then see if the learner is able
17:29to incorporate that feedback before
17:31the end of the experience together.
17:34And then of course at the end we have
17:37the opportunity to really make sure
17:39that the verbal and the written align
17:42and give and give sort of that double
17:45message to move people forward next slide.
17:50Great.
17:50And so this is our,
17:52this is the tool that we're interested
17:54in presenting to you today.
17:55It's called the BOSS model.
17:58We're not aware that BOSS has a
18:03particularly hierarchical reference
18:05and we're just acknowledging that it
18:09it's a tool for you to think about and
18:12see if it if it's a model that works for you.
18:14There may be some elements that do,
18:17but it's something that we're gonna practice
18:19today to see if it it kind of sticks,
18:22but it kind of has this BOSS.
18:24You know,
18:24we might use the term Riz if we
18:27had the opportunity to because
18:29that's the word of 2023, right?
18:32But they gave us BOSS instead.
18:34So BOSS stands for brief observation,
18:39significant and suggestions,
18:40and we'll take a minute here
18:43to look at it a little closer.
18:46But the brief observation
18:47part shouldn't be a surprise.
18:49We've been talking about how important
18:53directly observing behavior is,
18:55and so this reminds us to just take
18:57advantage of all the opportunities we
19:00can to directly observe and then it.
19:02I love this part about the model is
19:05it really reminds us of how important
19:07it is to frame our observations
19:09specifically for the learner to be able
19:12to receive it as effectively as possible.
19:15So we want to use a non judgmental,
19:18neutral if possible framing,
19:19and we'll look at that on the the
19:23next slide when we get there.
19:25But significance is is what we link to next.
19:29So we go from the observation to then
19:32thinking about what the significance is.
19:34And this is important because how we see the
19:38significance may not be the way that comes
19:40across as being valuable to the learner.
19:43So we really need to think
19:45about from their perspective,
19:47how they're going to see that what
19:49we care about is significant to them.
19:52And then we want a link at the end.
19:54Most importantly,
19:55as Matt pointed out from his study,
19:58that we really do need to be specific with
20:01a few suggestions and we want to aim for.
20:04You know, ideally 2 to 3 suggestions
20:07in how learners can improve.
20:09We want those to be specific and relevant
20:12to something that they could accomplish.
20:15Next slide, Matt, thanks.
20:18So we can go ahead and advance.
20:22So thinking about that judgmental aspect
20:25and trying to be neutral on the left side,
20:29we're going to see some comments that
20:31Matt and I have seen on evaluations.
20:33You know,
20:34Doctor X spends too much time on
20:36the computer in the patient room.
20:38So that's something that one of us might say.
20:41But how can we frame that observation to
20:43be a little more productive for the learner?
20:46So go ahead, Matt, look here.
20:49So another example might be eye contact
20:52between the doctor and the patient
20:54was less than 10% in this encounter.
20:57So making it again more objective and
21:00starting to lead into the significance,
21:03right. Next example, we just have one more.
21:07This one's a little more positive.
21:09Doctor X is very knowledgeable,
21:11but that doesn't really tell Doctor
21:13X what they're doing well or what
21:15was actually seen, right?
21:16So go ahead, Matt.
21:19So Doctor X routinely performs literature
21:22searches to develop their knowledge base.
21:25That gives the learner a little more of
21:27a sense of exactly what they're doing.
21:29Well, Next slide.
21:32So moving on to significance
21:35the first S and again,
21:38realizing that our framing as evaluators
21:40might be different from the learner,
21:43we might say something like, well,
21:44they tell me they're reading on
21:46their patients, but I can't tell.
21:48And instead if we're going to
21:49frame that for the learner,
21:51we might say, you know,
21:53sharing your learning points from your
21:56reading increases your team contribution
21:58and lets you practice your teaching skills.
22:01So really setting it out in
22:03quite a different way around,
22:05you know, learning specifically.
22:09And then the other example,
22:11they come late to clinic
22:13disrupting patient flow.
22:14So we might spin that one as well
22:17to how time management is important,
22:20effective time management maximizes
22:22learning opportunities in clinic.
22:26Next slide. So taking a minute to
22:29think about how we make suggestions,
22:32I think you'll all recognize these
22:34as common comments we see for what
22:37people can do to improve, teach more,
22:39read more, you know that's kind of
22:41the most common one ever, right.
22:43Instead we just want to be more
22:47specific and learner centered.
22:48So you know, I'd like to see you
22:51bring primary literature to rounds
22:53that is pertinent to how we are
22:55making our medical decisions, right?
22:57Or I we can just be much more specific
23:00about what someone should read,
23:03you know, some indications for
23:05diagnostic test or something else.
23:08So all right, on to the next slide.
23:12So that's the BOSS model and
23:15when we're thinking about BOSS,
23:16you know what are the things
23:18we want to observe,
23:19what are the points that we really
23:21want to make feedback on for learners.
23:24And I think we just wanted to remind you
23:27that we do have resources that can guide
23:29us and how we want to anchor our feedback,
23:32whether that's written or verbal.
23:36And just to remember for those of
23:38us in graduate medical education
23:40that all the primary specialties
23:42now and sub specialties all have
23:44their own milestones as evidenced
23:46here by a few examples.
23:49But also on the undergraduate
23:52medical education side,
23:53Yale in particular has we now have
23:56our own educational program objectives
23:59along with developmental milestones
24:02that we can reference in order to
24:05be more specific in our feedback.
24:08So with that,
24:09we're going to move on to part
24:11three and really get down to
24:14thinking about practicing boss.
24:16And before the next slide,
24:17I just want to reiterate what Matt
24:19had said at the very beginning.
24:22Our goal here is to have you try on BOSS,
24:25see how it feels.
24:27We are going to be providing you with
24:30little snippets to of comments and then
24:34asking you to improve on them using BOSS.
24:38And we're going to have you put your
24:40comments as you're comfortable into the
24:42chat after you've worked for a few minutes.
24:45And we just want to acknowledge that again,
24:47we're meaningless to be a safe,
24:50brave,
24:50low stakes setting where people can,
24:55you know,
24:56put out their,
24:57what they're working on and then we
24:59can give a little bit of specific
25:01feedback to the boss model.
25:03So nothing's going to be perfect
25:06here and we're just going to
25:08move forward in that spirit.
25:10So let's look at
25:11Laura before you move on.
25:13There's a question in the in the chat
25:16that I don't know if you can see it,
25:17I can read it. Go ahead, Dana.
25:21So Julio writes effective time
25:24management versus somebody who's
25:25arriving late sounds different to him.
25:27He was saying if he was arriving
25:30reading that arriving late
25:32might speak of professionalism.
25:33But time management could be
25:38something as takes too long with patience,
25:40takes too long writing notes.
25:41How do we navigate the line between honest
25:44feedback and not minimizing language
25:46due to fear of how folks will? Yeah,
25:49yeah, that's terrific Leo.
25:50Thank you. Yeah, Julio,
25:52I think that's that's really a nice
25:55comment and actually when I was looking
25:57at that I was thinking that to some
26:00extent too it can mean different things.
26:03I think there are ways though to also
26:06make learner centered suggestions
26:09and and create significance.
26:11You know, as far as things like being
26:15respectful to the rest of the team.
26:18I I think at some level it's a
26:21matter of again being creative and
26:23thinking a little bit deeply in a
26:26different direction about what is
26:28how we want to help the learner
26:30understand that their actions
26:32have different ramifications.
26:34Matt, I don't know if you
26:35wanted to add something.
26:36Yeah. I also think in the real world,
26:38I mean, when you do verbal feedback,
26:41often times, you know,
26:42you can tell them what you're observing and
26:44then there can be a dialogue to figure out.
26:46You know, it's often times there's
26:48a differential diagnosis for
26:49problems in the learner, right?
26:51If someone's showing up late first
26:52thing in the morning, I mean,
26:53I think more commonly that speaks
26:55to a professionalism issue.
26:57But if they're showing up
26:58late to afternoon clinic,
26:59I mean it could be that they're you know
27:02something from the morning they're doing
27:04something that's spilling over too long,
27:06their inefficiencies there.
27:07So I do think that before you write
27:10something you know evaluative about
27:12somebody you want to have that
27:14discussion we talked about where
27:15you're saying I'm observing this,
27:17what do you think is going on?
27:18And then it gives them a chance to explain
27:20and self assess what they think the issue is.
27:23And I think work together.
27:24So I agree.
27:25I mean I think you can.
27:27And as we move into this exercise,
27:29we're going to actually encourage
27:31you to take these little props or
27:34stimuli in different directions and
27:35be creative with it because a lot
27:37of times there can be many different
27:39reasons why you're observing a behavior.
27:41That's a great point.
27:43Yeah, I would agree.
27:45I think in that situation,
27:47a verbal feedback opportunity
27:50would would be really helpful
27:53to explore more before
27:55providing that kind of framing.
27:59All right, Matt.
28:00So the first writing exercises,
28:02we're calling it Boss it
28:04just to sort of think about,
28:09you know, being kind of funny about it,
28:11tongue in cheek, but also thinking about
28:14maybe Bruce Springsteen or whoever, right.
28:19But we're giving you a prompt.
28:20So in this case,
28:21one we've seen a lot as well as
28:24documentation needs to be more thorough.
28:26And we're gonna ask you to be
28:29creative and create some details
28:31maybe from your own experience that
28:33might fit just to practice the model.
28:36And so here we have an example where we said,
28:39you know I observed multiple times that
28:42Doctor Zee obtained history of present
28:44illness details at the bedside that
28:46didn't make it into their chart notes
28:49and obviously that's the observation.
28:53These were important to include
28:55as they provided the rationale
28:57to focus our diagnostic work up.
29:00So that gets at the significance
29:02and as we discussed Doctor Z may
29:05benefit from taking written notes
29:07at the bedside for their HPI.
29:09So one suggestion with
29:11some specificity there.
29:13So again,
29:14this is just playing around a little bit
29:17trying to see how this works for us.
29:19So we're gonna go on to the next slide
29:22and put up a couple prompts for you all.
29:25We put up two here,
29:27and we're gonna ask you to each
29:29of you to pick one of them,
29:31whichever 1 resonates a little bit more.
29:33And take about 3 minutes or so to try
29:37to work with this and write a specific
29:41behavior that you might have observed.
29:44Try to bring in some element of
29:47significance to motivate the learner,
29:49and then come up with a plan.
29:52And again, just be creative.
29:58And obviously folks are going to
30:02be working at different speeds,
30:03so we encourage anyone who's comfortable
30:05to just drop their writing, exercise
30:07in right into the chat and hit submit.
30:10And that way as some other people are
30:12finishing up during the three minutes,
30:13we can start to give.
30:15Because one of our goals here today
30:17is like not only to have you right,
30:18but to also have us give you some
30:21feedback about how we can, you know,
30:24get you using the the boss model.
30:25So encourage people to, you know,
30:30hit submit and there's going
30:31to be no judgement here.
30:32This is all for personal growth
30:35and this is a judgement free zone.
31:00Just let us know if you have any
31:02questions along the way as well.
32:43Oh thank you Julio.
32:49You want to take that one, Matt? Sure.
32:52So Julio started out with I observed Dr.
32:54Z checking on their phone
32:56frequently during rounds to perfect
33:00non judgmental observation.
33:04This led to distraction and
33:06orders not being correctly placed.
33:09Doctor Z explained he's
33:11having family problems.
33:12I suggested them to be forthcoming
33:15about these situations so we can
33:17allocate and protect their time to take
33:19care of emergent home situations and
33:21while also protecting patient care.
33:23Yeah, so this is,
33:24this is a really common situation
33:26that when we see people start to
33:29kind of underperform from a baseline
33:31of a higher performance level,
33:33a lot of times it's acute life stressors
33:36that are causing the decompensation.
33:37This is a great usage of the boss model.
33:41Julio, again,
33:42I think that you know for this
33:46particular situation,
33:47you know we probably wouldn't
33:49in like a Med hub evaluation,
33:52you know create written feedback
33:54that's specifically talking
33:56about someone's family problems.
33:58But you definitely you did exactly
34:02what we were looking for and
34:04this is the exact formatting.
34:05So kudos and definitely always
34:08thinking first like burnout depression.
34:11A sick family member.
34:14I'm glad
34:14I called you. Give me a second.
34:17These are always the be asking about you.
34:22All right. I see Michael
34:26Green wrote wrote an example.
34:28Let's see. Julio wrote something.
34:31Yeah, Julio's writing.
34:33I was thinking verbal makes
34:34sense not to include in Meadow.
34:36Yeah, I think that that is a a great,
34:39you know, using this great verbal
34:41feedback. It's perfect. Yeah.
34:44And I think what you're highlighting
34:46is that these principles can overlap
34:48into our verbal feedback as well.
34:50You know, the desire to try to be not
34:55judgmental in the way that we're framing
34:57things and also that specificity for
35:01the suggestions and the significance.
35:03So awesome Michael Green route.
35:07Reviewing their notes and presentations.
35:09Important diagnosis were missing.
35:11This could lead to missed opportunities
35:14for further evaluation or treatment.
35:19Perhaps begin with illness script
35:22to expand diagnosis possibilities.
35:26So I like that reviewing their
35:29notes is is that observation.
35:34I suppose it could be helpful to,
35:37you know in a certain instance
35:40give an example of a specific thing
35:43that was discussed between them,
35:45maybe during the rotation I'm presuming,
35:50but the significance is great.
35:51This could lead to missed opportunities
35:54for further evaluation or treatment.
35:56And then like the suggestion of
35:59beginning with an illness script,
36:01we would just need to make sure the
36:04learner knows what that terminology
36:06is referring to specifically.
36:07Probably.
36:10Thanks, Michael, for putting that in there.
36:15Looks like we have one
36:17from Kirsten. Matt, Yeah,
36:19I can. I can go through Kirsten.
36:20So, Kirsten said, I observe student aid
36:22looking at their phone several times during
36:25psych consultations with new patients.
36:27Though the student was
36:28sometimes in the observer role,
36:29this still may signal to the patient
36:30a lack of attention or empathy
36:32on the part of the team,
36:33and may erode patient trust.
36:35And it can also cause this student to
36:37miss key information Going forward,
36:39it might help student aid to limit
36:42cell phone usage to before or after
36:45sorry limit usage to also usage to
36:47before and after visits in order to
36:50ensure full attention on the patient.
36:52If urgent personal issues are present,
36:53the student is encouraged to
36:56briefly excuse themselves.
36:57This is fantastic, Kirsten.
36:59So you're saying you observe
37:01them looking at the cell phone
37:03during new patient visits.
37:05The significance is this kind of
37:07erosion of trust and lack of empathy.
37:11And it's you know kind of giving a a
37:14place for both like you know appropriate
37:17cell phone usage before and after
37:19the visit and then if something is
37:22truly timely to excuse themselves
37:24and acknowledge that they're they
37:26have a true crisis on their hand.
37:28Dana did you have a comment.
37:30Well, I was just gonna
37:30make sure that people that I
37:32think most people do know,
37:33but some might be thinking with respect
37:36to summative evaluations and if some
37:39of these are on summative clerkship
37:41evaluations or medical evaluations for
37:44residents that's that's not going to.
37:48They might worry about how that
37:50looks to in the Dean's letter or
37:52odd letters of recommendation.
37:54But you know there are these separate
37:57spots in the Med student evaluations
37:59where you can put the corrective
38:01and the reinforcing feedback.
38:02So this is such an important thing to not
38:06lose in documentation for the students.
38:08But and there is a spot for this
38:09that won't go into the Dean's letter.
38:11So people do recognize that
38:14for the students in particular.
38:16Yeah, yeah, yeah. I'll say I mean I love,
38:20I understands example because it
38:23goes into that professionalism realm
38:25and you know frames it in terms
38:27of the effects that this might
38:29have on patients and losing trust.
38:31And I think it's important to see that
38:34it's possible to frame it that way.
38:37And in my own experience on the CCC,
38:42one of my colleagues attended
38:44this workshop last year,
38:45I think and has actually been using
38:48the BOSS model now and most of the
38:51written feedback and it's been
38:53interesting on the CCC side to see
38:55that coming through and really it's
38:58it's been really helpful, I would say.
39:01So
39:02I also want to mirror what Dana said.
39:05The same is true for
39:06graduate medical education.
39:07I mean when we are providing letters
39:10of recommendation for fellowship
39:12or when you know I know for a fact
39:14when Mark Siegel's writing his
39:16letters like they're consciously
39:18omitting the constructive feedback.
39:20They're not gonna they're you're
39:23consciously leaving that out with
39:25creating the the letter recommendation.
39:27So we we, we know enough that we're
39:29going to select the reinforcing
39:31positive feedback and that
39:33really the constructive feedback
39:34is for the learners growth.
39:36Everybody has strengths and weaknesses
39:38and if we're not giving our learners
39:41constructive feedback then we're not
39:43allowing them to become the best
39:44versions of themselves they can,
39:46they can become.
39:49I was just looking ahead.
39:51I wonder, it looks like people are
39:54really getting the hang of the the
39:58BOSS components and doing a really.
40:02Lovely job with that. I wonder, Matt,
40:04if maybe we might move on to the
40:06next exercise just to switch it up.
40:08What do you think?
40:10Yeah, I know that sounds.
40:10I mean, I'm just skimming Andre's
40:13and it looks, it looks really great.
40:16Yeah. He did a similar similarly. Nice job
40:21of really highlighting the significance.
40:27All right. So in this last
40:32written exercise for you,
40:34we're going to kind of give you a
40:35little bit of a narrative here and
40:36then you can take this in whatever
40:38direction you want to take this.
40:39There could be,
40:40there is a differential diagnosis
40:43for what has been observed here and
40:45whatever direction you want to take it in.
40:46That's fine.
40:47So you provided final verbal
40:49feedback to an intern you've worked
40:51with for the last two weeks.
40:52In the fall, strengths were highlighted.
40:55You expressed you expressed some concern,
40:57however, though that they seemed
40:58to be putting in much longer
41:00hours than their peers,
41:01finishing notes late in the evenings,
41:03and making very late in the
41:04day phone calls to families.
41:06They were frequently yawning
41:08during rounds and falling asleep
41:09during midday didactics.
41:11So we want you to do the exact
41:13same thing with this last example,
41:15and you can preface it that you had
41:18a verbal discussion with this and
41:20then you kind of kind of maybe got
41:23different aspects of a differential
41:24diagnosis that came forward what
41:25was going on with the learner.
41:30I could prompt you here,
41:31but I think you guys are are
41:32going to do great creative stuff.
41:34So we'll give you a couple
41:36minutes to come up with a boss
41:39framework for this situation.
41:41This is your opportunity to kind of
41:43link a little more directly to verbal
41:46feedback and make sure it's aligned
42:03and it I should know,
42:04It's totally fine to write an
42:07observation and then say, you know,
42:10in discussion with the learner,
42:11you know it came up that you know,
42:14it seemed like this is what was
42:15at the root of the situation.
44:28Knowing you're all working at
44:30different speeds, we'll just
44:31invite you to There we go.
44:33We've got our first one coming in.
44:37It's an editorial comment. OK, thanks,
44:45Diana. Now it's just like saying,
44:46like, where it all goes bad is that people
44:49miss having the discussion and people jump
44:52to inferences about what they're seeing.
44:55And then they'll write down that they're
44:57disinterested and not and disengaged and
45:00not being respectful and all sorts of
45:02assumptions based on what they're seeing.
45:05But to Matt's point,
45:06there's this differential diagnosis.
45:07So you didn't get to explore it.
45:10So I just love keeping
45:12the the acronym in mind.
45:15Yeah. And I think,
45:18I think that's such an important point
45:21between the verbal and the written.
45:24The, the written is really quite limited,
45:27you know, compared to the verbal
45:29because we can't really explore
45:32the differential and we do need
45:35to be careful to not be making
45:37assumptions and what we're writing.
45:47So it's been a couple of minutes.
45:49We'll invite anyone who wants to go
45:52ahead and forward to us in the chat.
45:58You know, while people type,
46:02I have a comment.
46:03I have two caveats, one about the
46:05previous case and one about this one.
46:08You know, Kristen did a very good
46:10job talking about the iPhone case.
46:13I also pick up the same thing.
46:14But I also wanted to highlight
46:15something that I've come across,
46:16which is that not all iPhone use
46:20is evil and that especially now
46:23that we have Epic Haiku, you know,
46:26we're at risk of double messaging
46:27in a very serious way, right?
46:29We've made it really incredible for you to
46:31have Epic at your fingertips all the time,
46:34but don't use it.
46:35So I think that we need
46:36to be very aware of that,
46:37that there is quote UN quote
46:38good use and also good use,
46:40which I divided as patient related
46:42versus not patient related.
46:43So just a caveat for all of us,
46:44I know that I seen with my
46:47iPhone more than most.
46:49And the second thing in this is wearing my,
46:52perhaps my psychiatry hat.
46:53But I I'm sure that we've all dealt
46:56with residents or students or whatnot
46:58who have a lot of difficulty completing
47:01tasks and written tasks on time.
47:03And the more that we need to feed
47:05the beast of epic, the more so.
47:08And so I'm interested to,
47:10I was interested in,
47:11I'm interested in hearing
47:12what people will have to say.
47:13But something that happens often
47:15enough and yet we rarely talk about is
47:18that some of these folks can't do it.
47:20You know,
47:20there are learning disabilities.
47:22You know,
47:22being a physician doesn't make you
47:23immune to having a learning disability,
47:25a writing disability,
47:26to having ADD to having all sorts
47:29of things that are not only common,
47:31but they're horribly stigmatized.
47:32So you can be putting all of
47:34this feedback and just for us
47:36to be mindful of that,
47:38especially around written,
47:39which is such an important issue.
47:42Anyway,
47:48thanks. I just want to encourage
47:51anyone to put something in the chat,
47:53but we may be getting a message that
47:56not Nora just dropped
47:58something. Thanks, Nora.
48:01Oh, OK. Thanks. I can take that one.
48:04Thanks, Nora. Love how you use
48:08the letters to sort of guide you.
48:10That's what I find my colleague
48:12has been doing as well.
48:16So as previously shared verbally, Dr.
48:18Y has struggled with time management and
48:21maintaining boundaries around clinical work.
48:24This has in turn negatively
48:27impacted their ability to perform
48:29as they appear sleep deprived.
48:32We have discussed how to perform
48:34a root cause analysis. Oops.
48:37Somehow I'm not getting able to move my
48:44I I can take it.
48:44We've discussed how to perform a
48:46root analysis in their workflow to
48:48identify which is becoming consuming
48:50to continue to meet and review,
48:53where to prioritize the work and
48:54when to ask for help from colleagues.
48:56I encourage Dr.
48:57Y to continue in these efforts.
49:00Yeah, I think that's a,
49:02it's AI love the way you've linked it
49:04to the verbal feedback and you know,
49:07really then written it in a
49:10way that I think someone can
49:12carry forward in a helpful way.
49:16The significance is definitely there,
49:20it's turned, can negatively
49:22impact the ability to perform
49:25is is clearly significant.
49:27And then I think you offer some
49:31really nice suggestions as well
49:33about you know prioritizing the
49:35work and when to ask for help.
49:39So it's kind of like a a progress
49:42report almost at the end from what
49:44was said previously. So that's no
49:48except I forgot to continue.
49:52Yeah I love, I love these points
49:55about you know like continuing to
49:57come back and re engage with the
49:59issue and then you know I think
50:00a lot of physicians have a lot of
50:02trouble asking for help. So I think
50:05receptors and and mentors that
50:07are encouraging, you know,
50:09reaching out to others and asking for help.
50:11It's something we need to
50:12do better as a profession.
50:13So I I love that encouragement
50:15from a from a mentor.
50:18Anyone else want to put something
50:21in the chat We'll shift our
50:23focus here in a minute. All
50:30right.
50:33Well, maybe it's appropriate, Matt,
50:35to move on to our next slide.
50:40So we wanted to end by just having
50:43you all before you go take just a
50:46minute here to go ahead and scan and
50:48fill out the short evaluation for us
50:50so so we can know how to improve.
50:53So this is your first opportunity to
50:56practice written feedback in an official way,
51:00Boss them
51:04and but stay tuned, don't leave people
51:06because they have a little take home
51:08slide next and we can then go over
51:10any questions that you might have.
51:12Yeah. And I see John is putting in in
51:16the chat an example for us as well.
51:20So please take a minute to fill out
51:22the user boss for us and hold on
51:26just a second, we'll we'll kind of
51:27close things out with a summary.
51:32And I I want to thank
51:34Yvonne for also contributing
51:39and I like how in Yvonne's he kind of
51:43gave some reinforcement that you know
51:45he's encouraging not to take shortcuts
51:49and also kind of self-care
51:50not to overstretch ourselves.
51:58I think the last piece of on is like
52:01I know I think as long as this was
52:05like associated with some resources,
52:08right, Because I mean time management,
52:10task management, note taking techniques
52:12like this could be an overwhelming thing
52:14to recommend to somebody in isolation.
52:16But as long as you verbally provided
52:18them with some, you know a book,
52:20a website, something like that,
52:22then this would that would
52:23be that would be perfect.
52:26I'm also seeing John and Candela
52:28wrote one for us and what I'm loving
52:32about his is the observation.
52:34He actually lists examples of
52:37behaviors that might suggest
52:39fatigue or time management issues.
52:41And I can. I'm expecting John worked
52:45really hard to list those in a as
52:48non judgmental way as possible.
52:50Trying to make those objective
52:53late day calls, yawning,
52:56sleeping during didactic so,
52:59and giving feedback.
53:00I reported these observations
53:02and they explained having had
53:04a newborn at home
53:07and some other personal challenges.
53:11So I offered my willingness to to
53:14continue to support Doctor X and
53:16working through these changing patterns
53:19as well as to check in with them on
53:21these issues in the next few weeks.
53:24Yeah, so I wasn't I wasn't
53:26sure I was quite ready to.
53:28I felt like they needed to write more.
53:30But I just had some questions about it.
53:33And I guess my question for you is, you know,
53:37it seems like the person is working this out,
53:41but what are your recommendations for I
53:44I think that I wrote this note to make
53:48sure that to monitor the process so that,
53:51you know, the situation does
53:53improve and that this person is,
53:55you know, sort of adjusting to
53:57the changes in their life.
53:58So I wondered your recommendations
54:01for including this as a note?
54:04Or would you just not include it,
54:06'cause it seems like it's working out,
54:08but just monitor it in the future?
54:11What are your recommendations for that?
54:15I I would say that I mean in something
54:17that's so tied to a major life event.
54:19I mean I I think this is probably
54:24more verbal feedback appropriate
54:27that necessarily written feedback
54:30appropriate that my personal opinion.
54:31The other thing I also wanted
54:33to point out is that, you know,
54:34I think no matter who you are,
54:36you're going to come into situations
54:38where you may not have the answers.
54:40And right, like so many of the clinical
54:44preceptors at Yale are not great problem
54:47solvers when it comes to managing
54:49a newborn and a resident schedule.
54:52So I think, you know,
54:54reaching out to program directors GMEI
54:58mean trying to get reached like you,
54:59you can't solve,
55:00we can't solve all of our
55:03president's problems.
55:03And I I think you need to know when
55:05you've reached the limit of your
55:06expertise and when someone's in a
55:08better position to help somebody.
55:10Yeah, As I was writing it,
55:12I was thinking it as more a note to
55:14myself to remember to monitor the
55:17situation rather than something that I
55:19would begin the residents record. Yeah,
55:22I think if you were like a clinic
55:24preceptor and you're going to
55:25see them on a routine basis,
55:26that makes a lot of sense.
55:27You know, if you're if it was
55:29like a one time, you know,
55:30you were like their cardiology
55:31attending on a subspecialty rotation
55:33and might not apply as much.
55:35But I think in the right situation
55:37where you hadn't expected kind
55:39of recurring preceptor role,
55:41that really makes sense. John.
55:43I I agree. I think this is more
55:45suited for verbal probably.
55:47But I think at the same time it
55:50could be shifted into a a point in
55:54an evaluation about the person,
55:56how they were able to receive feedback,
55:59you know, how open they were
56:01to it and how they were able to
56:04act on it in a productive way.
56:06Sort of, yeah. Because that's those
56:08are our competencies too, right?
56:10You know, being able to receive
56:12feedback and implement change.
56:16I was
56:16thinking as sorry, Doctor Morrison,
56:19just a a related question to that.
56:20But I was thinking that maybe another
56:23person may make a a remark about the
56:25person's leaping yawning and not be.
56:28I was wondering if mentioning the
56:30newborn and mentioning the situation
56:31could be a way of us protecting that
56:34learner and explaining in the letter
56:35why that is happening and what measures
56:38they're taking to try to work around it,
56:40'cause maybe if we don't mention it,
56:42nobody else will and someone else
56:44might just say person is sleeping
56:46and then there's no explanation.
56:50Yeah, I think, I think Matt and I were
56:53agreeing both because we supervise,
56:55you know, our trainees as program director,
56:58roles that we would need to
57:00be aware of this for sure.
57:02So it would definitely be
57:04verbal feedback at some level.
57:07And then we would need to sort of coach
57:09them forward to be able to, you know,
57:12reveal what they're comfortable in
57:14the next learning setting so they
57:16can work on it in a productive way.
57:20I'm gonna move,
57:21just have our last slide here.
57:23Thank you all for the comments
57:25and for for jumping into our
57:27hopefully safe and brave space,
57:29putting yourselves out there
57:31and trying out the BOSS model.
57:34I think we wanted to just
57:36remind all of us that you know,
57:38doing feedback whether
57:39it's verbal or written,
57:41but specifically if you're going
57:43to do a written evaluation,
57:45doing it sooner is always better
57:47because you can be more specific and
57:49and target your words to what actually
57:53just happened and and not forget.
57:55So always proximal,
57:57a reminder that feedback can
58:01really fall into sort of different
58:04categories of appreciation,
58:06coaching and evaluation.
58:08So maybe being clear with yourself
58:11kind of which mode you're in,
58:14Please carry forward the BOSS
58:16framework in whatever way it's
58:18going to be helpful to you.
58:19Think about milestones or,
58:22you know,
58:23developmental milestones or
58:25educational program objectives as
58:28ways that we can anchor our feedback.
58:31Be specific, you know,
58:33be as specific as possible always,
58:35but especially in your suggestions
58:39for improvement.
58:40Make sure that the written aligns
58:42with the verbal as best we can,
58:45and always complete.
58:46Consider completing the verbal and
58:49written together certainly at the
58:51end of an experience at the end of a
58:54rotation or at the end of a course.
58:57And Matt is thanking people as well.
59:01So we'll end there and welcome
59:03any additional questions we'll
59:04stay but we're we're already
59:06at the end of our time.
59:08I I just want to add that there's
59:11some upcoming talks listed here.
59:13The upper right QR code is the
59:15Yes series and allows you to
59:17register for future events.
59:19And then if you're interested in any of
59:21the videos or articles we referenced,
59:24the lower right QR code is a
59:26PDF document of both those video
59:29links and the article links.
59:32But thank you so much for coming and
59:35participating and this was really,
59:37this was really great and generative
59:41and last little plug for that,
59:42especially the well both
59:43Andre's session on the posters,
59:45but Katie Gilson's session also
59:47about bias which could dovetail
59:49into additional considerations as
59:50we're as we're writing some of the
59:53feedback and giving feedback sets.
59:55Good pairing.
59:56Thank you so much both of
59:59you and everyone for coming.
01:00:01Happy Friday, everyone.
01:00:02Yeah, Happy New Year.