2/2 YES!: Bias in Assessment
February 02, 2024ID11265
To CiteDCA Citation Guide
- 00:00Let's go. And we're being recorded, Sir.
- 00:04Thank you, Dana. Thank you all for
- 00:07joining us at this Friday noon time.
- 00:11We recognize some some repeat participants,
- 00:15which is heart warming and
- 00:18wonderful and some new faces,
- 00:20which is also lovely. Today,
- 00:22we're presenting as part of the Yes series,
- 00:25Yes with an exclamation point,
- 00:27which is a Yes,
- 00:29the Yale Medical Educator Series.
- 00:32And this is a brainchild of my
- 00:34partner in crime, Dana and I,
- 00:36and this is this is the first
- 00:39year that we have done it.
- 00:41So we are really eager to know what works,
- 00:45what you want more of,
- 00:46what you want less of as we optimize it.
- 00:50The Yes series is loosely
- 00:51divided into 3 topical areas.
- 00:53One is preclinical,
- 00:55one is clinical bedside and the
- 00:58other is more scholarly, academic.
- 01:00But all of them have two things in common.
- 01:03One, they're meant to be very,
- 01:05very practical, very, very practical.
- 01:08And what's the second thing
- 01:09they have in common?
- 01:10I don't know.
- 01:11They're,
- 01:11they're fun and we have wonderful speakers
- 01:13and today is certainly no exception
- 01:16because backed by popular demand.
- 01:18Dana Dunn,
- 01:18please tell us who is our speaker today.
- 01:21Yes, happy to see everybody
- 01:23backed by popular demand.
- 01:25I'm so excited to see on screen my
- 01:29friend and expert educator Katie
- 01:32Gillison who was here after going to
- 01:35Med school at University of Chicago.
- 01:37She was here as a Med PEDs resident
- 01:40and then did a two year general
- 01:44medicine fellowship with Donna
- 01:46Windish during which time she was
- 01:48enrolled in the Master's program
- 01:49at the NOW Center for Medical
- 01:51Education and got her master's and
- 01:53is real expert and shooting Uprising
- 01:56star in the world of assessment.
- 01:59We were happy for her but sad for us
- 02:02when she got scooted away when she
- 02:04got the the position as the inaugural
- 02:07program director for a new Meds PEDs
- 02:10program that has been approved,
- 02:12that she's going to head up as
- 02:14the new program director at Emory.
- 02:16But thanks to Zoom,
- 02:17it's like she never left,
- 02:19except she's got kind of a plant
- 02:20that looks like it thrives at
- 02:22warmer climbs in the background.
- 02:24So she's going to talk to us
- 02:26about bias and assessment.
- 02:27So Katie,
- 02:28take it away.
- 02:29Thanks Dana. It's really exciting
- 02:32to see some familiar faces,
- 02:35some familiar names.
- 02:38I feel like I've never really left
- 02:40Yale because I'm still involved
- 02:41in a number of things at Yale.
- 02:43I still edit the Yale office based medicine.
- 02:46I'm excited to talk with you guys
- 02:48about a a factor of assessment that
- 02:51I think has been getting a lot more
- 02:54attention in recent years and I'd
- 02:56be excited to hear perspectives
- 02:57and thoughts as we go through.
- 03:00I did this session last year,
- 03:01but I'm really open to feedback and
- 03:04you know helping this session to
- 03:06grow and change as as time goes on.
- 03:08Mostly today we're going to be sort
- 03:10of just having some thought experience
- 03:13experiments and just being a little bit
- 03:15more actively conscious of the bias
- 03:17that exists in the assessment space.
- 03:19And my hope is that I can offer some
- 03:22new insights and some new food for
- 03:24thought for those of you who work
- 03:26with learners or who are in charge of
- 03:29assessment in whatever space you work in.
- 03:32So one of the things I'll just alert
- 03:35you to just to be aware of is later
- 03:37in the session I'm going to be asking
- 03:39you to pull up an evaluation that
- 03:41you've done in the recent past.
- 03:43So that could be in Med,
- 03:44hub or whatever interface that
- 03:46you currently use as an educator.
- 03:49But ideally it should be a written
- 03:50assessment of a learner that you
- 03:52work with directly.
- 03:55So let's start with a bit of a Riddle,
- 03:59just to start to start
- 04:01thinking about these themes.
- 04:03So the Riddle is a father and son are in a
- 04:06horrible car crash that kills the father.
- 04:09The son is rushed to the hospital and
- 04:11just as he's about to go under the knife,
- 04:13the surgeon says I can't operate.
- 04:16This boy is my son.
- 04:18Explain So you can pop in the chat.
- 04:22Why? How? How could this be?
- 04:36Joseph has seen this Riddle before.
- 04:39OK, yes, we I think this is a
- 04:41common one that we see come up.
- 04:43When they first did a study
- 04:45on this in college students,
- 04:47only 14% of the students guessed
- 04:50that the surgeon was the mom.
- 04:52Other guesses that came up was
- 04:54that the father who died in the
- 04:56car crash was actually a priest,
- 04:58or maybe it was a same sex couple.
- 05:00There was a a number of things,
- 05:01but very few people statistically
- 05:04speaking actually guessed
- 05:06that the surgeon was a woman,
- 05:09which I think just exemplifies just
- 05:13straightforward bias that often
- 05:15for myself as a woman in medicine
- 05:17have encountered many times.
- 05:19I've often been assumed to be a resident
- 05:21or often been assumed to be a nurse,
- 05:24even after I introduced
- 05:25myself as a physician.
- 05:26So this is something that we see
- 05:29in our everyday practice today.
- 05:31For the session,
- 05:32I would like to just talk about
- 05:35the impact of bias on learners and
- 05:37also systems and how that impacts
- 05:40our learners in the long term.
- 05:42Identify some different sources
- 05:43and types of bias that we often
- 05:46see in the learning space.
- 05:48And I want,
- 05:48I hope and and expect that this
- 05:50session will allow you to reflect
- 05:52a little bit on your own practice
- 05:55of assessing learners and how bias
- 05:57could be entering into that practice.
- 06:02So what is unconscious bias?
- 06:05It is our natural people preferences.
- 06:08As humans,
- 06:09our brains are hard wired to
- 06:12categorize and group things together.
- 06:14It is an unconscious process.
- 06:17It just is an adaptive thing
- 06:18that our brain does because we
- 06:20have to process so many bits of
- 06:22information throughout the day.
- 06:23Sights, sounds, smells,
- 06:25thoughts,
- 06:25all the things that we combine together.
- 06:28Our brain is hard wired to bucket
- 06:31things together into categories
- 06:33and those categories can often
- 06:36introduce things of bias.
- 06:38It is sort of our own predisposition
- 06:40towards certain things that are familiar
- 06:44that that our brain naturally does.
- 06:46And this can impact us a lot and impacts
- 06:49us as clinicians if we practice in
- 06:52taking care of patients and it can
- 06:54impact our perception of learners.
- 06:56It can impact the way we
- 06:57think of their knowledge,
- 06:58their ability,
- 07:00their readiness for independent practice.
- 07:05I think this can be a hard thing for
- 07:07to embrace because it it forces us to
- 07:11actually recognize that there are things
- 07:14that we naturally do that are against
- 07:16what we think is perhaps morally right.
- 07:19And I think most of us have the assumption
- 07:21of ourselves that I'm fair, I'm unbiased.
- 07:24I really do try to treat all my trainees
- 07:27the same way, if at all possible.
- 07:31But part of that is that our brain sort of
- 07:35protects itself from the reality of bias.
- 07:38You know, what we think societally and what
- 07:40we think personally is morally correct.
- 07:43Our brain kind of refuses to recognize
- 07:46that belief that conflicts with what we
- 07:49think is good or right and society also
- 07:52what society thinks is good or right.
- 07:56But like I mentioned before,
- 07:57biases are just evolutionary
- 08:00adaptive behaviors.
- 08:01We make many, many conscious and
- 08:04unconscious decisions daily.
- 08:05I think for those of us who are
- 08:07clinicians or those those of us
- 08:08working in the learning space,
- 08:10we are constantly making decisions
- 08:12under a lot of time pressures,
- 08:14under a lot of informational pressures,
- 08:17cognitive burden.
- 08:18And so our brain can only functionally
- 08:21deal with so much information at one time.
- 08:24And so we use these categories,
- 08:26these systems of thinking to allow
- 08:29us to make cognitive leaps.
- 08:31And in those cognitive leaps,
- 08:32bias can often enter into those decisions.
- 08:35So for those of you who are interested in
- 08:37or have learned about clinical reasoning,
- 08:40one of the things we talk a lot about
- 08:42in clinical reasoning is system one
- 08:44and system two thought processes.
- 08:46So system one is our fast intuitive
- 08:50emotional leaps of cognitive
- 08:53thought where a system conscious and
- 08:57effortful thinking or think through
- 08:59things step by step.
- 09:01And those system one thought
- 09:03processes can be really where a
- 09:04lot of biases enter into clinical
- 09:06reasoning and the same can be true
- 09:08in the assessment space as well.
- 09:12There are a lot of different ways that
- 09:14bias can manifest in learning spaces.
- 09:16So affinity bias is one that comes up a lot.
- 09:20I as a, you know, Medpedes program
- 09:22director like if I know I'm going to
- 09:24have a student who's interested in
- 09:26Medpedes working with me on service,
- 09:28I'm immediately going to have
- 09:29an affinity bias towards them
- 09:31because I feel connected to them.
- 09:32I feel like I want them to join into my
- 09:35tribe, as my program director calls it,
- 09:38perception bias.
- 09:40So stereotyping individuals,
- 09:43you know categorizing them in a way
- 09:45that doesn't necessarily align with
- 09:48themselves as an individual person.
- 09:51Halo effects is something that
- 09:52we see a lot in assessment.
- 09:54So just an underlying belief that
- 09:57somebody's has good traits and
- 09:59therefore I like them and their
- 10:01likability is going to affect the
- 10:02way that I assess them rather than
- 10:04objectively thinking about the
- 10:06behaviors that I saw in the workspace.
- 10:09Or confirmation bias,
- 10:10which I think for those of us,
- 10:12our clinicians see a lot where we kind
- 10:14of want to confirm our pre-existing
- 10:17notions about things and aren't
- 10:19really able to see outside of them.
- 10:22These are just some of the biases that
- 10:25can enter into the assessment space.
- 10:27I'll talk about a couple different
- 10:29examples to sort of cement things
- 10:30down in just a second.
- 10:34The other things to kind of
- 10:36think about in the bias spaces,
- 10:38ways that they manifest systemically.
- 10:42So learning environments have various cues.
- 10:45I think historically in medicine we
- 10:48have recruited certain subpopulations
- 10:50of people into our academic spaces and
- 10:53have become accustomed and communicating
- 10:55in certain types of way or teaching
- 10:57in certain types of ways and that
- 10:59might not be applicable to all types
- 11:02of backgrounds that we encounter.
- 11:03Now that we are a little bit more diverse
- 11:06in our recruitment and thoughtful about
- 11:09our recruitment of individuals into
- 11:11medicine and other healthcare specialties,
- 11:13we can also bake assessment the bias
- 11:17into our actual assessments which
- 11:19I'll show some examples of from
- 11:21the literature in just a second.
- 11:23And then when we do assessments,
- 11:25especially when we're doing
- 11:27competency based assessments,
- 11:28so assessing you know someone's
- 11:30ability to take care of patients
- 11:32or to make complex decisions,
- 11:34we often bring ourselves to the table
- 11:37when we make those assessments as
- 11:39as supervisors or as assessors in
- 11:42clinical workspaces in particular,
- 11:44we bring our own preconceptions
- 11:46of what's right,
- 11:47what's wrong and sometimes
- 11:48impose that on our learners.
- 11:50And that's something that increasingly
- 11:52I've gotten interested in as somebody
- 11:54who does research on entrustable
- 11:56professional activities where trust
- 11:58is the measure of assessment.
- 11:59So how much bias enters into that
- 12:03decision of trusting someone?
- 12:06So let's let's talk about some
- 12:08specific examples of bias in
- 12:10assessment in medical education.
- 12:12I think these are really highlights.
- 12:14There's a ton of literature out there
- 12:16on bias that is very interesting
- 12:18and helps to reveal some of the
- 12:20different ways that things manifest.
- 12:22But I picked out a couple studies
- 12:24that I thought revealed different
- 12:26aspects of bias that I thought
- 12:29were interesting and enlightening.
- 12:31So here's an example of how Halo
- 12:34effect can really change the way
- 12:37that we think about learners.
- 12:39So this is a really well done study.
- 12:40It was published in academic
- 12:42medicine just last year.
- 12:44It was done in family medicine and
- 12:47the investigators created 2 videos.
- 12:49One was with a female standardized
- 12:53resident and one was with a
- 12:55male standardized resident.
- 12:57The two videos were exactly the same.
- 12:59They follow the exact same script
- 13:02and for each of those two videos
- 13:05they labeled each one as either
- 13:08above or below average for male and
- 13:10above and below average for female.
- 13:12They were the exact same video.
- 13:15They were just labeled as above
- 13:17average or below average.
- 13:18So they had faculty look at these videos.
- 13:21They said, aha,
- 13:23this is a above average trainee.
- 13:25We want you to assess them on this form.
- 13:29So they had 70 faculty.
- 13:30Observers were randomized to one of
- 13:33the four videos, Above average male,
- 13:35below average male, above average female,
- 13:38below average female.
- 13:39And they analyze the results.
- 13:41And what they found was that just
- 13:44by putting the words below average
- 13:47on any of the videos,
- 13:49they were rated significantly
- 13:50lower compared to above average,
- 13:52regardless of gender.
- 13:54So what that told
- 13:56what the investigators concluded
- 13:58was that just one word above or
- 14:01below was associated with systemic
- 14:03differences in assessment score.
- 14:05And you might ask,
- 14:06why is this applicable to my
- 14:08learners and my learning environment?
- 14:10I think this is a really
- 14:12good lesson on labels.
- 14:14So using labels on learners,
- 14:16like a struggling learner or somebody
- 14:19who's stellar can actually impact
- 14:21the way that you feel about them
- 14:23irregardless of other factors.
- 14:25So labels become a really
- 14:28powerful factor in assessment.
- 14:30So something to really think about
- 14:31when you're working with learners.
- 14:35Let's talk about gender.
- 14:37There's actually a lot of studies on
- 14:39the effect of gender on assessment.
- 14:41This was a really interesting
- 14:43one and robust in the numbers of
- 14:45individuals that they analyzed.
- 14:46So they looked at emergency medicine
- 14:50residents across five programs and they
- 14:53were looking at narrative comments
- 14:54and they were interested in the
- 14:56differences in narrative comments,
- 14:57comments based on gender.
- 14:59So this was like 10,000 comments
- 15:02that they were looking at.
- 15:04When they analyzed these based
- 15:06on their narrative responses,
- 15:08they found that men were more likely
- 15:11to receive specific feedback feedback
- 15:13based on their competence and to be
- 15:16rated above expected performance.
- 15:18And this was irregardless of
- 15:20faculty gender of the assessor.
- 15:22For women,
- 15:23they were more likely to get comments
- 15:26about low skill levels and get get
- 15:29more comments about their quote UN
- 15:32quote confidence versus men who often
- 15:34when they were at a low skill level,
- 15:37they received comments that were
- 15:39associated with some actionable items
- 15:42that they could actually work on.
- 15:45Lastly,
- 15:45the assessor gender actually
- 15:47had an impact as well,
- 15:49so women faculty assessors were more likely
- 15:53to rate residents as performing below level.
- 15:57So women were stricter graders than men.
- 16:02Here's a schematic of their findings from
- 16:04this JAMA article that was published in 2022.
- 16:07So you can see some of the
- 16:09things that favored men were
- 16:11comments about professionalism,
- 16:13critical thinking, trustworthiness,
- 16:15whereas women were more associated with
- 16:18comments about confidence and care,
- 16:20planning for example,
- 16:21or adaptability, excuse me.
- 16:24So there were really substantial
- 16:26differences in the assessment language that
- 16:29was used in describing these learners,
- 16:32and that can really have an impact
- 16:34on training. It can have impact.
- 16:36The authors commented on things like
- 16:39eligibility for becoming a chief
- 16:42resident and other opportunities
- 16:44that came up.
- 16:45This is a different study that was
- 16:47done in internal medicine for medical
- 16:50students that was published in Jgym.
- 16:52They used a process called natural
- 16:55language processing where they looked
- 16:57at narrative comments together and
- 16:59they looked on two different axes,
- 17:02women and men and honors grades
- 17:05and past grades.
- 17:06And they found that women who received
- 17:09honors grades received words like wonderful,
- 17:14empathetic, delightful,
- 17:15whereas men got words more like relevant,
- 17:19active, certain.
- 17:21So those types of things came up.
- 17:24So 62% of words that differ by gender
- 17:28represented personal attributes and
- 17:31then they also looked at URM students
- 17:33versus non URM and the honors and
- 17:36pass schematic as well and personal
- 17:39attribute words were more common for URM.
- 17:42So things like lovely, kind,
- 17:44those types of things whereas
- 17:46competency based language, able,
- 17:48smart,
- 17:49superb were more common with non
- 17:53URM students.
- 17:54So language differs quite a lot between
- 17:57different categories of individual.
- 18:00So that's where bias gets introduced
- 18:05and the last thing I will talk about
- 18:07is how bias can be folded into systemic
- 18:11or programmatic aspects that can really
- 18:14impact not only learners individually,
- 18:17but it can impact their careers.
- 18:19So this is a really fascinating
- 18:22perspective piece that was put
- 18:24out by UCSF in 2018 and they were
- 18:27really curious about why they were
- 18:29seeing their URM students were not
- 18:32achieving awards at the same rate
- 18:34as some of their non URM students.
- 18:37So they took it upon themselves to
- 18:39do almost a quality improvement
- 18:40study to understand a little bit
- 18:43more about why that was the case,
- 18:45Why did they observe these differences?
- 18:48So they looked at 4 cohorts of students
- 18:51at their institution and they found
- 18:54that if they looked over the course of
- 18:57all of the training from pre clerkship
- 18:59all the way to the end of training
- 19:02that there were differences in grading
- 19:04that consistently favored non URM students.
- 19:07And these differences were
- 19:09absolutely been minuscule.
- 19:11We're talking about on an average,
- 19:13you know on a one to five point scale an
- 19:16average score difference of 4.7 for the
- 19:20non URM and 4.4 for the URM students,
- 19:23so .3 difference.
- 19:24I think any of us who are
- 19:26in an educational space,
- 19:28who fill out these types of forms would
- 19:30agree that a difference of .3 on a one
- 19:33to five scale means absolutely nothing
- 19:35for the ability for that individual
- 19:37to care for patients and their ability
- 19:40to become a physician or a clinician.
- 19:42But despite the size being
- 19:45small of those differences,
- 19:47the actual effect of those
- 19:49differences were huge.
- 19:51So because of grading policies at the school,
- 19:54having a quota of certain number
- 19:56of honors grades at that school,
- 19:57so you know 30% of the internal medicine
- 20:01clerkship students could get honors.
- 20:03That meant that URM students
- 20:05received about as half as many
- 20:07honors grades as non URM students.
- 20:10And URM students,
- 20:12therefore cascading down,
- 20:13were three times less likely to
- 20:16be selected for things like AOA,
- 20:18which is a big deal if you're thinking
- 20:20about applying into a competitive specialty
- 20:23like orthopedics or that type of thing.
- 20:26These types of wards can have really
- 20:29big impacts on your ability to match
- 20:31into your intended field or to
- 20:33match into competitive residencies.
- 20:36So they called this their
- 20:37amplification cascades.
- 20:38So these tiny little differences
- 20:41that occurred over the course of
- 20:43their training resulted in very
- 20:46big differences in these individual
- 20:49selection selected for awards.
- 20:51They decided to take a really honest
- 20:54look at themselves and really
- 20:56pinpoint where they were seeing
- 20:59bias being introduced at various
- 21:01parts of the grading cascade.
- 21:03So they looked at the individual students.
- 21:06They looked at the cultural and
- 21:08structural aspects of the school.
- 21:10And I think in particular for
- 21:12many of you attending here,
- 21:13the faculty and resident reader factors
- 21:16were they had faculty taken on this,
- 21:19look at the way that they were doing
- 21:22assessments in the actual real workspaces
- 21:24and also the grading policies that
- 21:27were present. So it was a really
- 21:29fascinating paper in that they
- 21:31were able to make some definite
- 21:33changes in their grading policies
- 21:36in order to be less biased overall.
- 21:40So I wanted to take a pause here.
- 21:43Does anyone have any clarifying
- 21:46questions or there's anything that
- 21:48came up in the chat that would be
- 21:50important to talk about before we
- 21:51take a little small group pause?
- 21:58Cool. Well, I'll do a quick poll.
- 22:03Oh, I don't see the poll.
- 22:06Here we go. So I'm going.
- 22:09Can you guys see the poll now?
- 22:12Yes. Oh, here
- 22:15you can, because I can't see it.
- 22:18Let's try relaunch questions.
- 22:21There we go. Do you see it now?
- 22:25All right. So the first question is,
- 22:27have you seen bias manifest in the assessment
- 22:29of learners where you work or teach?
- 22:50OK Can you see the results of the fall?
- 22:54I'm just. OK, perfect.
- 22:56So it looks like about 60% have said yes,
- 23:01definitely 81 is unsure in which I can
- 23:03understand where that's coming from,
- 23:05where I'm not sure if what
- 23:06I saw was actually bias.
- 23:08I don't want to label it as such,
- 23:10but maybe and then it's about 6% said no.
- 23:15Now I'm going to give you an
- 23:18opportunity to share if you have ever
- 23:21seen assessment bias manifest in
- 23:23your clinical learning environment,
- 23:26how have you seen that occur?
- 23:28So let's see if I can,
- 23:31Katie, can I ask a question?
- 23:33Yeah. So first of all, hi,
- 23:35nice to see you. Hey, so you're,
- 23:39you're familiar with this and I'm,
- 23:41I'm sure that everyone here has
- 23:43encountered this or thought about this.
- 23:46Would it be considered a form of
- 23:47sort of systemic or institutional
- 23:49bias if we consistently
- 23:53grade inflate or I'm trying to
- 23:54think of a more political way to
- 23:56say that if we if we give glowing
- 23:58sort of assessments across the board
- 24:00without really giving sort of,
- 24:01you know what I mean?
- 24:03Yeah, If we just if we're giving
- 24:05great assessments to the overwhelming
- 24:07majority of learners. Yeah,
- 24:09I like, that's such a complex question,
- 24:12Bennett, and I love that you ask it,
- 24:13'cause I think that's a national
- 24:16debate that's going on. You know,
- 24:18like what is the purpose of assessment?
- 24:20What is, what are we hoping to
- 24:23accomplish by use of assessment?
- 24:25And for me, like in the post
- 24:27purest sense of the, the answer,
- 24:29it's to get the right information
- 24:30to the right people about
- 24:32how this individual's doing.
- 24:33That includes the individual themselves.
- 24:35That includes the person leading the course,
- 24:38That includes the person
- 24:40overseeing their education.
- 24:42And I think there's a lot of
- 24:44different pressures that come
- 24:46into the conversation where we're
- 24:48talking about grades in particular.
- 24:50Like for example,
- 24:51the medical school has a,
- 24:54you know,
- 24:55like a invested desire to make
- 24:57sure that that individual gets
- 24:59to the next step of their career
- 25:01in the best fashion possible.
- 25:05And so and that individual wants
- 25:08to accomplish the things that they
- 25:10want to do in their career as well.
- 25:13So I think it's a form of like
- 25:16Halo bias that I think nationally
- 25:18a lot of clerkship directors are
- 25:20discussing and certainly a lot of
- 25:23residency directors are discussing
- 25:24about how useful is this information
- 25:27actually when we're looking at,
- 25:28you know, thousands of applications.
- 25:31So I'll leave it at that.
- 25:32I don't know if you have any
- 25:34thoughts about that.
- 25:37Oh, I'll, I'll, I'll happily hear
- 25:39other people's questions and
- 25:40suggestions and contributions.
- 25:42It's, it's complex, as you say, Katie.
- 25:44Gary has a question.
- 25:45Yeah. Hey, Gary.
- 25:48Hey Katie. I'm wondering if you know most,
- 25:51most of those those words were complimentary.
- 25:53I'm wondering if people have looked
- 25:56at at critical language as well
- 25:58and and how that impacts bias.
- 26:02Yeah, I mean the the gender study and
- 26:05emergency medicine that I mentioned before
- 26:07did look at critical language and I just,
- 26:10I glossed over it a little
- 26:12bit which they, for example,
- 26:14if a a woman was underperforming,
- 26:17they would talk more about confidence
- 26:19and they wouldn't use really necessarily
- 26:22specific language on what could be improved.
- 26:25Whereas male residents were more
- 26:27likely to receive like very specific
- 26:29competency based language on what
- 26:32specifically they could improve.
- 26:34There's a little bit of variation
- 26:37in especially around gender and
- 26:40language in the literature.
- 26:41So some studies will show no difference
- 26:44and it sort of depends on how the
- 26:46investigators are looking at the information.
- 26:48I just actually reviewed an article
- 26:51from the A/C GME that got accepted
- 26:53that they looked at standardized
- 26:55videos and there was a huge difference
- 26:57in assessed scores and competencies.
- 26:59So there's a little bit of variation
- 27:01in the literature out there,
- 27:03but there is some negative like
- 27:05negative language is associated
- 27:07with gender for example.
- 27:10Does that make sense
- 27:11and is that is that largely presenting
- 27:13as as less specific versus more specific
- 27:18And that I I I think my question is
- 27:21ultimately is there almost a a rebound
- 27:24effect of some of the bias training
- 27:26that we're we're almost being overly
- 27:29cautious and not you know and not
- 27:31as productive as as we want to be
- 27:33with some of these learners. I think
- 27:35that's a really interesting question.
- 27:38I don't, I don't know if
- 27:40anyone has actually asked that
- 27:42question in the literature.
- 27:43I will say the propensity of
- 27:45information that I've seen has
- 27:47been that women are more likely,
- 27:49for example, to receive feedback
- 27:52on personal characteristics
- 27:55rather than actual actions.
- 27:59Like you were kind
- 28:05that type of thing or you
- 28:08weren't confident enough.
- 28:09And as somebody who's been a
- 28:10recipient of that type of feedback,
- 28:12it's extremely frustrating
- 28:13because what does that mean?
- 28:16Like, objectively speaking,
- 28:20any other questions?
- 28:23Now, Katie, before we get the
- 28:24response back from question two,
- 28:25there was some in the chat that said that
- 28:27they couldn't see the question anymore.
- 28:30Let me try. I did.
- 28:32I did put it in the chat.
- 28:33One person had responded,
- 28:36which we can, I can, I can relate.
- 28:39Unless you want to try to
- 28:40launch it again or just.
- 28:42Yeah, I James, would you be willing to
- 28:45share, if you're if you're open to it,
- 28:47about the experience that you had.
- 28:51Yeah. It's interesting.
- 28:52It's it's going it's going it's going
- 28:54back to my own training, which is
- 28:59sorry. It's going
- 29:01back to my own training
- 29:03which is now 35 years ago.
- 29:06I I I felt perhaps it was more
- 29:08of a personality trait that they
- 29:10were looking for as opposed to clinical
- 29:13skills or or or or compassion traits. I
- 29:15don't know if anyone else had that
- 29:17experience during their own training years.
- 29:21Sounds like that first kind of bias Katie,
- 29:23that you were identifying.
- 29:24I can't remember its name or
- 29:26they're looking for somebody who
- 29:27looks like that with respect. Yeah,
- 29:31it's it's, I think it's really
- 29:33fascinating the way that we
- 29:36group in medicine in particular,
- 29:38you know like is was I attracted
- 29:41to Med PEDs because I, you know,
- 29:44like there was a affinity that
- 29:46was generated around it or
- 29:48is it because I was genuinely
- 29:50interested in the the field itself?
- 29:52I like to think it's the latter,
- 29:54but I do think that there's some very
- 29:57interesting cultural things that
- 29:58happen in medicine that by assess
- 30:00one way or the other or might even,
- 30:03for example, if you were interested
- 30:05in surgery and you know,
- 30:07got this implicit discouragement
- 30:09to enter into surgery,
- 30:11how we're biasing our students
- 30:13away or towards certain things?
- 30:14Well, it may have been a projection
- 30:16of the fact that I was
- 30:17more interested in internal medicine
- 30:18and eventually pursued internal
- 30:20medicine. So maybe that was
- 30:21reflected in my own behavior.
- 30:23So that's fair. Totally.
- 30:26All right, I'll see what we have for
- 30:30our responses here, which I cannot see.
- 30:38Huh. I'm sorry guys. I can't see the
- 30:41written responses on the question too.
- 30:43I know some people entered in their thoughts
- 30:47it might it shut down before I back to me.
- 30:50Sorry, it it shut down before I could save.
- 30:55I'm sorry about that.
- 30:56Would anybody else be willing to
- 30:57share an experience that they had?
- 31:11I just can I just add one thing.
- 31:13I mean the the Halo effect does strike me.
- 31:17Just having that evidence is really
- 31:19useful because I always remember as when
- 31:22I was a clerkship director I used to
- 31:24question why we wouldn't kind of feed
- 31:26forward information about the trainees,
- 31:29the students as they were going
- 31:30from 1 clerkship to the another so
- 31:32that we'd be able to support them.
- 31:34And the administration was concerned
- 31:36that that would kind of poison the
- 31:39well and that you want to have
- 31:40somebody have this fresh start.
- 31:41And I I it's really nice to have the
- 31:44evidence to support the fact that that
- 31:46was potentially a really good idea
- 31:48in a high stakes grading environment.
- 31:51I think in a pass fail environment
- 31:52we would hope that we could just
- 31:54continue to support them.
- 31:55But I could see the tension if it was a
- 31:57higher stakes and somebody could be labeled.
- 31:59Yeah,
- 32:00I think that's such a a double edged
- 32:03sword conversation that I think
- 32:04all educators need to have about
- 32:08our. Often times our education systems
- 32:10are chopped up such that when a student
- 32:13enters into a new educational space,
- 32:15it's almost like they're starting over.
- 32:17Which I think could be a really great thing.
- 32:19And that we're not biased towards or
- 32:21against anything with this new learner.
- 32:23But it also could be a detrimental
- 32:25thing in that a learner might not
- 32:28have the ongoing support they need
- 32:30to continue to build their skills
- 32:32with the support of a experienced
- 32:34educator or experienced clinician or
- 32:36whatever field they're working in.
- 32:38So it can often be really challenging to
- 32:41think about these types of things because
- 32:43we do want to be fair to our learners,
- 32:45but we also want to make sure they're
- 32:47supported in the reaching the ultimate goal,
- 32:48which is to practice medicine or to become
- 32:52a medical librarian or to become a,
- 32:54you know, a physician's associate
- 32:57or whatever their goal is.
- 33:00So the question is how?
- 33:01How can we move forward?
- 33:03I think this brought up a lot of
- 33:05really controversial and also just
- 33:07challenging things that we wrestle
- 33:09with that I don't think are just going
- 33:11to disappear with a lot of training.
- 33:13They're they're going to be ongoing
- 33:15things that we have to be attentive to.
- 33:17So number one is to just be aware
- 33:20that bias is part of the lexicon.
- 33:22Bias is part of our daily lives.
- 33:25You know,
- 33:25we used to really see bias as like bad,
- 33:28that it's a aberrant, intentional,
- 33:31conscious thing that we do.
- 33:34But in fact,
- 33:35we understand bias now as normative.
- 33:37It's unconscious,
- 33:38it's often simmering under the surface,
- 33:41and it's largely unintentional.
- 33:42I don't any of us go into encounters
- 33:45with our learners intending to be
- 33:48biased towards or against them.
- 33:50We want to be fair.
- 33:53And so we just have to recognize
- 33:55that this is part of our hard wiring
- 33:58and sometimes even though we go into
- 34:00interactions with the best of intentions,
- 34:02depending on what might be going on,
- 34:05you know whether it be a high stakes
- 34:08situation or past patterns or those
- 34:10types of things that can result
- 34:13in actions that we don't intend.
- 34:15And just as another piece of evidence
- 34:17of that hard wiring in our brain,
- 34:19I think all of us when we're looking
- 34:21at Block A and Block B we can,
- 34:23we are looking at it.
- 34:24They are definitely different colors,
- 34:26but when we actually line up
- 34:29that they're the exact same color
- 34:31using these columns here.
- 34:33And Despite that,
- 34:34I still look at Block A and Block
- 34:36B and I see two different colors.
- 34:38It's kind of like that phenomenon
- 34:40with the dress online,
- 34:41that meme with the blue dress
- 34:43and the gold dress.
- 34:44No matter how many times
- 34:45I looked at that dress,
- 34:46I could never see gold and yet I
- 34:48knew people who saw it as as blue
- 34:51or as as gold themselves.
- 34:53So it's it can be really hard to
- 34:55unweigh themselves of that hard
- 34:57wiring of our brain.
- 34:58So first of all, one of the things
- 35:00that can be helpful is just being
- 35:02conscious of the things that
- 35:04make us more prone towards bias.
- 35:05So stress, time constraints,
- 35:08doing multiple tasks at the
- 35:11same time need foreclosure.
- 35:13So I don't know about you,
- 35:15but sometimes I get a little bit
- 35:17behind on my evaluations and I
- 35:19need to get them done to meet that
- 35:21deadline for that clerkship director.
- 35:24So I need to do that evaluation as
- 35:26quickly as possible to get that closure fear.
- 35:28And I think Ben brought up that good,
- 35:32that good point of you know,
- 35:34like what is the impact of this
- 35:36learner if I were to evaluate them
- 35:39based on this rubric fatigue.
- 35:41So these are all things that all
- 35:43of us no matter what our workspace
- 35:46is encounter all the time.
- 35:47So these are things that make us
- 35:49a little bit more prone to bias,
- 35:50more prone to make those cognitive leaps.
- 35:55Can we control this?
- 35:56Well, it can't be totally
- 35:58trained out of our brains.
- 35:59Just as we looked at
- 36:00those block A and Block B,
- 36:02it can be hard to escape that hard wiring.
- 36:05But we can try to be very
- 36:07intentional about our implicit
- 36:09attitudes and try our best to curb
- 36:12their effects on our assessments.
- 36:14One of my very good friends was just
- 36:17telling me about a mental exercise she
- 36:19does in this respect about patients,
- 36:21where every single day she and her
- 36:24residents would say something kind
- 36:26and positive about every patient
- 36:28on their service just to create an
- 36:31implicit attitude of positivity
- 36:33towards the patients Being objective.
- 36:37Honestly, self reflecting,
- 36:39recognizing that you will have biases
- 36:41and trying to pinpoint where those
- 36:43might be coming from and what kind
- 36:45of implicit assumptions that you come
- 36:47to the table with can be really helpful.
- 36:49I would say external feedback
- 36:52can also be extremely helpful.
- 36:54One of the exercises that I have
- 36:56found personally helpful is
- 36:58for letters of recommendation,
- 37:00having someone else read them to
- 37:02make sure that you're using language
- 37:04that actually encapsulates the
- 37:06ability of the individual that you
- 37:09are writing that that letter For.
- 37:11Some things that you can do to help
- 37:14to reduce bias and assessment.
- 37:15First and foremost,
- 37:17attending sessions like this.
- 37:19Starting to recognize where you're
- 37:21making inferences about learners
- 37:23assumptions or categorizing
- 37:25learners into certain buckets.
- 37:27Starting to move your language into
- 37:29more behaviorally based language.
- 37:30I understand there was a session recently
- 37:34for YES where there was some talk
- 37:37about creating a good written assessments.
- 37:39So avoiding personality focused
- 37:41language like somebody was kind and
- 37:44instead focusing more on what specific
- 37:47actions that they took to made them
- 37:50seem kind or made them seem in that way.
- 37:53And then also focusing on how you're
- 37:57writing words using instruments and
- 37:59guides and actually reading through
- 38:02what language you're expecting to rate
- 38:04or grade that person on can be very helpful.
- 38:07So using those actual criteria
- 38:09about what you physically observed
- 38:11in the workspace or in the learning
- 38:14space can be very helpful.
- 38:16Using checklists can be very helpful too,
- 38:18to objectify things a little bit more,
- 38:22and then also for those of you who are
- 38:24course directors or clerkship directors More.
- 38:27Observations from different faculty
- 38:28of different backgrounds can also
- 38:30be helpful in mitigating biases
- 38:32on a single learner
- 38:36individually. Just recognize that
- 38:39you have bias and give feedback.
- 38:42You can review your assessments individually,
- 38:44so taking a step back from your assessments
- 38:47and just reading them specifically
- 38:49only to try to detect bias in language.
- 38:52Have a trusted colleague read them,
- 38:54or I'll introduce a tool on gender
- 38:56bias in just a second that you can
- 38:58use to see to start to detect some of
- 39:01the language that you might be using.
- 39:04Practice constructive uncertainty.
- 39:06So really try to pause as you're
- 39:10making assessments,
- 39:11observe yourself in action and
- 39:13more thoughtfully consider the way
- 39:15that you're seeing that situation.
- 39:17And I'll give you a little acronym
- 39:20in the next slide to kind of think
- 39:23about embrace the awkwardness
- 39:25and discomfort of this.
- 39:27Again, I think a lot of us feel uncomfortable
- 39:30recognizing that we do have biases.
- 39:32And so embracing that and recognizing
- 39:35that that is part of our hard
- 39:39wiring as humans and by recognizing
- 39:41where we have implicit assumptions
- 39:43and addressing those head on,
- 39:45that can be very helpful and engage
- 39:48with those who are different,
- 39:49hear their perspectives,
- 39:51be interested in their viewpoints
- 39:54and the way that they see the world.
- 39:57I found this acronym to be really helpful.
- 39:59It's called pause.
- 40:00And this could be a way that you as you
- 40:03fill out those mud hub evaluations or
- 40:05filling out an intrusible professional
- 40:07activity or any assessment that you
- 40:10do in your learning space could be a
- 40:12great way to think about your learner
- 40:14as you're doing that assessment.
- 40:16So 1st,
- 40:17pay attention to what you're assessing.
- 40:19You know,
- 40:19what is the objective activity or skill
- 40:22set that I'm trying to look at here?
- 40:25So what are the goals for
- 40:28this particular learner?
- 40:29Acknowledge that you come to
- 40:30the table with your own biases
- 40:33and reactions and judgments,
- 40:34and really try to think about
- 40:36why you're seeing it that way.
- 40:38And understand that maybe somebody might
- 40:40see it a different way than you would
- 40:42try to be as objective as possible,
- 40:45really be actionable in the way that
- 40:47you provide that feedback to that
- 40:49learner and execute the assessment
- 40:51in a way that minimizes bias.
- 40:55So I wanted to share with you an
- 40:58assessment that I did on on a learner.
- 41:00This is anonymized and this is person
- 41:02that graduated many years ago.
- 41:03So I wanted to see if you guys could see
- 41:08if there might be any biased language
- 41:10in this individual's assessment.
- 41:12Would anyone be willing to volunteer
- 41:14to read this out loud to the group
- 41:20I don't mind reading.
- 41:21All right. Thank you so much.
- 41:25I very much enjoyed working with
- 41:27M They worked hard to ensure
- 41:28their patients were attended
- 41:29to and was a great team player.
- 41:31I appreciate the effort they went
- 41:33to and caring for their patients,
- 41:34calling consults, pharmacies and
- 41:36doing other intern level tasks.
- 41:38They were very open to feedback
- 41:39and actively worked to improve
- 41:41their performance every day.
- 41:42It was clear they heard and
- 41:45incorporated feedback consistently.
- 41:46They also worked very hard on
- 41:47improving their cardiac exam and by
- 41:49the end of our two weeks together
- 41:51was better able to identify and
- 41:52describe the murmurs They heard.
- 41:54M could work on developing their
- 41:56differential diagnosis of common
- 41:57inpatient internal medicine complaints,
- 41:59GI bleed, tachycardia, angina.
- 42:01Often their differentials were based on
- 42:03either previously established diagnosis,
- 42:05even in the setting of new data,
- 42:07or a limited list of possible alternatives.
- 42:09Some of the assigned cases
- 42:11simulation online courses provided
- 42:12by the clerkship would be helpful.
- 42:16All right. What do you guys think?
- 42:19Do you see any language in here?
- 42:21And this is not a loaded question at all.
- 42:24Is this a unbiased?
- 42:26Do you see some language in here that
- 42:29could potentially point to certain
- 42:31gender or other affinity biases?
- 42:41I think you expand on the
- 42:43characterization as a great team player.
- 42:46But in the next sentence,
- 42:47but maybe maybe also there were
- 42:50other ideas or emotions you're
- 42:52experiencing when you typed that, that,
- 42:57that, that biased that characterization.
- 43:00What what do you what are
- 43:01you thinking, Ben? Oh,
- 43:04like maybe you liked the person,
- 43:06'cause they had a, you know,
- 43:08they had appropriate use of humor and they
- 43:11made cookies and they were willing to do,
- 43:15I don't know, you know,
- 43:17they're willing to go pick up
- 43:19paperwork or something like that.
- 43:21I don't know. It's not the best example.
- 43:23It's just kind of the first
- 43:25thing I landed upon.
- 43:25Yeah, the cookie bias.
- 43:27I I will say if somebody makes cookies,
- 43:29I do. I do feel positive thoughts
- 43:32about them. Anyone else want to
- 43:34thank you so much Ben for reading. I
- 43:36would add, you
- 43:37know normally I would say I
- 43:38don't see a lot of bias in this,
- 43:40looks like actually a pretty good
- 43:42evaluation with some specific details.
- 43:43I think after what we spoke about
- 43:45I I'm just going to take a wild
- 43:47guess and say this was someone
- 43:49evaluating a man because they were
- 43:51very specific as opposed to saying
- 43:53like they lacked confidence or so
- 43:55in a way I wonder about that but but
- 43:57honestly I don't see a lot of bias.
- 44:01Great. I would
- 44:04I would say leading with
- 44:07with a subjective assessment.
- 44:09I very much enjoyed working with
- 44:11so and so automatically sets up the
- 44:14reader to interpret the the assessment
- 44:18in a very specific way as opposed to
- 44:22leading with objective measurements.
- 44:25I would wholeheartedly agree with that,
- 44:26and reading this again and again has
- 44:29made me realize that I start almost
- 44:31all my evaluations that way, which,
- 44:35like I did, enjoy working with them.
- 44:37But is that the most helpful thing?
- 44:39I don't know.
- 44:42OK, go ahead, Ben. I just noticed
- 44:44one peculiar turn of phrase
- 44:47in the second sentence.
- 44:48I appreciate the effort they went
- 44:50to and caring for their patients
- 44:52and that struck me as odd,
- 44:54like caring take is work rather
- 44:58than saying I appreciate the care
- 45:01they provided to their patients.
- 45:04So you know, that struck me as a little odd.
- 45:09And perhaps you're trying to say that
- 45:11this is not an empathetic patient person.
- 45:14They have to work hard to show
- 45:17that they cared perhaps and.
- 45:20And and of one perspective.
- 45:23Yeah, no, I think that's
- 45:24a really thoughtful and
- 45:26interesting perspective for sure.
- 45:27I hadn't even noticed that.
- 45:29See this is how external
- 45:30feedback can be quite helpful.
- 45:33Can I just point out that there's
- 45:35another sort of tension in the whole
- 45:38medical school and you know and course
- 45:41director domain which is that to me,
- 45:44I think starting with a statement
- 45:45like I very much enjoyed working
- 45:47with M and casting that, you know,
- 45:49throwing out that Halo to begin
- 45:51with in a way that's kind of like I
- 45:55think it's helpful potentially in a
- 45:57departmental letter for example, right.
- 45:59I mean people are going quickly
- 46:01you know program directors are
- 46:03reading a lot of those letters.
- 46:05They get they're getting nice
- 46:06positive feelings at the beginning
- 46:07of each of these evaluations.
- 46:08They're getting good feedback,
- 46:09good feedback, good feedback.
- 46:11I don't know.
- 46:11I mean it's it's the the goal of
- 46:14that letter is very different than
- 46:16the goal of the feedback here and
- 46:18yet we directly import the feedback
- 46:20into that letter as an example of,
- 46:23you know, how the student was assessed.
- 46:24So I'm not sure if that has more
- 46:26of a place in this conversation,
- 46:28but it's just something that comes to mind.
- 46:30No, I think that's just another
- 46:32example of that cascade we were talking
- 46:35about before about like how little
- 46:37things just feed together to change
- 46:40someone's course in their educational,
- 46:41you know, trajectory, which is unfair.
- 46:45And there are so many dynamics there
- 46:47that it's it's just hard, I think,
- 46:49for us as educators to navigate.
- 46:51It's also hard for us to even talk
- 46:54with students about it, you know,
- 46:56'cause we want them to be successful,
- 46:58not only to achieve their goals,
- 46:59but also to achieve the goal of becoming
- 47:02the physician that they want to be
- 47:05or whatever the educational goal is.
- 47:07And to me that that requires hard work
- 47:09and an honest reflection on your ability.
- 47:12And I don't know if we always are
- 47:14doing our due diligence with that for
- 47:16students in that they're we're really
- 47:18actively reflecting on what are they
- 47:20able to do when work they actually grow.
- 47:23Instead we're writing things like I
- 47:25enjoyed working with them, which is good,
- 47:28it's good to enjoy working with someone,
- 47:30but how can that help that person
- 47:34grow so great?
- 47:36These are excellent reflections.
- 47:38I ended up putting my evaluation into
- 47:41a gender bias calculator so you can
- 47:45in the worksheet that I provided,
- 47:47which should have been dropped in the chat.
- 47:49You can find this gender bias
- 47:52calculator online and I found that
- 47:55the words working and effort were
- 47:58very female associated words.
- 48:01I had no idea that that was the case,
- 48:04but it made me really think about that
- 48:06this was indeed a female trainee.
- 48:08It was a female medical student and I
- 48:12wondered why I was using Hard worker so
- 48:15much when actually they had achieved,
- 48:18as you pointed out,
- 48:20Frederick,
- 48:20they had achieved good patient care.
- 48:23So it it just made me a little bit
- 48:26more thoughtful about the types
- 48:28of language that I I used.
- 48:30Moving forward,
- 48:31I think this can be incredibly hard,
- 48:34though,
- 48:34because a good evaluation in and
- 48:36of itself takes a lot of thought.
- 48:38And then to add another step of
- 48:41making sure you're not gendered
- 48:43or biased in your assessment can
- 48:45take another step of extra thought.
- 48:47And that takes a lot of cognitive load.
- 48:49So it can just take that extra,
- 48:52extra step,
- 48:53but it's worthwhile and that you're
- 48:55going to be able to get your learner the
- 48:58information they need to be successful.
- 49:00So I I wanted to, as a thought exercise,
- 49:03just take 5 minutes.
- 49:05I think we have enough time.
- 49:06Yeah, 5 minutes. Pull up a recent evaluation.
- 49:11I thought we could do this.
- 49:12Just have you do it individually.
- 49:15Ideally,
- 49:15pick something that has a lot
- 49:17of words or narrative.
- 49:19You'll find the worksheet in the chat.
- 49:21You can use the worksheet to
- 49:23evaluate your written assessments.
- 49:24I dropped the link for the gender
- 49:26calculator in there and in about,
- 49:30we'll say 4 minutes.
- 49:31So at 12:55,
- 49:33I'll just ask volunteers to share any
- 49:36reflections they had from that activity.
- 49:50I apologize. Katie, where's the worksheet?
- 49:57I'll drop it in the chat
- 49:59myself. Oh, thank you.
- 50:02And Linda put it in there a couple of oh,
- 50:05oh, that was that one. Yeah. OK. Thanks.
- 52:39Nice. Thanks Ben for sharing
- 52:41your results in the chat.
- 52:43I love your honest self reflection.
- 52:46It looks like you have a nice balance
- 52:48of male and female gendered terms.
- 52:52So I think you have a good question there,
- 52:55Ben, about like how do you interpret,
- 52:58I think it's just mostly for you to become
- 53:00more attentive to the types of words
- 53:03that you're using in your assessments.
- 53:05If you are gearing more
- 53:07towards one side or the other,
- 53:09really thinking about, OK, well,
- 53:11if I'm using mostly female gendered terms,
- 53:15am I on a female trainee,
- 53:17Do I really need to think about whether
- 53:19or not I'm being objective in my
- 53:22assessment of their clinical ability?
- 53:24And So what I would suggest for
- 53:26if you are noticing a skew on one
- 53:29side or the other is to look at
- 53:31the actual assessment form and look
- 53:33at the objective language that is
- 53:35recommended on that form and see if
- 53:37there's something that fits with the
- 53:39learner's ability in terms of what
- 53:41they can or they they're growing in to
- 53:43see if you can reflect some of that
- 53:45objective language in your assessment
- 53:50E was execute an unbiased assessment. James,
- 53:56any other thoughts about or
- 53:58reflections after doing that activity?
- 54:00I'm just putting the evaluation
- 54:01pull up just in case folks
- 54:03need to leave a little early.
- 54:12I, I, I have a comment
- 54:13Katie oh great to see you.
- 54:17I I pulled out one of my evaluations.
- 54:19It was of a female trainee and I I
- 54:24realized I do the same thing that
- 54:25you do which is I said oh this
- 54:27person is very pleasant that you.
- 54:29But I think I do that almost
- 54:30for everybody, you know,
- 54:32and I personally maybe I'm old school,
- 54:35think that's not bad because it gives
- 54:37color to the evaluation, you know,
- 54:39the whole bunch of just traits traits,
- 54:41traits can be a little dry in my opinion.
- 54:44But anyway, so this particular trainee I,
- 54:48I, I emphasize things like
- 54:50cooperation puts patients at ease,
- 54:52you know, diffuse while you,
- 54:53you know volatile situations,
- 54:54which this person was really good at.
- 54:57And I put it into your gender
- 54:59bias calculator and I thought
- 55:00it's going to tell me I'm very
- 55:02female biased and it was opposite.
- 55:05It said I used a lot of
- 55:07male associated words.
- 55:08So it's like that wasn't my initial
- 55:11you know impression but anyway I I,
- 55:15I do think some of that emotionally
- 55:18based language if it's the only thing
- 55:21is is perhaps bias.
- 55:24But it does give color to to
- 55:27otherwise people just appear
- 55:29like they're all the same.
- 55:31I I mean, I I don't know. That's
- 55:32just. Yeah. And there's nothing
- 55:34wrong with that language at all.
- 55:36I would, I think having a caring
- 55:39physician is really nice.
- 55:40You know, I think somebody who
- 55:42shows that they care and have
- 55:44compassion for another person is a
- 55:46good trait to having a physician.
- 55:48But if that's all the language that
- 55:50is being used on one individual,
- 55:51that's where things become
- 55:53a little problematic.
- 55:55But I I'm glad that you use strong
- 55:57language to describe a female training.
- 55:59That's great.
- 56:02Frederick.
- 56:03I was interested in the pyramid, excuse me,
- 56:07that you shared in an earlier slide.
- 56:10It didn't have a reference
- 56:12associated with it.
- 56:13So is that something you've
- 56:15created yourself or is there,
- 56:18you know, a publication we can
- 56:19read to learn more about it?
- 56:22Which one are you referring to, Frederick?
- 56:25Way back in
- 56:26the beginning. Yeah, way back in
- 56:27the beginning where there's a
- 56:28there was a there in in the
- 56:30lower right corner of the slide.
- 56:32There was a pyramid. And
- 56:35I would love to Google image
- 56:36search where I got that,
- 56:37that that's not of my creation.
- 56:40Oh, OK. I'll see if I can find the
- 56:42reference and I can share it with the
- 56:46formerly known as the teaching
- 56:48and Learning Center folks who can
- 56:49maybe send it out to the group.
- 56:51Yeah, we can definitely send it out. Katie,
- 56:54thank you very. Thank you very much.
- 56:56It was, it was interesting,
- 56:58but you'd need to sit with
- 57:00it a little bit to really to
- 57:02really process it. Thank you.
- 57:04Yeah, my pleasure.
- 57:07Just in the interest of time.
- 57:10And then I'll happy to talk about
- 57:13any additional thoughts people have.
- 57:15There are some additional
- 57:17yes sessions coming up.
- 57:19I would love to attend both of them.
- 57:21Getting published is definitely
- 57:22something that we all enjoy doing
- 57:24So or to potentially attending that
- 57:27one with Doctor Martin and then Bill
- 57:30Coutreir who's one of the faculty
- 57:33at Vanderbilt and a wonderful
- 57:34educator is going to talk about
- 57:36becoming a master adaptive learning.
- 57:38I've attended this session at a national
- 57:40conference and it's incredibly useful.
- 57:42So highly recommend any
- 57:46last remaining thoughts.
- 57:48Thankfully,
- 57:49we ended exactly on time.
- 57:54Thank you, Katie.
- 57:55Hopefully people will
- 57:55fill out the evaluation.
- 57:57Tell us what you find useful.
- 57:58We'll get this feedback to
- 58:00Katie and we'll see you 2
- 58:05weeks from now.
- 58:09Thanks again for joining.
- 58:11Thank you and all of these
- 58:14slides and the talks are on on
- 58:17Saved to the Center website.
- 58:21People want to go back.