Faculty and staff gathered on Jan. 27 for an update on several projects proposed during last year’s inaugural Ideathon.
Framed by Kathleen Ludewig, director of health IT strategy & portfolio, as innovations in the ways we listen, see, reason, and search, the showcase caught up with four teams exploring the use of artificial intelligence (AI) and spatial computing in medical education to improve student outcomes and enhance efficiency.
Jaideep Talwalkar, MD, associate dean for educational technology and innovation, shared an update on using ambient scribes in the classroom to improve the quality and efficiency of student feedback during standardized patient interviews.
Feedback is a critical component of competency-based medical education and an accreditation requirement for both the Liaison Committee for Medical Education (LCME) and the Accreditation Council of Graduate Medical Education (ACGME).
During standardized patient (SP) encounters, students interview an individual trained to portray a patient while being observed by their classmates and professor. Following the interview, the professor provides verbal feedback which they later summarize in writing on an assessment form.
This process is time-consuming and often results in an incomplete summary of what was said during the session. To address these challenges, Talwalkar led a study using an ambient AI scribe during SP interviews.
The scribe captures and summarizes verbal feedback which faculty can edit and share with the student. Thirteen faculty supervising 104 students participated in the study. The summaries generated with AI assistance scored better than the human summaries in terms of quality.
Building on the study's success, the Educational Technology and Innovation (ETI) team developed an in-house ambient scribe that captures, transcribes, and generatively organizes and summarizes the SP interview feedback.
Additionally, faculty can edit the AI-generated summaries and email them directly from the platform. To prevent errors, the tool is trained only to summarize feedback that was said during the interview.
The tool’s capabilities extend beyond feedback summaries; it can also create clinical notes and conversational summaries.
The scribe is still in the pilot phase, but Talwalkar has validation studies planned in the spring for various use cases.
Edouard Aboian, MD, associate professor of surgery (vascular), described how extended reality and three-dimensional models are being used to improve student outcomes in YSM’s human anatomy course.
In December 2024, Ray Hill, consultant for educational technology, conceptualized and developed a pipeline for generating 3D anatomy specimens. Using photogrammetry, the process of creating a 3D model by overlapping two-dimensional images, Hill created detailed 3D models of dissected and pre-dissected specimens that could be examined using extended reality (XR) headsets.
To test the efficacy of these models as teaching tools, Aboian recruited 19 student volunteers to examine models of the pelvis and abdomen ahead of their dissections using Apple Vision Pro XR headsets.
All students said they found the XR system easy to use, helpful for learning anatomy, and useful as a study aid.
Currently, four XR headsets are available for students to use: two in the Medical Library (located in the computing center locker), and two in the anatomy lab. The 3D models are also accessible via the Guided Anatomy website.
Aboian said he and the team are exploring future XR applications, such as linking anatomical structures to radiology images.
Jeffrey Dewey, MD, MHS, assistant professor of neurology, presented his work building and refining a neurologic reasoning coach using Gemini3’s customizable large language model known as Gems.
While Stanford’s Clinical Mind AI addresses many aspects of clinical reasoning, Dewey recognized a need for a specialist AI coach that could help students learn how to think like a neurologist.
Dewey started building the AI coach on Gems using the RODES framework for AI prompt engineering. The goal was to create a coach that both challenged and guided students in neurologic clinical reasoning using existing frameworks and a knowledge base developed by Dewey.
The process has been iterative. Dewey continually adjusts objectives, clinical case files, ideal responses, and expert knowledge to refine the chatbot’s coaching abilities.
Dewey noted that the chatbot sometimes draws conclusions that are not grounded in physiology or creates its own clinical reasoning process. Though it can be challenging to fix the AI's decision-making process, he’s beginning to see success.
Next steps aim to expand the clinical case repertoire, develop an in-house reference document, expand cases and models to include contextual analysis, introduce assessment statements, and eventually pilot the chatbot with neurology students.
Finally, Lei Wang, MSI, head of technology and innovation at the Cushing/Whitney Medical Library, shared an update on the MD curriculum search tool he and members of the ETI team developed last year.
The tool can search thousands of pages of content (including PowerPoint slides, syllabi, charts, images, and PDFs) across 18 pre-clerkship courses in the MD curriculum.
The applications are broad: administrators can use the tool to identify curriculum strengths and opportunities for improvement, faculty can use it to help streamline their content development process, and students can use it as a study aid.
The tool is currently in a phased launch, and the ETI team is developing training sessions and onboarding materials to help users explore the tool’s capabilities.