Yale NLP/LLM Interest Group Talks

Reception is an essential process for patients seeking medical care and a critical component influencing the healthcare experience. Addressing patients’ concerns and easing their anxiety is the primary objective of receptionist nurses. However, current communication system mainly relied on efforts of human, which is labor and knowledge dual-intense, resulting in frequent burn-out and compromised patient experience. An attractive alternative is to leverage the capabilities of large language models (LLMs)to assist the communication in reception sites of medical centers. Yet, several limitations have hindered the deployment of LLMs, including shortage in context-specific knowledge, lack of real-world benchmarks, and model uncertainty. In this study, we curated a unique dataset comprising 35,418 cases of real-world conversation audio corpus between outpatients and receptionist nurses from 10reception sites across 2 medical centers, to develop SSPEC, a site-specific prompt engineering chatbot. By integrating context-specific real-world knowledge and prompt strategies, SSPEC efficiently resolved patient queries, with a higher proportion of queries addressed in fewer rounds of queries and responses compared to nurse-led sessions.