NLP/LLM Interest Group

This session will feature two exciting talks:

1. A Collaborative Reasoning Agent-based Framework with Built-in Verification for Safe Medical Decision-Making by Fan Ma, PhD
   Abstract: Large language models (LLMs) have demonstrated expert-level capabilities on medical benchmarks, yet translating these achievements into clinical practice is impeded by persistent risks of hallucination and a lack of verifiable reasoning. While emerging agentic frameworks have begun to address these limitations through multi-step planning, existing systems often prioritize performance optimization over rigorous safety checks and fail to emulate the collective decision-making of multidisciplinary teams. To address these critical gaps, we introduce OpenDx, a multi-agent framework designed to bridge the divide between experimental prototypes and reliable clinical decision support. OpenDx is built upon three core principles: collaboration among specialized agents that simulate distinct clinical roles, integrated verification modules that strictly cross-check outputs for safety and consistency, and an architectural alignment with clinical auditability standards. We present the design and evaluation of OpenDx, demonstrating how structured collaboration significantly enhance reliability compared to baseline models. Our work advocates for a new paradigm of trustworthy medical AI, where performance gains are inseparable from the interpretability and safety assurances required for frontline healthcare deployment.
   
   
2. A Federated and Parameter-Efficient Framework for Large Language Model Training in Medicine: Applications to Clinical Information Extraction by Anran Li, PhD
   Abstract: Large language models (LLMs) are advancing medical applications such as patient question answering and diagnosis. Yet extracting structured information from unstructured clinical narratives across healthcare systems remains challenging. Current LLMs struggle with such clinical information extraction (IE) due to complex language, limited annotations, and data silos. We present a federated, model-agnostic framework for training LLMs in medicine, applied to clinical IE. The proposed Fed-MedLoRA enables parameter-efficient federated fine-tuning by transmitting only low-rank adapter parameters, substantially reducing communication and computation costs. Accuracy was assessed across five patient cohorts through comparisons with baselines for LLMs under (1) in-domain training and testing, (2) external patient cohorts, and (3) a case study on new-site adaptation using real-world clinical notes.