2024
SEETrials: Leveraging large language models for safety and efficacy extraction in oncology clinical trials
Lee K, Paek H, Huang L, Hilton C, Datta S, Higashi J, Ofoegbu N, Wang J, Rubinstein S, Cowan A, Kwok M, Warner J, Xu H, Wang X. SEETrials: Leveraging large language models for safety and efficacy extraction in oncology clinical trials. Informatics In Medicine Unlocked 2024, 50: 101589. PMID: 39493413, PMCID: PMC11530223, DOI: 10.1016/j.imu.2024.101589.Peer-Reviewed Original ResearchAntibody-drug conjugatesOverall response rateMultiple myelomaF1 scoreCAR-TComplete responseBispecific antibodiesComparative performance analysisClinical trial studyClinical trial outcomesLanguage modelAccurate data extractionTherapy subgroupFine granularityOncology clinical trialsAdverse eventsClinical decision-makingPerformance analysisClinical trialsInnovative therapiesDiverse therapiesClinical trial abstractsCancer domainData elementsTherapyAugmenting biomedical named entity recognition with general-domain resources
Yin Y, Kim H, Xiao X, Wei C, Kang J, Lu Z, Xu H, Fang M, Chen Q. Augmenting biomedical named entity recognition with general-domain resources. Journal Of Biomedical Informatics 2024, 159: 104731. PMID: 39368529, DOI: 10.1016/j.jbi.2024.104731.Peer-Reviewed Original ResearchBioNER datasetsMulti-task learningNER datasetsEntity typesBiomedical datasetsBaseline modelGeneral domain datasetsBiomedical language modelNeural network-basedYield performance improvementsBioNER modelsEntity recognitionBiomedical corporaHuman annotatorsLabel ambiguityLanguage modelTransfer learningF1 scoreBioNERHuman effortNetwork-basedBiomedical resourcesPerformance improvementDatasetSuperior performanceAscle—A Python Natural Language Processing Toolkit for Medical Text Generation: Development and Evaluation Study
Yang R, Zeng Q, You K, Qiao Y, Huang L, Hsieh C, Rosand B, Goldwasser J, Dave A, Keenan T, Ke Y, Hong C, Liu N, Chew E, Radev D, Lu Z, Xu H, Chen Q, Li I. Ascle—A Python Natural Language Processing Toolkit for Medical Text Generation: Development and Evaluation Study. Journal Of Medical Internet Research 2024, 26: e60601. PMID: 39361955, PMCID: PMC11487205, DOI: 10.2196/60601.Peer-Reviewed Original ResearchConceptsNatural language processingNatural language processing toolkitQuestion-answering taskLanguage modelText generationText processingDomain-specific language modelsNatural language processing functionsMinimal programming expertiseText generation tasksMedical knowledge graphMachine translation tasksROUGE-L scoreDomain-specific challengesAll-in-one solutionROUGE-LText summarizationBLEU scoreKnowledge graphMachine translationUnstructured textQuestion-answeringHugging FaceProcessing toolkitLanguage processingRelation extraction using large language models: a case study on acupuncture point locations
Li Y, Peng X, Li J, Zuo X, Peng S, Pei D, Tao C, Xu H, Hong N. Relation extraction using large language models: a case study on acupuncture point locations. Journal Of The American Medical Informatics Association 2024, 31: 2622-2631. PMID: 39208311, PMCID: PMC11491641, DOI: 10.1093/jamia/ocae233.Peer-Reviewed Original ResearchAcupuncture point locationsAcupoint locationLocation of acupointsClinical decision supportAcupuncture knowledgeAcupuncture trainingAcupuncture therapyAcupunctureAcupointsComplementary medicineEducational moduleWestern Pacific RegionInformatics applicationsDecision supportScoresGenerative Pre-trained TransformerWHO standardsF1 scoreLanguage modelPacific regionWHODomain-specific fine-tuningTrainingStudyMicro-averaged F1 scoreExtracting Systemic Anticancer Therapy and Response Information From Clinical Notes Following the RECIST Definition
Zuo X, Kumar A, Shen S, Li J, Cong G, Jin E, Chen Q, Warner J, Yang P, Xu H. Extracting Systemic Anticancer Therapy and Response Information From Clinical Notes Following the RECIST Definition. JCO Clinical Cancer Informatics 2024, 8: e2300166. PMID: 38885475, DOI: 10.1200/cci.23.00166.Peer-Reviewed Original ResearchConceptsNatural language processingDomain-specific language modelsNatural language processing systemsInformation extraction systemRule-based moduleNarrative clinical textsNLP tasksEntity recognitionText normalizationAssertion classificationLanguage modelInformation extractionClinical textElectronic health recordsLearning-basedClinical notesLanguage processingTest setSystem performanceHealth recordsResponse extractionTime-consumingAnticancer therapyInformationAssessment informationIntroduction to Natural Language Processing of Clinical Text
Demner Fushman D, Xu H. Introduction to Natural Language Processing of Clinical Text. Cognitive Informatics In Biomedicine And Healthcare 2024, 3-11. DOI: 10.1007/978-3-031-55865-8_1.Peer-Reviewed Original ResearchNatural language processingLanguage processingComplex language processingBiomedical natural language processingClinical natural language processingLanguage generation tasksClinical language processingBiomedical language processingLanguage modelClinical textGeneration taskMachine learningDelivery of informationClinical languageLanguageDevelopment of Clinical NLP Systems
Xu H, Demner Fushman D. Development of Clinical NLP Systems. Cognitive Informatics In Biomedicine And Healthcare 2024, 301-324. DOI: 10.1007/978-3-031-55865-8_11.Peer-Reviewed Original ResearchMapping Study Variables to Common Data Elements Using GPT for Sheets: Towards Standardized Data Collection and Sharing
Ram P, Hong N, Xu H, Jiang X. Mapping Study Variables to Common Data Elements Using GPT for Sheets: Towards Standardized Data Collection and Sharing. 2024, 00: 320-325. DOI: 10.1109/ichi61247.2024.00048.Peer-Reviewed Original ResearchA Study of Biomedical Relation Extraction Using GPT Models.
Zhang J, Wibert M, Zhou H, Peng X, Chen Q, Keloth V, Hu Y, Zhang R, Xu H, Raja K. A Study of Biomedical Relation Extraction Using GPT Models. AMIA Joint Summits On Translational Science Proceedings 2024, 2024: 391-400. PMID: 38827097, PMCID: PMC11141827.Peer-Reviewed Original ResearchLarge language models for biomedicine: foundations, opportunities, challenges, and best practices
Sahoo S, Plasek J, Xu H, Uzuner Ö, Cohen T, Yetisgen M, Liu H, Meystre S, Wang Y. Large language models for biomedicine: foundations, opportunities, challenges, and best practices. Journal Of The American Medical Informatics Association 2024, 31: 2114-2124. PMID: 38657567, PMCID: PMC11339493, DOI: 10.1093/jamia/ocae074.Peer-Reviewed Original ResearchNatural language processingPrompt tuningNLP applicationsLanguage modelState-of-the-art performanceNLP practitionersNatural language processing applicationsBiomedical NLP applicationsPre-training datasetNatural language understandingNeural network architecture modelNatural language generationBiomedical informatics communityNetwork architecture modelAmerican Medical Informatics Association (AMIAPrompt-tuningFew-shotZero-ShotNLP challengeNLP tasksReinforcement learningHuman feedbackLanguage generationLanguage understandingEvaluation metricsEnsemble pretrained language models to extract biomedical knowledge from literature
Li Z, Wei Q, Huang L, Li J, Hu Y, Chuang Y, He J, Das A, Keloth V, Yang Y, Diala C, Roberts K, Tao C, Jiang X, Zheng W, Xu H. Ensemble pretrained language models to extract biomedical knowledge from literature. Journal Of The American Medical Informatics Association 2024, 31: 1904-1911. PMID: 38520725, PMCID: PMC11339500, DOI: 10.1093/jamia/ocae061.Peer-Reviewed Original ResearchNatural language processingNatural language processing systemsLanguage modelExpansion of biomedical literatureZero-shot settingManually annotated corpusKnowledge graph developmentTask-specific modelsDomain-specific modelsZero-ShotEntity recognitionBillion parametersEnsemble learningLocation informationKnowledge basesBiomedical entitiesLanguage processingFree textGraph developmentBiomedical conceptsAutomated techniqueBiomedical literatureDetection methodPredictive performanceBiomedical knowledgeAdvancing entity recognition in biomedicine via instruction tuning of large language models
Keloth V, Hu Y, Xie Q, Peng X, Wang Y, Zheng A, Selek M, Raja K, Wei C, Jin Q, Lu Z, Chen Q, Xu H. Advancing entity recognition in biomedicine via instruction tuning of large language models. Bioinformatics 2024, 40: btae163. PMID: 38514400, PMCID: PMC11001490, DOI: 10.1093/bioinformatics/btae163.Peer-Reviewed Original ResearchNamed Entity RecognitionSequence labeling taskNatural language processingBiomedical NER datasetsLanguage modelNER datasetsEntity recognitionLabeling taskText generationField of natural language processingBiomedical NERFew-shot learning capabilityReasoning tasksMulti-domain scenariosDomain-specific modelsEnd-to-endMinimal fine-tuningSOTA performanceF1 scoreHealthcare applicationsBiomedical entitiesBiomedical domainLanguage processingMulti-taskingPubMedBERT modelPrompt Tuning in Biomedical Relation Extraction
He J, Li F, Li J, Hu X, Nian Y, Xiang Y, Wang J, Wei Q, Li Y, Xu H, Tao C. Prompt Tuning in Biomedical Relation Extraction. Journal Of Healthcare Informatics Research 2024, 8: 206-224. PMID: 38681754, PMCID: PMC11052745, DOI: 10.1007/s41666-024-00162-9.Peer-Reviewed Original ResearchFew-shot scenariosBiomedical relation extractionNatural language processingBiomedical RERelation extractionPrompt tuningState-of-the-art performanceText mining applicationsTuning modelBioCreative VISemEval-2013Knowledge graphLanguage modelMining applicationsBiomedical textOriginal inputComputational resourcesLanguage processingExternal knowledgeSpecific textsSuperior performanceDatasetEfficient approachTaskModel performanceImproving large language models for clinical named entity recognition via prompt engineering
Hu Y, Chen Q, Du J, Peng X, Keloth V, Zuo X, Zhou Y, Li Z, Jiang X, Lu Z, Roberts K, Xu H. Improving large language models for clinical named entity recognition via prompt engineering. Journal Of The American Medical Informatics Association 2024, 31: 1812-1820. PMID: 38281112, PMCID: PMC11339492, DOI: 10.1093/jamia/ocad259.Peer-Reviewed Original ResearchClinical NER tasksNER taskTask-specific promptsEntity recognitionLanguage modelTraining samplesState-of-the-art modelsFew-shot learningState-of-the-artMinimal training dataTask-specific knowledgeF1-socreAnnotated samplesConcept extractionModel performanceAnnotated datasetsTraining dataF1 scoreTask descriptionFormat specificationsComplex clinical dataOptimal performanceTaskEvaluation schemaGPT modelStandardizing Multi-site Clinical Note Titles to LOINC Document Ontology: A Transformer-based Approach.
Zuo X, Zhou Y, Duke J, Hripcsak G, Shah N, Banda J, Reeves R, Miller T, Waitman L, Natarajan K, Xu H. Standardizing Multi-site Clinical Note Titles to LOINC Document Ontology: A Transformer-based Approach. AMIA Annual Symposium Proceedings 2024, 2023: 834-843. PMID: 38222429, PMCID: PMC10785935.Peer-Reviewed Original Research
2023
AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models
Datta S, Lee K, Paek H, Manion F, Ofoegbu N, Du J, Li Y, Huang L, Wang J, Lin B, Xu H, Wang X. AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models. Journal Of The American Medical Informatics Association 2023, 31: 375-385. PMID: 37952206, PMCID: PMC10797270, DOI: 10.1093/jamia/ocad218.Peer-Reviewed Original ResearchConceptsLanguage modelInformation extraction systemOverall F1 scoreCriteria informationF1 scoreManual annotationScalable solutionContextual informationComplex scenariosContextual attributesExtraction systemReal-world settingsSystem evaluationModeling capabilitiesClinical trial protocol documentsInformationProtocol documentsLeveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration
Yu P, Xu H, Hu X, Deng C. Leveraging Generative AI and Large Language Models: A Comprehensive Roadmap for Healthcare Integration. Healthcare 2023, 11: 2776. PMID: 37893850, PMCID: PMC10606429, DOI: 10.3390/healthcare11202776.Peer-Reviewed Original ResearchLarge language modelsGenerative artificial intelligenceArtificial intelligenceLanguage modelInformation retrievalAI systemsShot learningData managementHuman feedbackReinforcement learningInformation managementSystem implementationCo-design processData acquisitionComprehensive roadmapDecision-making processLearningTechnologyFull potentialHealthcareIntelligenceHealthcare qualityRetrievalIntegrationPromising advancement
2022
A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora
Li J, Wei Q, Ghiasvand O, Chen M, Lobanov V, Weng C, Xu H. A comparative study of pre-trained language models for named entity recognition in clinical trial eligibility criteria from multiple corpora. BMC Medical Informatics And Decision Making 2022, 22: 235. PMID: 36068551, PMCID: PMC9450226, DOI: 10.1186/s12911-022-01967-7.Peer-Reviewed Original ResearchConceptsPre-trained language modelsNER taskUnstructured textEntity recognitionLanguage modelNatural language processing techniquesClinical trial eligibility criteriaLanguage processing techniquesData augmentation resultsData augmentation approachDomain-specific corpusBetter performanceTransformer modelCross-validation showMultiple data sourcesEligibility criteria textBiomedical domainEmbedding modelsNER performanceAugmentation approachContextual embeddingsMeaningful informationEvaluation resultsSuch documentsProcessing techniques
2021
From Tokenization to Self-Supervision: Building a High-Performance Information Extraction System for Chemical Reactions in Patents
Wang J, Ren Y, Zhang Z, Xu H, Zhang Y. From Tokenization to Self-Supervision: Building a High-Performance Information Extraction System for Chemical Reactions in Patents. Frontiers In Research Metrics And Analytics 2021, 6: 691105. PMID: 35005421, PMCID: PMC8727901, DOI: 10.3389/frma.2021.691105.Peer-Reviewed Original ResearchEvent extractionEntity recognitionNatural language processing techniquesAccurate information extractionInformation extraction systemLanguage processing techniquesKnowledge-based rulesInformation extractionAutomatic toolEnd systemArt resultsSemantic rolesLanguage modelSelf-SupervisionFree textChemical patentsSubtask 1Reaction extractionDifferent semantic rolesHybrid approachEvent triggersProcessing techniquesSubtasksTokenizationHigh performance
2020
Relation Extraction from Clinical Narratives Using Pre-trained Language Models.
Wei Q, Ji Z, Si Y, Du J, Wang J, Tiryaki F, Wu S, Tao C, Roberts K, Xu H. Relation Extraction from Clinical Narratives Using Pre-trained Language Models. AMIA Annual Symposium Proceedings 2020, 2019: 1236-1245. PMID: 32308921, PMCID: PMC7153059.Peer-Reviewed Original ResearchConceptsPre-trained language modelsNatural language processingLanguage modelRE tasksNLP tasksClinical narrativesRecent deep learning methodsDeep learning methodsClinical NLP tasksRelation extraction taskTraditional word embeddingsTraditional machineExtraction taskArt performanceRelation extractionBERT modelLanguage processingLearning methodsWord embeddingsShared TaskPrevious stateBiomedical literatureDifferent implementationsTaskOpen domain