Achievements

Background: Alcohol use disorder is the seventh leading cause of death and disability worldwide, and in the United States approximately 500,000 episodes of withdrawal require pharmacologic treatment each year. Longer duration and higher volume of alcohol intake increase the risk for developing severe alcohol withdrawal syndrome (SAWS), characterized by intense autonomic and neuropsychiatric symptoms, withdrawal seizures, and/or delirium tremens. Assertive, early treatment can prevent progression to SAWS, but over-treatment can result in oversedation and respiratory depression. Given recent interest in artificial intelligence (AI) to aid clinical decision-making, we examined whether a leading AI model was able to accurately calculate the mass of ethanol contained in plain-language descriptions of alcohol consumption. We also designed our own evidence-based tool that enabled us to quickly and accurately calculate the mass of ethanol in common plain-language alcohol consumption descriptions. Methods: Utilizing container volumes and alcohol by volume (ABV) values defined by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) and several US-based retailers of alcoholic products, we created a table of parameters describing alcoholic beverages including number, container volume, and beverage descriptions incorporating generic, brand, and recipe names. Using Excel (v. 16.88, Microsoft), we generated 30 multiparametric descriptions of consumption (e.g., “a sleeve of nips,” “two tall boys of beer and a pint of brandy”) and calculated the mass of ethanol contained in each description using the above created table and ethanol’s density of 0.789 g/mL. We then provided these descriptions to ChatGPT-4 (v. 12-2023, OpenAI) and instructed it to calculate the mass of ethanol in each, then compared ChatGPT-4’s results with our table’s calculations. For each of the 30 multiparametric descriptions, the mass of ethanol calculated by ChatGPT was assessed for accuracy and any assumptions made by the model. Assumptions affecting result values ≤5% were classified as "minor assumptions;" those affecting result values >5% were classified as "major assumptions." ChatGPT results were classified as correct (C), correct with minor assumptions (CMi), correct with major assumptions (CMa), or incorrect (I). Results: Compared with our evidence-based tool (table of parameters), ChatGPT-4 calculated the mass of ethanol per description of consumption with the following results: 50.0% C, 23.3% CMi, 6.7% CMa, 20% I. ChatGPT-4 produced C or CMi results when prompts contained unambiguous volumes and brand names, but generated CMa and I results when prompted with generic beverage descriptors and with some slang terms for volume. Of the parameters used to generate the consumption descriptions, the standard volume of “pint [of liquor]” resulted in an incorrect volumetric calculation each time, leading to incorrect results. When ambiguous volumes were used, e.g. “box [of wine],” ChatGPT-4 made major assumptions without seeking clarification. Conclusion: Our tool outperformed ChatGPT-4 in terms of calculating accurate alcohol amounts based on typical plain language quantifications of alcohol. For example, ChatGPT-4 generally failed to ask clarifying questions regarding volume or ABV. Thus, the table of parameters we created fills the need for a reliable, evidence-based tool in estimating alcohol consumption amounts to guide patient management without AI assistance.