Wright T, Salyers A, Howell K, Harrison J, Silvasstar J, Bull S. A Pilot Study of an AI Chatbot for the Screening of Substance Use Disorder in a Healthcare Setting. Ai. 2025;6(6)doi:10.3390/ai6060113
This pilot study assessed the usability of an artificial intelligence (AI) chatbot, Be Well Buddy, designed to provide substance use education and facilitate screening. Substance use disorder (SUD) screening strategies remain limited, and traditional Screening, Brief Intervention, and Referral to Treatment (SBIRT) models have demonstrated efficacy for alcohol misuse but have been criticized for limited success in linking individuals to treatment, particularly for opioid use. Be Well Buddy, developed by Clinic Chat, LLC, was implemented within the Be Well Texas program at the University of Texas Health Science Center at San Antonio. The primary objectives were to evaluate system functionality and user acceptability prior to a larger efficacy trial. The system was designed using three AI-based learning models, including a natural language processing model and a fine-tuned large language model (Llama) to classify user queries. Unlike generative AI systems, responses were retrieved from a curated closed library to reduce the risk of misinformation or hallucinated responses. The library included 150 intents addressing perceptions of SUD risk, treatment access, and mental health, and 250 intents related to substances and treatments such as medication-assisted therapy, naloxone, buprenorphine, and methadone. Screening tools included the two-item Patient Health Questionnaire (PHQ-2), the two-item Generalized Anxiety Disorder scale (GAD-2), and the 10-item Drug Abuse Screening Test (DAST). PHQ-2 and GAD-2 scores ranged from 0–6, with scores ≥3 triggering referral recommendations. Be Well Buddy was assessed in a mixed-methods pilot study; participants were adults (≥18 years) who were recruited to interact with the chatbot over a seven-day period. Of the 150 individuals screened, 92 enrolled and 91 engaged with the chatbot (n=91). Over 90 days, Be Well Buddy delivered 4,173 messages, including 1,418 push notifications (34%) and 2,755 responses to user queries (66%). The system correctly responded to 80% of user queries (2,204/2,755). Screening was initiated by 33% of participants (n = 30), with 97% completing at least one screener (n = 29). Among screened participants, referrals were recommended for anxiety (83%, n=24), depression (41%, n=12), and SUD (52%, n=15). These findings indicate preliminary usability and feasibility for AI-supported SUD screening and referral.