Depression - Major Depressive DisorderGPT4-RAG-PHQNCT06801925
Evaluating the Effectiveness and Acceptability of a GPT-4o and RAG-Based Voice Chatbot for Depression Screening Using PHQ-9
An observational study of Depression - Major Depressive Disorder and Depression Anxiety Disorder, run by University College, London at 1 site in United Kingdom.
Trial summary
This study aims to assess the feasibility and acceptability of a voice-based chatbot, powered by GPT-4o and Retrieval-Augmented Generation (RAG), for conducting depression screening using the Patient Health Questionnaire-9 (PHQ-9). Read more about this study
- Who is it for?
- Ages 18 to 65 yearsHealthy volunteers can take part
- Conditions the study is for
- Depression - Major Depressive Disorder, Depression Anxiety Disorder
- What is being tested?
- GPT-4o and RAG Voice Chatbot for PHQ-9 Screening
- Type of treatment
- Surgery or procedure
- How many people
- 100 participants
- Where
- 1 siteUnited Kingdom
- Study dates
- 1 Feb 2025to 31 May 2025
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 to 65 years
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Study population: The study population consists of healthy individuals aged 18 to 65 years who do not have severe mental illnesses. We welcome participants who have an interest in artificial intelligence technologies. Recruitment will be conducted both online and offline. Participants will engage with a sophisticated AI chatbot designed to simulate a consultation with a mental health professional. This interaction enables them to complete a psychometric evaluation, such as the PHQ-9 test, in a structured yet conversational manner. Unlike conventional online versions of the PHQ-9 test, the chatbot offers enhanced interactivity. For individuals who may experience difficulties in articulating or summarizing their responses, the chatbot provides clarifications and assistance using a comprehensive language model. After the assessment, participants will receive an interpretation of their test results along with tailored advice, delivered in accordance with established mental health guidelines.
Who may be able to join
- Adults aged 18-65 years.
- Fluent in English.
- Access to a device capable of voice interaction and stable internet connection.
- Willing to participate in chatbot interaction and a follow-up interview.
Who cannot join
- Current severe psychiatric diagnoses (e.g., psychosis, bipolar disorder).
- Participants undergoing active treatment for depression with a psychiatrist.
- Discomfort with voice-based technology or inability to provide informed consent.
Trial locations and contacts
1 site in 1 country. Contact the study team or a site near you to ask about taking part.
United Kingdom1 site
UCL Institute of Health Informatics
London, NW1 2DA
About this study
This study aims to assess the feasibility and acceptability of a voice-based chatbot, powered by GPT-4o and Retrieval-Augmented Generation (RAG), for conducting depression screening using the Patient Health Questionnaire-9 (PHQ-9). The PHQ-9 is a validated self-report instrument widely used to screen, diagnose, and monitor the severity of depression. It consists of nine questions that correspond to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria for major depressive disorder. Respondents rate the frequency of symptoms experienced over the past two weeks on a scale from 0 ("not at all") to 3 ("nearly every day"). The total score (ranging from 0 to 27) indicates the severity of depressive symptoms, categorized into minimal, mild, moderate, moderately severe, or severe depression. The PHQ-9 is also used to assess functional impairment and guide treatment decisions in clinical and research settings.
The voice-based chatbot integrates GPT-4o, with RAG to enhance its ability to provide informed and contextualized responses during interactions. GPT-4o serves as the conversational engine, capable of generating human-like, empathetic, and contextually appropriate dialogue. RAG, on the other hand, enables the chatbot to retrieve and incorporate external, up-to-date knowledge from a curated database or knowledge repository, ensuring the accuracy and reliability of its responses.
Read the detailed description
Depression is a prevalent mental health challenge with significant personal, social, and economic costs. Traditional mental health resources face barriers such as stigma, limited availability, and long wait times. Technology, particularly AI-powered tools, provides an opportunity to bridge these gaps. This study utilizes GPT-4o and RAG to create a voice-interactive chatbot capable of conversational engagement, administering the PHQ-9 questionnaire, and delivering personalized feedback.
Participants will fill in the PHQ-9 for self-testing before interacting with the chatbot (the results will not be disclosed to the public and will only be used for accuracy comparisons), and the results of their self-tests will be compared with the results given by the chatbot in terms of accuracy.
The chatbot interaction comprises three phases:
- Warm-up conversations for rapport-building and general support.
- The chatbot initiates casual, empathetic dialogues to build rapport with users, helping them feel comfortable and at ease before transitioning to the PHQ-9 screening.
- Users can ask general questions related to mental health, and the chatbot provides informed and supportive responses.
- Administration of the PHQ-9 questionnaire for depression screening.
- The chatbot introduces the PHQ-9 questionnaire, explaining its purpose and how the results will help assess the user's mental health.
- Through voice interaction, users respond to the nine PHQ-9 questions, and the chatbot records their responses. The chatbot can clarify questions or provide additional context if users have difficulty understanding specific items.
- Analysis of results and delivery of tailored recommendations.
- After the user completes the PHQ-9, the chatbot analyzes the responses, calculates the total score, and categorizes the results into severity levels (e.g., mild, moderate).
- Based on the score, the chatbot provides personalized recommendations, such as self-help strategies for mild symptoms or suggesting professional mental health services for more severe cases.
Participants will interact with the chatbot and then participate in a 1-hour semi-structured interview to provide feedback on their experience. The study focuses on evaluating the acceptability and feasibility of using such LLM-based chatbots in mental health screening and identifying potential improvements and risks.
Study Objectives Primary Objectives
- To evaluate the acceptability, feasibility, and accuracy of a GPT-4o and RAG-based voice chatbot (HopeBot) for depression screening using PHQ-9. Hypothesis: Participants showed high acceptance of HopeBot (higher than 65%) and high willingness to use such LLM-based chatbot for mental health screening in the future (higher than 65%), indicating recognition of the credibility of LLM as a supportive tool in mental health screening (higher than 65%). Participants use of the HopeBot for depression screening matched their self-test PHQ-9 results by 100%
- To analyze the chatbot's effectiveness in identifying depressive symptoms and delivering actionable recommendations.
Hypothesis: HopeBot can help users take the PHQ-9 test in a friendly way, help users categorize the answers accurately, and give accurate test results, the advice they provide is based on the official PHQ-9 guidelines, and more than 70% of the users say that their responses are very effective and helpful.
Secondary Objectives
- To assess the feasibility and performance of integrating RAG with LLM in creating a voice-interactive chatbot for mental health. Hypothesis: Over 65% of participants recognized that responses using RAG were more helpful and effective.
- To explore the strengths, limitations, and risks of deploying LLMs in the mental health domain.
Hypothesis: More than 65% of users say that HopeBot is very convenient, more accessible, and cost-free to provide non-judgmental advice. However, 50% still expressed concerns about its privacy and data security.
- Being tested
- GPT-4o and RAG Voice Chatbot for PHQ-9 Screening
- Keywords
- mental health, depression, chatbot, GPT, PHQ-9, RAG
- Medical subject headings
- Psychological Well-Being, Depression
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
GPT-4o and RAG Voice Chatbot for PHQ-9 ScreeningProcedure or surgery
This study involves the use of a voice-based chatbot powered by GPT-4o and Retrieval-Augmented Generation (RAG) to conduct depression screening using the Patient Health Questionnaire-9 (PHQ-9).
The chatbot aims to evaluate the feasibility and acceptability of using AI-powered conversational tools for mental health screening.
Participants interact with the chatbot in a single session, answering PHQ-9 questions and receiving responses generated using GPT-4o and RAG technologies.
What is being measured
The primary outcome is the main question the study is designed to answer.
Feasibility and Acceptability of the GPT-4o and RAG Voice Chatbot
Participants' perceptions of the chatbot's feasibility, acceptability, and effectiveness are assessed through semi-structured interviews conducted after the interaction session. These interviews explore themes such as the chatbot's empathy, usability, and the overall user experience.
Measured: Interviews are conducted immediately following the chatbot interaction.
Secondary outcomes (1)
Accuracy of PHQ-9 Scoring by the Chatbot
The accuracy of the PHQ-9 scores generated by the chatbot is measured by comparing the chatbot's results to participants' self-reported PHQ-9 scores collected prior to the interaction. Agreement will be assessed using statistical methods such as Cohen's kappa or intraclass correlation coefficient (ICC).
Measured: Measured immediately after the interaction session, once the chatbot has generated PHQ-9 scores
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted30 Jan 2025
- Study start1 Feb 2025 (estimated)
- Primary completion31 Mar 2025 (estimated)
- Study completion31 May 2025 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for Depression - Major Depressive DisorderStudies of the same condition, open and under way.
- More trials run by University College, LondonThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Other
- Time perspective
- Cross-sectional
- Enrolment
- 100 participants (estimated)
Sponsor and investigators
- Lead sponsor
- University College, London
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
- Investigators
- Kezhi Li, Study director, University College, London
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06801925
- Sponsor's protocol ID
- 26133.001
- Other IDs
- 26133.001
- Acronym
- GPT4-RAG-PHQ
- Status last verified
- Jan 2025
- First submitted
- 24 Jan 2025
- Last updated
- 30 Jan 2025
- Results posted
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No: IPD will not be shared to ensure participant confidentiality and comply with ethical guidelines related to mental health research. Additionally, participants did not provide explicit consent for data sharing with external researchers.
Publications
Guo Z, Lai A, Ive J, Petcu A, Wang Y, Qi L, Thygesen JH, Li K. Feasibility and user evaluation of HopeBot: An LLM-powered conversational chatbot for depression screening. PLOS Digit Health. 2026 Jun 25;5(6):e0001446. doi: 10.1371/journal.pdig.0001446. eCollection 2026 Jun.
PubMed 42348542
Study at a glance
From the registry record, last updated by the sponsor on 30 Jan 2025. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Evaluating the Effectiveness and Acceptability of a GPT-4o and RAG-Based Voice Chatbot for Depression Screening Using PHQ-9
- ClinicalTrials.gov ID
- NCT06801925
- Acronym / sponsor ID
- GPT4-RAG-PHQ / 26133.001
- Keywords
- mental health, depression, chatbot, GPT, PHQ-9, RAG
- Study type
- Observational
- Recruitment status
- Enrolling by invitation
- Enrolment
- 100 participants (estimated)
- Lead sponsor
- University College, LondonUniversity, hospital or other
- Study start
- 1 Feb 2025 (estimated)
- Primary completion
- 31 Mar 2025 (estimated)
- Study completion
- 31 May 2025 (estimated)
- First posted
- 30 Jan 2025 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study aims to assess the feasibility and acceptability of a voice-based chatbot, powered by GPT-4o and Retrieval-Augmented Generation (RAG), for conducting depression screening using the Patient Health Questionnaire-9 (PHQ-9). The PHQ-9 is a validated self-report instrument widely used to screen, diagnose, and monitor the severity of depression. It consists of nine questions that correspond to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) criteria for major depressive disorder. Respondents rate the frequency of symptoms experienced over the past two weeks on a scale from 0 ("not at all") to 3 ("nearly every day"). The total score (ranging from 0 to 27) indicates the severity of depressive symptoms, categorized into minimal, mild, moderate, moderately severe, or severe depression. The PHQ-9 is also used to assess functional impairment and guide treatment decisions in clinical and research settings.
The voice-based chatbot integrates GPT-4o, with RAG to enhance its ability to provide informed and contextualized responses during interactions. GPT-4o serves as the conversational engine, capable of generating human-like, empathetic, and contextually appropriate dialogue. RAG, on the other hand, enables the chatbot to retrieve and incorporate external, up-to-date knowledge from a curated database or knowledge repository, ensuring the accuracy and reliability of its responses.
Detailed description
Depression is a prevalent mental health challenge with significant personal, social, and economic costs. Traditional mental health resources face barriers such as stigma, limited availability, and long wait times. Technology, particularly AI-powered tools, provides an opportunity to bridge these gaps. This study utilizes GPT-4o and RAG to create a voice-interactive chatbot capable of conversational engagement, administering the PHQ-9 questionnaire, and delivering personalized feedback.
Participants will fill in the PHQ-9 for self-testing before interacting with the chatbot (the results will not be disclosed to the public and will only be used for accuracy comparisons), and the results of their self-tests will be compared with the results given by the chatbot in terms of accuracy.
The chatbot interaction comprises three phases:
- Warm-up conversations for rapport-building and general support.
- The chatbot initiates casual, empathetic dialogues to build rapport with users, helping them feel comfortable and at ease before transitioning to the PHQ-9 screening.
- Users can ask general questions related to mental health, and the chatbot provides informed and supportive responses.
- Administration of the PHQ-9 questionnaire for depression screening.
- The chatbot introduces the PHQ-9 questionnaire, explaining its purpose and how the results will help assess the user's mental health.
- Through voice interaction, users respond to the nine PHQ-9 questions, and the chatbot records their responses. The chatbot can clarify questions or provide additional context if users have difficulty understanding specific items.
- Analysis of results and delivery of tailored recommendations.
- After the user completes the PHQ-9, the chatbot analyzes the responses, calculates the total score, and categorizes the results into severity levels (e.g., mild, moderate).
- Based on the score, the chatbot provides personalized recommendations, such as self-help strategies for mild symptoms or suggesting professional mental health services for more severe cases.
Participants will interact with the chatbot and then participate in a 1-hour semi-structured interview to provide feedback on their experience. The study focuses on evaluating the acceptability and feasibility of using such LLM-based chatbots in mental health screening and identifying potential improvements and risks.
Study Objectives Primary Objectives
- To evaluate the acceptability, feasibility, and accuracy of a GPT-4o and RAG-based voice chatbot (HopeBot) for depression screening using PHQ-9. Hypothesis: Participants showed high acceptance of HopeBot (higher than 65%) and high willingness to use such LLM-based chatbot for mental health screening in the future (higher than 65%), indicating recognition of the credibility of LLM as a supportive tool in mental health screening (higher than 65%). Participants use of the HopeBot for depression screening matched their self-test PHQ-9 results by 100%
- To analyze the chatbot's effectiveness in identifying depressive symptoms and delivering actionable recommendations.
Hypothesis: HopeBot can help users take the PHQ-9 test in a friendly way, help users categorize the answers accurately, and give accurate test results, the advice they provide is based on the official PHQ-9 guidelines, and more than 70% of the users say that their responses are very effective and helpful.
Secondary Objectives
- To assess the feasibility and performance of integrating RAG with LLM in creating a voice-interactive chatbot for mental health. Hypothesis: Over 65% of participants recognized that responses using RAG were more helpful and effective.
- To explore the strengths, limitations, and risks of deploying LLMs in the mental health domain.
Hypothesis: More than 65% of users say that HopeBot is very convenient, more accessible, and cost-free to provide non-judgmental advice. However, 50% still expressed concerns about its privacy and data security.
Study design
- Observational model
- Other
- Time perspective
- Cross-sectional
Who may qualify
- Ages
- 18 to 65 years
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
- Study population
- The study population consists of healthy individuals aged 18 to 65 years who do not have severe mental illnesses. We welcome participants who have an interest in artificial intelligence technologies. Recruitment will be conducted both online and offline. Participants will engage with a sophisticated AI chatbot designed to simulate a consultation with a mental health professional. This interaction enables them to complete a psychometric evaluation, such as the PHQ-9 test, in a structured yet conversational manner. Unlike conventional online versions of the PHQ-9 test, the chatbot offers enhanced interactivity. For individuals who may experience difficulties in articulating or summarizing their responses, the chatbot provides clarifications and assistance using a comprehensive language model. After the assessment, participants will receive an interpretation of their test results along with tailored advice, delivered in accordance with established mental health guidelines.
Key inclusion criteria (4 listed)
- Adults aged 18-65 years.
- Fluent in English.
- Access to a device capable of voice interaction and stable internet connection.
- Willing to participate in chatbot interaction and a follow-up interview.
Key exclusion criteria (3 listed)
- Current severe psychiatric diagnoses (e.g., psychosis, bipolar disorder).
- Participants undergoing active treatment for depression with a psychiatrist.
- Discomfort with voice-based technology or inability to provide informed consent.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| GPT-4o and RAG Voice Chatbot for PHQ-9 ScreeningThis study involves the use of a voice-based chatbot powered by GPT-4o and Retrieval-Augmented Generation (RAG) to conduct depression screening using the Patient Health Questionnaire-9 (PHQ-9). The chatbot aims to evaluate the feasibility and acceptability of using AI-powered conversational tools for mental health screening. Participants interact with the chatbot in a single session, answering PHQ-9 questions and receiving responses generated using GPT-4o and RAG technologies. | Procedure or surgery | – |
Primary outcome
Feasibility and Acceptability of the GPT-4o and RAG Voice Chatbot
Time frame: Interviews are conducted immediately following the chatbot interaction.
Participants' perceptions of the chatbot's feasibility, acceptability, and effectiveness are assessed through semi-structured interviews conducted after the interaction session. These interviews explore themes such as the chatbot's empathy, usability, and the overall user experience.
1 secondary outcome
Accuracy of PHQ-9 Scoring by the Chatbot
Time frame: Measured immediately after the interaction session, once the chatbot has generated PHQ-9 scores
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 30 Jan 2025.
- Derived from this study: PubMed 42348542Guo Z, Lai A, Ive J, Petcu A, Wang Y, Qi L, Thygesen JH, Li K. Feasibility and user evaluation of HopeBot: An LLM-powered conversational chatbot for depression screening. PLOS Digit Health. 2026 Jun 25;5(6):e0001446. doi: 10.1371/journal.pdig.0001446. eCollection 2026 Jun.
- Individual participant dataSharing planned: No. IPD will not be shared to ensure participant confidentiality and comply with ethical guidelines related to mental health research. Additionally, participants did not provide explicit consent for data sharing with external researchers.
Related trials for Depression - Major Depressive Disorder
Not yet recruiting
Preventive and Personalized Digital Emotion Regulation Skills Intervention to Reduce Depression and Anxiety in Youths With Adverse Childhood Experiences
Adverse Childhood Experience, Depression / Major Depressive Disorder
RecruitingPhase 4
Biomarker-Guided Antidepressant Selection
Depression - Major Depressive Disorder, Treatment-resistant Depression (TRD)
Recruiting
Targeting Network Dysfunction in Apathy of Late-life Depression Using Digital Therapeutics
Major Depressive Disorder, Apathy
RecruitingObservational study
Student Precarity and Psychiatry
Depressive Disorder, Major, Generalized Anxiety Disorder
Not yet recruiting
Acute Intermittent Hypoxia Therapy for Depression
Major Depressive Disorder (MDD)
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
Kezhi Li study director
University College, London
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Depression - Major Depressive Disorder and Depression Anxiety Disorder, run by University College, London at 1 site in United Kingdom.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 4 inclusion and 3 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: enrolling by invitation, as of the sponsor's last update on 30 Jan 2025. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United Kingdom. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Feasibility and Acceptability of the GPT-4o and RAG Voice Chatbot (time frame: Interviews are conducted immediately following the chatbot interaction.). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT06801925, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 30 Jan 2025.