Hay FeverEPLLMMRDCNCT06457269
Evaluating the Potential of Large Language Models for Respiratory Disease Consultations
A study testing Diagnosis by three human doctors, Diagnosis by ChatGPT-3.5 (with search capabilities) and Diagnosis by ChatGPT-3.5 (without search capabilities) in people with Hay Fever and Asthma, run by North Sichuan Medical College at 1 site in China.
Trial summary
The clinical trial aimes to evaluate multiple large language models in respiratory disease consultations by comparing their performance to that of human doctors across three major medical consultation scenarios. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Hay Fever, Asthma, Acute Upper Respiratory Infection
- What is being tested?
- Diagnosis by three human doctors, Diagnosis by ChatGPT-3.5 (with search capabilities) and Diagnosis by ChatGPT-3.5 (without search capabilities)
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 703 participants
- Where
- 1 siteChina
- Study dates
- 1 Oct 2023to 12 Oct 2024
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Who may be able to join
- Self-reported symptoms of common respiratory diseases, such as cough, chest tightness, fever, and wheezing
- Ability to engage in LLM dialog operations independently or with minimal peer training
- A health status deemed suitable for study participation by the pulmonology experts
Who cannot join
- 1) Excessively poor health status
Trial locations and contacts
1 site in 1 country.
China1 site
The Affiliated Hospital of North Sichuan Medical College
Nanchong, Sichuan, 637000
About this study
Official title: Evaluating the Potential of Large Language Models for Respiratory Disease Consultations: A Randomized Crossover Trial
The clinical trial aimes to evaluate multiple large language models in respiratory disease consultations by comparing their performance to that of human doctors across three major medical consultation scenarios.
The main question aims to answer are:
Read more
- How do large language models perform in comparison to human doctors in diagnosing and consulting on respiratory diseases across various clinical scenarios?
In three clinical scenarios including the online query section, the disease diagnosis section and the medical explanation section, research assistants or volunteers will be asked to cross-question all LLMs or real doctors using predefined online questions and their own issues. After each questioning session, a short washout period is implemented to eliminate potential biases.
- Conditions
- Hay Fever, Asthma, Acute Upper Respiratory Infection, Pulmonary Embolism, Pneumonia, Tuberculosis
- Being tested
- Diagnosis by three human doctors, Diagnosis by ChatGPT-3.5 (with search capabilities), Diagnosis by ChatGPT-3.5 (without search capabilities), Diagnosis by ChatGPT-4.0 (with search capabilities), Diagnosis by ChatGPT-4.0 (without search capabilities)
- Medical subject headings
- Rhinitis, Allergic, Seasonal, Asthma, Pulmonary Embolism, Pneumonia, Tuberculosis, Bronchitis, Pulmonary Fibrosis, Lung Neoplasms
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Diagnosis by three human doctorsDiagnostic test
This intervention involves answering patient inquiries by different human doctors. Each patient is randomly assigned by the system to three doctors from different provinces in China selected from the database of doctors. The doctors all come from multiple online consultation platforms in China, and their diagnostic qualifications and medical licenses have undergone strict verification.
Diagnosis by ChatGPT-3.5 (with search capabilities)Diagnostic test
This intervention involves answering patient inquiries by ChatGPT-3.5 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-3.5 (without search capabilities)Diagnostic test
This intervention involves answering patient inquiries by ChatGPT-3.5 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (with search capabilities)Diagnostic test
This intervention involves answering patient inquiries by ChatGPT-4.0 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (without search capabilities)Diagnostic test
This intervention involves answering patient inquiries by ChatGPT-4.0 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (with search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Claude instant with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (without search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Claude instant without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (with search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Claude 2 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (without search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Claude 2 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (with search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Gemini Pro with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (without search capabilities)Diagnostic test
This intervention involves answering patient inquiries by Gemini Pro without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
| Study group | Role | What they receive |
|---|---|---|
| Cross-comparison group(the disease diagnosis section) | Other | Cross-comparison group (including human doctor controls and all LLMs) Receives: Diagnostic Test: Diagnosis by three human doctors; Diagnostic Test: Diagnosis by ChatGPT-3.5 (with search capabilities); Diagnostic Test: Diagnosis by ChatGPT-3.5 (without search capabilities); Diagnostic Test: Diagnosis by ChatGPT-4.0 (with search capabilities); Diagnostic Test: Diagnosis by ChatGPT-4.0 (without search capabilities); Diagnostic Test: Diagnosis by Claude instant (with search capabilities); Diagnostic Test: Diagnosis by Claude instant (without search capabilities); Diagnostic Test: Diagnosis by Claude 2 (with search capabilities); Diagnostic Test: Diagnosis by Claude 2 (without search capabilities); Diagnostic Test: Diagnosis by Gemini Pro (with search capabilities); Diagnostic Test: Diagnosis by Gemini Pro (without search capabilities) |
| Cross-comparison group(the medical explanation section) | Other | Cross-comparison group (including human doctor controls and all LLMs) Receives: Diagnostic Test: Diagnosis by three human doctors; Diagnostic Test: Diagnosis by ChatGPT-3.5 (with search capabilities); Diagnostic Test: Diagnosis by ChatGPT-3.5 (without search capabilities); Diagnostic Test: Diagnosis by ChatGPT-4.0 (with search capabilities); Diagnostic Test: Diagnosis by ChatGPT-4.0 (without search capabilities); Diagnostic Test: Diagnosis by Claude instant (with search capabilities); Diagnostic Test: Diagnosis by Claude instant (without search capabilities); Diagnostic Test: Diagnosis by Claude 2 (with search capabilities); Diagnostic Test: Diagnosis by Claude 2 (without search capabilities); Diagnostic Test: Diagnosis by Gemini Pro (with search capabilities); Diagnostic Test: Diagnosis by Gemini Pro (without search capabilities) |
What is being measured
The primary outcome is the main question the study is designed to answer.
Expert indicators-Accuracy
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5- The responses are completely accurate, addressing all of the patient's questions or diagnosing by identifying the key points of the patient's complaint. 4- The responses are mostly accurate, generally addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 3- The responses are moderately accurate, addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 2- The responses are rarely accurate, barely addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 1- The responses are very inaccurate, not addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint at all.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Expert indicators-Comprehensiveness
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5-The responses are highly comprehensive, addressing various aspects of potential diseases corresponding to the patient's symptoms, providing detailed advice, and offering its own extended interpretations. 4-The responses are mostly comprehensive, covering most aspects of potential common diseases related to the patient's symptoms, and providing fairly detailed advice. 3-The responses are moderately comprehensive, addressing some aspects of potential common diseases related to the patient's symptoms, and offering basic advice. 2-The responses are rarely comprehensive, failing to consider various aspects of potential common diseases related to the patient's symptoms, and providing very limited advice. 1-The responses are not comprehensive at all, overlooking most potential diseases related to the patient's symptoms, and failing to provide any advice.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Expert indicators-Correctness
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5- The responses are completely correct, with no inappropriate or ambiguous statements. 4- The responses are mostly correct, with most statements being appropriate and unambiguous. 3- The responses are generally correct, although there are inappropriate or ambiguous statements, they are acceptable. 2- The responses are partially correct, with few statements being appropriate or unambiguous. 1- The responses are completely incorrect, with nearly all statements being inappropriate and full of ambiguities.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Expert indicators-Ethical compliance
Based on the doctor's response to the patient's question, an expert panel will review each item in accordance with the Declaration of Helsinki and the International Code of Medical Ethics which aims to determine whether there are any responses or suggestions that could potentially harm the patient or violate ethical guidelines. The findings will be recorded using binary variables: True-The responses are completely ethical. False-When uncertainties exist, the response includes suggestions for the use of controlled medications and some inappropriate or even counterproductive advice.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Empathy indicators
Results from CARE scales concerning the doctor-patient relationship, which were completed by patients following each diagnostic session. Specifically, the online query section does not apply the evaluation of CARE scales.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective empathy indicators, the evaluation will be conducted within two months.
Secondary outcomes (7)
Regular indicators-Total number of questions
The number of follow-up questions asked by the LLM or real doctor to the patient after providing basic answers in a complete conversation.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Follow-up words
The number of words in follow-up questions asked by the LLM or real doctor to the patient after providing basic answers in a complete conversation.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total number of conversations
The total number of dialogs in a complete conversation between a user and LLMs or a real doctor, where each dialog consists of one question and one answer.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total conversation cost ($)
The total cost in dollars for completing the entire conversation.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total conversation time (min)
Timing starts from the user's input and stops when the LLMs or real doctors completes the output of the last sentence.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Number of output statements
The total number of words output by the LLMs or real doctors.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Number of input statements
The sum of the number of characters entered by the user.
Measured: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted13 Jun 2024
- Study start1 Oct 2023
- Primary completion12 Dec 2023
- Study completion12 Oct 2024
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (1)Papers the sponsor has linked to this study.
- Other trials for Hay FeverStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Crossover
- Masking
- Quadruple blind
- Who is masked
- Participant, Care provider, Investigator, Outcomes assessor
- Enrolment
- 703 participants (actual)
Sponsor and investigators
- Lead sponsor
- North Sichuan Medical College
- Sponsor type
- University, hospital or other
- Collaborators
- University of Glasgow, Nanchong Central Hospital, Affiliated Hospital of North Sichuan Medical College
- Responsible party
- Principal investigator, Zining Luo, North Sichuan Medical College
- Investigators
- Jiebin Xie, Doctor, Principal investigator, North Sichuan Medical College
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06457269
- Sponsor's protocol ID
- 1426887-2024-1
- Other IDs
- 22XQT0309, CBY22-QDA15, 2022LC005, 23JCYJPT0014
- Acronym
- EPLLMMRDC
- Status last verified
- Nov 2024
- First submitted
- 4 Jun 2024
- Last updated
- 27 Nov 2024
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
Publications
Mercer SW, Maxwell M, Heaney D, Watt GC. The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004 Dec;21(6):699-705. doi: 10.1093/fampra/cmh621. Epub 2004 Nov 4.
PubMed 15528286
Study at a glance
From the registry record, last updated by the sponsor on 27 Nov 2024. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Evaluating the Potential of Large Language Models for Respiratory Disease Consultations: A Randomized Crossover Trial
- ClinicalTrials.gov ID
- NCT06457269
- Acronym / sponsor ID
- EPLLMMRDC / 1426887-2024-1
- Conditions
- Hay Fever, Asthma, Acute Upper Respiratory Infection, Pulmonary Embolism, Pneumonia, Tuberculosis
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 703 participants (actual)
- Lead sponsor
- North Sichuan Medical CollegeUniversity, hospital or other
- Collaborators
- University of Glasgow, Nanchong Central Hospital, Affiliated Hospital of North Sichuan Medical College
- Study start
- 1 Oct 2023 (actual)
- Primary completion
- 12 Dec 2023 (actual)
- Study completion
- 12 Oct 2024 (actual)
- First posted
- 13 Jun 2024 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The clinical trial aimes to evaluate multiple large language models in respiratory disease consultations by comparing their performance to that of human doctors across three major medical consultation scenarios.
The main question aims to answer are:
- How do large language models perform in comparison to human doctors in diagnosing and consulting on respiratory diseases across various clinical scenarios?
In three clinical scenarios including the online query section, the disease diagnosis section and the medical explanation section, research assistants or volunteers will be asked to cross-question all LLMs or real doctors using predefined online questions and their own issues. After each questioning session, a short washout period is implemented to eliminate potential biases.
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Crossover
- Masking
- Quadruple blind (participant, care provider, investigator, outcomes assessor)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Cross-comparison group(the disease diagnosis section) | Other | Diagnostic Test: Diagnosis by three human doctors, Diagnostic Test: Diagnosis by ChatGPT-3.5 (with search capabilities), Diagnostic Test: Diagnosis by ChatGPT-3.5 (without search capabilities), Diagnostic Test: Diagnosis by ChatGPT-4.0 (with search capabilities), Diagnostic Test: Diagnosis by ChatGPT-4.0 (without search capabilities), Diagnostic Test: Diagnosis by Claude instant (with search capabilities), Diagnostic Test: Diagnosis by Claude instant (without search capabilities), Diagnostic Test: Diagnosis by Claude 2 (with search capabilities), Diagnostic Test: Diagnosis by Claude 2 (without search capabilities), Diagnostic Test: Diagnosis by Gemini Pro (with search capabilities), Diagnostic Test: Diagnosis by Gemini Pro (without search capabilities)Cross-comparison group (including human doctor controls and all LLMs) |
| Cross-comparison group(the medical explanation section) | Other | Diagnostic Test: Diagnosis by three human doctors, Diagnostic Test: Diagnosis by ChatGPT-3.5 (with search capabilities), Diagnostic Test: Diagnosis by ChatGPT-3.5 (without search capabilities), Diagnostic Test: Diagnosis by ChatGPT-4.0 (with search capabilities), Diagnostic Test: Diagnosis by ChatGPT-4.0 (without search capabilities), Diagnostic Test: Diagnosis by Claude instant (with search capabilities), Diagnostic Test: Diagnosis by Claude instant (without search capabilities), Diagnostic Test: Diagnosis by Claude 2 (with search capabilities), Diagnostic Test: Diagnosis by Claude 2 (without search capabilities), Diagnostic Test: Diagnosis by Gemini Pro (with search capabilities), Diagnostic Test: Diagnosis by Gemini Pro (without search capabilities)Cross-comparison group (including human doctor controls and all LLMs) |
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (3 listed)
- Self-reported symptoms of common respiratory diseases, such as cough, chest tightness, fever, and wheezing
- Ability to engage in LLM dialog operations independently or with minimal peer training
- A health status deemed suitable for study participation by the pulmonology experts
Key exclusion criteria (1 listed)
- 1) Excessively poor health status
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Diagnosis by three human doctorsThis intervention involves answering patient inquiries by different human doctors. Each patient is randomly assigned by the system to three doctors from different provinces in China selected from the database of doctors. The doctors all come from multiple online consultation platforms in China, and their diagnostic qualifications and medical licenses have undergone strict verification. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by ChatGPT-3.5 (with search capabilities)This intervention involves answering patient inquiries by ChatGPT-3.5 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by ChatGPT-3.5 (without search capabilities)This intervention involves answering patient inquiries by ChatGPT-3.5 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by ChatGPT-4.0 (with search capabilities)This intervention involves answering patient inquiries by ChatGPT-4.0 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by ChatGPT-4.0 (without search capabilities)This intervention involves answering patient inquiries by ChatGPT-4.0 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Claude instant (with search capabilities)This intervention involves answering patient inquiries by Claude instant with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Claude instant (without search capabilities)This intervention involves answering patient inquiries by Claude instant without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Claude 2 (with search capabilities)This intervention involves answering patient inquiries by Claude 2 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Claude 2 (without search capabilities)This intervention involves answering patient inquiries by Claude 2 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Gemini Pro (with search capabilities)This intervention involves answering patient inquiries by Gemini Pro with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
| Diagnosis by Gemini Pro (without search capabilities)This intervention involves answering patient inquiries by Gemini Pro without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement. | Diagnostic test | Cross-comparison group(the disease diagnosis section), Cross-comparison group(the medical explanation section) |
Primary outcome
Expert indicators-Accuracy
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5- The responses are completely accurate, addressing all of the patient's questions or diagnosing by identifying the key points of the patient's complaint. 4- The responses are mostly accurate, generally addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 3- The responses are moderately accurate, addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 2- The responses are rarely accurate, barely addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint. 1- The responses are very inaccurate, not addressing the patient's questions or diagnosing by identifying the key points of the patient's complaint at all.
Expert indicators-Comprehensiveness
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5-The responses are highly comprehensive, addressing various aspects of potential diseases corresponding to the patient's symptoms, providing detailed advice, and offering its own extended interpretations. 4-The responses are mostly comprehensive, covering most aspects of potential common diseases related to the patient's symptoms, and providing fairly detailed advice. 3-The responses are moderately comprehensive, addressing some aspects of potential common diseases related to the patient's symptoms, and offering basic advice. 2-The responses are rarely comprehensive, failing to consider various aspects of potential common diseases related to the patient's symptoms, and providing very limited advice. 1-The responses are not comprehensive at all, overlooking most potential diseases related to the patient's symptoms, and failing to provide any advice.
Expert indicators-Correctness
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Based on the doctors' responses to patients' issues, a 5-point scale will be used for scoring by an expert panel: 5- The responses are completely correct, with no inappropriate or ambiguous statements. 4- The responses are mostly correct, with most statements being appropriate and unambiguous. 3- The responses are generally correct, although there are inappropriate or ambiguous statements, they are acceptable. 2- The responses are partially correct, with few statements being appropriate or unambiguous. 1- The responses are completely incorrect, with nearly all statements being inappropriate and full of ambiguities.
Expert indicators-Ethical compliance
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.
Based on the doctor's response to the patient's question, an expert panel will review each item in accordance with the Declaration of Helsinki and the International Code of Medical Ethics which aims to determine whether there are any responses or suggestions that could potentially harm the patient or violate ethical guidelines. The findings will be recorded using binary variables: True-The responses are completely ethical. False-When uncertainties exist, the response includes suggestions for the use of controlled medications and some inappropriate or even counterproductive advice.
Empathy indicators
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective empathy indicators, the evaluation will be conducted within two months.
Results from CARE scales concerning the doctor-patient relationship, which were completed by patients following each diagnostic session. Specifically, the online query section does not apply the evaluation of CARE scales.
7 secondary outcomes
Regular indicators-Total number of questions
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Follow-up words
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total number of conversations
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total conversation cost ($)
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Total conversation time (min)
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Number of output statements
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Regular indicators-Number of input statements
Time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. After the completion of the dialogues, the system will automatically summarize all objective indicators and dialogue information.
Results and safety
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 27 Nov 2024.
- Background: PubMed 15528286Mercer SW, Maxwell M, Heaney D, Watt GC. The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004 Dec;21(6):699-705. doi: 10.1093/fampra/cmh621. Epub 2004 Nov 4.
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Jiebin Xie, Doctor principal investigator
North Sichuan Medical College
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Diagnosis by three human doctors, Diagnosis by ChatGPT-3.5 (with search capabilities) and Diagnosis by ChatGPT-3.5 (without search capabilities) in people with Hay Fever and Asthma, run by North Sichuan Medical College at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 3 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 27 Nov 2024. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study is completed, but results are not available in the source record. That does not mean the treatment did or did not work.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Expert indicators-Accuracy (time frame: For each participant, starting from the day of random conversation, a maximum participation time of one week will be given. As for subjective expert indicators, the evaluation will be conducted within two months.). The study lists 5 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT06457269, which also lists 1 linked publication.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 27 Nov 2024; AzTrials last refreshed its copy on 26 Oct 2026.