Clinical Decision-makingNCT07711600
Large Language Model Assistance for Clinical Decision-Making Among Rural Physicians
A study testing LLM-Use Training, Conventional Non-LLM Resources and LLM Second-Opinion Review in people with Clinical Decision-making, run by Peking University Third Hospital at 1 site in China.
Trial summary
This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings. Read more about this study
- Who is it for?
- Ages 18 to 65 yearsHealthy volunteers can take part
- Conditions the study is for
- Clinical Decision-making
- What is being tested?
- LLM-Use Training, Conventional Non-LLM Resources and LLM Second-Opinion Review
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 240 participants
- Where
- 1 siteChina
- Study dates
- Jul 2026to Sep 2026
- Phase
- Interventional
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 to 65 years
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Who may be able to join
- Currently engaged in clinical practice at a rural primary healthcare institution in northwestern China.
- Has received formal medical education and holds a relevant diploma or degree.
- Able to read and understand clinical case materials in Chinese.
- Able to use a computer to complete the study tasks.
- Willing to participate and able to provide written informed consent.
Who cannot join
- Previously involved in the development of the clinical case tasks, reference answers, or scoring rubric for this study.
- Previously participated in pilot testing involving the same clinical case tasks or study procedures.
Trial locations and contacts
1 site in 1 country. Contact the study team or a site near you to ask about taking part.
China1 site
Xinjiang Second Medical College
Karamay, Xinjiang Uygur Autonomous Region
Zejiang Ma, Master of Medicine, +86 18811331959, 18811331959@163.com
About this study
Official title: Effect of Large Language Model Assistance on Clinical Decision-Making Among Rural Physicians: A Randomized Controlled Trial
This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings.
Read the detailed description
Rural physicians play an essential role in the diagnosis and management of common and frequently occurring conditions, referral decision-making, chronic disease management, and patient education. In resource-constrained primary care settings, they often face limited access to medical information and specialist support, delays in updating clinical knowledge and guidelines, and substantial pressure in clinical decision-making. These challenges are particularly relevant in northwestern China, where primary care resources are relatively limited. Improving rural physicians' abilities in diagnostic assessment, recognition of clinical warning signs, and rational prescribing is therefore an important priority for strengthening primary healthcare services.
Large language models (LLMs) can support medical information retrieval, organization of diagnostic and management approaches, differential diagnosis, medication-related decision-making, patient education, and follow-up planning, and may therefore serve as accessible tools for supporting clinical decision-making in primary care. However, general-purpose LLMs were not specifically developed for use in resource-constrained primary care settings and have not been adequately evaluated among rural physicians. Their responses may contain factual errors or fabricated evidence, overlook warning signs, provide insufficient medication safety warnings, or recommend investigations and treatments that are not feasible in local primary care settings. Without adequate verification skills, physicians may fail to benefit from LLM assistance and may even introduce new safety risks. It is therefore important to evaluate how rural physicians use LLMs and whether structured training can improve the safe and effective use of these tools before their wider implementation.
This randomized controlled trial will evaluate the effects of LLM assistance and brief training on clinical decision-making among rural physicians. Participants will complete clinical cases involving common conditions encountered in primary care, with tasks assessing diagnostic judgment, recognition of warning signs, rational treatment, and patient education. Some participants will also use the LLM as a second-opinion tool to review and revise their initial decisions. All responses will be independently evaluated by reviewers blinded to group assignment using standardized scoring criteria to assess overall clinical decision-making performance and safety.
- Conditions
- Clinical Decision-making
- Being tested
- LLM-Use Training, Conventional Non-LLM Resources, LLM Second-Opinion Review, Direct LLM Assistance
- Keywords
- Large Language Models, Clinical Decision Support, Rural Physicians
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
LLM-Use TrainingBehavioural
Before completing the clinical cases, participants receive brief structured training on the safe and effective use of LLMs. The training covers the role and limitations of LLMs, structured prompting and follow-up questioning, identification of warning signs and referral indications, medication safety, verification of LLM-generated information, high-risk situations in which LLMs should not be relied upon, and protection of patient privacy.
Conventional Non-LLM ResourcesOther
During the initial 60-minute assessment, participants complete primary care clinical cases using conventional non-LLM resources only, including clinical guidelines, textbooks, drug labels, training materials, medical websites, and standard search engines. Participants are not permitted to use LLMs during this phase.
LLM Second-Opinion ReviewOther
After completing and submitting their initial responses using conventional non-LLM resources, participants receive an additional 30 minutes to use the study-provided DeepSeek-V4 as a second-opinion tool. They may review, verify, and revise their initial clinical decisions before submitting their final responses.
Direct LLM AssistanceOther
During the initial 60-minute assessment, participants may use the study-provided DeepSeek-V4 to assist with medical information retrieval, diagnostic and management reasoning, identification of warning signs, referral decisions, rational prescribing, patient education, and follow-up planning. Participants remain responsible for their final clinical decisions and responses.
| Study group | Role | What they receive |
|---|---|---|
| Conventional Resources Followed by LLM Second Opinion | Active comparator | Participants receive no LLM-use training. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses. Receives: Other: Conventional Non-LLM Resources; Other: LLM Second-Opinion Review |
| Direct LLM Assistance | Experimental | Participants receive no LLM-use training and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase. Receives: Other: Direct LLM Assistance |
| LLM Training and Conventional Resources Followed by LLM Second Opinion | Experimental | Participants receive brief structured training on the safe and effective use of LLMs. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses. Receives: Behavioral: LLM-Use Training; Other: Conventional Non-LLM Resources; Other: LLM Second-Opinion Review |
| LLM Training and Direct LLM Assistance | Experimental | Participants receive brief structured training on the safe and effective use of LLMs and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase. Receives: Behavioral: LLM-Use Training; Other: Direct LLM Assistance |
What is being measured
The primary outcome is the main question the study is designed to answer.
Overall Clinical Decision-Making Score
Participants' responses to primary care clinical cases will be evaluated using a prespecified scoring rubric. The overall score will reflect performance across key components of clinical decision-making. Higher scores indicate better overall clinical decision-making performance.
Measured: At the end of the initial 60-minute assessment
Secondary outcomes (4)
Diagnostic Judgment Domain Score
Participants' diagnostic judgment in response to primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better diagnostic judgment.
Measured: At the end of the initial 60-minute assessment
Clinical Warning Sign Recognition Domain Score
Participants' ability to identify clinically important warning signs in primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better recognition of clinical warning signs.
Measured: At the end of the initial 60-minute assessment
Treatment Plan Domain Score
Participants' proposed treatment plans for primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better treatment planning performance.
Measured: At the end of the initial 60-minute assessment
Change in Overall Clinical Decision-Making Score After LLM Review
Among participants assigned to the LLM second-opinion phase, the change in overall clinical decision-making score will be calculated as the score after LLM-assisted review minus the score before LLM-assisted review. Positive values indicate improvement in overall clinical decision-making performance.
Measured: Change from 60 to 90 minutes after the start of the assessment
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted17 Jul 2026
- Study startJul 2026 (estimated)
- Primary completionAug 2026 (estimated)
- Study completionSep 2026 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Clinical Decision-makingStudies of the same condition, open and under way.
- More trials run by Peking University Third HospitalThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Factorial
- Model description
- Participants will be randomly assigned to four groups according to two intervention factors: receipt of brief training in LLM use and direct access to the LLM during the initial assessment. Participants assigned to selected groups will subsequently use the LLM as a second-opinion tool to review and revise their initial responses.
- Masking
- Single blind
- Who is masked
- Outcomes assessor
- Enrolment
- 240 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Peking University Third Hospital
- Sponsor type
- University, hospital or other
- Collaborators
- Xinjiang Second Medical College
- Responsible party
- Principal investigator, tao liyuan, Peking University Third Hospital
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07711600
- Sponsor's protocol ID
- IRB00006761-M20260614
- Status last verified
- Jul 2026
- First submitted
- 14 Jul 2026
- Last updated
- 17 Jul 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 17 Jul 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Effect of Large Language Model Assistance on Clinical Decision-Making Among Rural Physicians: A Randomized Controlled Trial
- ClinicalTrials.gov ID
- NCT07711600
- Acronym / sponsor ID
- IRB00006761-M20260614
- Conditions
- Clinical Decision-making
- Keywords
- Large Language Models, Clinical Decision Support, Rural Physicians
- Study type
- Interventional
- Recruitment status
- Not yet recruiting
- Enrolment
- 240 participants (estimated)
- Lead sponsor
- Peking University Third HospitalUniversity, hospital or other
- Collaborators
- Xinjiang Second Medical College
- Study start
- Jul 2026 (estimated)
- Primary completion
- Aug 2026 (estimated)
- Study completion
- Sep 2026 (estimated)
- First posted
- 17 Jul 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings.
Detailed description
Rural physicians play an essential role in the diagnosis and management of common and frequently occurring conditions, referral decision-making, chronic disease management, and patient education. In resource-constrained primary care settings, they often face limited access to medical information and specialist support, delays in updating clinical knowledge and guidelines, and substantial pressure in clinical decision-making. These challenges are particularly relevant in northwestern China, where primary care resources are relatively limited. Improving rural physicians' abilities in diagnostic assessment, recognition of clinical warning signs, and rational prescribing is therefore an important priority for strengthening primary healthcare services.
Large language models (LLMs) can support medical information retrieval, organization of diagnostic and management approaches, differential diagnosis, medication-related decision-making, patient education, and follow-up planning, and may therefore serve as accessible tools for supporting clinical decision-making in primary care. However, general-purpose LLMs were not specifically developed for use in resource-constrained primary care settings and have not been adequately evaluated among rural physicians. Their responses may contain factual errors or fabricated evidence, overlook warning signs, provide insufficient medication safety warnings, or recommend investigations and treatments that are not feasible in local primary care settings. Without adequate verification skills, physicians may fail to benefit from LLM assistance and may even introduce new safety risks. It is therefore important to evaluate how rural physicians use LLMs and whether structured training can improve the safe and effective use of these tools before their wider implementation.
This randomized controlled trial will evaluate the effects of LLM assistance and brief training on clinical decision-making among rural physicians. Participants will complete clinical cases involving common conditions encountered in primary care, with tasks assessing diagnostic judgment, recognition of warning signs, rational treatment, and patient education. Some participants will also use the LLM as a second-opinion tool to review and revise their initial decisions. All responses will be independently evaluated by reviewers blinded to group assignment using standardized scoring criteria to assess overall clinical decision-making performance and safety.
Study design
- Primary purpose
- Health services research
- Allocation
- Randomised
- Intervention model
- Factorial
- Masking
- Single blind (outcomes assessor)
- Number of arms
- 4
- Model description
- Participants will be randomly assigned to four groups according to two intervention factors: receipt of brief training in LLM use and direct access to the LLM during the initial assessment. Participants assigned to selected groups will subsequently use the LLM as a second-opinion tool to review and revise their initial responses.
| Arm | Type | Receives |
|---|---|---|
| Conventional Resources Followed by LLM Second Opinion | Active comparator | Other: Conventional Non-LLM Resources, Other: LLM Second-Opinion ReviewParticipants receive no LLM-use training. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses. |
| Direct LLM Assistance | Experimental | Other: Direct LLM AssistanceParticipants receive no LLM-use training and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase. |
| LLM Training and Conventional Resources Followed by LLM Second Opinion | Experimental | Behavioral: LLM-Use Training, Other: Conventional Non-LLM Resources, Other: LLM Second-Opinion ReviewParticipants receive brief structured training on the safe and effective use of LLMs. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses. |
| LLM Training and Direct LLM Assistance | Experimental | Behavioral: LLM-Use Training, Other: Direct LLM AssistanceParticipants receive brief structured training on the safe and effective use of LLMs and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase. |
Who may qualify
- Ages
- 18 to 65 years
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (5 listed)
- Currently engaged in clinical practice at a rural primary healthcare institution in northwestern China.
- Has received formal medical education and holds a relevant diploma or degree.
- Able to read and understand clinical case materials in Chinese.
- Able to use a computer to complete the study tasks.
- Willing to participate and able to provide written informed consent.
Key exclusion criteria (2 listed)
- Previously involved in the development of the clinical case tasks, reference answers, or scoring rubric for this study.
- Previously participated in pilot testing involving the same clinical case tasks or study procedures.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| LLM-Use TrainingBefore completing the clinical cases, participants receive brief structured training on the safe and effective use of LLMs. The training covers the role and limitations of LLMs, structured prompting and follow-up questioning, identification of warning signs and referral indications, medication safety, verification of LLM-generated information, high-risk situations in which LLMs should not be relied upon, and protection of patient privacy. | Behavioural | LLM Training and Conventional Resources Followed by LLM Second Opinion, LLM Training and Direct LLM Assistance |
| Conventional Non-LLM ResourcesDuring the initial 60-minute assessment, participants complete primary care clinical cases using conventional non-LLM resources only, including clinical guidelines, textbooks, drug labels, training materials, medical websites, and standard search engines. Participants are not permitted to use LLMs during this phase. | Other | Conventional Resources Followed by LLM Second Opinion, LLM Training and Conventional Resources Followed by LLM Second Opinion |
| LLM Second-Opinion ReviewAfter completing and submitting their initial responses using conventional non-LLM resources, participants receive an additional 30 minutes to use the study-provided DeepSeek-V4 as a second-opinion tool. They may review, verify, and revise their initial clinical decisions before submitting their final responses. | Other | Conventional Resources Followed by LLM Second Opinion, LLM Training and Conventional Resources Followed by LLM Second Opinion |
| Direct LLM AssistanceDuring the initial 60-minute assessment, participants may use the study-provided DeepSeek-V4 to assist with medical information retrieval, diagnostic and management reasoning, identification of warning signs, referral decisions, rational prescribing, patient education, and follow-up planning. Participants remain responsible for their final clinical decisions and responses. | Other | Direct LLM Assistance, LLM Training and Direct LLM Assistance |
Primary outcome
Overall Clinical Decision-Making Score
Time frame: At the end of the initial 60-minute assessment
Participants' responses to primary care clinical cases will be evaluated using a prespecified scoring rubric. The overall score will reflect performance across key components of clinical decision-making. Higher scores indicate better overall clinical decision-making performance.
4 secondary outcomes
Diagnostic Judgment Domain Score
Time frame: At the end of the initial 60-minute assessment
Clinical Warning Sign Recognition Domain Score
Time frame: At the end of the initial 60-minute assessment
Treatment Plan Domain Score
Time frame: At the end of the initial 60-minute assessment
Change in Overall Clinical Decision-Making Score After LLM Review
Time frame: Change from 60 to 90 minutes after the start of the assessment
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 17 Jul 2026.
- Individual participant dataSharing planned: No
Related trials for Clinical Decision-making
RecruitingObservational study
The Decision To Enroll In Therapeutic Clinical Trials In A Pediatric Cancer Clinical Network
Knowledge, Attitudes, Practice
Recruiting
GenAI-Assisted Rheumatoid Arthritis Bone Erosion Risk Assessment and DMARD Selection
Rheumatoid Arthritis
RecruitingPhase 2
Circulating Tumor DNA Methylation-Guided Adjuvant Therapy Decision-Making in Patients With Resectable Colorectal Cancer Liver Metastases After Resection
Colorectal Cancer Liver Metastases (CRLM), ctDNA
RecruitingObservational study
A Study for the PanCystPro Assay in the Management of Pancreatic Cystic Lesions
Pancreatic Cystic Lesions
Not yet recruitingObservational study
PAtient Similarity for Decision-Making in Prevention of Cardiovascular Toxicity (PACT): A Feasibility Study
Heart Failure, Coronary Artery Disease
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing LLM-Use Training, Conventional Non-LLM Resources and LLM Second-Opinion Review in people with Clinical Decision-making, run by Peking University Third Hospital at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 5 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: opening soon, as of the sponsor's last update on 17 Jul 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
Contact Liyuan Tao, MD (+86 82265732, tendytly@163.com), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Overall Clinical Decision-Making Score (time frame: At the end of the initial 60-minute assessment). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 17 Jul 2026.