Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urologyNCT04011761
Effects of Expert Arbitration on Clinical Outcomes When Disputes Over Diagnosis Arise Between Physicians and Their Artificial Intelligence Counterparts: a Randomized, Multicenter Trial in Pediatric Outpatients
A study testing expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively in people with Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urology, run by Guangzhou Women and Children's Medical Center at 1 site in China.
Trial summary
We have recently developed an artificial intelligence (AI) framework to diagnose common pediatric diseases. This randomized controlled clinical trial aims to investigate the effects of expert arbitration on clinical outcomes in the situation where the AI-based diagnosis differs from the diagnosis made by pediatricians. Read more about this study
- Who is it for?
- Ages Up to 18 years
- Conditions the study is for
- Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urology
- What is being tested?
- expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 10,000 participants
- Where
- 1 siteChina
- Study dates
- 1 Nov 2019to 30 Apr 2021
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Up to 18 years
- All sexes
- No healthy volunteers
- Child, Adult
Who may be able to join
- Outpatients who visits the respirology clinics or the gastroenterology clinics during the recruitment period.
- Written informed consent is provided by parents/guardians
Who cannot join
- 1. Patients with any conditions that require immediate diagnosis and treatment.
Trial locations and contacts
1 site in 1 country.
China1 site
Guangzhou Women and Children's Medical Center
Guangzhou, Guangdong, 510623
Huiying Liang, PhD, +86-20-3885 7692, lianghuiying@hotmail.com
Kuanrong Li, PhD, +86-20-3885 7716, lik@gwcmc.org
About this study
We have recently developed an artificial intelligence (AI) framework to diagnose common pediatric diseases. This randomized controlled clinical trial aims to investigate the effects of expert arbitration on clinical outcomes in the situation where the AI-based diagnosis differs from the diagnosis made by pediatricians.
Read the detailed description
Based on the historical clinical data of more than 1 million pediatric outpatients in the Guangzhou Women and Children's Medical Center, an AI diagnostic framework has recently been developed for common pediatric diseases [Liang H et al. evaluation and accurate diagnosis of pediatric disease using artificial intelligence. Nat Med. 2019;25(3):433-8]. This AI framework utilizes predefined schema to extract informative clinical data from free text and reaches clinical diagnoses by hypothetico-deductive reasoning. In internal validation, the AI system showed accuracy rates ranging from 0.85 for gastrointestinal disease to 0.98 for neuropsychiatric disorders, suggesting that it might be a promising assisting diagnostic tool in clinical practice. However, there is a lack of evidence-based strategy on how to handle the scenarios where the AI-based diagnosis and the diagnosis made by pediatricians are discordant. It is legitimate to assume that diseases with discordant diagnoses present more similar clinical features; in this case it is necessary to introduce an extra arbitrator for differential and decisive diagnosis. Therefore, we conduct this randomized controlled trial to: 1) compare the accuracy of the two diagnostic modes in a real-world clinical setting where the AI-based diagnosis and the diagnosis made by pediatricians are discordant by introducing an expert arbitrator; and 2) look further into the change of clinical outcomes (hospital revisit and hospitalization in the next 3 months after initial visit; average total outpatient cost) due to introduction of the expert arbitrator. Please note that although the aforementioned AI framework was designed for diagnosis of a wide range of diseases, this clinical trial is limited to outpatients encountered in three specialty clinics, i.e. respirology, gastroenterology, and genito-urology. The reason for this selection is that the discordant diagnoses are assumed to be more common for these two specialties according to the internal validation result.
- Being tested
- expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectivelyOther
Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator.
| Study group | Role | What they receive |
|---|---|---|
| Experimental Arm | Active comparator | Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator. Receives: Other: expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively |
| Control Arm | No intervention | Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the control arm will receive treatment prescribed by pediatricians. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Hospital revisit
Within the first 3 months after the initial visit, active follow-up via phone call will be performed each month to collect the information on hospital revisit.
Measured: The next 3 months after the initial visit
Hospitalization in the next 3 months after the initial visit
be performed each month to collect the information on hospitalization.
Measured: The next 3 months after the initial visit
Average total outpatient cost
be performed each month to collect the information on the amount of money spending on healthcare.
Measured: The next 3 months after the initial visit
Secondary outcomes (2)
Accuracy rate of AI-based diagnosis and accuracy rate of the diagnoses made by pediatricians, using the diagnoses made by the expert arbitrator as the decisive diagnoses.
Measured: The next 3 months after the initial visit
Counseling time spent with each patient
Measured: The next 3 months after the initial visit
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted8 Jul 2019
- Study start1 Nov 2019 (estimated)
- Primary completion31 Oct 2020 (estimated)
- Study completion30 Apr 2021 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urologyStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Model description
- parallel assignment
- Masking
- None (open label)
- Enrolment
- 10,000 participants (estimated)
Sponsor and investigators
- Lead sponsor
- Guangzhou Women and Children's Medical Center
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Huiying Liang, Guangzhou Women and Children's Medical Center
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04011761
- Sponsor's protocol ID
- aiwcmc
- Status last verified
- Jun 2019
- First submitted
- 4 Jul 2019
- Last updated
- 8 Jul 2019
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 8 Jul 2019. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Effects of Expert Arbitration on Clinical Outcomes When Disputes Over Diagnosis Arise Between Physicians and Their Artificial Intelligence Counterparts: a Randomized, Multicenter Trial in Pediatric Outpatients
- ClinicalTrials.gov ID
- NCT04011761
- Acronym / sponsor ID
- aiwcmc
- Study type
- Interventional
- Recruitment status
- Unknown status
- Enrolment
- 10,000 participants (estimated)
- Lead sponsor
- Guangzhou Women and Children's Medical CenterUniversity, hospital or other
- Study start
- 1 Nov 2019 (estimated)
- Primary completion
- 31 Oct 2020 (estimated)
- Study completion
- 30 Apr 2021 (estimated)
- First posted
- 8 Jul 2019 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
We have recently developed an artificial intelligence (AI) framework to diagnose common pediatric diseases. This randomized controlled clinical trial aims to investigate the effects of expert arbitration on clinical outcomes in the situation where the AI-based diagnosis differs from the diagnosis made by pediatricians.
Detailed description
Based on the historical clinical data of more than 1 million pediatric outpatients in the Guangzhou Women and Children's Medical Center, an AI diagnostic framework has recently been developed for common pediatric diseases [Liang H et al. evaluation and accurate diagnosis of pediatric disease using artificial intelligence. Nat Med. 2019;25(3):433-8]. This AI framework utilizes predefined schema to extract informative clinical data from free text and reaches clinical diagnoses by hypothetico-deductive reasoning. In internal validation, the AI system showed accuracy rates ranging from 0.85 for gastrointestinal disease to 0.98 for neuropsychiatric disorders, suggesting that it might be a promising assisting diagnostic tool in clinical practice. However, there is a lack of evidence-based strategy on how to handle the scenarios where the AI-based diagnosis and the diagnosis made by pediatricians are discordant. It is legitimate to assume that diseases with discordant diagnoses present more similar clinical features; in this case it is necessary to introduce an extra arbitrator for differential and decisive diagnosis. Therefore, we conduct this randomized controlled trial to: 1) compare the accuracy of the two diagnostic modes in a real-world clinical setting where the AI-based diagnosis and the diagnosis made by pediatricians are discordant by introducing an expert arbitrator; and 2) look further into the change of clinical outcomes (hospital revisit and hospitalization in the next 3 months after initial visit; average total outpatient cost) due to introduction of the expert arbitrator. Please note that although the aforementioned AI framework was designed for diagnosis of a wide range of diseases, this clinical trial is limited to outpatients encountered in three specialty clinics, i.e. respirology, gastroenterology, and genito-urology. The reason for this selection is that the discordant diagnoses are assumed to be more common for these two specialties according to the internal validation result.
Study design
- Primary purpose
- Treatment
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
- Model description
- parallel assignment
| Arm | Type | Receives |
|---|---|---|
| Experimental Arm | Active comparator | Other: expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectivelyEach participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator. |
| Control Arm | No intervention | –Each participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the control arm will receive treatment prescribed by pediatricians. |
Who may qualify
- Ages
- Up to 18 years
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult
Key inclusion criteria (2 listed)
- Outpatients who visits the respirology clinics or the gastroenterology clinics during the recruitment period.
- Written informed consent is provided by parents/guardians
Key exclusion criteria (1 listed)
- 1. Patients with any conditions that require immediate diagnosis and treatment.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectivelyEach participant receives two diagnoses: one from the AI diagnostic system and the other from pediatricians, and the two diagnoses are discordant. Participants in the experimental arm will be referred to an expert arbitrator for differential and decisive diagnosis and will receive treatment prescribed by the expert arbitrator. | Other | Experimental Arm |
Primary outcome
Hospital revisit
Time frame: The next 3 months after the initial visit
Within the first 3 months after the initial visit, active follow-up via phone call will be performed each month to collect the information on hospital revisit.
Hospitalization in the next 3 months after the initial visit
Time frame: The next 3 months after the initial visit
be performed each month to collect the information on hospitalization.
Average total outpatient cost
Time frame: The next 3 months after the initial visit
be performed each month to collect the information on the amount of money spending on healthcare.
2 secondary outcomes
Accuracy rate of AI-based diagnosis and accuracy rate of the diagnoses made by pediatricians, using the diagnoses made by the expert arbitrator as the decisive diagnoses.
Time frame: The next 3 months after the initial visit
Counseling time spent with each patient
Time frame: The next 3 months after the initial visit
Results and safety
Results are not available in the source record.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 8 Jul 2019.
- Individual participant dataSharing planned: No
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing expert arbitration over discordant diagnoses made by AI diagnostic system and human doctors, respectively in people with Pediatric Outpatients Encountered in Three Specialty Clinics, i.e. Respirology, Gastroenterology, and Genito-urology, run by Guangzhou Women and Children's Medical Center at 1 site in China.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: status not confirmed, as of the sponsor's last update on 8 Jul 2019. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in China. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Results are not available in the source record.
How can a doctor refer a patient or contact the research team?
Contact Huiying Liang, PhD (+86-20-3885-7692, lianghuiying@hotmail.com), or a site near the patient. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Hospital revisit (time frame: The next 3 months after the initial visit). The study lists 3 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 8 Jul 2019; AzTrials last refreshed its copy on 7 Nov 2026.