Skin DiseasesNCT04636164
Deep Neural Networks on the Accuracy of Skin Disease Diagnosis in Non-Dermatologists
A study testing Model Dermatology (deep neural networks; Build 2020) in people with Skin Diseases, run by Pyoeng Gyun Choe at 1 site in South Korea.
Trial summary
Background: Deep neural networks (DNN) has been applied to many kinds of skin diseases in experimental settings. Objective: The objective of this study is to confirm the augmentation of deep neural networks for the diagnosis of skin diseases in non-dermatologist physicians in a real-world setting. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Skin Diseases
- What is being tested?
- Model Dermatology (deep neural networks; Build 2020)
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 55 participants
- Where
- 1 siteSouth Korea
- Study dates
- 27 Nov 2020to 27 Dec 2021
- Phase
- Interventional
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Who may be able to join
- non-dermatologist physician (residents) who agree to participate in this study
Who cannot join
- dermatology residents
- non-dermatology residents who use other deep neural networks for skin lesion diagnosis
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Effect of Using Deep Neural Networks on the Accuracy of Skin Disease Diagnosis in Non-Dermatologist Physician
Background: Deep neural networks (DNN) has been applied to many kinds of skin diseases in experimental settings.
Objective: The objective of this study is to confirm the augmentation of deep neural networks for the diagnosis of skin diseases in non-dermatologist physicians in a real-world setting.
Methods: A total of 40 non-dermatologist physicians in a single tertiary care hospital will be enrolled. They will be randomized to a DNN group and control group. By comparing two groups, the investigators will estimate the effect of using deep neural networks on the diagnosis of skin disease in terms of accuracy.
Read the detailed description
In the DNN group and control group, these steps are the same process.
- Routine exam and capture photographs of skin lesions for all eligible consecutive series patient.
- Make a clinical diagnosis (BEFORE-DX)
- Make a clinical diagnosis (AFTER-DX)
- consult to dermatologist
In the DNN group, after making the BEFORE-DX, physicians use deep neural networks and make an AFTER-DX considering the results of the deep neural networks (Model Dermatology, build 2020).
In the control group, after making the BEFORE-DX, physicians make an AFTER-DX after reviewing the pictures of skin lesions once more.
Ground truth will be based on the biopsy if available, or the consensus diagnosis of the dermatologists.
The investigators will compare the accuracy between the DNN group and control group after 6 consecutive months study.
- Conditions
- Skin Diseases
- Being tested
- Model Dermatology (deep neural networks; Build 2020)
- Keywords
- skin diseases, deep neural networks
- Medical subject headings
- Skin Diseases
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Model Dermatology (deep neural networks; Build 2020)Diagnostic test
Physicians in the DNN group take pictures of the skin lesion and use the algorithm by uploading pictures.
| Study group | Role | What they receive |
|---|---|---|
| DNN group | Experimental | using deep neural networks for skin lesion diagnosis Receives: Diagnostic Test: Model Dermatology (deep neural networks; Build 2020) |
| Control group | No intervention | conventional diagnosis |
What is being measured
The primary outcome is the main question the study is designed to answer.
Top-1 diagnostic accuracy
frequency of correct Top-1 prediction
Measured: 6 consecutive months
Secondary outcomes (3)
Top-2 and 3 diagnostic accuracy
frequency of correct Top-2 and 3 prediction
Measured: 6 consecutive months
Infection sensitivity
positive rate of infection diagnosis
Measured: 6 consecutive months
Malignancy sensitivity
Positive rate of malignancy diagnosis
Measured: 6 consecutive months
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted19 Nov 2020
- Study start27 Nov 2020
- Primary completion27 Nov 2021
- Study completion27 Dec 2021
Why it stopped: The rate of data collection was too slow.
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (7)Papers the sponsor has linked to this study.
- Other trials for Skin DiseasesStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Enrolment
- 55 participants (actual)
Sponsor and investigators
- Lead sponsor
- Pyoeng Gyun Choe
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor-investigator, Pyoeng Gyun Choe, Seoul National University Hospital
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT04636164
- Sponsor's protocol ID
- 2020-3233
- Status last verified
- Oct 2022
- First submitted
- 13 Nov 2020
- Last updated
- 27 Oct 2022
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Wang P, Liu X, Berzin TM, Glissen Brown JR, Liu P, Zhou C, Lei L, Li L, Guo Z, Lei S, Xiong F, Wang H, Song Y, Pan Y, Zhou G. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol. 2020 Apr;5(4):343-351. doi: 10.1016/S2468-1253(19)30411-X. Epub 2020 Jan 22.
PubMed 31981517Lin H, Li R, Liu Z, Chen J, Yang Y, Chen H, Lin Z, Lai W, Long E, Wu X, Lin D, Zhu Y, Chen C, Wu D, Yu T, Cao Q, Li X, Li J, Li W, Wang J, Yang M, Hu H, Zhang L, Yu Y, Chen X, Hu J, Zhu K, Jiang S, Huang Y, Tan G, Huang J, Lin X, Zhang X, Luo L, Liu Y, Liu X, Cheng B, Zheng D, Wu M, Chen W, Liu Y. Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial. EClinicalMedicine. 2019 Mar 17;9:52-59. doi: 10.1016/j.eclinm.2019.03.001. eCollection 2019 Mar.
PubMed 31143882Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S, Gupta V, Singh N, Natarajan V, Hofmann-Wellenhof R, Corrado GS, Peng LH, Webster DR, Ai D, Huang SJ, Liu Y, Dunn RC, Coz D. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020 Jun;26(6):900-908. doi: 10.1038/s41591-020-0842-3. Epub 2020 May 18.
PubMed 32424212Han SS, Park I, Eun Chang S, Lim W, Kim MS, Park GH, Chae JB, Huh CH, Na JI. Augmented Intelligence Dermatology: Deep Neural Networks Empower Medical Professionals in Diagnosing Skin Cancer and Predicting Treatment Options for 134 Skin Disorders. J Invest Dermatol. 2020 Sep;140(9):1753-1761. doi: 10.1016/j.jid.2020.01.019. Epub 2020 Mar 31.
PubMed 32243882Sellheyer K, Bergfeld WF. A retrospective biopsy study of the clinical diagnostic accuracy of common skin diseases by different specialties compared with dermatology. J Am Acad Dermatol. 2005 May;52(5):823-30. doi: 10.1016/j.jaad.2004.11.072.
PubMed 15858472Cui X, Wei R, Gong L, Qi R, Zhao Z, Chen H, Song K, Abdulrahman AAA, Wang Y, Chen JZS, Chen S, Zhao Y, Gao X. Assessing the effectiveness of artificial intelligence methods for melanoma: A retrospective review. J Am Acad Dermatol. 2019 Nov;81(5):1176-1180. doi: 10.1016/j.jaad.2019.06.042. Epub 2019 Jun 27.
PubMed 31255749Tschandl P, Codella N, Akay BN, Argenziano G, Braun RP, Cabo H, Gutman D, Halpern A, Helba B, Hofmann-Wellenhof R, Lallas A, Lapins J, Longo C, Malvehy J, Marchetti MA, Marghoob A, Menzies S, Oakley A, Paoli J, Puig S, Rinner C, Rosendahl C, Scope A, Sinz C, Soyer HP, Thomas L, Zalaudek I, Kittler H. Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol. 2019 Jul;20(7):938-947. doi: 10.1016/S1470-2045(19)30333-X. Epub 2019 Jun 12.
PubMed 31201137
Study at a glance
From the registry record, last updated by the sponsor on 27 Oct 2022. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Effect of Using Deep Neural Networks on the Accuracy of Skin Disease Diagnosis in Non-Dermatologist Physician
- ClinicalTrials.gov ID
- NCT04636164
- Acronym / sponsor ID
- 2020-3233
- Conditions
- Skin Diseases
- Keywords
- skin diseases, deep neural networks
- Study type
- Interventional
- Recruitment status
- TerminatedReason stopped: The rate of data collection was too slow.
- Enrolment
- 55 participants (actual)
- Lead sponsor
- Pyoeng Gyun ChoeUniversity, hospital or other
- Study start
- 27 Nov 2020 (actual)
- Primary completion
- 27 Nov 2021 (actual)
- Study completion
- 27 Dec 2021 (actual)
- First posted
- 19 Nov 2020 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Background: Deep neural networks (DNN) has been applied to many kinds of skin diseases in experimental settings.
Objective: The objective of this study is to confirm the augmentation of deep neural networks for the diagnosis of skin diseases in non-dermatologist physicians in a real-world setting.
Methods: A total of 40 non-dermatologist physicians in a single tertiary care hospital will be enrolled. They will be randomized to a DNN group and control group. By comparing two groups, the investigators will estimate the effect of using deep neural networks on the diagnosis of skin disease in terms of accuracy.
Detailed description
In the DNN group and control group, these steps are the same process.
- Routine exam and capture photographs of skin lesions for all eligible consecutive series patient.
- Make a clinical diagnosis (BEFORE-DX)
- Make a clinical diagnosis (AFTER-DX)
- consult to dermatologist
In the DNN group, after making the BEFORE-DX, physicians use deep neural networks and make an AFTER-DX considering the results of the deep neural networks (Model Dermatology, build 2020).
In the control group, after making the BEFORE-DX, physicians make an AFTER-DX after reviewing the pictures of skin lesions once more.
Ground truth will be based on the biopsy if available, or the consensus diagnosis of the dermatologists.
The investigators will compare the accuracy between the DNN group and control group after 6 consecutive months study.
Study design
- Primary purpose
- Diagnosis
- Allocation
- Randomised
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| DNN group | Experimental | Diagnostic Test: Model Dermatology (deep neural networks; Build 2020)using deep neural networks for skin lesion diagnosis |
| Control group | No intervention | –conventional diagnosis |
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (1 listed)
- non-dermatologist physician (residents) who agree to participate in this study
Key exclusion criteria (2 listed)
- dermatology residents
- non-dermatology residents who use other deep neural networks for skin lesion diagnosis
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Model Dermatology (deep neural networks; Build 2020)Physicians in the DNN group take pictures of the skin lesion and use the algorithm by uploading pictures. | Diagnostic test | DNN group |
Primary outcome
Top-1 diagnostic accuracy
Time frame: 6 consecutive months
frequency of correct Top-1 prediction
3 secondary outcomes
Top-2 and 3 diagnostic accuracy
Time frame: 6 consecutive months
Infection sensitivity
Time frame: 6 consecutive months
Malignancy sensitivity
Time frame: 6 consecutive months
Results and safety
The study was stopped early (reason given: The rate of data collection was too slow.). Results are not available in the source record.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 27 Oct 2022.
- Background: PubMed 31981517Wang P, Liu X, Berzin TM, Glissen Brown JR, Liu P, Zhou C, Lei L, Li L, Guo Z, Lei S, Xiong F, Wang H, Song Y, Pan Y, Zhou G. Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study. Lancet Gastroenterol Hepatol. 2020 Apr;5(4):343-351. doi: 10.1016/S2468-1253(19)30411-X. Epub 2020 Jan 22.
- Background: PubMed 31143882Lin H, Li R, Liu Z, Chen J, Yang Y, Chen H, Lin Z, Lai W, Long E, Wu X, Lin D, Zhu Y, Chen C, Wu D, Yu T, Cao Q, Li X, Li J, Li W, Wang J, Yang M, Hu H, Zhang L, Yu Y, Chen X, Hu J, Zhu K, Jiang S, Huang Y, Tan G, Huang J, Lin X, Zhang X, Luo L, Liu Y, Liu X, Cheng B, Zheng D, Wu M, Chen W, Liu Y. Diagnostic Efficacy and Therapeutic Decision-making Capacity of an Artificial Intelligence Platform for Childhood Cataracts in Eye Clinics: A Multicentre Randomized Controlled Trial. EClinicalMedicine. 2019 Mar 17;9:52-59. doi: 10.1016/j.eclinm.2019.03.001. eCollection 2019 Mar.
- Background: PubMed 32424212Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S, Gupta V, Singh N, Natarajan V, Hofmann-Wellenhof R, Corrado GS, Peng LH, Webster DR, Ai D, Huang SJ, Liu Y, Dunn RC, Coz D. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020 Jun;26(6):900-908. doi: 10.1038/s41591-020-0842-3. Epub 2020 May 18.
- Background: PubMed 32243882Han SS, Park I, Eun Chang S, Lim W, Kim MS, Park GH, Chae JB, Huh CH, Na JI. Augmented Intelligence Dermatology: Deep Neural Networks Empower Medical Professionals in Diagnosing Skin Cancer and Predicting Treatment Options for 134 Skin Disorders. J Invest Dermatol. 2020 Sep;140(9):1753-1761. doi: 10.1016/j.jid.2020.01.019. Epub 2020 Mar 31.
- Background: PubMed 15858472Sellheyer K, Bergfeld WF. A retrospective biopsy study of the clinical diagnostic accuracy of common skin diseases by different specialties compared with dermatology. J Am Acad Dermatol. 2005 May;52(5):823-30. doi: 10.1016/j.jaad.2004.11.072.
- Background: PubMed 31255749Cui X, Wei R, Gong L, Qi R, Zhao Z, Chen H, Song K, Abdulrahman AAA, Wang Y, Chen JZS, Chen S, Zhao Y, Gao X. Assessing the effectiveness of artificial intelligence methods for melanoma: A retrospective review. J Am Acad Dermatol. 2019 Nov;81(5):1176-1180. doi: 10.1016/j.jaad.2019.06.042. Epub 2019 Jun 27.
- Background: PubMed 31201137Tschandl P, Codella N, Akay BN, Argenziano G, Braun RP, Cabo H, Gutman D, Halpern A, Helba B, Hofmann-Wellenhof R, Lallas A, Lapins J, Longo C, Malvehy J, Marchetti MA, Marghoob A, Menzies S, Oakley A, Paoli J, Puig S, Rinner C, Rosendahl C, Scope A, Sinz C, Soyer HP, Thomas L, Zalaudek I, Kittler H. Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol. 2019 Jul;20(7):938-947. doi: 10.1016/S1470-2045(19)30333-X. Epub 2019 Jun 12.
- Individual participant dataSharing planned: No
Related trials for Skin Diseases
Not yet recruitingPhase 2
Extension Study of Safety and Tolerability of DDY391 in Participants With Moderate to Severe Psoriatic Arthritis (CDDY391A12201-extension)
Moderate to Severe Psoriatic Arthritis
RecruitingPhase 1/2
Study of INBRX-106 and INBRX-106 in Combination With Pembrolizumab (Keytruda®) in Subjects With Locally Advanced or Metastatic Solid Tumors (Hexavalent OX40 Agonist)
Solid Tumor, Non-Small Cell Lung Cancer
RecruitingPhase 1
IL-17 Blockade to Decrease irAEs (REPLAY)
Metastatic Melanoma
Active, not recruiting
Streamlined Geriatric and Oncological Evaluation Based On IC Technology
Breast Cancer, Comorbidities and Coexisting Conditions
RecruitingPhase 2
Gene Signatures to Guide HR+MBC Therapy in a Diverse Cohort
Invasive Mammary Carcinoma, Metastatic Breast Cancer
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
The record lists no contact. Use the official record to reach the sponsor.
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing Model Dermatology (deep neural networks; Build 2020) in people with Skin Diseases, run by Pyoeng Gyun Choe at 1 site in South Korea.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: stopped early, as of the sponsor's last update on 27 Oct 2022. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in South Korea. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
The study was stopped early (reason given: The rate of data collection was too slow.). Results are not available in the source record.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Top-1 diagnostic accuracy (time frame: 6 consecutive months). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT04636164, which also lists 7 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 27 Oct 2022.