Proximal Femoral FractureNCT06351943
Proximal Femur Image Database Validation
An observational study of Proximal Femoral Fracture, run by AO Innovation Translation Center at 1 site in Switzerland.
Trial summary
The AO@AI Turin project is a collaborative project with a Turin group and the AO (Arbeitsgemeinschaft für Osteosynthesefragen, or in English, Association for the Study of Internal Fixation) foundation. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Proximal Femoral Fracture
- What is being tested?
- Fracture classification annotations provided by the Turin group and Fracture classification annotations provided by the AO expert surgeon group
- How many people
- 2,932 participants
- Where
- 1 siteSwitzerland
- Study dates
- 1 May 2021to 30 Jun 2025
- Phase
- Observational
- Are there results?
- None posted in the registry
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Study population: Anonymized anteroposterior x-ray images in the Image database (DB). No patients will be enrolled for purposes of this study. The selection of 300 images for the pilot validation is random; therefore, the sampling method indicated below refers to this process of random selection of images from the Image DB.
Who may be able to join
- Not applicable.
- The study utilizes the anonymized images in the Image database (DB). No patients will be enrolled for purposes of this study.
Who cannot join
- Not applicable.
Trial locations and contacts
1 site in 1 country.
About this study
Official title: Validation of the Fracture Classification Accuracy (Ground Truth) of Anteroposterior X-ray of the Proximal Femur According to the Arbeitsgemeinschaft für Osteosynthesefragen/Orthopedic Trauma Association Classification Done by a Single Center: A Pilot Validation Study
The AO@AI Turin project is a collaborative project with a Turin group and the AO (Arbeitsgemeinschaft für Osteosynthesefragen, or in English, Association for the Study of Internal Fixation) foundation. An Image database (DB) has been built to host AP pelvic radiographs ready for artificial intelligence (AI) development.
The goal of this project is to determine the agreement between the Turin annotation of fracture status and the annotation from an external group of AO expert surgeons for a random subset of the Turin images.
Read the detailed description
The AO@AI Turin project is a collaborative project with a Turin group who has collected 2,932 anteroposterior (AP) pelvic radiographs, of which 1,811 are fracture images, and 1,121 are non-fracture images. The Turin group has developed an artificial intelligence (AI) algorithm for fracture classification using these images. These anonymized images (with all metadata or personal identifiers removed) have been uploaded to a cloud-based image database (DB) hosted and managed by the AO Foundation.
The Turin group has established the "ground truth" using the methods of "consensus by experts". Two radiologists from their medical team have reviewed and classified the fracture status (fracture vs non-fracture, and, if fracture, the AO/Orthopedic Trauma Association [OTA] classification).
The next step's goal is the ground truth validation plan to test the accuracy of the Turin annotation of fracture classification of the already uploaded AP pelvic images. This is to ensure that the image DB offers accurate quality annotations to allow AI development.
For the pilot phase, a random subset of the Turin images (300 of images) will be drawn from the image DB. These images will be reviewed by an external group of AO expert surgeons who will annotate the images per their fracture status, i.e., fracture vs non-fracture, and, if fracture, the AO/OTA classification.
The group of AO expert surgeons consists of four surgeons who will independently review the 300 images and a fifth surgeon who serves as an adjudicator if necessary. The expert surgeons will be given access to the 300 images via the cloud-based image DB and annotate the images. The expert surgeons will be blinded to the Turin annotations. The expert surgeons' annotations will be entered into a DB built for the purpose for the pilot study.
To determine the ground truth, the annotations of the four surgeons will be compared, and discrepancies will be identified. A meeting will then be arranged among the surgeons to resolve, by consensus, the discrepancies, with the potential involvement of the fifth surgeon as the adjudicator. After the resolution meeting, there will be a single set of annotations for the 300 images from the exert surgeon group.
The Turin annotations will also be entered into the study DB to allow comparisons with the expert surgeon group's annotation.
In case of disagreement between the Turin annotation and the AO expert surgeon annotations, a consensus will be sought to establish a new ground truth. If this process results in significant revisions to the annotations, the entire dataset will be reviewed to set this new standard. Following such a comprehensive dataset revision, the algorithm for automated fracture classification of the proximal femur, which has already been developed by the Turin group, will be re-trained. After re-training, the algorithm's performance will be evaluated through metrics such as precision, recall, and F1-score to ensure its accuracy and effectiveness in classifying proximal femur fractures.
- Conditions
- Proximal Femoral Fracture
- Being tested
- Fracture classification annotations provided by the Turin group, Fracture classification annotations provided by the AO expert surgeon group
- Keywords
- Artificial Intelligence, Hip Fractures, Radiographic Image Interpretation, Computer-Assisted, Reference Standards
- Medical subject headings
- Proximal Femoral Fractures, Hip Fractures
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
Fracture classification annotations provided by the Turin groupDiagnostic test
Fracture classification annotations provided by the Turin group: fracture vs non-fracture, and, if fracture, the Arbeitsgemeinschaft für Osteosynthesefragen (AO, in English, Association for the Study of Internal Fixation)/Orthopedic Trauma Association (OTA) classification.
Fracture classification annotations provided by the AO expert surgeon groupDiagnostic test
Fracture classification annotations provided by the AO expert surgeon group: fracture vs non-fracture, and, if fracture, the AO/OTA classification.
What is being measured
The primary outcome is the main question the study is designed to answer.
Annotations of fracture status of the image
Fracture status (fracture vs no fracture) classification
Measured: Day 0/Baseline
In case of fracture, Arbeitsgemeinschaft für Osteosynthesefragen (AO, in English, Association for the Study of Internal Fixation)/Orthopedic Trauma Association (OTA) classification: Type
AO/OTA classification: Type: 31A/31B/31C
Measured: Day 0/Baseline
In case of fracture, AO/OTA classification: Group
AO/OTA classification: Group: A1/A2/A3, B1/B2//B3, C1/C2
Measured: Day 0/Baseline
In case of fracture, AO/OTA classification: Subgroup
AO/OTA classification: Subgroup: A1.1/A1.2/A1.3/A2.2/A2.3/A3.1/A3.2/A3.3/B1.1/B1.2/B1.3/B2.1/B2.2/B2.3/C1.1/C1.2/C1.3/C2.1/C2.2/C2.3
Measured: Day 0/Baseline
In case of fracture, AO/OTA classification: Qualifier for 31A1.1
AO/OTA classification: Qualifier for 31A1.1: n/o
Measured: Day 0/Baseline
In case of fracture, AO/OTA classification: Qualifier for 31B2
AO/OTA classification: Qualifier for 31B2: p/q/r
Measured: Day 0/Baseline
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted8 Apr 2024
- Study start1 May 2021
- Primary completion1 Oct 2024 (estimated)
- Study completion30 Jun 2025 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Publications (4)Papers the sponsor has linked to this study.
- Other trials for Proximal Femoral FractureStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Other
- Time perspective
- Cross-sectional
- Enrolment
- 2,932 participants (estimated)
Sponsor and investigators
- Lead sponsor
- AO Innovation Translation Center
- Sponsor type
- University, hospital or other
- Collaborators
- University of Turin, Italy
- Responsible party
- Sponsor
- Investigators
- Alessandro Aprato, MD, Principal investigator, University of Turin, Italy
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06351943
- Sponsor's protocol ID
- ImageDB pilot
- Status last verified
- Mar 2024
- First submitted
- 26 Mar 2024
- Last updated
- 8 Apr 2024
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- Undecided
Publications
Tanzi L, Vezzetti E, Moreno R, Aprato A, Audisio A, Masse A. Hierarchical fracture classification of proximal femur X-Ray images using a multistage Deep Learning approach. Eur J Radiol. 2020 Dec;133:109373. doi: 10.1016/j.ejrad.2020.109373. Epub 2020 Oct 23.
PubMed 33126175Audige L, Bhandari M, Hanson B, Kellam J. A concept for the validation of fracture classifications. J Orthop Trauma. 2005 Jul;19(6):401-6. doi: 10.1097/01.bot.0000155310.04886.37.
PubMed 16003200Langerhuizen DWG, Janssen SJ, Mallee WH, van den Bekerom MPJ, Ring D, Kerkhoffs GMMJ, Jaarsma RL, Doornberg JN. What Are the Applications and Limitations of Artificial Intelligence for Fracture Detection and Classification in Orthopaedic Trauma Imaging? A Systematic Review. Clin Orthop Relat Res. 2019 Nov;477(11):2482-2491. doi: 10.1097/CORR.0000000000000848.
PubMed 31283727Meinberg EG, Agel J, Roberts CS, Karam MD, Kellam JF. Fracture and Dislocation Classification Compendium-2018. J Orthop Trauma. 2018 Jan;32 Suppl 1:S1-S170. doi: 10.1097/BOT.0000000000001063. No abstract available.
PubMed 29256945
Study at a glance
From the registry record, last updated by the sponsor on 8 Apr 2024. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Validation of the Fracture Classification Accuracy (Ground Truth) of Anteroposterior X-ray of the Proximal Femur According to the Arbeitsgemeinschaft für Osteosynthesefragen/Orthopedic Trauma Association Classification Done by a Single Center: A Pilot Validation Study
- ClinicalTrials.gov ID
- NCT06351943
- Acronym / sponsor ID
- ImageDB pilot
- Conditions
- Proximal Femoral Fracture
- Keywords
- Artificial Intelligence, Hip Fractures, Radiographic Image Interpretation, Computer-Assisted, Reference Standards
- Study type
- Observational
- Recruitment status
- Unknown status
- Enrolment
- 2,932 participants (estimated)
- Lead sponsor
- AO Innovation Translation CenterUniversity, hospital or other
- Collaborators
- University of Turin, Italy
- Study start
- 1 May 2021 (actual)
- Primary completion
- 1 Oct 2024 (estimated)
- Study completion
- 30 Jun 2025 (estimated)
- First posted
- 8 Apr 2024 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The AO@AI Turin project is a collaborative project with a Turin group and the AO (Arbeitsgemeinschaft für Osteosynthesefragen, or in English, Association for the Study of Internal Fixation) foundation. An Image database (DB) has been built to host AP pelvic radiographs ready for artificial intelligence (AI) development.
The goal of this project is to determine the agreement between the Turin annotation of fracture status and the annotation from an external group of AO expert surgeons for a random subset of the Turin images.
Detailed description
The AO@AI Turin project is a collaborative project with a Turin group who has collected 2,932 anteroposterior (AP) pelvic radiographs, of which 1,811 are fracture images, and 1,121 are non-fracture images. The Turin group has developed an artificial intelligence (AI) algorithm for fracture classification using these images. These anonymized images (with all metadata or personal identifiers removed) have been uploaded to a cloud-based image database (DB) hosted and managed by the AO Foundation.
The Turin group has established the "ground truth" using the methods of "consensus by experts". Two radiologists from their medical team have reviewed and classified the fracture status (fracture vs non-fracture, and, if fracture, the AO/Orthopedic Trauma Association [OTA] classification).
The next step's goal is the ground truth validation plan to test the accuracy of the Turin annotation of fracture classification of the already uploaded AP pelvic images. This is to ensure that the image DB offers accurate quality annotations to allow AI development.
For the pilot phase, a random subset of the Turin images (300 of images) will be drawn from the image DB. These images will be reviewed by an external group of AO expert surgeons who will annotate the images per their fracture status, i.e., fracture vs non-fracture, and, if fracture, the AO/OTA classification.
The group of AO expert surgeons consists of four surgeons who will independently review the 300 images and a fifth surgeon who serves as an adjudicator if necessary. The expert surgeons will be given access to the 300 images via the cloud-based image DB and annotate the images. The expert surgeons will be blinded to the Turin annotations. The expert surgeons' annotations will be entered into a DB built for the purpose for the pilot study.
To determine the ground truth, the annotations of the four surgeons will be compared, and discrepancies will be identified. A meeting will then be arranged among the surgeons to resolve, by consensus, the discrepancies, with the potential involvement of the fifth surgeon as the adjudicator. After the resolution meeting, there will be a single set of annotations for the 300 images from the exert surgeon group.
The Turin annotations will also be entered into the study DB to allow comparisons with the expert surgeon group's annotation.
In case of disagreement between the Turin annotation and the AO expert surgeon annotations, a consensus will be sought to establish a new ground truth. If this process results in significant revisions to the annotations, the entire dataset will be reviewed to set this new standard. Following such a comprehensive dataset revision, the algorithm for automated fracture classification of the proximal femur, which has already been developed by the Turin group, will be re-trained. After re-training, the algorithm's performance will be evaluated through metrics such as precision, recall, and F1-score to ensure its accuracy and effectiveness in classifying proximal femur fractures.
Study design
- Observational model
- Other
- Time perspective
- Cross-sectional
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
- Study population
- Anonymized anteroposterior x-ray images in the Image database (DB). No patients will be enrolled for purposes of this study. The selection of 300 images for the pilot validation is random; therefore, the sampling method indicated below refers to this process of random selection of images from the Image DB.
Key inclusion criteria (2 listed)
- Not applicable.
- The study utilizes the anonymized images in the Image database (DB). No patients will be enrolled for purposes of this study.
Key exclusion criteria (1 listed)
- Not applicable.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| Fracture classification annotations provided by the Turin groupFracture classification annotations provided by the Turin group: fracture vs non-fracture, and, if fracture, the Arbeitsgemeinschaft für Osteosynthesefragen (AO, in English, Association for the Study of Internal Fixation)/Orthopedic Trauma Association (OTA) classification. | Diagnostic test | – |
| Fracture classification annotations provided by the AO expert surgeon groupFracture classification annotations provided by the AO expert surgeon group: fracture vs non-fracture, and, if fracture, the AO/OTA classification. | Diagnostic test | – |
Primary outcome
Annotations of fracture status of the image
Time frame: Day 0/Baseline
Fracture status (fracture vs no fracture) classification
In case of fracture, Arbeitsgemeinschaft für Osteosynthesefragen (AO, in English, Association for the Study of Internal Fixation)/Orthopedic Trauma Association (OTA) classification: Type
Time frame: Day 0/Baseline
AO/OTA classification: Type: 31A/31B/31C
In case of fracture, AO/OTA classification: Group
Time frame: Day 0/Baseline
AO/OTA classification: Group: A1/A2/A3, B1/B2//B3, C1/C2
In case of fracture, AO/OTA classification: Subgroup
Time frame: Day 0/Baseline
AO/OTA classification: Subgroup: A1.1/A1.2/A1.3/A2.2/A2.3/A3.1/A3.2/A3.3/B1.1/B1.2/B1.3/B2.1/B2.2/B2.3/C1.1/C1.2/C1.3/C2.1/C2.2/C2.3
In case of fracture, AO/OTA classification: Qualifier for 31A1.1
Time frame: Day 0/Baseline
AO/OTA classification: Qualifier for 31A1.1: n/o
In case of fracture, AO/OTA classification: Qualifier for 31B2
Time frame: Day 0/Baseline
AO/OTA classification: Qualifier for 31B2: p/q/r
Results and safety
Results are not available in the source record.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 8 Apr 2024.
- Results publication: PubMed 33126175Tanzi L, Vezzetti E, Moreno R, Aprato A, Audisio A, Masse A. Hierarchical fracture classification of proximal femur X-Ray images using a multistage Deep Learning approach. Eur J Radiol. 2020 Dec;133:109373. doi: 10.1016/j.ejrad.2020.109373. Epub 2020 Oct 23.
- Results publication: PubMed 16003200Audige L, Bhandari M, Hanson B, Kellam J. A concept for the validation of fracture classifications. J Orthop Trauma. 2005 Jul;19(6):401-6. doi: 10.1097/01.bot.0000155310.04886.37.
- Results publication: PubMed 31283727Langerhuizen DWG, Janssen SJ, Mallee WH, van den Bekerom MPJ, Ring D, Kerkhoffs GMMJ, Jaarsma RL, Doornberg JN. What Are the Applications and Limitations of Artificial Intelligence for Fracture Detection and Classification in Orthopaedic Trauma Imaging? A Systematic Review. Clin Orthop Relat Res. 2019 Nov;477(11):2482-2491. doi: 10.1097/CORR.0000000000000848.
- Results publication: PubMed 29256945Meinberg EG, Agel J, Roberts CS, Karam MD, Kellam JF. Fracture and Dislocation Classification Compendium-2018. J Orthop Trauma. 2018 Jan;32 Suppl 1:S1-S170. doi: 10.1097/BOT.0000000000001063. No abstract available.
- Individual participant dataSharing planned: Undecided
Related trials for Proximal Femoral Fracture
Not yet recruiting
X-BS Ortho Bone Graft Study
Orthopedic Disorder, Trauma Injury
Recruiting
A Study to Verify if the Surgery for the Proximal Femur Fracture Can be Improved With the Help of a Navigation System
Femoral Fracture
RecruitingPhase 2
CITADEL: Effectiveness of CITicoline in Preventing Cognitive Decline After DELirium
Cognitive Decline, Delirium
Recruiting
Pericapsular Nerve Group Block Versus Femoral Nerve Block Study
Hip Fractures
Active, not recruiting
A Randomized Trial to Assess Patient Quality of Life and Function After Alternative Surgeries for Pathologic Fractures of the Femur
Proximal Femoral Metastases
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Alessandro Aprato, MD principal investigator
University of Turin, Italy
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Proximal Femoral Fracture, run by AO Innovation Translation Center at 1 site in Switzerland.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: status not confirmed, as of the sponsor's last update on 8 Apr 2024. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Switzerland. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Results are not available in the source record.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Annotations of fracture status of the image (time frame: Day 0/Baseline). The study lists 6 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT06351943, which also lists 4 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 8 Apr 2024.