Emergency CareKINA-CONCT07725965
Artificial Intelligence(AI) in the Emergency Department (ED) in Cologne
An observational study of Emergency Care and Clinical Decision Support, run by University of Cologne at 1 site in Germany.
Trial summary
This retrospective, non-interventional study evaluates the prognostic performance of open-weight Large Language Models (LLMs) in the setting of a German academic emergency department. Read more about this study
- Who is it for?
- Any age
- Conditions the study is for
- Emergency Care, Clinical Decision Support, Triage
- How many people
- 100,000 participants
- Where
- 1 siteGermany
- Study dates
- 1 Jul 2026to 31 Dec 2027
- Phase
- Observational
- Are there results?
- Not yet: the study is under way
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- No healthy volunteers
- Child, Adult, Older adult
Study population: All consecutive treatment cases at the Central Emergency Department of University Hospital Cologne during 01 January 2023 - 31 December 2025 (full census, approximately 100,000 cases).
Who may be able to join
- All consecutive treatment cases at the Central Emergency Department of University Hospital Cologne during the period 01 January 2023 - 31 December 2025 (full census, consecutive inclusion).
Who cannot join
- Documented objection to the scientific use of the data pursuant to Art. 21 General data protection Regulation (GDPR).
- Cases lacking the minimum data required for analysis (triage/history and documented outcome).
Trial locations and contacts
1 site in 1 country.
Germany1 site
Department of Internal Medicine II, University Hospital Cologne
Cologne, 50937
About this study
Official title: Retrospective Validation of Large Language Models (LLM) for the Prognostic Assessment of Clinical Parameters in Emergency Department and Evaluation of the Impact of Automated Anonymisation Methods
This retrospective, non-interventional study evaluates the prognostic performance of open-weight Large Language Models (LLMs) in the setting of a German academic emergency department. Using a full census of all consecutive emergency department cases at University Hospital Cologne between 01 January 2023 and 31 December 2025 (approximately 100,000 cases), the study assesses whether LLMs can make reliable prognostic predictions (e.g., hospital admission, imaging, diagnosis, placement) based on the initial history, vital signs, and triage category. In addition, it quantifies how strongly automated anonymization and perturbation procedures affect the models' diagnostic accuracy. This is an Investigator-Initiated Trial (IIT) with no intervention on patients.
Read the detailed description
The study analyzes a retrospective cohort of all emergency department cases at the Central Emergency Department of University Hospital Cologne (01 January 2023 - 31 December 2025). Data originate from the hospital information system (HIS) and are provided in pseudonymized form via the Medical Data Integration Center (MeDIC) of University Hospital Cologne, acting as an independent trusted third party. Extracted data include sociodemographic data (age/year of birth, sex), clinical vital signs (blood pressure, heart rate, respiratory rate, oxygen saturation, temperature, GCS), medical free text (triage records, physician history and admission findings), and process/outcome data serving as the gold standard (ICD-10 diagnoses, imaging performed, admission status, timestamp/length of stay). LLM processing takes place on premise on local compute clusters or in a contractually secured enterprise environment with zero data retention; open-weight models are used. Two arms are compared: Arm A (original data) vs. Arm B (anonymized/perturbed/synthesized data). The primary endpoint is diagnostic accuracy (AUROC, F1 score) against the documented clinical outcome. Working hypotheses: (1) modern LLMs are non-inferior to the human assessment (non-inferiority); (2) modern anonymization procedures reduce model performance by less than 5% (relative performance loss).
- Conditions
- Emergency Care, Clinical Decision Support, Triage
- Keywords
- Emergency Care, Medical AI, Large Language Models, Triage, Anonymization, De-Identification, Clinical Decision Support, AI Real World Data Validation
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
| Study group | Role | What they receive |
|---|---|---|
| Analytic Arm A | Original data will be used for the analysis | |
| Analytic Arm B | Anonymized/perturbed data will be used for the analysis |
What is being measured
The primary outcome is the main question the study is designed to answer.
Diagnostic accuracy of the LLM predictions (AUROC, F1 score) compared with the clinical gold standard
assessment at the level of the individual emergency department encounter).]
Measured: From enrollment to the end of retrospective observation period at 1 year
Secondary outcomes (2)
Relative performance loss of the models between original data (Arm A) and anonymized/perturbed data (Arm B); hypothesis < 5%.
Relative loss of diagnostic accuracy (AUROC, F1 score) when the models are applied to anonymized/perturbed data (Arm B) compared with original data (Arm A), expressed as the relative percentage change. Non-inferiority is assumed if the relative performance loss is below 5%.]
Measured: From enrollment to the end of retrospective observation period at 1 year
Sensitivity, specificity, positive predictive value(PPV)/negative predictive value (NPV) for binary endpoints and agreement of the triage assessment (Cohen's kappa / Krippendorff's alpha).
Agreement between LLM output and the documented reference for binary endpoints (e.g., admission yes/no), reported as sensitivity, specificity, and positive/negative predictive value; agreement on the ordinal triage category is reported using Cohen's kappa or Krippendorff's alpha
Measured: From enrollment to the end of retrospective observation period at 1 year
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted24 Jul 2026
- Study start1 Jul 2026
- Primary completion31 Jul 2027 (estimated)
- Study completion31 Dec 2027 (estimated)
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Other trials for Emergency CareStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Other
- Time perspective
- Retrospective
- Enrolment
- 100,000 participants (estimated)
Sponsor and investigators
- Lead sponsor
- University of Cologne
- Sponsor type
- University, hospital or other
- Responsible party
- Principal investigator, Volker Burst, University of Cologne
- Investigators
- Volker Burst, Prof., Principal investigator, Department of Internal Medicine II, University Hospital Cologne
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT07725965
- Sponsor's protocol ID
- KINA-CO
- Other IDs
- 26-1064
- Acronym
- KINA-CO
- Status last verified
- Jul 2026
- First submitted
- 21 Jul 2026
- Last updated
- 28 Jul 2026
- Results posted
- No
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Study at a glance
From the registry record, last updated by the sponsor on 28 Jul 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Retrospective Validation of Large Language Models (LLM) for the Prognostic Assessment of Clinical Parameters in Emergency Department and Evaluation of the Impact of Automated Anonymisation Methods
- ClinicalTrials.gov ID
- NCT07725965
- Acronym / sponsor ID
- KINA-CO / KINA-CO
- Conditions
- Emergency Care, Clinical Decision Support, Triage
- Keywords
- Emergency Care, Medical AI, Large Language Models, Triage, Anonymization, De-Identification, Clinical Decision Support, AI Real World Data Validation
- Study type
- Observational
- Recruitment status
- Active, not recruiting
- Enrolment
- 100,000 participants (estimated)
- Lead sponsor
- University of CologneUniversity, hospital or other
- Study start
- 1 Jul 2026 (actual)
- Primary completion
- 31 Jul 2027 (estimated)
- Study completion
- 31 Dec 2027 (estimated)
- First posted
- 24 Jul 2026 (actual)
- Results
- Not posted in the source record
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
This retrospective, non-interventional study evaluates the prognostic performance of open-weight Large Language Models (LLMs) in the setting of a German academic emergency department. Using a full census of all consecutive emergency department cases at University Hospital Cologne between 01 January 2023 and 31 December 2025 (approximately 100,000 cases), the study assesses whether LLMs can make reliable prognostic predictions (e.g., hospital admission, imaging, diagnosis, placement) based on the initial history, vital signs, and triage category. In addition, it quantifies how strongly automated anonymization and perturbation procedures affect the models' diagnostic accuracy. This is an Investigator-Initiated Trial (IIT) with no intervention on patients.
Detailed description
The study analyzes a retrospective cohort of all emergency department cases at the Central Emergency Department of University Hospital Cologne (01 January 2023 - 31 December 2025). Data originate from the hospital information system (HIS) and are provided in pseudonymized form via the Medical Data Integration Center (MeDIC) of University Hospital Cologne, acting as an independent trusted third party. Extracted data include sociodemographic data (age/year of birth, sex), clinical vital signs (blood pressure, heart rate, respiratory rate, oxygen saturation, temperature, GCS), medical free text (triage records, physician history and admission findings), and process/outcome data serving as the gold standard (ICD-10 diagnoses, imaging performed, admission status, timestamp/length of stay). LLM processing takes place on premise on local compute clusters or in a contractually secured enterprise environment with zero data retention; open-weight models are used. Two arms are compared: Arm A (original data) vs. Arm B (anonymized/perturbed/synthesized data). The primary endpoint is diagnostic accuracy (AUROC, F1 score) against the documented clinical outcome. Working hypotheses: (1) modern LLMs are non-inferior to the human assessment (non-inferiority); (2) modern anonymization procedures reduce model performance by less than 5% (relative performance loss).
Study design
- Observational model
- Other
- Time perspective
- Retrospective
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Analytic Arm A | – | –Original data will be used for the analysis |
| Analytic Arm B | – | –Anonymized/perturbed data will be used for the analysis |
Who may qualify
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Child, Adult, Older adult
- Study population
- All consecutive treatment cases at the Central Emergency Department of University Hospital Cologne during 01 January 2023 - 31 December 2025 (full census, approximately 100,000 cases).
Key inclusion criteria (1 listed)
- All consecutive treatment cases at the Central Emergency Department of University Hospital Cologne during the period 01 January 2023 - 31 December 2025 (full census, consecutive inclusion).
Key exclusion criteria (2 listed)
- Documented objection to the scientific use of the data pursuant to Art. 21 General data protection Regulation (GDPR).
- Cases lacking the minimum data required for analysis (triage/history and documented outcome).
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Primary outcome
Diagnostic accuracy of the LLM predictions (AUROC, F1 score) compared with the clinical gold standard
Time frame: From enrollment to the end of retrospective observation period at 1 year
assessment at the level of the individual emergency department encounter).]
2 secondary outcomes
Relative performance loss of the models between original data (Arm A) and anonymized/perturbed data (Arm B); hypothesis < 5%.
Time frame: From enrollment to the end of retrospective observation period at 1 year
Sensitivity, specificity, positive predictive value(PPV)/negative predictive value (NPV) for binary endpoints and agreement of the triage assessment (Cohen's kappa / Krippendorff's alpha).
Time frame: From enrollment to the end of retrospective observation period at 1 year
Results and safety
This study is still under way. Results are not available in the source record yet.
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 28 Jul 2026.
- Individual participant dataSharing planned: No
Related trials for Emergency Care
Recruiting
The Impact of Telemedicine, Using the Caller's Mobile Phone Camera, on the Quality of Care Provided to Adult Patients Contacting the 15 Centre for Acute Dyspnea
Dyspnea
Active, not recruitingObservational study
Potential Impact of the Use of the Application VentilO
Mechanical Ventilation Complication
Not yet recruiting
Prevention and Rapid Outreach for Treatment, Engagement, and Care Transition From the Emergency Department
Opioid Overdose Prevention, HIV Prevention and Care
Recruiting
AI-Assisted Interpretation of Cardiac CT in the Emergency Department
Chest Pain, Acute Coronary Syndrome
Not yet recruitingObservational study
Comparison of IV Dexmedetomidine Versus Clonidine Versus no Alpha 2 Agonist on Emergency Delirium, PACU Length of Stay and Time to Hospital Discharge in Children Undergoing (Adeno)Tonsillectomy: a Prospective Observational Study
Emergency Delirium, Postoperative Delirium (POD)
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Volker Burst, Prof. principal investigator
Department of Internal Medicine II, University Hospital Cologne
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Emergency Care and Clinical Decision Support, run by University of Cologne at 1 site in Germany.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 2 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: under way, not looking for participants, as of the sponsor's last update on 28 Jul 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in Germany. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
This study is still under way. Results are not available in the source record yet.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Diagnostic accuracy of the LLM predictions (AUROC, F1 score) compared with the clinical gold standard (time frame: From enrollment to the end of retrospective observation period at 1 year). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 28 Jul 2026.