Gastrointestinal HemorrhageNCT05816473
Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance
A study testing LLM in people with Gastrointestinal Hemorrhage, run by Yale University at 1 site in United States.
Trial summary
The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Read more about this study
- Who is it for?
- Any ageHealthy volunteers can take part
- Conditions the study is for
- Gastrointestinal Hemorrhage
- What is being tested?
- LLM
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 108 participants
- Where
- 1 siteUnited States
- Study dates
- 23 May 2023to 31 Dec 2024
- Phase
- Interventional
- Are there results?
- Yes, posted in the registrySummary in the doctors' view
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages Any age
- All sexes
- Healthy volunteers accepted
- Child, Adult, Older adult
Who may be able to join
- Internal Medicine residency trainees at study institution
- Emergency Medicine residency trainees at study institution
Who cannot join
- N/A
Trial locations and contacts
1 site in 1 country.
United States1 site
Yale New Haven Hospital
New Haven, Connecticut, 06510
About this study
Official title: Artificial Intelligent Clinical Decision Support System Simulation Center Study: Trust and Usefulness of Machine Learning Risk Stratification Tool for Acute Gastrointestinal Bleeding Using the Technology Acceptance Model
The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.
Read the detailed description
The experiment will deploy a previously validated machine learning algorithm trained on existing clinical datasets within simulation scenarios in which a patient with acute gastrointestinal bleeding (at low, moderate, and high risk for poor outcome) is evaluated.
Prior to the simulation, a baseline educational module about artificial intelligence, machine learning, and clinical decision support will be provided to all participants. The investigators will establish psychological safety by detailing what is available in the room, the opportunity to call a consultant, and availability of laboratory and radiographic studies. Each clinical scenario will run for approximately 10 minutes based on real patient cases where vital signs change over time and laboratory values are made available at specific points in the assessment. The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability dashboard compared to the machine learning algorithm with interpretability dashboard alone. Each participant will receive three scenarios in randomized order of risk.
For the large language model interaction arm, participants will be provided the computer workstation a LLM chatbot interface of the algorithm and interpretability dashboard For the machine learning dashboard arm, participants will be provided the computer workstation with the algorithm and interpretability dashboard.
- Conditions
- Gastrointestinal Hemorrhage
- Being tested
- LLM
- Keywords
- Implementation science, Artificial intelligence, Decision Support Systems, Clinical, Simulation Training
- Medical subject headings
- Gastrointestinal Hemorrhage
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
LLMOther
Use of a Large Language Model (LLM) chatbot interface to Interact with the Machine Learning Algorithm and interpretability dashboard.
| Study group | Role | What they receive |
|---|---|---|
| Large Language Model-based Interaction | Experimental | LLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language. Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard. Receives: Other: LLM |
| Machine Learning Dashboard | No intervention | Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score. Participants will have access to the machine learning dashboard only. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT
The study will use a common set of dependent variables to assess baseline and post-intervention attitudes towards machine learning algorithms in clinical care using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey assessing perceived usefulness of the system, perceived ease of use, attitudes towards using it, behavioral intentions, and trust, measured with a 5-point Likert scale. Percent change in UTAUT survey response between Large Language Model-based Interaction and Machine Learning Dashboard at recruitment prior to administration of scenarios and immediately after completion of scenarios. The difference in time between the two will be approximately 60 minutes. Higher change indicates greater acceptance/intention to use the GutGPT+Dashboard.
Measured: Approximately 60 minutes
Secondary outcomes (1)
Clinician Decision Making of Triage of GI Bleeding
Mean percentage of decision accuracy per participant. Accuracy is defined as the percentage of times participants accurately choose the correct clinical decision for each simulation scenario of acute upper GI bleeding for each treatment condition. Immediately after completion of scenarios (60 minutes from initiation of study for each participant). No further follow up afterwards.
Measured: Approximately 60 minutes
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted18 Apr 2023
- Study start23 May 2023
- Primary completion31 Dec 2024
- Study completion31 Dec 2024
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Posted resultsParticipant flow, outcome measures and adverse events reported by the sponsor.
- Publications (5)Papers the sponsor has linked to this study.
- Other trials for Gastrointestinal HemorrhageStudies of the same condition, open and under way.
- More trials run by Yale UniversityThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Health services research
- Allocation
- Not applicable
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Enrolment
- 108 participants (actual)
Sponsor and investigators
- Lead sponsor
- Yale University
- Sponsor type
- University, hospital or other
- Responsible party
- Sponsor
- Investigators
- Dennis Shung, MD, Principal investigator, Yale School of Medicine Section of Digestive Diseases
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT05816473
- Sponsor's protocol ID
- 2000034521
- Other IDs
- 1K23DK125718-01A1
- Status last verified
- May 2026
- First submitted
- 13 Mar 2023
- Last updated
- 22 May 2026
- Results posted
- Yes, on ClinicalTrials.gov
- FDA-regulated drug
- No
- FDA-regulated device
- No
- Individual data sharing
- No
Publications
Laine L. Risk Assessment Tools for Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2016 Nov;14(11):1571-1573. doi: 10.1016/j.cgh.2016.08.003. Epub 2016 Aug 10. No abstract available.
PubMed 27521511Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012 Mar;107(3):345-60; quiz 361. doi: 10.1038/ajg.2011.480. Epub 2012 Feb 7.
PubMed 22310222Leonardi, P. M. 2009. Why do people reject new technologies and stymie organizational changes of which they are in favor? Exploring misalignments between social interactions and materiality. Human Communication Research, 35(3): 407-441.
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425-478
Chung S, Giuffre M, Rajashekar N, Pu Y, Shin YE, Kresevic S, Chan C, Nakamura-Sakai S, You K, Saarinen T, Hsiao A, Wong AH, Evans L, McCall T, Kizilcec RF, Sekhon J, Laine L, Shung DL. Usability and adoption in a randomized trial of GutGPT a GenAI tool for gastrointestinal bleeding. NPJ Digit Med. 2025 Aug 18;8(1):527. doi: 10.1038/s41746-025-01896-5.
PubMed 40825997
Study at a glance
From the registry record, last updated by the sponsor on 22 May 2026. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Artificial Intelligent Clinical Decision Support System Simulation Center Study: Trust and Usefulness of Machine Learning Risk Stratification Tool for Acute Gastrointestinal Bleeding Using the Technology Acceptance Model
- ClinicalTrials.gov ID
- NCT05816473
- Acronym / sponsor ID
- 2000034521
- Conditions
- Gastrointestinal Hemorrhage
- Keywords
- Implementation science, Artificial intelligence, Decision Support Systems, Clinical, Simulation Training
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 108 participants (actual)
- Lead sponsor
- Yale UniversityUniversity, hospital or other
- Collaborators
- National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)
- Study start
- 23 May 2023 (actual)
- Primary completion
- 31 Dec 2024 (actual)
- Study completion
- 31 Dec 2024 (actual)
- First posted
- 18 Apr 2023 (actual)
- Results
- Posted 22 May 2026
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.
Detailed description
The experiment will deploy a previously validated machine learning algorithm trained on existing clinical datasets within simulation scenarios in which a patient with acute gastrointestinal bleeding (at low, moderate, and high risk for poor outcome) is evaluated.
Prior to the simulation, a baseline educational module about artificial intelligence, machine learning, and clinical decision support will be provided to all participants. The investigators will establish psychological safety by detailing what is available in the room, the opportunity to call a consultant, and availability of laboratory and radiographic studies. Each clinical scenario will run for approximately 10 minutes based on real patient cases where vital signs change over time and laboratory values are made available at specific points in the assessment. The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability dashboard compared to the machine learning algorithm with interpretability dashboard alone. Each participant will receive three scenarios in randomized order of risk.
For the large language model interaction arm, participants will be provided the computer workstation a LLM chatbot interface of the algorithm and interpretability dashboard For the machine learning dashboard arm, participants will be provided the computer workstation with the algorithm and interpretability dashboard.
Study design
- Primary purpose
- Health services research
- Allocation
- Not applicable
- Intervention model
- Parallel groups
- Masking
- None (open label)
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| Large Language Model-based Interaction | Experimental | Other: LLMLLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language. Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard. |
| Machine Learning Dashboard | No intervention | –Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score. Participants will have access to the machine learning dashboard only. |
Who may qualify
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Child, Adult, Older adult
Key inclusion criteria (2 listed)
- Internal Medicine residency trainees at study institution
- Emergency Medicine residency trainees at study institution
Key exclusion criteria (1 listed)
- N/A
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| LLMUse of a Large Language Model (LLM) chatbot interface to Interact with the Machine Learning Algorithm and interpretability dashboard. | Other | Large Language Model-based Interaction |
Primary outcome
Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT
Time frame: Approximately 60 minutes
The study will use a common set of dependent variables to assess baseline and post-intervention attitudes towards machine learning algorithms in clinical care using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey assessing perceived usefulness of the system, perceived ease of use, attitudes towards using it, behavioral intentions, and trust, measured with a 5-point Likert scale. Percent change in UTAUT survey response between Large Language Model-based Interaction and Machine Learning Dashboard at recruitment prior to administration of scenarios and immediately after completion of scenarios. The difference in time between the two will be approximately 60 minutes. Higher change indicates greater acceptance/intention to use the GutGPT+Dashboard.
1 secondary outcome
Clinician Decision Making of Triage of GI Bleeding
Time frame: Approximately 60 minutes
Results and safety
Loading the posted results from ClinicalTrials.gov…
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 22 May 2026.
- Posted resultsParticipant flow, baseline, all outcome measures and adverse events.
- Background: PubMed 27521511Laine L. Risk Assessment Tools for Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2016 Nov;14(11):1571-1573. doi: 10.1016/j.cgh.2016.08.003. Epub 2016 Aug 10. No abstract available.
- Background: PubMed 22310222Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012 Mar;107(3):345-60; quiz 361. doi: 10.1038/ajg.2011.480. Epub 2012 Feb 7.
- BackgroundLeonardi, P. M. 2009. Why do people reject new technologies and stymie organizational changes of which they are in favor? Exploring misalignments between social interactions and materiality. Human Communication Research, 35(3): 407-441.
- BackgroundVenkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425-478
- Derived from this study: PubMed 40825997Chung S, Giuffre M, Rajashekar N, Pu Y, Shin YE, Kresevic S, Chan C, Nakamura-Sakai S, You K, Saarinen T, Hsiao A, Wong AH, Evans L, McCall T, Kizilcec RF, Sekhon J, Laine L, Shung DL. Usability and adoption in a randomized trial of GutGPT a GenAI tool for gastrointestinal bleeding. NPJ Digit Med. 2025 Aug 18;8(1):527. doi: 10.1038/s41746-025-01896-5.
- Individual participant dataSharing planned: No
Related trials for Gastrointestinal Hemorrhage
Recruiting
OTSC vs. Angiographic Embolization in Patients With Refractory Non-variceal Upper Gastrointestinal Bleeding
Upper Gastrointestinal Bleeding
RecruitingObservational study
Evaluation of the Clinical Utility of Endoscopic Closure Using a Dual Action Tissue (DAT) Clip
GI Bleeding, Perforation Colon
Enrolling by invitationObservational study
Case-Specific Health Care Professional Clinical Survey for Concerto Versa™ Detachable Coils
Arteriovenous Malformation, Gastrointestinal Bleed
Recruiting
Onyx™ Liquid Embolic IDE Clinical Study
Peripheral Arterial Hemorrhage, Trauma
Recruiting
Detection of Blood in the GI Lumen With HemoPill Acute
Upper GI Bleeding
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Dennis Shung, MD principal investigator
Yale School of Medicine Section of Digestive Diseases
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing LLM in people with Gastrointestinal Hemorrhage, run by Yale University at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 2 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 22 May 2026. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Yes. Results are posted in the registry (first posted 22 May 2026). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT (time frame: Approximately 60 minutes). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT05816473, which also lists 5 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 22 May 2026; AzTrials last refreshed its copy on 27 Oct 2026.