Machine Intelligence in the PharmacyNCT06245044
Preventing Medication Dispensing Errors in Pharmacy Practice With Interpretable Machine Intelligence
A study testing No MI Help, Scenario #1 and Scenario #2 in people with Machine Intelligence in the Pharmacy, run by University of Michigan at 1 site in United States.
Trial summary
Pharmacists currently perform an independent double-check to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. Read more about this study
- Who is it for?
- Ages 18 years and older
- Conditions the study is for
- Machine Intelligence in the Pharmacy
- What is being tested?
- No MI Help, Scenario #1 and Scenario #2
- Type of treatment
- Lifestyle or therapy programme
- Could I receive a placebo?
- No, everyone receives an active treatment
- How many people
- 68 participants
- Where
- 1 siteUnited States
- Study dates
- 11 Apr 2024to 4 Dec 2024
- Phase
- Interventional
- Are there results?
- Yes, posted in the registrySummary in the doctors' view
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 18 years and older
- All sexes
- No healthy volunteers
- Adult, Older adult
Who may be able to join
- Licensed pharmacist in the United States
- Age 18 years and older at screening
- PC/Laptop with Microsoft Windows 10 or Mac (Macbook, iMac) with MacOS with Google Chrome, Edge, Opera, Safari, or Firefox web browser installed on the device
- Screen resolution of 1024x968 pixels or more
- A laptop integrated webcam or USB webcam is also required for the eye tracking purpose.
Who cannot join
- Participated in Wave 1 or Wave 2
- Eyeglasses
- Uncorrected cataracts, intraocular implants, glaucoma, or permanently dilated pupil
- Require a screen reader/magnifier or other assistive technology to use the computer
- Eye movement or alignment abnormalities (lazy eye, strabismus, nystagmus)
Trial locations and contacts
1 site in 1 country.
About this study
Pharmacists currently perform an independent double-check to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. This research is being conducted to examine the effectiveness of the timing of machine intelligence (MI) advice on to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
Read the detailed description
Pharmacists currently perform an independent double-check currently to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. Instead, pharmacists rely solely on reference images of the medication which they can compare to the prescription vial contents. Previous research has shown that decision support systems can effectively improve healthcare delivery efficiency and accuracy, while preventing adverse drug events. However, little is known about how MI technologies impact pharmacists' work performance and cognitive demand.
To facilitate the long-term symbiotic relationship between the pharmacists and the MI system, proper trust needs to be established. While trust has been identified as the central factor for effective human-machine teaming, issues arise when humans place unjustified trust in automated technologies do not place enough trust in them. Over trust in automation can lead to complacency and automation bias. For instance, the pharmacists may rely on the MI system to the extent that they blindly accept any recommendation by the system. Under trust can result in pharmacist disuse and potential abandonment of the MI system.
Furthermore, little is known about the timing of the MI advice on pharmacists' work performance. For example, showing the MI's advice while the pharmacist is performing the medication verification task may yield different results than showing the MI's advice after the pharmacist made their decision.
The study investigators have developed a MI system for medication images classification. The objective of this study is to examine the effectiveness of the timing of MI advice to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
- Conditions
- Machine Intelligence in the Pharmacy
- Being tested
- No MI Help, Scenario #1, Scenario #2
- Keywords
- pharmacy, medication verification, artificial intelligence, machine intelligence
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
No MI HelpBehavioural
Participants will complete the medication verification task without any MI help
Scenario #1Behavioural
Participants will receive MI in the form of a pop-up message if their decision differs from the MI's determination.
Scenario #2Behavioural
MI help will be displayed concurrently with the filled and reference images.
| Study group | Role | What they receive |
|---|---|---|
| No MI Help | Experimental | No MI help will be presented during the verification tasks Receives: Behavioral: No MI Help; Behavioral: Scenario #1; Behavioral: Scenario #2 |
| Scenario #1 | Experimental | MI help will be presented in the form of a pop-up message the participant's decision differs from the MI's determination. Receives: Behavioral: No MI Help; Behavioral: Scenario #1; Behavioral: Scenario #2 |
| Scenario #2 | Experimental | MI help will be displayed concurrently with the filled and reference images. Receives: Behavioral: No MI Help; Behavioral: Scenario #1; Behavioral: Scenario #2 |
What is being measured
The primary outcome is the main question the study is designed to answer.
Reaction Time
Difference in task time measured by the number of seconds from starting the task to accepting or rejecting a medication image
Measured: Throughout the verification task
Decision Accuracy
Difference in detection rate measured by the number of medication verification errors across all participants in the Arm/Group.
Measured: Throughout the verification task
Trust Change
Participants will complete 100 mock medication verification trials in each of the study arms (i.e., Scenario 1, Scenario 2, and No Help). After each trial in Scenario 1 and Scenario 2, participants will use a visual analog scale (VAS) to respond to the question: "How much do you trust the AI advice?" The endpoints of the 100-point VAS are 'Not at all' to 'Completely trust'. Participants indicate their level of trust in the MI advice after every trial on a scale from 1-100, with higher scores indicating greater levels of trust.
The trust change, as measured by the visual analog scale, will be calculated using the following formula:
Trust change (i) = Trust(i) - Trust(i - 1), where i=2, 3, ..., 100.
To compute a single, summarized value for the Trust Change variable within a specific scenario, the individual Trust Change scores measured from the trials are averaged. This averaging method provides a comprehensive measure of how trust shifted across the duration of the scenario.
Measured: After every trial in Scenarios 1 and 2
Trust
Trust will be assessed using the Muir \& Moray's (1996) Trust in Automation scale. Scores range from 0 to 100 with higher scores indicating greater levels of trust.
Measured: Post-intervention in Scenarios 1 and 2.
Secondary outcomes (4)
Cognitive Effort
Participants' eye movements were tracked using a browser-based online eye tracking system. The outcome measure is the difference in cognitive effort as measured by fixation count in the defined areas of interest: fill image, reference image, or MI plot. Higher fixation rates indicate repeated interest in a certain area.
Measured: Throughout the verification task
Cognitive Effort
Participants' eye movements were tracked using a browser-based online eye tracking system. The outcome measure is the difference in cognitive effort as measured by the duration of fixations in the defined areas of interest: fill image, reference image, or MI plot. Longer fixation duration indicates a higher cognitive load.
Measured: Throughout the verification task
Workload
Participants will complete 100 mock medication verification trials in each of the 3 arms. The workload of each arm will be measured by the NASA Task Load Index (TLX). The 5 TLX dimensions assessed are: mental demand, effort, temporal demand, performance, and frustration. For each dimension, participants will indicate their response to a single question. For 4 of the dimensions, the endpoints of the Likert scale are 'very low' and 'very high'. The performance dimension is reverse-scored, and the endpoints are 'perfect' and 'failure'. Participants then complete 10 pairwise comparisons of the dimensions by indicating which dimension they consider to be a more important factor (e.g., effort vs frustration).
Each category score multiplied by its respective pairwise comparison count is summed and divided by 10 to get an overall weighted workload score. The result is an overall workload score between 1 and 20, with higher scores indicating higher workload.
Measured: After completing 100 mock verification trials in each arm
Usability
Participants will complete 100 mock medication verification trials in each of the 3 arms (No MI Help, Scenario 1, and Scenario 2). After completing 100 trials, participants will assess the mock verification interface using the System Usability Scale (SUS). The SUS is comprised of 10 statements that participants indicate their agreement with using a 5-point Likert scale ranging from strongly agree to strongly disagree. Odd-numbered questions have a positive response and even-numbered questions are reverse-scored. Scores are summed and multiplied by 2.5 to get a final SUS score. SUS scores range from 0 to 100 with higher scores indicating greater usability. An average SUS score is considered to be 68. Anything below 50 is "Not Acceptable. Scores between 51-70 are considered "Marginal", those above 71 are considered "Acceptable", and those at 80 or above are indicative of high usability.
Measured: After completing 100 mock verification trials in each arm
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted7 Feb 2024
- Study start11 Apr 2024
- Primary completion4 Dec 2024
- Study completion4 Dec 2024
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Posted resultsParticipant flow, outcome measures and adverse events reported by the sponsor.
- Other trials for Machine Intelligence in the PharmacyStudies of the same condition, open and under way.
- More trials run by University of MichiganThe sponsor's other registered studies.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Interventional
- Phase
- Phase N/A
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Crossover
- Masking
- None (open label)
- Enrolment
- 68 participants (actual)
Sponsor and investigators
- Lead sponsor
- University of Michigan
- Sponsor type
- University, hospital or other
- Collaborators
- National Library of Medicine (NLM)
- Responsible party
- Principal investigator, Corey Lester, University of Michigan
- Investigators
- Corey A Lester, PharmD, PhD, Principal investigator, University of Michigan
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT06245044
- Sponsor's protocol ID
- HUM00241223
- Other IDs
- 5R01LM013624
- Status last verified
- Sep 2025
- First submitted
- 5 Jan 2024
- Last updated
- 26 Nov 2025
- Results posted
- Yes, on ClinicalTrials.gov
- Data monitoring committee
- No
- FDA-regulated drug
- No
- FDA-regulated device
- No
Study at a glance
From the registry record, last updated by the sponsor on 26 Nov 2025. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Preventing Medication Dispensing Errors in Pharmacy Practice With Interpretable Machine Intelligence
- ClinicalTrials.gov ID
- NCT06245044
- Acronym / sponsor ID
- HUM00241223
- Conditions
- Machine Intelligence in the Pharmacy
- Keywords
- pharmacy, medication verification, artificial intelligence, machine intelligence
- Study type
- Interventional
- Recruitment status
- Completed
- Enrolment
- 68 participants (actual)
- Lead sponsor
- University of MichiganUniversity, hospital or other
- Collaborators
- National Library of Medicine (NLM)
- Study start
- 11 Apr 2024 (actual)
- Primary completion
- 4 Dec 2024 (actual)
- Study completion
- 4 Dec 2024 (actual)
- First posted
- 7 Feb 2024 (actual)
- Results
- Posted 26 Nov 2025
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
Pharmacists currently perform an independent double-check to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. This research is being conducted to examine the effectiveness of the timing of machine intelligence (MI) advice on to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
Detailed description
Pharmacists currently perform an independent double-check currently to identify drug-selection errors before they can reach the patient. However, the use of machine intelligence (MI) to support this cognitive decision-making work by pharmacists does not exist in practice. Instead, pharmacists rely solely on reference images of the medication which they can compare to the prescription vial contents. Previous research has shown that decision support systems can effectively improve healthcare delivery efficiency and accuracy, while preventing adverse drug events. However, little is known about how MI technologies impact pharmacists' work performance and cognitive demand.
To facilitate the long-term symbiotic relationship between the pharmacists and the MI system, proper trust needs to be established. While trust has been identified as the central factor for effective human-machine teaming, issues arise when humans place unjustified trust in automated technologies do not place enough trust in them. Over trust in automation can lead to complacency and automation bias. For instance, the pharmacists may rely on the MI system to the extent that they blindly accept any recommendation by the system. Under trust can result in pharmacist disuse and potential abandonment of the MI system.
Furthermore, little is known about the timing of the MI advice on pharmacists' work performance. For example, showing the MI's advice while the pharmacist is performing the medication verification task may yield different results than showing the MI's advice after the pharmacist made their decision.
The study investigators have developed a MI system for medication images classification. The objective of this study is to examine the effectiveness of the timing of MI advice to determine if it results in lower task time, increased accuracy, and increased trust in the MI.
Study design
- Primary purpose
- Other
- Allocation
- Randomised
- Intervention model
- Crossover
- Masking
- None (open label)
- Number of arms
- 3
| Arm | Type | Receives |
|---|---|---|
| No MI Help | Experimental | Behavioral: No MI Help, Behavioral: Scenario #1, Behavioral: Scenario #2No MI help will be presented during the verification tasks |
| Scenario #1 | Experimental | Behavioral: No MI Help, Behavioral: Scenario #1, Behavioral: Scenario #2MI help will be presented in the form of a pop-up message the participant's decision differs from the MI's determination. |
| Scenario #2 | Experimental | Behavioral: No MI Help, Behavioral: Scenario #1, Behavioral: Scenario #2MI help will be displayed concurrently with the filled and reference images. |
Who may qualify
- Ages
- 18 years and older
- Sex
- All
- Healthy volunteers
- Not accepted
- Age groups
- Adult, Older adult
Key inclusion criteria (5 listed)
- Licensed pharmacist in the United States
- Age 18 years and older at screening
- PC/Laptop with Microsoft Windows 10 or Mac (Macbook, iMac) with MacOS with Google Chrome, Edge, Opera, Safari, or Firefox web browser installed on the device
- Screen resolution of 1024x968 pixels or more
- A laptop integrated webcam or USB webcam is also required for the eye tracking purpose.
Key exclusion criteria (5 listed)
- Participated in Wave 1 or Wave 2
- Eyeglasses
- Uncorrected cataracts, intraocular implants, glaucoma, or permanently dilated pupil
- Require a screen reader/magnifier or other assistive technology to use the computer
- Eye movement or alignment abnormalities (lazy eye, strabismus, nystagmus)
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| No MI HelpParticipants will complete the medication verification task without any MI help | Behavioural | No MI Help, Scenario #1, Scenario #2 |
| Scenario #1Participants will receive MI in the form of a pop-up message if their decision differs from the MI's determination. | Behavioural | No MI Help, Scenario #1, Scenario #2 |
| Scenario #2MI help will be displayed concurrently with the filled and reference images. | Behavioural | No MI Help, Scenario #1, Scenario #2 |
Primary outcome
Reaction Time
Time frame: Throughout the verification task
Difference in task time measured by the number of seconds from starting the task to accepting or rejecting a medication image
Decision Accuracy
Time frame: Throughout the verification task
Difference in detection rate measured by the number of medication verification errors across all participants in the Arm/Group.
Trust Change
Time frame: After every trial in Scenarios 1 and 2
Participants will complete 100 mock medication verification trials in each of the study arms (i.e., Scenario 1, Scenario 2, and No Help). After each trial in Scenario 1 and Scenario 2, participants will use a visual analog scale (VAS) to respond to the question: "How much do you trust the AI advice?" The endpoints of the 100-point VAS are 'Not at all' to 'Completely trust'. Participants indicate their level of trust in the MI advice after every trial on a scale from 1-100, with higher scores indicating greater levels of trust. The trust change, as measured by the visual analog scale, will be calculated using the following formula: Trust change (i) = Trust(i) - Trust(i - 1), where i=2, 3, ..., 100. To compute a single, summarized value for the Trust Change variable within a specific scenario, the individual Trust Change scores measured from the trials are averaged. This averaging method provides a comprehensive measure of how trust shifted across the duration of the scenario.
Trust
Time frame: Post-intervention in Scenarios 1 and 2.
Trust will be assessed using the Muir \& Moray's (1996) Trust in Automation scale. Scores range from 0 to 100 with higher scores indicating greater levels of trust.
4 secondary outcomes
Cognitive Effort
Time frame: Throughout the verification task
Cognitive Effort
Time frame: Throughout the verification task
Workload
Time frame: After completing 100 mock verification trials in each arm
Usability
Time frame: After completing 100 mock verification trials in each arm
Results and safety
Loading the posted results from ClinicalTrials.gov…
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 26 Nov 2025.
- Posted resultsParticipant flow, baseline, all outcome measures and adverse events.
Related trials for Machine Intelligence in the Pharmacy
Recruiting
Evaluating an AI-Based Mobile Application for Chemotherapy Support in Breast Cancer Patients
Breast Cancer, Breast Neoplasm
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
Corey A Lester, PharmD, PhD principal investigator
University of Michigan
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
A study testing No MI Help, Scenario #1 and Scenario #2 in people with Machine Intelligence in the Pharmacy, run by University of Michigan at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 5 inclusion and 5 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 26 Nov 2025. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Yes. Results are posted in the registry (first posted 26 Nov 2025). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Reaction Time (time frame: Throughout the verification task). The study lists 4 primary outcomes in all. It is what the study was designed to measure, not a finding.
Where can I read the original study information?
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 26 Nov 2025.