Quality ImprovementNCT00754377
Developing a Practice-Based Learning and Improvement Quality Improvement (QI) Systems Impact Assessment Questionnaire
An observational study of Quality Improvement, run by US Department of Veterans Affairs at 1 site in United States.
Trial summary
The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed practice-based learning and improvement (PBLI) -- a competency that is typically omitted from medical curriculum. Read more about this study
- Who is it for?
- Ages 25 to 75 yearsHealthy volunteers can take part
- Conditions the study is for
- Quality Improvement
- What is being tested?
- PBLI Curriculum
- How many people
- 92 participants
- Where
- 1 siteUnited States
- Study dates
- Jan 2008to Dec 2011
- Phase
- Observational
- Are there results?
- Yes, posted in the registrySummary in the doctors' view
Key requirements
Who can take part, as listed by the sponsor. Only the study team can confirm whether you are eligible, so take this list to your doctor.
- Ages 25 to 75 years
- All sexes
- Healthy volunteers accepted
- Adult, Older adult
Study population: Internal Medicine residents
Who may be able to join
- None, data being used has already been collected from a previous study which the inclusion criteria included all Internal Medicine and Medicine-Pediatrics residents completing a 4 week ambulatory block from 2005-2006 were required to participate in the PBLI curriculum to satisfy an ACGME's core competency.
Who cannot join
- None, the residents from 2004 that did not complete a 4 week ambulatory block and residents participating in ambulatory block 7 and 13 were excluded from the study previously Institutional Review Board (IRB) approved and exempted study. Blocks 7 and 13 are not structured to permit teaching.
Trial locations and contacts
1 site in 1 country.
United States1 site
Louis Stokes VA Medical Center, Cleveland, OH
Cleveland, Ohio, 44106
About this study
Official title: Developing a PBLI QI Systems Impact Assessment Questionnaire
The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed practice-based learning and improvement (PBLI) -- a competency that is typically omitted from medical curriculum. The goal is to have residents competent to investigate and evaluate their own patient care practices, integrate scientific evidence and be able to improve their practices. Available assessment tools do not adequately address all of the components of PBLI and few assessment tools attempt to capture the residents' ability to develop and implement clinically-based Continuous Quality Improvement (CQI) projects that involve the practice setting. Curriculums without such foci miss the importance of system perspectives and opportunities for interprofessional team development. Our aim is to evaluate preliminary data on the curriculum we developed to address the gaps, to develop an assessment tool, and to provide methods for assessing the sustainability of system projects.
The key component of the curriculum is the integration of system quality improvement projects. PBLI curriculum was offered on alternate rotations. Preliminary data is available from 6 PBLI QI Systems Curriculum blocks (n=50) and 5 comparison blocks (n=42). Data includes closed- and open-ended questions designed to assess resident PBLI application skills, the notes and presentation slides for the residents' presentation.
Read the detailed description
Physicians in training operate in complex healthcare delivery systems but many have not been equipped with the knowledge or skills to analyze clinical environments and continually improve patient care.[1] Instead, their training emphasizes the clinical management of individual patients. The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed one of two novel core competencies that are typically omitted from formal medical curriculum: practice-based learning and improvement (PBLI).[2,3] The ACGME's PBLI competency involves six points. The overall objective is to have residents competent to investigate and evaluate their own patient care practices, evaluate and integrate scientific evidence into their clinics and be able to improve their practices. However, the ACGME was not prescriptive about how to successfully implement and evaluate PBLI, and an established curriculum and validated assessment tool did not exist.[4] Efforts to date to develop an assessment tool provide a foundation, but none adequately addresses all of the components of PBLI as described by the ACGME. [3, 5-10] In particular, few assessment tools attempt to capture the residents' ability to develop and implement clinically-based CQI projects that involve the practice setting and assess impact on the practice setting and/or organization. Curriculums without such foci miss the importance of system perspectives, opportunities for interprofessional team development, and meaningful impacts on patient care at a broader organizational level.
After several iterations, a PBLI QI curriculum that addressed the gaps identified in many other curriculums was instituted. [11] The key component was the integration of system quality improvement projects that could evolve over several resident outpatient rotation blocks. Data from 11 blocks of the new curriculum is available for analysis. Data includes closed- and open-ended questions designed to assess resident PBLI application skills and the notes and presentation slides for the residents' presentation at the internal medicine's morbidity and mortality conference (IM MMC) at the end of each block. Funding would make it possible to more quickly evaluate the existing preliminary data. Our overall aim is to evaluate the preliminary data on the new curriculum and develop an assessment tool that addresses the six ACGME points and provides methods for assessing the sustainability of system projects. The next step will be to apply for a grant to more fully evaluate the curriculum and assessment tool.
- Conditions
- Quality Improvement
- Being tested
- PBLI Curriculum
- Keywords
- Education, Program Evaluation, Quality of Health Care, Medical Residents
What participants receive
What participants receive. In a randomised study each person is assigned to one group by chance.
PBLI CurriculumOther
PBLI curriculum was offered on alternate rotations
| Study group | Role | What they receive |
|---|---|---|
| PBLI Curriculum group | To evaluate preliminary data on a PBLI curriculum grounded on QI system projects. Receives: Other: PBLI Curriculum | |
| Comparison group | Received a different curriculum. |
What is being measured
The primary outcome is the main question the study is designed to answer.
Beliefs About Ability to Implement a CQI Project
Residents' belief about their ability to implement a CQI project was measured using a single efficacy item (values ranged from 1, strongly disagree, to 5, strongly agree). The item is from the Systems Quality Improvement Training and Assessment Tool. Differences (post minus pre) in this belief item were looked at with positive and higher difference values reflecting more positive change/increase in belief.
Measured: 1 month
Secondary outcomes (1)
Change in Knowledge Scores About Quality Improvement
Residents' knowledge was assessed using the knowledge scale (e.g., describe change concept, how a cause-effect diagram is created, elements of the improvement model) from the SQI TAT and scores could range from 0 to 54 points. Difference scores were used based on total score of the scale with larger positive values indicating more increase in knowledge.
Measured: 1 Month
Timeline
From the start of the study to its completion; the mark is today. Estimated dates are the sponsor’s plan.
- First posted18 Sep 2008 (estimated)
- Study startJan 2008
- Primary completionJul 2008
- Study completionDec 2011
Resources
- Official record on ClinicalTrials.govThe registry entry this page is built from, with its full change history.
- Posted resultsParticipant flow, outcome measures and adverse events reported by the sponsor.
- Publications (3)Papers the sponsor has linked to this study.
- Other trials for Quality ImprovementStudies of the same condition, open and under way.
- What the trial phases meanHow a treatment moves from first tests in people to approval.
More details
For doctors and researchers: how the study is designed, who runs it, and its registry IDs.
Study design
- Study type
- Observational
- Observational model
- Cohort
- Time perspective
- Retrospective
- Enrolment
- 92 participants (actual)
Sponsor and investigators
- Lead sponsor
- US Department of Veterans Affairs
- Sponsor type
- US federal agency
- Responsible party
- Sponsor
- Investigators
- David C Aron, MD MS, Principal investigator, Louis Stokes VA Medical Center, Cleveland, OH
IDs, dates and publications
- ClinicalTrials.gov ID
- NCT00754377
- Sponsor's protocol ID
- SHP 08-194
- Status last verified
- Apr 2015
- First submitted
- 16 Sep 2008
- Last updated
- 12 May 2015
- Results posted
- Yes, on ClinicalTrials.gov
- Data monitoring committee
- No
Publications
Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010 Jun 23;303(24):2479-85. doi: 10.1001/jama.2010.841.
PubMed 20571014Lawrence RH, Tomolo AM. Development and preliminary evaluation of a practice-based learning and improvement tool for assessing resident competence and guiding curriculum development. J Grad Med Educ. 2011 Mar;3(1):41-8. doi: 10.4300/JGME-D-10-00102.1.
PubMed 22379522Tomolo AM, Lawrence RH, Watts B, Augustine S, Aron DC, Singh MK. Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills. J Grad Med Educ. 2011 Mar;3(1):49-58. doi: 10.4300/JGME-D-10-00104.1.
PubMed 22379523
Study at a glance
From the registry record, last updated by the sponsor on 12 May 2015. Recruitment status is as the sponsor reported it: confirm with the study team.
- Official title
- Developing a PBLI QI Systems Impact Assessment Questionnaire
- ClinicalTrials.gov ID
- NCT00754377
- Acronym / sponsor ID
- SHP 08-194
- Conditions
- Quality Improvement
- Keywords
- Education, Program Evaluation, Quality of Health Care, Medical Residents
- Study type
- Observational
- Recruitment status
- Completed
- Enrolment
- 92 participants (actual)
- Lead sponsor
- US Department of Veterans AffairsUS federal agency
- Study start
- Jan 2008
- Primary completion
- Jul 2008 (actual)
- Study completion
- Dec 2011 (actual)
- First posted
- 18 Sep 2008 (estimated)
- Results
- Posted 14 Nov 2014
Clinical summary
The research question as the sponsor registered it. This describes what the study plans to test, not what it found.
The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed practice-based learning and improvement (PBLI) -- a competency that is typically omitted from medical curriculum. The goal is to have residents competent to investigate and evaluate their own patient care practices, integrate scientific evidence and be able to improve their practices. Available assessment tools do not adequately address all of the components of PBLI and few assessment tools attempt to capture the residents' ability to develop and implement clinically-based Continuous Quality Improvement (CQI) projects that involve the practice setting. Curriculums without such foci miss the importance of system perspectives and opportunities for interprofessional team development. Our aim is to evaluate preliminary data on the curriculum we developed to address the gaps, to develop an assessment tool, and to provide methods for assessing the sustainability of system projects.
The key component of the curriculum is the integration of system quality improvement projects. PBLI curriculum was offered on alternate rotations. Preliminary data is available from 6 PBLI QI Systems Curriculum blocks (n=50) and 5 comparison blocks (n=42). Data includes closed- and open-ended questions designed to assess resident PBLI application skills, the notes and presentation slides for the residents' presentation.
Detailed description
Physicians in training operate in complex healthcare delivery systems but many have not been equipped with the knowledge or skills to analyze clinical environments and continually improve patient care.[1] Instead, their training emphasizes the clinical management of individual patients. The Accreditation Council for Graduate Medical Education (ACGME) acknowledged the changing needs of physicians in training when it endorsed one of two novel core competencies that are typically omitted from formal medical curriculum: practice-based learning and improvement (PBLI).[2,3] The ACGME's PBLI competency involves six points. The overall objective is to have residents competent to investigate and evaluate their own patient care practices, evaluate and integrate scientific evidence into their clinics and be able to improve their practices. However, the ACGME was not prescriptive about how to successfully implement and evaluate PBLI, and an established curriculum and validated assessment tool did not exist.[4] Efforts to date to develop an assessment tool provide a foundation, but none adequately addresses all of the components of PBLI as described by the ACGME. [3, 5-10] In particular, few assessment tools attempt to capture the residents' ability to develop and implement clinically-based CQI projects that involve the practice setting and assess impact on the practice setting and/or organization. Curriculums without such foci miss the importance of system perspectives, opportunities for interprofessional team development, and meaningful impacts on patient care at a broader organizational level.
After several iterations, a PBLI QI curriculum that addressed the gaps identified in many other curriculums was instituted. [11] The key component was the integration of system quality improvement projects that could evolve over several resident outpatient rotation blocks. Data from 11 blocks of the new curriculum is available for analysis. Data includes closed- and open-ended questions designed to assess resident PBLI application skills and the notes and presentation slides for the residents' presentation at the internal medicine's morbidity and mortality conference (IM MMC) at the end of each block. Funding would make it possible to more quickly evaluate the existing preliminary data. Our overall aim is to evaluate the preliminary data on the new curriculum and develop an assessment tool that addresses the six ACGME points and provides methods for assessing the sustainability of system projects. The next step will be to apply for a grant to more fully evaluate the curriculum and assessment tool.
Study design
- Observational model
- Cohort
- Time perspective
- Retrospective
- Number of arms
- 2
| Arm | Type | Receives |
|---|---|---|
| PBLI Curriculum group | – | Other: PBLI CurriculumTo evaluate preliminary data on a PBLI curriculum grounded on QI system projects. |
| Comparison group | – | –Received a different curriculum. |
Who may qualify
- Ages
- 25 to 75 years
- Sex
- All
- Healthy volunteers
- Accepted
- Age groups
- Adult, Older adult
- Study population
- Internal Medicine residents
Key inclusion criteria (1 listed)
- None, data being used has already been collected from a previous study which the inclusion criteria included all Internal Medicine and Medicine-Pediatrics residents completing a 4 week ambulatory block from 2005-2006 were required to participate in the PBLI curriculum to satisfy an ACGME's core competency.
Key exclusion criteria (1 listed)
- None, the residents from 2004 that did not complete a 4 week ambulatory block and residents participating in ambulatory block 7 and 13 were excluded from the study previously Institutional Review Board (IRB) approved and exempted study. Blocks 7 and 13 are not structured to permit teaching.
Eligibility is decided by the study team after screening; this overview does not establish that a patient qualifies. Full inclusion and exclusion criteria.
Interventions
| Intervention | Type | Arms |
|---|---|---|
| PBLI CurriculumPBLI curriculum was offered on alternate rotations | Other | PBLI Curriculum group |
Primary outcome
Beliefs About Ability to Implement a CQI Project
Time frame: 1 month
Residents' belief about their ability to implement a CQI project was measured using a single efficacy item (values ranged from 1, strongly disagree, to 5, strongly agree). The item is from the Systems Quality Improvement Training and Assessment Tool. Differences (post minus pre) in this belief item were looked at with positive and higher difference values reflecting more positive change/increase in belief.
1 secondary outcome
Change in Knowledge Scores About Quality Improvement
Time frame: 1 Month
Results and safety
Loading the posted results from ClinicalTrials.gov…
Research links
- Official record on ClinicalTrials.govLast updated by the sponsor 12 May 2015.
- Posted resultsParticipant flow, baseline, all outcome measures and adverse events.
- Results publication: PubMed 20571014Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA. 2010 Jun 23;303(24):2479-85. doi: 10.1001/jama.2010.841.
- Results publication: PubMed 22379522Lawrence RH, Tomolo AM. Development and preliminary evaluation of a practice-based learning and improvement tool for assessing resident competence and guiding curriculum development. J Grad Med Educ. 2011 Mar;3(1):41-8. doi: 10.4300/JGME-D-10-00102.1.
- Results publication: PubMed 22379523Tomolo AM, Lawrence RH, Watts B, Augustine S, Aron DC, Singh MK. Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills. J Grad Med Educ. 2011 Mar;3(1):49-58. doi: 10.4300/JGME-D-10-00104.1.
Related trials for Quality Improvement
Not yet recruitingObservational study
Clinical Audit of Cervical Cytology Services in an Early Cancer Detection Unit
Cervical Intraepithelial Neoplasia, Uterine Cervical Neoplasms
Active, not recruiting
OR Video-Laryngoscope Evaluations
Video Laryngoscopy, Quality Improvement
Not yet recruiting
Multimodal Stewardship Programme to Improve Antibiotic Prescribing for Children With Lower Respiratory Tract Infections
Bronchitis, Acute, Community-acquired Pneumonia
Active, not recruitingObservational study
Focused Intensive Repeated Stepping During Inpatient Rehabilitation Poststroke - A Quality Improvement Project
Stroke, Acute, Gait, Hemiplegic
Not yet recruiting
Effects of Daily Avocado Consumption on Sleep and Skin Health in Postmenopausal Females
Poor Sleep Quality
Referral and contact
AzTrials does not enrol participants or take referrals. Contact the study team or a site listed below; they decide who can take part. This study is not recruiting at the moment.
David C Aron, MD MS principal investigator
Louis Stokes VA Medical Center, Cleveland, OH
1 site in 1 country: see every site and its status.
Questions about this trial
What is this clinical trial studying?
An observational study of Quality Improvement, run by US Department of Veterans Affairs at 1 site in United States.
The full registered description is in the About this study section and the official record.
Who may be eligible to take part?
The sponsor lists 1 inclusion and 1 exclusion criteria. Eligibility is decided by the study team after screening, so read the full criteria and contact a listed site.
Is this clinical trial currently recruiting?
The record says: finished, as of the sponsor's last update on 12 May 2015. Status can change between updates, so confirm with the study contact before making plans.
Where is the study taking place?
The record lists 1 site in United States. Whether a site is taking part right now should be confirmed with that site. See every site.
What are the possible benefits and risks?
They depend on the study and the treatment, and no study can promise a benefit. The study team explains the known risks during informed consent, before anyone joins; the record itself does not list them.
Are results available for this study?
Yes. Results are posted in the registry (first posted 14 Nov 2014). A summary is in the For doctors view, and the full results are on ClinicalTrials.gov.
How can a doctor refer a patient or contact the research team?
The record lists no contact. The official record names the sponsor, who can be asked directly. AzTrials does not enrol participants or take referrals.
What does the primary outcome measure?
Beliefs About Ability to Implement a CQI Project (time frame: 1 month). It is what the study was designed to measure, not a finding.
Where can I read the original study information?
On the ClinicalTrials.gov record for NCT00754377, which also lists 3 linked publications.
Does a listing mean the treatment is approved or proven to work?
No. A listing means the study is registered. It does not show that the treatment is safe, works, or is approved by a regulator.
How often is this information updated?
AzTrials copies changed records from ClinicalTrials.gov every night and reloads the whole registry every week. The sponsor last updated this record on 12 May 2015.