Dynamic CDSA to Manage Sick Children in Rwanda

Children are a well-recognized vulnerable population that still suffers from a high rate of acute infectious diseases and preventable deaths. This is especially true in fragile health systems of Sub-Saharan Africa, where under-five mortality is 10 times higher than in high-income countries. The management of sick children at the primary care level in these environments remains of insufficient quality as front-line clinicians lack appropriate diagnostics, supervision to improve their skills, and decision support tools. Clinically validated point-of-care (POC) diagnostic tests are often not available, and practice guidelines are quickly outdated by new evidence and changing epidemiology. When an epidemic arises, these static, generic guidelines can even become deleterious if the event is not detected on time and integrated into the recommendations.

In the absence of reliable guidance, health care workers (HCWs) tend to over-prescribe antibiotics (Hopkins et al. 2017, Fink et al. 2019). Approximately 9 out of 10 children at the primary care level in Tanzania receive an antibiotic, while only 1 in 10 needs one (D'Acremont et al. 2014). Inappropriate antibiotic use disrupts the gut flora, favoring the proliferation of pathogens and weakening a child's immune response (Benoun et al. 2016). It is also a major driver of antibiotic resistance, which is estimated to be responsible for up to 10 million deaths per year by 2050 (Holmes et al. 2016, Fink et al. 2019). Equally important to antibiotic overuse, is its underuse. Missing a child in need of antibiotic treatment or providing a child with an inappropriate type or dosage of antibiotic puts them at risk of preventable morbidity and death. The same occurs with antimalarials that are not always prescribed to the children in need: those with a positive malaria test result.

Misdiagnoses have consequences that reach beyond the patient. They increase re-attendance rates, further congesting primary health facilities and accruing economic losses not only for families but for the entire health system. Systematic errors in patient-level data accumulate, and as they are aggregated to measure population-level indicators, they have the potential to bias the statistics used to prioritize health interventions and, importantly, identify epidemics.

The WHO has identified digital health interventions and predictive tools in primary care as key accelerators in achieving the 2030 Sustainable Development Goal 3 of ensuring good health and well-being for all. New simple and cheap technologies, such as mobile devices, coupled with the advances in computing and data science, could help mitigate several of the aforementioned challenges. The proposed digital intervention is a third-generation clinical decision support algorithm (CDSA) intended to help HCWs at the primary care level manage children with acute illnesses. The first two versions of the algorithm have undergone rigorous evaluations in controlled research conditions as summarized below:

The first-generation algorithm called ALMANACH was tested in Tanzania in 2010-2011, achieving improved clinical cure (from 92% to 97%) and a decrease in antibiotic prescription (from 84% to 15%) as compared to routine care (Shao et al. 2015A). ALMANACH also led to more consistent clinical assessments without taking more time than a conventional consultation and was perceived by clinicians as "a powerful and useful" tool (Shao et al. 2015B).

The second-generation algorithm called ePOCT was trialed in Tanzania in 2014-2016. In addition to symptoms and signs, it made use of several POC tests to help detect children with severe infections requiring hospital-based treatment (oximetry and hemoglobin level) and/or children with serious bacterial infection (CRP). The use of ePOCT resulted in higher clinical cure (98%) as compared to ALMANACH (96%) and routine care (95%). The algorithm also further reduced antibiotic prescription to 11%, as compared to 30% with the use of ALMANACH and 95% in routine care (Keitel et al. 2017).

Electronic algorithms can thus be successfully implemented to improve clinical guidance and provide feedback to clinicians, as well as allow near-real-time analyses of data for M\&E of health interventions, disease surveillance and outbreak detection. The goal of this study is to improve clinical diagnosis, decrease morbidity and mortality of children, and mitigate antimicrobial resistance using novel dynamic POC technologies that help front-line HCWs manage sick patients, enhanced by smart disease surveillance and outbreak detection mechanisms.

More specifically, this study seeks to:

Objective 1: Improve the integrated management of children with an acute illness through the provision of an electronic CDSA (ePOCT+) to clinicians working at primary care level;

Objective 2: Improve the accuracy of the clinical algorithm and adapt it to spatiotemporal variations in epidemiology and resources, based on the data generated through the ePOCT+ tool, analyzed using machine learning and checked by clinical experts;

Objective 3: Enhance the health management information system for monitoring and evaluation and conducting supportive supervision (increased number of meaningful indicators that are reviewed and actioned regularly) in HFs using the clinical data generated by the ePOCT+ tool and enhanced by additional data analysis and visualization dashboards;

Objective 4: Create a framework for the development and implementation of dynamic CDSA and disease surveillance tools, for large-scale, sustainable, and clinically responsible use of machine learning and data science.

The primary intervention study will be conducted in two phases.

Phase 1: open-label two-arm parallel-group cluster non-randomized controlled superiority trial implemented in a staggered/sequential manner in a total of 32 health facilities. The intervention consists of ePOCT+ clinical decision support algorithm (CDSA) displayed on tablets (medAL-reader), point-of-care tests and devices that are not part of routine care (pulse oximeter, CRP rapid test, additional hemoglobin cuvettes), complementary training on the tool, regular monitoring and mentorship/supervision visits by the study team and/or the Douncil Health Management Team (DHMT). Mentorship and supervision will be enabled by a complementary dashboard (medAL-monitor), used to visualize and monitor study-related indicators. Due to the pragmatic nature of the study, the design is adaptive, in that changes in the implementation across the three cluster studies may be made based on monitoring data and feedback from the health facilities. These implementation changes (excluding significant adaptations to algorithm content) will be thoroughly documented and accounted for in comparing the effects across the three cluster studies.

Phase 2: scale-up of the intervention to more health facilities and transformation into a dynamic algorithm The ePOCT+ tool will be extended to the health facilities serving as controls in Phase 1, as well as to additional neighboring facilities of the target area, to reach a total of up to 40 facilities in Rwanda. In Phase 2, an adaptive study design will be used to measure the same outcome indicators as in Phase 1. The medical content of the algorithm will not be fixed anymore, but rather modifiable. Each potential modification will first be evaluated by the Tanzanian clinical expert group for its clinical coherence, safety and potential benefit and then applied to the retrospective data. If these analyses confirm both a clinically relevant positive impact and estimate that there will be sufficient future cases during the study period to detect this improvement, the change in the algorithm will be tested in a randomized sub-study using the same study design as in Phase 1, except that randomization will take place at patient level rather than health facility level. If the positive impact is confirmed in the sub-study, the modification will be implemented in all relevant locations/patient sub-groups.

Additional cross-sectional mixed-methods operational research studies will take place throughout the intervention period to study the implementation context, facilitators and barriers to the scale-up of this intervention and its integration into the primary health system of Rwanda.