Digital diagnoSis of Cardiac sOUNd in peDiatric Patients [DI-SOUND Study]

Introduction:

Congenital heart diseases (CHD) are the most common birth defects in humans. Timely diagnosis of cardiac structural abnormalities in newborns and children is associated with improved outcomes in the general pediatric population. Within CHD, ductal-dependent CHD are a rare group of cardiovascular malformation with heterogeneous anatomical features, sharing the inability to sustain either the pulmonary (ductal-dependent pulmonary circulation) or the systemic (ductal-dependent systemic circulation) circulation at the time of ductal closure. Examples of such condition are hypoplastic left heart syndrome (prevalence of 2/10.000), severe coarctation of the aorta (prevalence of 3/10.000), pulmonary atresia with intact ventricular septum (<1/10.000), critical neonatal aortic valve stenosis (prevalence ~5/10.000), critical pulmonary valve stenosis (1-5/10.000) and other rare more complex congenital lesions. In this cases, it is imperative to timely establish the correct diagnosis to ensure ductal patency through prostaglandin infusion and refer patient for care to tertiary pediatric cardiovascular centers.

Current newborn screening for CHD predominantly relies on brachial and lower extremity pulse oximetry screening (POS) and cardiac auscultation. Diagnostic performance of such practice is limited. POS is plagued by moderate sensitivity, in particular if performed during the first 24 hours of life. Cardiac auscultation is probably even more limited with sensitivity ranging between 75-85%.

Although prenatal and neonatal screening of CHD has been associated with increased recognition of disease in newborns a significant number of patients is not correctly identified and delayed diagnosis is still present in western and even more so in developing countries.

Digital elaboration of cardiac sounds with diagnostic purposes has been explored in the recent past in adults and older children. We propose to develop a dedicated software for automatic dichotomous clinical classification of heart sounds (normal versus abnormal) in newborns to improve neonatal recognition of structural heart disease in this population.

The Digital dIagnosis of cardiac SOUND in pediatric patients (DI_SOUND) study aims to develop and validate a tool with the overall goal of improving neonatal recognition of CHD.

Study Aims Aim 1: Develop a binary classifier for normal versus abnormal cardiac sounds in newborns Aim 2: Validate the binary classifier in a consecutive, independent cohort of newborns Aim 3: Cost-effective analysis of digital versus standard screening modality for CHD in newborns

Methods and study design Thisis a multicenter study and itwill be conducted in fourpediatriccardiologyprograms in Italy (IRCCS Azienda Ospedaliero-Universitaria di Bologna, IRCCS Ospedale Pediatrico Bambin Gesù in Roma, Azienda Ospedaliero-Universitaria Policlinico Umberto I in Roma and Ospedale Monaldi in Napoli) along with a Engineeringunit (Politecnico di Milano). The study is composed of two sequential phases: a derivation/training phase (binary classification algorithm development, Aim 1) and validation phase (binary classification algorithm validation and cost-effective analysis Aims 2 and 3) (Graphical Abstract). The study design is based on SPIRIT 2025 Guideline. IRB approval has been obtained by each clinical unit.

Study population:

Derivation phase. Study population will include neonates with known cardiovascular status including newbornswith and without CHD.

Validation phase. Study population will include newborns without previous cardiovascular examination and unknown cardiovascular status. Pre-test probability for CHD will be defined: high-risk newborn for structural cardiovascular abnormalities versus neonates at low-risk (general population risk level). High-risk sub-group will include newborns with existing fetal ultrasound suggesting cardiovascular abnormalities and those with clinical indication for pediatric cardiology evaluation (abnormal neonatal screening, signs/symptoms). Low- risk sub-group (approaching patient level prevalence for structural cardiovascular abnormalities) will include consecutive neonates specifically enrolled for such research aim and without any indication for cardiovascular examination.

Inclusion criteria

• Age < 30 days
• Signed informed consent obtained from parent(s) or representative(s) Exclusion criteria
• Inability to acquire a diagnostic echocardiogram
• Weight less than 1.5Kg

Study tool:

Cardiac sounds will be recorded using the Littmann Core Stethoscope (Eko Software) (3M Company, Minnesota/USA). Digital recordings will be stored as wav files in a dedicate encrypted platform for data sharing among centers.

Each newborn enrolled in the study will undergo a standardized and complete echocardiogram by experienced operators as reported in Table 1 and in the full Study Protocol submitted as Online Supplementary Material. Echocardiographic evaluation will be performed according to existing guidelines for neonatal echocardiography.28 Exams will be performed with standardized and reproducible approach (Table 1). They will be stored on a digital support. After anonymization, the exams will be transferred to theEchocardiographic Study Core Imaging Laboratory for formal assessment and adjudication.

A web-based, encrypted Case Report Form will be created using the institutional REDCap (Vanderbilt University) license of IRCCS Azienda Ospedaliero-Universitaria di Bologna.

Study procedure:

Patient enrollment For the derivation phase newborns without CHD will be enrolled at the time of discharge from Ob/Gyn program and newborns with CHD will be enrolled at the time of cardiovascular examination. For the validation phase consecutive newborns will be enrolled at the time of discharge from Ob/Gyn unit.

Graphical abstractsummarizes the study pipeline. Derivation phase After screening of eligible patients, written informed consent will be obtained from parents or caregivers.

The first echocardiographic evaluation will take place at a post natal age< 30days.Age at recording and echocardiographic evaluation will be planned not before 7 days of post-natal life to allow completion of proper cardio-circulatory transition in the healthy newborns (i.e. ductal closure, pulmonary vascular remodeling, foramen ovale physiologic shunt). The echocardiogram will be linked to an anonymous identifier which will be used to link the exam to the patient without breaching patient privacy. The examination will be digitally stored and transferred to the Echocardiography Study Core Lab for formal revision. The cardiovascular neonatal status will be appropriately labeled as being with or without cardiovascular abnormalities.

Heart sound will be digitally recorded using as acquisition device Littmann Core Stethoscope (Eko Software) (3M Company, Minnesota/USA). Cardiac sound tracings will be recorded and stored in wav format and encrypted transferred to the Bioengineer Research Unit for further elaboration. Cardiovascular neonatal status will be un-blinded to the Bioengineer Research Unit to allow for proper handling of classifier training.

The details of the acquisition method are as follows:

1. Acquisition device: Littmann Core Stethoscope (Eko Software) (3M Company, Minnesota/USA)
2. Position of the auscultation: mid precordium. Gentle pressure is applied on chest, spontaneous breathing, patients are kept as calm as possible using maternage without any sedation
3. Average recording duration: 15 seconds Phonocardiographic signals will be stored as wav files in an encrypted cloud system to be transferred for further analysis (Graphical abstract). Tracings will undergo preliminary processing to remove background noise, and will be segmented to generate sound samples of homogeneous suitable tuned duration. As the last step of the preprocessing, the filtered records will be normalized for preventing inhomogeneity in the extracted features. Data analysis will be then implemented based on handcrafted feature extraction. The main objective of feature extraction is to identify a small number of representative features, i.e., characteristic properties of a sound sample, that replace the high- dimensional raw signals still preserving its informative content with respect to the phenomenon under investigation. Relevant features are then fed to the classification algorithm to discriminate between healthy and abnormal subjects. Multiple different techniques are typically used for feature extraction in sound signals, and can be loosely grouped into frequency-domain techniques (e.g., Fast Fourier transform (FFT), Discrete Cosine Transform (DCT), Short Time Fourier transform (STFT)), time-frequency domain techniques (e.g., Linear Frequency Band Cepstral (LFBC), Mel Frequency Cepstrum Coefficients (MFCC) and linear predictive coding (LPC)), and time-domain techniques (e.g., zero-crossing detection and peak finding). Each of these techniques provides descriptors of potentially relevant properties of the sound sample, which can be used as features or further processed to extract higher level characteristics of the signal (such as systole and diastole variability). To maximize the robustness of our results to less-than-ideal acquisition conditions, we will train the classifiers using features from all the above-mentioned domains and will perform a feature selection procedure to select those features which are mostly correlated to the outcome of interest.We will consider as possible classifiers Logistic Regression, AdaBoost, XGBoost, Random Forest, Support Vector Machines and Hidden Markov Models. Neural networks may be implemented if appropriate and feasible.Following the standard best practice, the dataset will be split into training set, validation set and test data set. For evaluating the performance of classifiers, K-Fold cross-validation with different fold numbers will be used: 10-fold, 5-fold, and Leave- One-Out- Cross-Validation (LOOCV). In LOOCV, the number of folds is equal to number of records.Gini importance for Random Forest classifiers will be used as a tool to validate parameters with least predictive contribution.Once a proper classifier has been trained a post training analysis to determine the most relevant features will be conducted. The purpose of this analysis is to investigate to which extent each feature is contributing to the selection of one class with respect to the other. Techniques for feature relevance estimation based on SHAP values, or permutation analysis will be applied also with the aim of providing a proper explanation to the model decision (Graphical Abstract).

Validation phase Clinical software validation (Aim 2) After screening of eligible patients written informed consent will be obtained from parents or caregivers.

Clinical screening will be performed as mandated by the italian law. Pre-ductal (right arm) oxygen saturation and post-ductal (leg) oxygen saturation will be recored. Physical examination will include femoral arterial pulse detection and heart auscultation.

Clinical screening output threshold criteria is summarized in Table 2. For this phase we will enroll patients using a block stratification for high CHD risk versus low (standard) CHD risk status for each newborn.

Heart recordings will be acquired (30 seconds length) before comprehensive echocardiography by a trained research investigator. Age at recording and echocardiographic evaluation will be planned not before 7 days of post-natal life to allow completion of proper cardio-circulatory transition in the healthy newborns (i.e. ductal closure, pulmonary vascular remodeling, foramen ovale physiologic shunt).

The echocardiography will be performed according to current guidelines by expert pediatric cardiovascular imager blinded to clinical and digital screening results. Preliminary inter- and intra-observer variability between dedicated expert cardiovascular imagers among centers will be performed in small patient subset.

Pertinent anonymized demographics, cardiovascular status, and pertinent research variables will be stored in an encrypted, web-based, research-focused online software.

Digitally anonymized echocardiographic clips will be transferred to the Echocardiography Study Core Lab who will perform the final, formal reading of the exams dichotomizing patient population into normal and abnormal examination. Any potential conflict of interpretation will be solved with consensus and among all Units cardiac imagers.

Concordance/discordance pattern between patient-centered comparisons between digital and and clinical screening results will be performed using the echocardiography as primary modality of outcome ascertainment (Figure 1).

Cost-effectiveness validation (Aim 3) Aim 3 will explore potentials for direct effect on health care and expenditure containment of the digital screening compared to current practice. For this research aim the targeted population will be composed only of low CHD risk newborns without any indication for cardiovascular examination to approximate un-selected cardiovascular abnormalities prevalence in the general population.

The number of prevented un-necessary examinations using digital screening and the number of missed potentially necessary examination using digital screening will be modeled as outcome of interest. Disease related group national reimbursement will used to model marginal cost-effectiveness of neonatal digital screening using standard of care screening modality as comparison.

Statistical considerations:

Methods of data collection Patient-level data (demographics, obstetrical history, family history) will be collected at the time of digital acquisition of heart records, and stored (as previously indicated) in a customary, web-based, properly encrypted database with digital anonymization, by dedicated research personnel. Digital acquisition of heart records will be acquired as previously specified. Echocardiography will be performed by dedicated experienced pediatric cardiovascular imagers. A preliminary inter- and intra-observer variability for the echocardiographic evaluation will be run on 1% of total population (variability testing cohort).

Statistic plan Between-group comparisons for clinical and outcome variables will be performed using independent samples t-test, Wilcoxon rank sum test, chi-square analysis, or Fisher's exact test using appropriate variable-specific denominators.

For experimental design aim 1, limited data are available to build up a reliable power calculation at this stage, but we expect to enroll 600 patients with 2:1 diseased:healthy newborn ratio in this experimental phase.

Specific potential classifiers will be studied in the spectral features, cepstral features and time-domain features. Logistic regression, boosting algorithms (AdaBoost, XGBoost, Random Forest) will be used. Dataset will be split into training set, validation set and test data set.The performance of the classifiers will be evaluated using K-Fold cross-validation with different fold numbers will be used. Different folding levels will be tested: 10-fold, 5-fold, and Leave-One-Out- Cross-Validation (LOOCV), aiming for >95% sensitivity and >90% specificity.Gini importance for Random Forest classifiers will be used as a tool to validate parameters with least predictive contribution.

Gini importance can be defined as the probability of making a false classification of a randomly chosen record if it were randomly labeled in the class distribution. During the training of the decision trees of the random forest, each tree is a set of internal nodes and leaves. In the internal node, selected feature is used to make decision of dividing the dataset into two separate splits. Selection of the feature is done with Gini impurity criterion. The feature that brings the highest decrease of Gini impurity is selected for the internal nodes. Even after training, how each feature decreases the impurity can be calculated and accepted as the importance of that feature.

Net classification improvement will be used as the major statistical analysis for Experimental Aim 2, using 'standard of care' screening modality as comparison group. Sensitivity, specificity, positive and negative predictive values will be calculated, and likelihood ratio will be analyzed. C-statistics will be used to compute diagnostic performance of newly validated algorithm. Bootstrapping will be used to generate confidence intervals.

For experimental design aim 2, sample size calculation is being presented for sensitivity. Fixing alpha error to 0.05 and aiming for 0.9 power (1-beta error), ~1900 patients will be necessary to validate our software if disease prevalence is 0.01. Accordingly, an independent validation cohort of appropriate size will be consecutively screened.

Data will be reported as mean ±standard deviation, median (first and third quartile) or frequency (%). All tests were two-sided. A p-value <0.05 was considered significant. Standard statistica analysis will be performed using STATA® 17h Release data analysis software (StataCorp LP, College Station, TX).

Discussion The DI_SOUND study principal target is to develop and validate a digital classifier of newborn cardiac sounds to improve efficiency and effectiveness of neonatal CHD screening. Current clinical neonatal screening relies on arm and leg oxygen saturation reading along with physical examination by a neonatologist at the time of pre-discharge post-natal evaluation. Cardiovascular auscultation is historically a major component of neonatal cardiovascular screening, but it has limitations. Diagnostic performance of current neonatal cardiovascular screening protocols is limited. Potential factors affecting screening sensitivityinclude: a) pre-natal recognition of CHD that usually leads to central care of neonates with cardiovascular anomalies that may affect exposure to newborn with CHD by the general community of pediatric residents and junior pediatric care providers; b) early post-natal discharge (usually within 48-72 hours after birth) that may limit the acoustic correlates of large post-tricuspid shunt and complex CHD in the setting of residual high pulmonary vascular resistance and early circulatory transition; c) reduced ability to perform a physical examination in the setting of advanced medical technologies. Moreover, timely recognition of newborn with CHD is particularly problematic in developing countries, where general neonatal care is reduced and the availability of neonatologists is usually very limited. From a pure cost-effectiveness profile, neonatal recognition and appropriate care of critical (usually ductal-dependent) is very favorable because CHD treatment leads to a substantial gain of quality-adjusted life years even in developing countries with limited health care resources.

CHD screening has limited specificity as well. This usually generates a substantial number of un-necessary cardiovascular neonatal evaluations with significant health-care ineffective resource utilization, families anxiety, prolonged neonatal hospitalization.

We believe that a tool easily implementable in commercially available digital health care platforms may have a substantial positive impact in the management of newborns with CHD. Improving screening sensitivity will reduce the number of "false negative" screening results with a potential major impact on patient safety, prognosis, patient care, hospital logistics and this effect may be even more pronounced in low-income setting such as developing countries or remote areas with challenging access to tertiary referral centers, and longer transportation time.33 Better screening specificity will reduce the number of "false positive" results with positive effect on health-care cost, reduce family anxiety and newborn discomfort. More importantly, a digital classifier can be easily implemented into tele-medicine web-based platform optimizing screening scalability, penetration and real-world efficiency.

Limitation The most notable limitation is the absence of reliable data to calculate power for the development/training phase. Second, some severe CHD (among others transposition of the great arteries with intact ventricular septum) may have minimal acoustic correlates, and,therefore, our approach may be of limited efficiency for these lesions. Third, clinical screening modality integrates a variety of non-acoustic information, such as general newborn appearance, breathing pattern, femoral pulses, oxygen saturation. It is possible that clinical screening method may outperform digital screening for specific subsets of congenital lesions. Lastly, development and validation of diagnostic tool in a specific world region may constitute per se a limit affecting world-wide generalizability.

Conclusion The DI_SOUND study has the overall aim to transform cardiovascular newborn screening by leveraging technology to enhance diagnostic performance, promote worldwide penetration of telemonitoring screening, reduce health-care cost, improving patient safety. The study will prove if development and validation of a digital classifier of neonatal cardiac sounds is feasible and how this will impact clinical outcomes, patient care, hospital admissions and screening efficiency.