A Realist Evaluation of Autism ServiCe Delivery (RE-ASCeD)

Scientific Summary (following Rameses 2 reporting guidelines)

Study Title

A Realist Evaluation of Autism ServiCe Delivery (RE-ASCeD): Which diagnostic pathways work best, for whom, when, and at what cost?

Research Question Which autism diagnostic pathways work best, for whom, in what circumstances, and at what cost when providing timely and high quality diagnostic services?

Secondary Research Questions

• What factors underpin timely diagnosis service delivery success or failure?
• What needs to be in place to identify the needs and create individualized, holistic pathways of diagnosis for the child and their family?

Background The NHS England Long Term Plan states: "Children and young people with suspected autism wait too long before being provided with a diagnostic assessment. Over the next three years the investigators aim to test and implement the most effective ways to reduce waiting times…achieving timely diagnostic assessments in line with best practice guidelines...(and) to support children with autism and their families, throughout the diagnostic process." Set against a background of increasing demand for diagnostic assessment, and the need, as recommended by the NICE guidelines, for a multidisciplinary approach to assessment, taking at least 13 hours of professional time and £800 per child, evidence is needed to identify models of diagnostic assessment to achieve timely diagnoses, and how families can best be supported through this process. The NHSE Long Term Plan aims to reduce the wait time for diagnosis, and the aim of this project is to find ways/models to directly address that need.

Aims and Objectives This study aims to examine existing models of Autism diagnostic service delivery, and to investigate which approaches offer the most timely, cost-effective, high quality and child and family friendly solutions, and in line with realist thinking, to identify the factors that are likely to underpin their success, or failure, if rolled out more generally across providers.

Method

The study will use Realist Evaluation methodology, which enables evaluation of complex interventions in health care by building on existing research, to identify "what works, for whom and in what circumstances". Mixed methods, including Rapid Realist (literature) Review, survey of current practice, and case studies of centres delivering promising approaches, will be used to build a picture in an iterative manner, to identify underlying Contexts and Mechanisms that contribute to the desired Outcomes (CMO configurations) of improved Autism diagnostic service delivery and user experience, that in turn can provide plausible explanations of why the service models work or do not in certain circumstances. Data analysis from each phase of the study will be synthesised through a realist evaluative process comprising of four stages:

1. Articulation of programme theories and propositions from WP1 of the study (rapid realist review). Identification of candidate CMO configurations.
2. Data collection from WP4 (case studies) of the study to test and refine propositions.
3. Map the outcomes of each model across the cases; interrogate what contexts and mechanisms explain the pattern of outcomes.
4. Through stakeholder consolidation (WP5), refine explanatory CMO configurations to evaluate what works best and in what circumstances to deliver timely and cost-effective diagnostic services that meet the needs of children with possible Autism

Initial Programme Theory (Based on NICE principles) If there is an MDT assessment by a team with competencies in child neurodevelopment and mental health (context), then Autism will be recognised as a complex condition that relies on detailed history and observation across settings (mechanism) to diagnose it, leading to an accurate diagnosis, recognition of associated comorbidity such as ADHD and intellectual disability (outcome), and the ruling out of complex differential diagnoses.

This will also create - whilst not an explicit part of this project - accurate pictures of child strengths and needs to inform individualized packages of support and intervention through health, education and social care (outcome).

Work Package One (months 1-7). Rapid Realist Review. Evidence will be sought from published studies and evaluations of Autism diagnostic services, including controlled trials, uncontrolled studies, cost-effectiveness studies, process evaluations, grey literature and qualitative studies, including unpublished information about locally implemented Autism services. The purpose will be to refine or develop initial programme theories.

Work Package Two (months 1-6): National/International Survey of Current Diagnostic Practice: An electronic survey will be sent to UK CDCs, CAMHS teams, and tertiary centres, delivering Autism diagnostic services. This will ask teams to describe current challenges to, and any new models of diagnostic service delivery they are using. Analysis will use both qualitative and quantitative methods, to identify common themes, and services using novel approaches.

Work Package Three (months 7-9): Purposive Sampling Frame. The findings of WP 1 \& 2 will be used to develop a matrix of the key features of service delivery, plus other contextual factors such as lower SES, rural, urban etc. This will be used as a sampling frame. It is anticipated that 6-8 service delivery models will be purposefully selected as case studies across the matrix to ensure the investigators have a full range of delivery models and population characteristics.

Work Package Four: Case studies (months 10-29): In-depth exploration of individual service models will be carried out to determine the interrelationship of context, intervention, mechanisms and outcomes as they occur in their natural setting. Data will be gathered within case studies using mixed methods at two levels: (i) service model and (ii) individual child/ family. Methods will include semi structured realist Interviews, focus groups, and document reviews to assemble a comprehensive description of the service and model. Interviewees will include commissioners, service managers, clinicians, referrers, parents and young people. Interviews will be recorded and transcribed, proceeding to qualitative analysis based on components of the programme theory. Regular team meetings will discuss coding, address inconsistencies, and refine collective understanding of coding framework. Coded data will be collated using NVivo software.

For each case study the journeys of at last 58 children completing the diagnostic pathway will be recorded from case notes to measure pathway, hours of professional input and time to diagnosis. Costs incurred by families for example travel to clinic, loss of earnings, will be collected by interview. Associations between service use costs and characteristics of the child and model will be assessed by regression analysis.

Work Package Five: Consolidation event, Months 27-28 : The investigators will run two iterative national consensus workshops, inviting key stakeholders including NHSE and parents/young people, to refine and confirm the context-mechanism-outcome configurations, and salient actions required to deliver timely, reliable and costed diagnostic services that meet the needs of children with possible Autism. This will lead to dissemination of models to inform the ongoing development of service models within the NHSE Long-Term Plan, commissioning guidelines and local service development.