Generative AI Radiologist's Workstation

This study aims to develop and validate a generative AI-assistant designed to optimize radiologists' workflow by automatically processing electronic medical records (EMRs) and generating structured clinical summaries. The AI tool will extract and prioritize relevant patient data to support accurate and efficient interpretation of diagnostic imaging studies.

The study rationale originates from increasing radiology workloads and the need to reduce time spent reviewing EMRs while maintaining diagnostic accuracy. The proposed AI solution specifically targets these issues through advanced natural language processing capabilities, with particular attention to optimizing time efficiency while maintaining or improving diagnostic accuracy.

The study consists of 9 Stages:

Stage 1: Theoretical Foundation.

1.1 Systematic review: comprehensive analysis of existing LLM applications in radiology.

1.2 Healthcare system analysis: evaluation of LLM implementations in clinical settings.

1.3 Expert consensus: semi-structured interviews with 30 practicing radiologists (stratified by experience: junior [<3 years], mid-career [3-10 years], senior [>10 years]) to establish:

• Minimum required clinical data elements;
• Optimal summary format (structured vs. narrative);
• Critical alert thresholds.

Stage 2: Technical Development.

2.1 Medical text processing: formalization of methods for extraction, standardization, and annotation.

2.2 Dataset Curation: methodology for creating representative training datasets from UMIAS (Unified Medical Information and Analytical System).

2.3 Validation Framework: creation of validation methodology for the generative AI-based assistant.

Development and validation of a questionnaire assessing:

• Relevance;
• Completeness (missing critical data);
• Hallucination frequency;
• Terminology/grammar;
• Radiologist satisfaction.

Stage 3: Dataset Development.

Data Source: Retrospective extraction of anonymized EMRs from UMIAS.

Inclusion Criteria:

• Age of the patient at the moment of medical image acquisition >18 years;
• Pathology types: pleuritis, ascites, unspecified masses/lesions, neuropathy;
• Imaging study modalities (performed between January 1, 2020, and May 31, 2025): CT chest (pleural/pulmonary pathologies); CT abdomen/pelvis (ascites/abdominal masses); MRI brain (neuropathy/neurological conditions);
• Complete EMR data (clinical notes, prior imaging reports, lab results, discharge summaries).

Exclusion Criteria:

Cases with technical artifacts on medical images compromising diagnostic quality.

Per-case data collected: physical examination results; two prior imaging reports (same modality) for progression assessment; three laboratory test results; consultation notes from three clinical specialists; discharge summaries; AI-Generated summaries (three summaries of different quality), including:

• High quality summary: complete, well-structured, clinically relevant;
• Medium quality summary: partial omissions, acceptable structure;
• Low quality summary: significant omissions/poor structure.

Stage 4: Comparative analysis of open-license generative AI architectures.

Stage 5: Model selection according to pre-defined selection criteria.

Stage 6: Model adaptation (fine-tuning and prompt optimization).

Stage 7: Development and UMIAS integration of a minimum viable product (MVP).

Stage 8: Pilot Testing.

Participants: 27 radiologists divided into three groups (A, B and C; n=9 each). Detailed description of each group is in section 'Groups and Interventions'. The group B will evaluate AI-summary quality via specially developed and validated questionnaire (scores: ≤8=low, 9-15=medium, >15=high).

In the end of pilot testing primary and secondary outcomes will be assessed.

Stage 9: Comparative analysis across all groups. Formulation of conclusions and assessment of the AI-assistant's applicability.