Statistical analysis plan for the ARtificially Intelligent image fusion system versus standard treatment to guide endovascular Aortic aneurysm repair (ARIA): a multi-centre randomised controlled trial

Abstract

Background

Aortic aneurysms, a significant cause of mortality, particularly in individuals aged 55 years and older, have witnessed a transformative shift in treatment strategies with the advent of endovascular surgery. Cydar-EV is an innovative image fusion technology that can augment preoperative planning and surgical guidance of endovascular aneurysm repair (EVAR). The ARIA trial aims to evaluate the efficacy of using Cydar-EV with EVAR procedures to reduce operating time while enhancing procedural precision, patient outcomes, and cost-effectiveness. This paper describes the statistical analysis plan for the study.

Methods/design

The ARIA trial, a phase III, multi-centre, open-label, two-armed, parallel groups randomised controlled surgical trial, seeks to recruit 340 patients diagnosed with abdominal or thoraco-abdominal aortic aneurysms. Participants are randomly assigned to receive either standard endovascular repair or an endovascular repair assisted by Cydar-EV for planning and surgical guidance. Primary and secondary outcomes are assessed at baseline, 4–12 weeks, and 52 weeks. The primary outcome measure is procedure duration at baseline, while additional secondary outcomes are recorded at various time points and include indicators for technical effectiveness, patient outcomes, procedure efficiency, and cost-effectiveness. We plan to analyse the patient outcome data according to the treatment they received regardless of initial allocation. The statistical analysis plan outlines methods for handling missing data, covariates for adjusted analyses, and planned sensitivity analyses to ensure robust evaluation of treatment effects.

Trial registration

The trial was registered with the ISRCTN register on 03/12/2021, number ISRCTN13832085.

Background

Aortic aneurysm, the second most prevalent condition affecting the aorta after atherosclerosis, stands as the fifteenth leading cause of death among individuals aged 55 years and older and nineteenth overall. The advent of endovascular surgery marks a significant shift towards minimally invasive procedures. Endovascular aneurysm repair (EVAR) has rapidly gained precedence over open aortic surgery, driven by perceived advantages in patient survival, reduced postoperative complications, and shorter hospital stays [1].

The pre-operative planning of EVAR surgery involves the use of computed tomography (CT) scans to reconstruct 3D images for assessing access and determining the optimal type, configuration, and sizing of the implantable medical device. During surgery, a 2D X-ray fluoroscopy, coupled with the injection of nephrotoxic contrast material, is employed to visualise blood vessels and guide the procedure. However, device positioning errors may still occur resulting in variability in patient outcomes which in many cases necessitates secondary interventions [2]. In addition, imprecise positioning can lead to serious and even fatal complications raising concerns about cost-effectiveness [3, 4].

Previous solutions to enhance visualisation during EVAR have included manually aligned, operating table-tracked 3D-2D image overlay. This technology, available in from GE, Siemens, and Philips, can be deemed too costly as it necessitates design, installation, and maintenance of a hybrid operating room. A survey across 10 US centres revealed that although equipped with 3D overlay capabilities, clinicians refrained from its use due to disruptions in clinical workflow and clinically significant image positioning errors [5]. In contrast, Cydar-EV image fusion, a CE-marked medical device, employs computer vision to automatically and in real-time fuse pre-procedural 3D images with intra-operative 2D fluoroscopy. Unlike other methods, Cydar-EV provides surgeons with real-time fully integrated 3D visualisation throughout the EVAR procedure. The computer vision, a form of artificial intelligence utilising NHS digital-approved, GDPR-compliant high-performance cloud computing, relies solely on existing patient data without introducing new imaging. Importantly, it seamlessly integrates into existing clinical workflows, requires no user interaction, involves no additional ionising radiation or iodinated contrast, and is compatible with both fixed and mobile X-ray systems.

The ‘ARIA’ (ARtificially Intelligent image fusion system versus standard treatment to guide endovascular Aortic aneurysm repair) trial is a multi-centre randomised controlled trial designed to compare the operation time of patients undergoing EVAR. The study aims to determine whether the implementation of Cydar-EV during EVAR offers superiority over the current standard procedure, without Cydar-EV, in terms of achieving a shorter operation time, reduced costs, and enhanced procedural efficiency, while maintaining equally effective technical outcomes and positive patient results. Here we outline the statistical analysis plan intended for generating the main results of the ARIA trial. This plan has been finalised while the data collection is still ongoing.

Study design

The ARIA trial protocol has been published previously [6]. In brief, ARIA is a multi-centre, open label, two-armed, parallel groups randomised controlled surgical trial comparing Cydar-EV to conventional surgical procedure. Patients are recruited from 10 sites across the UK. After obtaining written informed consent and confirming eligibility, participants undergoing endovascular aortic aneurysm repair are being randomly assigned to procedures with or without Cydar-EV for planning and surgical guidance.

Randomisation method

The randomisation, performed at the patient level, follows a 1:1 ratio and utilises minimisation methods. The King’s Clinical Trials Unit (KCTU) at King’s College London manages the randomisation process through a web-based bespoke randomisation system. Minimisation factors include surgeon, procedure urgency (emergency or elective), and procedure type (simple or complex)—a simple procedure involves the repair of infra-renal aneurysm ± internal iliac embolisation, while a complex procedure encompasses all other types of AAA and TAAA repair to include branched and fenestrated devices.

Intervention description

Patients will undergo endovascular aneurysm repair that is pre-operatively planned and intra-operatively guided by Cydar-EV, which provides tools to:

• Import and visualise CT data

• Segment and annotate vascular anatomy from CT data

• Place and edit virtual guidewires and measure lengths on them

• Make measurements of anatomical structures on planar sections of the CT data

• Produce an operative plan from measurements and segmentation of preoperative vessel anatomy

• Overlay planning information such as preoperative vessel anatomy onto live fluoroscopic images, aligned based on the position of anatomical features present in both

• Non-rigidly transform the visualisation of anatomy when intra-operative vessel deformation is observed

• Post-operatively review data relating to procedures where the system was used

Comparator description

Patients will undergo endovascular aneurysm repair using standard planning technology and X-ray fluoroscopy imaging during the procedure.

Primary outcome definition

The primary efficacy parameter for the study is procedure duration for endovascular repair, measured at baseline surgery. This duration is defined as the time in minutes from the insertion of the first wire, initiated after percutaneous access is achieved (if applicable), at the start of the endovascular procedure to the last frame of the completion angiogram.

Secondary outcomes definitions

The main secondary outcome measures are outlined in Table 1, with additional outcomes detailed in Appendix 1. These outcomes include assessments of procedure efficiency, technical success, length of admission and hospital stay, and other patient outcomes.

Table 1 Secondary outcomes

Main hypothesis

For individuals diagnosed with abdominal or thoraco-abdominal aortic aneurysm (AAA/TAAA), incorporating Cydar-EV in the planning and surgical guidance during endovascular repair is associated with a reduction in procedure time when compared to the standard surgical treatment without Cydar-EV.

Secondary hypotheses

Performing endovascular repair with Cydar-EV reduces costs and improves procedural efficiency, technical effectiveness, and patient outcomes, compared to standard procedures conducted without Cydar-EV.

Sample size

The sample size for the study is 340 participants. This number of participants is required to detect a difference in procedure time of 22.5 min between the two arms with 90% power and a two-sided significance level of 5%, assuming a t-test for ratio of means 1.2 (fold change), with a lognormal distribution for the calculations. This sample size allows for a 7.5% attrition rate.

Trial duration

The trial duration per participant is 52 weeks from baseline treatment to the last visit. Evaluation occurs pre-discharge, followed by two subsequent follow-up visits at weeks 4–12 and week 52. Table 2 outlines the scheduled events for each participant from screening to the final visit.

Table 2 Schedule of events

General considerations

Analysis populations

All valid assessments from every randomised patient will be included or excluded in the analysis populations defined below:

• i. Per-protocol population (PP).

  The majority of the analyses, including those for the primary and secondary endpoints, will be conducted based on the PP population. This population consists of all subjects who adhered to the trial protocol, by undergoing endovascular treatment according to the initial allocation. The PP population will include participants with partial data as well as any patients who have withdrawn from the trial provided they have consented for the continued use of their data. Any subject who did not receive the treatment randomly allocated to them will be excluded.

• ii. Safety population.

  The safety population includes all individuals from the PP population, without excluding those who may have received a different treatment than originally assigned. This population will be used to report summary statistics on non-serious adverse events (NSAEs) and serious adverse events (SAEs), with results presented according to the treatment actually received rather than the treatment assigned at randomisation.

• iii. Intention-to-treat population (ITT).

  The ITT population includes all participants who were randomised into the trial, analysed according to their originally assigned group regardless of whether they completed the treatment as planned, switched treatments, or dropped out. This analysis method preserves the benefits of randomisation by reflecting real-world conditions where perfect adherence to treatment is not always possible. In this trial, ITT analysis will be conducted as a sensitivity analysis for the primary outcome, specifically to assess the impact of treatment switches or misallocations. The ITT approach will provide a conservative perspective on the effectiveness of Cydar-EV, offering valuable insights into how the treatment performs in less controlled, everyday settings.

Levels of confidence and P values

We will employ two-sided statistical tests and provide estimates of treatment effect with 95% confidence intervals. Significance will be assessed at the 5% level.

Correction for multiplicity

Only one primary outcome will be evaluated in this trial, while all other outcomes are considered hypothesis-generating. As a result, we have used the specified level of confidence as an acceptable risk of type I error.

Protocol violations and exclusions from the study

Any deviations from the protocol and exclusions, along with reasons for such exclusions, will be documented for each trial arm. Additionally, we will conduct an ITT analysis of the primary outcome to explore the possible impact of such violations.

Unit of analysis

The unit of analysis is the patient.

Unadjusted and adjusted analyses

For each outcome variable, we will provide unadjusted and adjusted estimates. The primary analysis will be the covariate-adjusted analysis, which incorporates the randomisation (minimisation) factors, as outlined earlier.

Missing data

Non-response will be reported wherever present. The proportion of participants with missing data for each outcome will be summarised for each arm and at each time point. The main analysis for the primary outcome will utilise data at surgery (i.e. baseline stage). In the event of missingness in such data, a worst-case best-case sensitivity analysis will be performed (as described later). All findings will be presented in adherence to the most recent CONSORT statement guidelines.

Interim analyses and stopping rules

In this study, the treatment is given at baseline and it is not possible to subsequently stop treatment for a given participant. Therefore, there will be no planned formal interim analyses of the primary and secondary outcomes. The DMC will examine the recruitment rate, data completeness and monitor safety, and will recommend whether the study should continue, stop, be suspended, or be modified, based on their findings. If necessary for urgent safety reasons the sponsor may stop or pause the trial immediately, without DMC review.

Start of data analysis

The main analysis will begin once the final randomised patient has completed the first follow-up visit (week 4–12). The results will be published in an open-source peer-reviewed medical journal and will include procedure time and secondary outcome data available up to that point. A subsequent analysis will be conducted and published after all participants have reached the 52-week follow-up, and the data has been cleaned and locked. This analysis will include the remaining outcomes and a comprehensive safety evaluation.

Proposed analyses

Trial profile

The study’s recruitment, randomisation, and follow-up will be summarised for each arm in a CONSORT flow diagram. This visual representation will include the number of screened participants, the count of consenting and eligible patients, and, within each treatment arm, the breakdown of compliant and non-compliant participants (assessed by whether a patient received the randomised treatment or not). Additionally, it will include figures for those continuing through the trial, withdrawals at each time point, and individuals lost to follow-up at each visit. Furthermore, the diagram will highlight the numbers excluded and those included in the final analysis.

Baseline

The baseline characteristics will include patient demographics, randomisation (minimisation) stratifiers, medical history, intraoperative processes, and other baseline clinical measures. This will allow an assessment of whether there is clinically important imbalance in any variables.

Baseline characteristics of each group will be summarised as mean and standard deviation for continuous variables with median and interquartile range for highly skewed data, and count and percentage for categorical variables. These summaries will be based on complete data only and the number of missing observations will be reported. No significance testing will be performed.

Primary outcome

Anticipating a potentially skewed distribution in the primary outcome (procedure time), normalisation will be considered if necessary and feasible, utilising an appropriate transformation. Linear regression techniques will then be applied for the analysis of procedure duration. This analysis will include stratification (minimisation) factors as covariates. If a suitable transformation cannot be identified, quantile regression will be employed. This approach allows for the incorporation of stratification factors as covariates while accommodating the unique characteristics of the data distribution.

Secondary outcomes

Secondary outcomes, including those defined in Appendix 1, will be analysed using the appropriate regression methods for the type of data. Continuous measures will be analysed as for the primary outcome (described above), whereas binary outcomes will be compared between arms using logistic regression adjusting for stratification factors. Results from all secondary analyses will be presented as adjusted differences in means (or median) or odds ratios, as appropriate, with confidence intervals. Such analyses will be treated as exploratory.

Safety evaluation

We will summarise adverse events (AE), serious adverse events (SAE), and important medical events (IME) as counts and percentages for each trial arm. Furthermore, we will provide a summary of event types, intensity, and their relationship to the study intervention.

Analysis of missing data

We will assess potential bias due to missing data by conducting a descriptive comparison of the baseline characteristics between trial participants with complete measurements for the primary outcome and those with incomplete or no outcome data. Additionally, we will provide a summary of the reasons for withdrawal and non-compliance to the allocated treatment. For further insights into treatment compliance, we will undertake a descriptive comparison of baseline characteristics between patients who adhere to their treatment allocation and those who do not.

Sensitivity analyses

First, we will perform analyses that account for variables associated with missing data. Utilising logistic regression analyses with the presence or absence of missing data as the outcome, we aim to identify predictors of missingness. If these predictors are linked to both missing data and the primary outcome, we will then refit the primary analysis model by incorporating these predictors. Second, a worst-case and best-case scenario analysis will be executed. This involves imputing missing data in the treatment arm (Cydar-EV group) with the longest observed operation time from the overall sample, while replacing missing data in the control arm with the shortest observed operation time, and vice versa. Moreover, we will explore the option of conducting analyses using the ‘intention-to-treat’ principle, providing an additional perspective on the impact of treatment allocation on the outcomes.

Health economic analyses

As outlined in the trial protocol, health economic analyses will be conducted [6]. The specifics of such analyses are detailed in a separate document.

Trial status

The statistical analysis plan is based on the published protocol (version 1.3, 24.05.2023) [6]. Recruitment started in May 2022 and is now finalised. The collection of follow-up data will be completed in May 2025.

Appendix 1. Additional analyses

• Image analysis—analysis of technical outcome

CT image data acquired pre-operatively and at the two post-operative intervals will be uploaded to the system for analysis. All image data will be reviewed independently by two experienced clinicians blinded to the image guidance method used during endovascular aortic repair. Anatomical measurements will be performed with central luminal line reconstructions using dedicated software. Measurements will include aneurysm size, aortic neck (diameter, length, α and β angulation), iliac diameter and stenosis, distance from the lowermost renal artery to the beginning of the covered part of the endograft, the length of the proximal sealing zone, length of the distal sealing zone, and detection of endoleak. Technical success will be defined as proximal and distal seal zone at least 10 mm with no evidence of endoleak [1].

• System efficiency

A key link between the primary outcome measure (procedure time) and the cost-effectiveness of the intervention is measured in terms of improvements in the planning and utilisation of operating theatre resources. The average procedure time in England for a standard EVAR procedure is 110 min. Assuming operating theatre capacity of 420 min (7 h) daily, it would currently be possible to complete three EVAR procedures daily with an allowance for turn-around time. Assuming a similar reduction in procedure time as was observed in the Duke University study (18%), with Cydar-EV it would be possible to complete four procedures daily with the same capacity, an increase of 33%. The HRG EVAR tariff can be used as a proxy for the value to the NHS of the additional procedure. Because Cydar-EV is also expected to reduce variability in procedure times, there should also be a reduction in the number of cancelled operations because of over-runs, and more predictability in waiting list planning and bed occupancy.

We will explore the implications of improvements in system efficiency by comparing the distributions of procedure times for Cydar-EV and standard fluoroscopy and assessing these against current capacity constraints (e.g. operating theatre capacity, turn-around times). We will also assess the potential implications of any ‘learning curve’ effects in the procedure times for Cydar-EV. We will use these analyses to develop a series of scenarios which capture the potential impact on Cydar-EV on improving the planning and utilisation of operating resources in terms of costs and potential health consequences. The impact of these scenarios on the overall cost-effectiveness of Cydar-EV will be assessed using sensitivity analysis.

• Value of information

Decisions based on 12-month follow-up (and the exploratory model based analysis) for Cydar-EV will be subject to uncertainty and there will always be a chance that the wrong decision could be made. If the wrong decision is made, there will be costs in terms of health benefit and resources forgone. The maximum amount the NHS should be willing to invest to further reduce remaining uncertainty in the decision can be informed by the expected value of perfect information (EVPI). EVPI evaluates the expected cost of current decision uncertainty, based on results from the ARIA trial, by accounting for both the probability that a decision based on existing evidence is wrong and for the magnitude of the consequences of making the wrong decision.

The EVPI estimates will be used to assess the potential value of further research and to inform future research priorities. EVPI also represents the maximum amount that a decision-maker should be willing to pay for additional evidence to inform this decision in the future. EVPI provides an upper bound on the value of additional research. This valuation provides an initial hurdle, acting as a necessary requirement for determining the potential efficiency of further primary research. Applying this decision rule, additional research should only be considered if the EVPI exceeds the expected cost of the research.