Core outcome sets in cancer clinical trials: current status and future opportunities—an EORTC perspective

Abstract

Background

Inconsistent, varied and selective outcome reporting is problematic in clinical trials. Core outcome sets (COS) standardise the outcomes that should be measured and reported in all trials in a specific area of health or health care. We reviewed available cancer COS and assessed their uptake in cancer clinical trials through surveying members of the European Organisation for Research and Treatment of Cancer (EORTC).

Methods

This study employs an exploratory cross-sectional design across two phases. The Core Outcome Measures in Effectiveness Trials (COMET) Initiative database was searched for cancer-specific COS on June 1st, 2023. Awareness and use of COS amongst EORTC trialists was assessed in November 2023 via an online survey.

Results

We identified a total of 85 cancer-related COS on the COMET database. Of these, 69 related to the tumour types as categorised by the EORTC and their disease orientated groups. A total of 710 EORTC members responded, of whom half (50%) stated they were unfamiliar with COS. Relevant COS were available to over a quarter of respondents, with a tenth utilising available COS. Those who chose not to use an available COS cited volume of outcomes, lack of time and infrastructure for implementation as key barriers.

Conclusions

While COS are becoming increasingly available to, and acknowledged by, cancer clinical trialists, their implementation is currently still limited. Our findings indicate that further development of COS to fill gaps for missing tumour types, greater involvement of trialists in the COS development process, and increased awareness and understanding of COS amongst trialists are all required to ensure widespread implementation of COS in cancer clinical trials.

Introduction

The clinical need for core outcome sets (COS)

Regulatory trials (or clinical trials) evaluate new medical devices/treatments to determine if they can be safely and effectively used in patient care [1, 2]. Clinical decisions are made based on the outcomes reported in these trials [3]. However, the choice of outcomes measured in clinical trials is often based on researcher’s preference, leading to inconsistent outcomes reported, and variability in the definitions or instruments used across trials. This poses a challenge for systematic reviews and meta-analyses, as evidence from individual trials is incommensurable, preventing increased accuracy and generalisability of results.

Core outcome sets offer a solution to this problem. A core outcome set (COS) is a minimum set of standardised and agreed-upon outcomes that should be measured and reported in all clinical trials in specific areas of health or health care [4]. COS are often organised by domains, or outcome areas, with commonly observed domains including mortality, physiological, life impact, resource use and adverse events [5]. The involvement of patients, their families and carers, relevant healthcare workers and other key stakeholders is a crucial aspect of COS development, as outlined by the Core Outcome Set-STAndards for Development (COS-STAD) recommendations [6]. Thus, COS help ensure that the priorities and expertise of relevant stakeholders, including patients, inform the recommended set of outcomes to measure in clinical trials for a given health condition [7]. Moreover, by ensuring that all trials report the same core outcomes as a minimum, COS can reduce heterogeneity across studies, allowing for more efficient and appropriate meta-analysis, leading to increased power, precision and higher confidence in estimates of effect, as well as reduced research waste in systematic reviews and meta-analyses [2].

The Core Outcome Measures in Effectiveness Trials (COMET) Initiative was established to provide easy access to COS for researchers, trialists and other key stakeholders. COMET collates known COS, including those in development, into one database. Cancer currently has the highest number of COS available on the COMET database (N = 146). The number of cancer treatments undergoing regulatory testing is increasingly rapidly [8] and thus, so is the number of trials. It is imperative that the outcomes included in these trials are relevant to patients and health care professionals, and that outcomes across trials are homogeneous to allow for meta-analysis, using COS.

The regulatory need for COS

Increasingly, key stakeholders in clinical research are recommending COS use, including trial funders, policymakers and international medical agencies. Organisations aimed at supporting healthcare services improvement and informing decision-making also encourage COS use [9, 10]. Moreover, global regulators including the Food and Drug Administration (FDA) and the European Medicines Agency (EMA) are highly influential to clinical trialists and recommend the use of patient-reported outcomes (PROs) and COS, respectively. However, neither recommendation is legally binding, meaning trialists are under no obligation to implement COS. Nonetheless, the benefits of COS can only be realised if trialists measure and report COS. Supplementary Table 1 outlines the COS recommendations made by key stakeholders in clinical cancer research.

The need for COS in clinical cancer research—from the perspective of the EORTC

The European Organisation for Research and Treatment of Cancer (EORTC), an independent, non-governmental, non-profit, cancer research organisation, evaluates new cancer treatments in collaboration with the pharmaceutical industry and patients [11]. The scope of research conducted by the EORTC is broad, with ongoing studies spanning from clinical trials at various phases of the life cycle, to population-based studies using real-world data, and tumour-specific health-related quality of life questionnaire development. Clinical research activities within the EORTC cover all types of cancer tumours, with activities conducted through disease-oriented groups or DOGs, such as breast, genito-urinary and lymphoma. A complete list of the EORTC’s DOGs is presented in Table 1. Membership within the EORTC is granted based on personal and institutional track records, as well as research quality. EORTC members can participate in EORTC clinical trials, meetings, conferences and training. They currently do not have an official endorsement of, or position on, the use of COS in their trials. It is currently unclear if, and to what extent, COS uptake has been implemented in EORTC trials. To our knowledge, there is no existing research assessing COS uptake within cancer clinical trials specifically, yet COS uptake is essential to realising their benefits. Therefore, we aimed to (1) obtain a snapshot of the available COS from the COMET database for each of the disease orientated groups (i.e. tumour types) within the EORTC [12] and (2) assess COS uptake within clinical trials affiliated with the EORTC.

Table 1 Demographic information of EORTC members who responded to the COS survey questions

Methods

Review of COS on the COMET database

An exploratory search of the COMET database was conducted to identify cancer-specific COS on 1 June 2023. Using the advanced search function on the COMET database (https://www.comet-initiative.org/Studies), each of the EORTC tumour types, also referred to as disease-oriented groups, were used as search terms (e.g. ‘breast’, ‘endocrine’, ‘lung’) and cancer was selected under the ‘disease category’ search function on the COMET database, as shown in Appendix 1. Withdrawn COS, narrative review papers, general assembly and consortium reports, and COS pertaining specifically to Traditional Chinese Medicine (TCM) were excluded. TCM COS were beyond the scope of this review as they do not relate to clinical trials conducted by the EORTC. All other COS listed on the COMET database were included in this overview. No statistical analyses were performed.

Assessment of COS uptake in EORTC clinical trials

EORTC members were queried on their awareness and use of COS over two survey questions: ‘Are you aware of a core outcome set for the disease you specialise in?’ and, where the participant indicated they did not use a relevant, available COS, ‘Please provide the main reasons why you do not use the relevant/available core outcome set.’ (Appendix 2). Response items for the latter question included ‘There are too many outcomes’; ‘It is too general’; ‘It is too specific’; ‘I do not understand how to use it’; and ‘I do not trust how it was developed’. Participants were encouraged to select all relevant items from the list. A text box was also included for open-ended responses. A short summary outlining and contextualising COS (Appendix 3) was presented to respondents prior to the items. The summary stated that the existence or use of COS does not imply that outcomes in a particular study should be restricted to those included in the COS, rather that COS outcomes are included as a minimum. Participant demographics, awareness of COS and the close-ended question assessing barriers to COS uptake were analysed using descriptive statistics. Open-ended responses regarding barriers to COS uptake were grouped according to similarity to establish categories.

Results

Review of COS on the COMET database

We identified 85 cancer-related COS relevant to the EORTC tumour groups. Appendix 4 presents the search results; an overview of the number of COS by tumour type is provided in Fig. 1. Of these, 69 were specific to the EORTC tumour types and 16 were general cancer COS. Gastro-intestinal and genito-urinary cancers had the greatest number of COS—14 and 11, respectively. Several tumour types had less than five COS available (brain, endocrine, leukaemia and melanoma). One tumour type (sarcoma) had only one COS available, and cutaneous lymphoma had no available COS on the COMET database. COS were specific to a country in 11 cases, with an additional two COS developed in specific countries (Italy, Australia). In addition, nine COS were specific to certain age ranges, such as children and adolescents; survivors of childhood cancer; and those aged over 70. Of all identified COS, 40 (47%) were specific to a treatment or treatments. Over a third of all COS identified mentioned patient-reported or patient-centred outcomes in their title. Most COS are completed (61 [72%]) or ongoing (22 [26%]), with two (2%) in the planning phase. There were a total of 34 completed COS unspecific to a particular country or age group across the EORTC’s tumour types.

Fig. 1

Number of COS per tumour type available on the COMET database

Assessment of COS uptake in EORTC clinical trials

The survey was shared with 3733 EORTC members on November 28th, 2023, and closed on April 5th, 2024. A total of 710 EORTC members (19%) answered the COS questions. Table 1 presents demographic characteristics of respondents. Most respondents were physicians (93%) based in Europe (93%), and there was an equal spread of responses from men and women. Breast, genito-urinary, gastrointestinal tract and lung were the most common cancer specialties that trialists reported working within.

Awareness of COS amongst EORTC clinical trialists

Half (352 [50%]) of respondents reported that COS were outside their area of expertise, e.g. they had not heard of core outcome sets. Over a fifth (164 [23%]) indicated that while they are aware of COS, there is no relevant COS for their disease speciality. A further 126 (18%) reported that while there is a relevant COS available for their disease speciality, they do not use it. A tenth (68) stated that there is an available COS and that they use it.

Barriers to COS uptake and implementation amongst EORTC clinical trialists

Participants who indicated they were aware of a suitable COS for their disease speciality but chose not to use it were asked to provide the main reasons for their decision. Volume of outcomes (45 (37%)) and a lack of understanding on how to use COS (22 (18%)) were the most common reasons for not implementing a COS. Amongst open-ended responses, lack of time (20 (16%)) and lack of infrastructure and/or capacity (13 (11%)) were commonly cited reasons for not implement COS (Fig. 2). A further 24 responses were provided by respondents and are presented in Fig. 3.

Fig. 2

Main reasons reported by EORTC trialists who chose not to implement a suitable and available COS. Note. Participants could select multiple responses

Fig. 3

EORTC trialists’ open-ended responses as why they chose not to implement a suitable and available COS

Discussion

This is the first study to obtain a snapshot of the availability of cancer COS on the COMET database and uptake of COS in the context of EORTC cancer clinical trials. Our findings demonstrate that while there are many cancer COS available to trialists on the COMET database, they are often specific to a tumour subtype, country or age range. As such, there is a clear need for further development of COS to address existing gaps in availability. Only one tenth of surveyed EORTC trialists reported implementing a suitable COS in their trials. The vast majority of trialists reported three central issues hindering COS uptake: (1) lack of COS awareness; (2) lack of suitable COS for trialists’ disease speciality; (3) barriers to implementing a suitable and available COS. To achieve widespread COS uptake in cancer clinical trials, trialists must be aware of COS, a suitable COS must be available to them, and barriers to COS uptake must be addressed.

Review of COS in the COMET database

Our review of the COMET database found that almost all tumour types had at least one COS available, except for cutaneous lymphoma. However, many COS were specific to a particular geographical location, age range or intervention type (e.g. surgery). Of the 85 COS identified in this study, only 34 completed COS were unspecific to a geographical location and age range. Consequently, even where there are larger number of COS available for a tumour group, there are still gaps in COS availability for certain tumour types or patient groups (e.g. uterine cancer).

Awareness of COS amongst EORTC clinical trialists

The majority of EORTC trialists described COS as ‘outside of their area of expertise’ or were aware of COS but reported no available COS for their disease speciality. Similar findings were observed in a review of 95 RCTs published in five major medical journals [13]. Relevant COS were identified for an additional 31 trials; however, an applicable COS could not be identified for 62 trials [13]. Only two of the 95 trials had implemented an applicable COS [13]. Additionally, in 2021, over one third of trialists surveyed by Bellucci and colleagues reported they were unfamiliar with COS, with almost half having never used a COS [14]. These findings highlight a substantial need for COS relevant for all tumour types regardless of patient characteristics, intervention type or geographical location, in clinical trials. Moreover, trialists’ awareness of existing and future COS must be improved.

Barriers to COS uptake amongst EORTC clinical trialists

Almost one in five trialists surveyed indicate they choose not to use the COS available to them. Commonly cited barriers to implementation included the number of outcomes, a lack of time and infrastructure, a lack of understanding on how to use COS, and that COS were both too general and too specific. Similar barriers to uptake were reported by Bellucci and colleagues who found 61% of trialists surveyed preferred to choose their own outcomes, and 52% of trialists cited COS increased patient burden [14]. Additionally, over one third of trialists reported challenges in how to measure the outcomes included in a COS [14].

COS implemented by EORTC cancer clinical trialists

In our study, only 10% of trialists stated that they used COS in their research. Likewise, in the review of 95 effectiveness trials identified, 93 did not report using a COS, with the remaining two trials using only some of the outcomes from the selected COS [14]. As uptake is difficult to quantify, few studies have assessed COS uptake in clinical trials, with no existing study assessing the uptake of cancer COS specifically [15, 16]. However, the field of rheumatoid arthritis demonstrates that COS can be widely implemented, as 81% of phase III/IV drug clinical trials now report using the full rheumatoid arthritis COS, an increase since its development in 1994 [15]. Widespread implementation of COS across cancer clinical trials is possible, but COS must be high quality, available to trialists, and trialists must be aware of COS and their benefits.

Opportunities and implications

Our review of the COMET database and snapshot of EORTC trialists uptake demonstrate that to improve COS implementation in cancer clinical trials, COS must be available, trialists must be aware of and understand COS, and barriers to COS must be addressed. Below we outline how to address these challenges.

Development of COS for implementation in cancer clinical trials

Once consensus on the outcomes to be included in the COS has been reached, developers should focus on how to define and measure the included outcomes. The estimand framework, a systematic description of the treatment effect to be quantified to answer the trial’s objective, [15, 16] is a useful tool for defining an outcome. Defining an estimand involves specifying five attributes: treatment condition(s), population variable, population-level summary and post-randomisation (intercurrent) events [17]. By using the estimand framework to define the treatment effect(s) in the design phase, clarity and transparency is brought. Moreover, estimands result in clearer answers to trial questions, thus improving regulatory and clinical decision-making [17]. As such, COS and estimands can be used together to improve the quality of clinical trials. While using both COS and estimands in practice does not yet appear to be commonplace for clinical trials, there are some instances of their joint use in other studies. For example, Kahan and colleagues [18] proposed a set of estimands that could be used for trials of in-hospital treatments for COVID-19 using outcomes described in the COS for COVID-19. Using estimands to define outcome would also aid in addressing the issue of outcome definition heterogeneity, wherein trials include the same outcomes, but different definitions are used.

In addition, COS should be developed with the continued involvement of key stakeholders including trialists and patients. Patient involvement in COS development ensure that outcomes considered most important to patients are measured, which is not always the case currently [4, 19]. Moreover, engagement with the COS development process by key stakeholders allows for barriers such as awareness, buy-in and understanding, to be addressed early on, improving COS implementation.

Awareness of COS for implementation in cancer clinical trials

Awareness of COS also positively contributes to their uptake [20]. Bellucci et al. reported that having seen a COS reported in research or a clinical trial, having attended a conference presentation on COS, or being informed about COS by a colleague were the most commonly cited ways in which trialists became familiar with COS [14]. Furthermore, trialists suggested that awareness should be increased through training; publicity across social media and clinical/academic forums; publication and journal requirements; and funding application requirements [20]. Such methods have demonstrated success in increasing COS uptake. For example, endorsement of COS by the NIHR Health Technology Assessment programme was found to encourage trialists to search for a COS [21], with similar results shown in Ireland after HRB guidance to report COS. Developers can also increase COS awareness during the development process by designing an implementation plan at the start of the development process [22]. Awareness of COS can for instance be improved through a combination of funder and journal requirements, research training (which can include conference seminars) and implementation strategies developed alongside the COS itself.

Removing barriers for implementation of COS in cancer clinical trials

Improved awareness of COS alone is insufficient to increase COS implementation, and further barriers should be identified [14]. Many EORTC trialists reported the volume of outcomes in COS as a barrier to implementation, likely due to perceived patient burden as indicated by previous research [14]. When deciding on outcomes and their measurement instruments, COS developers should consider the burden on patients and trialists using the COS [20]. Similarly, some trialists reported COS to be too specific for their needs. While this again emphasises the need for coordination on how COS which contain the same/similar outcomes regardless of tumour types and patient characteristics, understanding of COS and how COS outcomes are selected (e.g. by patients and other relevant stakeholders) should be promoted amongst trialists. Likewise, COS being too general was also described as a barrier to implementation by EORTC trialists. However, COS are the minimum set of outcomes that should be measured in trials; implementing a COS does not prevent additional outcomes being included. Lack of understanding on how to use COS was also reported as a barrier to implementation. Taken together, it is evident these barriers can be significantly reduced by increasing understanding amongst trialists of how COS are developed and how they can be used in clinical trials.

As outlined by Hughes et al., [21] to increase knowledge and understanding of COS, organisations involved in the promotion of COS (e.g. COMET Initiative, trial funders, journals) should consider providing educational programmes such as standalone workshops at conferences, or training embedded into existing education courses, e.g. those delivered for continued professional development. Moreover, recent research indicates that COS uptake may be increased by using behavioural strategies to overcome barriers to COS adoption [23].

Strengths, limitations and recommendations for future research

This study has provided insights into COS availability to, and uptake by, EORTC trialists. To our knowledge, it is the first study to assess COS uptake in the cancer clinical trial setting. Our findings indicate distinct areas that require strengthening to improve the implementation of COS in cancer clinical trials. However, our sample may be influenced by self-selection bias. Of the 3733 EORTC trialists invited to participate, 710 participants responded to our COS survey. As the wider survey focused on PROMS, it is possible participants who responded to the survey may have been more familiar with standardised outcome reporting, including COS, or had a particular interest in outcome measurement and reporting in trials. Thus, our findings may overestimate the proportion of trialists within the EORTC who are aware of COS. Conversely, we did not assess respondents’ level of involvement in selecting outcomes for trials. As such, some respondents may not be involved in outcome selection and may be unaware COS are implemented in their trials.

Moreover, our sample is heavily comprised of physicians working in Europe, many of whom specialise in multiple tumour types. This posed a challenge for examining the associations between respondent characteristics and COS awareness and uptake using formal statistical analyses, as the results were unlikely to be reliable or meaningful. Future research should ensure a balanced sample of trialists to enable a more in-depth analysis of the potential associations between COS implementation and trialists’ characteristics.

Future research should employ more resource-intensive methods of assessing COS uptake, which avoid the potential for self-selection bias, such as reviewing outcomes listed in trial registries to provide reasonable estimates of COS uptake [24]. While less efficient than surveying trialists, this method is less resource-intensive than other methods such as identifying and examining reports of RCTs, and citation analysis which has also proved unreliable [24], more recent publications propose COS uptake is assessed by reviewing outcomes listed in trial registries to provide reasonable estimates of COS uptake efficiently [24]. Such assessment of COS uptake could also be complemented by a more in-depth review of the cancer COS available on the COMET database. To our knowledge, this study is the first to provide a snapshot into cancer COS availability on the COMET database. However, a detailed review of COS availability that establishes the quality of available COS and the relevance of COS to ongoing cancer clinical trials would provide further insight into the extent to which COS uptake may be hindered by suitable COS availability.

Strategies to increase COS awareness and uptake amongst cancer trialists

Strategies to increase COS awareness and uptake within the EORTC

Following the findings of this COS survey, the EORTC began planning activities aimed at addressing low awareness amongst trialists. In March of 2024, a presentation titled ‘What are core outcome sets and how can they facilitate PROM collection?’ was given at the EORTC Quality-of-Life (QLG) Spring meeting by author S.M. EORTC QLG meetings are typically attended by 100 to 150 EORTC members who conduct research on patients’ quality-of-life across the cancer care pathway and beyond to survivorship. Moreover, plans are in motion to create an educational webinar for members of the Young Early Cancer Investigator (Y-ECI) group aimed at increasing awareness of COS. This may also mark the beginning of a larger initiative in the Y-ECI group to educate members on COS to promote uptake. Finally, initial education efforts will focus on EORTC headquarter members who support protocol writing for trials. This would help disseminate awareness of COS amongst trialists and promote awareness.

COS in rheumatoid arthritis: an example of widespread COS implementation

The field of cancer could learn from rheumatoid arthritis (RA) where COS have been implemented since 1994 and were subsequently endorsed by the EMA and the US FDA in the late 1990s [25]. The proportion of RA clinical trials reporting the full COS started to rise shortly after the EMA and FDA regulatory guidelines were published. Implementation was further improved following a publication which demonstrated agreement regarding the important of these COS outcomes in RA trials [25]. Studies examining the uptake of COS report relevant RA COS implementation in 60% to over 80% of RA clinical trials, showing that widespread uptake is possible [26]. Notably, over 80% of RA trials assessed received commercial funding and would have followed EMA or FDA guidance including on COS [26]. Recent research indicates higher uptake of COS in non-commercial RA trials, compared to commercial trials [19]. For cancer clinical trials, future research could also establish if COS uptake differs according to funding source and by geographical region, based on national regulatory guidelines. If COS uptake in cancer clinical trials does differ by regulatory guidelines and funding sources, then COS developers should be encouraged to work closely with regulatory and funding bodies to mandate COS uptake where possible, to achieve similar COS uptake in cancer trials as is observed in RA trials.

Conclusion

COS can reduce outcome heterogeneity across studies, resulting in more appropriate meta-analysis of cancer clinical trials, with consistently defined outcome measures, leading to increased power, precision and higher confidence in estimates of effect, as well as reduced research waste. Through a review of the COMET database and a obtaining a snapshot of EORTC clinical trialists’ awareness and usage of COS, we identified key barriers to COS uptake and recommend evidence-based strategies for overcoming them. The EORTC has already started initiatives to help address these challenges and will continue their efforts to increase COS awareness and uptake amongst trialists.