If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
1 These authors contributed equally and are joint first authors.
Katharina Beyer
Correspondence
Corresponding author. Translational Oncology and Urology Research, School of Cancer and Pharmaceutical Studies, Guy’s Hospital, Great Maze Pond, London SE1 9RT, UK. Tel. +44 207 188 5594.
University Hospital of Psychiatry II, Medical University of Innsbruck, Innsbruck, AustriaSpecialist Centre for Kidney Cancer, Royal Free Hospital, London, UK
Outcomes in renal cell carcinoma (RCC) are reported inconsistently, with variability in definitions and measurement. Hence, it is difficult to compare intervention effectiveness and synthesise outcomes for systematic reviews and to create clinical practice guidelines. This uncertainty in the evidence makes it difficult to guide patient-clinician decision-making. One solution is a core outcome set (COS): an agreed minimum set of outcomes.
Objective
To describe outcome reporting, definitions, and measurement heterogeneity as the first stage in co-creating a COS for localised renal cancer.
Evidence acquisition
We systematically reviewed outcome reporting heterogeneity in effectiveness trials and observational studies in localised RCC. In total, 2822 studies (randomised controlled trials, cohort studies, case-control studies, systematic reviews) up to June 2020 meeting our inclusion criteria were identified. Abstracts and full texts were screened independently by two reviewers; in cases of disagreement, a third reviewer arbitrated. Data extractions were double-checked.
Evidence synthesis
We included 149 studies and found that there was inconsistency in which outcomes were reported across studies and variability in the definitions used for outcomes that were conceptually the same. We structured our analysis using the outcome classification taxonomy proposed by Dodd et al. Outcomes linked to adverse events (eg, bleeding, outcomes linked to surgery) and renal injury outcomes (reduced renal function) were reported most commonly. Outcomes related to deaths from any cause and from cancer were reported in 44% and 25% of studies, respectively, although the time point for measurement and the analysis methods were inconsistent. Outcomes linked to life impact (eg, global quality of life) were reported least often. Clinician-reported outcomes are more frequently reported than patient-reported outcomes in the renal cancer literature.
Conclusions
This systematic review underscores the heterogeneity of outcome reporting, definitions, and measurement in research on localised renal cancer. It catalogues the variety of outcomes and serves as a first step towards the development of a COS for localised renal cancer.
Patient summary
We reviewed studies on localised kidney cancer and found that multiple terms and definitions have been used to describe outcomes. These are not defined consistently, and often not defined at all. Our review is the first phase in developing a core outcome set to allow better comparisons of studies to improve medical care.
]. With the increase in reporting of incidental findings, a greater proportion of patients newly diagnosed with renal cancer currently present with stage I disease [
A stage-for-stage and grade-for-grade analysis of cancer-specific mortality rates in renal cell carcinoma according to age: a competing-risks regression analysis.
]. Historically, surgery has been the standard of care for localised renal cancer, but international guidelines have more recently proposed ablative treatments and active surveillance as alternative options [
Across many clinical areas including urology, patient-reported outcomes and clinical outcomes are reported inconsistently, with variability in definitions and measurement, for instance in the settings of localised prostate cancer and bladder cancer [
]. This makes it very difficult to compare and synthesise outcomes to improve guidelines to better direct and support patients and clinicians during treatment decision-making and ultimately improve results in clinical practice [
]. A core outcome set (COS) is a standardised set of prioritised outcomes and is proposed by current research as a solution to decrease heterogeneity in collection, reporting, and analysis of outcomes. COS in urology are needed because inconsistencies and variability cause not only frustration but also potentially problematic conclusions [
]. This issue is also clearly apparent for localised renal cancer, and ultimately results in barriers for the multifactorial process of decision-making [
The aim of this systematic review was to identify which outcomes are reported in intervention effectiveness research in localised kidney cancer and to assess heterogeneity in outcome definitions and measurements. It constitutes the initial stage in the development of a COS for localised renal cancer with the intention of identifying a minimum set of outcomes that are potentially important to health care professionals and patients. The outcomes identified in this systematic review are organised under the taxonomy developed by Dodd et al. [
], which helps to structure general health research vocabularies to reduce inconsistencies. It is embedded in a larger project registered in the Core Outcome Measures in Effectiveness Trials (COMET) database [
Updating and integrating core outcome sets for localised, locally advanced, metastatic, and nonmetastatic castration-resistant prostate cancer: an update from the PIONEER Consortium.
] developed in collaboration with the European Association of Urology.
2. Evidence acquisition
This systematic review followed the guidelines of the COMET initiative, an international expert body that established guidelines on how to develop methodologically robust COS. We report our study in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and the COS-STAR reporting guidelines, which are relevant to this stage of COS development [
]. A project steering committee (S.M., M.V.H., P.Z., A.B., L.M., S.D., R.B., N.K.) supported the development from a methodological and clinical perspective. The study protocol was registered on PROSPERO (ID: CRD42020198605).
2.1 Aims and objectives
The aim of this project was to systematically review which outcomes have been reported in effectiveness trials and observational studies in localised renal cancer, and how they were defined and measured.
2.2 Identification of relevant studies
We searched Medline, EMBASE, Cochrane CENTRAL, and Cochrane Database of Systematic Reviews (via Ovid) from inception to June 2020. We worked with an information scientist to design the search strategy (Supplementary Fig. 1). To balance the feasibility and precision of the search, we used a two-step approach. First, we identified all published systematic reviews and intervention trials related to RCC without limiting the search to localised renal cancer, and we screened the reference lists in all the articles as a pragmatic way to identify primary studies potentially meeting our inclusion criteria. Second, we searched for and screened all interventional studies on localised RCC from 2015 onwards without limiting the study design. We included randomised controlled trials (RCTs), cohort studies, and case-control studies that reported on eligible interventions for localised renal cancer. We excluded case studies owing to their low level of evidence according to the Oxford Centre for Evidence-Based Medicine (level of evidence 4 or lower [
]) and the unlikelihood of changing clinical practice. We also excluded conference abstracts.
2.2.1 Study participants
Adults (male and female) with suspected localised renal cancer (N0M0 according to the TNM classification; all versions of the TNM staging system) on magnetic resonance imaging, computed tomography, or ultrasound imaging were included.
Those undergoing treatment for renal metastasis or other tumours were excluded.
2.2.2 Intervention and comparator
Studies reporting on any intervention for localised renal cancer were retained, including but not limited to active surveillance, radical nephrectomy (all modes and approaches), partial nephrectomy (all modes and approaches), cryoablation, radiofrequency ablation, microwave ablation, irreversible electroporation, watchful waiting, high-intensity focused ultrasound, or radiotherapy.
2.2.3 Eligibility of studies
All abstracts and full texts were screened independently by at least two reviewers (C.W., K.B.). Any disagreements were arbitrated by a third review author (S.M).
2.3 Data extraction
Data were independently extracted from the studies included by two researchers (C.W., K.B.) and checked for accuracy by another reviewer (S.M.). We extracted data on study design; author details; year and journal of publication; intervention(s) under investigation; each effectiveness outcome reported; whether the outcome was defined or not; the definition used; the indicators and/or tool(s) used to operationalise or measure the outcome; the time point or period for outcome measurement; and how the outcome was reported.
2.4 Data analysis and synthesis
The outcome names extracted were coded and categorised according to the outcome reporting taxonomy developed by Dodd et al. [
], which has been suggested by COMET for classification of outcomes and group domains.
2.5 Assessment of risk of bias
A risk-of-bias assessment was not conducted, as no estimation of the effect size of treatments was conducted and only qualitative information containing terminology was extracted.
3. Evidence synthesis
3.1 Characteristics of the studies included
Our initial search returned 2785 abstracts. Of these, we assessed 319 full-text articles, of which 149 were included (Fig. 1). Of the 149 studies included, 97% were observational studies and five (3%) were RCTs.
Fig. 1Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.
3.2 Heterogeneity in outcome reporting, detection, and definitions
A suitable outcome taxonomy for health research must differentiate between high-level outcome domain classifications, and comprehensively classify all outcomes, while also proposing a standardised terminology. Therefore, we reported and organised the outcomes in the studies under the taxonomy developed by Dodd et al. [
] and recommended by the COMET initiative. Taxonomies help to structure vocabularies for general health research to reduce inconsistencies and ambiguities in how current studies describe and define outcomes. The Dodd taxonomy has been proposed to increase the reuse value of outcome data. The taxonomy comprises 38 outcome domains within five core areas: death, adverse events, life impact, physiological/clinical, and resource use [
Table 1 lists the outcomes reported by the studies by domain and highlights the heterogeneity of outcomes identified. We have merged the synonyms and redundant terms. The next section explains the heterogeneity of the terminology in more detail. Table 2 shows which outcomes were reported in the studies included in the review.
Table 1Outcomes classified according to the taxonomy of Dodd et al.
Death was reported 103 times. We categorised these outcomes according to the Dodd mortality/survival classification into “overall survival” and “cancer-specific survival”. Overall survival (OS) was measured in 65 studies (44%; Table 2) as OS, death, or mortality; more details are provided in Table 1. Cancer-specific survival (CSS) was reported in 43 (29%) of the studies (Table 2) as CSS, death from renal cancer, or cancer-specific mortality.
Definitions of OS and CSS differed across studies (Table 1). The heterogeneity for the definitions was linked to time points. For instance, some studies started measurement at diagnosis, whereas others used the treatment date as the starting point for survival. The time endpoint also differed, with studies reporting either a rate at a defined time (eg, at 10 yr) or a hazard ratio based on survival analyses.
3.4 Adverse events (adverse events/effects)
Adverse events (AEs) were the most common outcome reported (n = 101, 68%). However, many different types of AE were reported, sometimes as the number or percentage of patients experiencing the outcome and sometimes subsumed in a classification system linked to severity or consequences (eg, the Clavien-Dindo scheme). Examples of events that play a role in AE assessment include bleeding, operation time, warm ischaemia time, intra-abdominal pressure, surgical time, drainage time, serum creatinine, blood loss, trifecta/pentafecta outcomes, and dialysis, which are linked to the complexity of the surgery. Many articles reported several AEs within one study, but the AEs reported varied across studies (eg, surgical complications were measured as intraoperative complications, conversion to nephrectomy, or short-term complications).
“Adverse events/effects or resource use: hospital” outcomes were reported in six studies (4%). Outcome reporting and measurement were inconsistent; examples include dialysis-free probability, number requiring dialysis, temporary dialysis, and permanent dialysis (Table 1).
3.5 Life impact/functioning (perceived health status; global quality of life)
Only eight studies (5%) reported outcomes reflecting life impact. Five studies (3%) reported outcomes classified as perceived health status. Three studies (2%) reported on global quality of life (QoL), one study using the Short Form (SF)-36 and another using the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI)-15 patient-reported outcome measures (PROMs; Table 2).
3.6 Physiological/clinical (physiological or clinical)
Physiological or clinical outcomes were subclassified as “renal and injury outcomes” (eg, new chronic kidney disease [CKD], CKD stage, time to CKD), which were defined very heterogeneously and reported in 87 (58%) of the studies; and “outcomes relating to neoplasms” (linked to cancer follow-up and progression, reported in 55 studies [37%]) according to the Dodd taxonomy [
3.7 Resource use (economic, need for further intervention, hospital)
Resource use consisted of the subcategories “economic resource” (eg, health care expenditure; reported in four studies (3%) as mean or median costs), “need for further intervention” (eg, readmission; reported in eight studies [5%] as a binary yes/no result or median value), and “hospital” (eg, length of hospital stay, reported in 58 studies [39%] as mean or median length of hospital stay in days).
3.8 Discussion
To the best of our knowledge, this is the first systematic review of outcome reporting heterogeneity in the literature on localised renal cancer. Our results build a framework for developing a COS for localised renal cancer with the aim of reducing heterogeneity for outcome definitions, measurement, and reporting.
Our systematic review highlights the persisting problem of outcome reporting heterogeneity in studies on localised renal cancer. Multiple terms are used to refer to conceptually similar outcomes, and there is variation in the outcome definitions used. This has not improved over time and is problematic when summarising the evidence base for treatment effectiveness to inform decision-making, because it is not advisable to synthesise data with different outcome definitions within a meta-analysis. Such a practice can produce meaningless summary statistics that may be given more credibility than they are due. Therefore, a cumbersome and often less-informative narrative synthesis must be undertaken instead. Furthermore, our work highlights variety in data reporting and measurement. For instance, if dichotomous outcomes such as OS and CSS are reported using different methods (eg, some studies report adjusted and some unadjusted hazard ratios, others report a rate at median follow-up or at specified time points such as 1 yr or 5 yr), then these data cannot be easily or reliably synthesised in a meta-analysis. When these problems all occur together, then it is difficult to interpret the body of evidence and clinical practice guideline panels encounter challenges in drawing up recommendations and applying certainty-of-evidence attributes such as those proposed by the GRADE working group [
Worryingly, we identified very few patient-reported outcomes (PROs), which might be related to the limited number of specific tools available for capturing QoL for renal cancer. In their systematic review, Rossi et al. [
] identified three generic PROMs (RAND medical outcome survey SF-36 and SF-12, EuroQol [EQ-5D], Convalescence and Recovery Evaluation [CARE]) and eight cancer-specific PROMs (Cancer Rehabilitation Evaluation System-Short Form [CARES-SF], European Organisation for Research and Treatment of Cancer [EORTC] Quality of Life Questionnaire [QLQ]-C30, Functional Assessment of Cancer Therapy-General [FACT-G], FKSI, Renal Cell Carcinoma-Symptom Index [RCC-SI], Instruments to assess psychological wellbeing Impact of Events Scale [IES], Hospital Anxiety and Depression Scale [HADS], Mishel Uncertainty in Illness Scale (MUIS)) which are currently being used in renal cancer. However, of the PROM instruments used, only two are specific to renal cancer and are not stage-specific (FKSI, RCC-SI) [
A psychometric evaluation of the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI-19) among renal cell carcinoma patients suggesting an alternative two-factor structure.
] assessed the psychometric properties of the FKSI-19 (which captures physical and emotional disease-related symptoms, function/wellbeing, and treatment side effects) among patients with RCC and reported that it is barely fit for this purpose. Decat Bergerot et al. [
Decat Bergerot C, Psutka S, Plimack E, et al. Relevance of items in the FACT Kidney Cancer Symptom Inventory-19 (FKSI-19): results of a patient survey. Presented at the 2022 European International Kidney Cancer Symposium (EIKCS), Antwerp, April 22–24, 2022.
] conducted a patient survey that identified many of the FKSI-19 questions as irrelevant from a patient perspective and stressed the need to incorporate patients in the development of PRO tools to determine areas of importance to them. The EORTC Quality of Life group is currently developing an RCC-specific module to be used in combination with their QLQ-C30 instrument.
Our project steering group includes clinical RCC experts, patient advocacy groups, methodologists, and guideline developers from the European Association of Urology (most are co-authors of this study). We aim to use these networks to improve recruitment to our research stages, and to subsequently endorse and disseminate the final COS as part of our implementation strategy.
We curated the different terms used and collated them using a standardised outcome classification taxonomy [
] as a first step in creating a COS for localised renal cancer. In the next step we will use consensus processes in a multistakeholder group to prioritise which outcomes are core and to recommend definitions for each outcome. Once we know which outcomes are considered core by our stakeholders, we will systematically review the psychometric properties of PROMs available, with coverage of core outcomes using the COSMIN criteria [
] and will recommend one to be used in future research on treatment effectiveness. This is a medium- to long-term vision to standardise the definition, measurement, and reporting of outcomes in research on localised renal cancer, with the ultimate aim of improving the decision-making process at all levels.
3.9 Limitations
We may have missed studies reporting PROs and/or QoL because we did not search specifically for primary qualitative studies of patient experiences of renal cancer treatment. However, we will supplement the list of outcomes presented here with outcomes identified in our own primary interview study with patients who have been treated for renal cancer, and further review work. Furthermore, as part of our prioritisation process, participants will be able to propose outcomes they think are missing from our list.
4. Conclusions
Our review indicates that clinical research on localised renal cancer is impeded by heterogeneity in outcome selection, definitions, and reporting. This work represents the first step in the development of a COS that will ultimately improve the evidence basis for treatment of patients with localised renal cancer and the process for creating clinical practice guidelines, and will facilitate treatment decision-making by health care professionals and patients.
Author contributions: Katharina Beyer had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Beyer, Widdershoven, Van Hemelrijck, Bex, Zondervan, MacLennan.
Acquisition of data: Beyer, Widdershoven, MacLennan.
Analysis and interpretation of data: Beyer, Zondervan, MacLennan.
Drafting of the manuscript: Beyer, Zondervan, MacLennan.
Critical revision of the manuscript for important intellectual content: Beyer, Widdershoven, Wintner, Dabestani, Marconi, Moss, Kinsella, Yuan, Giles, Barod, Van Hemelrijck, Bex, Zondervan, MacLennan.
Statistical analysis: None.
Obtaining funding: None.
Administrative, technical, or material support: Beyer, Widdershoven, Yuan.
Supervision: Zondervan, MacLennan.
Other: None.
Financial disclosures: Katharina Beyer certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.
Funding/Support and role of the sponsor: None.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
A stage-for-stage and grade-for-grade analysis of cancer-specific mortality rates in renal cell carcinoma according to age: a competing-risks regression analysis.
Updating and integrating core outcome sets for localised, locally advanced, metastatic, and nonmetastatic castration-resistant prostate cancer: an update from the PIONEER Consortium.
A psychometric evaluation of the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI-19) among renal cell carcinoma patients suggesting an alternative two-factor structure.
Decat Bergerot C, Psutka S, Plimack E, et al. Relevance of items in the FACT Kidney Cancer Symptom Inventory-19 (FKSI-19): results of a patient survey. Presented at the 2022 European International Kidney Cancer Symposium (EIKCS), Antwerp, April 22–24, 2022.
02702-1&id=gr1.jpg){kind=link}
02702-1&id=gr2.jpg){kind=link}