Advertisement

A Systematic Review of Heterogeneity in Outcome Definition and Reporting in Localised Renal Cancer

Open AccessPublished:December 16, 2022DOI:https://doi.org/10.1016/j.euros.2022.11.014

      Abstract

      Context

      Outcomes in renal cell carcinoma (RCC) are reported inconsistently, with variability in definitions and measurement. Hence, it is difficult to compare intervention effectiveness and synthesise outcomes for systematic reviews and to create clinical practice guidelines. This uncertainty in the evidence makes it difficult to guide patient-clinician decision-making. One solution is a core outcome set (COS): an agreed minimum set of outcomes.

      Objective

      To describe outcome reporting, definitions, and measurement heterogeneity as the first stage in co-creating a COS for localised renal cancer.

      Evidence acquisition

      We systematically reviewed outcome reporting heterogeneity in effectiveness trials and observational studies in localised RCC. In total, 2822 studies (randomised controlled trials, cohort studies, case-control studies, systematic reviews) up to June 2020 meeting our inclusion criteria were identified. Abstracts and full texts were screened independently by two reviewers; in cases of disagreement, a third reviewer arbitrated. Data extractions were double-checked.

      Evidence synthesis

      We included 149 studies and found that there was inconsistency in which outcomes were reported across studies and variability in the definitions used for outcomes that were conceptually the same. We structured our analysis using the outcome classification taxonomy proposed by Dodd et al. Outcomes linked to adverse events (eg, bleeding, outcomes linked to surgery) and renal injury outcomes (reduced renal function) were reported most commonly. Outcomes related to deaths from any cause and from cancer were reported in 44% and 25% of studies, respectively, although the time point for measurement and the analysis methods were inconsistent. Outcomes linked to life impact (eg, global quality of life) were reported least often. Clinician-reported outcomes are more frequently reported than patient-reported outcomes in the renal cancer literature.

      Conclusions

      This systematic review underscores the heterogeneity of outcome reporting, definitions, and measurement in research on localised renal cancer. It catalogues the variety of outcomes and serves as a first step towards the development of a COS for localised renal cancer.

      Patient summary

      We reviewed studies on localised kidney cancer and found that multiple terms and definitions have been used to describe outcomes. These are not defined consistently, and often not defined at all. Our review is the first phase in developing a core outcome set to allow better comparisons of studies to improve medical care.

      Keywords

      1. Introduction

      Renal cell carcinoma (RCC) represents 2.2% of all new cancers worldwide [
      • Ferlay J.
      • Steliarova-Foucher E.
      • Lortet-Tieulent J.
      • et al.
      Cancer incidence and mortality patterns in Europe: estimates for 40 countries in 2012.
      ,
      • Sung H.
      • Ferlay J.
      • Siegel R.L.
      • et al.
      Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
      ]. With the increase in reporting of incidental findings, a greater proportion of patients newly diagnosed with renal cancer currently present with stage I disease [
      • Sun M.
      • Abdollah F.
      • Bianchi M.
      • et al.
      A stage-for-stage and grade-for-grade analysis of cancer-specific mortality rates in renal cell carcinoma according to age: a competing-risks regression analysis.
      ,
      • Rosiello G.
      • Larcher A.
      • Montorsi F.
      • et al.
      Renal cancer: overdiagnosis and overtreatment.
      ]. Historically, surgery has been the standard of care for localised renal cancer, but international guidelines have more recently proposed ablative treatments and active surveillance as alternative options [
      • Martini A.
      • Larcher A.
      • Bravi C.A.
      • et al.
      How to select the optimal candidates for renal mass biopsy.
      ,
      • Ljungberg B.
      • Albiges L.
      • Abu-Ghanem Y.
      • et al.
      European Association of Urology guidelines on renal cell carcinoma: the 2022 update.
      ]. Currently, oncological outcomes across treatments are similar and treatment decision-making is multifactorial [
      • Beyer K.
      • Barod R.
      • Fox L.
      • et al.
      The current evidence for factors that influence treatment decision making in localized kidney cancer: a mixed methods systematic review.
      ].
      Across many clinical areas including urology, patient-reported outcomes and clinical outcomes are reported inconsistently, with variability in definitions and measurement, for instance in the settings of localised prostate cancer and bladder cancer [
      • Williamson P.R.
      • Altman D.G.
      • Bagley H.
      • et al.
      The COMET handbook: version 1.0.
      ,
      • MacLennan S.
      • Williamson P.R.
      The need for core outcome sets in urological cancer research.
      ,
      • Meregaglia M.
      • Ciani O.
      • Banks H.
      • et al.
      A scoping review of core outcome sets and their ‘mapping’ onto real-world data using prostate cancer as a case study.
      ]. This makes it very difficult to compare and synthesise outcomes to improve guidelines to better direct and support patients and clinicians during treatment decision-making and ultimately improve results in clinical practice [
      • Violette P.D.
      • Jewett M.A.S.
      • Richard P.O.
      Evidence-based urology: trustworthy guidelines.
      ,
      • Kirkham J.J.
      • Dwan K.M.
      • Altman D.G.
      • et al.
      The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews.
      ]. A core outcome set (COS) is a standardised set of prioritised outcomes and is proposed by current research as a solution to decrease heterogeneity in collection, reporting, and analysis of outcomes. COS in urology are needed because inconsistencies and variability cause not only frustration but also potentially problematic conclusions [
      • MacLennan S.
      • Williamson P.R.
      The need for core outcome sets in urological cancer research.
      ]. This issue is also clearly apparent for localised renal cancer, and ultimately results in barriers for the multifactorial process of decision-making [
      • Beyer K.
      • Barod R.
      • Fox L.
      • et al.
      The current evidence for factors that influence treatment decision making in localized kidney cancer: a mixed methods systematic review.
      ].
      The aim of this systematic review was to identify which outcomes are reported in intervention effectiveness research in localised kidney cancer and to assess heterogeneity in outcome definitions and measurements. It constitutes the initial stage in the development of a COS for localised renal cancer with the intention of identifying a minimum set of outcomes that are potentially important to health care professionals and patients. The outcomes identified in this systematic review are organised under the taxonomy developed by Dodd et al. [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ], which helps to structure general health research vocabularies to reduce inconsistencies. It is embedded in a larger project registered in the Core Outcome Measures in Effectiveness Trials (COMET) database [
      • MacLennan S.
      • Marconi L.
      • Van Hemelrijck M.
      • Beyer K.
      • Bex A.
      • Zondervan P.
      Renal cancer Core Outcome Sets (R-COS) programme encompassing 1) localised renal cell carcinoma (RCC) 2) locally advanced RCC and 3) metastatic RCC.
      ], and uses the same robust methodology that was already followed for the prostate cancer COS [
      • Beyer K.
      • Moris L.
      • Lardas M.
      • et al.
      Updating and integrating core outcome sets for localised, locally advanced, metastatic, and nonmetastatic castration-resistant prostate cancer: an update from the PIONEER Consortium.
      ] developed in collaboration with the European Association of Urology.

      2. Evidence acquisition

      This systematic review followed the guidelines of the COMET initiative, an international expert body that established guidelines on how to develop methodologically robust COS. We report our study in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and the COS-STAR reporting guidelines, which are relevant to this stage of COS development [
      • Kirkham J.J.
      • Gorst S.
      • Altman D.G.
      • et al.
      Core Outcome Set-Standards for Reporting: the COS-STAR statement.
      ]. A project steering committee (S.M., M.V.H., P.Z., A.B., L.M., S.D., R.B., N.K.) supported the development from a methodological and clinical perspective. The study protocol was registered on PROSPERO (ID: CRD42020198605).

      2.1 Aims and objectives

      The aim of this project was to systematically review which outcomes have been reported in effectiveness trials and observational studies in localised renal cancer, and how they were defined and measured.

      2.2 Identification of relevant studies

      We searched Medline, EMBASE, Cochrane CENTRAL, and Cochrane Database of Systematic Reviews (via Ovid) from inception to June 2020. We worked with an information scientist to design the search strategy (Supplementary Fig. 1). To balance the feasibility and precision of the search, we used a two-step approach. First, we identified all published systematic reviews and intervention trials related to RCC without limiting the search to localised renal cancer, and we screened the reference lists in all the articles as a pragmatic way to identify primary studies potentially meeting our inclusion criteria. Second, we searched for and screened all interventional studies on localised RCC from 2015 onwards without limiting the study design. We included randomised controlled trials (RCTs), cohort studies, and case-control studies that reported on eligible interventions for localised renal cancer. We excluded case studies owing to their low level of evidence according to the Oxford Centre for Evidence-Based Medicine (level of evidence 4 or lower [
      • MacLennan S.
      • Marconi L.
      • Van Hemelrijck M.
      • Beyer K.
      • Bex A.
      • Zondervan P.
      Renal cancer Core Outcome Sets (R-COS) programme encompassing 1) localised renal cell carcinoma (RCC) 2) locally advanced RCC and 3) metastatic RCC.
      ]) and the unlikelihood of changing clinical practice. We also excluded conference abstracts.

      2.2.1 Study participants

      Adults (male and female) with suspected localised renal cancer (N0M0 according to the TNM classification; all versions of the TNM staging system) on magnetic resonance imaging, computed tomography, or ultrasound imaging were included.
      Those undergoing treatment for renal metastasis or other tumours were excluded.

      2.2.2 Intervention and comparator

      Studies reporting on any intervention for localised renal cancer were retained, including but not limited to active surveillance, radical nephrectomy (all modes and approaches), partial nephrectomy (all modes and approaches), cryoablation, radiofrequency ablation, microwave ablation, irreversible electroporation, watchful waiting, high-intensity focused ultrasound, or radiotherapy.

      2.2.3 Eligibility of studies

      All abstracts and full texts were screened independently by at least two reviewers (C.W., K.B.). Any disagreements were arbitrated by a third review author (S.M).

      2.3 Data extraction

      Data were independently extracted from the studies included by two researchers (C.W., K.B.) and checked for accuracy by another reviewer (S.M.). We extracted data on study design; author details; year and journal of publication; intervention(s) under investigation; each effectiveness outcome reported; whether the outcome was defined or not; the definition used; the indicators and/or tool(s) used to operationalise or measure the outcome; the time point or period for outcome measurement; and how the outcome was reported.

      2.4 Data analysis and synthesis

      The outcome names extracted were coded and categorised according to the outcome reporting taxonomy developed by Dodd et al. [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ], which has been suggested by COMET for classification of outcomes and group domains.

      2.5 Assessment of risk of bias

      A risk-of-bias assessment was not conducted, as no estimation of the effect size of treatments was conducted and only qualitative information containing terminology was extracted.

      3. Evidence synthesis

      3.1 Characteristics of the studies included

      Our initial search returned 2785 abstracts. Of these, we assessed 319 full-text articles, of which 149 were included (Fig. 1). Of the 149 studies included, 97% were observational studies and five (3%) were RCTs.
      Figure thumbnail gr1
      Fig. 1Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.

      3.2 Heterogeneity in outcome reporting, detection, and definitions

      A suitable outcome taxonomy for health research must differentiate between high-level outcome domain classifications, and comprehensively classify all outcomes, while also proposing a standardised terminology. Therefore, we reported and organised the outcomes in the studies under the taxonomy developed by Dodd et al. [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ] and recommended by the COMET initiative. Taxonomies help to structure vocabularies for general health research to reduce inconsistencies and ambiguities in how current studies describe and define outcomes. The Dodd taxonomy has been proposed to increase the reuse value of outcome data. The taxonomy comprises 38 outcome domains within five core areas: death, adverse events, life impact, physiological/clinical, and resource use [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ]. The core outcomes are further subclassified as shown in Figure 2.
      Figure thumbnail gr2
      Fig. 2The Dodd
      [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ]
      taxonomy applied for classification.
      Table 1 lists the outcomes reported by the studies by domain and highlights the heterogeneity of outcomes identified. We have merged the synonyms and redundant terms. The next section explains the heterogeneity of the terminology in more detail. Table 2 shows which outcomes were reported in the studies included in the review.
      Table 1Outcomes classified according to the taxonomy of Dodd et al.
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      DeathAdverse events/effectsPhysiological or clinicalResource useLife impact
      Mortality/survival38. Adverse events/effects2–24. Physiological/clinicalResource useFunctioning
      Mortality/survival

      Overall survival
      • OS rate
      • Cumulative survival
      • Stage-related OS
      • Mean OS
      • Survival probability 1 yr


      Deaths
      • Deaths
      • Death from any cause


      Mortality
      • Other-cause mortality
      • Mortality events
      • Overall mortality
      • Total mortality
      • X-day mortality


      Cancer-specific survival
      • RCC-specific survival
      • Recurrence-free survival


      Death from kidney cancer
      • Number of patients deceased at last follow-up
      • Death from kidney cancer
      • RCC death
      • Death from RCC
      • Death due to cancer


      Cancer-specific mortality
      • Cancer-specific mortality
      • Death from nonRCC, other-cause mortality
      38. Adverse events

      Complications
      • Surgical complications
      • Intraoperative complications
      • Conversion to nephrectomy
      • Short-term complications
      • Conversions
      • Grade I and grade II complications
      • Highest complication grade
      • Overall complications
      • 30-d postoperative complications


      Bleeding
      • Bleeding severity
      • Units of blood transfused during hospitalisation
      • Estimated bleeding
      • Bleeding-related complications
      • Haemoglobin postoperatively


      Perioperative

      Surgical margins
      • Surgical margins
      • Negative margins
      • Positive surgical margins


      Outcomes linked to surgery
      • Operation time
      • WIT
      • Surgical time
      • Drainage time
      • Procedure time
      • Pneumoperitoneum time
      • Suture time
      • WIT ≤25 min
      • Conversions
      • Open conversion
      • Average clamping time
      • Haemoglobin after surgery
      • Postoperative drainage time
      • Intra-abdominal pressure
      • Adverse health outcomes
      • Clampless rate


      Blood loss
      • Mean estimated blood loss
      • Estimated blood loss
      • Changes in estimated blood loss
      • Units of blood transfused during surgery
      • Transfusion requirement
      • Transfusion rate
      • Intraoperative transfusion
      • Intraoperative ES transfusion
      • Transfusions received
      • Perioperative allogenic blood transfusion
      • Percentage blood transfusion
      • BUN after 1 d and 1 mo


      Trifecta/pentafecta
      • Trifecta
      • Trifecta rate
      • Pentafecta reached
      • Highest complication grade
      • Low-grade complication
      19. Renal and injury outcomes

      New CKD
      • CKD probability


      CKD stage
      • CKD stage
      • Upgrading to CKD grade III–V
      • CKD upstaging
      • No CKD upstaging
      • Postoperative CKD stage
      • Postoperative new onset of stage III or IV CKD
      • Final CKD stage
      • Patients with acquired stage III–V CKD at follow-up, compared to preoperative


      Time to CKD
      • Decline in CKD stage
      • Progression to CKD
      • CKD upstaged–free survival
      • De novo CKD stage III
      • Survival without CKD upstaging
      • Time to diagnosis of CKD


      Outcomes linked to procedure
      • Mean ablation time
      • Laser excision time
      • Median procedure time
      • Renal outcomes
      • Urinary function
      • Oncological outcomes
      • Collecting system entry
      • Haemostatic agent


      eGFR
      • Mean eGFR change
      • Median eGFR preservation
      • Median percentage eGFR change
      • Change in GFR
      • eGFR preservation (%)
      • Latest eGFR preservation
      • Δ GFR change
      • Last eGFR
      • eGFR 1-yr post operation
      • Percentage change in eGFR
      • eGFR decrease
      • Postoperative eGFR change (%) from baseline to 1-yr follow‐up


      Serum creatinine
      • Preoperative creatinine
      • Creatinine level
      • Serum creatinine
      • Difference in serum creatinine between preoperative and postoperative levels
      • Postoperative creatinine level
      • Postoperative creatinine
      • Latest creatinine level
      • Percentage change in creatinine
      • Variation of creatinine
      • Δ creatinine


      Recurrence-free survival
      • RFS + time
      34. Economic

      Cost
      • Health care expenditure
      • Medical cost
      • Total cost
      • Imaging (linked to costs)
      • Medications?








      36. Need for further intervention
      • Readmission








      35. Hospital
      • Length of stay
      • Postoperative HSP time
      • HSP time
      • Hospital stay
      • Average hospital stay
      • Duration of HSP
      • Duration of postoperative hospital stay
      • Median hospital stay
      • Surgical supplies and devices
      • Operating room
      31. Perceived health status
      • Perceived health
      • Pain
      • Adverse health outcomes








      30. Global quality of life
      • Health-related quality of life
      38. Adverse events/effects or 35. Resource use: hospital
      • Dialysis free probability
      • No. requiring dialysis
      • Temporary dialysis
      • Permanent dialysis
      16. Outcomes relating to neoplasms: benign, malignant and unspecified (including cysts and polyps)

      Metastasis
      • Distant metastasis–free survival
      • Distant metastasis
      • Extrarenal metastasis


      Follow-up
      • Follow-up
      • Long-term outcomes
      • Median postoperative follow-up time
      • Average length of follow-up
      • Median follow-up time


      Progression-free survival
      • PFS
      • Systemic PFS
      • Clinical PFS


      Progression
      • Local tumour progression
      • Disease progression


      Recurrence
      • Local recurrence
      • Disease-free survival
      • Recurrence rate
      • Recurrence-free survival
      • Recurrence result


      Recurrence linked to time
      • Time to local recurrence
      • Events of local recurrence
      • Delayed recurrence
      • Time to recurrence
      • Local recurrence rate
      • Local recurrence–free survival
      • Recurrence (local or metastatic)
      • Local ipsilateral recurrence
      • Disease-free survival
      BUN = blood urea nitrogen; CKD = chronic kidney disease; eGFR = estimated glomerular filtration rate; ES = erythrocyte suspension; HSP = hospitalisation; OS = overall survival; PFS = progression-free survival; RCC = renal cell carcinoma; WIT = warm ischaemia time.
      Table 2Outcomes reported in each study after classification according to the taxonomy suggested by Dodd et al.
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      First authorDesignLocationDeathAdverse eventsPhysiological or clinicalResource useLife impact
      OSCSSAEsAEs/EHNFIPHIGQL
      ERUHRIOORN
      PatelOBSUSXXX
      LiOBSChinaXXX
      WangOBSChinaXXXX
      MorkosOBSUSAXXXXX
      AlshyarbaOBSSaudi ArabiaXX
      WuOBSChinaXXX
      PackiamOBSUSAXXX
      YangOBSNot statedXXX
      RembeyoOBSFranceXXXXX
      UhligOBSUSAX
      YuOBSChinaXXXXX
      KartalOBSTurkeyXXXX
      JalbaniOBSPakistanXXX
      SeonOBSSouth KoreaXXXXX
      ChoiOBSSouth KoreaXXX
      TanOBSUSA, Puerto RicoX
      GrantOBSUSAX
      ChenRCTChinaXXX
      LiuOBSChinaXXX
      SandbergenOBSNetherlandsXX
      ShapiroOBSUSAXXXX
      De CobelliOBSItalyXX
      NayanOBSCanadaXXXX
      JinOBSChinaXXX
      MouraoOBSUSA, SpainXXXXX
      AnglickisOBSLithuaniaXXXXX
      MarchioniOBSUSAXX
      LiOBSChinaX
      LiaoOBSUSAXX
      SimoneOBSItalyXXXXX
      ShaoOBSTaiwanXXX
      AntonelliRCTItalyXXX
      KitleyOBSUSAX
      ZhouOBSChinaXX
      AndrewsOBSUSAXX
      ZhouOBSUSAXXX
      FraisseOBSFranceXXXX
      HuOBSChinaX
      Abu-GhanemOBSIsraelXXXX
      KavaricOBSMontenegroXXX
      ZiegelmuellerOBSGermanyXXX
      TalenfeldOBSUSAXXX
      BhindiOBSUSAXXXXX
      LarcherOBSNetherlands, ItalyXXXXX
      XingOBSUSAXXX
      RistauOBSUSA, Puerto RicoX
      ZhaoOBSChinaXXXX
      GershmanOBSUSAXXXX
      BenoitOBSFranceXXXX
      PaulucciOBSUSAXXXX
      Abdel RaheemOBSSouth KoreaXXXXX
      LourencoOBSCanadaXXXXX
      HasegawaOBSJapanXXX
      StrejaOBSUSAXX
      BorghesiOBSGloballyXXX
      UhligOBSUSAXXX
      YeOBSChinaXXX
      ParkOBSNot stated
      VenkatramaniOBSUSAXX
      UhligOBSUSAX
      ZhangOBSUSAXX
      LeeOBSSouth KoreaXXXX
      ChongOBSUSAXXX
      ChangOBSTaiwanXXX
      YangOBSChinaXXX
      VeysOBSBelgiumXXXXX
      BanapourOBSUSAXXX
      CaiOBSChinaXXX
      LanchonOBSFranceXXX
      VenkatramaniOBSUSAXX
      KaralliRCTSwedenX
      DongOBSNot statedX
      WangOBSUSAX
      TangOBSUSAXX
      YinOBSChinaXX
      ShahOBSUSAX
      AnninoOBSItalyXX
      WangOBSChinaXX
      ShumOBSUSAXXXX
      LuoOBSUSAXX
      LeeOBSSouth KoreaXXXX
      CaputoOBSUSAXXXX
      LuOBSChinaXXX
      MaricOBSSerbiaX
      MateiOBSItalyXXXXX
      PaulucciOBSUSAXX
      RassweilerOBSGermanyX
      LarcherOBSUSAXXXXXX
      LenisOBSUSAXX
      WangOBSUSAXX
      PengOBSChinaXXXXX
      MalkocOBSUSAXXXX
      LongOBSFranceXXXXXXX
      YooOBSSouth KoreaX
      RedondoOBSSpainXX
      CarrionOBSSpainXX
      ShahOBSUSAXX
      MoskowitzOBSUSAX
      HuangRCTChinaXXX
      LarcherOBSUSAX
      JangOBSSouth KoreaXXXXX
      ForbesOBSCanadaXXXX
      KaraOBSNot statedXXXX
      TakagiOBSNot statedXXXX
      OhOBSNot statedX
      AndradeOBSNot statedXXXX
      DongOBSNot statedXXXXX
      TrudeauOBSUSAXX
      LaiOBSChinaXXXX
      LiuOBSChinaXXXX
      PantelidouOBSUKXXXX
      LiuOBSChinaXXXX
      LarcherOBSUSAXX
      HosseinOBSIranXX
      KomatsudaOBSJapanXX
      JanicicOBSSerbiaXX
      LyonOBSUSAXX
      SatkunasivamOBSUSAXXXXX
      ThompsonOBSUSAXXX
      TabayoyongOBSUSAX
      AlaneeOBSUSAX
      ZargarOBSUSAXXX
      ManoOBSUSAXXXXX
      ChangOBSChinaXXXXXX
      SerniOBSItalyXXXXX
      ChungOBSKoreaXXX
      YuOBSNot statedXX
      WeinbergOBSUSAXXX
      ParkRCTSouth KoreaXXXX
      BalasarOBSTurkeyXX
      O’MalleyOBSUSAXXX
      KimOBSSouth KoreaXX
      ChangOBSChinaXX
      CooperOBSUSAXX
      AlamOBSUSAXXX
      ÇömezOBSTurkeyXXX
      KoppOBSUSAXXXX
      DanzigOBSUSAXX
      HusseinOBSEgyptX
      SimsekOBSTurkeyXXX
      FossatiOBSItalyXXX
      JiOBSItaly, ChinaXXXXXXX
      MasonOBSUSAX
      ChehabOBSNot statedX
      AnOBSUSAXX
      RosenOBSUSAXX
      RamirezOBSUSAXX
      MalkocOBSTurkeyXXX
      AEs = adverse events; CSS = cancer-specific survival; E = economic resource use; ERUH = effects or resource use: hospital; GQL = global quality of life; H = hospital resource use; NFI = need for further intervention; OBS = observational study; ORN = outcomes relating to neoplasms; OS = overall survival; PHI = perceived health impact; RCT = randomised controlled trial; RIO = renal and injury outcomes.

      3.3 Death (mortality/survival)

      Death was reported 103 times. We categorised these outcomes according to the Dodd mortality/survival classification into “overall survival” and “cancer-specific survival”. Overall survival (OS) was measured in 65 studies (44%; Table 2) as OS, death, or mortality; more details are provided in Table 1. Cancer-specific survival (CSS) was reported in 43 (29%) of the studies (Table 2) as CSS, death from renal cancer, or cancer-specific mortality.
      Definitions of OS and CSS differed across studies (Table 1). The heterogeneity for the definitions was linked to time points. For instance, some studies started measurement at diagnosis, whereas others used the treatment date as the starting point for survival. The time endpoint also differed, with studies reporting either a rate at a defined time (eg, at 10 yr) or a hazard ratio based on survival analyses.

      3.4 Adverse events (adverse events/effects)

      Adverse events (AEs) were the most common outcome reported (n = 101, 68%). However, many different types of AE were reported, sometimes as the number or percentage of patients experiencing the outcome and sometimes subsumed in a classification system linked to severity or consequences (eg, the Clavien-Dindo scheme). Examples of events that play a role in AE assessment include bleeding, operation time, warm ischaemia time, intra-abdominal pressure, surgical time, drainage time, serum creatinine, blood loss, trifecta/pentafecta outcomes, and dialysis, which are linked to the complexity of the surgery. Many articles reported several AEs within one study, but the AEs reported varied across studies (eg, surgical complications were measured as intraoperative complications, conversion to nephrectomy, or short-term complications).
      “Adverse events/effects or resource use: hospital” outcomes were reported in six studies (4%). Outcome reporting and measurement were inconsistent; examples include dialysis-free probability, number requiring dialysis, temporary dialysis, and permanent dialysis (Table 1).

      3.5 Life impact/functioning (perceived health status; global quality of life)

      Only eight studies (5%) reported outcomes reflecting life impact. Five studies (3%) reported outcomes classified as perceived health status. Three studies (2%) reported on global quality of life (QoL), one study using the Short Form (SF)-36 and another using the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI)-15 patient-reported outcome measures (PROMs; Table 2).

      3.6 Physiological/clinical (physiological or clinical)

      Physiological or clinical outcomes were subclassified as “renal and injury outcomes” (eg, new chronic kidney disease [CKD], CKD stage, time to CKD), which were defined very heterogeneously and reported in 87 (58%) of the studies; and “outcomes relating to neoplasms” (linked to cancer follow-up and progression, reported in 55 studies [37%]) according to the Dodd taxonomy [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ] (Table 1, Table 2).

      3.7 Resource use (economic, need for further intervention, hospital)

      Resource use consisted of the subcategories “economic resource” (eg, health care expenditure; reported in four studies (3%) as mean or median costs), “need for further intervention” (eg, readmission; reported in eight studies [5%] as a binary yes/no result or median value), and “hospital” (eg, length of hospital stay, reported in 58 studies [39%] as mean or median length of hospital stay in days).

      3.8 Discussion

      To the best of our knowledge, this is the first systematic review of outcome reporting heterogeneity in the literature on localised renal cancer. Our results build a framework for developing a COS for localised renal cancer with the aim of reducing heterogeneity for outcome definitions, measurement, and reporting.
      Our systematic review highlights the persisting problem of outcome reporting heterogeneity in studies on localised renal cancer. Multiple terms are used to refer to conceptually similar outcomes, and there is variation in the outcome definitions used. This has not improved over time and is problematic when summarising the evidence base for treatment effectiveness to inform decision-making, because it is not advisable to synthesise data with different outcome definitions within a meta-analysis. Such a practice can produce meaningless summary statistics that may be given more credibility than they are due. Therefore, a cumbersome and often less-informative narrative synthesis must be undertaken instead. Furthermore, our work highlights variety in data reporting and measurement. For instance, if dichotomous outcomes such as OS and CSS are reported using different methods (eg, some studies report adjusted and some unadjusted hazard ratios, others report a rate at median follow-up or at specified time points such as 1 yr or 5 yr), then these data cannot be easily or reliably synthesised in a meta-analysis. When these problems all occur together, then it is difficult to interpret the body of evidence and clinical practice guideline panels encounter challenges in drawing up recommendations and applying certainty-of-evidence attributes such as those proposed by the GRADE working group [

      Schünemann H, Brożek J, Guyatt G, Oxman A. GRADE handbook. Updated October 2013. https://gdt.gradepro.org/app/handbook/handbook.html.

      ].
      Worryingly, we identified very few patient-reported outcomes (PROs), which might be related to the limited number of specific tools available for capturing QoL for renal cancer. In their systematic review, Rossi et al. [
      • Rossi S.H.
      • Klatte T.
      • Stewart G.D.
      Quality of life outcomes in patients with localised renal cancer: a literature review.
      ] identified three generic PROMs (RAND medical outcome survey SF-36 and SF-12, EuroQol [EQ-5D], Convalescence and Recovery Evaluation [CARE]) and eight cancer-specific PROMs (Cancer Rehabilitation Evaluation System-Short Form [CARES-SF], European Organisation for Research and Treatment of Cancer [EORTC] Quality of Life Questionnaire [QLQ]-C30, Functional Assessment of Cancer Therapy-General [FACT-G], FKSI, Renal Cell Carcinoma-Symptom Index [RCC-SI], Instruments to assess psychological wellbeing Impact of Events Scale [IES], Hospital Anxiety and Depression Scale [HADS], Mishel Uncertainty in Illness Scale (MUIS)) which are currently being used in renal cancer. However, of the PROM instruments used, only two are specific to renal cancer and are not stage-specific (FKSI, RCC-SI) [
      • Rossi S.H.
      • Klatte T.
      • Stewart G.D.
      Quality of life outcomes in patients with localised renal cancer: a literature review.
      ].
      In their study of the symptom index most commonly used for renal cancer, Rosenblad et al. [
      • Rosenblad A.K.
      • Sundqvist P.
      • Westman B.
      • et al.
      A psychometric evaluation of the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI-19) among renal cell carcinoma patients suggesting an alternative two-factor structure.
      ] assessed the psychometric properties of the FKSI-19 (which captures physical and emotional disease-related symptoms, function/wellbeing, and treatment side effects) among patients with RCC and reported that it is barely fit for this purpose. Decat Bergerot et al. [

      Decat Bergerot C, Psutka S, Plimack E, et al. Relevance of items in the FACT Kidney Cancer Symptom Inventory-19 (FKSI-19): results of a patient survey. Presented at the 2022 European International Kidney Cancer Symposium (EIKCS), Antwerp, April 22–24, 2022.

      ] conducted a patient survey that identified many of the FKSI-19 questions as irrelevant from a patient perspective and stressed the need to incorporate patients in the development of PRO tools to determine areas of importance to them. The EORTC Quality of Life group is currently developing an RCC-specific module to be used in combination with their QLQ-C30 instrument.
      Our project steering group includes clinical RCC experts, patient advocacy groups, methodologists, and guideline developers from the European Association of Urology (most are co-authors of this study). We aim to use these networks to improve recruitment to our research stages, and to subsequently endorse and disseminate the final COS as part of our implementation strategy.
      We curated the different terms used and collated them using a standardised outcome classification taxonomy [
      • Dodd S.
      • Clarke M.
      • Becker L.
      • et al.
      A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
      ] as a first step in creating a COS for localised renal cancer. In the next step we will use consensus processes in a multistakeholder group to prioritise which outcomes are core and to recommend definitions for each outcome. Once we know which outcomes are considered core by our stakeholders, we will systematically review the psychometric properties of PROMs available, with coverage of core outcomes using the COSMIN criteria [
      • Mokkink L.B.
      • de Vet H.C.W.
      • Prinsen C.A.C.
      • et al.
      COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
      ,
      • Prinsen C.A.C.
      • Mokkink L.B.
      • Bouter L.M.
      • et al.
      COSMIN guideline for systematic reviews of patient-reported outcome measures.
      ] and will recommend one to be used in future research on treatment effectiveness. This is a medium- to long-term vision to standardise the definition, measurement, and reporting of outcomes in research on localised renal cancer, with the ultimate aim of improving the decision-making process at all levels.

      3.9 Limitations

      We may have missed studies reporting PROs and/or QoL because we did not search specifically for primary qualitative studies of patient experiences of renal cancer treatment. However, we will supplement the list of outcomes presented here with outcomes identified in our own primary interview study with patients who have been treated for renal cancer, and further review work. Furthermore, as part of our prioritisation process, participants will be able to propose outcomes they think are missing from our list.

      4. Conclusions

      Our review indicates that clinical research on localised renal cancer is impeded by heterogeneity in outcome selection, definitions, and reporting. This work represents the first step in the development of a COS that will ultimately improve the evidence basis for treatment of patients with localised renal cancer and the process for creating clinical practice guidelines, and will facilitate treatment decision-making by health care professionals and patients.
      Author contributions: Katharina Beyer had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
      Study concept and design: Beyer, Widdershoven, Van Hemelrijck, Bex, Zondervan, MacLennan.
      Acquisition of data: Beyer, Widdershoven, MacLennan.
      Analysis and interpretation of data: Beyer, Zondervan, MacLennan.
      Drafting of the manuscript: Beyer, Zondervan, MacLennan.
      Critical revision of the manuscript for important intellectual content: Beyer, Widdershoven, Wintner, Dabestani, Marconi, Moss, Kinsella, Yuan, Giles, Barod, Van Hemelrijck, Bex, Zondervan, MacLennan.
      Statistical analysis: None.
      Obtaining funding: None.
      Administrative, technical, or material support: Beyer, Widdershoven, Yuan.
      Supervision: Zondervan, MacLennan.
      Other: None.
      Financial disclosures: Katharina Beyer certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.
      Funding/Support and role of the sponsor: None.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Ferlay J.
        • Steliarova-Foucher E.
        • Lortet-Tieulent J.
        • et al.
        Cancer incidence and mortality patterns in Europe: estimates for 40 countries in 2012.
        Eur J Cancer. 2013; 49: 1374-1403https://doi.org/10.1016/j.ejca.2012.12.027
        • Sung H.
        • Ferlay J.
        • Siegel R.L.
        • et al.
        Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
        CA Cancer J Clin. 2021; 71: 209-249https://doi.org/10.3322/caac.21660
        • Sun M.
        • Abdollah F.
        • Bianchi M.
        • et al.
        A stage-for-stage and grade-for-grade analysis of cancer-specific mortality rates in renal cell carcinoma according to age: a competing-risks regression analysis.
        Eur Urol. 2011; 60: 1152-1159https://doi.org/10.1016/j.eururo.2011.07.064
        • Rosiello G.
        • Larcher A.
        • Montorsi F.
        • et al.
        Renal cancer: overdiagnosis and overtreatment.
        World J Urol. 2021; 39: 2821-2823https://doi.org/10.1007/s00345-021-03798-z
        • Martini A.
        • Larcher A.
        • Bravi C.A.
        • et al.
        How to select the optimal candidates for renal mass biopsy.
        Eur Urol Oncol. 2021; 4: 506-509https://doi.org/10.1016/j.euo.2020.10.001
        • Ljungberg B.
        • Albiges L.
        • Abu-Ghanem Y.
        • et al.
        European Association of Urology guidelines on renal cell carcinoma: the 2022 update.
        Eur Urol. 2022; 82: 399-410https://doi.org/10.1016/j.eururo.2022.03.006
        • Beyer K.
        • Barod R.
        • Fox L.
        • et al.
        The current evidence for factors that influence treatment decision making in localized kidney cancer: a mixed methods systematic review.
        J Urol. 2021; 206: 827-839https://doi.org/10.1097/ju.0000000000001901
        • Williamson P.R.
        • Altman D.G.
        • Bagley H.
        • et al.
        The COMET handbook: version 1.0.
        Trials. 2017; 18: 280https://doi.org/10.1186/s13063-017-1978-4
        • MacLennan S.
        • Williamson P.R.
        The need for core outcome sets in urological cancer research.
        Transl Androl Urol. 2020; 10: 2832-3285
        • Meregaglia M.
        • Ciani O.
        • Banks H.
        • et al.
        A scoping review of core outcome sets and their ‘mapping’ onto real-world data using prostate cancer as a case study.
        BMC Med Res Methodol. 2020; 20: 41https://doi.org/10.1186/s12874-020-00928-w
        • Violette P.D.
        • Jewett M.A.S.
        • Richard P.O.
        Evidence-based urology: trustworthy guidelines.
        Eur Urol Focus. 2021; 7: 1243-2126https://doi.org/10.1016/j.euf.2021.10.012
        • Kirkham J.J.
        • Dwan K.M.
        • Altman D.G.
        • et al.
        The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews.
        BMJ. 2010; 340: c365
        • Dodd S.
        • Clarke M.
        • Becker L.
        • et al.
        A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery.
        J Clin Epidemiol. 2018; 96: 84-92https://doi.org/10.1016/j.jclinepi.2017.12.020
        • MacLennan S.
        • Marconi L.
        • Van Hemelrijck M.
        • Beyer K.
        • Bex A.
        • Zondervan P.
        Renal cancer Core Outcome Sets (R-COS) programme encompassing 1) localised renal cell carcinoma (RCC) 2) locally advanced RCC and 3) metastatic RCC.
        COMET Initiative. 2019;
        • Beyer K.
        • Moris L.
        • Lardas M.
        • et al.
        Updating and integrating core outcome sets for localised, locally advanced, metastatic, and nonmetastatic castration-resistant prostate cancer: an update from the PIONEER Consortium.
        Eur Urol. 2022; 81: 503-514https://doi.org/10.1016/j.eururo.2022.01.042
        • Kirkham J.J.
        • Gorst S.
        • Altman D.G.
        • et al.
        Core Outcome Set-Standards for Reporting: the COS-STAR statement.
        PLoS Med. 2016; 13: e1002148
      1. Schünemann H, Brożek J, Guyatt G, Oxman A. GRADE handbook. Updated October 2013. https://gdt.gradepro.org/app/handbook/handbook.html.

        • Rossi S.H.
        • Klatte T.
        • Stewart G.D.
        Quality of life outcomes in patients with localised renal cancer: a literature review.
        World J Urol. 2018; 36: 1961-1972https://doi.org/10.1007/s00345-018-2415-3
        • Rosenblad A.K.
        • Sundqvist P.
        • Westman B.
        • et al.
        A psychometric evaluation of the Functional Assessment of Cancer Therapy-Kidney Symptom Index (FKSI-19) among renal cell carcinoma patients suggesting an alternative two-factor structure.
        Qual Life Res. 2021; 30: 2663-2670https://doi.org/10.1007/s11136-021-02839-9
      2. Decat Bergerot C, Psutka S, Plimack E, et al. Relevance of items in the FACT Kidney Cancer Symptom Inventory-19 (FKSI-19): results of a patient survey. Presented at the 2022 European International Kidney Cancer Symposium (EIKCS), Antwerp, April 22–24, 2022.

        • Mokkink L.B.
        • de Vet H.C.W.
        • Prinsen C.A.C.
        • et al.
        COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.
        Qual Life Res. 2018; 27: 1171-1179https://doi.org/10.1007/s11136-017-1765-4
        • Prinsen C.A.C.
        • Mokkink L.B.
        • Bouter L.M.
        • et al.
        COSMIN guideline for systematic reviews of patient-reported outcome measures.
        Qual Life Res. 2018; 27: 1147-1157https://doi.org/10.1007/s11136-018-1798-3