Advertisement
Prostatic Disease| Volume 53, P46-54, July 2023

Comparison of Rotterdam and Barcelona Magnetic Resonance Imaging Risk Calculators for Predicting Clinically Significant Prostate Cancer

  • Author Footnotes
    † These authors contributed equally as co-first authors.
    Juan Morote
    Correspondence
    Corresponding author. Department of Urology, Vall d́Hebron Hospital, Po Vall d́Hebron 119-129, 08035 Barcelona, Spain. Tel. +34 2746100; Fax: +34 2746028.
    Footnotes
    † These authors contributed equally as co-first authors.
    Affiliations
    Department of Urology, Vall d́Hebron Hospital, Barcelona, Spain

    Department of Surgery, Universitat Autònoma de Barcelona, Bellaterra, Spain
    Search for articles by this author
  • Author Footnotes
    † These authors contributed equally as co-first authors.
    Ángel Borque-Fernando
    Footnotes
    † These authors contributed equally as co-first authors.
    Affiliations
    Department of Urology, Hospital Miguel Servet, IIS-Aragon, Zaragoza, Spain
    Search for articles by this author
  • Marina Triquell
    Affiliations
    Department of Urology, Vall d́Hebron Hospital, Barcelona, Spain

    Department of Surgery, Universitat Autònoma de Barcelona, Bellaterra, Spain
    Search for articles by this author
  • Miriam Campistol
    Affiliations
    Department of Urology, Vall d́Hebron Hospital, Barcelona, Spain

    Department of Surgery, Universitat Autònoma de Barcelona, Bellaterra, Spain
    Search for articles by this author
  • Pol Servian
    Affiliations
    Department of Urology, Hospital Germans Trias I Pujol, Badalona, Spain
    Search for articles by this author
  • José M. Abascal
    Affiliations
    Department of Urology, Parc de Salut Mar, Barcelona, Spain

    Department of Surgery, Universitat Pompeu Fabra, Badalona, Spain
    Search for articles by this author
  • Jacques Planas
    Affiliations
    Department of Urology, Vall d́Hebron Hospital, Barcelona, Spain

    Department of Surgery, Universitat Autònoma de Barcelona, Bellaterra, Spain
    Search for articles by this author
  • Olga Méndez
    Affiliations
    Biomedical Research in Urology Unit, Vall d́Hebron Research Institute, Barcelona, Spain
    Search for articles by this author
  • Author Footnotes
    ‡ These authors contributed equally as co-last authors.
    Luis M. Esteban
    Footnotes
    ‡ These authors contributed equally as co-last authors.
    Affiliations
    Department of Applied Mathematics, Escuela Universitaria Politécnica La Almunia, Universidad de Zaragoza, Zaragoza, Spain
    Search for articles by this author
  • Author Footnotes
    ‡ These authors contributed equally as co-last authors.
    Enrique Trilla
    Footnotes
    ‡ These authors contributed equally as co-last authors.
    Affiliations
    Department of Urology, Vall d́Hebron Hospital, Barcelona, Spain

    Department of Surgery, Universitat Autònoma de Barcelona, Bellaterra, Spain
    Search for articles by this author
  • Author Footnotes
    † These authors contributed equally as co-first authors.
    ‡ These authors contributed equally as co-last authors.
Open AccessPublished:May 22, 2023DOI:https://doi.org/10.1016/j.euros.2023.03.013

      Abstract

      Background

      Magnetic resonance imaging (MRI)-based risk calculators (MRI-RCs) individualise the likelihood of clinically significant prostate cancer (csPCa) and improve candidate selection for prostate biopsy beyond the Prostate Imaging Reporting and Data System (PI-RADS).

      Objective

      To compare the Barcelona (BCN) and Rotterdam (ROT) MRI-RCs in an entire population and according to the PI-RADS categories.

      Design, setting, and participants

      A prospective comparison of BCN- and ROT-RC in 946 men with suspected prostate cancer in whom systematic biopsy was performed, as well as target biopsies of PI-RADS ≥3 lesions.

      Outcome measurements and statistical analysis

      Saved biopsies and undetected csPCa (grade group ≥2) were determined.

      Results and limitations

      The csPCa detection was 40.8%. The median risks of csPCa from BCN- and ROT-RC were, respectively, 67.1% and 25% in men with csPCa, whereas 10.5% and 3% in those without csPCa (p < 0.001). The areas under the curve were 0.856 and 0.844, respectively (p = 0.116). BCN-RC showed a higher net benefit and clinical utility over ROT-RC. Using appropriate thresholds, respectively, 75% and 80% of biopsies were needed to identify 50% of csPCa detected in men with PI-RADS <3, whereas 35% and 21% of biopsies were saved, missing 10% of csPCa detected in men with PI-RADS 3. BCN-RC saved 15% of biopsies, missing 2% of csPCa in men with PI-RADS 4, whereas ROT-RC saved 10%, missing 6%. No RC saved biopsies without missing csPCa in men with PI-RADS 5.

      Conclusions

      ROT-RC provided a lower and narrower range of csPCa probabilities than BCN-RC. BCN-RC showed a net benefit over ROT-RC in the entire population. However, BCN-RC was useful in men with PI-RADS 3 and 4, whereas ROT-RC was useful only in those with PI-RADS 3. No RC seemed to be helpful in men with negative MRI and PI-RADS 5.

      Patient summary

      Barcelona risk calculator was more helpful than Rotterdam risk calculator to select candidates for prostate biopsy.

      Keywords

      1. Introduction

      The early detection of prostate cancer (PCa) has evolved towards clinically significant PCa (csPCa), avoiding unnecessary prostate biopsies and overdetection of insignificant tumours. However, PCa suspicion remains based on serum prostate-specific antigen (PSA) elevation and abnormal digital rectal examination (DRE) [
      • Mottet N.
      • van den Bergh R.C.N.
      • Briers E.
      • et al.
      EAU-EANM-ESTRO-ESUR-SIOG guidelines on prostate cancer—2020 update. Part 1: screening, diagnosis, and local treatment with curative intent.
      ,
      • Van Poppel H.
      • Roobol M.J.
      • Chapple C.R.
      • et al.
      Prostate-specific antigen testing as part of a risk-adapted early detection strategy for prostate cancer: European Association of Urology position and recommendations for 2021.
      ,
      • Van Poppel H.
      • Hogenhout R.
      • Albers P.
      • van den Bergh R.C.N.
      • Barentsz J.O.
      • Roobol M.J.
      European model for an organised risk-stratified early detection programme for prostate cancer.
      ]. This paradigm shift has resulted from the spread of prebiopsy multiparametric magnetic resonance imaging (mpMRI), which grades the likelihood of csPCa through Prostate Imaging Reporting and Data System (PI-RADS) [
      • Schoots I.G.
      • Padhani A.R.
      • Rouvière O.
      • Barentsz J.O.
      • Richenberg J.
      Analysis of magnetic resonance imaging-directed biopsy strategies for changing the paradigm of prostate cancer diagnosis.
      ]. The negative predictive value of mpMRI reaches 95%, which makes it possible to avoid prostate biopsies for PI-RADS <3 [
      • Sathianathen N.J.
      • Omer A.
      • Harriss E.
      • et al.
      Negative predictive value of multiparametric magnetic resonance imaging in the detection of clinically significant prostate cancer in the prostate imaging reporting and data system era: a systematic review and meta-analysis.
      ,
      • Wagaskar V.G.
      • Levy M.
      • Ratnani P.
      • et al.
      Clinical utility of negative multiparametric magnetic resonance imaging in the diagnosis of prostate cancer and clinically significant prostate cancer.
      ]. Target biopsies of suspicious lesions (PI-RADS ≥3) increase the csPCa sensitivity of systematic-biopsies. However, csPCa detection does not exceed 20% in men with PI-RADS 3, is approximately 50% in men with PI-RADS 4, and reaches 90% in men with PI-RADS 5 [
      • Mazzone E.
      • Stabile A.
      • Pellegrino F.
      • et al.
      Positive predictive value of Prostate Imaging Reporting and Data System version 2 for the detection of clinically significant prostate cancer: a systematic review and meta-analysis.
      ].
      MRI-predictive models individualise the likelihood of csPCa and improve candidate selection for prostate biopsy, although available risk calculators (RCs) and external validations are essential [
      • Osses D.F.
      • Roobol M.J.
      • Schoots I.G.
      Prediction medicine: biomarkers, risk calculators and magnetic resonance imaging as risk stratification tools in prostate cancer diagnosis.
      ]. Among the 18 MRI-predictive models developed in the last 5 yr, seven have been validated externally and only two RCs are available [
      • Triquell M.
      • Campistol M.
      • Celma A.
      • et al.
      Magnetic resonance imaging-based predictive models for clinically significant prostate cancer: a systematic review.
      ]. Rotterdam (ROT) RC was initially designed from the Rotterdam population of the European Randomised Screening Prostate Cancer (ERSPC) trial [

      Steyberg R, Roobol-Bouts MJ, Kranse M, Schroder FH. Data storage device and method for determining the dependency of the risk for prostate cancer, device and method for indicating a risk for a disease in an individual. U.S. Patent Office, 8,087,576.

      ,
      • Roobol M.J.
      • Steyerberg E.W.
      • Kranse R.
      • et al.
      A risk-based strategy improves prostate-specific antigen-driven detection of prostate cancer.
      ]. ROT MRI-RC has resulted from an adjustment of the previous RCs 3 and 4 to predict PCa and high-grade PCa likelihoods in biopsy-naïve men and those with previous negative prostate biopsy, in 961 men with serum PSA ≥3.0 ng/ml and/or abnormal DRE, in whom systematic biopsy was always performed as a target biopsy to PI-RADSv.1 ≥3 lesions. PI-RADSv.1 score (1–5), age (50–75 yr), biopsy status (initial vs repeat), serum PSA (0.4–50 ng/ml), DRE (normal vs abnormal), and prostate volume (10–110 ml) were included as predictors [
      • Alberts A.R.
      • Roobol M.J.
      • Verbeek J.F.M.
      • et al.
      Prediction of high-grade prostate cancer following multiparametric magnetic resonance imaging: improving the Rotterdam European Randomized Study of Screening for Prostate Cancer risk calculators.
      ]. ROT MRI-RC has been validated in some populations, although recalibrations and risk threshold adjustments have been needed to assure accurate predictions [
      • Püllen L.
      • Radtke J.P.
      • Wiesenfarth M.
      • et al.
      External validation of novel magnetic resonance imaging-based models for prostate cancer prediction.
      ,
      • Chen R.
      • Verbeek J.F.M.
      • Yang Y.
      • Song Z.
      • Sun Y.
      • Roobol M.J.
      Comparing the prediction of prostate biopsy outcome using the Chinese Prostate Cancer Consortium (CPCC) risk calculator and the Asian adapted Rotterdam European Randomized Study of Screening for Prostate Cancer (ERSPC) risk calculator in Chinese and European men.
      ,
      • Petersmann A.L.
      • Remmers S.
      • Klein T.
      • et al.
      External validation of two MRI-based risk calculators in prostate cancer diagnosis.
      ,
      • De Nunzio C.
      • Lombardo R.
      • Baldassarri V.
      • et al.
      Rotterdam mobile phone app including MRI data for the prediction of prostate cancer: a multicenter external validation.
      ]. Barcelona (BCN) MRI-RC was recently designed among 1486 men with PSA ≥3.0 ng/ml and/or abnormal DRE, in whom systematic biopsies were always performed as target biopsies to PI-RADSv.2 ≥3 lesions. BCN MRI-RC has been validated externally and includes the same predictors as ROT MRI-RC without range limitation, PI-RADSv.2, and PCa family history (first degree vs no). The csPCa was defined as the International Society of Urologic Pathology (ISUP) grade group ≥2. BCN MRI-RC has been the first predictive model analysed according to the PI-RADS categories, showing that the efficacy in the entire population does not represents that in each PI-RADS category. In addition, the BCN MRI-RC has the possibility to select the appropriate threshold for adjustments in validation studies and according to the PI-RADS categories [
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ].
      Owing to differences between ROT and BCN MRI-RCs, we hypothesise different behaviour for improving candidate selection for prostate biopsy. We aim to compare the clinical usefulness of ROT and BCN MRI-RCs in a whole population of suspected PCa men and according to each PI-RADS category.

      2. Patients and methods

      2.1 Design, setting, and participants

      A prospective head-to-head comparison of ROT- and BCN-RC in 946 men with serum PSA ≥3.0 ng/ml and/or abnormal DRE, recruited in two academic centres from the Barcelona metropolitan area (PSM and GTiP), between 2018 and 2021 was performed. Prebiopsy 3-Tesla mpMRI and 12-core transrectal ultrasound (TRUS) systematic biopsy were always performed, and two- to four-core TRUS visual target biopsies were added to PI-RADSv.2 ≥3 lesions. Men undergoing 5-ARIs who had previous PCa, atypical small acinar proliferation, or high-grade prostatic intraepithelial neoplasia with atypia were excluded. The project was approved by the ethical committee of VHH (PR/AG-317/2017).

      2.2 Intervention

      The likelihoods of csPCa from ROT-RC (www.prostatecancer-riskcalculator.com) [
      • Alberts A.R.
      • Roobol M.J.
      • Verbeek J.F.M.
      • et al.
      Prediction of high-grade prostate cancer following multiparametric magnetic resonance imaging: improving the Rotterdam European Randomized Study of Screening for Prostate Cancer risk calculators.
      ] and BCN-RC (https://mripcaprediction.shinyapps.io/MRIPCaPrediction/, second tab ‘BCN2RC: MRI-based model’) [
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ] were assessed. PI-RADSv.2 was introduced in both RCs as the closest value of serum PSA, prostate volume, and age when out of the accepted range in ROT-RC.

      2.3 Clinically significant PCa definition

      ROT-RC predicts high-grade PCa likelihood, defined as a Gleason score of ≥3 + 4, although the current RC shows this as csPCa [
      • Püllen L.
      • Radtke J.P.
      • Wiesenfarth M.
      • et al.
      External validation of novel magnetic resonance imaging-based models for prostate cancer prediction.
      ]. BCN-RC predicts csPCa risk defined as an ISUP grade group of ≥2 [
      • Epstein J.I.
      • Zelefsky M.J.
      • Sjoberg D.D.
      • et al.
      A contemporary prostate cancer grading system: a validated alternative to the Gleason score.
      ].

      2.4 Endpoint variables

      Saved prostate biopsies and missed csPCa were determined.

      2.5 Statistical analysis

      Quantitative variables were expressed as median and 25–75 percentiles (interquartile range [IQR]), and qualitative variables in rates. Mann-Witney U test and chi-square test were used to compare medians and proportions [
      • Whitney J.
      Testing for differences with the nonparametric Mann-Whitney U test.
      ,
      • Plackett R.L.
      Karl Pearson and the chi-squared test.
      ]. The discrimination ability of csPCa was analysed with receiver operating characteristic (ROC) curves [
      • Creelman C.D.
      • Donaldson W.
      ROC curves for discrimination of linear extent.
      ]; areas under the curve (AUCs) were compared with the test of DeLong et al [
      • DeLong E.R.
      • DeLong D.M.
      • Clarke-Pearson D.L.
      Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
      ]. Calibration curves analysed the correspondence between predictive probabilities and observed occurrence of csPCa. The net benefit of RCs over biopsying all men or none was analysed using a decision curve analysis (DCA) [
      • Vickers A.J.
      • Elkin E.B.
      Decision curve analysis: a novel method for evaluating prediction models.
      ]. Specificities corresponding to 85%, 90%, and 95% sensitivities were assessed as their 95% confidence intervals (CIs). Clinical utility curves (CUCs) exploring potential rates of avoided biopsies and missed csPCa according to continuous probability of csPCa were generated [
      • Borque Á.
      • Rubio-Briones J.
      • Esteban L.M.
      • et al.
      Implementing the use of nomograms by choosing threshold points in predictive models: 2012 updated Partin tables vs a European predictive nomogram for organ-confined disease in prostate cancer.
      ]. Tests with two-sided p < 0.05 were considered statistically significant. Statistical analyses were computed using R programming language v.4.0.3 (The R Foundation for Statistical Computing, Vienna, Austria) and SPSS v.25 (Statistical Package for Social Sciences; IBM, San Francisco, CA, USA).

      3. Results

      3.1 Population characteristics

      The characteristics of participants are summarised in Table 1. In 386 men (40.8%), csPCa was detected—17.9% in men with PI-RADS <3, 20.4% for PI-RADS 3, 51.9% for PI-RADS 4, and 84% for PI-RADS 5. A subset of 209 men (22.1%) had age (129), serum PSA (18), or prostate volume (68) out of the accepted range of ROT-RC.
      Table 1Population characteristics
      CharacteristicMeasurement
      Number of men946
      Age (yr), median (IQR)67 (61–72)
      Total PSA (ng/ml), median (IQR)7.4 (5.5–10.9)
      Abnormal DRE, n (%)307 (32.5)
      Prostate volume (ml), median (IQR)55 (40–79)
      Prior negative prostate biopsy, n (%)293 (31.0)
      Family history of PCa, n (%)34 (3.6%)
      PI-RADS, n (%)
       1–2235 (24.8)
       3201 (21.2)
       4391 (41.3)
       5119 (12.6)
      PCa detection, n (%)521 (55.1)
      csPCa detection, n (%)386 (40.8)
      csPCa detection by PI-RADS category, n (%)
       <342 (17.9)
       341 (20.4)
       4203 (51.9)
       5100 (84.0)
      csPCa = clinically significant PCa; DRE = digital rectal examination; IQR = interquartile range; n = number; PCa = prostate cancer; PI-RADS = Prostate Imaging Reporting and Data System; PSA = prostate-specific antigen.

      3.2 Behaviour of ROT- and BCN-RC in the whole population

      The median csPCa likelihood of ROT-RC was 3% (IQR: 2–10) in men without csPCa and 25% (9–46) in men with csPCa (p < 0.001), whereas those of BCN-RC were 10.5% (3.6–27.2) and 67.1% (39.3–85.6), respectively (p < 0.001; Fig. 1).
      Figure thumbnail gr1
      Fig. 1Violin plots of csPCa likelihoods of men without and with csPCa estimated with ROT MRI-RC and BCN MRI-RC. BCN = Barcelona; csPCa = clinically significant prostate cancer; MRI-RC = magnetic resonance imaging–based risk calculator; ROT = Rotterdam.
      The calibration curves of both RCs showed certain underestimation of csPCa (Fig. 2A and 2B). BCN-RC showed a slight underestimation with a calibration in the large of 0.25 showing a minimum difference between the mean observed and the mean predicted values, with a difference of 1.96 for ROT-RC. The slopes were 0.81 and 1.04 for ROT- and BCN-RC, respectively. BCN-RC showed values near the ideal value of 1.
      Figure thumbnail gr2
      Fig. 2Calibration curves of (A) ROT MRI-RC and (B) BCN MRI-RC. BCN = Barcelona; MRI-RC = magnetic resonance imaging–based risk calculator; ROT = Rotterdam.
      The discrimination ability of ROT- and BCN-RC for csPCa is presented by ROC curves in Figure 3A. The AUCs were 0.856 (95% CI: 0.831–0.881) and 0.844 (0.819–0.869) respectively (p = 0.116). BCN-RC showed a net benefit over biopsying all men from a 15% probability threshold; ROT MRI-RC showed the benefit from a 32% probability threshold (Fig. 3B). CUCs showed a larger clinical utility area for BCN-RC (Fig. 3C); additionally, the morphology and positioning of CUCs were different between the two RCs, shifted up and to the left of ROT-RC with respect to BCN-RC. The number of missed csPCa cases and avoided biopsies in each threshold from 1% to 100% are presented in Supplementary Table 1. The specificities corresponding to 85%, 90%, and 95% sensitivities of csPCa provided by both RCs are presented in Table 2. There were similar specificities at 85% and 90% sensitivities, with 42.1% (95% CI: 38.0–46.3) for ROT-RC and 31.8% (28.0–35.8) for BCN-RC (p < 0.001) at 95% sensitivity.
      Figure thumbnail gr3
      Fig. 3(A) Discrimination ability of csPCa of BCN MRI-RC and ROT MRI-RC presented with ROC curves, (B) net benefit of BCN MRI-RC and ROT MRI-RC over biopsying all men presented by DCAs, and (C) clinical utility of BCN MRI-RC and ROT MRI-RC showing the percentage of avoided biopsies and missed csPCa according to the threshold probability of csPCa by CUCs. BCN = Barcelona; csPCa = clinically significant prostate cancer; CUC = clinical utility curve; DCA = decision curve analysis; MRI-RC = magnetic resonance imaging–based risk calculator; ROT = Rotterdam.
      Table 2Specificities of BCN MRI-RC and ROT MRI-RC corresponding to 85%, 90%, and 95% sensitivities for csPCa
      Risk calculatorSpecificity (95% CI) for sensitivities of
      85%p value90%p value95%p value
      BCN MRI-RC (%)67.3 (63.2–71.1)0.05552.0 (47.7–56.2)0.39231.8 (28.0–35.8)<0.001
      ROT MRI-RC (%)61.7 (57.5–65.7)54.7 (50.5–58.9)42.1 (38.0–46.3)
      BCN = Barcelona; CI = confidence interval; MRI-RC = magnetic resonance imaging–based risk calculator; ROT = Rotterdam.

      3.3 Behaviour of ROT- and BCN-RC according to the PI-RADS categories

      Violin plots of csPCa likelihoods in men with and without csPCa, assessed with both RCs according to PI-RADSv.2 categories, show lower values and a narrow range of ROT-RC predictions (Fig. 4A) than those from BCN-RC (Fig. 4B), although significant differences existed between the medians of both subsets in all PI-RADS categories.
      Figure thumbnail gr4
      Fig. 4Violin plots of csPCa likelihoods, estimated from (A) ROT MRI-RC and (B) BCN MRI-RC, in men without and with csPCa according to the PI-RADS categories. BCN = Barcelona; csPCa = clinically significant prostate cancer; MRI-RC = magnetic resonance imaging–based risk calculator; PI-RADS = Prostate Imaging Reporting and Data System; ROT = Rotterdam.
      The AUC of ROT-RC in men with PI-RADS <3 was 0.776 (95% CI: 0.661–0.832), whereas the AUC of BCN-RC was 0.774 (0.697–0.850, p = 0.529; Fig. 5A). The AUCs were, respectively, 0.836 (0.756–0.916) and 0.838 (0.761–0.914) in men with PI-RADS 3 (p = 0.954; Fig. 5D), 0.829 (0.789–0.869) and 0.737 (0.677–0.787) in men with PI-RADS 4 (p < 0.001; Fig. 5G), and 0.866 (0.785–0.948) and 0.822 (0.723–0.920) in men with PI-RADS 5 (p = 0.323; Fig. 5J).
      Figure thumbnail gr5
      Fig. 5Discrimination ability presented by ROC curves, net benefit presented by DCAs, and clinical utility presented by CUCs, of BCN MRI-RC and ROT MRI-RC according to the PI-RADS categories: (AC) PI-RADS <3, (DF) PI-RADS 3, (G–I) PI-RADS 4, and (J–L) PI-RADS 5. BCN = Barcelona; csPCa = clinically significant prostate cancer; CUC = clinical utility curve; MRI-RC = magnetic resonance imaging–based risk calculator; PI-RADS = Prostate Imaging Reporting and Data System; ROT = Rotterdam.
      A small benefit over no-biopsy men with PI-RADS <3 was observed in both RCs (Fig. 5B). A clear net benefit of BCN-RC was observed over biopsying all men with PI-RADS 3 from a 10% csPCa probability, with the benefit of ROT-RC being lower (Fig. 5E). In men with PI-RADS 4, BCN-RC showed a net benefit over biopsying all men from a 17% csPCa probability; however, ROT-RC showed a small benefit between 50% and 75% csPCa probability (Fig. 5H). In men with PI-RADS 5, BCN-RC showed a benefit over biopsy-all men from 38% probability of csPCa; ROT-RC exhibited a minimal benefit from the 83% csPCa probability (Fig. 5K).
      CUCs showed a striking behaviour of both RCs. ROT-RC started in PI-RADS <3 with curves displaced up and to the left, evolving with the increase of the PI-RADS category towards the graph diagonal line; BCN-RC run through the entire area of the graph, evolving from the top left to the bottom right with an increase in the PI-RADS category. CUCs of BCN-RC showed clear clinical utility in PI-RADS 3 and 4. The number of missed csPCa cases and saved biopsies in each PI-RADS category of ROT- and BCN-RC are presented in Supplementary Tables 2–5. The specificities of both RCs from 85%, 90%, and 95% sensitivities in each PI-RADS category are presented in Table 3.
      Table 3Specificities of BCN MRI-RC and ROT MRI-RC corresponding to 85%, 90%, and 95% sensitivities for csPCa according to the PI-RADSv.2 category
      Risk calculatorSpecificity (95% CI) for sensitivities of
      85%p value90%p value95%p value
      PI-RADS <3
      BCN MRI-RC (%)49.7 (45.5–54.0)0.18844.0 (39.9–48.3)0.4569.8 (7.6–12.7)<0.001
      ROT MRI-RC (%)53.8 (49.6–58.0)41.7 (37.6–45.9)24.5 (21.1–28.3)
      PI-RADS 3
      BCN MRI-RC (%)78.1 (74.4–81.4)<0.00150.6 (46.4–54.8)0.10215.6 (12.8–19–0)<0.001
      ROT MRI-RC (%)60.0 (55.8–64.0)45.6 (41.4–49.8)25.5 (22.0–29.4)
      PI-RADS 4
      BCN MRI-RC (%)62.2 (58.1–66.2)<0.00149.5 (45.3–53.7)<0.00139.9 (32.9–41.0)<0.001
      ROT MRI-RC (%)29.7 (26.0–33.7)20.2 (17.0–23.8)4.6 (3.1–6.8
      PI-RADS 5
      BCN MRI-RC (%)63.2 (59.0–67.1)1.00052.6 (48.4–56.8)0.45636.8 (32.9–41.0)<0.001
      ROT MRI-RC (%)63.2 (59.0–67.1)60.5 (56.3–64.6)52.6 (48.4–56.8)
      BCN = Barcelona; CI =confidence interval; MRI-RC = magnetic resonance imaging–based risk calculator; PI-RADS = Prostate Imaging Reporting and Data System; ROT = Rotterdam.
      As the decision of prostate biopsy is made after mpMRI, it is appropriate to define the acceptable percentages of missed csPCa according to each PI-RADS category [
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ]. CUCs show how no RC appeared clinically helpful for PI-RADS <3 because, to identify 50% of 42 csPCa detected (5% of all csPCa detected), 75% of men needed biopsy (18.6% of all biopsies) with ROT-RC and 80% (20% of all biopsies) with BCN-RC. The rate of csPCa detection in men with PI-RADS 3 was 20.4%, and it was acceptable to miss up to 10% of these csPCa (5.2% of all csPCa) cases. ROT-RC saved 21% of biopsies, whereas BCN-RC saved 35%. In men with PI-RADS 4, BCN-RC saved 15% of biopsies (6% of all biopsies), missing 2% of csPCa (1% of all csPCa), whereas ROT-RC saved 10% of biopsies, with 6% of csPCa remaining undetected, which seems clinically unacceptable. Finally, in PI-RADS 5 where only 16% of biopsies were unnecessary, no RC assured to avoid any biopsy without missing csPCa.

      4. Discussion

      We were surprised at how far from the ideal was the calibration curve of ROT MRI-RC compared with that of BCN MRI-RC. The most probable cause was the low and narrow csPCa risk prediction range generated by ROT-RC mainly located at low predicted probabilities. We also noted that 22.1% of analysed suspected PCa men showed age, serum PSA, or prostate volume out of the accepted range of ROT MRI-RC. This drawback, not reported until now, can be a consequence of the strict inclusion criteria of the ERSPC trial in which RCs 3 and 4 were initially developed [

      Steyberg R, Roobol-Bouts MJ, Kranse M, Schroder FH. Data storage device and method for determining the dependency of the risk for prostate cancer, device and method for indicating a risk for a disease in an individual. U.S. Patent Office, 8,087,576.

      ,
      • Roobol M.J.
      • Steyerberg E.W.
      • Kranse R.
      • et al.
      A risk-based strategy improves prostate-specific antigen-driven detection of prostate cancer.
      ]. These limited ranges were not observed in the population of suspected PCa men in whom the ROT MRI-RC was adjusted [
      • Alberts A.R.
      • Roobol M.J.
      • Verbeek J.F.M.
      • et al.
      Prediction of high-grade prostate cancer following multiparametric magnetic resonance imaging: improving the Rotterdam European Randomized Study of Screening for Prostate Cancer risk calculators.
      ], which had similar characteristics as our population study [
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ].
      The discrimination ability of csPCa of ROT and BCN MRI-RCs was similar in the entire population; however, DCAs showed a net benefit of BCN MRI-RC over ROT MRI-RC. Since three-quarters of csPCa likelihoods predicted by ROT MRI-RC were below 24%, there was no benefit over biopsying all men from a threshold of >20%, and misclassification of csPCa > 60% remained within these thresholds that are not in the utility range [
      • Vickers A.J.
      • Elkin E.B.
      Decision curve analysis: a novel method for evaluating prediction models.
      ]. In contrast, BCN MRI-RC showed a net benefit over biopsying all men above the 15% threshold. This threshold avoided 40% of prostate biopsies, missing 10% of csPCa. To know the true clinical value of MRI-RCs is essential to assess their behaviour according to the PI-RADS categories, because the discrimination ability in the whole population does not represent that of each PI-RADS [
      • Moonesingh S.R.
      • Bashford T.
      • Wagstaff D.
      Implementing risk calculators: time for the Trojan horse?.
      ]. ROT MRI-RC showed no benefit in men with PI-RADS <3, 4, and 5, as did BCN MRI-RC in PI-RADS <3 and 5. BCN MRI-RC showed a net benefit in men with PI-RADS 4, saving between 6% and 23% of prostate biopsies and missing between 1% and 5% of csPCa detected for risk thresholds between 13% and 28%. In men with PI-RADS 3, ROT MRI-RC with thresholds between 1% and 3% missed between 0% and 12% of csPCa, saving 0.5–45% of biopsies. BCN MRI-RC at thresholds between 1% and 11% missed between 0% and 12% csPCa, saving between 5% and 47% of biopsies.
      The possible drawback of comparing both MRI-RCs in a population from the same metropolitan area where the BCN MRI-RC was developed, we note that csPCa detection rate in the cohort of suspected PCa men in whom ROT MRI-RC was adjusted was 35.8%, very close to that of 36.9% observed in the BCN MRI-RC development cohort [
      • Püllen L.
      • Radtke J.P.
      • Wiesenfarth M.
      • et al.
      External validation of novel magnetic resonance imaging-based models for prostate cancer prediction.
      ,
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ]. The characteristics of participants of this head-to-head comparison were different from those of both the development and the adjustment cohorts of BCN and ROT MRI-RCs in terms of age, serum PSA, DRE, and csPCa detection rate of 40.8%. The close origin of this population to that of BCN MRI-RC development did not influence the results. Rather, we believe that the differences between BCN and ROT MRI-RCs justify their different usefulness. BCN MRI-RC predicted the likelihood of csPCa defined as grade group ≥2, while ROT MRI-RC defined as Gleason ≥3 + 4; the age, serum PSA, and prostate volume ranges were limited in ROT MRI-RC and PI-RADSv.1 was used in its adjustment cohort, whereas PI-RADSv.2 was used in the BCN MRI-RC development cohort. The expression of csPCa likelihoods without and with decimals in ROT and BCN MRI-RCs, respectively, may represent any bias. A limitation of both MRI-RCs and the present study may be that a transperineal approach is currently recommended for prostate biopsy, whereas a transrectal approach was used in the present ROT MRI-RC adjustment and BCN MRI-RC development comparative cohorts [
      • Mottet N.
      • van den Bergh R.C.N.
      • Briers E.
      • et al.
      EAU-EANM-ESTRO-ESUR-SIOG guidelines on prostate cancer—2020 update. Part 1: screening, diagnosis, and local treatment with curative intent.
      ], which appears to improve the overall detection of csPCa [
      • Zattoni F.
      • Marra G.
      • Kasivisvanathan V.
      • et al.
      The detection of prostate cancer with magnetic resonance imaging-targeted prostate biopsies is superior with the transperineal vs the transrectal approach. A European Association of Urology-Young Academic Urologists Prostate Cancer Working Group multi-institutional study.
      ].
      There are specific limitations of predictive models. A developed predictive model reflects the probability of a condition based on the characteristics at that time. However, changes arising in the same population and those from the validation cohorts justify the need of recalibrations of the models and adjustment of risk thresholds to ensure accurate predictions [
      • Morote J.
      • Borque-Fernando Á.
      • Triquell M.
      • Esteban L.M.
      • Trilla E.
      The true utility of predictive models based on magnetic resonance imaging in selecting candidates for prostate biopsy.
      ,
      • Diniz M.A.
      Statistical methods for validation of predictive models.
      ]. BCN MRI-RC reports the novelty of selecting the risk threshold that can be useful in external validations and selecting appropriate thresholds for each PI-RADS category [
      • Morote J.
      • Borque-Fernando A.
      • Triquell M.
      • et al.
      The Barcelona predictive model of clinically significant prostate cancer.
      ]. Real-time updating is a great challenge for future RCs [
      • Strobl A.N.
      • Vickers A.J.
      • Van Calster B.
      • et al.
      Improving patient prostate cancer risk assessment: moving from static, globally-applied to dynamic, practice-specific risk calculators.
      ]. Continuous feedback of new cases, big data integration, appropriate machine learning algorithms, and federated networking can lead to future RCs validated in each site and ensuring accurate and long-lasting predictions in many places [
      • Nandi A.
      • Xhafa F.
      A federated learning method for real-time emotion state classification from multi modal streaming.
      ].

      5. Conclusions

      BCN and ROT MRI-RCs showed different behaviour in this head-to-head comparative analysis. ROT MRI-RC reported a lower and narrower range of csPCa likelihoods than BCN MRI-RC. BCN MRI-RC presented a net benefit over ROT MRI-RC and grater clinical utility in the entire population. According to the PI-RADS category, BCN MRI-RC was helpful in men with PI-RADS 3 and 4, whereas ROT MRI-RC was helpful only in men with PI-RADS 3. No MRI-RC was helpful in men with PI-RADS <3 and 5.
      Author contributions: Juan Morote had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
      Study concept and design: Morote, Borque-Fernando, Esteban.
      Acquisition of data: Triquell, Campistol, Servian, Abascal, Mendez, Planas.
      Analysis and interpretation of data: Morote, Borque-Fernando, Esteban.
      Drafting of the manuscript: Morote.
      Critical revision of the manuscript for important intellectual content: Borque-Fernando, Esteban, Planas, Mendez, Trilla.
      Statistical analysis: Esteban, Borque-Fernando, Morote.
      Obtaining funding: Morote.
      Administrative, technical, or material support: Morote.
      Supervision: Morote, Trilla.
      Other: None.
      Financial disclosures: Juan Morote certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.
      Funding/Support and role of the sponsor: This study was supported by the Instituto Carlos III (SP) and the European Union (project ref: PI20/01666).

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Mottet N.
        • van den Bergh R.C.N.
        • Briers E.
        • et al.
        EAU-EANM-ESTRO-ESUR-SIOG guidelines on prostate cancer—2020 update. Part 1: screening, diagnosis, and local treatment with curative intent.
        Eur Urol. 2021; 79: 243-262
        • Van Poppel H.
        • Roobol M.J.
        • Chapple C.R.
        • et al.
        Prostate-specific antigen testing as part of a risk-adapted early detection strategy for prostate cancer: European Association of Urology position and recommendations for 2021.
        Eur Urol. 2021; 80: 703-711
        • Van Poppel H.
        • Hogenhout R.
        • Albers P.
        • van den Bergh R.C.N.
        • Barentsz J.O.
        • Roobol M.J.
        European model for an organised risk-stratified early detection programme for prostate cancer.
        Eur Urol Oncol. 2021; 10: 731-739
        • Schoots I.G.
        • Padhani A.R.
        • Rouvière O.
        • Barentsz J.O.
        • Richenberg J.
        Analysis of magnetic resonance imaging-directed biopsy strategies for changing the paradigm of prostate cancer diagnosis.
        Eur Urol Oncol. 2020; 3: 32-41
        • Sathianathen N.J.
        • Omer A.
        • Harriss E.
        • et al.
        Negative predictive value of multiparametric magnetic resonance imaging in the detection of clinically significant prostate cancer in the prostate imaging reporting and data system era: a systematic review and meta-analysis.
        Eur Urol. 2020; 78: 402-414
        • Wagaskar V.G.
        • Levy M.
        • Ratnani P.
        • et al.
        Clinical utility of negative multiparametric magnetic resonance imaging in the diagnosis of prostate cancer and clinically significant prostate cancer.
        Eur Urol Open Sci. 2021; 28: 9-16
        • Mazzone E.
        • Stabile A.
        • Pellegrino F.
        • et al.
        Positive predictive value of Prostate Imaging Reporting and Data System version 2 for the detection of clinically significant prostate cancer: a systematic review and meta-analysis.
        Eur Urol Oncol. 2020; 4: 697-713
        • Osses D.F.
        • Roobol M.J.
        • Schoots I.G.
        Prediction medicine: biomarkers, risk calculators and magnetic resonance imaging as risk stratification tools in prostate cancer diagnosis.
        Int J Mol Sci. 2020; 20: 1637
        • Triquell M.
        • Campistol M.
        • Celma A.
        • et al.
        Magnetic resonance imaging-based predictive models for clinically significant prostate cancer: a systematic review.
        Cancers (Basel). 2022; 14: 4747
      1. Steyberg R, Roobol-Bouts MJ, Kranse M, Schroder FH. Data storage device and method for determining the dependency of the risk for prostate cancer, device and method for indicating a risk for a disease in an individual. U.S. Patent Office, 8,087,576.

        • Roobol M.J.
        • Steyerberg E.W.
        • Kranse R.
        • et al.
        A risk-based strategy improves prostate-specific antigen-driven detection of prostate cancer.
        Eur Urol. 2010; 57: 79-85
        • Alberts A.R.
        • Roobol M.J.
        • Verbeek J.F.M.
        • et al.
        Prediction of high-grade prostate cancer following multiparametric magnetic resonance imaging: improving the Rotterdam European Randomized Study of Screening for Prostate Cancer risk calculators.
        Eur Urol. 2019; 75: 310-318
        • Püllen L.
        • Radtke J.P.
        • Wiesenfarth M.
        • et al.
        External validation of novel magnetic resonance imaging-based models for prostate cancer prediction.
        BJU Int. 2020; 125: 407-416
        • Chen R.
        • Verbeek J.F.M.
        • Yang Y.
        • Song Z.
        • Sun Y.
        • Roobol M.J.
        Comparing the prediction of prostate biopsy outcome using the Chinese Prostate Cancer Consortium (CPCC) risk calculator and the Asian adapted Rotterdam European Randomized Study of Screening for Prostate Cancer (ERSPC) risk calculator in Chinese and European men.
        World J Urol. 2021; 39: 73-80
        • Petersmann A.L.
        • Remmers S.
        • Klein T.
        • et al.
        External validation of two MRI-based risk calculators in prostate cancer diagnosis.
        World J Urol. 2021; 39: 4109-4116
        • De Nunzio C.
        • Lombardo R.
        • Baldassarri V.
        • et al.
        Rotterdam mobile phone app including MRI data for the prediction of prostate cancer: a multicenter external validation.
        Eur J Surg Oncol. 2021; 47: 2640-2645
        • Morote J.
        • Borque-Fernando A.
        • Triquell M.
        • et al.
        The Barcelona predictive model of clinically significant prostate cancer.
        Cancers (Basel). 2022; 14: 1589
        • Epstein J.I.
        • Zelefsky M.J.
        • Sjoberg D.D.
        • et al.
        A contemporary prostate cancer grading system: a validated alternative to the Gleason score.
        Eur Urol. 2016; 69: 428-435
        • Whitney J.
        Testing for differences with the nonparametric Mann-Whitney U test.
        J. Wound Ostomy Continence Nurs. 1997; 24: 12
        • Plackett R.L.
        Karl Pearson and the chi-squared test.
        Int Stat Rev. 1983; 51: 59-72
        • Creelman C.D.
        • Donaldson W.
        ROC curves for discrimination of linear extent.
        J Exp Psychol. 1968; 77: 514-516
        • DeLong E.R.
        • DeLong D.M.
        • Clarke-Pearson D.L.
        Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
        Biometrics. 1988; 44: 837-845
        • Vickers A.J.
        • Elkin E.B.
        Decision curve analysis: a novel method for evaluating prediction models.
        Med Decis Making. 2006; 26: 565-574
        • Borque Á.
        • Rubio-Briones J.
        • Esteban L.M.
        • et al.
        Implementing the use of nomograms by choosing threshold points in predictive models: 2012 updated Partin tables vs a European predictive nomogram for organ-confined disease in prostate cancer.
        BJU Int. 2014; 113: 878-886
        • Moonesingh S.R.
        • Bashford T.
        • Wagstaff D.
        Implementing risk calculators: time for the Trojan horse?.
        Anesth Analg. 2018; 121: 1192-1196
        • Zattoni F.
        • Marra G.
        • Kasivisvanathan V.
        • et al.
        The detection of prostate cancer with magnetic resonance imaging-targeted prostate biopsies is superior with the transperineal vs the transrectal approach. A European Association of Urology-Young Academic Urologists Prostate Cancer Working Group multi-institutional study.
        J Urol. 2022; 208: 830-837
        • Morote J.
        • Borque-Fernando Á.
        • Triquell M.
        • Esteban L.M.
        • Trilla E.
        The true utility of predictive models based on magnetic resonance imaging in selecting candidates for prostate biopsy.
        Eur Urol Open Sci. 2022; 42: 40-41
        • Diniz M.A.
        Statistical methods for validation of predictive models.
        J Nucl Cardiol. 2022; 29: 3248-3255
        • Strobl A.N.
        • Vickers A.J.
        • Van Calster B.
        • et al.
        Improving patient prostate cancer risk assessment: moving from static, globally-applied to dynamic, practice-specific risk calculators.
        J Biomed Inform. 2015; 56: 87-93
        • Nandi A.
        • Xhafa F.
        A federated learning method for real-time emotion state classification from multi modal streaming.
        Methods. 2022; 204: 340-344