Advertisement

Evidence-based Medicine: Perceptions, Attitudes, and Skills Among European Urology Residents

Open AccessPublished:September 29, 2022DOI:https://doi.org/10.1016/j.euros.2022.08.023

      Abstract

      Background

      Evidence-based medicine (EBM) was introduced as a concept in the early 1990s as an integration of the best available evidence with clinical expertise and patient values.

      Objective

      To evaluate the current status of EBM training and EBM perception, attitudes, and self-perceived skills among European urology residents.

      Design, setting, and participants

      Our online open survey comprised 28 multiple-choice items, including ten questions with responses on a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. The survey was distributed via the mailing lists and social media accounts of the European Society of Residents in Urology, German Society of Residents in Urology, French Society of Urologists in Training, Spanish Urology Residents Working Group, Italian Society of Residents in Urology, and the Urology Social Media Working Group in two rounds (May–July 2019 and July 2020). We excluded responses from non-European countries.

      Outcome measurements and statistical analysis

      The online open survey comprised 28 multiple-choice items. These included ten questions with answers on a five-point Likert scale with response items ranging from strongly disagree (score of 1) to strongly agree (score of 5).

      Results and limitations

      We received 210 responses, of which 181 from 23 European countries were eligible. Approximately three-quarters (73.7%) of the respondents were male, with a mean age of 31 yr. Only 28.2% reported EBM training as part of their urology curriculum and 19.3% felt that the training they received was sufficient to guide their daily practice. An overwhelming majority (91.5%) stated that they would be interested in more formalized EBM training or additional training. There was a strong level of agreement (median score 5, interquartile range 4–5) that EBM is important for daily medical and surgical practice and that it improves patient care. Overall, the mean self-perceived understanding of basic EBM concepts was good. Limitations include concerns about generalizability given its internet-based format, the inability to calculate a response rate, poor representation from some European regions, and limited sample size.

      Conclusions

      Our survey suggests that European urology residents receive a limited amount of EBM training despite considerable appreciation, interest, and self-perceived deficits for more advanced concepts. Formal integration of EBM teaching in all European residency programs should be considered.

      Patient summary

      We performed an online survey of urology residents in Europe. We found that residents have positive perceptions of and attitudes to evidence-based medicine but most programs lack formal training in this area.

      Keywords

      1. Introduction

      Evidence-based medicine (EBM) is defined as “integration of the best available research evidence with clinical expertise and patients’ unique values and preferences” [
      • Sackett D.L.
      • Rosenberg W.M.
      • Gray J.A.
      • Haynes R.B.
      • Richardson W.S.
      Evidence based medicine: what it is and what it isn’t.
      ] and was introduced as a concept in the early 1990s as a new approach to teaching and practicing medicine [
      • Evidence-Based Medicine Working Group
      Evidence-based medicine. A new approach to teaching the practice of medicine.
      ]. Since the term was introduced by Gordon Guyatt, EBM has found widespread uptake as a guiding principle in medicine that has been formally integrated into graduate and postgraduate curricula in many countries.
      An initial assessment of the status of EBM teaching was published by Wiking Månsson [
      • Månsson W.
      Evidence-based urology—a utopia?.
      ] in 2004 and suggested that urology as a specialty had come late to the field of EBM, emphasizing the need to provide formal teaching. Since then, several urology-specific resources have been developed, including a users’ guide to the urological literature series and a series on evidence-based urology in practice. EBM is also part of the American Urological Association (AUA) core curriculum that identifies foundational knowledge for urology trainees in accredited North American programs, and is also integrated into the educational milestones used for resident assessments twice a year. To date, no such resources and requirements have been developed for European Board of Urology accredited training programs and little is known about if and how residents in Europe are being instructed in EBM core competencies and related skills such as critical appraisal and biostatistics. We conducted this survey to determine if and how EBM knowledge and skills are being taught in European training programs and what the attitudes and perceptions are among urology residents.

      2. Materials and methods

      We designed an online open survey using the Survey Monkey platform (http://www.surveymonkey.com/) comprising 28 items, including ten questions with responses rated on a five-point Likert scale (Supplementary material) to evaluate perception, attitudes, and self-perceived skills. The three-part survey was an adapted version of a prior work [
      • Dahm P.
      • Poolman R.W.
      • Bhandari M.
      • et al.
      Perceptions and competence in evidence-based medicine: a survey of the American Urological Association Membership.
      ] and a first draft was distributed to the European Society of Residents in Urology (ESRU) board members and was revised based on their input. Demographic variables we collected included age, gender, country, year of residency training, and prior formal training in research (MSc, PhD, or similar). We did not collect identifiable data.
      The first part of the survey (10 questions) assessed participants’ attitudes and perceptions about the role of EBM in the daily clinical practice of urology, as well as self-perceived skills. Respondents were asked to indicate their level of agreement on a five-point Likert scale ranging from 1 (strongly disagree) through 3 (neither agree nor disagree) to 5 (strongly agree). The second part of the survey (3 questions) inquired about participants’ EBM background and interest in further training. In the third and final part (15 questions), respondents were asked to self-assess their own knowledge and understanding of EBM and statistics-related concepts on a four-point scale: 1, not familiar; 2, recognize it but do not understand it; 3, understand it but could not explain it to others; or 4, understand it and can explain it to others. All the content of the survey was displayed through four screens/pages.
      The survey was distributed in two rounds, first from May to July 2019 and a reminder in July 2020 via mailing lists of the ESRU (n = 412), German Society of Residents in Urology (n = 332), French Society of Urologists in Training (n = 275), Spanish Urology Residents Working Group (n = 238), and Urology Social Media working group (n = 146). In addition, the survey was disseminated using the Twitter accounts of the same groups. The platform had an internet provider address block so that each user could only respond once; participation was voluntary and there were no incentives provided for completion. Only complete surveys from European urology residents were included in the final analysis. We followed the Checklist for Reporting Results of Internet e-Surveys (CHERRIES) [
      • Eysenbach G.
      Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
      ] guidelines recommended by the EQUATOR network. Since no identifiable personal information was collected, it was deemed exempt from formal institutional review board approval, in line with US Department of Health & Human Services regulations for the protection of human subjects in research [

      US Department of Health & Human Services. Office for Human Research Protections. Exemptions (2018 requirements). https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/common-rule-subpart-a-46104/index.html.

      ].

      2.1 Statistical analysis

      All the answers were collected in a database and analyzed using descriptive statistics with GraphPad Prism 8. Categorical data were summarized using frequency tables, and data for continuous variables using the mean (and standard deviation) or median (and interquartile range [IQR]) as indicated by their distribution. Likert-scale responses were summarized using box plots with representations of the mean, median, and IQR.

      3. Results

      We received a total of 210 responses, of which we excluded 29 either for being incomplete or originating from residents of non-European countries, leaving 181 responses from 23 European countries as the study sample. Most respondents were male (72.9%), with a mean age of 31 yr; the distribution of junior (years 1–3) and senior (years ≥4 of training) residents was similar. Table 1 summarizes the demographic data for the survey respondents.
      Table 1Demographic characteristics of the survey respondents
      ParameterResult
      Gender, n (%)
       Male132 (72.9)
       Female49 (27.1)
      Median age, yr (interquartile range)30 (27–32)
      Year of training, n (%)
       Year 120 (11)
       Year 227 (14.9)
       Year 341 (22.7)
       Year 437 (20.5)
       Year 544 (24.3)
       Year ≥612 (6.6)
      European regions, n (%)
      European regions according to EuroVoc [18].
       Southern Europe108 (59.7)
       Central and Eastern Europe23 (12.7)
       Northern Europe14 (7.7)
       Western Europe36 (19.9)
      Prior formal training in evidence-based medicine, epidemiology or statistics, n (%)
       Yes99 (54.7)
       No82 (45.3)
      a European regions according to EuroVoc

      EuroVoc - EUR-Lex [Internet]. [cited 2021 Nov 14]. Available from: https://eur-lex.europa.eu/browse/eurovoc.html?params=72%2C7206.

      .
      In the first part of the survey, nearly half of the respondents (54.8%) reported prior formal training in EBM, epidemiology, or statistics, while only 28.2% had it as part of their urology residency curriculum.
      When participants were asked if they believe that the EBM/statistics education received as part of their residency training was enough for their daily practice, only 19.3% answered affirmatively, and 91.5% said they were interested in receiving formal EBM training or further training.
      According to the second part of the survey evaluating the perception and attitudes of residents to EBM, measured using a Likert scale from 1 to 5, most residents considered that EBM is important for daily medical and surgical practice. When asked if most of their medical and surgical therapy decisions are supported by EBM, the median score was 4. Similarly, expert opinion was considered to play an important role in daily diagnostic and therapeutic decisions (median score 4). Participants also considered that EBM improves patient care (median score 5). Nevertheless, when asked if most urologists in their department practice EBM, the median score was 3.
      Most participants consider the European Association of Urology guidelines as a trusted source of information for evidence-based decisions (median score 5, IQR 4–5). However, residents do not feel as confident in performing critical appraisal (median score 4, IQR 3–4) or interpreting methodology and statistics in journal articles (median score 3, IQR 2–4). These findings are summarized in Figure 1.
      Figure thumbnail gr1
      Fig. 1Level of agreement with statements about perception and attitudes towards EBM in urology on a scale from 1 (strongly disagree) to 5 (strongly agree). Thick vertical lines indicate the median score and + symbols the mean score; box outlines indicate the first and third quartiles; horizontal lines indicate the range; and whiskers show the minimum and maximum values. EBM = evidence-based medicine; EAU = European Association of Urology.
      The final section of the survey evaluated the self-rated understanding of 15 EBM-related concepts, for which the results are summarized in Figure 2.
      Figure thumbnail gr2
      Fig. 2Self-assessed understanding of EBM-related concepts. Each color denotes the percentage of respondents for each category. EBM = evidence-based medicine.
      We found a positive correlation between respondents with prior EBM training and better self-rated understanding of EBM-related concepts. Some 72% of the respondents who understood all the concepts provided had prior EBM training, while 38.7% of respondents without prior formal EBM training reported understanding of less advanced concepts (p value, likelihood ratio, relative risk, confidence interval, double blinding, selection bias).

      4. Discussion

      4.1 Principal findings

      Findings from this first survey of urology trainees in European urology programs indicate strong support for the foundational role of EBM in urology to guide clinical decision-making for both medical and surgical care. At the same time, they also suggest that urology trainees have at least some concerns about their own methodological and critical appraisal skills when applying the principles of EBM. On the basis of the participants’ self-assessment, basic concepts appear to be well understood, but more advanced terms such as Kaplan-Meier curves and forest plots are not, despite their ubiquitous presence in the urological literature. This study further bears witness to the fact that the degree of integrated EBM focus in European urology residency programs may be limited, while there appears to be a strong desire for more opportunities to acquire further knowledge and skills in EBM, critical appraisal, and related statistical concepts.

      4.2 Strength and weakness of the study

      One major strength of this study is the sampling of urology trainees across a broad spectrum of European urology training, thereby allowing identification of common themes that may lend themselves to educational initiatives, for example by the European School of Urology (ESU). Second, the survey included a broad range of EBM-related questions related to both EBM knowledge and skills, but also participants’ perceptions and attitudes towards EBM. It used a format that was like that of a somewhat dated survey of AUA members that sampled mainly US-based urologists (not limited to trainees), thereby allowing some indirect comparisons. Shortcomings of this survey include concerns regarding generalizability. First, given its social media–based format, we were unable to determine the number and characteristics of trainees who became aware of this survey to calculate a response rate and assess the representativeness. Second, there are data to suggest that health care providers overestimate their own understanding of EBM concepts and that misunderstandings are common [
      • Young J.M.
      General practitioners’ self ratings of skills in evidence based medicine: validation study.
      ]. This implies that actual EBM knowledge may be even lower than estimated. Lastly, the overall number of respondents was relatively small, thereby negating the possibility of meaningful subgroup analysis (eg, by country) or statistical modeling. Nevertheless, we believe that our findings are informative and meaningful in guiding future educational efforts related to EBM.

      4.3 Strength and weakness in relation to other studies

      The first similar study on EBM in urology was conducted by the AUA in the form of a large e-mail–based survey of its membership in 2005 [
      • Sur R.L.
      • Scales C.D.
      • Preminger G.M.
      • Dahm P.
      Evidence-based medicine: a survey of American Urological Association members.
      ]. It similarly found that there was widespread agreement (median score of 9 on a scale of 1–10, with higher values indicating greater agreement) with the notion that EBM improves patient care and that every urologist should be familiar with critical appraisal techniques. Meanwhile, it also raised concerns about over-reliance on expert opinion (median score 7) and industry-sponsored symposia (median score 6) as the most important sources of evidence. However, in contrast to our study, the vast majority of the 714 respondents (95%) were in independent practice and largely based in North America (71%), thereby limiting comparability. A follow-up mail-based study surveyed a random sample of 2000 AUA members in 2006, of whom 889 (44.5%) responded [
      • Dahm P.
      • Poolman R.W.
      • Bhandari M.
      • et al.
      Perceptions and competence in evidence-based medicine: a survey of the American Urological Association Membership.
      ]. This study also found that urologists thought that practicing EBM improved patient care (median score 8) and that every urologist should be familiar with critical appraisal techniques (median score 9). The first assessment of EBM teaching in residency targeted the program directors of 117 American Council for Graduate Medical Education programs in North America and achieved a very high response rate of 92.3% [
      • Dahm P.
      • Preminger G.M.
      • Scales C.D.
      • Fesperman S.F.
      • Yeung L.L.
      • Cohen M.S.
      Evidence-based medicine training in residency: a survey of urology programme directors.
      ]. Nearly all program directors endorsed the importance of EBM teaching, and 77.8% indicated that their curricula included formal educational activities related to EBM, with a median of 10 h devoted to EBM per year. However, the study did not address if and to what extent formal EBM curricula existed in these programs and no input from the actual trainees was sought. The first survey to do so was a smaller study of 29 senior residents of Canadian urology residency programs [
      • Roth K.
      • Siemens D.R.
      The status of evidence-based medicine education in urology residency.
      ]. Very similar to our study, only 28% of residents indicated that they received any formal EBM teaching within their program. With regard to understanding technical terms, the responses indicating “understand and could explain to others” from this survey versus ours were: 59% versus 74% for meta-analysis; 59% versus 56% for number needed to treat; 48% versus 43% for relative risk; and 41% versus 48% for confidence interval), indicating largely similar results. The authors concluded from their findings that there was “a need for redoubling efforts to ensure appropriate exposure and instruction” of EBM in Canadian urology training programs [
      • Roth K.
      • Siemens D.R.
      The status of evidence-based medicine education in urology residency.
      ]. Lastly, a more recent study from India surveyed urology trainees, of whom only 11% reported receiving formal EBM training during their residency [
      • Mukherjee P.
      • Chaudhary K.
      • Yadav B.K.
      • Kumar S.
      • Kekre N.
      • Devasia A.
      Attitude and perceived barriers towards the practice of evidence-based urology amongst urological trainees in India.
      ].

      4.4 Meaning of the study: possible explanations and implications

      Our study findings point to the need for more and improved formal EBM training in European urology residency programs. Ideally, curriculum development and the creation of core resources would be spearheaded by the ESU, with stakeholder input from a broad spectrum of member countries with diverse training backgrounds and structures. The development of such an EBM curriculum for urology trainees could build on the outcomes from a recent consensus document for core competencies in evidence-based practice for health professionals [
      • Albarqouni L.
      • Hoffmann T.
      • Straus S.
      • et al.
      Core Competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey.
      ]. It could also build on existing urology-specific training resources in print form, such as the users’ guide to the urological literature [
      • Dahm P.
      • Preminger G.M.
      A users’ guide to the urological literature: introducing a series of evidence based medicine review articles.
      ], the recent mini-series on evidence-based urology in European Urology Focus [
      • Tikkinen K.A.O.
      • Guyatt G.H.
      Evidence-based urology: introduction to our series of articles.
      ], and a series of eight urology-specific video tutorials presented by an international forum of EBM experts developed jointly by Société Internationale de Urologie and Cochrane Urology [

      Société Internationale d’Urologie. SIU Academy® – official eLearning portal of SIU. https://academy.siu-urology.org/siu/#!*listing=0*browseby=8*sortby=2*search=evaluate+urological%20literature%20series.

      ]. Although a European initiative would probably seek to identify its own training priorities and corresponding resources, it might also be able to draw from the EBM chapter included in the AUA core curriculum [

      American Urological Association. Evidence-based medicine – research. urology core curriculum. https://university.auanet.org/core/research/evidence-based-medicine/index.cfm.

      ].

      4.5 Unanswered questions and future research

      An important question that this study was unable to address is the perspective of directors of European urology residency programs. It appears that at least some programs already have robust and perhaps innovative educational curricula in place to teach the principles and application of EBM. It is also unclear how the educational content should best be delivered. Since existing surveys from the USA and Canada have suggested the lack of appropriately trained and motivated faculty as an important barrier to EBM teaching [
      • Roth K.
      • Siemens D.R.
      The status of evidence-based medicine education in urology residency.
      ,
      • Scales C.D.
      • Voils C.I.
      • Fesperman S.F.
      • et al.
      Barriers to the practice of evidence-based urology.
      ], centrally developed and virtually delivered on-demand resources might be welcomed. It would also be important to identify other barriers that residents and program directors perceive to stand in the way of more in-depth EBM training as part of urology residency.

      5. Conclusions

      According to this first study exploring the current landscape for EBM training in European urology residency programs, perceptions and attitudes are very positive, but formal training opportunities appear to be underdeveloped. The study findings point to a need for more and improved formal EBM training in European urology residency programs that could be centrally developed and administered.
      Author contributions: Daniel A. González Padilla had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
      Study concept and design: González Padilla, Dahm.
      Acquisition of data: González Padilla, Subiela, Carrion, Esperto, Gómez Rivas, Khadhouri, Mantica, Mattigk, Rodríguez Socarrás, Segui Moya, Teoh, Tortolero-Blanco, Uçar.
      Analysis and interpretation of data: González Padilla, Subiela, Dahm.
      Drafting of the manuscript: González Padilla, Subiela, Dahm.
      Critical revision of the manuscript for important intellectual content: González Padilla, Dahm.
      Statistical analysis: González Padilla, Dahm.
      Obtaining funding: None.
      Administrative, technical, or material support: González Padilla, Gómez Rivas, Uçar, Dahm.
      Supervision: Dahm.
      Other: None.
      Financial disclosures: Daniel A. González Padilla certifies that all conflicts of interest, including specific financial interests and relationships and affiliations relevant to the subject matter or materials discussed in the manuscript (eg, employment/affiliation, grants or funding, consultancies, honoraria, stock ownership or options, expert testimony, royalties, or patents filed, received, or pending), are the following: None.
      Funding/Support and role of the sponsor: None.
      Ethics considerations: The principles of the Helsinki Declaration were respected in the present work.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Sackett D.L.
        • Rosenberg W.M.
        • Gray J.A.
        • Haynes R.B.
        • Richardson W.S.
        Evidence based medicine: what it is and what it isn’t.
        BMJ. 1996; 312: 71-72
        • Evidence-Based Medicine Working Group
        Evidence-based medicine. A new approach to teaching the practice of medicine.
        JAMA. 1992; 268: 2420-2425
        • Månsson W.
        Evidence-based urology—a utopia?.
        Eur Urol. 2004; 46: 143-146
        • Dahm P.
        • Poolman R.W.
        • Bhandari M.
        • et al.
        Perceptions and competence in evidence-based medicine: a survey of the American Urological Association Membership.
        J Urol. 2009; 181: 767-777
        • Eysenbach G.
        Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES).
        J Med Internet Res. 2004; 6: e34
      1. US Department of Health & Human Services. Office for Human Research Protections. Exemptions (2018 requirements). https://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/common-rule-subpart-a-46104/index.html.

        • Young J.M.
        General practitioners’ self ratings of skills in evidence based medicine: validation study.
        BMJ. 2002; 324: 950-951
        • Sur R.L.
        • Scales C.D.
        • Preminger G.M.
        • Dahm P.
        Evidence-based medicine: a survey of American Urological Association members.
        J Urol. 2006; 176: 1127-1134
        • Dahm P.
        • Preminger G.M.
        • Scales C.D.
        • Fesperman S.F.
        • Yeung L.L.
        • Cohen M.S.
        Evidence-based medicine training in residency: a survey of urology programme directors.
        BJU Int. 2009; 103: 290-293
        • Roth K.
        • Siemens D.R.
        The status of evidence-based medicine education in urology residency.
        Can Urol Assoc J. 2013; 4: 114
        • Mukherjee P.
        • Chaudhary K.
        • Yadav B.K.
        • Kumar S.
        • Kekre N.
        • Devasia A.
        Attitude and perceived barriers towards the practice of evidence-based urology amongst urological trainees in India.
        Indian J Urol. 2019; 35: 116-119
        • Albarqouni L.
        • Hoffmann T.
        • Straus S.
        • et al.
        Core Competencies in evidence-based practice for health professionals: consensus statement based on a systematic review and Delphi survey.
        JAMA Netw Open. 2018; 1: e180281
        • Dahm P.
        • Preminger G.M.
        A users’ guide to the urological literature: introducing a series of evidence based medicine review articles.
        J Urol. 2007; 178: 1149
        • Tikkinen K.A.O.
        • Guyatt G.H.
        Evidence-based urology: introduction to our series of articles.
        Eur Urol Focus. 2021; 7: 1215-1216
      2. Société Internationale d’Urologie. SIU Academy® – official eLearning portal of SIU. https://academy.siu-urology.org/siu/#!*listing=0*browseby=8*sortby=2*search=evaluate+urological%20literature%20series.

      3. American Urological Association. Evidence-based medicine – research. urology core curriculum. https://university.auanet.org/core/research/evidence-based-medicine/index.cfm.

        • Scales C.D.
        • Voils C.I.
        • Fesperman S.F.
        • et al.
        Barriers to the practice of evidence-based urology.
        J Urol. 2008; 179: 2345-2350
      4. EuroVoc - EUR-Lex [Internet]. [cited 2021 Nov 14]. Available from: https://eur-lex.europa.eu/browse/eurovoc.html?params=72%2C7206.