Barriers and facilitators to answering clinical questions in the Americas: a cross-sectional study of surgical trauma care providers

Introduction

Nearly one-third of all deaths worldwide occur due to conditions potentially treated with surgical care.1 Many surgical systems across resource settings lack the necessary tools, including those required to advance human resources through continuous education, to achieve the best outcomes.2 The Lancet Commission on Global Surgery prioritizes high-quality, safe care2 as a crucial part of surgical systems, yet one USA-based analysis measuring quality of care in various healthcare settings found that only 50% to 60% of surgical patients receive quality, evidence-based care.3

Clinical practice that is based on evidence may be described to follow the sequence “ask, acquire, appraise, apply, assess.”4 Barriers arise from the very start of this process. The proven method to frame vague questions into researchable phrases using the mnemonic “population, intervention, comparison, outcome” (PICO) is not widespread in provider groups globally.5 Providers across resource settings also feel stuck at the “acquire” stage in evidence-based practice.6 To improve practical acquisition of evidence-based answers to clinical questions, the Lancet Commission on Global Surgery promoted standardized clinical protocols using the most updated evidence.2 However, cited reasons why providers do not use guidelines include lack of adequate dissemination,7 awareness, familiarity, or agreement on guidelines.8 Some practitioners feel that guidelines slow their practice or do not apply to low-resource settings.8 Institutions pay millions of dollars per year for access to research behind paywalls, and individuals pay on average $30 per paper.9 Even for publicly funded research, paywalls exist for the first year after publication, slowing the already sluggish pace of scientific progress.10 To push for faster access to scientific results, the US government drafted policies requiring immediate public access to papers funded by taxpayers’ dollars, but academic publishers effectively lobbied against the proposed policies.11 In 2016, only 19% of published articles were immediately available on publication.12 A new plan in 11 European countries, known as “Plan S,” aims to increase the number of articles immediately available on publication by requiring that all research with grants from European research councils or funding bodies publish directly to a public-access forum.12 13

Even as the scientific community pushes for public access, such as that provided through “Plan S,” many providers receive no training in the “appraise” portion of the evidence-based framework. The “appraise” stage of evidence-based practice requires financial resources, expertise, and time. In a 2010 study, only 33% of surgeons used primary literature to inform their practice; lack of education in appraising primary literature was described as a primary barrier.14 Most initiatives for education on evidence-based practice require hours of training,15 16 further preventing already time-strapped surgeons from acquiring the skills required to appraise literature.17 As one attempt to overcome these barriers, evidence-based search engines may provide predigested answers, providing an effective “short-cut” when primary literature is inaccessible or too dense to quickly appraise. The American College of Surgeons promotes the global use of these search engines,18 yet many surgeons still lack access to all papers referenced by the search engines and instead use abstracts to guide their practice.19 Aside from one study reporting that cost and language were barriers to practice of evidence-based medicine (EBM),6 very few data exist related to sociodemographic factors related to EBM or using these search engines.

Without adequate support for “acquiring” and “appraising,” the flow of evidence-based practice ends before “application” and “assessment,” never reaching the patient. This study aimed to better understand influencing factors on surgical trauma providers in the Americas as they acquire answers to clinical questions and to assess their level of confidence in appraising that information. We hypothesized that increased English proficiency and country income level would improve providers’ acquisition and application of clinical knowledge.

Patients and methods

For this cross-sectional study we partnered with the Pan American Trauma Society (PTS), an international professional society of trauma providers across the full spectrum of education and clinical care, who collaborate to improve surgical trauma systems throughout the Americas.20 We used their listserv and Facebook account to electronically invite all 619 current members to anonymously participate in this survey (online supplemental appendix A). We partnered with a surgeon (FV) with a history of leadership within the PTS to design the approach and promote participation. The survey was distributed to participants to choose between English, Spanish, and Portuguese. The 23 multiple-choice question survey asked about inperson, paper, and electronic resources most often used when answering clinical questions (online supplemental file 1). It queried participant perception of language barriers including English reading comprehension and quality of translation resources (online supplemental file 2). Finally, the survey included five knowledge questions designed to assess participants’ ability to acquire information on common and life-saving clinical scenarios found in international guidelines for sepsis and appendicitis, relevant across resource settings and updated in the past 4 years (table 1, online supplemental appendix A).21 The survey link was available through REDCap (Research Electronic Data Capture) until over 100 participants completed the survey.

Each participant’s knowledge score was the sum of their correct answers to knowledge questions. STATA, version 7 (StataCorp, 2015, College Station, TX) was used for data analysis. All variables of interest were ordinal categorical variables. For simplicity in reporting and interpreting our results, we used linear regression with robust SE to test for association between variables.22 In a post-hoc analysis, ordinal logistic regression resulted in the same statistical inference. In univariate analyses, a single independent variable was included in the regression model. In the multivariate analysis, the dependent variable was knowledge score, and the independent variables were selected a priori: age, country income level, English reading comprehension, translation quality, frequency of unanswerable clinical questions, frequency of uncertainty regarding clinical practice, and access to effective resources. For each variable of interest, we report a crude (univariate) and adjusted (multivariate) regression coefficient and 95% CI. For analyses related to knowledge score, the regression coefficient can be interpreted as the average change in knowledge score (ie, 1–5) associated with a one-unit increase in the independent variable. Statistical significance was set at p<0.05.

Results

Discussion

This study set out to understand how surgical trauma providers in the Americas acquire and appraise information to properly inform their clinical practice. Overall, these providers felt they had adequate resources to address their clinical uncertainties and were confident in their ability to interpret and apply those resources to their clinical practice. However, most reported difficulty resolving clinical questions even when using all resources available to them and answered at least one survey knowledge question incorrectly. Although there will always be room for growth in clarification of clinical issues, and the available literature may not provide direction in complex individual patient scenarios, we also captured a significant degree of uncertainty regarding more discrete issues which do have relevant, publicly available, recommendations based on a strong evidence base.

Interestingly, higher reported confidence in interpreting and applying literature to clinical practice correlated to lower knowledge score in univariate but not multivariate analysis. Previous studies parallel our findings, underlining the Dunning-Kruger effect describing the cognitive bias in which people overestimate their medical knowledge and ability.23 24

Participants perceived their translation services, when required, to be sufficient for clinical practice. This mirrors prior research on Google Translate working well in medical settings.25 However, participants with lower reported English reading comprehension also reported lower confidence in interpreting and applying literature to their clinical practice, showing that even with the use of translation services, language persists as a barrier to answering life-saving clinical questions. One in five providers felt that language prevented them from practicing medicine using evidence. Language competency has been previously reported to be an explicit barrier to evidence-based care6 and is here seen to undermine provider confidence in evidence interpretation.

In this analysis, when holding other variables constant, increased age correlated with decreased score on the included knowledge assessment. Although experience likely improves various other skills relevant to the provider, previous research also connects more years in practice with decreased adherence to current standards of practice and therapies and decreased knowledge.26 More senior providers are further from medical education and may not be exposed to practice updates outside of the academic setting. In some reports, age correlates with lower rates of technological literacy and thus potentially difficulty accessing guidelines related to our survey knowledge questions.27 The authors recognize that endorsement of and adherence to EBM is only one part of clinical care. These data do not capture other important components of clinical decision-making, including judgment, cognitive biases, collaborative decision-making, and other features of clinical care that may have different trends throughout providers’ careers. Providers may depend on their early formal clinical training years to acquire clinical knowledge, and then have limited and compromised options for new knowledge acquisition throughout their career. Further research is indicated on customized interventions to address barriers to EBM unique to various groups.

Country income level also significantly influenced knowledge scores, when holding other variables constant, yet all answers to the survey knowledge questions were available in well-established and public-access online guidelines. This confirms the complexity of the socioeconomic barriers to evidence-based practice. These data do not capture reasons for higher knowledge scores in high-income countries, and future research could focus on underlying factors. Most participants reported interest in learning more about research and evidence-based practice, indicating that providers recognize the value of, and would appreciate more focus on, these topics.

Conclusions

Most surveyed surgical trauma providers felt confident in their evidence-based practice yet do not answer all clinical questions correctly even in an “open book” assessment. Lower country income level and higher age of provider corresponded to fewer clinical questions answered in congruence with international guidelines. Almost one out of every five participants felt their English competency was a barrier to evidence-based practice. These data highlight the broken, and even sometimes absent, dissemination and implementation schemes for new, effective interventions—many of these interventions only available in English.

Our participants were motivated to expand their knowledge of EBM and research, indicating a disconnect between opportunity and desire for practicing using the latest recommendations. Equity in healthcare necessitates overcoming linguistic barriers so all providers can use evidence-based research written in any language to drive their clinical practice. Furthermore, the logistical, resource, and behavioral barriers that contribute to an age and resource gap in clinical knowledge scores merit further research and ultimately customized intervention.

Every single patient depends on their provider to adequately acquire and appraise new data and practice EBM. In surgical trauma settings, lives are often at stake. These data underscore the urgency to holistically improve global trauma systems by investing in continuous education and building accurate, rapid language translation to improve dissemination and implementation of evidence-based practice.