By Heidi Reyst Ph.D., CBIST

Good doctors use both individual clinical expertise and the best available external evidence and neither alone is enough.

The Problem

In 2004, Dr. David Cutler provided America’s healthcare system a checkup. In his book, Your Money or Your Life: Strong Medicine for America’s Healthcare System, Cutler painted a picture of a “patient” in distress, in this case the healthcare system itself. While his examination was metaphorical, there was nothing figurative about the results. The patient’s symptoms included high costs, a lack of access to care and uneven or widely varied quality of care or clinical practices.(1) He called these the “holy trinity of health reform,” in a time when there was little promise for any chance of reform.(2)

Fast forward to 2015, where healthcare costs continue to rise. Health expenditure per capita was $4,791 in 2000, and $8,745 in 2012,(3) and healthcare’s share of the GDP is expected to rise from 16 percent in 2006 to 19.5 percent by 2017.(4) The continual rise in the cost of healthcare is now coupled with fewer individuals who are uninsured.(5) Taking the 30,000 foot view on this might yield the following: I may be lucky enough to have access to care by having healthcare insurance, and I may be able to pay the costs of that care, but now will I have to worry about whether I will receive top-notch care?

In discussing this issue of variable quality of care, Dr. Cutler noted the usual suspect one would worry about, namely errors, which is beyond the scope of this article. However, he noted other concerns such as too much care, too little care, or higher spending on the same healthcare with no discernible change in mortality or patient quality of life.(1) In a very well-articulated argument, he noted that a significant way to improve healthcare based on these three elements would be to focus on increasing the value we get for the dollars we spend.

The question begs how, exactly, do we go about that?

The Answer?

One answer to that question comes in the form of Evidence-Based Medicine, or EBM. The core aspect of EBM is that practitioners will utilize the best available evidence, in conjunction with clinical expertise, when determining treatment options for a patient. The history of EBM is a relatively short one, yet one that ensued at an incredibly fast pace.

The Origins of EBM

EBM’s “modern” origins date back to the late ‘60s and early ‘70s. Dr. David Eddy, an influential force in the burgeoning field noted that it was the existing environment of medicine that created the need for the movement towards the idea of “evidence based”.(6) He described this environment as follows:

Up until about 40 years ago, medical decisions were doing very well on their own, or so people thought. The complacency was based on a fundamental assumption that through the rigors of medical education, followed by continuing education, journals, individual experiences, and exposure to colleagues, each physician always thought the right thoughts and did the right things. The idea was that when a physician faced a patient, by some fundamentally human process called the “art of medicine” or “clinical judgment,” the physician would synthesize all of the important information about the patient, relevant research, and experiences with previous patients to determine the best course of action. “Medical decision making” as a field worthy of study did not exist. Analytical methods and mathematical models were limited to research projects.

In the context of the late ‘60s and early ‘70s, in which clinical judgement was firmly entrenched in the status quo, one can only imagine how difficult it would be to bring about a significant change in culture. Hindsight being what it is, clearly there were game-changing insights, or “aha” moments, and a fair amount of elbow grease that actually resulted in the culture moving. What, then, spurred this movement?

Dr. Eddy pointed out two such assumptions that moved the world of medicine towards a more evidence-based paradigm. The first assumption found to be critically flawed was the premise of how clinical judgments were made, namely those based on the rigors of medical training, continuing education and tradition. The second faulty assumption was the idea that what was happening in current clinical practice correlated with research available to inform clinical practices.

EBM—The Rise of Clinical Epidemiology

The game-changers, as I will call them, came from a variety of places like Canada, Great Britain, and the United States, as well as from a variety of disciplines like medicine, epidemiology, public health, mathematics, and statistics. Much of what the game-changers accomplished was to fundamentally discredit the prevailing assumptions about “the art of medicine.”

More on the origins of EBM can be found in the sidebar at right.

EBM—The Modern Era

The term ‘evidence-based’ was coined and used by Dr. Eddy in the latter half of the 1980s.(7) He first published the term in a 1990 Journal of the American Medical Association article titled “Practice policies: where do they come from?” Two years later, the term ‘evidence-based medicine’ was published in JAMA by The Evidence-Based Medicine Working Group headed by Gordon Guyatt, with his colleagues from McMaster University, along with a multi-national team of experts from Britain, the United States and elsewhere.

Though seemingly indistinguishable from one another the two terms came from distinctly different contexts. For Eddy, use of the term was in relation to evidence-based guidelines policy.(7) Guyatt and colleagues’ use of the term stemmed from a much different frame of reference—that of “a new approach to teaching the practice of medicine”.(8)

The latter term, along with the seminal work of the McMaster University program, helped propel the concepts of evidence-based medicine forward and distinctly brought about the modern era of EBM.

Indeed, the most widely referenced definition of EBM came directly from that group in 1996:(9) Evidence based medicine is the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients.

Though widely referenced, it does not tell the entire story of EBM—as essentially two distinct branches of EBM developed out of this. Eddy himself classified the two models. One model, which he termed Evidence-Based Individual Decision Making (EBID), was championed by Sackett, Guyatt and the McMaster group. The other model Eddy termed Evidence-Based Guidelines (EBG) and focused more on the development of guidelines.

The importance of these two models lies in their distinction of how they affect the patient. The EBG model will impact the patient indirectly via the development of guidelines that are designed to impact groups of people, whereas the EBID model affects the patient directly by providingmedical science to clinical judgment. Table 1, outlines the key differences in the models.

EBID Model

When EBM was first beginning to be conceptualized, it was presented as a stark contrast to the “art of medicine.”(10) Like any new concept, it evolved over time. To a degree, it has come full circle, as Sackett, Rosenberg, Gray, Haynes and Richardson9 noted: “Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannized by evidence, for even excellent external evidence may be inapplicable to or inappropriate to an individual patient.” This idea has been represented in a Clinical Decision Model, shown in Figure 1.(11, 12, 13)

The model incorporates three critical aspects of the evidence-based clinical decision-making model. One aspect of the model, Patient Values, takes into account the patient’s preferences, circumstances and values.(11, 12, 13) Examples of this include the level of risk the patient is willing to accept, their ability to follow treatment recommendations, cultural factors, insurance issues, personal values and a host of other considerations.(11)

A second aspect of the model pertains to the Clinical Circumstances with which the patient presents. Examples of this include symptoms, diagnoses, accessibility to care and other considerations which may be central to the decision making process.(11)

The third aspect is Research Evidence. This is the aspect which takes medical decision making from the “art of medicine” to “evidence-based.” In particular, research evidence includes “systematic observations from the laboratory, preliminary pathophysiological studies in humans and more advanced applied clinical research, such as randomized controlled trials with outcomes that are immediately important to patients.”(11) It builds on the aspects that have always guided clinical decision making, namely, patient values and clinical circumstances. Collectively, all three components impact clinical judgment, with the goal being to assist practitioners to make the best decision for care, given all the relevant information.

The Clinical Decision Model also places importance on all the aspects that go into the practitioner having the information they need to make the best decision for each patient. Beyond basic skills and experience, the ability to keep up with growing evidence and expanding requirements that impact skills is tremendous. In fact, Archie Cochrane noted this fact decades ago. He recognized that there is no utility in clinical trials if the information contained in them is inaccessible to physicians.

In an article published in 2010, it noted there were 75 clinical trials and 11 systematic reviews published in the medical literature on a daily basis, or more than 30,000 per year!(14) In the age of managed care, the time needed to keep up to date is untenable. What then is the answer to getting the right information into physician’s hands? The likely answer is Eddy’s(6) second EBM Model: evidenced-based guidelines, or EBG – Model.

EBG – Model

The starting point for the discussion of evidence-based guidelines has to be with the word “evidence.” Without evidence, medical decision making falls back solely to the “art of medicine.” Early on, the randomized controlled trial (RCT) was considered the gold standard of evidence obtainment. The problem with RCTs lies in the reality that they may not be the definitive answer, nor are they practical to conduct for every diagnosis or treatment. If RCT is impractical in certain situations, or (gasp!) if more than one RCT provides contradictory evidence, where do practitioners turn?

The key to establishing the best available evidence is to complete a systematic review of all available evidence—the great, the good, the bad and the ugly. Once a systematic review is conducted, evidence-based guidelines can be developed. The key to the development is ensuring robust methodology is explicitly established and explicitly followed.

In addition, it is crucial that the process be transparent and free from bias. This ensures that the users of the final guidelines have knowledge of how the guidelines came about, understand the strength of evidence that was used to inform the guidelines and that the process was free of conflicts of interest.

The companion article to this one titled ‘Evidence-Based Guidelines: Development and Dissemination’ reviews guideline development, and delineates the systematic reviews process.

At the beginning of this article, three problems were identified in healthcare today—rising costs, lack of access to care and uneven or varied quality of care. One way to combat this problem is to focus on obtaining a better value for the healthcare dollars we spend.

The question loomed—how, exactly do we do that? The initial answer was a global one. Improve clinical decision making through the incorporation of evidence-based medicine. Or, more specifically, enhance clinical judgment by bringing direct evidence of treatment efficacy to bear on the patient. The more succinct answer is to use the principles of evidence-based medicine (which emphasize examining the evidence) to create evidence-based guidelines (which have an explicit methodology for review of all available evidence).

When the latter is done, for a given treatment of a given diagnosis, variability of treatment for the patient is reduced. When successfully reduced, we should no longer see wide-ranging treatments that ultimately increase the cost of care, yet provide no better (or even worse) efficacy in terms or mortality or morbidity. ❚

Evidence-Based Guidelines: The Development and Dissemination Process

“Guidelines help clinicians translate best evidence into best practice. A well-crafted guideline promotes quality by reducing healthcare variations, improving diagnostic accuracy, promoting effective therapy, and discouraging ineffective—or potentially harmful—interventions.”(5)

Evidence-Based Guidelines (EBG) came out of the evidence-based medicine (EBM) revolution which transpired over the last 40 or so years. The need arose from an understanding that, for any given standard medical treatment which was assumed to be effective and in the best interest of the patient, there was no real science to indicate that it was actually effective.

In fact, as the EBM revolution gained traction, research started to debunk many tried and true medical treatments finding they were not only ineffective, but in some cases were more harmful than good. This turned old assumptions about how medical decisions are made on their head. The “art of medicine” as the old model is often referred, could no longer be the sole impetus for clinical judgments. The new model included the basic aspects of the art of medicine, but also included medical science, in the form of evidence, to inform clinical decision making.

The goal of EBM is to link supporting evidence to clinical practice.(1) In order to bring evidence derived from medical science to physicians, which then informs clinical judgment, a systematic process is needed. An important output of that process is evidence-based guidelines (EBGs). At a philosophical level, EBGs involve five core concepts:(2)

1. Medical decisions should be based on the best available evidence.
2. The patient’s clinical problem determines the type of evidence sought to inform the clinical judgment.
3. Identifying the best evidence means using epidemiological and biostatistical methods.
4. The conclusions from the evidence then must be applied to patients.
5. The performance should be continually assessed.

These core concepts are directly derived from EBM and make up the philosophical starting point for effective medical care. The key to moving from this philosophical starting point to effective treatment is to develop EBGs. The development of EBG is a rigorous, complex, and time-consuming process. But at the end of the day, the benefits can far outweigh the barriers. If EBG result in better information getting to practitioners, better diagnosis and care, less variability in care, and improved patient outcome, the development process is well worth the effort.

Guideline Development —the guiding principles

The development of clinical guidelines has many important guiding principles, the sum of which are designed to limit bias and enhance trustworthiness in the guidelines and is based on the standards set forth by the Institute of Medicine (IOM).(3, 4)

The first principle is to have a transparent process that clearly describes how the guideline is developed. This is considered to be one of the most, if not the most, important standard. This gets to the heart of guideline trustworthiness. When announcing and disseminating guidelines, a transparent process allows stakeholders the ability to see precisely how the guidelines were developed should there be any concerns or questions. The second principle is to manage conflicts of interest, with a clear eye toward having very few conflicts overall. The third principle is to have a guideline development group that have appropriate proficiency in the applicable field. The fourth principle is to complete a robust, methodologically sound systematic review. The fifth principle is to be clear about evidence quality and recommendation strength. The sixth principle is to clearly articulate the recommendations. The seventh principle is to submit the guidelines for external review. The final principle is to have a plan to update the guidelines as new evidence warrants.

For groups developing EBG, the greater the adherence to each of these steps, the greater the level of trust that stakeholders, including the public and potential beneficiaries of these guidelines, may hold. The use of these guiding principles will ultimately determine if the guideline is welcomed, trusted, and ultimately followed by those stakeholders for whom it was intended. Failure to use them can leave stakeholders with questions as to the integrity of the process, and most importantly, the plausibility of the guideline itself.

The systematic review

The previous section focused on principles of guideline development. This section is focused on detailed elements of the system review process which ultimately leads to guideline completion. In 2011, the IOM published a book on standards for systematic reviews titled Finding What Works in Health Care: Standards for Systematic Reviews.(6) The following sections summarize the IOM standards for guideline development, with the major steps presented in Figure 1.

Initiate Systematic Review

The first steps in a systematic review involve the development of standards for the review, which are clear and explicit. These steps are designed to ensure that conflicts of interest and bias are mitigated throughout the review process. The steps of this development are as follows:

Establish a review team. The team of individuals who comprise the guideline development group should have clinical expertise and experience as it relates to the focus of the guideline. The members of the team should collectively have experience with systematic review methods and strong knowledge of quantitative methods, i.e., statistical prowess.

Obtain stakeholder input. Stakeholder input should be solicited and should be planned throughout the entire process. This can assist in preventing the derailment of the project because a crucial aspect was overlooked. It is also central to the transparency aspect of guideline development.

Manage conflict of interest and bias. All members of the team (at any stage of the process) should disclose any conflicts of interest, financial conflicts or potential biases that may exist as it relates to the project. The team should exclude any members whose bias would impact the credibility of the project. This is crucial, as there are a host of ways in which guidelines could be self-serving or disreputable.

Imagine for a moment that a new guideline is distributed on the management of a chronic heart condition. Imagine also that the guidelines were bereft of transparency and made no mention of guideline developer conflicts of interest. If at the end of the day, imagine that the guideline development team was predominately from a pharmaceutical company whose recently developed drug has been targeted as the treatment for that condition. Should physicians, patients, policy-makers or any other stakeholder have good faith that conflicts of interest did not inject bias into the process? This example highlights the importance of managing bias in the guideline development process. It means not only having full disclosure of any conflicts, but also limiting the sum total of conflicts.

Develop the review topic. This is the starting point for the active guideline development stage. The guideline topic must be clearly articulated and link the treatment to the outcome of interest. In addition, once topic development is complete, it should be evident that the area of review is truly needed.

Topic development is done through formulation of a clinical question utilizing a uniform process.

At a basic level the questions relate the diagnosis/treatment to the outcome of interest. For example, the guideline developers may want to know about the best, most effective health screen. A question might be “what is the best method to screen for breast cancer?” Or, it may be a treatment of interest. A question for this scenario might be “what is the best treatment for diabetes mellitus?”

These broad level questions get at the heart of the systematic review topic, but they are too broad in scope to begin the next step of the process. What is needed is a very clear question that will lend itself to a structured review of the available evidence. The side bar at left provides an example of one such process for question generation.

Develop the systematic review protocol. The next major step in the systematic review process is to develop a well-defined protocol that will be followed explicitly. Figure 2 lists the steps in this process.

Submit the protocol for peer review and make the final protocol available to the public. This consists of a public comment period and a public report on the outcomes from the commentary. When that process is complete, the entire protocol should be made available to the public. When this is complete the next phase is to conduct a comprehensive search to find all available evidence.

Find and Assess Individual Studies

This is the phase of guideline development where the available evidence is garnered. Figure 3 outlines the steps of this process.
Conduct a comprehensive, systematic search for evidence. The amount of data in searchable databases is overwhelming. This is the starting point for evidence collection. But this voluminous data may only tell part of the story, as much research goes unpublished or the data in published studies may only be partially reported. To accomplish this step, specialists trained in performing systematic reviews must be employed. They are tasked with searching bibliographic databases, citation indexes and other means to find potential studies which meet the pre-ordained selection criteria.

Address potential bias in reported results. This step involves looking beyond the usual published studies. Unpublished, or grey literature (that which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers, e.g., www.greylit.org), should also be sought after to reduce the potential bias of using only traditionally published studies.

Screen and select studies. Strict adherence to the inclusion or exclusion criteria is key. Either abstracts or full-text articles are reviewed. Typically two members of the team review each abstract/article against the inclusion criteria. If both reviewers concur, the study is either in or out depending on the determination. If there is a conflicting answer, there is a protocol to determine the final outcome.

Document the search. This entails meticulous documentation of the search strategy, as well as the final outcome for each study and the reason for inclusion or exclusion.

Manage the data collection. Once the studies are selected, the data must be extracted, using standard protocols.

Critically appraise each study. This involves assessing bias, the relevance of the population, intervention and outcome measure in each study.

Synthesize the Body of Evidence

Use a systematic method to evaluate the body of evidence. This stage involves looking at a variety of qualities for each outcome examined. This includes assessing risk of bias, consistency, precision, directness and reporting bias. It also entails systematically assessing observational studies (where assignment of subjects cannot be directed), as well as using clear characterizations of confidence in effect sizes. The Figure 4 reviews the criteria for judging evidence.

Conduct a qualitative synthesis. This entails describing the methodological aspects of the evidence, the strengths and limitations of the studies, as well as the flaws and biases. The team also looks at the individual studies in relationship to the overall findings as well as patterns across studies, and assesses their relevance to the study population and the PICOT question posed.

Determine if a meta-analysis is needed. This entails justifying the need for pooled estimates.

Meta-analysis process. If meta-analysis is warranted, use experts to conduct it and ensure all relevant statistical norms are followed.

Reporting Systematic Review Findings

The final aspect of the systematic review entails preparing the final report using a structured format, initiating a peer review of the draft report and publishing the final report ensuring free public access. ❚

Definitions

Meta-analyses Meta-analysis is a systematic, objective way to combine data from many studies, usually from randomized controlled trials (RCTs), and arrive at a pooled estimate of treatment effectiveness and statistical significance. Meta-analysis can also combine data from case/control and cohort studies. The advantage to merging these data is that it increases sample size and allows for analyses that would not otherwise be possible. They should not be confused with reviews of the literature or systematic reviews.

Systematic Reviews A systematic review is a comprehensive survey of a topic that takes great care to find all relevant studies of the highest level of evidence, published and unpublished, assess each study, synthesize the findings from individual studies in an unbiased, explicit, and reproducible way and present a balanced and impartial summary of the findings with due consideration of any flaws in the evidence. In this way it can be used for the evaluation of either existing or new technologies and practices.

Randomized Controlled Trials An RCT is a study in which 1. There are two groups, one treatment group and one control group. The treatment group receives the treatment under investigation, and the control group receives either no treatment (placebo) or standard treatment. 2. Patients are randomly assigned to all groups.

Quasi-Experimental Studies Quasi-experimental research shares similarities with the traditional experimental design RCTs, but they specifically lack the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other
than random assignment (e.g., an eligibility cutoff mark).

Cohort Studies Cohort studies or longitudinal studies involve a case-defined population who presently have a certain exposure and/or receive a particular treatment that are followed over time and compared with another group who are not affected by the exposure under investigation.

Case Control Studies Patients who already have a certain condition are compared to those who do not.

Case Series and Case Reports These consist of collections of reports on the treatment of individual patients with the same condition, or of reports on a single patient. Case series/reports are used to illustrate an aspect of
a condition, the treatment or the adverse reaction to treatment.

The criteria for judging evidence

What are the criteria for judging evidence? There are a variety of aspects with which to judge. Is the evidence relevant to the clinical question (PICOT)? How much evidence, or how many pieces, is sufficient? What is the veracity of the information? When looking at the body of evidence as it relates to the clinical question each of these areas must be ascertained.

When looking at evidence there are two levels of examination. The first level examination is the individual studies. The team assesses the study’s relevance to the clinical question, its methodological design, the quality of its implementation, the number of subjects, and the precision of the study findings.(8) The second examination is at the level of the entire body of evidence. The team assesses the consistency of findings, examines the total number of subjects and the precision of findings after pooling subjects/studies (i.e., meta-analysis), as well as the strength of the studies making up the entire body.(8)

There are numerous determinants of the quality of evidence. One critical factor is the study design. Certain study designs will rate higher than others and a general rating level is best describe in a levels of evidence pyramid (Figure 4). For rating individual studies, the ‘unfiltered’ section is utilized. Generally, the randomized control trial is rated highest followed by cohort studies, case control studies and cross-sectional studies.

The quality of the study design itself also impacts the overall study quality. Important factors include the power analysis, quality of the outcome measure, quality control on the intervention, the use of blinding, methods for blocking/subject matching, as well as study implementation (e.g., small sample sizes, failures in blinding, attrition of subjects or missing data.)(8)

Lastly, directness refers to the extent to which the subjects, interventions and outcome measures are similar to the clinical question. Well designed, randomized control trial with consistent effect sizes are of little usefulness if they don’t directly pertain to the outcome measures.

[accordian][toggle title=”About the Author” open=”no”]

By Heidi Reyst Ph.D., CBIST

Heidi Reyst Ph.D., CBIST began her career in the field of Brain Injury in 1991. She has experience on both the care side and the administrative side as a Program Director, Systems Director, Director of Clinical Administration, and Vice President of Clinical Administration. Dr. Reyst is an advocate for ensuring that individuals working in brain injury are trained in the specific issues salient to brain injury. She is a Certified Brain Injury Specialist Trainer and is a member of the Academy of Certified Brain Injury Specialists. In 2008, Dr. Reyst was the recipient of the Brain Injury Association of Michigan’s Legacy Society “Professional Service Award”; this was awarded based on her involvement in advocating for those with brain injuries and for her training of those working in the field of Brain Injury through the ACBIS program.

Dr. Reyst has been a member of the ACBIS community since 2004, when she joined the ACBIS Corporate Alliance Council as Rainbow’s representative. In 2005 she became a member of ACBIS Board of Governors, serving as a member as well as the Vice Chairperson of Information Management. In January of 2016, Dr. Reyst became Chairperson of the ACBIS Board, and additionally, joined the Board of Directors of the Brain Injury Association of America. She was one of the 100 professionals from across the country selected to participate in the Guidelines for the Rehabilitation and Chronic Disease Management of Adults with Moderate to Severe Traumatic Brain Injury project sponsored by the Brain Injury Association of America and the Brain Injury Research Center at the Icahn School of Medicine at Mount Sinai.

Evidence-Based Medicine

1. Cutler D. Your money or your life: strong medicine for America’s health care system. New York, NY. Oxford University Press. 2004.

2. Cutler D. Your money or your life: strong medicine for America’s health care system. http://www.chrp.org/pdf/Cutler_041405.pdf. Power Point presentation accessed 9/5/2015.

3. OECD Health Statistics 2014. How does the United States Compare? http://www.oecd.org/unitedstates/Briefing-Note-UNITED-STATES-2014.pdf. Accessed July 28, 2015.

4. Keehan S, Sisko A, Truffer C, Cowan P, Clemens M. Health Spending Projections Through 2017: The Baby-Boom Generation Is Coming To Medicare. Health Affairs. 2008; 27(2): W145-W155.

5. Majerol M, Newkirk V, Garfield R. The Uninsured: A Primer. Key facts about health insurance and the uninsured in America. http://files.kff.org/attachment/the-uninsured-a-primer-key-facts-about-health-insurance-and-the-uninsured-in-america-primer. Accessed 9/5/2015.

6. Eddy D. Evidence-Based Medicine: A Unified Approach. Health Affairs. 2005; 24(1): 8-17.

7. Eddy D. History of Medicine. The Origins of Evidence-Based Medicine – A Personal Perspective. Virtual Mentor. 2011; 13: 55-60.

8. The Evidence-Based Working Group. Evidence-Based Medicine: As New Approach to Teaching the Practice of Medicine. JAMA. 1992; 268 (17): 2420-2425.

9. Sackett D, Rosenberg W, Gray M, Haynes B, Richardson W. Evidence-Based Medicine: What it is and what it isn’t. BMJ. 1996; 312: 71-72.

10. Timmermans S, Mauck A. The Promises and Pitfalls of Evidence-Based Medicine. Health Affairs. 2005; 24(1): 18-28.

11. Haynes B, Devereaux P, Guyatt G. Clinical Expertise in the Era of Evidence Based Medicine and Patient Choice. EBM Notebook. 2002; 38(7): 36-38.

12. Shah H, Chung K. Archie Cochrane and his Vision for Evidence Based Medicine. Plast Reconstr Surg. 2009; 124(3): 982-988.

13. Bhandari M, Giannoudis, P. Evidence-Based Medicine: What it is and what it is not. Injury, Int. J. Care Injured. 2006; 37: 302-306.

14. Bastian H, Glasziou P, Chalmers I. Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up? PLOS Medicine. 2010; 7 (9): 1-6.

Evidence-Based Guidelines

1. Fletcher R, Fletcher S. Evidence-Based Approach to the Medical Literature. JGIM. 1997; 12 (supplemental 2): S5-S14.

2. Davidoff F, Haynes B, Sackett D, Smith R. Evidence-Based Medicine. BMJ. 1995; 310 (6987): 1085-1086.

3. Graham R, Mancher M, Wolman D, Greenfield s, and Steinberg E. Clinical Practice Guidelines We Can Trust. Institute of Medicine of the National Academies Press. 2011.

4. Ransohoff D, Pignone M, Sox H. How to Decide Whether a Clinical Practice Guideline is Trustworthy. JAMA; 2013; 309 (2): 139-140.

5. Rosenfeld R, Shiffman R. Clinical Practice Guidelines Development Manual: A Quality Driven Approach for Translating Evidence into Action. Otolaryngol Head Neck Surg. 2009; 140 (Suppl 6): 1-73.

6. Eden R, Levit L, Berg A, Morton S. Finding What Works in Health Care. Standard for Systematic Reviews. Institute of Medicine of the National Academies Press. 2011.

7. Davies K. Formulating the Evidence Based Practice Questions: A Review of Frameworks. Evidence Based Library and Information Practice. 2011. https://ejournals.library.ualberta.ca/index.php/EBLIP/article/viewFile/9741/8144.

8. Dijkers M. TBI Guidelines Training Document. TBI Guidelines Project.

o earn CBIS and CCM credits please complete the forms below. A copy of your completed form will be emailed to you for your records after submission.