The Evidence

Evidence-based medicine seeks to prioritize medical research from the most valuable to the least valuable, from the most accurate to the least accurate, from the most biased to the least biased and from the most applicable to the least applicable.

There are Levels of Evidence for different clinical questions, regarding therapy, prognosis, diagnosis etc: 

Evidence also varies with regards to study designs. The most useful study designs are randomized controlled trials, systematic reviews and meta-analysis for therapy. This would be considered Level I Evidence. On the other hand, the least useful study designs would be unsubstantiated anecdotal information by an “expert”. This would be considered Level V Evidence. All other evidence lies somewhere in between the two.

The Evidence that can be achieved varies for different subjects, scientifically.  Pharmacological research, following existing protocols, can produce dependable Level I Evidence for an EBP to use when making decisions on what medications, dosage and precautions should be considered when answering clinical questions.  I may start a patient out on Keflex during an initial encounter with an infection, but I will use Level I Evidentiary studies to determine my eventual antibiotic of choice.   However, variables such as pedal variations among subjects, the environment and in function and quality of life goals, often prevents the development of Level I Evidence when it comes to biomechanics.  This means that when it comes to biomechanics, Level II, III, IV or even V evidence may be the best available to use when answering clinical questions in an evidence based biomechanical practice and all evidence deserves scrutiny as to clinical relevance.

Two Examples:

1. Gross NL, Davlin LB, Evanski PM. Effectiveness of orthotic shoe inserts in the long-distance runner. Am J Sports Med 1991; 19:409-412 (Level IV, Orthotic Poorly Defined, Weak Criteria, Lacks Longevity)
2. Gross MT, Byers JM, Krafft JL, Lackey EJ, Melton KM: The impact of custom semirigid foot orthotics on pain and disability for individuals with plantar fasciitis. J Ortho Sp Phys Ther, 32:149-157, 2002. (Level IV, Orthotic Well Defined, Strong Criteria, Long Follow-up),

Diagnostic, therapeutic, and prognostic questions arise when practicing biomechanics.  Often, one case may raise questions from all three categories. 

Waiting for Level I Evidence to surface or avoiding examining new hypotheses lacking Level I Evidence, when it comes to biomechanics, often doesn’t make clinical sense when it comes to addressing patients needs clinically.

Systems to stratify evidence have been developed, such as this one by the U.S. Preventive Services Task Force. 

• Level I: Evidence obtained from at least one properly designed Randomized Control trial (RTC).
• Level II: Evidence obtained from well-designed controlled trials without randomization.
• Level III: Evidence obtained from well-designed Cohort or Case Control Analytic Studies, preferably from more than one center or research group.
• Level IV: Evidence obtained from multiple time series with or without the intervention. Dramatic results in uncontrolled trials might also be regarded as this type of evidence.
• Level V: Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees as well as cadaver and animal study.

Using techniques from science, engineering and statistics, such as meta-analysis, risk benefit analysis and RTC’s, "experts" should strive to reference their declarations to scientific studies.

The systematic review of published research studies for the benefit of EBP’s, like the Cochrane Collaboration, allows authors to categorizes evidence as "likely to be beneficial", "likely to be harmful", or "evidence did not support either benefit or harm."

A 2007 analysis of 1016 systematic reviews from all 50 Cochrane Collaboration Review Groups found that 44% of the reviews concluded that the intervention was "likely to be beneficial", 7% concluded that the intervention was "likely to be harmful", 49% concluded that evidence "did not support either benefit or harm" and 96% recommended further research.

The Levels of Evidence

When discussing Biomechanics, it is best to discuss the Levels of Evidence from V to I, since the Level I Evidence is scarse and lacks clinical acceptance.

Level V: Unsystematic anecdotal information and cadaver and animal studies are highly biased and impossible to verify and is considered the least valid form of evidence.  Level V information is best used to develop hypotheses, which are tested using more rigorous study designs.

Level IV: Case Reports and Series are collections of reports on the treatment of individual patients or a report on a single patient. They are reports of cases, use no control groups and have no statistical validity.  They are easy to perform. They are usually retrospective, but may be prospective. They lack a control group, and suffer from selection bias. These studies are best utilized to develop hypotheses rather than test hypotheses.

Level III: Case Control Studies are studies where patients already having a specific condition or outcome are compared with people who do not have the outcome, to retrospectively identify possible interventions. They rely on medical records and patient recall for data collection. These types of studies are often less reliable than randomized controlled trials and cohort studies, because showing a statisticalrelationship does not mean than one factor necessarily caused the other.  These are retrospective studies using a comparator. Since it is difficult to find a good control group for these studies, selection bias is always a concern. Recall bias is another common concern regarding case-control studies.

Level II: Cohort Studies take a large population of patients who are taking a treatment or intervention and follow them over time (prospectively) to observe an outcome and compare it to a similar group not receiving the treatment.  Cohort studies are sometimes flawed because the two groups differ in ways other than in the variable under study.  These studies are prospective, minimizing bias, but may be retrospective.  These studies are expensive, time-consuming and require careful attention to detail in evaluating the control group. As with any control group study, selection bias is always a concern. Results typically are expressed as a relative risk rather than odds ratios.

Level I – Randomized Controlled Trials and Systematic Reviews and Meta-analysis: Randomized controlled trials (RCT’s) are true controlled experiments. Two or more groups of subjects receive different interventions and are followed forward in time and at some point, they are compared using an outcome. RTC’s are carefully planned projects that study the effect of a therapy on real patients. They include methodologies that reduce the potential for bias (randomization, blinding, follow-up, etc.) and that allow for comparison between intervention groups and control (no intervention) groups.

RTC’s minimize selection bias, but they are difficult to perform correctly and incur considerable expense. RTC’s in Biomechanics are rare and even when published remain questionable because they lack acceptable error rates or incorporate biased influence of predetermination by authors and those controlling funding or lack acceptable error rates.

Systematic reviews usually focus on a clinical topic and answer a specific question.  An extensive literature search is conducted to identify studies with sound methodology. The studies are reviewed, assessed, and the results summarized according to predetermined criteria of the review question.  They can be considered Level I Evidence when randomized controlled trials are utilized.  They require careful planning and methods which are transparent and allow other researchers to replicate the results and reach similar conclusions, minimizing bias.  

Meta-analysis takes a systematic review one step further by combining the results of multiple studies using accepted statistical methodology and reporting them as one large study. The Cochrane Collaboration has done a lot of work in the area of meta-analysis.  Because Meta-analysis uses methodology including critical appraisal of the selected RCTs for analysis, it produces more of a quantitative rather than a qualitative result and yields Level I Evidence if limited to RTC’s.

Diagnostic Test Studies are prospective trials that blindly compare two diagnostic tests. This is a controlled trial that looks at patients with varying degrees of an illness and administers both diagnostic tests to all of the patients in the study group. The sensitivity and specificity of the two tests are compared to determine potential usefulness.

Figure 1. Shows the Pyramid of Evidence, starting at the bottom as low level up to the highest levels at the top.

In a Biomechanics EBP, it is fundamental to utilize the existing evidence to answer diagnosis, treatment and prognostic questions that arise as patient care is rendered.  The evidence must be identified and then appraised as to Level and then reviewed to determine relevance for applying into practice by determining accuracy, numbers of subjects and bias in order to make a decision as to whether the evidence deserves incorporating into your patient encounter and EBP (conisider the dangerous bias revealed in the recent Avandia Study).

References:

1. Timmermans S, Mauck A (2005). "The promises and pitfalls of evidence-based medicine". Health Aff (Millwood) 24 (1): 18–28.
2. Elstein AS (2004). "On the origins and development of evidence-based medicine and medical decision making". Inflamm. Res. 53 Suppl 2: S184–9.
3. Eddy DM (2005). "Evidence-based medicine: a unified approach". Health affairs (Project Hope) 24 (1): 9–17.
4. Gray J, Muir L: Evidence-based health care. Edinburgh: Churchill Livingstone. 1997.
5. Eddy DM (2005). "Evidence-based medicine: a unified approach". Health Aff (Millwood) 24 (1): 9–17.
6. El Dib RP, Atallah AN, Andriolo RB (August 2007). "Mapping the Cochrane evidence for decision making in health care". J Eval Clin Pract 13 (4): 689–92.
7. Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V (2001). "Reviewing the reviews. How strong is the evidence? How clear are the conclusions?". Int J Technol Assess Health Care 17 (4): 457–466.
8. Atkins D, Best D, Briss PA, et al. (2004). "Grading quality of evidence and strength of recommendations". BMJ 328 (7454): 1490.
9. ScienceDirect – Editorial: Medical Hypotheses : Pursuing treatments that are not evidence based: How DSM IV clarifies, how it blinds psychiatrists to issues in need of investigation;Volume 72, May 2009, 491-498
10. Sackett DL, Straus S, Richardson S, Rosenberg W, Haynes B. Evidence based medicine: how to practice and teach EBM. 2nd ed. London: Churchill Livingston
11. Guyette, G: Evidence-based medicine. A new approach to teaching the practice of medicine". JAMA 268 (17): 2420–5. November 1992.

Next Installment: Does Heel Contact Gait Promote Injury?

Major Sponsors