By Desmond Chan, SRT and Shirley Quach, CRE RRT HBSc
Similar to medicine and nursing, Respiratory Therapists (RRT) also utilize scientific evidence to guide respiratory interventions. It is imperative for RTs to understand the importance and the process of research appraisal because this helps to avoid the use of bias research conclusions and results when treating patients. In order to reinforce evidence-based practice, the skill in identifying and selecting studies with robust research methodology should be further developed to ensure the provision of reliable and appropriate respiratory care.
A core concept of the research appraisal process is the hierarchy of evidence. This hierarchy ranks different types of studies based on how rigorous the research methodology is (Evidence-Based Practice in Health, 2019). In general, studies that are positioned higher in this pyramid scheme use more thorough research methodologies and cumulate verified data for their investigation. Hence, results from these sources contain the least research or experimenter bias, a process where scientists conducting the study influences the results in order to obtain the desired outcome. On the other hand, experts’ opinions, editorials are at the bottom of the hierarchy, where the information generated at this level tend to be more biased and derived from anecdotal experience (Evidence-Based Practice in Health, 2019). Information and data from the bottom of the pyramid do not provide the best evidence for clinical decisions.
Filtered information: These studies assessed the qualities of multiple primary research with similar investigation and recommend the application of the synthesized conclusion in practice (Evidence-Based Practice in Health, 2019). Since critical appraisal has already been performed, they are well-suited for decision making at point-of-care. Examples include:
- Cochrane Database of Systematic Reviews
- BMJ Clincial Evidence
- ACP Journal Club.
Unfiltered information: Primary studies that have not been synthesized. Thus, it is harder to read, interpret, and apply results or conclusions. Examples include:
Systematic Reviews and Meta-analysis
Systematic reviews summarize all relevant scientific studies on a particular health topic. Its goal is to analyze existing literature by identifying, appraising, and synthesizing all research evidence pertaining to its review question (Uman, 2011). To do so, it utilizes a comprehensive and rigorous methodology, which includes eight stages, to reduce bias and improve the reliability of its conclusion. In addition, systematic reviews often include a meta-analysis, which synthesizes the data/results from multiple independent studies to calculate an overall “effect size” (Shorten & Shorten, 2012). This effect essentially assesses the strength of the relationship between the statistical significance and clinical significance of the collective studies, therefore providing information on the magnitude of the intervention effect (i.e. the extent of efficacy of a particular intervention clinically) (Uman, 2011).
Critically Appraised Topic
Critically appraised topics (CAT) are short synopsis of a few research studies, with a goal to answer a clinical question. These studies are synthesized, evaluated, and critiqued based on their internal, external, and statistical validity (Fetters et al., 2004). As a result, the CAT addresses the strength and applicability of the evidence pertaining to the clinical question. Unlike a systematic review, a CAT is shorter in length and uses less robust research methodology.
Randomized Controlled trial
Randomized controlled trials are the gold-standard when it comes to studying the efficacy and safety of therapeutic treatment. A typical RCT involves an experimental group that receives the intervention that is being tested, whereas the other group is either a comparison or control group receiving a placebo or an alternative intervention (Kendall, 2003). The purpose of this design and subsequent analysis is to determine whether a cause-effect relationship exists between the intervention (e.g. blood pressure medication) and the desired outcome (e.g. improve blood pressure).
Cohort studies are a type of longitudinal study – an approach that follows a group of subjects with specific exposure characteristics for a period of time. The goal is to determine whether exposing the research subjects to certain risk factors over a period of time would affect the occurrence of a disease. Essentially, these types of studies help identify what factors may increase or decrease the likelihood of developing a disease, also known as the odds ratio (Barrett & Noble, 2019). Cohort studies may be prospective or retrospective. In a prospective cohort experiment, two groups of participants with different baseline characteristics are recruited (e.g. smokers versus non-smokers) that are observed over time to see which group develops a certain outcome (e.g. lung cancer). On the other hand, a retrospective cohort study usually occurs when researchers use medical records to identify the factors associated with the outcome of interest.
Case controlled/ Cross-sectional studies
Case-control studies are a type of retrospective study that determines the relationship between one or more risk factors and an outcome of interest (Case-Control Study – An Overview, ScienceDirect Topics, n.d.). In particular, it investigates whether cases with the outcome of interest (e.g. specific birth defect) and cases without event of interest (i.e. control group) had similar or different levels of exposure to a risk factor (e.g. a medication), also known as relative risk.
Expert Opinions, Anecdotes, and Editorials
In scientific journals, editorials are short, invited opinion pieces that discuss a particular topic of interest. They typically do not contain any abstract, figures, tables, and results (Instructions for authors of editorial, focus, perspective, and commentary articles, 2018). The author of editorials can be an expert in the field of interest. Although the ideas and opinions presented come from experienced individual(s), their anecdotes and observation can be biased due to their beliefs, practices and judgment (Ingham-Broomfield, 2016). Usually, the evidence and information presented can be under-researched and outdated. Therefore, expert opinions are the least reliable in the hierarchy of evidence.
Evidence leading to Practice
When RTs use this hierarchy to collect evidence, one needs to beware that the position of systematic reviews at the top of the hierarchy may not be absolute at all times (Evidence-Based Practice in Health, 2019). Counterintuitively, primary studies can sometimes be a better source of information. The reason being is that a robust systematic review can take years to complete. Besides, the strength and rigour of the systematic review itself must also be evaluated by experts before being applied to patients. Therefore, a systematic review can be superseded by more recent evidence from primary studies before it becomes applicable. If a systematic review is dated or is not available for a present clinical situation, primary studies may offer more reliable and useful information (Evidence-Based Practice in Health, 2019).
Question Type or Domain
Best Research Design to Answer the Question
|Therapy (Treatment)||Randomized Controlled Trial (RCT)|
|Prevention||RCT or Prospective Study|
|Diagnosis||RCT or Cohort Study|
|Prognosis (Forecast)||Cohort Study and/or Case-Control Series|
|Etiology (Causation)||Cohort Study|
Table 1: List of the most effective research methodology depending on the type of clinical
investigation (Evidence-Based Practice in Health, 2019).
Barrett, D., & Noble, H. (2019).
What are cohort studies? Evidence-Based Nursing, 22(4), 95–96. https://doi.org/10.1136/ebnurs-2019-103183
Case-Control Study—An overview | ScienceDirect Topics. (n.d.).
Retrieved January 10, 2020, from https://www.sciencedirect.com/topics/nursing-and-health-professions/case-control-study
Evidence-Based Practice in Health: Hierarchy of Evidence. (2019, September 6).
Retrieved January 10, 2020, from https://canberra.libguides.com/c.php?g=599346&p=4149721
Fetters, L., Figueiredo, E., Keane-Miller, D., McSweeney, D., & Tsao, C.-C. (2004).
Critically Appraised Topics. Pediatric Physical Therapy, 16(1), 19–21. https://doi.org/10.1097/01.PEP.0000114628.87352.C8
Ingham-Broomfield, B. (2016). A nurses’ guide to the hierarchy of research designs and evidence.
Instructions for authors of editorial, focus, perspective, and commentary articles. (2018, January 31). Science | AAAS. https://www.sciencemag.org/journals/immunology/authors/editorial-focus-commentary-perspectives
Kendall, J. M. (2003). Designing a research project: Randomized controlled trials and their principles. Emergency Medicine Journal, 20(2), 164–168. https://doi.org/10.1136/emj.20.2.164
Shorten, A., & Shorten, B. (2013). What is meta-analysis? Evidence-Based Nursing, 16(1), 3-4. https://doi.org/10.1136/eb-2012-101118
Uman, L. S. (2011). Systematic Reviews and Meta-Analyses. Journal of the Canadian Academy of Child and Adolescent Psychiatry, 20(1), 57 -59.