Now that you have finished screening, you need to critically appraise your selected studies for quality and risk of bias. It is an important process in the review as it assesses your included studies for trustworthiness, value and relevance. Depending on who you are drafting your review for (Cochrane, JBI) there could be existing guidelines and standards that you need to follow when critically appraising the studies. The types of studies included in the review will also dictate which checklist could be used for the appraisal process. Check our list of Appraisal Tools for more information.
Things to consider when appraising the studies:
Reporting levels of evidence is also an important part of critical appraisal.
Meta Analysis - systematic review that uses quantitative methods to synthesize and summarize the results.
Systematic Review - summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.
Randomised Controlled Trial - Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.
Cohort Study - Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.
Case Control Study - study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.
Case Series - report on a series of patients with an outcome of interest. No control group is involved. (Definitions from Centre for Evidence Based Medicine)
Systematic review and bias tools
The below critical appraisal tools (CATs) can be used to assess the selected studies:
Reporting tools
Next you need to extract necessary data from your included studies. This will help you assess the validity of the included studies and will make analysis, synthesis and interpretation easier. There are guides that outline how to conduct appropriate data extraction and depending on what type of review you are conducting and where you are publishing you might need to follow specific guidelines for data extraction.
Things to remember when extracting data:
If you have screened your review in Covidence it has data extraction capabilities. Please see Introduction video below:
Guidelines & Standards
Once you have extracted your data, you need to synthesise your data. This involves bringing together your findings from the included studies to describe and explain what your systematic review has found.
Meta-analysis
One way to synthesise data is by using meta-analysis. This approach requires the studies to be similar or contain qualitative data. A meta-analysis is a statistical combination of the results from each included study to create one large study. A forest plot or meta-view is an excellent way to present the synthesised data.
Meta-synthesis (Narrative or descriptive synthesis)
If the studies are not similar you will need to synthesise the data using a narrative or descriptive synthesis. This type of synthesis is more subjective and provides an overall view of the included studies. As narrative or descriptive synthesis has no standard process, you could use a guide or framework to consider quality, quantity, consistency, impact and applicability of the evidence.
Forest plots and other methods
Tables and figures can help help to present included studies and their findings in a clear way. A ‘Findings Summary’ table provides key information regarding the quality of evidence, the effect and outcomes of the included studies. Forest plots are often used to illustrate results of individual studies.