Analysing search results
Once the search results are obtained the analysis phase of the systematic review commences where each paper must be judged on validity and quality. The decisions on which papers to include and exclude are based on the eligibility criteria specified in the protocol.
Best Practice Tip
It is recommended that you use the PRISMA flow diagram to record the number of search results from each database and the changes to those numbers as the screening process is conducted. This diagram should be included in the reported findings of the systematic review.
What is PRISMA?
PRISMA is the Preferred Reporting Items for Systematic reviews and Meta-Analyses. PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.
Screening
The screening process of a systematic review will usually involve more than one reviewer, with the reviewers assessing each paper against the eligibility criteria for inclusion or exclusion from the review as defined in the protocol.
Each reviewer will need to provide a rationale for how the papers match the eligibility criteria or have some value to the systematic review. This is done separately by each reviewer to ensure minimal bias. The results are then compared.
The screening is done incrementally; first, the titles and abstracts of each paper is assessed against the eligibility criteria. Agreement must be reached by all reviewers on which papers are then to be assessed in the second phase. The same process is repeated with the screening of the full-text of papers against the eligibility criteria. The remaining papers are those that will contribute to the systematic review.
Critical appraisal
The critical appraisal process examines the validity and applicability of the studies. It is important to critically evaluate the literature to:
- assess benefits and strengths of the research against flaws and weaknesses
- decide whether studies have been undertaken in a way that makes their findings reliable
- make sense of the results
- know what these results mean in the context of the clinical decision being made
- assess the usefulness of the evidence for clinical decisions
There are several checklists available to assist in the process of determining the quality of the studies. A checklist should be chosen to assess sources of bias that are likely to affect results in relation to the research question.
Activity
Take a look at the following sites for examples of critical appraisal tools that can be used to assess quality, validity, and bias of papers.
- CASP (Critical Appraisal Skills Programme) Checklists
- Joanna Briggs Critical Appraisal Tools
- Risk of bias assessment tools from the University of Bristol
