Systematic Review Overview: 6. Conduct a Quality Appraisal of Included Studies

In order to ensure that the conclusions of your systematic review are based on the highest quality of evidence available, it is important to appraise the included studies in a standardized and explicit manner. Using a systematic procedure to appraise these studies will make certain that only relevant studies with an acceptable level of quality are included. The quality of the studies used is a measure of the strength of the evidence for the recommendations made in a systematic review.

When conducting the quality appraisal of included studies there should be a minimum of two independent reviewers. The appraisal process includes the assessment of internal and external validity as well as determining biases affecting methodological quality. Validity answers the question of how close the results are to the truth.

When critically appraising studies, it is often helpful to use a checklist of necessary elements for a quality study. There are various checklists available depending upon the type of study being assessed.

Here are some key issues* to consider when appraising various studies.

Key Issues in Appraising Therapy Studies:

• Randomization and concealed allocation
• Follow-up of all patients (ideally 80% or better)
• Blinding (concealment) of patients, clinicians and study personnel to the treatment being provided
• Intention to Treat analysis
• Baseline similarities between groups (established at the start of the trial)

Key Issues in Appraising Diagnostic Studies:

• Independent blind comparison with a gold standard
• Appropriate spectrum of patients
• All patients receive both tests

Key Issues in Appraising Prognosis Studies:

• Well-defined sample of patients
• Similar prognostic factors
• Follow-up
• Objective outcome criteria

Key Issues for Etiology/Harm Studies:

• Similarity of comparison groups
• Outcomes and exposure measured the same for both groups
• Follow-up of sufficient length

*Key issues checklists are adapted from “EMB and the Medical Librarian”. University of North Carolina School of Information and Library Science. Unpublished course material, 2008. Instructors Connie Schardt and Angela Myatt.

GRADE: Grading of Recommendations, Assessment, Development, and Evaluation. Journal of Clinical Epidemiology series.

"...[the] objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations."