Background
A systematic review is an evidence-based research approach that collates existing empirical evidence, guided by eligibility criteria, to answer a research question. Chapman (2021) states that systematic reviews are highly critical in research as they synthesize the state of knowledge in a given field of study. The findings help generate or evaluate theories, identify future research priorities and problems, answer questions that cannot be satisfied by a single study, and separate the issues in primary research that need rectification. Systematic reviews are distinguished by characteristics that ensure accurate, concise, reliable, thorough, and understandable summaries of the best available information on a given issue, supporting research conclusions.
Protocols
Researchers should develop protocols before conducting a systematic review. They define the scope of the systematic review and describe the methodology and approach adopted throughout the study, which provides the necessary direction. According to Owen (2021), a systematic review’s defined scope and method make it easier for those who are reading or reviewing the article to rapidly gauge the review’s rigor and validity. Protocols act as a guide for carrying out a systematic review in an organized, open, and reproducible manner while reducing bias and error.
One of the most preferred protocol resources in systematic reviews is the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). According to Xiao and Watson (2019), PRISMA was first published in 2009. The framework offers a structured outline for authors to report on the purpose of the review transparently, the methods used, and the findings. It helps ensure that the review process is transparent, comprehensive, and reproducible.
Xiao and Watson (2019) further indicate that the tool underwent significant improvements in its first decade of production, and a better version was subsequently offered in 2020. The latest version provides new reporting guidelines that reflect advances in methods to identify, select, appraise, and synthesize studies. PRISMA generally offers a more comprehensive structure for reporting systematic reviews, as outlined in a checklist containing 21 items. It is mainly applied during the reporting stage of a systematic review. However, it is recommended that it should be observed from the planning phase to the final reporting of the systematic review.
Another applicable protocol in systematic reviews is the one proposed by the Cochrane Collaboration, known as the Cochrane protocol. The International non-profit body, founded in 1993, seeks to ensure that researchers produce high-quality, evidence-based systematic reviews and other synthesized research evidence to inform better decision-making. This is attained through the production of comprehensive systematic reviews that rigorously assess the obtainable evidence on a specific research topic.
The body provides guidelines to ensure reviews are conducted using transparent and well-defined methodologies. The Campbell Collaboration is a protocol resource established in 1999, encouraged by the success and methodology of the Cochrane Collaboration in healthcare (Chapman, 2021). The global research web produces systematic reviews in social and economic policy domains.
Time and Team
A second key feature of systematic reviews is that they require a long period to conduct. Based on the guidelines proposed by the Cochrane Collaboration, a systematic review typically requires an average of 18 months to complete, as researchers must thoroughly search existing literature to determine if a particular solution is available (Xiao & Watson, 2019). The time is broken down by activity, starting from, and each spans a period ranging from one month to over seven months.
Given the scale of activities involved in a systematic review, it is clear that one person cannot complete the exercise alone. It requires the involvement of a team, preferably drawn from subject matter experts, to clarify issues related to the topic. The project leader undertakes the roles of coordinating the review and writing the final report. Reviewers can be included in the systematic review to read complete texts and abstracts. Librarians can help create thorough search techniques, and statisticians may be required to assist with data analysis.
Clearly Defined Question(s)
A systematic review has to be guided by a clear research question (s), through which it is possible to search relevant information from all available published and unpublished literature. In developing a clear research question and rationale for a systematic review, the PICO framework is applied as it helps to identify key concepts applicable to the review. Owens (2021) indicates that the framework is mainly applied in healthcare research. P stands for the patient population or problem as defined by the characteristics. I stands for the primary intervention or treatment to be applied. C represents comparison to another intervention, and O stands for the outcome, which is the effect of the intervention/treatment and how it will be measured.
Different variations of PICO can be applied in a systematic review, including PICOS (S = study design), PICOT (T = time), and PICOC (C = context). Several other frameworks that can be applied in the research include:
- PEO (Population, Exposure, Outcomes) for questions about risks or causes
- SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) for qualitative research
- ECLIPSE (Expectation, Client group, Location, Impact, Professionals, Service) and SPICE (Setting, Population or Perspective, Intervention, Comparison, Evaluation) for evaluations of interventions or policies.
Rigorous and Systematic Search of the Literature, and Inclusion & Exclusion Criteria
Searching for the desired information is a critical feature of a systematic review, as it seeks to uncover the evidence base for the research. A thorough systematic review must be conducted based on an exhaustive, objective, understandable, and repeatable search approach. Chapman (2021) suggests several processes, beginning with the formulation of the search strategy following preliminary exploratory searching. The technique helps the researcher locate pertinent publications. By clicking on links to references and publications cited in the paper, the articles can then assist in identifying key search phrases and locating related papers. The most crucial concepts to include in the search are guided by the search framework and research question.
The second stage is to ponder search filters and text mining. Text mining is a technique that assists researchers in identifying search terms by utilizing software to scan text for recurring terms or phrases. There are numerous free text mining tools available, like PubMed’s PubReMiner. Search filters, also known as hedges, are tried-and-true search tactics designed to locate specific items in a particular database (Xiao & Watson, 2019). They are quite helpful and aim for sensitivity and precision in the search. Some examples of search filters include CareSearch, which is used in PubMed searches. They are mainly helpful in healthcare research, particularly in areas such as palliative care and related topics.
Choosing where to seek and conducting searches are equally important. The most practical method to use in a systematic review is a database search. The subject area and the exact research issue influence the databases that are selected. It is recommended that the search be limited to the most comprehensive and pertinent databases in the area of study, or to use individual databases rather than databases in packages. Where they are available, using thesaurus phrases and keywords is recommended.
There are different databases for different disciplines; for instance, in health, the common databases are MEDLINE and PubMed. Owens (2021) states that it is also crucial for the researcher to document the review and update the search strategy to ensure it is reproducible and verifiable. This entails recording the name of the database, the years covered, the date of the search, and the exact search strategy, including all the terms used in the search. The documentation can be done by ensuring that the search history from each database is recorded.
The inclusion and exclusion criteria are crucial to the literature search. The parameters can be used to narrow the search as much as possible inside each database through a process known as filtering or refining. According to Xiao and Watson (2019), the criteria help in ensuring that articles based on title-and-abstract screening are rejected. In the methods section of the systematic review report, the information concerning the inclusion and exclusion criteria is typically presented as a paragraph or table. Language, age groups, time (date of publishing), publication type, study design, and location or environment are some of the often utilized criteria.
Critical Appraisal of Included Studies
The risk of bias for included studies is evaluated in the critical assessment. It is the process of thoroughly and methodically evaluating research to determine its reliability, as well as its worth and significance in a given context. When determining the trustworthiness of research, bias must be minimized, and the impact and significance of the findings are used to assess the value of the research. Owens (2021) indicates that the applicability of the research’s findings to the population, patient, or problem determines the study’s relevance. The included research can be evaluated using various techniques, including RAMMbo and AMSTAR. RAMMbo stands for:
- Recruitment of the participants, representative of the target population or the sample size.
- Allocation – trial randomization.
- Maintenance of the status of the study and control group throughout the trial. It inquires into whether the groups were treated the same way, other than in the intervention.
- Measurement (Blinding, Objective measures) relates to whether the outcomes were measured objectively and whether the participants were blinded to the intervention. It responds to the query of whether bias was eliminated as much as possible.
The AMSTAR measurement tool is used to evaluate the quality of systematic review methodology; it does not assess the content or therapeutic significance of the systematic review. Systematic reviews are subject to concerns, including variations in quality and a lack of empirical validation. The large number of systematic reviews at their disposal, which provide high-quality, pertinent, and well-supported literature, has been examined by decision-makers to determine how they can be used most effectively. AMSTAR helps evaluate systematic reviews critically to establish their validity and reliability. It provides a methodical and systematic approach to assessing the level of evidence synthesis in medical studies.
A higher score indicates a higher methodological quality of the systematic review, according to the AMSTAR rating methodology, which contains 11 important components (Owens, 2021). It is also employed as a training tool for systematic reviews, a manual for conducting reviews, and a tool for developing and evaluating reviews. To help consumers distinguish between systematic reviews, AMSTAR aims to encourage the creation of high-quality reviews and to provide valid, trustworthy, and usable instruments. Focusing on methodological excellence and professional consensus helps achieve these objectives.
Data Extraction and Management
Once the articles that need to be used in the review have been identified, the relevant data from those studies will be collected. In the final review report, data extraction is typically presented in tables and must be conducted in a structured and methodical manner. Several analysis tools can be used for a more detailed systematic review, such as NVivo, PSPP, and JASP (Chapman, 2021).
A variety of evidence can be evaluated through systematic reviews, including qualitative, quantitative, and mixed evidence. For each form of evidence, the right synthesis techniques should be utilized. Key conclusions from each of the collected articles should be incorporated into the overall summary of a critical overview. The researcher should take into account the methodological soundness and other essential aspects of each study (such as the demographic, sample size, or setting in which it was conducted).
Analysis and Interpretation of Results
The best estimate of any real effect should be provided by a systematic review that is carried out properly. This is accomplished through a thorough and organized analysis and the interpretation of results. The impact of the intervention can be fully summarized using a meta-analysis of quantitative data, whereas a thematic analysis of qualitative studies identifies recurring themes. To investigate heterogeneity and judge the reliability of results, subgroup and sensitivity analyses may be performed.
According to Xiao and Watson (2019), the quality, likelihood of bias, and relevance of the studies are taken into consideration when interpreting the synthesis data. The systematic review’s conclusions highlight the advantages and disadvantages of the evidence, guiding future practice, policy, and research. The findings are then independently and objectively presented in a report.
References
Chapman, K. (2021). Characteristics of systematic reviews in the social sciences. The Journal of Academic Librarianship, 47(5), 102396.
Owens, J. K. (2021). Systematic reviews: Brief overview of methods, limitations, and resources. Nurse Author & Editor, 31(3-4), 69–72.
Xiao, Y., & Watson, M. (2019). Guidance on conducting a systematic literature review. Journal of Planning Education and Research, 39(1), 93–112.