Systematic review and meta-analysis

[reading time 4 minutes]

Systematic reviews

The exponential proliferation of research studies published on indexed journals has forced scholars to elaborate methods to summarize the information by producing summary articles.

Amongst the summary articles the systematic reviews are of particular relevance thanks to the rigor of the methodological criteria that characterize them.

The elaboration of the concept of systematic review is attributed to Archie Cochrane (1909-1988), a British epidemiologist specialized in the cure of tuberculosis. Reflecting on the limited resources of public health care, which he thought would stay the same in the future, he considered his duty to commit himself in carefully evaluating the services provided while proving their efficacy.

Building on these thoughts, in 1993 the Cochrane Collaboration was founded, an international community operating in 45 countries which gathers more than ten thousand people organized into interdisciplinary groups. Each year the Cochrane publishes over two hundred systematic reviews, produced according to strict recommendations, which are made available in the CDSR database in order to promote decision-making informed by evidence in health care. The Cochrane also offers other resources, including training courses, documents and manuals for the conduct of the reviews.

A reliable systematic review can only be carried out by adhering to rigorous criteria, which require researchers to take into account a number of aspects, including, for example, the following:

• establishing clear objectives and consequently identifying the eligibility criteria to be followed in the studies included in the review;
• to adopt a rigorous and reproducible methodology;
• to carry out an accurate search of the “primary sources” available – that is to collect “systematically” all the studies published on the topic of interest, considering not only the white literature but also the gray one and other sources;
• to critically evaluate the validity of the results presented in the studies scrutinized, especially for what concerns the risk of bias;
• to present a summary and an interpretation compliant with the guidelines for the composition of this type of study, for example the PRISMA guidelines.

The systematic reviews can be useful also for a bibliographical research, since the researchers have already carried out the search work, have evaluated the studies and excluded the non eligible ones – therefore they offer “filtered” information.

Systematic reviews are very powerful means, however, they are particularly susceptible to some kinds of bias, in particular:

• the publication bias, ie, the possibility that the studies with “negative” results are not transmitted to the scientific community. Therefore it would be impossible to find them during the course of the initial systematic search of the sources in the literature.
• the selection bias, since the inclusion or exclusion of a study can heavily affect the outcomes of the meta-analysis.

Other weaknesses are the heterogeneity of the studies which can make the comparison of the outcomes problematic and the need of updating in the case of the publication of new studies on the same subject.

Meta-analysis

When the data of the considered studies are comparable in accordance with the methods of the statistical analysis then a meta-analysis can be carried out. This type of procedure allows the accumulation of the results obtained in the individual studies, with the advantage of obtaining an overview of the topic researched, based on a bigger sample size and therefore of greater power.

To mention an example, tens of thousands of deaths caused by sudden infant death syndrome (SIDS) could have been avoided had a meta-analysis been carried out in 1970 in order to cumulate the results of the studies already available at that time about the correct way to position the infants for sleeping . Instead this work was done in the ’90s¹. The campaign Back to sleep (now Safe to Sleep®) to reduce the risk of SIDS was launched in the USA in 1994.

The results of the meta-analysis are visualized in a so-called forest plot (or blobbogram), which provides numerous pieces of information at a glance. The diamond in the lower part of the graphic shows if the results are favorable or unfavorable to the examined intervention. The squares represent the single studies while their size indicates the size of the effect and their lateral whiskers indicate the confidence interval as they are directly proportional to the uncertainty. The vertical line which divides the graphic into two parts indicates a null effect (or no effect). When the horizontal line of a study crosses the vertical line of “no effect” the result, conventionally, cannot be considered statistically significant.

Systematic reviews and meta-analysis in the osteopathic research

In the field of osteopathy numerous systematic reviews have been carried out, many of which with meta-analysis (a search launched by PubMed in 2021 with the keyword “osteopathic manipulative treatment”, activating the filter to visualize only the systematic reviews and the meta-analysis, obtained more than 50 results).

For further in-depth analysis please refer to the Osteopedia’s page reporting the reviews of scientific articles and to the volume edited by Francesco Cerritelli and Diego Lanaro².

Bibliography

1. Gilbert R, Salanti G, Harden M, See S. Infant sleeping position and the sudden infant death syndrome: systematic review of observational studies and historical review of recommendations from 1940 to 2002. Int J Epidemiol. 2005 Aug;34(4):874-87.
2. Cerritelli F, Lanaro D. Elementi di ricerca in osteopatia e terapie manuali. Napoli: Edises, 2018.