What are systematic reviews and meta-analyses?

Systematic reviews can only provide reliable answers if the studies they are based on are searched for and selected very carefully. The individual steps needed before they can be published are usually quite complex.

• Research question: First of all, the researchers have to decide exactly what question they want to find the answer to. Which treatment should be looked at in which group of people, and what should it be compared with? What should be measured? This set of key questions is also referred to as the PICO framework. PICO stands for Population (patient group), Intervention (the treatment or diagnostic test under investigation), Control (comparison group) and Outcome (variable to be measured). The research question also determines which criteria to use when selecting studies to include in the review – for instance, only certain types of studies.
• Research: Once they know what they are looking for, the researchers have to search as thoroughly and comprehensively as possible for all the studies that might help answer the question. This can easily add up to as many as several hundred studies. Searches for studies are usually done in international databases. Most study results are published online and in English. The relevant information is filtered out using sophisticated methods.
  The researchers often try to find any unpublished data by contacting and asking other scientists, looking through lists of sources used in other publications, and sometimes even by looking at conference transcripts. One big problem is that some studies are never published. Compared to studies in which treatments are found to have positive outcomes, studies that don’t find any benefits are often published later or never published at all. As a result, the studies that are found and included in reviews might make a treatment seem better than it really is. This kind of systematic bias is also known as “publication bias.”
• Selection: The suitability of every study that is found has to be checked using very specific pre-defined criteria. Studies that do not fulfill the criteria are not included in the review. The suitability of a study is usually assessed by at least two researchers who go through all the studies separately and then compare and discuss their conclusions. This is done in order to try to avoid including unsuitable studies in the review.
• Assessment: The studies that fulfill all the inclusion criteria are carefully assessed. This analysis should provide a comprehensive overview of what is known, and what isn’t known, about the topic in question.
• Peer review: The researchers provide a detailed report of the steps they took, their research methods and what they found. A draft version is critically assessed and commented on by experts. This is called "peer reviewing."
• Publication: If the systematic review “passes” the peer review, it can be published in scientific journals and relevant databases. One important source of systematic reviews is the “Cochrane Library” database. It is run by the Cochrane Collaboration – an international network of researchers who have specialized in producing systematic reviews.
• Keeping the information up-to-date: In order to stay up-to-date, systematic reviews must be updated regularly.

Sometimes the results of all of the studies found and included in a systematic review can be summarized and expressed as an overall result. This is known as a meta-analysis. The overall outcome of the studies is often more conclusive than the results of individual studies.

But it only makes sense to do a meta-analysis if the results of the individual studies are fairly similar (homogeneous). If there are big differences between the results, there are likely to be important differences between the studies. These should be looked at more closely. It is then sometimes possible to split the participants into smaller subgroups and summarize the results separately for each subgroup.