Every second we are bombarded with claims and counter-claims on every possible topic from climate change to cancer treatments.

When some scientific findings report one result, and others the opposite result, this can leave us at best confused, and at worst, reaching the wrong conclusions and making the wrong decisions.

This can affect our health, our environment, and our society, with sometimes very poor outcomes.

“The findings of a single scientific study may seem to tell us that a particular phenomenon or effect exists”, explains Prof Julia Koricheva, an expert in meta-analysis from Royal Holloway University of London, who has co-authored a paper on the subject which appears in Nature today.

“But any individual study is subject to potential limitations and biases. Particular outcomes may result by chance, especially when the number of people or samples studied are small, and various biases may tip the conclusions.

“Fortunately, over the last few decades statistical methods have been developed to more accurately and objectively summarize the results of multiple studies on the same topic.”

How has the power of meta-analysis been used to resolve problems, discover general patterns, and get closer to what’s true across a wide range of scientific areas?

In the paper, experts from Royal Holloway University, Newcastle University, UK, Stony Brook University (USA) and the University of New South Wales (Australia) show how meta-analysis is essential for progress in science and medicine.

Meta-analysis is the quantitative, scientific synthesis of research results, and the set of scientific and statistical tools which underpin evidence-based research.

First introduced in the 1970s, it has had a revolutionary impact on many scientific fields, helped to establish evidence-based practice and sometimes resolve seemingly contradictory results among different studies. Meta-analysis has been a powerful and less biased means of confirming – or disproving – assumed wisdom.

Forty years after its introduction, meta-analysis is widely used in medicine, many social sciences, and many other scientific disciplines, including ecology and conservation.

“By bringing the scientific method into the synthesis of results across independent studies, we can ask—and answer—questions we could never hope to resolve before, while new methods in meta-analysis open doors to the resolution of seemingly intractable problems” Dr Nakagawa, New South Wales University, remarks.

Professor Gurevitch, of Stony Brook University adds, “Meta-analysis is the grandmother of Big Data and Open Science and has been essential for progress in many fields. In medicine, we have been able to compare the effectiveness of treatments that have never been compared in any one study, and we can save lives by understanding what works long before it is apparent in any single experiment. In ecology and evolutionary biology, we can evaluate patterns across wide expanses of space and a diversity of animals and plants that are more than any one researcher could study in several lifetimes.

"In conservation, we may find out what measures really work best to achieve the goals of preserving ecosystems.”

But as the number of published meta-analyses increases exponentially, not everyone is happy. As the popularity of the meta-analysis approach has grown, some researchers have claimed that scientists conducting meta-analyses are “research parasites” who do not generate data themselves, but use those collected by others to publish high profile papers.

Other criticisms of meta-analysis focus on the sometimes poor methodological quality of particular papers, and of “combining apples and oranges.” The authors of this paper argue that higher standards and more consistent guidelines for meta-analysis publications and improved training in these methods can go a long way to addressing these criticisms.

Dr Gavin Stewart, of Newcastle University said: “We need to be using meta-analysis across the board to solve the big challenges facing society, such as climate change, education and future food security. If we can underpin these decisions with trustworthy data, looking at the evidence of multiple of studies and not just individual and possibly unreliable claims, then we can start to use accurate scientific evidence as the basis for best practices.”

More information can be found here about Royal Holloway's bio sciences.