Sloppy researchers beware. A new institute has you in its sights
“WHY most published research findings are false” is not, as the title of an academic paper, likely to win friends in the ivory tower. But it has certainly influenced people (including journalists at The Economist). The paper it introduced was published in 2005 by John Ioannidis, an epidemiologist who was then at the University of Ioannina, in Greece, and is now at Stanford. It exposed the ways, most notably the overinterpreting of statistical significance in studies with small sample sizes, that scientific findings can end up being irreproducible—or, as a layman might put it, wrong.
Dr Ioannidis has been waging war on sloppy science ever since, helping to develop a discipline called meta-research (ie, research about research). Later this month that battle will be institutionalised, with the launch of the Meta-Research Innovation Centre at Stanford.
METRICS, as the new laboratory is to be known for short, will connect enthusiasts of the nascent field in such corners of academia as medicine, statistics and epidemiology, with the aim of solidifying the young discipline. Dr Ioannidis and the lab’s co-founder, Steven Goodman, will (for this is, after all, science) organise conferences at which acolytes can meet in the world of atoms, rather than just online. They will create a “journal watch” to monitor scientific publishers’ work and to shame laggards into better behaviour. And they will spread the message to policymakers, governments and other interested parties, in an effort to stop them making decisions on the basis of flaky studies. All this in the name of the centre’s nerdishly valiant mission statement: “Identifying and minimising persistent threats to medical-research quality.”
The METRICS system
Irreproducibility is one such threat—so much so that there is an (admittedly tongue-in-cheek) publication called the Journal of Irreproducible Results. Some fields are making progress, though. In psychology, the Many Labs Replication Project, supported by the Centre for Open Science, an institute of the University of Virginia, has re-run 13 experiments about widely accepted theories. Only ten were validated. The centre has also launched what it calls the Cancer Biology Reproducibility Project, to look at 50 recent oncology studies.
Until now, however, according to Dr Ioannidis, no one has tried to find out whether such attempts at revalidation have actually had any impact on the credibility of research. METRICS will try to do this, and will make recommendations about how future work might be improved and better co-ordinated—for the study of reproducibility should, like any branch of science, be based on evidence of what works and what does not.
The Latest on: Bad science
via Google News
The Latest on: Bad science
- Not All Science Is Settled - How Journalists Can Convey Coronavirus Uncertainty Without Losing Truston March 24, 2020 at 10:51 am
The New York Times could practically co-brand Center for Science in the Public Interest when it comes to suspect food claims or Natural Resources Defense Council when it comes to chemicals. Bad ...
- Theranos CEO Elizabeth Holmes says setbacks tied to FDA, not bad scienceon March 23, 2020 at 10:42 am
The company founder sits down at Fortune's Global Forum to discuss the controversy around her company.
- When will Taxpayers have enough of Spending on Bad (Irrelevant) Science?on February 29, 2020 at 4:00 pm
So the question is other than wasted taxpayer money, what is the harm? That is an important question because the researchers didn’t appear to publish bad science (I trust the results are valid and ...
- Health Supplement Retailers Make Illegal Claims, Finds Bad Science Watch Studyon February 27, 2020 at 7:01 am
A study conducted by the Canadian science advocacy group Bad Science Watch shows it’s far more often than you might think. In fact, the results are alarming. Bad Science Watch looked ...
- How bad science infiltrates the publicon February 26, 2020 at 11:23 am
What I've done wrong 2. What the big deal is and why you should care Side story: How was I influenced by bad science Additionally, the author was selective in picking the research to present in ...
- The Bane of Billion Dollar Bad Scienceon February 18, 2020 at 3:31 am
The Bane of Billion Dollar Bad Science Rigor Mortis by Richard Harris is a real black pill (a grim truth) for those who are optimistic about medical science delivering us cures to disease.
- The verdict is in: Courtrooms seldom overrule bad scienceon February 17, 2020 at 8:14 am
In real-world courtrooms, however, the quality of scientific testimony can vary wildly, making it difficult for judges and juries to distinguish between solid research and so-called junk science. This ...
- The verdict is in: Courtrooms seldom overrule bad scienceon February 14, 2020 at 4:00 pm
In real-world courtrooms, however, the quality of scientific testimony can vary wildly, making it difficult for judges and juries to distinguish between solid research and so-called junk science.
via Bing News