Sloppy researchers beware. A new institute has you in its sights
“WHY most published research findings are false” is not, as the title of an academic paper, likely to win friends in the ivory tower. But it has certainly influenced people (including journalists at The Economist). The paper it introduced was published in 2005 by John Ioannidis, an epidemiologist who was then at the University of Ioannina, in Greece, and is now at Stanford. It exposed the ways, most notably the overinterpreting of statistical significance in studies with small sample sizes, that scientific findings can end up being irreproducible—or, as a layman might put it, wrong.
Dr Ioannidis has been waging war on sloppy science ever since, helping to develop a discipline called meta-research (ie, research about research). Later this month that battle will be institutionalised, with the launch of the Meta-Research Innovation Centre at Stanford.
METRICS, as the new laboratory is to be known for short, will connect enthusiasts of the nascent field in such corners of academia as medicine, statistics and epidemiology, with the aim of solidifying the young discipline. Dr Ioannidis and the lab’s co-founder, Steven Goodman, will (for this is, after all, science) organise conferences at which acolytes can meet in the world of atoms, rather than just online. They will create a “journal watch” to monitor scientific publishers’ work and to shame laggards into better behaviour. And they will spread the message to policymakers, governments and other interested parties, in an effort to stop them making decisions on the basis of flaky studies. All this in the name of the centre’s nerdishly valiant mission statement: “Identifying and minimising persistent threats to medical-research quality.”
The METRICS system
Irreproducibility is one such threat—so much so that there is an (admittedly tongue-in-cheek) publication called the Journal of Irreproducible Results. Some fields are making progress, though. In psychology, the Many Labs Replication Project, supported by the Centre for Open Science, an institute of the University of Virginia, has re-run 13 experiments about widely accepted theories. Only ten were validated. The centre has also launched what it calls the Cancer Biology Reproducibility Project, to look at 50 recent oncology studies.
Until now, however, according to Dr Ioannidis, no one has tried to find out whether such attempts at revalidation have actually had any impact on the credibility of research. METRICS will try to do this, and will make recommendations about how future work might be improved and better co-ordinated—for the study of reproducibility should, like any branch of science, be based on evidence of what works and what does not.
The Latest on: Bad science
via Google News
The Latest on: Bad science
- It’s Not Science Fictionon November 26, 2020 at 8:05 am
The prolific science-fiction writer Kim Stanley Robinson, who is at heart an optimist, opens his newest novel, The Ministry for the Future, with a long ...
- “Space Karen” Trends as Elon Musk Gets Owned for Bad Scienceon November 16, 2020 at 9:41 am
Musk being called out for bad science and being an insufferable git is pretty standard, yet also fun. But this one is extra great because Emma Bell is a scientist who knows what she’s talking about ...
- Disseminating Preliminary, Incomplete COVID-19 Vaccine Study Results by Press Release Represents Bad Scienceon November 9, 2020 at 8:02 am
Because of people like you, another world is possible. There are many battles to be won, but we will battle them together—all of us. Common Dreams is not your normal news site. We don't survive on ...
- Claims of Voter Fraud Supercharged by Bad Scienceon November 8, 2020 at 4:00 pm
During the 2016 primary season, Trump campaign staffer Matt Braynard had an unusual political strategy. Instead of targeting Republican base voters—the ones who show up for every election—he focused ...
- Trump’s Attacks on Climate Science Are Coming to Fruitionon November 2, 2020 at 12:50 am
“I’ve never seen anything like this before,” said one long-serving scientist who asked not to be identified for fear of reprisal. “We’re being asked to follow bad science.” ...
- How claims of voter fraud were supercharged by bad scienceon November 1, 2020 at 9:42 am
Published in 2014 by Jesse Richman, a political science professor at Old Dominion ... it is still susceptible to those operating in bad faith. I asked Richman earlier this summer if we should ...
- The pursuit of herd immunity is a folly – so who's funding this bad science?on October 18, 2020 at 11:21 am
There is no real scientific divide over this approach, because there is no science to justify its usage in the case of Covid-19. We know that when it comes to other coronaviruses, immunity is only ...
- Massaging minds with bad scienceon August 6, 2020 at 3:40 pm
So today’s column is a first. I want everyone to read Ben Goldacre’s book Bad Science. It should be in every school, and even if children are not taking science subjects (perhaps specifically ...
via Bing News