Scientists like to think of science as self-correcting. To an alarming degree, it is not
“I SEE a train wreck looming,” warned Daniel Kahneman, an eminent psychologist, in an open letter last year. The premonition concerned research on a phenomenon known as “priming”. Priming studies suggest that decisions can be influenced by apparently irrelevant actions or events that took place just before the cusp of choice. They have been a boom area in psychology over the past decade, and some of their insights have already made it out of the lab and into the toolkits of policy wonks keen on “nudging” the populace.
Dr Kahneman and a growing number of his colleagues fear that a lot of this priming research is poorly founded. Over the past few years various researchers have made systematic attempts to replicate some of the more widely cited priming experiments. Many of these replications have failed. In April, for instance, a paper in PLoS ONE, a journal, reported that nine separate experiments had not managed to reproduce the results of a famous study from 1998 purporting to show that thinking about a professor before taking an intelligence test leads to a higher score than imagining a football hooligan.
The idea that the same experiments always get the same results, no matter who performs them, is one of the cornerstones of science’s claim to objective truth. If a systematic campaign of replication does not lead to the same results, then either the original research is flawed (as the replicators claim) or the replications are (as many of the original researchers on priming contend). Either way, something is awry.
To err is all too common
It is tempting to see the priming fracas as an isolated case in an area of science—psychology—easily marginalised as soft and wayward. But irreproducibility is much more widespread. A few years ago scientists at Amgen, an American drug company, tried to replicate 53 studies that they considered landmarks in the basic science of cancer, often co-operating closely with the original researchers to ensure that their experimental technique matched the one used first time round. According to a piece they wrote last year in Nature, a leading scientific journal, they were able to reproduce the original results in just six. Months earlier Florian Prinz and his colleagues at Bayer HealthCare, a German pharmaceutical giant, reported in Nature Reviews Drug Discovery, a sister journal, that they had successfully reproduced the published results in just a quarter of 67 seminal studies.
The governments of the OECD, a club of mostly rich countries, spent $59 billion on biomedical research in 2012, nearly double the figure in 2000. One of the justifications for this is that basic-science results provided by governments form the basis for private drug-development work. If companies cannot rely on academic research, that reasoning breaks down. When an official at America’s National Institutes of Health (NIH) reckons, despairingly, that researchers would find it hard to reproduce at least three-quarters of all published biomedical findings, the public part of the process seems to have failed.
Academic scientists readily acknowledge that they often get things wrong. But they also hold fast to the idea that these errors get corrected over time as other scientists try to take the work further. Evidence that many more dodgy results are published than are subsequently corrected or withdrawn calls that much-vaunted capacity for self-correction into question. There are errors in a lot more of the scientific papers being published, written about and acted on than anyone would normally suppose, or like to think.
Various factors contribute to the problem. Statistical mistakes are widespread. The peer reviewers who evaluate papers before journals commit to publishing them are much worse at spotting mistakes than they or others appreciate. Professional pressure, competition and ambition push scientists to publish more quickly than would be wise. A career structure which lays great stress on publishing copious papers exacerbates all these problems. “There is no cost to getting things wrong,” says Brian Nosek, a psychologist at the University of Virginia who has taken an interest in his discipline’s persistent errors. “The cost is not getting them published.”
First, the statistics, which if perhaps off-putting are quite crucial.
Go deeper with Bing News on:
- Waste to Energy Market to Reach $50.1 Billion by 2027: Allied Market Researchon September 7, 2020 at 6:13 am
According to the report published, the global waste to energy industry was estimated at $35.1 billion in 2019, and ...
- Reliability of fNIRS for noninvasive monitoring of brain function and emotion in sheepon September 7, 2020 at 2:24 am
The aim of this work was to critically assess if functional near infrared spectroscopy (fNIRS) can be profitably used as a tool for noninvasive recording of brain functions and emotions in sheep. We ...
- On the reliability of highly magnified micrographs for structural analysis in materials scienceon September 7, 2020 at 2:13 am
Highly magnified micrographs are part of the majority of publications in materials science and related fields. They are often the basis for discussions and far-reaching conclusions on the nature of ...
- Slow, unreliable labor force data have consequences for out-of-work Americans and the economy.on September 3, 2020 at 4:22 am
Governments at all levels are trying to make decisions about how to reduce the cost of joblessness – including whether to provide supplemental unemployment benefits. And they are doing this without ...
- Abaco Announces $1.7M Win from European Astrophysics Research Instituteon September 2, 2020 at 11:57 am
Abaco Systems announced that it has won orders from a major European astrophysics research institute that will see ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
Science as self-correcting
- Liberia: Religiosity and Failed African Stateson September 1, 2020 at 3:18 am
The greatest tragedy of all African states is the introduction of foreign gods and languages, cemented by tribes, decimated by education without interrogation or intelligence.
- South Kingstown Election Profile: Cadence Hansenon August 25, 2020 at 5:08 pm
Cadence Hansen is running for the school committee in South Kingstown. The 41-year-old Democrat is a newcomer to elected office. Hansen and her husband, Jesse, have two children, Isaac, 6, and Clara, ...
- Q&A with Dr. Fauci: ‘We will get out of this and we will return to normal. Don’t despair.’on August 21, 2020 at 5:00 am
The nation’s top infectious disease specialist weighs in on school safety, voting, harassment and Tucker Carlson.
- Anthony Fauci gives Matthew McConaughey true directives in coronavirus chaton August 13, 2020 at 5:54 pm
For once, it was nice to have a movie star on hand in a national emergency. The U.S. is grappling with misinformation about the coronavirus pandemic.
- The 50 Best Things to Watch on Disney+ Right Nowon July 24, 2020 at 1:21 am
The Disney streaming platform has hundreds of movie and TV titles, drawing from its own deep reservoir classics and from Star Wars, Marvel and more. These are our favorites.