Introducing concept to undergrads could lead to more transparency in science
The ability to duplicate an experiment and its results is a central tenet of the scientific method, but recent research has shown an alarming number of peer-reviewed papers are irreproducible.
A team of math and statistics professors has proposed a way to address one root of that problem by teaching reproducibility to aspiring scientists, using software that makes the concept feel logical rather than cumbersome.
Researchers from Smith College, Duke University and Amherst College looked at how introductory statistics students responded to a curriculum modified to stress reproducibility. Their work is detailed in a paper published Feb. 25 in the journal Technological Innovations in Statistics Education.
In 2013, on the heels of several retraction scandals and studies showing reproducibility ratesas low as 10 percent for peer-reviewed articles, the prominent scientific journal Nature dedicated a special issue to the concerns over irreproducibility.
Nature’s editors announced measures to address the problem in its own pages, and encouraged the science community and funders to direct their attention to better training of young scientists.
“Too few biologists receive adequate training in statistics and other quantitative aspects of their subject,” the editors wrote. “Mentoring of young scientists on matters of rigour and transparency is inconsistent at best.”
The authors of the present study thus looked to their own classrooms for ways to incorporate the idea of reproducibility.
“Reproducing a scientific study usually has two components: reproducing the experiment, and reproducing the analysis,” said Ben Baumer, visiting assistant professor of math and statistics at Smith College. “As statistics instructors, we wanted to emphasize the latter to our students.”
The grade school maxim to “show your work” doesn’t hold in the average introductory statistics class, said Mine Cetinkaya-Rundel, assistant professor of the practice in the Duke statistics department. In a typical workflow, a college-level statistics student will perform data analysis in one software package, but transfer the results into something better suited to presentation, like Microsoft Word or Microsoft PowerPoint.
Though standard, this workflow divorces the raw data and analysis from the final results, making it difficult for students to retrace their steps. The process can give rise to errors, and in many cases, the authors write, “the copy-and-paste paradigm enables, and even encourages, selective reporting.”
“Usually, a data analysis report, even a published paper, isn’t going to include the code,” Cetinkaya-Rundel said. “But at the intro level, where this is the first time students are exposed to this workflow, it helps to keep intact both the final results and the code used to generate them.”
Enter R Markdown, a statistical package that integrates seamlessly with the programming language R. The team chose R Markdown for its ease of use — students wouldn’t have to learn a new computer syntax — and because it combines the raw data, computing and written analysis into one HTML document. The researchers hoped a single HTML file would give students a start-to-finish understanding of assignments, as well as make studying and grading easier.
The study introduced R Markdown to 417 introductory statistics students (272 from Duke University, 145 from Smith College) during the 2012-2013 school year. Instructors emphasized the lesson of reproducibility throughout each course and surveyed 70 students about their experience using R Markdown for homework assignments.
The Latest on: Scientific Reproducibility
via Google News
The Latest on: Scientific Reproducibility
- Process Spectroscopy Market - Global Industry Trends, Statistics, Competition Strategies and Industry Share by 2026on February 13, 2020 at 4:51 am
Top players in the Process Spectroscopy Market include ABB, Thermo Fisher Scientific, Agilent Technologies Ltd., Yokogawa, Danaher Corporation, Bruker, HORIBA, Shimadzu Corporation, BUCHI Labortechnik ...
- Irreproducibility is not a sign of failure, but an inspiration for fresh ideason February 12, 2020 at 4:03 pm
Everyone’s talking about reproducibility — or at least they are in the biomedical and social sciences ... If that were to happen — although physicists think it unlikely — it would be a good example of ...
- Geography and Environment Research Transparency and Reproducibility Training - Grantham Research Institute on climate change and the environmenton February 10, 2020 at 4:00 pm
The Geography & Environment Department is pleased to host you for the first Research Transparency and Reproducibility Training at the ... sustainable impacts on scientific norms and practices as ...
- IOP Publishing strengthens commitment to open scienceon February 7, 2020 at 8:40 pm
It is how we disseminate high-quality research, increase its visibility and encourage reproducibility and ... strengths is that we're embedded in the scientific communities we serve, and this ...
- Video Streamingon February 3, 2020 at 4:00 pm
Articles consist of high-quality video demonstrations and detailed text protocols which facilitate scientific reproducibility and productivity. The scope of the journal includes novel techniques, ...
- Problems in social science are being used to discredit climate scienceon January 28, 2020 at 12:18 pm
A conference in California next week says it aims to make scientific studies more reliable ... The conference’s programme focuses on the reproducibility crisis – the claim that science has an ...
- Software searches out reproducibility issues in scientific paperson January 22, 2020 at 10:09 am
In recent years, problems with identifying the antibodies used in research have been highlighted as a factor in the lack of reproducibility in the academic literature. “It is shocking that so ...
- Reproducibility Project Dismisses Landmark Cancer Studies Without Investigationon January 12, 2020 at 4:00 pm
February 3, 2017 | Getting scientific experiments right needs dedication ... Most scientists, especially those in biology, have probably experienced the same. The Reproducibility Project first made ...
- Advisory Boardon June 5, 2019 at 10:03 am
Stan Lazic Stan Lazic is Co-founder and Chief Scientific Officer at Prioris.ai Inc. and was ... include the design and analysis of biological experiments, improving the reproducibility of biomedical ...
- CCTS Concludes First Kaizen Game on Scientific Reproducibilityon July 30, 2018 at 1:16 pm
To help young researchers meet an impending NIH policy requiring formal training in scientific rigor and reproducibility, the CCTS launched a web-based quiz game using the Kaizen platform. To beta ...
via Bing News