International scientists are challenging their colleagues to make artificial intelligence (AI) research more transparent and reproducible to accelerate the impact of their findings for cancer patients.
In an article published in Nature on Oct. 14, 2020, scientists at Princess Margaret Cancer Centre, University of Toronto, Stanford University, Johns Hopkins University, Harvard University School of Public Health, Massachusetts Institute of Technology, and others, challenge scientific journals to hold computational researchers to higher standards of transparency, and call for their colleagues to share their code, models and computational environments in publications.
“Scientific progress depends on the ability of researchers to scrutinize the results of a study and reproduce the main finding to learn from,” says Dr. Benjamin Haibe-Kains, Senior Scientist at the Princess Margaret and first author of the article. “But in computational research, it’s not yet a widespread criterion for the details of an AI study to be fully accessible. This is detrimental to our progress.”
The authors voiced their concern about the lack of transparency and reproducibility in AI research after a Google Health study by McKinney et al., published in a prominent scientific journal in January 2020, claimed an artificial intelligence (AI) system could outperform human radiologists in both robustness and speed for breast cancer screening. The study made waves in the scientific community and created a buzz with the public, with headlines appearing in BBC News, CBC and CNBC.
A closer examination raised some concerns: the study lacked a sufficient description of the methods used, including their code and models. The lack of transparency prohibited researchers from learning exactly how the model works and how they could apply it to their own institutions.
“On paper and in theory, the McKinney et al. study is beautiful,” says Dr. Haibe-Kains, “But if we can’t learn from it then it has little to no scientific value.”
According to Dr. Haibe-Kains, who is jointly appointed as Associate Professor in Medical Biophysics at the University of Toronto and affiliate at the Vector Institute for Artificial Intelligence, this is just one example of a problematic pattern in computational research.
“Researchers are more incentivized to publish their finding rather than spend time and resources ensuring their study can be replicated,” explains Dr. Haibe-Kains. “Journals are vulnerable to the ‘hype’ of AI and may lower the standards for accepting papers that don’t include all the materials required to make the study reproducible – often in contradiction to their own guidelines.”
This can actually slow down the translation of AI models into clinical settings. Researchers are not able to learn how the model works and replicate it in a thoughtful way. In some cases, it could lead to unwarranted clinical trials, because a model that works on one group of patients or in one institution, may not be appropriate for another.
In the article titled Transparency and reproducibility in artificial intelligence, the authors offer numerous frameworks and platforms that allow safe and effective sharing to uphold the three pillars of open science to make AI research more transparent and reproducible: sharing data, sharing computer code and sharing predictive models.
“We have high hopes for the utility of AI for our cancer patients,” says Dr. Haibe-Kains. “Sharing and building upon our discoveries – that’s real scientific impact.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
Artificial intelligence research
- Artificial Intelligence (AI) in Fintech Market May See a Big Move : IBM, Amazon Web Services, Intelon November 27, 2020 at 11:57 am
Global Artificial Intelligence (AI) in Fintech Market Report 2020 by Key Players, Types, Applications, Countries, Market Size, Forecast to 2026 (Based on 2020 COVID-19 Worldwide Spread) is latest ...
- UCF Investigates Artificial Intelligence to Help Children Teach Coding to Classmates While Learning Social Skillson November 27, 2020 at 8:33 am
UCF Investigates Artificial Intelligence to Help Children Teach Coding to Classmates While Learning Social Skills | Read more about UCF Colleges & Campus News, Community, Research, Orlando and Central ...
- Overjet Tapped by AEGIS for Dental Artificial Intelligence Article Serieson November 27, 2020 at 6:00 am
Overjet Inc, the leading provider of artificial intelligence (AI) for the dental community, was selected by AEGIS to share a curated series on how AI is transforming dentistry. The series will appear ...
- Artificial Intelligence in Healthcare Global Market - Forecast to 2027on November 27, 2020 at 5:04 am
Reportlinker.com announces the release of the report "Artificial Intelligence in Healthcare Global Market - Forecast to 2027" - https:/ ...
- Artificial intelligence research projects get £20m booston November 26, 2020 at 9:23 pm
The Turing AI Acceleration Fellowships have been awarded to 15 researchers using AI on innovative and diverse projects.
Go deeper with Google Headlines on:
Artificial intelligence research
Go deeper with Bing News on:
Artificial intelligence research transparency
- Veterinary Artificial Insemination Market is Driven by Growing Awareness Regarding Benefits of Reduction of Disease Transmissionon November 27, 2020 at 5:36 am
Reasons why artificial insemination is preferred over natural mating is that it reduces risks, allows for easy transportation and storage for semen, and enables multiple females being impregnated at ...
- BasisAI included in Responsible AI Solutions New Tech report by Independent Research Firmon November 25, 2020 at 5:00 pm
Momentum for responsible artificial intelligence solutions continues to buildSINGAPORE, Nov. 26, 2020 /PRNewswire/ -- BasisAI, a provider of ...
- Artificial Intelligence (AI) in the context of transformation, transition, transparency & privacyon November 20, 2020 at 10:50 am
Our world is undergoing an information Big Bang, in which the universe of data doubles every two years and quintillions of bytes of data are generated every day. The impact of big data is commonly ...
- Unlock The Full Potential Of Artificial Intelligenceon November 20, 2020 at 3:05 am
The past year has been a learning experience for all of us, and collectively we’ve managed incredible things. Out of necessity, organizations have massively increased their speed of digitization and ...
- Lemonade via robots: UT research project looks into responsible artificial intelligenceon November 17, 2020 at 3:06 am
Jiao said their goal is to hopefully expand the technology and have service robots roaming the city in the next couple of years.