Researchers develop machine model that exponentially shortens expert review period
Northwestern University researchers are using artificial intelligence (AI) to speed up the search for COVID-19 treatments and vaccines. The AI-powered tool makes it possible to prioritize resources for the most promising studies — and ignore research that is unlikely to yield benefits.
In the midst of the pandemic, scientific research is being conducted at an unprecedented rate. The Food and Drug Administration and the U.S. Department of Health and Human Services announced plans to accelerate clinical trials, and hundreds of scientists are investigating possible treatments and vaccines.
We’re behind the ball, and this can help us catch up.”
But the question remains: Which research has the most potential to produce real, much-needed solutions?
The scientific community has been predicting the answer to such questions for decades using the Defense Advanced Research Projects Agency’s Systematizing Confidence in Open Research and Evidence (DARPA SCORE) program. The program relies on scientific experts to review and rate submitted research studies based on how likely they are to be replicable. On average, this process takes about 314 days – a long wait in the midst of global pandemic.
The machine model is just as accurate as the human scoring system at making such predictions, researchers said, and it can scale up to review a larger number of papers in a fraction of the time – minutes instead of months.
“The standard process is too expensive, both financially and in terms of opportunity costs,” said Northwestern’s Brian Uzzi, who led the study. “First, it takes take too long to move on to the second phase of testing and second, when experts are spending their time reviewing other people’s work, it means they are not in the lab conducting their own research.”
With their new AI tool, Uzzi and his team at the Kellogg School of Management bypass the human-scoring method, allowing the research community and policymakers to make faster decisions about how to prioritize time and funding on the studies that are most likely to succeed.
Uzzi is the corresponding author on the paper, titled “Estimating the ‘Deep-Replicability’ of Scientific Findings Using Human and Machine Intelligence,” which will be published the week of May 4 in PNAS.
“In the midst of a public health crisis, it is essential that we focus our efforts on the most promising research,” said Uzzi, the Richard L. Thomas Professor of Leadership at Kellogg and co-director of the Northwestern Institute on Complex Systems. “This is important not only to save lives, but also to quickly tamp down the misinformation that results from poorly conducted research.”
The team of Northwestern researchers developed an algorithm to predict which studies’ results are most likely to be replicable. Replication, which means that the results of the study can be produced a second time with a new test population, is a key signal that study conclusions are valid.
314 days turnaround time with human predictive model
5 minutes turnaround time with new AI model
The machine model’s prediction of the likelihood of replicability may actually be more accurate than the traditional human-scoring prediction, researchers said, because it considers more of the narrative of the study, while expert reviewers tend to focus on the strength of the relational statistics in a paper.
“There is a lot of valuable information in how study authors explain their results,” Uzzi said. “The words they use reveal their own confidence in their findings, but it is hard for the average human to detect that.”
Because the algorithm examines the words of thousands of papers, it recognizes word-choice patterns that might be hidden from human consciousness. It has a much bigger schema to draw upon for its predictions, which makes it an extraordinary partner for human reviewers, Uzzi said.
The researchers’ model can be used immediately to analyze COVID-related research papers and quickly determine which show the most promise.
“This tool is particularly useful in this crisis situation where we can’t act fast enough,” Uzzi said. “It can give us an accurate estimate of what’s going to work and not work very quickly. We’re behind the ball, and this can help us catch up.”
Used on its own, the model has comparable accuracy to the DARPA SCORE method. Paired together, the combination human-machine approach predicts which findings will be replicable with even greater accuracy than either method on its own, the researchers found.
“This tool will help us conduct the business of science with greater accuracy and efficiency,” Uzzi said. “Now more than ever, it’s essential for the research community to operate lean, focusing only on those studies which hold real promise.”
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
- 'Collateral Damage': COVID-19 Threatens Patients With COPDon June 23, 2020 at 12:44 pm
There is concern that the COVID-19 pandemic may be inflicting collateral damage on vulnerable patients with chronic conditions, in particular, those with COPD.
- NIH director: COVID-19's 'heartbreaking' harm to Black and Hispanic Americans demands testingon June 23, 2020 at 12:42 pm
Francis Collins, director of the NIH, calls COVID-19's harm on Black and Hispanic Americans "heartbreaking" and said trials must reach out to those communities.
- PCORI Board approves up to $50 million in funding for COVID-19 researchon June 23, 2020 at 12:41 pm
Board of Governors today, building on its rapid response to the COVID-19 health crisis, approved allocating up to $50 million in additional funding for new projects in PCORI's COVID-19 funding ...
- Surgical Theater Responds to COVID-19 Pandemic with Free Remote 360° VR Serviceson June 23, 2020 at 12:36 pm
Surgical Theater, Inc., the market leader in virtual and augmented reality-based healthcare services, announced today that it will make available its virtual reality technology at no charge to any ...
- Additional Cases Of Antiviral Drug Remdesivir Distributed to DFW Metroplex for COVID-19 Treatmentson June 23, 2020 at 10:39 am
Governor Greg Abbott has announced that the Texas Department of State Health Services will distribute 120 additional cases of the antiviral drug remdesivir to 37 hospitals across the Dallas-Fort Worth ...
Go deeper with Google Headlines on:
Go deeper with Bing News on:
- Watch live: Dr. Anthony Fauci optimistic about COVID-19 vaccine, he and health officials say Trump didn’t ask to slow down testingon June 23, 2020 at 11:59 am
With coronavirus cases rising in about half the states, Dr. Anthony Fauci returns to Capitol Hill on Tuesday at a fraught moment in the nation's pandemic response.
- Fauci hopeful for COVID-19 vaccine by end of the year or early 2021on June 23, 2020 at 11:33 am
Dr. Anthony Fauci and other top health officials also said they have not been asked to slow down testing for coronavirus, an issue that became controversial after President Donald ...
- Coronavirus Vaccine News: French, US Biotech Companies Expand Partnership, Will Evaluate COVID-19 Candidateson June 23, 2020 at 10:45 am
Sanofi is also developing potential coronavirus vaccines with English pharmaceutical company GlaxoSmithKline Plc.
- Fauci: No requests to slow down COVID-19 testing, vaccine timeline unchangedon June 23, 2020 at 10:35 am
Dr. Anthony Fauci told a House panel Tuesday that he has never been asked to slow down testing for COVID-19 and again expressed hope that a vaccine will be ready by the end of 2020 or early 2021.
- Translate Bio Joins With Sanofi to Make Covid-19 Vaccineson June 23, 2020 at 10:06 am
Sanofi will take a 7.2% stake in Translate Bio as part of the deal and give it an initial $425 million payment.