“Misinformation endangers the public debate on a broad range of global societal issues“
Network scientists at Indiana University have developed a new computational method that can leverage any body of knowledge to aid in the complex human task of fact-checking.
In the first use of this method, IU scientists created a simple computational fact-checker that assigns “truth scores” to statements concerning history, geography and entertainment, as well as random statements drawn from the text of Wikipedia, the well-known online encyclopedia.
In multiple experiments, the automated system consistently matched the assessment of human fact-checkers in terms of their certitude about the accuracy of these statements.
The results of the study, “Computational Fact Checking From Knowledge Networks,” are reported in today’s issue of PLOS ONE.
“These results are encouraging and exciting,” said Giovanni Luca Ciampaglia, a postdoctoral fellow at the Center for Complex Networks and Systems Research in the IU Bloomington School of Informatics and Computing, who led the study. “We live in an age of information overload, including abundant misinformation, unsubstantiated rumors and conspiracy theories whose volume threatens to overwhelm journalists and the public.
“Our experiments point to methods to abstract the vital and complex human task of fact-checking into a network analysis problem, which is easy to solve computationally.”
The team selected Wikipedia as the information source for their experiment due to its breadth and open nature. Although Wikipedia is not 100 percent accurate, previous studies estimate the online encyclopedia is nearly as reliable as traditional encyclopedias, but also covers many more subjects.
Using factual information from infoboxes on the site, IU scientists built a “knowledge graph” with 3 million concepts and 23 million links between them. A link between two concepts in the graph can be read as a simple factual statement, such as “Socrates is a person” or “Paris is the capital of France.”
In what the IU scientists describe as an “automatic game of trivia,” the team applied their algorithm to answer simple questions related to geography, history and entertainment, including statements that matched states or nations with their capitals, presidents with their spouses and Oscar-winning film directors with the movie for which they won the Best Picture awards, with the majority of tests returning highly accurate truth scores.
Lastly, the scientists used the algorithm to fact-check excerpts from the main text of Wikipedia, which were previously labeled by human fact-checkers as true or false, and found a positive correlation between the truth scores produced by the algorithm and the answers provided by the fact-checkers.
Significantly, the IU team found their computational method could even assess the truthfulness of statements about information not directly contained in the infoboxes. For example, the fact that Steve Tesich — the Serbian-American screenwriter of the classic Hoosier film “Breaking Away” — graduated from IU, despite the information not being specifically addressed in the infobox about him.
“The measurement of the truthfulness of statements appears to rely strongly on indirect connections, or ‘paths,’ between concepts,” Ciampaglia said. “If we prevented our fact-checker from traversing multiple nodes on the graph, it performed poorly since it could not discover relevant indirect connections. But because it’s free to explore beyond the information provided in one infobox, our method leverages the power of the full knowledge graph.”
Although the experiments were conducted using Wikipedia, the IU team’s method does not assume any particular source of knowledge. The scientists aim to conduct additional experiments using knowledge graphs built from other sources of human knowledge, such as Freebase, the open-knowledge base built by Google, and note that multiple information sources could be used together to account for different belief systems.
The Latest on: Fact-checking
via Google News
The Latest on: Fact-checking
- Fact-checking Facebook post about Saudi terrorists and Saudis on 45th floor of Trump toweron January 10, 2020 at 2:08 pm
A Facebook account that has more than 500,000 followers posted this claim, with a couple of grammatical errors, about Saudi Arabia and President Donald Trump: 1. Saudi terrorists has killed more ...
- “Rated false”: Here’s the most interesting new research on fake news and fact-checkingon January 10, 2020 at 9:04 am
Here, JR’s managing editor, Denise-Marie Ordway, sums up some of the most compelling papers on fake news and fact-checking published in 2019. (You can also read some of her other roundups focusing on ...
- Fact-checking Trump's claims about tanks, cars and Ohio's 'best' economic yearon January 10, 2020 at 4:20 am
President Donald Trump's first rally of 2020 was back in the Buckeye State, where he name-dropped everyone from Ohio inventor Thomas Edison to John Glenn – a one-time Democratic presidential candidate ...
- Donald Trump in Toledo: Fact-checking claims about tanks, cars and Ohio's 'best' economic yearon January 9, 2020 at 6:44 pm
Was 2019 the best year for Ohio's economy? Did President Trump save a Lima tank plant? Checking claims made at the Toledo rally.
- Fact-checking Trump's claims about Obama and the Iran dealon January 9, 2020 at 6:06 pm
CNN's Jake Tapper, in partnership with FactCheck.org, looks at statements made by President Trump about Iran and former President Barack Obama in his speech following Iran's attack on the Iraqi air ...
- Fact-checking Trump's claims on the Iran dealon January 8, 2020 at 11:30 pm
The US and Iran appear to have stepped away from the brink of full-blown conflict, after trading military strikes in recent days, but tensions remain high. Follow here for the latest updates.
- VERIFY: Fact-checking President Trump's speech after Iran's missile attackon January 8, 2020 at 7:23 pm
Following Iran’s missile attack on two Iraqi military bases that house American troops, President Donald Trump publicly addressed the attack Wednesday morning from the White House. The VERIFY team ...
- Fact-checking the president’s remarks on Iranon January 8, 2020 at 7:03 pm
President Donald Trump wrongly dismissed the continuing threat of the Islamic State group and spread a false tale of the U.S. paying out billions of dollars to Iran as part of the multinational ...
- Fact-checking Trump’s address on the Iran missile attackson January 8, 2020 at 12:25 pm
President Trump’s address to the nation after Iran’s missile attacks on bases in Iraq where U.S. troops are stationed contained a number of factually dubious statements. Here’s a quick roundup.
- Fact-Checking President Trump’s Address on Iranon January 8, 2020 at 10:15 am
Here is a look at some of the misleading statements made by President Donald Trump in a national address following Iran's attack on two military bases in Iraq that house American troops. President ...
via Bing News