Artificial Intelligence (AI) and machine learning algorithms such as Deep Learning have become integral parts of our daily lives: they enable digital speech assistants or translation services, improve medical diagnostics and are an indispensable part of future technologies such as autonomous driving. Based on an ever increasing amount of data and powerful novel computer architectures, learning algorithms appear to reach human capabilities, sometimes even excelling beyond.
The issue: so far it often remains unknown to users, how exactly AI systems reach their conclusions. Therefore it may often remain unclear, whether the AI’s decision making behavior is truly ‘intelligent’ or whether the procedures are just averagely successful.
Researchers from TU Berlin, Fraunhofer Heinrich Hertz Institute HHI and Singapore University of Technology and Design (SUTD) have tackled this question and have provided a glimpse into the diverse “intelligence” spectrum observed in current AI systems, specifically analyzing these AI systems with a novel technology that allows automatized analysis and quantification.
The most important prerequisite for this novel technology is a method developed earlier by TU Berlin and Fraunhofer HHI, the so-called Layer-wise Relevance Propagation (LRP) algorithm that allows visualizing according to which input variables AI systems make their decisions. Extending LRP, the novel Spectral relevance analysis (SpRAy) can identify and quantify a wide spectrum of learned decision making behavior. In this manner it has now become possible to detect undesirable decision making even in very large data sets.
This so-called ‘explainable AI’ has been one of the most important steps towards a practical application of AI, according to Dr. Klaus-Robert Müller, Professor for Machine Learning at TU Berlin. “Specifically in medical diagnosis or in safety-critical systems, no AI systems that employ flaky or even cheating problem solving strategies should be used.”
By using their newly developed algorithms, researchers are finally able to put any existing AI system to a test and also derive quantitative information about them: a whole spectrum starting from naive problem solving behavior, to cheating strategies up to highly elaborate “intelligent” strategic solutions is observed.
Dr. Wojciech Samek, group leader at Fraunhofer HHI said: “We were very surprised by the wide range of learned problem-solving strategies. Even modern AI systems have not always found a solution that appears meaningful from a human perspective, but sometimes used so-called ‘Clever Hans Strategies’.”
Clever Hans was a horse that could supposedly count and was considered a scientific sensation during the 1900s. As it was discovered later, Hans did not master math but in about 90 percent of the cases, he was able to derive the correct answer from the questioner’s reaction.
The team around Klaus-Robert Müller and Wojciech Samek also discovered similar “Clever Hans” strategies in various AI systems. For example, an AI system that won several international image classification competitions a few years ago pursued a strategy that can be considered naïve from a human’s point of view. It classified images mainly on the basis of context. Images were assigned to the category “ship” when there was a lot of water in the picture. Other images were classified as “train” if rails were present. Still other pictures were assigned the correct category by their copyright watermark. The real task, namely to detect the concepts of ships or trains, was therefore not solved by this AI system – even if it indeed classified the majority of images correctly.
The researchers were also able to find these types of faulty problem-solving strategies in some of the state-of-the-art AI algorithms, the so-called deep neural networks – algorithms that were so far considered immune against such lapses. These networks based their classification decision in part on artifacts that were created during the preparation of the images and have nothing to do with the actual image content.
“Such AI systems are not useful in practice. Their use in medical diagnostics or in safety-critical areas would even entail enormous dangers,” said Klaus-Robert Müller. “It is quite conceivable that about half of the AI systems currently in use implicitly or explicitly rely on such ‘Clever Hans’ strategies. It’s time to systematically check that, so that secure AI systems can be developed.”
With their new technology, the researchers also identified AI systems that have unexpectedly learned “smart” strategies. Examples include systems that have learned to play the Atari games Breakout and Pinball. “Here the AI clearly understood the concept of the game and found an intelligent way to collect a lot of points in a targeted and low-risk manner. The system sometimes even intervenes in ways that a real player would not,” said Wojciech Samek.
“Beyond understanding AI strategies, our work establishes the usability of explainable AI for iterative dataset design, namely for removing artefacts in a dataset which would cause an AI to learn flawed strategies, as well as helping to decide which unlabeled examples need to be annotated and added so that failures of an AI system can be reduced,” said SUTD Assistant Professor Alexander Binder.
“Our automated technology is open source and available to all scientists. We see our work as an important first step in making AI systems more robust, explainable and secure in the future, and more will have to follow. This is an essential prerequisite for general use of AI,” said Klaus-Robert Müller.
Learn more: How intelligent is artificial intelligence?
The Latest on: Artificial intelligence
via Google News
The Latest on: Artificial intelligence
- Artificial Intelligence in Computer Networks Market 2019 Global Trend, Segmentation and Opportunities Forecast To 2025on August 16, 2019 at 9:48 am
Aug 16, 2019 (Heraldkeepers) -- Wiseguyreports.Com adds "Artificial Intelligence in Computer Networks Market -Market Demand, Growth, Opportunities, Analysis of Top Key Players and Forecast to 2025" To ...
- Jeffrey Tarrant Saw Artificial Intelligence as the Future of Managing Moneyon August 16, 2019 at 7:34 am
Even while being treated for the brain cancer that would cut his life short, Jeffrey Tarrant was searching for the next frontier in investing. He figured it would involve the use of artificial ...
- Artificial Intelligence (AI) Robots Market with Growing CAGR in Forecast Period 2019 to 2026on August 16, 2019 at 6:30 am
Aug 16, 2019 (Global QYResearch via COMTEX) -- A new research report titled “Global Artificial Intelligence (AI) Robots Market” successfully exhibits the complete scenario of the global and an ...
- New York-based artificial intelligence unicorn co. hires 71 for Houston office, aims to hire moreon August 16, 2019 at 6:26 am
New York-based UiPath Inc., one of the world’s largest artificial intelligence-focused startups, is officially opening its Houston office Aug. 22. Marie Myers, the Houston-based CFO of UiPath and a ...
- 'Stereotyping' emotions is getting in the way of artificial intelligence. Scientists say they've discovered a better way.on August 16, 2019 at 5:07 am
If you want to understand what people are feeling, you have to look beyond surface-level cues. Technology like AI could fall short without this.
- Staying Smart: How artificial intelligence could impact companies, customers and competitionon August 16, 2019 at 4:00 am
Until recently, artificial intelligence (AI) was limited to movies and science fiction. That’s rapidly changing however as advances in computing power and the creation of massive data sets have made ...
- The Amazing Ways Babylon Health Is Using Artificial Intelligence To Make Healthcare Universally Accessibleon August 15, 2019 at 9:34 pm
Babylon, a UK start-up, plans to "put an accessible and affordable health service in the hands of every person on earth" by putting artificial intelligence (AI) tools to work. Currently, the company ...
- Artificial Intelligence Technology Solutions Receives PO for High End Retail Location Owned by Billion Dollar Real Estate Fundon August 15, 2019 at 11:10 am
RENO, NV, Aug. 15, 2019 (GLOBE NEWSWIRE) -- via NEWMEDIAWIRE – Artificial Intelligence Technology Solutions, Inc. (AITX:OTCPK), is pleased to announce that Robotic Assistance Devices (RAD), its ...
- Cab Aggregator Unicorn Ola Acquires Artificial Intelligence Platform Pickup.aion August 15, 2019 at 8:16 am
Bengaluru-based cab aggregator unicorn Ola has acquired Bengaluru-based artificial intelligence platform Pickup.ai. As part of the deal, the co-founders and employees of Pikup.ai will join the Ola ...
- Artificial Intelligence In Fintech - Two CEOs Perspectiveson August 15, 2019 at 5:11 am
Philippe Branche: First, could you describe your company in a few words? Simon Loong: WeLab is a fintech company providing seamless digital financial services. Today, we have more than 800 employees ...
via Bing News