New training technique would reveal the basis for machine-learning systems’ decisions.
In recent years, the best-performing systems in artificial-intelligence research have come courtesy of neural networks, which look for patterns in training data that yield useful predictions or classifications. A neural net might, for instance, be trained to recognize certain objects in digital images or to infer the topics of texts.
But neural nets are black boxes. After training, a network may be very good at classifying data, but even its creators will have no idea why. With visual data, it’s sometimes possible to automate experiments that determine which visual features a neural net is responding to. But text-processing systems tend to be more opaque.
At the Association for Computational Linguistics’ Conference on Empirical Methods in Natural Language Processing, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) will present a new way to train neural networks so that they provide not only predictions and classifications but rationales for their decisions.
“In real-world applications, sometimes people really want to know why the model makes the predictions it does,” says Tao Lei, an MIT graduate student in electrical engineering and computer science and first author on the new paper. “One major reason that doctors don’t trust machine-learning methods is that there’s no evidence.”
“It’s not only the medical domain,” adds Regina Barzilay, the Delta Electronics Professor of Electrical Engineering and Computer Science and Lei’s thesis advisor. “It’s in any domain where the cost of making the wrong prediction is very high. You need to justify why you did it.”
“There’s a broader aspect to this work, as well,” says Tommi Jaakkola, an MIT professor of electrical engineering and computer science and the third coauthor on the paper. “You may not want to just verify that the model is making the prediction in the right way; you might also want to exert some influence in terms of the types of predictions that it should make. How does a layperson communicate with a complex model that’s trained with algorithms that they know nothing about? They might be able to tell you about the rationale for a particular prediction. In that sense it opens up a different way of communicating with the model.”
Neural networks are so called because they mimic — approximately — the structure of the brain. They are composed of a large number of processing nodes that, like individual neurons, are capable of only very simple computations but are connected to each other in dense networks.
In a process referred to as “deep learning,” training data is fed to a network’s input nodes, which modify it and feed it to other nodes, which modify it and feed it to still other nodes, and so on. The values stored in the network’s output nodes are then correlated with the classification category that the network is trying to learn — such as the objects in an image, or the topic of an essay.
Over the course of the network’s training, the operations performed by the individual nodes are continuously modified to yield consistently good results across the whole set of training examples. By the end of the process, the computer scientists who programmed the network often have no idea what the nodes’ settings are. Even if they do, it can be very hard to translate that low-level information back into an intelligible description of the system’s decision-making process.
In the new paper, Lei, Barzilay, and Jaakkola specifically address neural nets trained on textual data. To enable interpretation of a neural net’s decisions, the CSAIL researchers divide the net into two modules. The first module extracts segments of text from the training data, and the segments are scored according to their length and their coherence: The shorter the segment, and the more of it that is drawn from strings of consecutive words, the higher its score.
The segments selected by the first module are then passed to the second module, which performs the prediction or classification task. The modules are trained together, and the goal of training is to maximize both the score of the extracted segments and the accuracy of prediction or classification.
One of the data sets on which the researchers tested their system is a group of reviews from a website where users evaluate different beers. The data set includes the raw text of the reviews and the corresponding ratings, using a five-star system, on each of three attributes: aroma, palate, and appearance.
What makes the data attractive to natural-language-processing researchers is that it’s also been annotated by hand, to indicate which sentences in the reviews correspond to which scores. For example, a review might consist of eight or nine sentences, and the annotator might have highlighted those that refer to the beer’s “tan-colored head about half an inch thick,” “signature Guinness smells,” and “lack of carbonation.” Each sentence is correlated with a different attribute rating.
As such, the data set provides an excellent test of the CSAIL researchers’ system. If the first module has extracted those three phrases, and the second module has correlated them with the correct ratings, then the system has identified the same basis for judgment that the human annotator did.
In experiments, the system’s agreement with the human annotations was 96 percent and 95 percent, respectively, for ratings of appearance and aroma, and 80 percent for the more nebulous concept of palate.
In the paper, the researchers also report testing their system on a database of free-form technical questions and answers, where the task is to determine whether a given question has been answered previously.
In unpublished work, they’ve applied it to thousands of pathology reports on breast biopsies, where it has learned to extract text explaining the bases for the pathologists’ diagnoses. They’re even using it to analyze mammograms, where the first module extracts sections of images rather than segments of text.
“There’s a lot of hype now — and rightly so — around deep learning, and specifically deep learning for natural-language processing,” says Byron Wallace, an assistant professor of computer and information science at Northeastern University. “But a big drawback for these models is that they’re often black boxes. Having a model that not only makes very accurate predictions but can also tell you why it’s making those predictions is a really important aim.”
“As it happens, we have a paper that’s similar in spirit being presented at the same conference,” Wallace adds. “I didn’t know at the time that Regina was working on this, and I actually think hers is better. In our approach, during the training process, while someone is telling us, for example, that a movie review is very positive, we assume that they’ll mark a sentence that gives you the rationale. In this way we train the deep-learning model to extract these rationales. But they don’t make this assumption, so their model works without using direct annotations with rationales, which is a very nice property.”
Learn more: Making computers explain themselves
The Latest on: Deep learning
via Google News
The Latest on: Deep learning
- What are Machine Learning and Deep Learning in Artificial Intelligenceon February 2, 2020 at 2:16 pm
AI can be called a superset of Machine Learning (ML) processes, and Deep Learning (DL) processes. AI usually is an umbrella term that is used for ML and DL. Deep Learning is again, a subset of Machine ...
- Deep transfer learning has now been used to classify galaxies with expert-level accuracyon January 31, 2020 at 11:56 pm
Last year in September, researchers used deep learning to probe into dark matter in an industry first. Now, the same technique has been used to study galaxies. In the paper titled "Deep Learning at ...
- 5 Soon-to-Be Trends in Artificial Intelligence And Deep Learningon January 31, 2020 at 5:10 pm
AI’s major headwind is the cost of the investment, which will skew returns in the short-term. When the turnaround occurs, however, companies who are making the investment can expect to be rewarded ...
- Deep Learning Models for Histopathological Classification of Gastric and Colonic Epithelial Tumourson January 30, 2020 at 2:40 am
In 2018, there were 1,033,701 new cases and 782,685 deaths due to stomach cancer, and 1,096,601 new cases and 551,269 deaths due to colon cancer. Here, we propose deep learning models to classify ...
- Deep Learning Chipset Market Size, Share 2020 Supply, Consumption, Cost and Profit analysis and forecast to 2024| Says Market Reports Worldon January 29, 2020 at 9:37 pm
Deep Learning Chipset Market 2020 :- The Deep Learning Chipset Market has been created based on an in-depth market analysis with inputs from industry experts. The report covers the growth prospects ...
- Concentric launches with a deep learning approach to fixing broken file permissionson January 29, 2020 at 2:08 pm
Concentric Inc. launched today with $7.5 million in a Series A funding and an approach to document-level security that relies upon the artificial intelligence technique of deep learning to identify ...
- Opening the AI box: can deep learning predict cancer recurrence?on January 29, 2020 at 2:13 am
The deep-learning algorithm automatically annotates a 3D whole-mount pathology image and presents the discovered features in an understandable way. The patches with high probability of cancer ...
- Lab Materials for MIT 6.S191: Introduction to Deep Learningon January 29, 2020 at 12:31 am
This repository contains all of the code and software labs for MIT 6.S191: Introduction to Deep Learning! All lecture slides and videos are available on the course website. The 2020 6.S191 labs will ...
- Reality Engines offers a deep learning tour de force to challenge Amazon et al in Enterprise AIon January 28, 2020 at 9:28 am
Most exciting of all is that the tiny 18-person team has some very novel takes on deep learning forms of AI, the product of seasoned vets in machine learning technology and products. This is no me-too ...
via Bing News