Neural networks could be implemented more quickly using new photonic technology
“Deep learning” computer systems, based on artificial neural networks that mimic the way the brain learns from an accumulation of examples, have become a hot topic in computer science. In addition to enabling technologies such as face- and voice-recognition software, these systems could scour vast amounts of medical data to find patterns that could be useful diagnostically, or scan chemical formulas for possible new pharmaceuticals.
But the computations these systems must carry out are highly complex and demanding, even for the most powerful computers.
Now, a team of researchers at MIT and elsewhere has developed a new approach to such computations, using light instead of electricity, which they say could vastly improve the speed and efficiency of certain deep learning computations. Their results appear today in the journal Nature Photonics in a paper by MIT postdoc Yichen Shen, graduate student Nicholas Harris, professors Marin Solja?i? and Dirk Englund, and eight others.
Solja?i? says that many researchers over the years have made claims about optics-based computers, but that “people dramatically over-promised, and it backfired.” While many proposed uses of such photonic computers turned out not to be practical, a light-based neural-network system developed by this team “may be applicable for deep-learning for some applications,” he says.
Traditional computer architectures are not very efficient when it comes to the kinds of calculations needed for certain important neural-network tasks. Such tasks typically involve repeated multiplications of matrices, which can be very computationally intensive in conventional CPU or GPU chips.
After years of research, the MIT team has come up with a way of performing these operations optically instead. “This chip, once you tune it, can carry out matrix multiplication with, in principle, zero energy, almost instantly,” Solja?i? says. “We’ve demonstrated the crucial building blocks but not yet the full system.”
By way of analogy, Solja?i? points out that even an ordinary eyeglass lens carries out a complex calculation (the so-called Fourier transform) on the light waves that pass through it. The way light beams carry out computations in the new photonic chips is far more general but has a similar underlying principle. The new approach uses multiple light beams directed in such a way that their waves interact with each other, producing interference patterns that convey the result of the intended operation. The resulting device is something the researchers call a programmable nanophotonic processor.
The result, Shen says, is that the optical chips using this architecture could, in principle, carry out calculations performed in typical artificial intelligence algorithms much faster and using less than one-thousandth as much energy per operation as conventional electronic chips. “The natural advantage of using light to do matrix multiplication plays a big part in the speed up and power savings, because dense matrix multiplications are the most power hungry and time consuming part in AI algorithms” he says.
The new programmable nanophotonic processor, which was developed in the Englund lab by Harris and collaborators, uses an array of waveguides that are interconnected in a way that can be modified as needed, programming that set of beams for a specific computation. “You can program in any matrix operation,” Harris says. The processor guides light through a series of coupled photonic waveguides. The team’s full proposal calls for interleaved layers of devices that apply an operation called a nonlinear activation function, in analogy with the operation of neurons in the brain.
To demonstrate the concept, the team set the programmable nanophotonic processor to implement a neural network that recognizes four basic vowel sounds. Even with this rudimentary system, they were able to achieve a 77 percent accuracy level, compared to about 90 percent for conventional systems. There are “no substantial obstacles” to scaling up the system for greater accuracy, Solja?i? says.
Englund adds that the programmable nanophotonic processor could have other applications as well, including signal processing for data transmission. “High-speed analog signal processing is something this could manage” faster than other approaches that first convert the signal to digital form, since light is an inherently analog medium. “This approach could do processing directly in the analog domain,” he says.
The team says it will still take a lot more effort and time to make this system useful; however, once the system is scaled up and fully functioning, it can find many user cases, such as data centers or security systems. The system could also be a boon for self-driving cars or drones, says Harris, or “whenever you need to do a lot of computation but you don’t have a lot of power or time.”
Learn more: New system allows optical “deep learning”
The Latest on: Neural networks
- Futures: “Dorkwood” paint? The neural networks making us laugh on September 21, 2017 at 3:23 am
Neural networks can raise a smile, but do they have a sense of humour? Nicole Kobie reveals the world of AI-created paint, Pokémon and recipes. Stummy Beige. Dorkwood. Sindis Poop. Turdly. These are not paints you’d choose to slather on the walls of ... […]
- Imagination reveals PowerVR Neural Network Accelerator (NNA) with 2x the performance and half the bandwidth of nearest competitor on September 21, 2017 at 12:01 am
First dedicated hardware solution with flexible bit depth support from 16-bit down to 4-bit Lowest bandwidth Neural Network (NN) solution Architected to support multiple operating systems, including Linux and Android Neural networks such as Convolutional ... […]
- Imagination creates neural network accelerator for custom processors on September 20, 2017 at 5:01 pm
Imagination Technologies will make it easy for chip designers to embed neural network accelerators in their chips in order to dramatically speed up artificial intelligence processing. London-based Imagination is introducing a design that chip makers can ... […]
- Finite time synchronization of memristor-based Cohen-Grossberg neural networks with mixed delays on September 20, 2017 at 2:35 pm
Finite time synchronization, which means synchronization can be achieved in a settling time, is desirable in some practical applications. However, most of the published results on finite time synchronization don’t include delays or only include discrete ... […]
- Using Recurrent Neural Networks to Predict Player Performance on September 19, 2017 at 1:00 am
Editor’s Note: This article initially was a presentation at the marvelous 2017 Saberseminar. The year is 3047. Advancements in technology allow Buck Showalter to unplug a USB drive from a port behind his left ear and transfer it to an iPad. They don’t ... […]
- Build Simple AI .NET Library - Part 5 - Artificial Neural Networks on September 18, 2017 at 1:00 am
My objective is to create a simple AI library that covers a couple of advanced AI topics such as Genetic algorithms, ANN, Fuzzy logics and other evolutionary algorithms. The only challenge to complete this series would be having enough time working on code ... […]
- A Neural Network Invents D&D Spells, and Decides “Dave” Is the Most Magical Name Of All on September 16, 2017 at 1:36 pm
Research scientist Janelle Shane, whose wonderful neural networks we’ve covered before, has turned her experiments from guinea pig names to D&D spells – and the results are just as entertaining. She first ran a dataset of 365 examples, and the results ... […]
- Visualizing convolutional neural networks on September 16, 2017 at 12:53 pm
Given all of the higher level tools that you can use with TensorFlow, such as tf.contrib.learn and Keras, one can very easily build a convolutional neural network with a very small amount of code. But often with these higher level applications, you cannot ... […]
- Recurrent Neural Networks for Email List Churn Prediction on September 12, 2017 at 10:27 pm
Find out how AI-Fueled APIs from Neura can make interesting products more exciting and engaging. Not very long after finishing writing my lessons learned from building a Hello World neural network, I thought that I could move on from a simple MLP to a more ... […]
- MIT Paper Sheds Light on How Neural Networks Think on September 11, 2017 at 1:00 am
MIT researchers have developed a new general-purpose technique sheds light on inner workings of neural nets trained to process language. Artificial-intelligence research has been transformed by machine-learning systems called neural networks, which learn ... […]
via Google News and Bing News