A team of researchers from the Department of Energy’s Oak Ridge National Laboratory has married artificial intelligence and high-performance computing to achieve a peak speed of 20 petaflops in the generation and training of deep learning networks on the laboratory’s Titan supercomputer.
Deep learning is a burgeoning field of artificial intelligence that uses networks modeled after the human brain to “learn” how to distinguish features and patterns in vast datasets. Such networks hold great promise in the realization of numerous technologies, from self-driving cars to intelligent robots.
Due to its ability to make sense of massive amounts of data, researchers across the scientific spectrum are eager to refine deep learning and apply it to some of today’s most challenging science problems. One such effort is ORNL’s Advances in Machine Learning to Improve Scientific Discovery at Exascale and Beyond (ASCEND) project, which aims to use deep learning to make sense of the massive datasets produced by the world’s most sophisticated scientific experiments, such as those located at ORNL.
Analysis of such datasets generally requires existing neural networks to be modified, or novel networks designed and then “trained” so that they know precisely what to look for and can produce valid results.
This is a time-consuming and difficult task, but one that an ORNL team led by Robert Patton and including Steven Young and Travis Johnston recently demonstrated can be dramatically expedited with a capable computing system such as ORNL’s Titan, the nation’s fastest supercomputer for science.
To efficiently design neural networks capable of tackling scientific datasets and expediting breakthroughs, Patton’s team developed two codes for evolving (MENNDL) and fine-tuning (RAvENNA) deep neural network architectures.
Both codes can generate and train as many as 18,600 neural networks simultaneously. Peak performance can be estimated by randomly sampling, and then carefully profiling, several hundred of these independently trained networks.
Both codes achieved a peak performance of 20 petaflops, or 20 thousand trillion calculations per second, on Titan (or just under half of Titan’s single precision total peak performance). In practical terms, that translates to training 40-50,000 networks per hour.
“The real measure of success in the deep learning community is time-to-solution,” said Johnston. “And with a machine like Titan we are able to train an unparalleled number of highly accurate networks.”
Titan is a Cray hybrid system, meaning that it uses both traditional CPUs and graphics processing units (GPUs) to tackle complex calculations for big science problems efficiently; the GPUs also happen to be the processor of choice for training deep learning networks.
The team’s work demonstrates that with the right high-performance computing system researchers can efficiently train large numbers of networks, which can then be used to help them tackle today’s increasingly data-heavy experiments and simulations.
This efficient design of deep neural networks will enable researchers to deploy highly accurate, custom-designed models, saving both time and money by freeing the scientist from the task of designing a network from the ground up.
And because the OLCF’s next leadership computing system, Summit, features a deep-learning friendly architecture with enhanced GPUs and complementary Tensor cores, the team is confident both codes will only get faster.
“Out of the box, without tuning to Summit’s unique architecture, we are expecting an increase in performance up to 50 times,” said Johnston.
With that sort of network training capability, Summit could be indispensable to researchers across the scientific spectrum looking to deep learning to help them tackle some of science’s most immense challenges.
Patton’s team is not waiting for the improved hardware to start tackling current scientific data challenges; they have already deployed their codes to assist domain scientists at the Department of Energy’s Fermilab in Batavia, Illinois.
Researchers at Fermilab used MENNDL to better understand how neutrinos interact with ordinary matter by producing a classification network to support their Main Injector Experiment for v-A (MINERvA), a neutrino scattering experiment. The task, known as vertex reconstruction, required a network to analyze images and precisely identify the location where neutrinos interact with one of many targets—a task akin to finding the aerial source of a starburst of fireworks.
In only 24 hours, MENNDL produced optimized networks that outperformed any previously handcrafted network—an achievement that could easily have taken scientists months to accomplish. To identify the high-performing network, MENNDL evaluated approximately 500,000 neural networks, training them on a data set consisting of 800,000 images of neutrino events, steadily using 18,000 of Titan’s nodes.
“You need something like MENNDL to explore this effectively infinite space of possible networks, but you want to do it efficiently,” Young said. “What Titan does is bring the time to solution down to something practical.”
And with Summit to come online this year, the future of deep learning in big science looks bright indeed.
The Latest on: Deep learning networks
- Micron Joins CERN openlab, Bringing New Machine Learning Capabilities to Advance Science and Research on November 12, 2018 at 7:03 am
Micron’s memory solutions that combine neural network capabilities will be tested ... between Micron and FWDNXT, a provider of deep learning and AI solutions. “Micron is committed to pushing ... […]
- Sophos Intercept X with Deep Learning Honored with 2018 CRN® Tech Innovator Award on November 12, 2018 at 7:00 am
Sophos Intercept X with deep learning took top honors in the Endpoint ... CRN named Sophos as an “Overall Winner” in Network Security and “Overall Winner” in Endpoint Security, ranking ... […]
- Facial recognition's failings: Coping with uncertainty in the age of machine learning on November 9, 2018 at 5:37 am
Deep learning relies on training deep neural networks — mathematical models loosely inspired by the structure of the brain — teaching these networks to make accurate predictions, typically by feeding ... […]
- Machine Learning: A Starter Pack on November 8, 2018 at 3:24 pm
This entire process mimics a human brain. It uses neural networks to make each voice search faster and more accurate than the last. Deep Learning, also known as the Deep Neural Network, is a strand of ... […]
- Mipsology Delivers Deep Learning Inference at 20X Speedup versus Midrange Xeon CPU Leveraging Mellanox' SmartNIC Adapters on November 8, 2018 at 11:34 am
Mipsology is a startup developing state-of-the-art FPGA-based accelerators targeted for deep learning applications in neural networks. It was founded in 2015 by a team of engineers and scientists ... […]
- Deep Learning for Medical Imaging Fares Poorly on External Data on November 7, 2018 at 7:07 am
Researchers at the Icahn School of Medicine at Mount Sinai trained a convolutional neural network (CNN), a form of deep learning, to flag evidence of pneumonia in chest x-rays. In three out of five co... […]
- Artificial Neural Networks. Beginners guide on November 6, 2018 at 1:24 pm
Only this precise level of learning, known as deep learning, is something that makes neural network become as smart as humans. What can it do ANN already proved itself as a technology that can gain va... […]
- Toshiba Memory Corporation Develops High-Speed and High-Energy-Efficiency Algorithm and Hardware Architecture for Deep Learning Processor on November 5, 2018 at 11:37 pm
 ResNet50: One of deep neural network, generally used to benchmark deep-learning for image recognition.  ImageNet: A large image database, generally used to benchmark image-recognition, the numb... […]
- Microcontroller Aimed at Neural Networks That Train Themselves on November 5, 2018 at 6:15 pm
Tensai can also handle convolutional neural networks (CNNs), the main building block of deep learning algorithms. These networks are a honeycomb of nodes, each connected to another node in the network ... […]
- New mobile device identifies airborne allergens using deep learning on November 5, 2018 at 10:55 am
Trained to recognize five common allergens — pollen from Bermuda grass, oak, ragweed and spores from two types of mold — the system classified samples with an accuracy of 94 percent using deep learnin... […]
via Google News and Bing News