Rice University researchers have demonstrated methods for both designing data-centric computing hardware and co-designing hardware with machine-learning algorithms that together can improve energy efficiency in artificial intelligence hardware by as much as two orders of magnitude.
Rice University researchers have demonstrated methods for both designing innovative data-centric computing hardware and co-designing hardware with machine-learning algorithms that together can improve energy efficiency by as much as two orders of magnitude.
Advances in machine learning, the form of artificial intelligence behind self-driving cars and many other high-tech applications, have ushered in a new era of computing — the data-centric era — and are forcing engineers to rethink aspects of computing architecture that have gone mostly unchallenged for 75 years.
“The problem is that for large-scale deep neural networks, which are state-of-the-art for machine learning today, more than 90% of the electricity needed to run the entire system is consumed in moving data between the memory and processor,” said Yingyan Lin, an assistant professor of electrical and computer engineering.
Lin and collaborators proposed two complementary methods for optimizing data-centric processing, both of which were presented June 3 at the International Symposium on Computer Architecture (ISCA), one of the premier conferences for new ideas and research in computer architecture.
The drive for data-centric architecture is related to a problem called the von Neumann bottleneck, an inefficiency that stems from the separation of memory and processing in the computing architecture that has reigned supreme since mathematician John von Neumann invented it in 1945. By separating memory from programs and data, von Neumann architecture allows a single computer to be incredibly versatile; depending upon which stored program is loaded from its memory, a computer can be used to make a video call, prepare a spreadsheet or simulate the weather on Mars.
But separating memory from processing also means that even simple operations, like adding 2 plus 2, require the computer’s processor to access the memory multiple times. This memory bottleneck is made worse by massive operations in deep neural networks, systems that learn to make humanlike decisions by “studying” large numbers of previous examples. The larger the network, the more difficult the task it can master, and the more examples the network is shown, the better it performs. Deep neural network training can require banks of specialized processors that run around the clock for more than a week. Performing tasks based on the learned networks — a process known as inference — on a smartphone can drain its battery in less than an hour.
“It has been commonly recognized that for the data-centric algorithms of the machine-learning era, we need innovative data-centric hardware architecture,” said Lin, the director of Rice’s Efficient and Intelligent Computing (EIC) Lab. “But what is the optimal hardware architecture for machine learning?
“There are no one-for-all answers, as different applications require machine-learning algorithms that might differ a lot in terms of algorithm structure and complexity, while having different task accuracy and resource consumption — like energy cost, latency and throughput — tradeoff requirements,” she said. “Many researchers are working on this, and big companies like Intel, IBM and Google all have their own designs.”
One of the presentations from Lin’s group at ISCA 2020 offered results on TIMELY, an innovative architecture she and her students developed for “processing in-memory” (PIM), a non-von Neumann approach that brings processing into memory arrays. A promising PIM platform is “resistive random access memory” (ReRAM), a nonvolatile memory similar to flash. While other ReRAM PIM accelerator architectures have been proposed, Lin said experiments run on more than 10 deep neural network models found TIMELY was 18 times more energy efficient and delivered more than 30 times the computational density of the most competitive state-of-the-art ReRAM PIM accelerator.
TIMELY, which stands for “Time-domain, In-Memory Execution, LocalitY,” achieves its performance by eliminating major contributors to inefficiency that arise from both frequent access to the main memory for handling intermediate input and output and the interface between local and main memories.
In the main memory, data is stored digitally, but it must be converted to analog when it is brought into the local memory for processing in-memory. In prior ReRAM PIM accelerators, the resulting values are converted from analog to digital and sent back to the main memory. If they are called from the main memory to local ReRAM for subsequent operations, they are converted to analog yet again, and so on.
TIMELY avoids paying overhead for both unnecessary accesses to the main memory and interfacing data conversions by using analog-format buffers within the local memory. In this way, TIMELY mostly keeps the required data within local memory arrays, greatly enhancing efficiency.
The group’s second proposal at ISCA 2020 was for SmartExchange, a design that marries algorithmic and accelerator hardware innovations to save energy.
“It can cost about 200 times more energy to access the main memory — the DRAM — than to perform a computation, so the key idea for SmartExchange is enforcing structures within the algorithm that allow us to trade higher-cost memory for much-lower-cost computation,” Lin said.
“For example, let’s say our algorithm has 1,000 parameters,” she added. “In a conventional approach, we will store all the 1,000 in DRAM and access as needed for computation. With SmartExchange, we search to find some structure within this 1,000. We then need to only store 10, because if we know the relationship between these 10 and the remaining 990, we can compute any of the 990 rather than calling them up from DRAM.
“We call these 10 the ‘basis’ subset, and the idea is to store these locally, close to the processor to avoid or aggressively reduce having to pay costs for accessing DRAM,” she said.
The researchers used the SmartExchange algorithm and their custom hardware accelerator to experiment on seven benchmark deep neural network models and three benchmark datasets. They found the combination reduced latency by as much as 19 times compared to state-of-the-art deep neural network accelerators.
The Latest Updates from Bing News & Google News
Go deeper with Bing News on:
AI energy efficiency
- Global E-Invoicing Market Report 2020-2025: Cloud Integration, AI, IoT and Predictive Analytical Tools are Contributing to Market Growthon November 12, 2020 at 9:04 am
Global Industry Trends, Share, Size, Growth, Opportunity and Forecast 2020-2025" report has been added to ResearchAndMarkets.com's offering. The global e-invoicing market grew at a CAGR of around 17% ...
- Ambiq Honored with Most Innovative AI Application for Best of Show Award at Flash Memory Summit 2020on November 11, 2020 at 6:02 am
Ambiq was honored with a Flash Memory Summit 2020 Best of Show Award for the Most Innovative AI Application at today’s Flash Memory Summit 2020 Best of Show Awards ceremony. The Flash Memory Summit, ...
- Young Innovators: New U of S artificial intelligence makes “smart” apps faster, more efficienton November 10, 2020 at 1:25 pm
Tired of Siri or Google Assistant draining your phone battery? A new University of Saskatchewan artificial intelligence computer model holds promise for making “smart” apps such as Amazon, Apple, and ...
- CLEAResult Announces Multi-Year Agreement with Streem to Deliver AR- and AI-Enabled Remote Energy Efficiency Solutions to Utilitieson November 10, 2020 at 8:56 am
CLEAResult, the leader in designing and implementing technology-enabled energy efficiency programs for utilities in North America, has signed a multi-year agreement with Streem expanding its use of St ...
- CLEAResult Announces Multi-Year Agreement with Streem to Deliver AR- and AI-Enabled Remote ...on November 10, 2020 at 8:47 am
Together, CLEAResult and Streem are leading the utility industry in leveraging AR and AI to provide valuable solutions -- and energy savings -- to consumers,” said CLEAResult CEO Scott Boose. “Using ...
Go deeper with Google Headlines on:
AI energy efficiency
Go deeper with Bing News on:
Machine learning energy use
- Report highlights potential of machine learning offshore Norwayon November 12, 2020 at 1:36 pm
A new DNV GL report, commissioned by OG21, demonstrates how machine learning should be developed, adopted, and scaled more quickly in the Norwegian continental shelf.
- Machine learning for making machines: Applying visual search to mechanical partson November 12, 2020 at 1:15 pm
Computer vision researchers use machine learning to train computers in visually recognizing objects – but very few apply machine learning to mechanical parts such as gearboxes, bearings, brakes, ...
- OctoML Announces Speaker Line-Up for 3rd Annual Apache TVM Conference Focused on Advances in Deep Learning Compilation and Optimizationon November 12, 2020 at 9:16 am
OctoML, the MLOps automation company for superior model performance, portability and productivity, today announced the speaker line-up for the Apache TVM and Deep Learning Compilation Conference. The ...
- How Amazon Retail Systems Run Machine Learning Predictions with Apache Sparkon November 12, 2020 at 8:54 am
Amazon Retail systems uses Apache Spark with Deep Java Library (DJL) to build propensity models to drive customer experience.
- Innovative machine-learning approach for future diagnostic advances in Parkinson's diseaseon November 12, 2020 at 8:50 am
Parkinson's disease (PD) is the second most common neurodegenerative disease, with patient numbers being expected to double worldwide in the next 20 years. The detailed molecular and cellular ...