VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

Arash Ardakani, Francois Leduc-Primeau, Naoya Onizawa, Takahiro Hanyu, Warren J. Gross

Research output: Contribution to journalArticlepeer-review

101 Citations (Scopus)

Abstract

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing (SC) has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral SC. The proposed architecture has been implemented on a Virtex7 field-programmable gate array, resulting in 45% and 62% average reductions in area and latency compared with the best reported architecture in the literature. We also synthesize the circuits in a 65-nm CMOS technology, and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared with the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation that yields 33% reduction in energy consumption with respect to the binary radix implementation without any compromise on performance.

Original languageEnglish
Article number7839313
Pages (from-to)2688-2699
Number of pages12
JournalIEEE Transactions on Very Large Scale Integration (VLSI) Systems
Volume25
Issue number10
DOIs
Publication statusPublished - 2017 Oct

Keywords

  • Deep neural network (DNN)
  • VLSI
  • hardware implementation
  • integral stochastic computation
  • machine learning
  • pattern recognition

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing'. Together they form a unique fingerprint.

Cite this