View
4
Download
0
Category
Preview:
Citation preview
Mixed-Signal Techniques for Embedded
Machine Learning Systems
Boris Murmann
June 16, 2019
Applications
Speed of response
Bandwidth utilized
Privacy
Power consumed
CloudEdge
Conversational
Interfaces
…natural?
…seamless?
…real-time?
2
Task Complexity, Memory and Classification Energy
Face Detection
SVM
Digit RecognitionImage Classification
10 classes
Binary CNN
Image Classification
1000 classes
MobileNet v1
Self-Driving
ResNet-50 HD
pJ
nJ
µJ
mJ
J
3
Task Complexity, Memory and Classification Energy
pJ
nJ
µJ
mJ
J
My main interest
4
Edge Inference System
5
A/D
A/D
Sensors
Wake-up
Detector
Pre-
Processin
g
Deep
Neural
Network
DRA
M
SRA
M
Buffer
PE
Array
Register
File
Compute
MAC
Opportunities for Analog/Mixed-Signal Design
6
A/D
A/D
Sensors
Wake-up
Detector
Pre-
Processin
g
Deep
Neural
Network
DRA
M
SRA
M
Buffer
PE
Array
Register
File
Compute
MAC
Analog
Feature
Extraction
Analog
MAC
In-Memory
Computing
Outline
7
Data-Compressive Imager for Object Detection
› Omid-Zohoor & Young, TCSVT 2018 & ISSCC 2019
Mixed-Signal ConvNet
› Bankman, ISSCC 2018 & JSSC 2019
RRAM-based ConvNet with In-Memory Compute
› Ongoing work
Wake-Up Detector with Hand-Crafted Features
Feature
Extractor
Simple
Classifier
Sensor
Signal LabelA/D
Training
Algorithm
Labeled
Training Data
High-dimensional
data
Low-dimensional
representation
Data Deluge
8
Analog Feature Extractor
Low-rate and/or low–resolution ADC
Low data rate digital I/O
Reduced memory requirements
Feature
Extractor
Simple
Classification
Sensor
SignalLabelA/D
Training
Algorithm
Labeled
Training Data
Low-dimensional
representation
9
Histogram of Oriented Gradients
10
Analog Gradient Computation
H1 H2𝐺𝐻 =
V1 V2𝐺𝑉 =
horizontal
vertical
V2
H
1
PH
2
V1
𝐺𝐻 = 400𝑚𝑉 − 100𝑚𝑉 = 𝟑𝟎𝟎𝒎𝑽
𝐺𝐻 =1
4400𝑚𝑉 −
1
4100𝑚𝑉 = 𝟕𝟓𝒎𝑽
Dark patch
Bright patch
11
High Dynamic Range Images
12
Gradient magnitude varies
significantly across image
Would require high-
resolution ADCs (≥ 9b) to
digitize computed gradients
› But, we want to produce
as little data as possible
Ratio-Based (“Log”) Gradients
V2
H
1
PH
2
V1
H1
H2
𝐺𝐻 =V1
V2
𝐺𝑉 =
H1
H2
𝐺𝐻 = =𝛼 ×
𝛼 ×
H1
H2
Illumination Invariant
13
Ratio Quantization
V2
H
1
PH
2
V1
H1
H2
𝐺𝐻 =
V1
V2
𝐺𝑉 =
𝑉𝑝𝑖𝑥1𝑉𝑝𝑖𝑥2
1
22
Negative
edge
Positive
edge
No
edge
*Empirically determined
thresholds
𝑉𝑝𝑖𝑥1𝑉𝑝𝑖𝑥2
1
22
1.5b
2.75b
4.21.31
1.3
1
4.2
H1
H2
V1
V2
𝐺𝑉 = 𝑄𝐺𝐻 = 𝑄
14
HOG Feature Compression with 1.5b Gradients
Quant
Fewer angle bins
Quantizing histogram
magnitudes
25× less data in HOG
features compared to 8-bit
image
15
Log vs. Linear GradientsLess Illu
min
atio
n
16
Log vs. Linear GradientsLess Illu
min
atio
n
17
Log vs. Linear GradientsLess Illu
min
atio
n
18
Prototype Chip
19
Row Buffers with Pixel Binning (Image Pyramid)
20
Ratio-to-Digital Converter (RDC)
21
Data-Driven Spec Derivation
22
𝐻1 + 𝝐𝟏𝐻2 + 𝝐𝟐
Chip Summary
• 0.13 µm CIS 1P4M
• 5µm 4T pixels
• QVGA 320(V) x 240(H)
• 229 mW @ 30 FPS
Supply Voltages
Pixel: 2.5V
Analog: 1.5V, 2.5V
Digital: 0.9V
23
Sample Images
24
25
Results using Deformable Parts Model detection & custom database (PascalRAW)
Comparison to State of the Art
26
Information Preservation
27
Use Log Gradients as ConvNet Input?
28
Ongoing work; comparable performance using ResNet-10 (PascalRaw dataset)
Digital ConvNet Fabric
Can We Play Mixed-Signal Tricks in a ConvNet?
29
BinaryNet
Courbariaux et al., NIPS 2016
Weights and activations constrained to +1
and -1, multiplication becomes XNOR
Minimizes D/A and A/D overhead
Nice option for small/medium-size
problems and mixed-signal exploration
30
Bankman et al., ISSCC 2018
Mixed-Signal Binary CNN Processor
1. Binary CNN with “CMOS-inspired” topology, engineered for minimal circuit-level path loading
2. Hardware architecture amortizes memory access across many computations, with all memory on chip (328 KB)
3. Energy-efficient switched-capacitor neuron for wide vector summation, replacing digital adder tree
31
CIFAR-10
Zhao et al., FPGA 2017
Original BinaryNet Topology
88.54% accuracy on CIFAR-10
1.67 MB weight memory (68% FC layers)
27.9 mJ/classification with FPGA
32
Mixed-Signal BinaryNet Topology
Sacrificed accuracy for regularity and energy efficiency
86.05% accuracy on CIFAR-10
328 KB weight memory
3.8 mJ per classification
33
Neuron
34
Naïve Sequential Computation
35
Weight-Stationary
256
x2 (north/south)
x4 (neuron mux)
36
Weight-Stationary and Data-ParallelParallel
broadcast
37
Complete Architecture
38
𝑧 = sgn
𝑖=0
1023
𝑤𝑖𝑥𝑖 + 𝑏 𝑧
𝑤 𝑏Filter Weights
𝑥Image Patch Filter Bias
9b sign-magnitude
𝑏 = −1 𝑠
𝑖=0
7
2𝑖𝑚𝑖
Output
2×2×256 2×2×256
Neuron Function
39
𝑣diff𝑉𝐷𝐷
=𝐶𝑢𝐶𝑡𝑜𝑡
𝑖=0
1023
𝑤𝑖𝑥𝑖 + 𝑏
𝑏 = −1 𝑠
𝑖=0
7
2𝑖𝑚𝑖
Switched-Capacitor Implementation
Bias & offset cal.Weights x inputs
Batch normalization
folded into weight
signs and bias
40
Behavioral Simulations
41
Significant margin in noise, offset, and mismatch (VFS = 460 mV)
𝐶𝑢 = 1 fF𝑣𝑜𝑠 = 4.6 mV RMS𝑣𝑛 = 460 µV RMS
“Memory-Cell-Like” Processing Element
1 fF metal-oxide-metal fringe capacitorStandard-cell-based
42 transistors
24107 F2
42
TSMC 28nm HPL 1P8M
6 mm2 area
328 KB SRAM
10 MHz clock
Supply Voltages
VDD – Digital Logic, 0.6V – 1.0V
VMEM – SRAM, 0.53V – 1.0V
VNEU – Neuron Array, 0.6V
VCOMP – Comparators, 0.8V
Die Photo
43
μ = 86.05%
σ = 0.40%
Measured Classification Accuracy
10 chips, 180 runs each through
10,000 CIFAR-10 test images
VDD = 0.8V, VMEM = 0.8V
3.8 μJ/classification
237 FPS, 899 μW
0.43 μJ in 1.8V I/O
Mean accuracy μ = 86.05% same
as perfect digital model
44
Comparison to Synthesized Digital
BinarEye(Moons et al., CICC 2018)
Synthesized Digital Mixed-Signal
45
Digital vs. Mixed-Signal Binary CNN Processor
Synthesized
Digital
Moons et al.
CICC 2018
Hand-Designed
Digital
(Projected)
Mixed-Signal
Bankman et al.
ISSCC 2018
Energy @
86.05%
CIFAR-10
46
CIFAR-10 Energy vs. Accuracy Neuromorphic› [1] TrueNorth, Esser PNAS
2016
GPU› [2] Zhao FPGA 2017
FPGA› [2] Zhao FPGA 2017
› [3] Umuroglu FPGA 2017
MCU› [4] CMSIS-NN, Lai arXiv 2018
Memory-like, mixed-signal› [5] Bankman ISSCC 2018
BinarEye, digital› [6] Moons CICC 2018
In-memory, mixed-signal› [7] Jia arXiv 2018
*energy excludes off-chip DRAM
47
Limitations of Mixed-Signal BinaryNet
48
Poor programmability
Relatively limited accuracy (even on CIFAR-10) due to 1b arithmetic
Energy advantage over customized digital is not revolutionary
› Same SRAM, essentially same dataflow
Need a more “analog” memory system to unleash larger gains
› In-memory computing
BinaryNet Synapse versus Resistive RAM
• TBD
• 25 F2
• Multi-bit (?)
49
• 0.93 fJ per 1b-MAC in 28 nm
• 24107 F2
• Single-bit
Tsai, 2018
Matrix-Vector Multiplication with Resistive Cells
50
Typically use two cells to
achieve pos/neg weights
(other schemes possible)
256 columns
…
10
24
row
s
25 F2 cell @ F = 90 nm
Side length s = 0.45 mm
2 x 1024 x 0.45 um x 256 x 0.45 um = 0.106 mm2
per layer
0.84 mm2 for the complete 8-layer network
RRAM Density – BinaryNet Example
51
(2x)
Ongoing Research
52
What is the best architecture?
How many levels can be stored in each cell?
What is the most efficient readout?
Can we cope with nonidealities using training techniques?
53
(Content deleted for online distribution)
54
(Content deleted for online distribution)
55
(Content deleted for online distribution)
56
(Content deleted for online distribution)
VGG-7 Experiment (4.8 Million Parameters)
57
8192
10
3×3×128
3×3×128
3×3×256 3×3×512
3×3×2563×3×3
2-bit weights, 2-bit
activations
Accuracy on CIFAR-10
2b quantization only: 93%
2b quantization + RRAM/ADC model: 92%
Work in progress!
Optimal energy/MAC = 0.38 fJ
Energy Model for Column in Conv6 Layer
58
Summary
59
Analog feature extraction is attractive for wake-up detectors
Adding analog compute in ConvNets can be beneficial when it
simultaneously lets us reduce data movement
› In-memory analog compute looks most promising
› Can consider SRAM or emerging memories (e.g. RRAM)
Expect significant progress as more application drivers for “machine
learning at the edge” emerge
Recommended