Embed Size (px)
Citation preview
Towards a Bayesian Approach for Assessing FaultTolerance of Deep Neural Networks
Subho S. Banerjee§, James Cyriac†, Saurabh Jha§, Zbigniew T. Kalbarczyk† and Ravishankar K. Iyer†§§Department of Computer Science, †Department of Electrical and Computer Engineering,
University of Illinois at Urbana-Champaign, Urbana IL 61801, USA.
Abstract—This paper presents Bayesian Deep Learning basedFault Injection (BDLFI), a novel methodology for fault injectionin neural networks (NNs) and more generally differentiableprograms. BDLFI uses (1) Bayesian Deep Learning to modelthe propagation of faults, and (2) Markov Chain Monte Carloinference to quantify the effect of faults on the outputs of aNN. We demonstrate BDLFI on two representative networks andpresent our results that challenge pre-existing results in the field.
I. INTRODUCTION
For the past decade, machine learning (ML) and particularlydeep neural network (DNN) models have had tremendoussuccess in solving problems in image classification, speechrecognition, text translation, and game playing. This ubiquitoussuccess has led to the application of a large number of thesemodels in safety critical systems, e.g., autonomous vehicles,and industrial control systems, where reliability and safetyguarantees are paramount. Traditional approaches for validationof these guarantees have relied on random Fault Injection(FI) [1] to test the resilience of a software/hardware platform inthe presence of faults. However, these traditional approaches arefundamentally challenged due to (1) the enormous space of faultlocations and program states that must be injected/investigated;(2) the need for significant system support (i.e., debug capabil-ities like ptrace support) to build system-specific injectors;and (3) the inability to provide statistical guarantees about“completeness” beyond performed injections. This presents asignificant challenge for today’s popular DNN models thatscale to millions of parameters and employ special purpose(often proprietary) hardware accelerators that do not support awide range of debugging capabilities (e.g., GPUs, TPUs).
In this paper, we address the above challenges by presentingBayesian Deep Learning based Fault Injection (BDLFI), amethodology for fault injection in neural networks (NNs).BDLFI uses (1) Bayesian Deep Learning (BDL) to modelthe propagation of faults, and (2) Markov Chain Monte Carlo(MCMC) inference to quantify the effect of faults on theoutputs of a NN [2]. BDL and MCMC provide a formalismand inference method to quantify uncertainty in the inputs,parameters (referred to as weights), activations, and outputs ofDNN models. BDLFI leverages this aspect of BDL and modelsuncertainty as being caused by transient hardware faults.
Advantages of BDLFI over traditional fault injection tech-niques include: (1) the ability to quantify “completeness” of aninjection campaign (i.e., when further injections do not changemeasured hypothesis) using MCMC-mixing; and (2) the abilityto use algorithmic acceleration techniques; (3) the ability touse specialized hardware (without needing system support todo injections) to directly accelerate inference and hence thefault injection campaigns. As a result, we believe that BDLFIcan subsume current source-level (e.g., [3]) and debugger-level(e.g., [4]) FIs as a method for rapidly validating the resilience ofML/DNN models executing on embedded accelerator platforms.BFI can be used to inject faults into programs other than neuralnetworks, with the only assumption being that of end-to-enddifferentiability (i.e., one can calculate the gradient of theoutput of a program over its input).
II. BAYESIAN FAULT INJECTOR
We now describe the operation of the BDLFI on a multi-layerperceptron (MLP) network shown as 1 in Fig. 1.
Fault Model. We consider transient faults in the memory
Original Classification Boundary
log(Error) Probability Due to Faults
3 Bayesian Inference
Weights learned from golden run…
Neural Network
Inputs
Output
FC Layer
Softmax
1 2 Bayesian Network based Failure Model
…b1<latexit sha1_base64="2Av6VIrMFFmteisJKMRbsrCLdOk=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lKTXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckblil11a/XGpY3sqr1ERpquW2siJ1cqkKM9Kr8PxxFJQio04VipgWPH2kux1IxwuigNE0VjTGZ4QgeGChxS5aXLUxfo3ChjFETSlNBoqX6fSHGo1Dz0TWeI9VT99jLxL2+Q6KDppUzEiaaCrBYFCUc6QtnfaMwkJZrPDcFEMnMrIlMsMdEmnZIJ4etT9D/p1qqO4Tf1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3Ty8XjaI=</latexit><latexit sha1_base64="2Av6VIrMFFmteisJKMRbsrCLdOk=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lKTXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckblil11a/XGpY3sqr1ERpquW2siJ1cqkKM9Kr8PxxFJQio04VipgWPH2kux1IxwuigNE0VjTGZ4QgeGChxS5aXLUxfo3ChjFETSlNBoqX6fSHGo1Dz0TWeI9VT99jLxL2+Q6KDppUzEiaaCrBYFCUc6QtnfaMwkJZrPDcFEMnMrIlMsMdEmnZIJ4etT9D/p1qqO4Tf1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3Ty8XjaI=</latexit><latexit sha1_base64="2Av6VIrMFFmteisJKMRbsrCLdOk=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lKTXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckblil11a/XGpY3sqr1ERpquW2siJ1cqkKM9Kr8PxxFJQio04VipgWPH2kux1IxwuigNE0VjTGZ4QgeGChxS5aXLUxfo3ChjFETSlNBoqX6fSHGo1Dz0TWeI9VT99jLxL2+Q6KDppUzEiaaCrBYFCUc6QtnfaMwkJZrPDcFEMnMrIlMsMdEmnZIJ4etT9D/p1qqO4Tf1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3Ty8XjaI=</latexit><latexit sha1_base64="2Av6VIrMFFmteisJKMRbsrCLdOk=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lKTXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckblil11a/XGpY3sqr1ERpquW2siJ1cqkKM9Kr8PxxFJQio04VipgWPH2kux1IxwuigNE0VjTGZ4QgeGChxS5aXLUxfo3ChjFETSlNBoqX6fSHGo1Dz0TWeI9VT99jLxL2+Q6KDppUzEiaaCrBYFCUc6QtnfaMwkJZrPDcFEMnMrIlMsMdEmnZIJ4etT9D/p1qqO4Tf1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3Ty8XjaI=</latexit>
b2<latexit sha1_base64="BaEcf+3zEns+kctWYsB36EOrFF0=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lCbXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckflil2tu7XGpY3sqr1ERpr1uttETq5UIEd7VH4fjiOShFRowrFSA8eOtZdiqRnhdFEaJorGmMzwhA4MFTikykuXpy7QuVHGKIikKaHRUv0+keJQqXnom84Q66n67WXiX94g0UHTS5mIE00FWS0KEo50hLK/0ZhJSjSfG4KJZOZWRKZYYqJNOiUTwten6H/SdauO4Te1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3TzCbjaM=</latexit><latexit sha1_base64="BaEcf+3zEns+kctWYsB36EOrFF0=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lCbXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckflil2tu7XGpY3sqr1ERpr1uttETq5UIEd7VH4fjiOShFRowrFSA8eOtZdiqRnhdFEaJorGmMzwhA4MFTikykuXpy7QuVHGKIikKaHRUv0+keJQqXnom84Q66n67WXiX94g0UHTS5mIE00FWS0KEo50hLK/0ZhJSjSfG4KJZOZWRKZYYqJNOiUTwten6H/SdauO4Te1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3TzCbjaM=</latexit><latexit sha1_base64="BaEcf+3zEns+kctWYsB36EOrFF0=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lCbXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckflil2tu7XGpY3sqr1ERpr1uttETq5UIEd7VH4fjiOShFRowrFSA8eOtZdiqRnhdFEaJorGmMzwhA4MFTikykuXpy7QuVHGKIikKaHRUv0+keJQqXnom84Q66n67WXiX94g0UHTS5mIE00FWS0KEo50hLK/0ZhJSjSfG4KJZOZWRKZYYqJNOiUTwten6H/SdauO4Te1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3TzCbjaM=</latexit><latexit sha1_base64="BaEcf+3zEns+kctWYsB36EOrFF0=">AAAB6nicdVBNS8NAEJ3Ur1q/qh69LFbBU0lCbXssePFY0X5AG8pmu2mXbjZhdyOU0J/gxYMiXv1F3vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82VXm9+6pVCwSd3oeUy/EE8ECRrA20q0/ckflil2tu7XGpY3sqr1ERpr1uttETq5UIEd7VH4fjiOShFRowrFSA8eOtZdiqRnhdFEaJorGmMzwhA4MFTikykuXpy7QuVHGKIikKaHRUv0+keJQqXnom84Q66n67WXiX94g0UHTS5mIE00FWS0KEo50hLK/0ZhJSjSfG4KJZOZWRKZYYqJNOiUTwten6H/SdauO4Te1Sussj6MIJ3AKF+BAA1pwDW3oAIEJPMATPFvcerRerNdVa8HKZ47hB6y3TzCbjaM=</latexit>
b32<latexit sha1_base64="DsxGgX3CFCjKC0qcvoL0tcJiWSY=">AAAB7nicdVBNS8NAEJ3Ur1q/qh69LFbBU0libXssePFYwX5AG8pmu2mXbjZhdyOU0B/hxYMiXv093vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82XXm9+6pVCwSd3oeUy/EE8ECRrA2Us8fpZfuojQqV+xq3a01rmxkV+0lMtKs190mcnKlAjnao/L7cByRJKRCE46VGjh2rL0US80Ip4vSMFE0xmSGJ3RgqMAhVV66PHeBzo0yRkEkTQmNlur3iRSHSs1D33SGWE/Vby8T//IGiQ6aXspEnGgqyGpRkHCkI5T9jsZMUqL53BBMJDO3IjLFEhNtEspC+PoU/U+6btUx/LZWaZ3lcRThBE7hAhxoQAtuoA0dIDCDB3iCZyu2Hq0X63XVWrDymWP4AevtE58jjwA=</latexit><latexit sha1_base64="DsxGgX3CFCjKC0qcvoL0tcJiWSY=">AAAB7nicdVBNS8NAEJ3Ur1q/qh69LFbBU0libXssePFYwX5AG8pmu2mXbjZhdyOU0B/hxYMiXv093vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82XXm9+6pVCwSd3oeUy/EE8ECRrA2Us8fpZfuojQqV+xq3a01rmxkV+0lMtKs190mcnKlAjnao/L7cByRJKRCE46VGjh2rL0US80Ip4vSMFE0xmSGJ3RgqMAhVV66PHeBzo0yRkEkTQmNlur3iRSHSs1D33SGWE/Vby8T//IGiQ6aXspEnGgqyGpRkHCkI5T9jsZMUqL53BBMJDO3IjLFEhNtEspC+PoU/U+6btUx/LZWaZ3lcRThBE7hAhxoQAtuoA0dIDCDB3iCZyu2Hq0X63XVWrDymWP4AevtE58jjwA=</latexit><latexit sha1_base64="DsxGgX3CFCjKC0qcvoL0tcJiWSY=">AAAB7nicdVBNS8NAEJ3Ur1q/qh69LFbBU0libXssePFYwX5AG8pmu2mXbjZhdyOU0B/hxYMiXv093vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82XXm9+6pVCwSd3oeUy/EE8ECRrA2Us8fpZfuojQqV+xq3a01rmxkV+0lMtKs190mcnKlAjnao/L7cByRJKRCE46VGjh2rL0US80Ip4vSMFE0xmSGJ3RgqMAhVV66PHeBzo0yRkEkTQmNlur3iRSHSs1D33SGWE/Vby8T//IGiQ6aXspEnGgqyGpRkHCkI5T9jsZMUqL53BBMJDO3IjLFEhNtEspC+PoU/U+6btUx/LZWaZ3lcRThBE7hAhxoQAtuoA0dIDCDB3iCZyu2Hq0X63XVWrDymWP4AevtE58jjwA=</latexit><latexit sha1_base64="DsxGgX3CFCjKC0qcvoL0tcJiWSY=">AAAB7nicdVBNS8NAEJ3Ur1q/qh69LFbBU0libXssePFYwX5AG8pmu2mXbjZhdyOU0B/hxYMiXv093vw3btoIKvpg4PHeDDPz/JgzpW37wyqsrW9sbhW3Szu7e/sH5cOjrooSSWiHRDySfR8rypmgHc00p/1YUhz6nPb82XXm9+6pVCwSd3oeUy/EE8ECRrA2Us8fpZfuojQqV+xq3a01rmxkV+0lMtKs190mcnKlAjnao/L7cByRJKRCE46VGjh2rL0US80Ip4vSMFE0xmSGJ3RgqMAhVV66PHeBzo0yRkEkTQmNlur3iRSHSs1D33SGWE/Vby8T//IGiQ6aXspEnGgqyGpRkHCkI5T9jsZMUqL53BBMJDO3IjLFEhNtEspC+PoU/U+6btUx/LZWaZ3lcRThBE7hAhxoQAtuoA0dIDCDB3iCZyu2Hq0X63XVWrDymWP4AevtE58jjwA=</latexit>
bi ⇠ Bernoulli(p) 8 i 2 [1, 32]<latexit sha1_base64="IywMQW6SHH8sbuIdGXZphe5eU1Y=">AAACGnicdVBNSwMxFMz6bf2qevQSLIKClN2t1h5FLx4VrArdpWTTVxuazS7JW7Es9W948a948aCIN/HivzGtFVR0IDDMzCPvTZRKYdB1352x8YnJqemZ2cLc/MLiUnF55cwkmeZQ54lM9EXEDEihoI4CJVykGlgcSTiPuocD//wKtBGJOsVeCmHMLpVoC87QSs2iFzUFDYyIaYBwjfkBaJVkUor+Zrp1E7QTzaS8sRGhaMPbrvhhs1hyy1V/Z2/XpW7ZHWJAatWqX6PeSCmREY6bxdeglfAsBoVcMmManptimDONgkvoF4LMQMp4l11Cw1LFYjBhPjytTzes0qJ2DfsU0qH6fSJnsTG9OLLJmGHH/PYG4l9eI8N2LcyFSjMExT8/ameSYkIHPdGW0MBR9ixhXAu7K+UdphlH22bBlvB1Kf2fnPllr1L2TnZK+3RUxwxZI+tkk3hkj+yTI3JM6oSTW3JPHsmTc+c8OM/Oy2d0zBnNrJIfcN4+AI86oG8=</latexit>
e = {bi}<latexit sha1_base64="85QdBqTtoimYxixxIfOv+Znlz38=">AAAB8nicdVBNS8NAEN3Ur1q/qh69LBbBU0lE23oQCl48VrAfkISy2W7apZtN2J0IJfRnePGgiFd/jTf/jZs2goo+GHi8N8PMvCARXINtf1illdW19Y3yZmVre2d3r7p/0NNxqijr0ljEahAQzQSXrAscBBskipEoEKwfTK9zv3/PlOaxvINZwvyIjCUPOSVgJJfhK+xlwZB782G1ZtcvnKbdamC7bi+Qk8Zlo+Vgp1BqqEBnWH33RjFNIyaBCqK169gJ+BlRwKlg84qXapYQOiVj5hoqScS0ny1OnuMTo4xwGCtTEvBC/T6RkUjrWRSYzojARP/2cvEvz00hbPkZl0kKTNLlojAVGGKc/49HXDEKYmYIoYqbWzGdEEUomJQqJoSvT/H/pHdWdwy/Pa+1cRFHGR2hY3SKHNREbXSDOqiLKIrRA3pCzxZYj9aL9bpsLVnFzCH6AevtE6NOkLs=</latexit><latexit sha1_base64="85QdBqTtoimYxixxIfOv+Znlz38=">AAAB8nicdVBNS8NAEN3Ur1q/qh69LBbBU0lE23oQCl48VrAfkISy2W7apZtN2J0IJfRnePGgiFd/jTf/jZs2goo+GHi8N8PMvCARXINtf1illdW19Y3yZmVre2d3r7p/0NNxqijr0ljEahAQzQSXrAscBBskipEoEKwfTK9zv3/PlOaxvINZwvyIjCUPOSVgJJfhK+xlwZB782G1ZtcvnKbdamC7bi+Qk8Zlo+Vgp1BqqEBnWH33RjFNIyaBCqK169gJ+BlRwKlg84qXapYQOiVj5hoqScS0ny1OnuMTo4xwGCtTEvBC/T6RkUjrWRSYzojARP/2cvEvz00hbPkZl0kKTNLlojAVGGKc/49HXDEKYmYIoYqbWzGdEEUomJQqJoSvT/H/pHdWdwy/Pa+1cRFHGR2hY3SKHNREbXSDOqiLKIrRA3pCzxZYj9aL9bpsLVnFzCH6AevtE6NOkLs=</latexit><latexit sha1_base64="85QdBqTtoimYxixxIfOv+Znlz38=">AAAB8nicdVBNS8NAEN3Ur1q/qh69LBbBU0lE23oQCl48VrAfkISy2W7apZtN2J0IJfRnePGgiFd/jTf/jZs2goo+GHi8N8PMvCARXINtf1illdW19Y3yZmVre2d3r7p/0NNxqijr0ljEahAQzQSXrAscBBskipEoEKwfTK9zv3/PlOaxvINZwvyIjCUPOSVgJJfhK+xlwZB782G1ZtcvnKbdamC7bi+Qk8Zlo+Vgp1BqqEBnWH33RjFNIyaBCqK169gJ+BlRwKlg84qXapYQOiVj5hoqScS0ny1OnuMTo4xwGCtTEvBC/T6RkUjrWRSYzojARP/2cvEvz00hbPkZl0kKTNLlojAVGGKc/49HXDEKYmYIoYqbWzGdEEUomJQqJoSvT/H/pHdWdwy/Pa+1cRFHGR2hY3SKHNREbXSDOqiLKIrRA3pCzxZYj9aL9bpsLVnFzCH6AevtE6NOkLs=</latexit><latexit sha1_base64="85QdBqTtoimYxixxIfOv+Znlz38=">AAAB8nicdVBNS8NAEN3Ur1q/qh69LBbBU0lE23oQCl48VrAfkISy2W7apZtN2J0IJfRnePGgiFd/jTf/jZs2goo+GHi8N8PMvCARXINtf1illdW19Y3yZmVre2d3r7p/0NNxqijr0ljEahAQzQSXrAscBBskipEoEKwfTK9zv3/PlOaxvINZwvyIjCUPOSVgJJfhK+xlwZB782G1ZtcvnKbdamC7bi+Qk8Zlo+Vgp1BqqEBnWH33RjFNIyaBCqK169gJ+BlRwKlg84qXapYQOiVj5hoqScS0ny1OnuMTo4xwGCtTEvBC/T6RkUjrWRSYzojARP/2cvEvz00hbPkZl0kKTNLlojAVGGKc/49HXDEKYmYIoYqbWzGdEEUomJQqJoSvT/H/pHdWdwy/Pa+1cRFHGR2hY3SKHNREbXSDOqiLKIrRA3pCzxZYj9aL9bpsLVnFzCH6AevtE6NOkLs=</latexit>
W<latexit sha1_base64="PGXoi0N1ZkLl416klmFGIBaVTfc=">AAAB8XicbVDLSsNAFL2pr1pfVZduBqvgqiQi6LLgxmUF+8A2lMn0ph06mYSZiVBC/8KNC0Xc+jfu/BsnbRbaemDgcM69zLknSATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zhVDFssFrHqBlSj4BJbhhuB3UQhjQKBnWBym/udJ1Sax/LBTBP0IzqSPOSMGis99iNqxkGYdWaDas2tu3OQVeIVpAYFmoPqV38YszRCaZigWvc8NzF+RpXhTOCs0k81JpRN6Ah7lkoaofazeeIZObfKkISxsk8aMld/b2Q00noaBXYyT6iXvVz8z+ulJrzxMy6T1KBki4/CVBATk/x8MuQKmRFTSyhT3GYlbEwVZcaWVLEleMsnr5L2Zd2z/P6q1jgr6ijDCZzCBXhwDQ24gya0gIGEZ3iFN0c7L86787EYLTnFzjH8gfP5A8IQkN0=</latexit><latexit sha1_base64="PGXoi0N1ZkLl416klmFGIBaVTfc=">AAAB8XicbVDLSsNAFL2pr1pfVZduBqvgqiQi6LLgxmUF+8A2lMn0ph06mYSZiVBC/8KNC0Xc+jfu/BsnbRbaemDgcM69zLknSATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zhVDFssFrHqBlSj4BJbhhuB3UQhjQKBnWBym/udJ1Sax/LBTBP0IzqSPOSMGis99iNqxkGYdWaDas2tu3OQVeIVpAYFmoPqV38YszRCaZigWvc8NzF+RpXhTOCs0k81JpRN6Ah7lkoaofazeeIZObfKkISxsk8aMld/b2Q00noaBXYyT6iXvVz8z+ulJrzxMy6T1KBki4/CVBATk/x8MuQKmRFTSyhT3GYlbEwVZcaWVLEleMsnr5L2Zd2z/P6q1jgr6ijDCZzCBXhwDQ24gya0gIGEZ3iFN0c7L86787EYLTnFzjH8gfP5A8IQkN0=</latexit><latexit sha1_base64="PGXoi0N1ZkLl416klmFGIBaVTfc=">AAAB8XicbVDLSsNAFL2pr1pfVZduBqvgqiQi6LLgxmUF+8A2lMn0ph06mYSZiVBC/8KNC0Xc+jfu/BsnbRbaemDgcM69zLknSATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zhVDFssFrHqBlSj4BJbhhuB3UQhjQKBnWBym/udJ1Sax/LBTBP0IzqSPOSMGis99iNqxkGYdWaDas2tu3OQVeIVpAYFmoPqV38YszRCaZigWvc8NzF+RpXhTOCs0k81JpRN6Ah7lkoaofazeeIZObfKkISxsk8aMld/b2Q00noaBXYyT6iXvVz8z+ulJrzxMy6T1KBki4/CVBATk/x8MuQKmRFTSyhT3GYlbEwVZcaWVLEleMsnr5L2Zd2z/P6q1jgr6ijDCZzCBXhwDQ24gya0gIGEZ3iFN0c7L86787EYLTnFzjH8gfP5A8IQkN0=</latexit><latexit sha1_base64="PGXoi0N1ZkLl416klmFGIBaVTfc=">AAAB8XicbVDLSsNAFL2pr1pfVZduBqvgqiQi6LLgxmUF+8A2lMn0ph06mYSZiVBC/8KNC0Xc+jfu/BsnbRbaemDgcM69zLknSATXxnW/ndLa+sbmVnm7srO7t39QPTxq6zhVDFssFrHqBlSj4BJbhhuB3UQhjQKBnWBym/udJ1Sax/LBTBP0IzqSPOSMGis99iNqxkGYdWaDas2tu3OQVeIVpAYFmoPqV38YszRCaZigWvc8NzF+RpXhTOCs0k81JpRN6Ah7lkoaofazeeIZObfKkISxsk8aMld/b2Q00noaBXYyT6iXvVz8z+ulJrzxMy6T1KBki4/CVBATk/x8MuQKmRFTSyhT3GYlbEwVZcaWVLEleMsnr5L2Zd2z/P6q1jgr6ijDCZzCBXhwDQ24gya0gIGEZ3iFN0c7L86787EYLTnFzjH8gfP5A8IQkN0=</latexit>
W0 = e � W<latexit sha1_base64="QJX5pAz0z32O7UxHGxNLQUypiQo=">AAACCnicdZDLSgMxFIYzXmu9jbp0Ey2iqzIj2taFUHDjsoK9QGcomTTThmaSIckIZejaja/ixoUibn0Cd76NmXYUFf0h8POdc8g5fxAzqrTjvFtz8wuLS8uFleLq2vrGpr213VIikZg0sWBCdgKkCKOcNDXVjHRiSVAUMNIORhdZvX1DpKKCX+txTPwIDTgNKUbaoJ6950VID4MwbR9O4Dkk0BMxSxT8wpOeXXLKp27VqVWgU3amykzlrFJzoZuTEsjV6NlvXl/gJCJcY4aU6rpOrP0USU0xI5OilygSIzxCA9I1lqOIKD+dnjKBB4b0YSikeVzDKf0+kaJIqXEUmM5sQ/W7lsG/at1EhzU/pTxONOF49lGYMKgFzHKBfSoJ1mxsDMKSml0hHiKJsDbpFU0In5fC/03ruOwaf3VSqsM8jgLYBfvgCLigCurgEjRAE2BwC+7BI3iy7qwH69l6mbXOWfnMDvgh6/UDm7WaHg==</latexit><latexit sha1_base64="QJX5pAz0z32O7UxHGxNLQUypiQo=">AAACCnicdZDLSgMxFIYzXmu9jbp0Ey2iqzIj2taFUHDjsoK9QGcomTTThmaSIckIZejaja/ixoUibn0Cd76NmXYUFf0h8POdc8g5fxAzqrTjvFtz8wuLS8uFleLq2vrGpr213VIikZg0sWBCdgKkCKOcNDXVjHRiSVAUMNIORhdZvX1DpKKCX+txTPwIDTgNKUbaoJ6950VID4MwbR9O4Dkk0BMxSxT8wpOeXXLKp27VqVWgU3amykzlrFJzoZuTEsjV6NlvXl/gJCJcY4aU6rpOrP0USU0xI5OilygSIzxCA9I1lqOIKD+dnjKBB4b0YSikeVzDKf0+kaJIqXEUmM5sQ/W7lsG/at1EhzU/pTxONOF49lGYMKgFzHKBfSoJ1mxsDMKSml0hHiKJsDbpFU0In5fC/03ruOwaf3VSqsM8jgLYBfvgCLigCurgEjRAE2BwC+7BI3iy7qwH69l6mbXOWfnMDvgh6/UDm7WaHg==</latexit><latexit sha1_base64="QJX5pAz0z32O7UxHGxNLQUypiQo=">AAACCnicdZDLSgMxFIYzXmu9jbp0Ey2iqzIj2taFUHDjsoK9QGcomTTThmaSIckIZejaja/ixoUibn0Cd76NmXYUFf0h8POdc8g5fxAzqrTjvFtz8wuLS8uFleLq2vrGpr213VIikZg0sWBCdgKkCKOcNDXVjHRiSVAUMNIORhdZvX1DpKKCX+txTPwIDTgNKUbaoJ6950VID4MwbR9O4Dkk0BMxSxT8wpOeXXLKp27VqVWgU3amykzlrFJzoZuTEsjV6NlvXl/gJCJcY4aU6rpOrP0USU0xI5OilygSIzxCA9I1lqOIKD+dnjKBB4b0YSikeVzDKf0+kaJIqXEUmM5sQ/W7lsG/at1EhzU/pTxONOF49lGYMKgFzHKBfSoJ1mxsDMKSml0hHiKJsDbpFU0In5fC/03ruOwaf3VSqsM8jgLYBfvgCLigCurgEjRAE2BwC+7BI3iy7qwH69l6mbXOWfnMDvgh6/UDm7WaHg==</latexit><latexit sha1_base64="QJX5pAz0z32O7UxHGxNLQUypiQo=">AAACCnicdZDLSgMxFIYzXmu9jbp0Ey2iqzIj2taFUHDjsoK9QGcomTTThmaSIckIZejaja/ixoUibn0Cd76NmXYUFf0h8POdc8g5fxAzqrTjvFtz8wuLS8uFleLq2vrGpr213VIikZg0sWBCdgKkCKOcNDXVjHRiSVAUMNIORhdZvX1DpKKCX+txTPwIDTgNKUbaoJ6950VID4MwbR9O4Dkk0BMxSxT8wpOeXXLKp27VqVWgU3amykzlrFJzoZuTEsjV6NlvXl/gJCJcY4aU6rpOrP0USU0xI5OilygSIzxCA9I1lqOIKD+dnjKBB4b0YSikeVzDKf0+kaJIqXEUmM5sQ/W7lsG/at1EhzU/pTxONOF49lGYMKgFzHKBfSoJ1mxsDMKSml0hHiKJsDbpFU0In5fC/03ruOwaf3VSqsM8jgLYBfvgCLigCurgEjRAE2BwC+7BI3iy7qwH69l6mbXOWfnMDvgh6/UDm7WaHg==</latexit> W0
<latexit sha1_base64="42X3G1RUbRppp153SYiIgbl8o3E=">AAAB8nicdVDLSgMxFM34rPVVdekmWEVXZUa0rbuCG5cV7AOmpWTSTBuaSYbkjlCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxIIbcN0PZ2l5ZXVtvbBR3Nza3tkt7e23jUo0ZS2qhNLdgBgmuGQt4CBYN9aMRIFgnWBynfmde6YNV/IOpjHrR2QkecgpASv5vYjAOAjTzulsUCq7lUuv5tar2K24c2SkelWte9jLlTLK0RyU3ntDRZOISaCCGON7bgz9lGjgVLBZsZcYFhM6ISPmWypJxEw/nUee4ROrDHGotH0S8Fz9vpGSyJhpFNjJLKL57WXiX56fQFjvp1zGCTBJFx+FicCgcHY/HnLNKIipJYRqbrNiOiaaULAtFW0JX5fi/0n7vOJZfntRbhzndRTQITpCZ8hDNdRAN6iJWogihR7QE3p2wHl0XpzXxeiSk+8coB9w3j4BeE6RRw==</latexit><latexit sha1_base64="42X3G1RUbRppp153SYiIgbl8o3E=">AAAB8nicdVDLSgMxFM34rPVVdekmWEVXZUa0rbuCG5cV7AOmpWTSTBuaSYbkjlCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxIIbcN0PZ2l5ZXVtvbBR3Nza3tkt7e23jUo0ZS2qhNLdgBgmuGQt4CBYN9aMRIFgnWBynfmde6YNV/IOpjHrR2QkecgpASv5vYjAOAjTzulsUCq7lUuv5tar2K24c2SkelWte9jLlTLK0RyU3ntDRZOISaCCGON7bgz9lGjgVLBZsZcYFhM6ISPmWypJxEw/nUee4ROrDHGotH0S8Fz9vpGSyJhpFNjJLKL57WXiX56fQFjvp1zGCTBJFx+FicCgcHY/HnLNKIipJYRqbrNiOiaaULAtFW0JX5fi/0n7vOJZfntRbhzndRTQITpCZ8hDNdRAN6iJWogihR7QE3p2wHl0XpzXxeiSk+8coB9w3j4BeE6RRw==</latexit><latexit sha1_base64="42X3G1RUbRppp153SYiIgbl8o3E=">AAAB8nicdVDLSgMxFM34rPVVdekmWEVXZUa0rbuCG5cV7AOmpWTSTBuaSYbkjlCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxIIbcN0PZ2l5ZXVtvbBR3Nza3tkt7e23jUo0ZS2qhNLdgBgmuGQt4CBYN9aMRIFgnWBynfmde6YNV/IOpjHrR2QkecgpASv5vYjAOAjTzulsUCq7lUuv5tar2K24c2SkelWte9jLlTLK0RyU3ntDRZOISaCCGON7bgz9lGjgVLBZsZcYFhM6ISPmWypJxEw/nUee4ROrDHGotH0S8Fz9vpGSyJhpFNjJLKL57WXiX56fQFjvp1zGCTBJFx+FicCgcHY/HnLNKIipJYRqbrNiOiaaULAtFW0JX5fi/0n7vOJZfntRbhzndRTQITpCZ8hDNdRAN6iJWogihR7QE3p2wHl0XpzXxeiSk+8coB9w3j4BeE6RRw==</latexit><latexit sha1_base64="42X3G1RUbRppp153SYiIgbl8o3E=">AAAB8nicdVDLSgMxFM34rPVVdekmWEVXZUa0rbuCG5cV7AOmpWTSTBuaSYbkjlCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxIIbcN0PZ2l5ZXVtvbBR3Nza3tkt7e23jUo0ZS2qhNLdgBgmuGQt4CBYN9aMRIFgnWBynfmde6YNV/IOpjHrR2QkecgpASv5vYjAOAjTzulsUCq7lUuv5tar2K24c2SkelWte9jLlTLK0RyU3ntDRZOISaCCGON7bgz9lGjgVLBZsZcYFhM6ISPmWypJxEw/nUee4ROrDHGotH0S8Fz9vpGSyJhpFNjJLKL57WXiX56fQFjvp1zGCTBJFx+FicCgcHY/HnLNKIipJYRqbrNiOiaaULAtFW0JX5fi/0n7vOJZfntRbhzndRTQITpCZ8hDNdRAN6iJWogihR7QE3p2wHl0XpzXxeiSk+8coB9w3j4BeE6RRw==</latexit>
x<latexit sha1_base64="6E1wU6rMmXaTO45JkRR27re54ZU=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tV8FQSEfRY8OKxBfsBbSib7aRdu9mE3Y1YQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOybl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJb/yMyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVZ9Sw3riq1szyOIpzAKVyAB9dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A3TWM3g==</latexit><latexit sha1_base64="6E1wU6rMmXaTO45JkRR27re54ZU=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tV8FQSEfRY8OKxBfsBbSib7aRdu9mE3Y1YQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOybl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJb/yMyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVZ9Sw3riq1szyOIpzAKVyAB9dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A3TWM3g==</latexit><latexit sha1_base64="6E1wU6rMmXaTO45JkRR27re54ZU=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tV8FQSEfRY8OKxBfsBbSib7aRdu9mE3Y1YQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOybl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJb/yMyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVZ9Sw3riq1szyOIpzAKVyAB9dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A3TWM3g==</latexit><latexit sha1_base64="6E1wU6rMmXaTO45JkRR27re54ZU=">AAAB6HicbZBNS8NAEIYn9avWr6pHL4tV8FQSEfRY8OKxBfsBbSib7aRdu9mE3Y1YQn+BFw+KePUnefPfuG1z0NYXFh7emWFn3iARXBvX/XYKa+sbm1vF7dLO7t7+QfnwqKXjVDFssljEqhNQjYJLbBpuBHYShTQKBLaD8e2s3n5EpXks780kQT+iQ8lDzqixVuOpX664VXcusgpeDhXIVe+Xv3qDmKURSsME1brruYnxM6oMZwKnpV6qMaFsTIfYtShphNrP5otOybl1BiSMlX3SkLn7eyKjkdaTKLCdETUjvVybmf/VuqkJb/yMyyQ1KNniozAVxMRkdjUZcIXMiIkFyhS3uxI2oooyY7Mp2RC85ZNXoXVZ9Sw3riq1szyOIpzAKVyAB9dQgzuoQxMYIDzDK7w5D86L8+58LFoLTj5zDH/kfP4A3TWM3g==</latexit> b0
<latexit sha1_base64="XSa9q8LMq8AN/ycpx8qQ8v3iUpE=">AAAB8nicdVDLSgMxFL3js9ZX1aWbYBVdlRnRtu4KblxWsA+YlpJJM21oJhmSjFCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxJxp47ofztLyyuraemGjuLm1vbNb2ttva5koQltEcqm6AdaUM0FbhhlOu7GiOAo47QST68zv3FOlmRR3ZhrTfoRHgoWMYGMlvxdhMw7CNDidDUplt3Lp1dx6FbkVd46MVK+qdQ95uVKGHM1B6b03lCSJqDCEY619z41NP8XKMMLprNhLNI0xmeAR9S0VOKK6n84jz9CJVYYolMo+YdBc/b6R4kjraRTYySyi/u1l4l+en5iw3k+ZiBNDBVl8FCYcGYmy+9GQKUoMn1qCiWI2KyJjrDAxtqWiLeHrUvQ/aZ9XPMtvL8qN47yOAhzCEZyBBzVowA00oQUEJDzAEzw7xnl0XpzXxeiSk+8cwA84b5+JEJFS</latexit><latexit sha1_base64="XSa9q8LMq8AN/ycpx8qQ8v3iUpE=">AAAB8nicdVDLSgMxFL3js9ZX1aWbYBVdlRnRtu4KblxWsA+YlpJJM21oJhmSjFCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxJxp47ofztLyyuraemGjuLm1vbNb2ttva5koQltEcqm6AdaUM0FbhhlOu7GiOAo47QST68zv3FOlmRR3ZhrTfoRHgoWMYGMlvxdhMw7CNDidDUplt3Lp1dx6FbkVd46MVK+qdQ95uVKGHM1B6b03lCSJqDCEY619z41NP8XKMMLprNhLNI0xmeAR9S0VOKK6n84jz9CJVYYolMo+YdBc/b6R4kjraRTYySyi/u1l4l+en5iw3k+ZiBNDBVl8FCYcGYmy+9GQKUoMn1qCiWI2KyJjrDAxtqWiLeHrUvQ/aZ9XPMtvL8qN47yOAhzCEZyBBzVowA00oQUEJDzAEzw7xnl0XpzXxeiSk+8cwA84b5+JEJFS</latexit><latexit sha1_base64="XSa9q8LMq8AN/ycpx8qQ8v3iUpE=">AAAB8nicdVDLSgMxFL3js9ZX1aWbYBVdlRnRtu4KblxWsA+YlpJJM21oJhmSjFCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxJxp47ofztLyyuraemGjuLm1vbNb2ttva5koQltEcqm6AdaUM0FbhhlOu7GiOAo47QST68zv3FOlmRR3ZhrTfoRHgoWMYGMlvxdhMw7CNDidDUplt3Lp1dx6FbkVd46MVK+qdQ95uVKGHM1B6b03lCSJqDCEY619z41NP8XKMMLprNhLNI0xmeAR9S0VOKK6n84jz9CJVYYolMo+YdBc/b6R4kjraRTYySyi/u1l4l+en5iw3k+ZiBNDBVl8FCYcGYmy+9GQKUoMn1qCiWI2KyJjrDAxtqWiLeHrUvQ/aZ9XPMtvL8qN47yOAhzCEZyBBzVowA00oQUEJDzAEzw7xnl0XpzXxeiSk+8cwA84b5+JEJFS</latexit><latexit sha1_base64="XSa9q8LMq8AN/ycpx8qQ8v3iUpE=">AAAB8nicdVDLSgMxFL3js9ZX1aWbYBVdlRnRtu4KblxWsA+YlpJJM21oJhmSjFCGfoYbF4q49Wvc+Tdm2hFU9EDgcM695NwTxJxp47ofztLyyuraemGjuLm1vbNb2ttva5koQltEcqm6AdaUM0FbhhlOu7GiOAo47QST68zv3FOlmRR3ZhrTfoRHgoWMYGMlvxdhMw7CNDidDUplt3Lp1dx6FbkVd46MVK+qdQ95uVKGHM1B6b03lCSJqDCEY619z41NP8XKMMLprNhLNI0xmeAR9S0VOKK6n84jz9CJVYYolMo+YdBc/b6R4kjraRTYySyi/u1l4l+en5iw3k+ZiBNDBVl8FCYcGYmy+9GQKUoMn1qCiWI2KyJjrDAxtqWiLeHrUvQ/aZ9XPMtvL8qN47yOAhzCEZyBBzVowA00oQUEJDzAEzw7xnl0XpzXxeiSk+8cwA84b5+JEJFS</latexit>
y0<latexit sha1_base64="r9Drz4GYDJ5Tr8A2n4d8XrLJfbU=">AAAB6XicdVBNS8NAEJ3Ur1q/qh69LFbRU0lE23orePFYxX5AG8pmu2mXbjZhdyOE0H/gxYMiXv1H3vw3btoIKvpg4PHeDDPzvIgzpW37wyosLa+srhXXSxubW9s75d29jgpjSWibhDyUPQ8rypmgbc00p71IUhx4nHa96VXmd++pVCwUdzqJqBvgsWA+I1gb6TY5GZYrdvXCqduNGrKr9hwZqV3WGg5ycqUCOVrD8vtgFJI4oEITjpXqO3ak3RRLzQins9IgVjTCZIrHtG+owAFVbjq/dIaOjTJCfihNCY3m6veJFAdKJYFnOgOsJ+q3l4l/ef1Y+w03ZSKKNRVksciPOdIhyt5GIyYp0TwxBBPJzK2ITLDERJtwSiaEr0/R/6RzVnUMvzmvNI/yOIpwAIdwCg7UoQnX0II2EPDhAZ7g2Zpaj9aL9bpoLVj5zD78gPX2CZGXjUk=</latexit><latexit sha1_base64="r9Drz4GYDJ5Tr8A2n4d8XrLJfbU=">AAAB6XicdVBNS8NAEJ3Ur1q/qh69LFbRU0lE23orePFYxX5AG8pmu2mXbjZhdyOE0H/gxYMiXv1H3vw3btoIKvpg4PHeDDPzvIgzpW37wyosLa+srhXXSxubW9s75d29jgpjSWibhDyUPQ8rypmgbc00p71IUhx4nHa96VXmd++pVCwUdzqJqBvgsWA+I1gb6TY5GZYrdvXCqduNGrKr9hwZqV3WGg5ycqUCOVrD8vtgFJI4oEITjpXqO3ak3RRLzQins9IgVjTCZIrHtG+owAFVbjq/dIaOjTJCfihNCY3m6veJFAdKJYFnOgOsJ+q3l4l/ef1Y+w03ZSKKNRVksciPOdIhyt5GIyYp0TwxBBPJzK2ITLDERJtwSiaEr0/R/6RzVnUMvzmvNI/yOIpwAIdwCg7UoQnX0II2EPDhAZ7g2Zpaj9aL9bpoLVj5zD78gPX2CZGXjUk=</latexit><latexit sha1_base64="r9Drz4GYDJ5Tr8A2n4d8XrLJfbU=">AAAB6XicdVBNS8NAEJ3Ur1q/qh69LFbRU0lE23orePFYxX5AG8pmu2mXbjZhdyOE0H/gxYMiXv1H3vw3btoIKvpg4PHeDDPzvIgzpW37wyosLa+srhXXSxubW9s75d29jgpjSWibhDyUPQ8rypmgbc00p71IUhx4nHa96VXmd++pVCwUdzqJqBvgsWA+I1gb6TY5GZYrdvXCqduNGrKr9hwZqV3WGg5ycqUCOVrD8vtgFJI4oEITjpXqO3ak3RRLzQins9IgVjTCZIrHtG+owAFVbjq/dIaOjTJCfihNCY3m6veJFAdKJYFnOgOsJ+q3l4l/ef1Y+w03ZSKKNRVksciPOdIhyt5GIyYp0TwxBBPJzK2ITLDERJtwSiaEr0/R/6RzVnUMvzmvNI/yOIpwAIdwCg7UoQnX0II2EPDhAZ7g2Zpaj9aL9bpoLVj5zD78gPX2CZGXjUk=</latexit><latexit sha1_base64="r9Drz4GYDJ5Tr8A2n4d8XrLJfbU=">AAAB6XicdVBNS8NAEJ3Ur1q/qh69LFbRU0lE23orePFYxX5AG8pmu2mXbjZhdyOE0H/gxYMiXv1H3vw3btoIKvpg4PHeDDPzvIgzpW37wyosLa+srhXXSxubW9s75d29jgpjSWibhDyUPQ8rypmgbc00p71IUhx4nHa96VXmd++pVCwUdzqJqBvgsWA+I1gb6TY5GZYrdvXCqduNGrKr9hwZqV3WGg5ycqUCOVrD8vtgFJI4oEITjpXqO3ak3RRLzQins9IgVjTCZIrHtG+owAFVbjq/dIaOjTJCfihNCY3m6veJFAdKJYFnOgOsJ+q3l4l/ef1Y+w03ZSKKNRVksciPOdIhyt5GIyYp0TwxBBPJzK2ITLDERJtwSiaEr0/R/6RzVnUMvzmvNI/yOIpwAIdwCg7UoQnX0II2EPDhAZ7g2Zpaj9aL9bpoLVj5zD78gPX2CZGXjUk=</latexit>
Input from previous layer
y0 = max(0,W0T x + b0)<latexit sha1_base64="Q6U05Yj7z6MhKB/NkX3DFTcCHBI=">AAACE3icbVDLSsNAFJ3UV62vqEs3g1VaH5REBN0IBTcuK/QFbSyT6aQdOnkwM5GGkH9w46+4caGIWzfu/BsnbRBtPTDM4Zx7ufceO2BUSMP40nILi0vLK/nVwtr6xuaWvr3TFH7IMWlgn/m8bSNBGPVIQ1LJSDvgBLk2Iy17dJ36rXvCBfW9uowCYrlo4FGHYiSV1NOPoxK8gl0XjcvGafrLoe3ErVJyVx/Dkx/BLiVHPb1oVIwJ4DwxM1IEGWo9/bPb93HoEk9ihoTomEYgrRhxSTEjSaEbChIgPEID0lHUQy4RVjy5KYGHSulDx+fqeRJO1N8dMXKFiFxbVaY7ilkvFf/zOqF0Lq2YekEoiYeng5yQQenDNCDYp5xgySJFEOZU7QrxEHGEpYqxoEIwZ0+eJ82ziqn47XmxepDFkQd7YB+UgQkuQBXcgBpoAAwewBN4Aa/ao/asvWnv09KclvXsgj/QPr4BZFOb5Q==</latexit><latexit sha1_base64="Q6U05Yj7z6MhKB/NkX3DFTcCHBI=">AAACE3icbVDLSsNAFJ3UV62vqEs3g1VaH5REBN0IBTcuK/QFbSyT6aQdOnkwM5GGkH9w46+4caGIWzfu/BsnbRBtPTDM4Zx7ufceO2BUSMP40nILi0vLK/nVwtr6xuaWvr3TFH7IMWlgn/m8bSNBGPVIQ1LJSDvgBLk2Iy17dJ36rXvCBfW9uowCYrlo4FGHYiSV1NOPoxK8gl0XjcvGafrLoe3ErVJyVx/Dkx/BLiVHPb1oVIwJ4DwxM1IEGWo9/bPb93HoEk9ihoTomEYgrRhxSTEjSaEbChIgPEID0lHUQy4RVjy5KYGHSulDx+fqeRJO1N8dMXKFiFxbVaY7ilkvFf/zOqF0Lq2YekEoiYeng5yQQenDNCDYp5xgySJFEOZU7QrxEHGEpYqxoEIwZ0+eJ82ziqn47XmxepDFkQd7YB+UgQkuQBXcgBpoAAwewBN4Aa/ao/asvWnv09KclvXsgj/QPr4BZFOb5Q==</latexit><latexit sha1_base64="Q6U05Yj7z6MhKB/NkX3DFTcCHBI=">AAACE3icbVDLSsNAFJ3UV62vqEs3g1VaH5REBN0IBTcuK/QFbSyT6aQdOnkwM5GGkH9w46+4caGIWzfu/BsnbRBtPTDM4Zx7ufceO2BUSMP40nILi0vLK/nVwtr6xuaWvr3TFH7IMWlgn/m8bSNBGPVIQ1LJSDvgBLk2Iy17dJ36rXvCBfW9uowCYrlo4FGHYiSV1NOPoxK8gl0XjcvGafrLoe3ErVJyVx/Dkx/BLiVHPb1oVIwJ4DwxM1IEGWo9/bPb93HoEk9ihoTomEYgrRhxSTEjSaEbChIgPEID0lHUQy4RVjy5KYGHSulDx+fqeRJO1N8dMXKFiFxbVaY7ilkvFf/zOqF0Lq2YekEoiYeng5yQQenDNCDYp5xgySJFEOZU7QrxEHGEpYqxoEIwZ0+eJ82ziqn47XmxepDFkQd7YB+UgQkuQBXcgBpoAAwewBN4Aa/ao/asvWnv09KclvXsgj/QPr4BZFOb5Q==</latexit><latexit sha1_base64="Q6U05Yj7z6MhKB/NkX3DFTcCHBI=">AAACE3icbVDLSsNAFJ3UV62vqEs3g1VaH5REBN0IBTcuK/QFbSyT6aQdOnkwM5GGkH9w46+4caGIWzfu/BsnbRBtPTDM4Zx7ufceO2BUSMP40nILi0vLK/nVwtr6xuaWvr3TFH7IMWlgn/m8bSNBGPVIQ1LJSDvgBLk2Iy17dJ36rXvCBfW9uowCYrlo4FGHYiSV1NOPoxK8gl0XjcvGafrLoe3ErVJyVx/Dkx/BLiVHPb1oVIwJ4DwxM1IEGWo9/bPb93HoEk9ihoTomEYgrRhxSTEjSaEbChIgPEID0lHUQy4RVjy5KYGHSulDx+fqeRJO1N8dMXKFiFxbVaY7ilkvFf/zOqF0Lq2YekEoiYeng5yQQenDNCDYp5xgySJFEOZU7QrxEHGEpYqxoEIwZ0+eJ82ziqn47XmxepDFkQd7YB+UgQkuQBXcgBpoAAwewBN4Aa/ao/asvWnv09KclvXsgj/QPr4BZFOb5Q==</latexit>
p based on AVF
E<latexit sha1_base64="iVWebS2dZLQIBn0Qzdn0ga+xfMw=">AAAB8XicdVDLSsNAFL2pr1pfVZduBqvgqiSiad0VRHBZwT6wDWUynbRDJ5MwMxFK6F+4caGIW//GnX/jpI2gogcGDufcy5x7/JgzpW37wyosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb8yWXmd+6pVCwSt3oaUy/EI8ECRrA20l0/xHrsB+nVbFCu2NVzp2bXXWRX7Tky4l64dQc5uVKBHM1B+b0/jEgSUqEJx0r1HDvWXoqlZoTTWamfKBpjMsEj2jNU4JAqL50nnqFjowxREEnzhEZz9ftGikOlpqFvJrOE6reXiX95vUQHdS9lIk40FWTxUZBwpCOUnY+GTFKi+dQQTCQzWREZY4mJNiWVTAlfl6L/Sfu06hh+c1ZpHOV1FOEADuEEHKhBA66hCS0gIOABnuDZUtaj9WK9LkYLVr6zDz9gvX0C+QyRBA==</latexit><latexit sha1_base64="iVWebS2dZLQIBn0Qzdn0ga+xfMw=">AAAB8XicdVDLSsNAFL2pr1pfVZduBqvgqiSiad0VRHBZwT6wDWUynbRDJ5MwMxFK6F+4caGIW//GnX/jpI2gogcGDufcy5x7/JgzpW37wyosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb8yWXmd+6pVCwSt3oaUy/EI8ECRrA20l0/xHrsB+nVbFCu2NVzp2bXXWRX7Tky4l64dQc5uVKBHM1B+b0/jEgSUqEJx0r1HDvWXoqlZoTTWamfKBpjMsEj2jNU4JAqL50nnqFjowxREEnzhEZz9ftGikOlpqFvJrOE6reXiX95vUQHdS9lIk40FWTxUZBwpCOUnY+GTFKi+dQQTCQzWREZY4mJNiWVTAlfl6L/Sfu06hh+c1ZpHOV1FOEADuEEHKhBA66hCS0gIOABnuDZUtaj9WK9LkYLVr6zDz9gvX0C+QyRBA==</latexit><latexit sha1_base64="iVWebS2dZLQIBn0Qzdn0ga+xfMw=">AAAB8XicdVDLSsNAFL2pr1pfVZduBqvgqiSiad0VRHBZwT6wDWUynbRDJ5MwMxFK6F+4caGIW//GnX/jpI2gogcGDufcy5x7/JgzpW37wyosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb8yWXmd+6pVCwSt3oaUy/EI8ECRrA20l0/xHrsB+nVbFCu2NVzp2bXXWRX7Tky4l64dQc5uVKBHM1B+b0/jEgSUqEJx0r1HDvWXoqlZoTTWamfKBpjMsEj2jNU4JAqL50nnqFjowxREEnzhEZz9ftGikOlpqFvJrOE6reXiX95vUQHdS9lIk40FWTxUZBwpCOUnY+GTFKi+dQQTCQzWREZY4mJNiWVTAlfl6L/Sfu06hh+c1ZpHOV1FOEADuEEHKhBA66hCS0gIOABnuDZUtaj9WK9LkYLVr6zDz9gvX0C+QyRBA==</latexit><latexit sha1_base64="iVWebS2dZLQIBn0Qzdn0ga+xfMw=">AAAB8XicdVDLSsNAFL2pr1pfVZduBqvgqiSiad0VRHBZwT6wDWUynbRDJ5MwMxFK6F+4caGIW//GnX/jpI2gogcGDufcy5x7/JgzpW37wyosLa+srhXXSxubW9s75d29tooSSWiLRDySXR8rypmgLc00p91YUhz6nHb8yWXmd+6pVCwSt3oaUy/EI8ECRrA20l0/xHrsB+nVbFCu2NVzp2bXXWRX7Tky4l64dQc5uVKBHM1B+b0/jEgSUqEJx0r1HDvWXoqlZoTTWamfKBpjMsEj2jNU4JAqL50nnqFjowxREEnzhEZz9ftGikOlpqFvJrOE6reXiX95vUQHdS9lIk40FWTxUZBwpCOUnY+GTFKi+dQQTCQzWREZY4mJNiWVTAlfl6L/Sfu06hh+c1ZpHOV1FOEADuEEHKhBA66hCS0gIOABnuDZUtaj9WK9LkYLVr6zDz9gvX0C+QyRBA==</latexit>
Injected Errors
E = kW0 � Wk<latexit sha1_base64="owOBjtpWCJo/G8r/G5XgkJ+WCs0=">AAACFHicdVDLSsNAFJ34rPUVdelmsIiCWBLRti6EggguK9gHNKVMppN26GQSZiZCifkIN/6KGxeKuHXhzr9x0qb4QA8MnHvOvcy9xw0ZlcqyPoyZ2bn5hcXcUn55ZXVt3dzYbMggEpjUccAC0XKRJIxyUldUMdIKBUG+y0jTHZ6nfvOGCEkDfq1GIen4qM+pRzFSWuqaB46P1MD14osEnkHnFk7r5l4CD7+qRHtds2AVT+yyVSlBq2iNkZLSaaliQztTCiBDrWu+O70ARz7hCjMkZdu2QtWJkVAUM5LknUiSEOEh6pO2phz5RHbi8VEJ3NVKD3qB0I8rOFa/T8TIl3Lku7ozXVL+9lLxL68dKa/SiSkPI0U4nnzkRQyqAKYJwR4VBCs20gRhQfWuEA+QQFjpHPM6hOml8H/SOCraml8dF6owiyMHtsEO2Ac2KIMquAQ1UAcY3IEH8ASejXvj0XgxXietM0Y2swV+wHj7BHCZnbY=</latexit><latexit sha1_base64="owOBjtpWCJo/G8r/G5XgkJ+WCs0=">AAACFHicdVDLSsNAFJ34rPUVdelmsIiCWBLRti6EggguK9gHNKVMppN26GQSZiZCifkIN/6KGxeKuHXhzr9x0qb4QA8MnHvOvcy9xw0ZlcqyPoyZ2bn5hcXcUn55ZXVt3dzYbMggEpjUccAC0XKRJIxyUldUMdIKBUG+y0jTHZ6nfvOGCEkDfq1GIen4qM+pRzFSWuqaB46P1MD14osEnkHnFk7r5l4CD7+qRHtds2AVT+yyVSlBq2iNkZLSaaliQztTCiBDrWu+O70ARz7hCjMkZdu2QtWJkVAUM5LknUiSEOEh6pO2phz5RHbi8VEJ3NVKD3qB0I8rOFa/T8TIl3Lku7ozXVL+9lLxL68dKa/SiSkPI0U4nnzkRQyqAKYJwR4VBCs20gRhQfWuEA+QQFjpHPM6hOml8H/SOCraml8dF6owiyMHtsEO2Ac2KIMquAQ1UAcY3IEH8ASejXvj0XgxXietM0Y2swV+wHj7BHCZnbY=</latexit><latexit sha1_base64="owOBjtpWCJo/G8r/G5XgkJ+WCs0=">AAACFHicdVDLSsNAFJ34rPUVdelmsIiCWBLRti6EggguK9gHNKVMppN26GQSZiZCifkIN/6KGxeKuHXhzr9x0qb4QA8MnHvOvcy9xw0ZlcqyPoyZ2bn5hcXcUn55ZXVt3dzYbMggEpjUccAC0XKRJIxyUldUMdIKBUG+y0jTHZ6nfvOGCEkDfq1GIen4qM+pRzFSWuqaB46P1MD14osEnkHnFk7r5l4CD7+qRHtds2AVT+yyVSlBq2iNkZLSaaliQztTCiBDrWu+O70ARz7hCjMkZdu2QtWJkVAUM5LknUiSEOEh6pO2phz5RHbi8VEJ3NVKD3qB0I8rOFa/T8TIl3Lku7ozXVL+9lLxL68dKa/SiSkPI0U4nnzkRQyqAKYJwR4VBCs20gRhQfWuEA+QQFjpHPM6hOml8H/SOCraml8dF6owiyMHtsEO2Ac2KIMquAQ1UAcY3IEH8ASejXvj0XgxXietM0Y2swV+wHj7BHCZnbY=</latexit><latexit sha1_base64="owOBjtpWCJo/G8r/G5XgkJ+WCs0=">AAACFHicdVDLSsNAFJ34rPUVdelmsIiCWBLRti6EggguK9gHNKVMppN26GQSZiZCifkIN/6KGxeKuHXhzr9x0qb4QA8MnHvOvcy9xw0ZlcqyPoyZ2bn5hcXcUn55ZXVt3dzYbMggEpjUccAC0XKRJIxyUldUMdIKBUG+y0jTHZ6nfvOGCEkDfq1GIen4qM+pRzFSWuqaB46P1MD14osEnkHnFk7r5l4CD7+qRHtds2AVT+yyVSlBq2iNkZLSaaliQztTCiBDrWu+O70ARz7hCjMkZdu2QtWJkVAUM5LknUiSEOEh6pO2phz5RHbi8VEJ3NVKD3qB0I8rOFa/T8TIl3Lku7ozXVL+9lLxL68dKa/SiSkPI0U4nnzkRQyqAKYJwR4VBCs20gRhQfWuEA+QQFjpHPM6hOml8H/SOCraml8dF6owiyMHtsEO2Ac2KIMquAQ1UAcY3IEH8ASejXvj0XgxXietM0Y2swV+wHj7BHCZnbY=</latexit>
e<latexit sha1_base64="7QcGseKTiBB4jN15miRR+sxCV/k=">AAAB6HicdVDLSgNBEOyNrxhfUY9eBqPgKeyKJvEW8OIxAfOAZAmzk95kzOyDmVkhhHyBFw+KePWTvPk3ziYrqGhBQ1HVTXeXFwuutG1/WLmV1bX1jfxmYWt7Z3evuH/QVlEiGbZYJCLZ9ahCwUNsaa4FdmOJNPAEdrzJdep37lEqHoW3ehqjG9BRyH3OqDZSEwfFkl2+dKp2rULssr1ASipXlZpDnEwpQYbGoPjeH0YsCTDUTFCleo4da3dGpeZM4LzQTxTGlE3oCHuGhjRA5c4Wh87JqVGGxI+kqVCThfp9YkYDpaaBZzoDqsfqt5eKf3m9RPs1d8bDONEYsuUiPxFERyT9mgy5RKbF1BDKJDe3EjamkjJtsimYEL4+Jf+T9nnZMbx5UaqfZHHk4QiO4QwcqEIdbqABLWCA8ABP8GzdWY/Wi/W6bM1Z2cwh/ID19gkSzo0E</latexit><latexit sha1_base64="7QcGseKTiBB4jN15miRR+sxCV/k=">AAAB6HicdVDLSgNBEOyNrxhfUY9eBqPgKeyKJvEW8OIxAfOAZAmzk95kzOyDmVkhhHyBFw+KePWTvPk3ziYrqGhBQ1HVTXeXFwuutG1/WLmV1bX1jfxmYWt7Z3evuH/QVlEiGbZYJCLZ9ahCwUNsaa4FdmOJNPAEdrzJdep37lEqHoW3ehqjG9BRyH3OqDZSEwfFkl2+dKp2rULssr1ASipXlZpDnEwpQYbGoPjeH0YsCTDUTFCleo4da3dGpeZM4LzQTxTGlE3oCHuGhjRA5c4Wh87JqVGGxI+kqVCThfp9YkYDpaaBZzoDqsfqt5eKf3m9RPs1d8bDONEYsuUiPxFERyT9mgy5RKbF1BDKJDe3EjamkjJtsimYEL4+Jf+T9nnZMbx5UaqfZHHk4QiO4QwcqEIdbqABLWCA8ABP8GzdWY/Wi/W6bM1Z2cwh/ID19gkSzo0E</latexit><latexit sha1_base64="7QcGseKTiBB4jN15miRR+sxCV/k=">AAAB6HicdVDLSgNBEOyNrxhfUY9eBqPgKeyKJvEW8OIxAfOAZAmzk95kzOyDmVkhhHyBFw+KePWTvPk3ziYrqGhBQ1HVTXeXFwuutG1/WLmV1bX1jfxmYWt7Z3evuH/QVlEiGbZYJCLZ9ahCwUNsaa4FdmOJNPAEdrzJdep37lEqHoW3ehqjG9BRyH3OqDZSEwfFkl2+dKp2rULssr1ASipXlZpDnEwpQYbGoPjeH0YsCTDUTFCleo4da3dGpeZM4LzQTxTGlE3oCHuGhjRA5c4Wh87JqVGGxI+kqVCThfp9YkYDpaaBZzoDqsfqt5eKf3m9RPs1d8bDONEYsuUiPxFERyT9mgy5RKbF1BDKJDe3EjamkjJtsimYEL4+Jf+T9nnZMbx5UaqfZHHk4QiO4QwcqEIdbqABLWCA8ABP8GzdWY/Wi/W6bM1Z2cwh/ID19gkSzo0E</latexit><latexit sha1_base64="7QcGseKTiBB4jN15miRR+sxCV/k=">AAAB6HicdVDLSgNBEOyNrxhfUY9eBqPgKeyKJvEW8OIxAfOAZAmzk95kzOyDmVkhhHyBFw+KePWTvPk3ziYrqGhBQ1HVTXeXFwuutG1/WLmV1bX1jfxmYWt7Z3evuH/QVlEiGbZYJCLZ9ahCwUNsaa4FdmOJNPAEdrzJdep37lEqHoW3ehqjG9BRyH3OqDZSEwfFkl2+dKp2rULssr1ASipXlZpDnEwpQYbGoPjeH0YsCTDUTFCleo4da3dGpeZM4LzQTxTGlE3oCHuGhjRA5c4Wh87JqVGGxI+kqVCThfp9YkYDpaaBZzoDqsfqt5eKf3m9RPs1d8bDONEYsuUiPxFERyT9mgy5RKbF1BDKJDe3EjamkjJtsimYEL4+Jf+T9nnZMbx5UaqfZHHk4QiO4QwcqEIdbqABLWCA8ABP8GzdWY/Wi/W6bM1Z2cwh/ID19gkSzo0E</latexit>
Figure 1. The BDLFI Approach: Faults are modeled (as Bayesian Networks) at every neuron of the network and propagated as probability distributionsthrough the network, and inferred at the output using Markov Chain Monte Carlo.
10 5 10 4 10 3 10 2 10 1
Flip Probability
5
10
15
20
25
Cla
ssifi
cati
on
Err
or(
%)
Golden Run
Figure 2. Error injections in all layers of multi-layerperceptron from Fig. 1.
Conv. Batch Norm. + ReLU DensePooling
0 1 2 3 4 5
Figure 3. ResNet-18 network and results of injec-tions on a layer-by-layer basis.
10 5 10 4 10 3 10 2 10 1
Flip Probability
30
40
50
60
70
Cla
ssifi
cati
on
Err
or(
%)
Golden Run
Figure 4. Error injections in all layers of ResNet-18from Fig. 3.
units for storing NN parameters, inputs, intermediate activationsand outputs. We model such faults by using the per-bitarchitectural vulnerability factor (AVF), i.e., each bit error istreated as a Bernoulli random variable with probability p. Wedo not make any assumptions about the number of bits in error;this is determined by p. All network parameters, inputs, andoutputs are encoded as 32-bit floating point numbers. BDLFIcan also be extended to other fault models.
Bayesian Network of Fault Model. We encode the abovefault model in a Bayesian Network shown as 2 in Fig. 1 foreach neuron in the NN. The weights and biases (W and b) aretransformed into their error injected version (W′ and b′) byperforming bitwise-XOR operations with flipped bits describedabove. As a result, output of each neuron is a probabilitydistribution over its output space given the original weightsand p. The key insight is that the fault behavior is modeledin the mathematical formulation of the Bayesian network. Thefault behavior is then propagated through the NN (using BDL)with every neuron activation capturing the fault behavior of itsinput, as well as its own fault behavior. We can hence captureend-to-end fault propagation in the NN as a statistical modelthat can be automatically inferred.
Inference. We quantify the effect of the injected faultsby comparing the classification accuracy of a golden run ofthe network with the distribution of classification accuracyproduced by the BDLFI. This distribution is produced byperforming Markov Chain Monte Carlo based inference onthe output of the fault injected network. The fault injection isperformed using the following steps:1) The network is trained to obtain the weights of the golden
network that provide good classification accuracy.2) The bit flip fault model and the trained weights of the
network are used to create the error distribution over thenetwork weights.
3) A Bayesian fault model is created for each neuron in thenetwork using the error distribution to construct a DBN.
4) Perform inference multiple times on the DBN using MCMCto obtain the classification uncertainty of the network fordifferent flip probabilities.
3 in Fig. 1 illustrates a golden run as well as the distributionof classification error produced by BDLFI.
III. PRELIMINARY RESULTS
We performed fault injection campaigns over an MLPin Fig. 1 and a ResNet-18 network trained on the CIFAR-
10 dataset. Our preliminary results in characterizing DNNresiliency, show several interesting behaviours that challengeexisting notions generated from traditional random fault injec-tion methods.
Effect of faults is most significant at the decision boundary. 3in Fig. 1 shows the effect of memory faults in an MLP network.The most likely classification errors are produced as a result offaults that happen at the decision boundary. This motivates theneed for having reliability features in safety-critical systems,where the points that are close to the decision boundary (i.e.,harder to classify) are more egregiously affected by errors. Byanalyzing the probability of errors near the boundaries, we canset a threshold on the regions of the feature space that needmore protection and verification of correctness.
Scope for trading off reliability and performance in DNNarchitectures. The relationship of NN classification accuracywith p can be seen in Fig. 2 and Fig. 4. In both cases, we cansee that there are two clear regimes explaining the effect ofbit flip probability on network classification error. In the firstregime consisting of smaller flip probability values, we observethat there is no significant increase in average classificationerror as the bit flip probability increases. In the second regime,we see that average classification error increases significantlywith flip probability. Hence operating at the knee of thesecurves provides the optimal performance-reliability trade-offsin building DNN systems.
Error propagation to the output of the neural network is notrelated to the depth of the injection layer. Contrary to previouswork [1] Fig. 3 shows that there is no direct relationshipbetween the layer in which the fault manifests and the networkclassification error. We believe that this artifact appears intraditional FI because of incomplete traversal of the entireinjection space. We are currently investigating this behavioron other NNs.
REFERENCES[1] G. Li et al., “Understanding error propagation in deep learning neural
network (dnn) accelerators and applications,” in Proceedings of theInternational Conference for High Performance Computing, Networking,Storage and Analysis. ACM, 2017, p. 8.
[2] Y. Gal, “Uncertainty in deep learning,” University of Cambridge, 2016.[3] B. Reagen et al., “Ares: A framework for quantifying the resilience of
deep neural networks,” in 2018 55th ACM/ESDA/IEEE Design AutomationConference (DAC). IEEE, 2018, pp. 1–6.
[4] G. Li, K. Pattabiraman, and N. DeBardeleben, “Tensorfi: A configurablefault injector for tensorflow applications,” in 2018 IEEE InternationalSymposium on Software Reliability Engineering Workshops (ISSREW).IEEE, 2018, pp. 313–320.
