BinFI: An Efficient Fault Injector for Safety-Critical …blogs.ubc.ca/karthik/files/2019/08/sc19-camera-ready.pdfsafety violations in practice [43]. Therefore, BinFI is the first

BinFI An Efficient Fault Injector for Safety-Critical MachineLearning Systems

Zitao Chenzitaoceceubcca

University of British Columbia

Guanpeng Ligplieceubcca

Karthik Pattabiramankarthikpeceubcca

Nathan DeBardelebenLos Alamos National Laboratory

ndebardlanlgov

ABSTRACTAs machine learning (ML) becomes pervasive in high performancecomputing ML has found its way into safety-critical domains (egautonomous vehicles) Thus the reliability of ML has grown in im-portance Specifically failures of ML systems can have catastrophicconsequences and can occur due to soft errors which are increasingin frequency due to system scaling Therefore we need to evaluateML systems in the presence of soft errors

In this work we propose BinFI an efficient fault injector (FI)for finding the safety-critical bits in ML applications We find thewidely-used ML computations are often monotonic Thus we canapproximate the error propagation behavior of a ML application as amonotonic function BinFI uses a binary-search like FI technique topinpoint the safety-critical bits (also measure the overall resilience)BinFI identifies 9956 of safety-critical bits (with 9963 precision)in the systems which significantly outperforms random FI withmuch lower costs

KEYWORDSError Resilience Machine Learning Fault InjectionACM Reference FormatZitao Chen Guanpeng Li Karthik Pattabiraman and Nathan DeBardeleben2019 BinFI An Efficient Fault Injector for Safety-Critical Machine LearningSystems In Proceeding of SC19 November 17-22 2019 Denver CO ACMNew York NY USA 14 pages httpsdoiorg101145nnnnnnnnnnnnnn

1 INTRODUCTIONThe past decade has seen the massive adoption of machine learning(ML) in a diverse set of areas [36 47 71 72] ML has also found itsway into the high performance computing (HPC) domain wherethe high computing capacity empowers ML to manage the largevolume of scientific and engineering data [35 56 65 81] Many ofthese ML applications are safety-critical [19 26 60 80 81] Oneemerging example is autonomous vehicles (AVs) in which the highthroughput low latency high reliability requirements make thehardware of these applications rival those of some supercomputersFor example Nvidia Drive AGX Xavier a system-on-chip (SoC)

ACMacknowledges that this contributionwas authored or co-authored by an employeecontractor or affiliate of the United States government As such the United Statesgovernment retains a nonexclusive royalty-free right to publish or reproduce thisarticle or to allow others to do so for government purposes onlySC19 November 17-22 2019 Denver CO USAcopy 2019 Association for Computing MachineryACM ISBN 978-x-xxxx-xxxx-xYYMM $1500httpsdoiorg101145nnnnnnnnnnnnnn

designed for AVs application is able to deliver 30 TOPS (trillionoperations per second) of performance at 30W [10] much like HPCsystems [49] While we focus on AVs in this paper similar trendsapply for ML applications in domains such as health-care [80 81]

As the trends toward exascale computing and AVs continueto grow the ability of ML to deliver high performance criticallydepends on the reliability of the system [23 70] The ISO 26262standard for functional safety of road vehicles specifies that therecan be nomore than 10 FIT (Failures in Time) which is 10 failures ina billion hours of operation [8 49] A recent study of failures in AVsshows that faults related to ML systems are the major culprits oftenrequiring the fallback of human driver intervention [18] Many ofthese failures are due to hardware transient faults also known assoft errors which are typically caused by high-energy particles dueto cosmic rays that interact with electronic components (eg aterrestrial neutron strike may cause current spikes in logic circuitsor storage elements and subsequently lead to a bit flip) Moreoversoft errors are increasing in frequency as system scale increases(especially in HPC applications [23 70 73]) and they could lead toundesirable consequences in ML systems [25 49 62] In additionhardware faults can also be deliberately induced by malicious at-tackers (eg selectively flipping specific bits) which can also causesignificant performance degradation in DNNs [38]

Traditional techniques (eg replicating hardware components[53] instruction duplication [51]) incur high overheads in hardwarecost energy and performance which make them impractical tobe deployed in HPC ML systems [49] For example an AV readyfor real-world deployment should be able to process the drivingdata within a few milliseconds (per iteration) [2 15] Deployingtraditional protection techniques may lead to delays in responsetime which may in turn lead to reaction-time-based accidents [18]Therefore we need to identify the parts of the ML systems thatare the most sensitive to hardware transient faults and selectivelyprotect the sensitive parts at low costs

In this paper we focus on the problem of identifying the safety-critical bits inML applications These are the bits in theML programwhich if affected by hardware transient faults result in a safety-condition violation A well-established approach to experimentalresilience assessment is random fault injection (FI) which worksby randomly sampling from a set of fault locations and injectingthe faults into the program to obtain a statistically significant esti-mate of their overall resilience Random FI has been used on MLapplications for overall resilience assessment [49 62] but it is un-fortunately not suitable for identifying the safety-critical bits in the

SC19 November 17-22 2019 Denver CO USA Zitao Chen Guanpeng Li Karthik Pattabiraman and Nathan DeBardeleben

program due to two reasons First because random FI relies on sta-tistical sampling it is not guaranteed to cover all the safety-criticalbits Second the safety-critical bits are often clustered in the statespace and random sampling is unable to find them (eg Fig 3)

The only known approach to identify the safety-critical bitsof a ML program is exhaustive FI which involves flipping eachbit of the program and checking if it resulted in a safety violationUnfortunately exhaustive FI incurs huge performance overheads asonly one fault is typically injected in each trial (for controllability)The time taken by exhaustive FI is therefore directly proportionalto the number of bits in the program which can be very large

In this paper we propose an efficient FI approach which canidentify the safety-critical bits in ML applications and also measurethe overall resilience with reasonable overheads The key insightof our approach is that the functions used in ML applications areoften tailored for specific purposes For example for a networkto classify an image of a vehicle the ML computations would bedesigned in a way that they can produce larger response upon thedetection of the vehicular feature in the image while keeping theresponse to irrelevant features small This results in the functionsexhibiting monotonicity based on how compatible is the input withthe target (eg vehicular features) and the composition of theseML functions can be approximated as a single monotonic compositefunction (Section 32) The monotonicity of the function helps usprune the FI space and efficiently identify the safety-critical bitsAnalogous to how binary search on a sorted array has an exponentialtime reduction compared to linear search our approach results in anexponential reduction in the FI space of ML applications to identifythe safety-critical bits compared to exhaustive FI Therefore we callour approach Binary fault injection or BinFI (in short)

Prior work has attempted to find safety violations in ML pro-grams for designsoftware bugs through random testing approaches [3154 59 78] However these papers do not examine the effects ofhardware faults more specifically soft errors Other papers have at-tempted to prune the FI space for general-purpose programs [33 69]or even eliminate FI altogether [51] Unfortunately these papers donot consider the specific properties of ML programs To the best ofour knowledge BinFI is the first fault injection technique to efficientlyfind safety violations in generic ML applications due to soft errors

Our main contributions in this paper are as follows

bull Analyze the common operations used in ML applications andidentify that many of them exhibitmonotonicity We approximatethe composition of these functions as a monotonic functionbull Present BinFI a fault injection approach that leverages the approx-imated monotonic composite function to find the safety-criticalbits and also measure the overall resilience of the programbull Extend an open-source FI tool TensorFI to incorporate BinFIand evaluate it over 8 ML applications with a total of 6 datasets(including a real-world driving dataset)

Our evaluation shows that BinFI can identify 9956 of the crit-ical bits with 9963 precision which significantly outperformsrandom FI Further it can also accurately estimate the overall SDCprobability of the application Finally BinFI incurs only around 20of the performance overhead of exhaustive FI which is the onlyother way to identify most of the critical bits in a program (to ourknowledge) ie it obtains a speedup of 5X over exhaustive FI

While the coverage of BinFI is not 100 in identifying safety-critical bits it presents an attractive alternative to traditional ex-haustive fault injection due to its significantly low overheads Fur-ther the proportion of critical bits missed by BinFI is less than05 which is acceptable in many situations given that soft errorsare relatively rare events and that not all bit-flips would lead tosafety violations in practice [43] Therefore BinFI is the first steptowards exploring the space of techniques that trade-off resiliencefor performance overheads in safety-critical ML applications

2 BACKGROUND AND FAULT MODELWe start by providing an overview of the fault injection tool weused We then discuss the requirements for AVs and finally presentthe fault model we assume

21 TensorFlow and Fault Injection ToolIn our work we use an open-source fault injector called TensorFIwhich is a configurable FI tool to inject faults in TensorFlow ap-plications [50] We choose it as it supports FI experiments on MLalgorithms implemented using the TensorFlow framework [16]which is the most popular ML framework in use today [14]1 Themain advantage of TensorFlow is that it abstracts the operationsin a ML program as a set of operators and a dataflow graph - thisallows programmers to focus on the high-level programming logic

There are two main components in the TensorFlow dataflowgraph (1) operator is the computational unit (eg matrix multiplica-tion) and tensor is the data unit Users can build the ML model usingthe built-in operators or define their customized operators Ten-sorFI duplicates the TensorFlow graph with customized operators[50] which are designed to not only be able to perform compu-tation as standard operators do but also inject faults at runtimeTensorFI is provided as a library so that users can easily integrateTensorFI into their TensorFlow programs It also allows differentconfiguration (eg how and where to inject fault) options througha YAML interface

22 AVs RequirementsAutonomous vehicles (AVs) are complex systems that use ML tointegrate data from various electronic components (eg LiDAR) anddeliver real-time driving decisions AVs entail several requirements(1) high throughput (eg large amounts of data must be processed asthey arrive from the sensors [1 15]) and (2) low latency (eg applythe brakes upon the detection of a pedestrian in front of the vehiclewithin a few ms) [2 15] These requirements present significantchallenges for reliability in AV applications

Asmentioned AVs reliability is mandated by stringent regulation- no more than 10 FIT as governed by ISO 26262 safety standardThere are two kinds of faults that can cause this standard to beviolated [2] namely (1) systematic faults and (2) transient faultsThe former are caused by design defects in hardware and softwarewhile the latter by cosmic rays and electromagnetic disturbancesSystematic faults in AVs can be mitigated at design time and havebeen well explored in the literature [59 67 78] Hardware transientfaults on the other hand need runtime mitigation and are less1Our approach is also applicable for applications using other ML frameworks such asPyTorch Keras etc as they are similar to TensorFlow in structure

BinFI An Efficient Fault Injector for Safety-Critical Machine Learning Systems SC19 November 17-22 2019 Denver CO USA

studied in the context of AVs FIT due to soft errors is orders ofmagnitude higher than the 10 FIT requirement [49] Therefore it isimportant to study soft errorsrsquo effect on AVrsquos reliability

23 Fault ModelIn this paper we consider hardware transient faults (ie soft errors)that randomly occur during the execution of ML programs Theerrors are caused by different sources such as alpha particles andelectro-magnetic effects In our fault model we assume faults occurin the processorrsquos data path ie in pipeline registers and ALUs Weassume that main memory cache and the register file are protectedby ECC or parity and hence we do not consider faults that originatein them This is a common assumption made in fault injectionstudies [17 29 51] We also assume that faults do not occur in theprocessorrsquos control logic as that constitutes only a small portionof the total area of the processor Note that our fault model onlyconsiders faults that are not masked before reaching the softwarelayer as masked faults do not affect the applicationrsquos execution

Since TensorFI operates on TensorFlow graphs we map the faultoccurrence to the interface of the operators in the graph Thisis because the details of each operation are hidden from the MLprogram and are also platform specific (eg the GPU version ofthese operators are often different from the CPU version) As aresult we inject faults directly to the output value of operators inML programs - this method is in line with prior work [20 50 51]

We follow the one-fault-per-execution model as soft errors arerelatively rare events with respect to the typical time of a programrsquosexecution - again this is a common assumption in the literature [1727 51 79] Soft errors can manifest in the software layer as singleor multiple-bit flips However recent work [20 68] has shown thatmultiple bit flip errors result in similar error propagation patternsand SDC probabilities as single-bit flip errors (at the program level)showing that studying single-bit flip fault injection is sufficient fordrawing conclusions about error resilience

We assume that faults do not modify the statestructure of themodel (eg change the modelrsquos parameters [42 54]) nor do weconsider faults in the inputs provided to the model (eg sensorfaults such as brightness change of the image [67 78]) as theyare outside the scope of the technique Finally we only considerfaults during the inference phase of ML programs as ML models areusually trained once offline and the inference phase is performedrepeatedly in deployment (typically hundreds of thousands of timesin AVs) which makes them much more prone to soft errors

3 METHODOLOGYWe consider Silent Data Corruption (SDC) as a mismatch betweenthe output of a faulty program and that of a fault-free program exe-cution For example in classifier models an image misclassificationdue to soft errors would be an SDC We leave the determinationof whether an SDC is a safety violation to the application as itdepends on the applicationrsquos context (see Section 4) Unlike tradi-tional programs where faults can lead to different control flows[33 51 58] faults in ML programs only result in numerical changesof the data within the ML models (though faults might also changethe execution time of the model this rarely happens in practice asthe control flow is not modified)

Critical bits are those bits in the programwhere the occurrence offault would lead to an SDC (eg unsafe scenarios in safety-criticalapplications) Our goal is to efficiently identify these critical bits inML programs without resorting to exhaustive fault injection intoevery bit

We first provide an example of how faults propagate in ML pro-gram in Section 31 We then define the terms monotonicity andapproximate monotonicity that we use throughout the paper Thenwe present our findings regarding the monotonicity of the func-tions used in common ML algorithms in Section 33 so that we canmodel the composition of all the monotonic functions involved infault propagation either as a monotonic or approximately mono-tonic function (Section 34) Finally we show how we leverage the(approximate) monotonicity property to design a binary-search likeFI algorithm to efficiently identify the critical bits in Section 35

31 Error Propagation ExampleThe principle of error propagation in different ML models is sim-ilar in that a transient fault corrupts the data which becomes anerroneous data and will be processed by all the subsequent com-putations until the output layer of the model In this section weconsider an example of error propagation in the k-nearest neigh-bor algorithm (kNN) in Fig 1 (k=1) The program is written usingTensorFlow and each TensorFlow operator has a prefix of tf (egtfadd) We use this code as a running example in this section

We assume that the input to the algorithm is an image (testImд)and the output is a label for the image Line 1 calculates the negativevalue of the raw pixel in testImд The program also has a set ofimages called neiдhbors whose labels are already known and thegoal of the program is to assign the label from one of the neiдhbors(called nearest neighbor) to the testImд Line 2 computes the rela-tive distance of testImд to each of neiдhbors Line 3 generates theabsolute distances and line 4 summarizes the per-pixel distance intoa total distance Line 5 looks for the index of the nearest neighborwhose label is the predicted label for testImд

Figure 1 An example of error propagation in kNN model(k=1) fault occurs at the tfadd operator - line 2

Assume a fault occurs at the add operator (line 2) and modifiesits output relativeDistance If each imagersquos dimension is (28 28)then the result from the tfadd operator is a matrix with shape( |N | 784) where |N | is the number of neighbors and each vectorwith 784 elements corresponds to each neighbor If the ith imageis the nearest neighbor we have disi lt disj forallj isin |N | i j wheredisi is the distance of the ith image to the test image The faultmight or might not lead to an SDC - we consider both cases below

SDC The fault occurs at (iy)y isin [1 784] (ie correspondingto the nearest neighbor) which incurs a positive value deviation(eg flipping 0 to 1 in a positive value) The fault would increasedisi which indicates a potential SDC as disi might no longer be thesmallest one among disj j isin |N | Similarly the fault at (jy)y isin[1 784] incurs a negative deviation and may result in an SDC

No SDC The fault at (iy)y isin [1 784] incurs a negative valuedeviation thus decreasing disi This will not cause SDC since disiis always the nearest neighbor Similarly a fault incurring positivedeviation at (jy)y isin [1 784] would always be masked

32 Monotonicity and ApproximateMonotonicity

We now define the terms monotonicity and approximate mono-tonicity that we use in this paperbull Non-strictly monotonic function A non-strictly monotonicfunction is either monotonically increasing f (xi ) ge f (x j )forallxi gtx j or monotonically decreasing f (xi ) le f (x j )forallxi gt x j We saya function has monotonicity when it is non-strictly monotonicbull Approximately monotonic We call a function approximatelymonotonic if it is non-strictly monotonic in a non-trivial intervalie the function does not exhibit both monotonically increasingand decreasing interchangeably (eg Sine function) We say afunction has approximate monotonicity when it is approximatelymonotonic For example f (x ) = 100 lowastmax (x minus 1 0) minusmax (x 0)is monotonically increasing when x gt 1 but not when x isin (0 1)Hence we consider f (x ) is approximately monotonicError propagation (EP) functionWedefine EP function which

is a composite function of those functions involved in propagat-ing the fault from the fault site to the modelrsquos output For ex-ample there are MatMul (matrix multiplication) and ReLu (rec-tified linear unit [57]) following the occurrence of the fault inthis case EP function is the composite function of both functionsEP (x ) = ReLu (MatMul (xorд minusxerr w )) wherew is the weight forthe MatMul computation xorд and xerr are the values before andafter the presence of fault Thus xorд minusxerr is the deviation causedby the fault at the fault site Note that we are more interested in thedeviation caused by the fault rather than the value of the affecteddata (xerr ) Thus the input to EP is the bit-flip deviation at the faultsite and the output of EP is the outcome deviation by the fault atthe modelrsquos output

The main observation we make in this paper is that most of thefunctions in ML model are monotonic as a result of which the EPfunction is either monotonic or approximately monotonic The mainreason for the monotonicity of ML functions (and especially DNNs)is that they are designed to recognize specific features in the inputFor example the ML model in Fig 2 is built for recognizing theimage of digit 1 so the ML computation is designed in a way that itwill have stronger responses to images with similar features as digit1 Specifically the ML computation would generate larger output ifthe image has stronger features that are consistent with the targetIn Fig 2 the output of the three inputs increases as they exhibithigher consistency (values in the middle column for each input)with the ML target And the final output by the model is usuallydetermined by the numerical magnitude of the outputs (eg largeroutput means higher confidence of the image to be digit 1)

The EP function can be monotonic or approximately monotonicbased on the ML model This (approximate) monotonicity can helpus to prune the fault injection space of the technique For simplicitylet us assume the EP function is monotonic We can model the EPfunction as |EP (x ) | ge |EP (y) | (x gt y ≫ 0) cup (x lt y ≪ 0) x yare faults at the bits of both 0 or 1 in the same data (Section 34

1 10 1

1 10 0

output =8Weak response

output = 120Medium response

output = 240Strong response

Input ML target(digit 1)

Output

Convolution(inner-product)

Input-1Weak feature

Input-2Medium feature

Input-3Strong feature

$ + hellip+$$$$

$ $ $$hellip

Figure 2 An example of how a ML model exhibits mono-tonicity for different inputs Assume the computation is asimple one-time convolution (inner-product)

has more details) We can therefore expect the magnitude of thedeviation caused by larger faults (in absolute value) to be greaterthan those by smaller faults Further a larger deviation at the finaloutput is more likely to cause an SDC Based on the above we canfirst inject x If it does not lead to an SDC we can reason that faultsfrom lower-order bits y will not result in SDCs without actuallysimulating them as these faults would have smaller outcomes

In practice the EP function is often approximately monotonic(rather than monotonic) especially in real-world complex ML mod-els such as DNNs Our approach for pruning the fault injectionspace remains the same This leads us to some inaccuracy in theestimation due to our approximation of monotonicity Nonethelesswe show later in our evaluation that this inaccuracy is quite small

33 Common ML Algorithms and MonotonicityIn this section we first survey the major ML functions within thestate-of-art ML models in different domains and then discuss theirmonotonicity Most of these models are comprised of DNNs

Image classification LeNet [48] AlexNet [47] VGGNet [72] In-ception [75ndash77] ResNet [36] Some of these networks [36 47 72 76]are the winning architectures that achieve the best performance onthe ILSVRC challenges [24] from 2012 to 2017

Object detection Faster-RCNN [66] YoLo [63] DarkNet [64]These networks produce outstanding performance in object detec-tion (eg can detect over 9000 categories [64]) and we are primarilyinterested in the ML thread for classification as these networksinclude both object localization and classification

Steering models Nvidia DAVE system [19] Commaairsquos steeringmodel [5] Rambo [12] Epoch [7] Autumn [3] These models are thepopular steering models available in the open-source community 23 and are used as the benchmarks in related studies [59 78]

Health care and others Detection of atrial fibrillation [80] ar-rythmia detection [60] skin cancer prediction [26] cancer reportanalysis [81] aircraft collision avoidance system [44]

These tasks are important in safety-critical domains (eg self-driving cars medical diagnosis) We are interested in those compu-tations for calculating the results (eg those in hidden layers) Notethat errors in the input layer such as during the reading the input

2httpsgithubcomudacityself-driving-car3httpsgithubcomcommaairesearch

Table 1 Major computations in state-of-art DNNs

Basic Conv MatMul Add (BiasAdd)Activation ReLu ELuPooling Max-pool Average-pool

Normalization Batch normalization (BN)Local Response Normalization (LRN)

Data transformation Reshape Concatenate DropoutOthers SoftMax Residual function

image are out of consideration The computations are summarizedin Table 1

We next discuss how most of the computations used in the abovetasks are monotonicbull Basic Convolution computation (Conv) is mainly used in thekernels in DNNs to learn the features Conv is essentially aninner-product computation X middot W = sumxiwi xi isin X wi isin W Assuming there are two faults at the same location andx1x2 (x1 gtx2 gt 0) are the deviations from the two faults The monotonicityproperty is satisfied as |x1wi | ge |x2wi | As mentioned that weare more interested in the effect caused by the single bit-flip faultso we do not consider data that are not affected by fault The mul-tiply function is monotonic thus Conv is monotonic (similarlyfor matrix multiplication - MatMul)bull Activation Activation (Act) function is often used to introducenon-linearity into the network which is important for the net-work to learn non-linear complicated mappings between theinputs and outputs [34] A widely used activation function isRectified linear unit (ReLu) [57] defined as f (x ) = max (0x )which is monotonic Exponential linear unit (ELu) [55] is similarto ReLu and is monotonicbull Pooling Pooling is usually used for non-linear down samplingMax-pooling and average-pooling are the two major poolingfunctions Max-pooling function extracts the maximum valuefrom a set of data for down-streaming and it is monotonic as fol-lowsmax (xi xk ) ge max (x j xk ) i f xi gt x j where xi x j couldbe faults at different bits and xk denotes all the data unaffectedby the fault Similarly average-pooling function calculates theaverage value from a group of data and it is also monotonic asfollows avд(xi xk ) ge avд(x j xk ) i f xi gt x j bull Normalization Normalization (Norm) is used to facilitate thetraining of the network by dampening oscillations in the distri-bution of activations which helps prevent problems like gradientvanishing [41] Local response Norm (LRN) and batch Norm (BN)are the two Norm approaches used in the above networksLRN implements a form of lateral inhabitation by creating com-petition for big activities among neuron outputs from differentkernels [47] However LRN does not satisfy the monotonicityproperty as it normalizes the values across different neurons in away that only needs to maintain competition (relative ordering)among the neighboring neurons Thus a larger input might gener-ate a smaller output as long as the normalized value maintains thesame relative ordering among the neighboring neurons (we alsovalidated this by experiments) However LRN was found to havesignificant limitations [72] and hence modern ML algorithmsusually use BN instead of LRN [12 19 36 64 75ndash77]

BN normalizes the value in a mini-batch during training phaseto improve the learning and normalization is neither necessarynor desirable during the inference as the output is expected tobe only dependent on the input deterministically [41] And thusin inference phase BN applies the same linear transformation toeach activation function given a feature map So we can describeBN in inference phase as f (x ) = wx + b where wb are thestatistics learned during training phase BN is thus monotonic asf prime(x ) = w

bull Data transformation There are computations that simply trans-form the data eg reshape the matrix and not alter the datavalue Dropout is considered as an identity mapping since it per-forms dropout only in training phase [74] These transformingfunctions are thus monotonic as xi gt x j i f xi gt x j bull Others SoftMax [37] is often used in the output layer to con-vert the logit (the raw prediction generated by the ML model)computed for each class into a probability within (01) and theprobabilities of all classes add up to 1 SoftMax function is de-fined as f (xi ) = exi

sumJj=1 e

x j ( f or i = 1 acircĂę J ) where xi thepredicted value for different class (J classes in total) The deriva-tive of SoftMax with respect to xi is as follows

part f (xi )

partxi=part

partxi(

exisumJj e

x j) =

(exi )prime lowastsumJj e

x j minus exi lowast exi

(sumJj e

x j )2

=exisumJj e

x jminus

exisumJj e

x jlowast

exisumJj e

x j= f (xi ) (1 minus f (xi ))

as f (xi ) ranges in (0 1) the derivative of SoftMax is alwayspositive thus SoftMax is monotonicResidual function used in ResNet [36] has the property that themapping from the input to output at each layer before activationfunction is added with an extra mapping H (x ) = F (x ) +Wx where H (x ) is the mapping from input to output F (x ) is theresidual function The insight is that it is easier to learn the resid-ual function F (x ) than the original mapping H (x ) For exampleif H (x ) is an identity mapping to be learned by the network itis easier for the network to learn F (x ) = 0 (W = 1) rather thanH (x ) = x There are different types of residual blocks and weconsider the original one in [36]H (x ) = BN (Conv (ReLu (BN (Conv (x ))))) +Wx whereWx is the extramapping Assuming a fault occurs at xi isin x H (xi ) is monotonic if the derivatives of the original mappingF (xi ) andWixi are always positive or negative However thederivative of F (xi ) might vary due to fault propagation For ex-ample assume H (x ) = 100 lowast ReLu (x minus 1) minus ReLu (x ) where onefault propagates into two state spaces (thus there are x minus 1 and x )The monotonicity of H (x ) discontinues according to the value ofx H (x ) = 99x minus 100x gt 1 and H (x ) = minusx x isin (0 1) Thereforewe call the residual block function as approximately monotonicThus we find that almost all the operations in Table 1 exhibit

monotonicity However this is not an exhaustive list eg thereare other activation functions some of which are non-monotonic(eg Swish [61] Sinusoid [28]) Nevertheless the above models arerepresentative of models used in domains such as object detection(as they yield state-of-art performance and are frequently referred

to by other studies) and the functions in Table 1 are common MLfunctions Thus we assume that the computations in most of theML models in the application domains mentioned above have themonotonicity property

34 Error Propagation and (Approximate)Monotonicity

As mentioned earlier the EP function is a composite function con-sisting of all the ML functions involved in error propagation The EPfunction therefore satisfies the monotonic or approximately mono-tonic property dependent on the ML modelbull EP with monotonicity Recall the example in Fig 1 (kNNmodel) where the fault occurs at the tfadd operator and it even-tually affects the distance of the test image to the neighbor imagevia tfabs function In this case EP (x ) = abs (x ) which is mono-tonicbull EP with approximate monotonicity Consider another func-tion in which a single fault x propagates into two state spaces(x minus 1x ) and ReLu is the subsequent function The weights asso-ciated with the respective data are (100minus1) thus we can modelthe EP function simply as EP (x ) = 100lowastmax (xminus1 0)minusmax (x 0)which is either monotonically increasing or decreasing in differ-ent intervals eg EP (x ) = 99x minus 100 x gt 1 (monotonically in-creasing) and EP (x ) = minusx x isin (0 1) (monotonically decreasing)The EP function is thus approximately monotonic as its mono-tonicity during x gt 1 becomes discontinued when x isin (0 1) andwe approximate that it is monotonic when x gt 0We leverage the (approximate) monotonicity of the EP function

for injecting faults Our methodology applies to EP functions in allML models with either monotonicity or approximate monotonicitythough our approach might incur minor inaccuracy in the lattersince it is an approximation In the following discussion we useboth terms (monotonicity and approximate monotonicity) inter-changeably and do not distinguish them if not explicitly specifiedEP functions with monotonicity satisfy the following property

|EP (x ) | ge |EP (y) | (x gt y ≫ 0) cup (x lt y ≪ 0) (2)

where x y are two faults at the bits of both 0 or 1 in the same datax occurs at high-order bit and y at low-order bit We consider thefaults that deviate considerably from 0 since faults around 0 onlyyield small deviations and would not lead to SDCs The monotonicEP function can be monotonically increasing or decreasing WhenEP is monotonically increasing EP (x ) le EP (y) le 0x lt y ≪0 thus Eq 2 is satisfied Monotonically decreasing only occursin multiply-related functions (eg Conv linear transformation)where the weights are negative For those functions also | f (x ) | gt| f (y) | (x gt y ≫ 0) cup (x lt y ≪ 0) thus Eq 2 is satisfied Hencethe EP functions in ML models satisfy Eq 2 Note that for EP withapproximate monotonicity Eq 2 might not always hold since it isan approximation

35 Binary Fault Injection - BinFIIn this section we discuss how we can leverage the (approximate)monotonicity of the EP functions in ML systems to efficiently pin-point the critical bits that lead to SDCs As mentioned earlier our

0 0 0 0 0 0 0SDC Non-SDC

Not result in SDCmove to higher-orderResults in SDC

move to lower-order2

+Fault occurs at

tfadd Op in Fig1

Figure 3 Illustration of binary fault injection Critical bitsare clustered around high-order bits

methodology applies to all ML programswhose EP functions exhibiteither monotonicity or approximate monotonicity

With (approximate) monotonicity of the EP function the out-come by different faults are sorted based on the original deviationcaused by the faults Faults at higher-order bits (larger input) wouldhave larger impact on the final outcome (larger output) and arethus more likely to result in SDCs Therefore we search for an SDC-boundary bit where faults at higher-order bits would lead to SDCsand faults from lower-order bits would be masked Fig 3 illustratesthis process Finding such a boundary is similar to searching for aspecific target within a sorted array (bits) and thus we decide touse binary-search like algorithm as the FI strategy We outline theprocedure of binFI in Algorithm 1 дetSDCBound function is thecore function to search for the SDC-boundary

BinFI is run separately for each operator in the model and itconsiders each data element in the output of the targeted operatorBinFI first converts the data into a binary expression (line 2) andthen obtains the index of the bits of 0 and 1 respectively (line 3 4)because faults causing positive and negative deviations would havedifferent impacts (this is why we separate the cases for x gt y ≫ 0and x lt y ≪ 0) We then find the SDC-boundary bit in the bits of0 and 1 (line 6 7)

We illustrate how binFI works on the tfadd operator in the ex-ample of Fig 1 (kNN model) Assuming the fault occurs at (iy)y isin[1 784] which corresponds to the nearest neighbor thus the 0 bitsin Fig 3 are those in the data at (iy) of the output by the tfaddoperator In this example whether an SDC will occur depends onwhether disi is still the nearest neighbor after a fault (Section 31)

BinFI first bisects the injection space and injects a fault in themiddle bit (line 16 17) The next injection is based on the resultfrom the current injection For example if the fault does not resultin an SDC (ie disi is still the nearest neighbor) the next injectionmoves to higher-order bit (from step 1 to step 2 in Fig 3) becausemonotonicity indicates that no fault from lower-order bits wouldlead to SDCs More specifically assume that the result from thefault at step 1 is disprimei = disi +abs (N ) where abs (N ) is the deviationcaused by the simulated fault We can express the results by faultsin lower-order bits as disprimeprimei = disi + abs (M ) According to Eq 2abs (N ) gt abs (M )N gt M gt 0 thus disprimei gt dis

primeprime

i where disprime

i disprimeprime

iare the nearest neighbors of the node

Moving the next FI to a higher- or lower-order bit is done byadjusting the front or rear index since we are doing binary-searchlike injection (eg line 19 moves the next injection to lower-orderbits by adjusting the front index) Step 2 in Fig 3 shows that theinjection causes an SDC and hence faults from higher-order bits

Algorithm 1 Binary-search fault injectionData opOutputlarr output from targeted operatorResult SDC boundary in the bits of 0 and 1 and SDC rate for

each element in the targeted operator1 for each in opOutput do2 binVal = binary(each) binary conversion3 0_list = дetIndexO f _0_bit (binVal) build the bit index (bit

of 0) from MSB to LSB4 1_list = дetIndexO f _1_bit (binVal)5 results for each element are stored in a list 6 sdcBound_0append(bound0=дetSDCBound(binVal 0_list))7 sdcBound_1append(bound1=дetSDCBound(binVal 1_list))8 sdcRateappend((bound0 + bound1)length(binVal))9 end for10 return sdcBound_1 sdcBound_0 sdcRate

Function дetSDCBound(binVal indexList)11 get the front and rear index for binary splitting 12 front = 0 higher-order bit13 rear = дetLenдth(indexList)-1 lower-order bit14 sdcBoundary = 0 by default there is no SDC15 while front le rear and front rear lastInjectedBit do16 currInjectedBit = indexList[(front + rear)2] binary split17 FIres = f ault_injection(binVal currInjectedBit)18 if FIres results in SDC then19 front = currInjectedBit+1 move next FI to low-order bit20 sdcBoundary = currInjectedBit index of critical bit21 else22 rear = currInjectedBit - 1 move next FI to high-order bit23 end if24 lastInjectedBit = currInjectedBit25 end while26 return sdcBoundary

would also lead to SDCs The next injection will move to lower-order bits (from step 2 to step 3) We also record the index of thelatest bit where a fault could lead to an SDC and it eventuallybecomes sdcBoundary (line 20) sdcBound in line 6 7 indicates theindex of the SDC boundary as well as how many critical bits (egsdcBound_1 = 5 means there are 5 critical bits in the bits of 1) Thuswe can use them to calculate the SDC rate by calculating the numberof critical bits over the total number of bits (line 8)

4 EVALUATIONAs mentioned in Section 21 we use an open-source FI tool devel-oped in our group called TensorFI 4 for performing FI experimentson TensorFlow-supported ML programs We modify TensorFI to(1) provide support for DNNs and (2) support BinFIrsquos approach forinjecting faults The former is necessary as the current version ofTensorFI does not have support for complex operations such asconvolutions used in DNNs - we added this support and made itcapable of injecting faults into DNNs (these modifications havesince been merged with the TensorFI mainline tool) The latter is

4httpsgithubcomDependableSystemsLabTensorFI

necessary so that we have a uniform baseline to compare BinFIwith

We have alsomade BinFI publicly available 5 For brevity we referto the implementation of TensorFI with the above modifications asBinFI in the rest of this paper

We evaluate BinFI by asking three research questions as followsRQ1 Among all the critical bits in a program how many of

them can be identified by BinFI compared with random FIRQ2 How close is the overall SDC rate measured by BinFI to

the ground truth SDC compared with that measured by randomFI

RQ3What is the overhead for BinFI compared with exhaustiveand random FI approaches and how does it vary by data type

41 Experimental SetupHardware All of our experiments were conducted on nodes run-ning Red Hat Enterprise Linux Server 64 with Intel Xeon 250GHzprocessors with 12 cores 64GB memory We also use a Linux desk-top running Ubuntu 1604 with an Intel i7-4930K 340GHz processorwith 6 cores 16GB memory and Nvidia GeForce GT610 GPU Notethat we measure the performance overheads in our experimentsin terms of ratios so the exact hardware configuration does notmatter (ie we can leverage more HPC resources to accelerate theexperiments both for our technique and exhaustive FI as FI is anembarrassingly parallel problem)

ML models and test datasets We consider 8 ML models in our eval-uation ranging from simple models eg neural network - NNkNN to DNNs that can be used in the self-driving car domains egNvidia DAVE systems [19] Commaai steering model [5]

We use 6 different datasets including general image classifica-tion datasets (Mnist Cifar-10 ImageNet) These are used for thestandard ML models In addition we use two datasets to representtwo different ML tasks in AVs motion planning and object detectionThe first dataset is a real-world driving dataset that contains imagescaptured by a camera mounted behind the windshield of a car [6]The dataset is recorded around Rancho Palos Verdes and San PedroCalifornia and labeled with steering angles The second one is theGerman traffic sign dataset which contains real-world traffic signimages [39] We use two different steering models for AVs from(1) Commaai which is a company developing AV technology andprovides several open-source frameworks for AVs such as open-pilot [5] (2) Nvidia DAVE self-driving system [19] which has beenimplemented in a real car for road tests [11] and has been usedas a benchmark in other studies of self-driving cars [59 78] Webuild a VGG11 model [72] to run on the traffic sign dataset Table 2summarizes the ML models and test datasets used in this study

FI campaigns We perform different FI campaigns on different op-erations in the network Due to the time-consuming nature of FIexperiments (especially considering that we need to perform ex-haustive FI to obtain the ground truth) we decide to evaluate 10inputs for each benchmark We also made sure that the inputs werecorrectly classified by the network in the absence of faults In thecase of the driving frame dataset there was no classification so we

5httpsgithubcomDependableSystemsLabTensorFI-BinaryFI

Table 2 ML models and datasets for evaluation

Dataset Dataset Description ML modelsMNIST [9] Hand-written digits 2-layer NN

LeNet-4 [48]Survive [13] Prediction of patient kNN

survivalCifar-10 [4] General images AlexNet [47]

ImageNet [24] General images VGG16 [72]German traffic sign [39] Traffic sign images VGG11 [72]

Driving [6] Driving video frames Nvidia Dave [19]Commaai [5]

checked if the steering angle was correctly predicted in the absenceof faults

We evaluate all of the operations in the following networks2-layer NN kNN LeNet-4 and AlexNet (without LRN) HoweverFI experiments on all operators in the other networks are verytime-consuming and hence we choose for injection one operatorper type in these networks eg if there are multiple convolutionoperations we choose one of them We perform injections on allthe data within the chosen operator except VGG16 which consistsof over 3 million data element in one operationrsquos output Thereforeit is impractical to run FI on all of those data (it will take more than276525 hours to do exhaustive FI on one operator for one input)So we decide to evaluate our approach on the first 100 data itemsin the VGG16 model (this took around 13 hours for one input inone operator) We use 32-bit fixed-point data type (1 sign bit 21integer bits and 10 mantissa bits) as the fixed-point datatype ismore energy efficient than the floating point datatype [22 32]

42 ResultsWe organize our results by the research questions (RQs)

RQ1 Identifying Critical Bits To answer this RQ we considerfour safety-critical ML systems used in AVs This is because SDCsin these systems would be potential safety violations and hencefinding such critical bits that lead to unsafe scenarios is importantWe use the steering systems from Nvidia [19] and Commaai [5] (ona real-world driving dataset) VGG11 [72] (on a real-world trafficsign dataset) and VGG16 [72] (vehicle images in ImageNet) Forthe latter two datasets we consider SDC as any misclassificationproduced by the system However for the two steering modelsthe output is the steering angle which is a continuous value andthere is no clear definition of what constitutes an SDC (to ourknowledge) Consequently we came up with three different valuesof the acceptable threshold for deviations of the steering angle fromthe correct angle to classify SDCs namely 5 30 and 60 degreesNote that these values include both positive and negative deviationsFig 4 shows the test images in our evaluation and exemplify theeffect of SDCs There are two types of deviation since the thresholdis for both positive and negative deviations

Apart from steering the vehicles it is also crucial for the AVs tocorrectly identify traffic signs and surrounding vehicles as incorrectclassification in such scenarios could lead to fatal consequences(eg misclassifying a ldquostoprdquo sign as ldquogo aheadrdquo sign) We show inFig 5 the potential effects of SDC in such scenarios

Figure 4 Single-bit flip outcome on the self-controlled steer-ing systems Blue arrows point to the expected steering an-gles and red arrows are faulty outputs by the systems

Figure 5 Single-bit flip outcome on the classifiermodels Im-ages at the first row are the input images those at the secondrow are the faulty outputs

We first perform exhaustive FI and record the results from flip-ping every bit in the injection space We also record all critical bitsidentified by BinFI and random FI for different numbers of trialsWe report the recall (ie how many critical bits were found) byBinFI and random FI Exhaustive FI is the baseline and it has a 100recall as it covers the entire injection space Fig 6 presents the recallfor different FI approaches on the four safety-critical ML systems Inthe interest of space we report only the recalls for the four simplemodels LeNet - 9984 AlexNet - 9961 kNN and NN - both 100We also found that the same operation (eg Conv operation) indifferent layers of a DNN exhibited different resilience (which is inline with the finding in prior work [49]) BinFI is able to achievehigh recall nonetheless The reason why BinFI has a recall of 100in kNN and NN is that the EP functions in these models can bemodeled as monotonic functions unlike the other models wherethe EP function is only approximately monotonic

Recall is computed as the number of critical bits (found by eachFI approach) in the whole state space divided by the total number ofcritical bits found by exhaustive FI (ground truth) We also providethe number of critical bits (obtained from exhaustive FI) found ineach benchmark and the total injection space in Table 3 across all 10inputs The critical bits found in the two steering models decreasealong with the larger SDC threshold since larger threshold impliesfewer SDCs are flagged

For the two steering models (the left six histograms in Fig 6) BinFIis able to find over 9871 of critical bits (an average of 9961)that would lead to safety violations across all three thresholds

100 100 100 100 100 100 100 1009956 9984 9985 9971 9871 9999 9999 9886

63456300

63466321

63146316

63196294

39563918

39413595

39293931

39403887

22252209

22302020

22092213

22082187

18821883

18951646

17781777

15611700

985987

1003863

933932

813892

502504

518442

478477

417444

Davethreshold = 5

Davethreshold = 30

Davethreshold = 60

Commaaithreshold = 5

VGG11(Traffic Sign)

VGG16(ImageNet)

all FI binFI ranFI - 1 ranFI - 05 ranFI - 025 ranFI ~ 02 ranFI ~ 01 ranFI ~ 005

Figure 6 Recall of critical bits by different FI techniques on four safety-critical ML systems ranFI minus 1 is random FI whose FItrial is the same as that by allF I FI trial for ranFI minus 05 is half of that by allF I ranFI sim 02 takes the same FI trials as BinFI

Table 3 Number of critical bits in each benchmark

Model amp Dataset Critical Bits Total Bits Percentagein FI Space ()

Dave-5 - Driving 2267185 4289160 5286Dave-30 - Driving 2092200 4289160 4878Dave-60 - Driving 1754487 4289160 4091

Commaai-5 - Driving 3352848 8144320 4171Commaai-30 - Driving 2679756 8144320 3290Commaai-60 - Driving 2217353 8144320 2723VGG11 - Traffic sign 808999 4483840 1804VGG16 - Vehicles 16919 186000 910

The consistently high recall of BinFI when using different SDCthresholds suggest that BinFI is agnostic to the specific thresholdused for classifying SDCs in these models Similarly for the othertwo models BinFI also achieves very high recalls We thus observea consistent trend across all the benchmarks each of which exhibitsdifferent sensitivity to transient faults as shown in Table 3

The coverage of random FI depends on the number of trialsperformed We consider different numbers of trials for randomFIranging from 5 to 100 of the trials for the exhaustive FI caseThese are labeled with the fraction of trials performed For exampleranFIminus05 means that the number of trials is 50 that of exhaustiveinjection BinFI performs about 20 of the trials as exhaustive injec-tion (see Fig 7 in RQ3) so the corresponding randomFI experiment(ranFI sim 02) with the same number of trials identifies about 19of the critical bits Even in the best case where the number of trialsof random FI is the same as that of exhaustive FI ie ranFI minus 10the recall is less than 65 which is much lower than BinFIrsquos recallof nearly 99 This is because the bits chosen by random FI are notnecessarily unique especially as the number of trials increases andhence not all bits are found by random FI

In addition to recall we alsomeasure the precision (ie howmanybits identified as critical are indeed critical bits) This is because lowprecision might raise false alarms and waste unnecessary resourceson over-protecting non-critical bits We report the precision inthe four safety-critical models in Table 4 (precision values for theremaining four models range from 9931 to 100) We find thatBinFI has precision of over 99 across all of the benchmarks whichmeans that it finds very few unnecessary critical bits

In summary we find that BinFI can achieve an average recallof 9956 with 9963 precision thus demonstrating its efficacy at

Table 4 Precision for BinFI on identifying critical bits

Model DAVE 1 Commaai 1 VGG11 VGG16(Driving) (Driving) (Traffic sign) (Vehicles)

Precision () 9960 9970 9999 99141 Results are averaged from using three thresholds

finding critical bits compared to random FI which only finds 19 ofthe critical bits with the same number of trials

RQ2 Overall SDC Evaluation We calculate the overall SDCprobabilities measured by BinFI and random FI for the ML modelsNote that the overall SDC probability is a product of the number ofbits in the operator as well as the SDC probability per bit So weweight the per bit SDC with the number of bits in the operator toobtain the overall SDC probability For example the overall SDCprobability for an operator with 100 bits (with SDC probability 20)and another operator with 10000 bits (with SDC probability 5)will be (100 lowast 020 + 10000 lowast 005)(100 + 10000) = 515

To quantify the accuracy of BinFI in measuring the overall SDCwe measure the deviation of the SDC probabilities from the groundtruth obtained through exhaustive FI in Table 5 (error bars at the95 confidence intervals are shown below the SDC rates in thetable) We limit the number of trials performed by random FI tothose performed by BinFI to obtain a fair comparison Overall wefind that the SDC probabilities measured by BinFI are very closeto the ground truth obtained through exhaustive FI Table 5 alsoshows that BinFI achieves 0 deviation in two ML models (NN andkNN) which is because the EP function in these two models aremonotonic

While the above results demonstrate that BinFI can be usedto obtain accurate estimates of the overall resilience random FIachieves nearly the same results with a much lower number of trialsTherefore if the goal is to only obtain the overall SDC probabilitiesthen random FI is more efficient than BinFI

RQ3 PerformanceOverhead Weevaluate the performance over-head for each FI technique in terms of the number of FI trials asthe absolute times are machine-dependent We show the resultsfor AlexNet and VGG11 in Fig 7 The overall FI trials for VGG11are lower than those for AlexNet as we do not inject fault intoall operators in VGG11 Fig 7 shows that the number of FI trialsperformed by BinFI is around 20 of that by exhaustive FI Thisis expected as BinFI performs a binary-search and we use a 32-bitdatatype - loд231 asymp 5 Note that the value is not 531 asymp 16 since

Table 5 Overall SDC deviation compared with ground truthDeviation shown in percentage () and error bars shown atthe 95 confidence intervals for random FI

Model BinFI ranFIsim02 ranFIsim01 ranFIsim005Dave-5 0070 0100 0101 0119(Driving) (plusmn001 sim plusmn031) (plusmn002 sim plusmn044) (plusmn002 sim plusmn062)Dave-30 0036 0096 0134 0172(Driving) (plusmn004 sim plusmn030) (plusmn006 sim plusmn042) (plusmn009 sim plusmn059)Dave-60 0125 0116 0118 0234(Driving) (plusmn005 sim plusmn029) (plusmn008 sim plusmn042) (plusmn011 sim plusmn059)

Commaai-5 0049 0064 0040 0095(Driving) (plusmn023 sim plusmn024) (plusmn033 sim plusmn034) (plusmn047 sim plusmn048)

Commaai-60 0008 0060 0094 0212(Driving) (plusmn021 sim plusmn022) (plusmn030 sim plusmn031) (plusmn043 sim plusmn044)VGG11 0002 0101 0156 0199

(Traffic sign) (plusmn023 sim plusmn026) (plusmn033 sim plusmn038) (plusmn047 sim plusmn053)VGG16 0042 1039 0778 0800

(ImageNet) (plusmn062 sim plusmn107) (plusmn086 sim plusmn158) (plusmn128 sim plusmn214)AlexNet 0068 0319 0420 0585(Cifar-10) (plusmn015 sim plusmn018) (plusmn022 sim plusmn025) (plusmn031 sim plusmn036)LeNet 0030 0228 0366 338(Mnist) (plusmn045 sim plusmn046) (plusmn064 sim plusmn065) (plusmn0887 sim plusmn095)NN 0000 0849 0930 0989

(Mnist) (plusmn033 sim plusmn11) (plusmn047 sim plusmn155) (plusmn067 sim plusmn220)kNN 0000 0196 0190 0197

(Survival) (plusmn005 sim plusmn02) (plusmn019 sim plusmn032) (plusmn0195 sim plusmn047)

we separately do injection for 0-bits and 1-bits Thus it is closer to631(asymp 20) We observe a similar trend in all the benchmarks (notshown due to space constraints)

1111970

448384

21756287993

1111970

44838455

962175

-100000100000300000500000

700000900000

11000001300000

Alexnet VGG11

allFI bin FI ranFI - 1 ranFI - 05 ranFI - 025 ranFI ~ 02 ranFI ~ 01 ranFI ~ 005

Figure 7 FI trials (overhead) for different FI techniques toidentify critical bits in AlexNet and VGG11

The performance overhead of BinFI also depends on the numberof bits used in the data representation In general the overheadgains increase as the number of bits increases (as BinFIrsquos time growslogarithmically with the number of bits while exhaustive FIrsquos timegrows linearly with the number of bits) To validate this intuitionwe evaluate BinFI on VGG11 using datatypes with different width(16-bit 32-bit and 64-bit) and compare its overhead with that ofexhaustive FI in Fig 8We find that the growth rate of BinFI is indeedlogarithmic with the number of bits We also measure the recalland precision of BinFI for the three data-types The recall valuesare 100 9997 9999 for 16-bit 32-bit and 64 bits respectivelywhile the respective precision values are 100 9996 9994 This

shows that the precision and recall values of BinFI are independentof the datatype

4296 5531 666015360

16-bit 32-bit 64-bit

binFIexhaustive FI

Figure 8 Numbers of FI trials by BinFI and exhaustive FI toidentify critical bits in VGG11 with different datatypes

5 DISCUSSIONWe start this section by discussing the inaccuracy of BinFI followedby the effects of non-monotonicity on its efficacy Finally we reflecton the implications of BinFI and its application to other HPC areas

51 Inaccuracy of BinFIAs mentioned in Section 34 the EP function is often only approxi-mately monotonic - this is the main source of inaccuracy for BinFIas it may overlook the critical bits in the non-monotonic portionsof the function Assuming EP (x ) = 2 lowastmax (x minus 1 0) minusmax (x 0)and a fault raises a deviation of x = 2EP (2) = 0 which does notresult in an SDC According to Eq 2 BinFI regards that all thefaults from 0 lt y lt 2 will not cause SDCs as |Eq(y) | le 0 However|EP (05) | = 05 which violates Eq 2 A fault incurring a deviationof 05 could be a critical bit unidentified by BinFI (thus resultingin inaccuracy) This is the reason why BinFI incurs minor inaccu-racy except for two models in Table 5 in which the EP functionis monotonic (not just approximately so) Nevertheless our eval-uation shows that BinFI incurs only minor inaccuracy as it hasan average recall of 9956 with 9963 precision and the overallSDC rate is very close to ground truth as well This is because thenon-monotonicity occurs in most cases when the fault is small inmagnitude and is hence unlikely to lead to an SDC

52 Effect of Non-MonotonicityWhile our discussion in Section 32 shows that many computationswithin the state-of-art ML models are monotonic there are alsosome models that use non-monotonic functions Though BinFI re-quires the functions to be monotonic so that the EP function is(approximately) monotonic we also want to measure the effectswhen BinFI is run on those models using functions that are notmonotonic Therefore we conduct an experiment to evaluate BinFIon two networks using different non-monotonic functions Specifi-cally we use a Neural Network using a Swish activation function[61] and AlexNet with LRN [47] As before we measure the recalland precision of BinFI on these models

For the NN model Fig 9 shows that BinFI has a recall of 989and a precision of 973 which is quite high The main reason isthat Swish function is monotonic across a non-trivial interval thusexhibiting approximate monotonicity so BinFI works well on it Onthe other hand when it comes to AlexNet with LRN BinFI has highrecall but low precision which means BinFI is not suitable for thismodel as LRN is non-monotonic in nature (Section 33)

0989 098809730764

NN with Switsh AlexNet with LRN

Recall

Precision

Figure 9 Recall and precision for BinFI on ML models withnon-monotonic functions

53 Implications for ProtectionKnowing the critical bits in ML systems can lead to the adoptionof configurable fault-tolerance techniques with low costs eg [4546 52 82] without resorting to conservative approaches [40 4951] For example Li et al [52] enable dynamic fault tolerance atruntime given that they know when the protection is needed tosave energy Similarly Krause et al [46] propose a data-drivenvoltage over-scaling technique where the supply voltage can bedynamically increased if the errors exceed a given threshold in acertain region of code Given that BinFI is able to identify the criticalbits one can adopt these approaches to dynamically protect theML systems with low costs Moreover BinFI can also identify theparts of application that might be exploited by malicious attackershence guiding the security protection against hardware fault attackstowards ML systems [38]

However while BinFI is able to identify 9956 of the criticalbits in an application its coverage is not 100 This means thatthere may be a few critical bits that are left unidentified when usingBinFI This is a limitation of our approachrsquos assumption of mono-tonicity Unfortunately there is no easy remedy for this problem asidentifying 100 of the bits requires exhaustive FI which is highlytime-consuming Our study is a first step towards low-cost protec-tion on safety-critical ML applications and we believe that missinga small proportion of the critical bits is an acceptable tradeoff given(1) the significant cost savings in BinFI (2) the relatively rare oc-curence of soft errors (3) not all critical bits would result in actualfailures in the systems [43]

54 Application to Other Areas of HPCIn this paper we mainly use BinFI to evaluate safety-critical MLsin the AVs domain which is an emerging example of ML in HPCHowever BinFI is not confined to this domain and there are manyother areas of application of BinFI in the HPC context We consider3 examples below (1) Xiong et al [80] use thousands of GPU coresto implement a DNN for the detection of atrial fibrillation (2) Yoonet al [81] use DNNs to extract information from cancer pathologyreports in cancer registries using supercomputer facility (3) Conget al [21] use a cluster of GPUs to accelerate ML tasks for actionrecognition which exhibits better performance in terms of bothspeedup and accuracy There are many other examples of ML usedin safety-critical domains [26 44 60] In these domains transientfaults can have serious implications on safety and hence BinFI canbe used to find the safety-critical bits in the application We deferexploration of ML in other safety critical domains to future work

In Section 33 we discuss our observation of monotonicity foundin common ML functions based on which we design BinFI to ef-ficiently identify safety-critical bits in ML programs While it ispossible that BinFI can be applied in other application domains if

the computations in the application exhibit the monotonicity prop-erty we believe this is unlikely to be the case in general purposeprograms This is because different faults in these programs couldlead to the execution of different program paths [33 51 58] Forexample a larger fault could cause a different program path to beexecuted whose output is not necessarily larger than the outputfrom another program path caused by a smaller fault (eg due to thedifferent computations performed) thereby violating monotonicityWe defer the exploration of non-ML applications to future work

6 RELATEDWORKWe classify related work into three broad areas

Testing of ML Pei et al [59] propose a novel approach to gen-erate corner case inputs to trigger unexpected behaviors of themodel They analyze the decision boundary of different ML modelsand leverage gradient ascent to modify the image to efficientlytrigger differential behaviors in different DNNs Tian et al [78] usetransformation matrices (eg scale rotate the image) to automati-cally generate corner case images Ma et al [54] design a mutationframework to mutate the ML model which is then used to evaluatethe quality of the test data The idea of mutating the ML model [54]is similar to fault injection However unlike our work which injectstransient faults the faults they inject have to do with emulatingsoftware faults such as removing a neuron or shuffling the dataRubaiyat et al [67] build a strategic software FI framework whichleverages hazard analysis to identify potential unsafe scenarios toprune the injection space However their approach suffers from lowerror coverage (less than 36) None of these approaches considerhardware transient faults which are growing in frequency andcan trigger undesirable consequences in ML systems Moreovernone of the above approaches leverage monotonicity of the modelsrsquofunctions to perform efficient fault injection

Error resilience of ML Li et al [49] build a fault injector torandomly inject transient hardware faults in ML application run-ning on specialized hardware accelerators Using the injector theystudy the resilience of the program under different conditions byvarying different system parameters such as data types A recentstudy designs a DNN-specific FI framework to inject faults into realhardware and studies the trade off between the model accuracyand the fault rate [62] Santos et al [25] investigate the resilienceof ML under mixed-precision architectures by conducting neutronbeam experiments These papers measure the overall resilience ofML systems using random FI while our approach identifies the bitsthat can lead to safety violations in these systems As we showed inthis paper random FI achieves very poor coverage for identifyingthe critical bits in ML systems

Accelerating fault injection Given the high overhead of FIexperiments numerous studies have proposed to accelerate the FIexperiments by pruning the FI space or even predicting the errorresilience without performing FI [30 33 51 69] Hari et al [33]propose Relyzer a FI technique that exploits fault equivalence (anobservation that faults that propagate in similar paths are likely toresult in similar outputs) to selectively perform FI on the pilot in-structions that are representative of fault propagation In follow upwork the authors propose GangES which finds that faults resultingin the same intermediate execution state will produce the same

faulty output [69] thus pruning the FI space further Li et al [51]propose Trident a framework that can predict the SDC rate of theinstructions without performing any FI The key insight is that theycan model the error propagation probability in data-dependencycontrol-flow and memory levels jointly to predict the SDC rateThey find that the Trident model is able to predict both the overallSDC rate of the program and those of individual instructions How-ever none of these studies are tailored for ML programs and theirfocus is on measuring the overall resilience of the system unlikeBinFI that can identify the critical bits

7 CONCLUSIONIn this work we propose an efficient fault injector to identify thecritical bits in ML systems under the presence of hardware tran-sient faults Our insight is based on the observation that many ofthe ML computations are monotonic which constrains their faultpropagation behavior We thus identify the existence of the SDCboundary where faults from higher-order bits would result in SDCswhile faults at lower-order bits would be masked Finally we designa binary-search like fault injector to identify the SDC boundaryand implement it as a tool called BinFI for ML programs writtenusing the TensorFlow framework

We evaluate BinFI on 8 ML benchmarks including ML systemsthat can be deployed in AVs Our evaluation demonstrates thatBinFI can correctly identify 9956 of the critical bits with 9963precision which significant outperforms conventional random FI-based approaches It also incurs significantly lower overhead thanexhaustive FI techniques (by 5X) BinFI can also accurately measurethe overall resilience of the application

As future work we plan to (1) extend BinFI to other ML frame-works than TensorFlow (2) consider other safety-critical ML appli-cations than AVs and (3) explore selective protection techniquesbased on the results from BinFI

BinFI is publicly available at the following URLhttpsgithubcomDependableSystemsLabTensorFI-BinaryFI

ACKNOWLEDGMENTSThis work was funded in part by a grant from the Natural Sciencesand Engineering Research Council of Canada (NSERC) through theDiscovery grant and Strategic grant programmes We thank theanonymous reviewers of SCrsquo19 for their comments which helpedimprove the paper

This manuscript has been approved for unlimited release and hasbeen assigned LA-UR-19-27921 This work has been co-authoredby an employee of Triad National Security LLC which operates LosAlamosNational Laboratory under Contract No 89233218CNA000001with the US Department of EnergyNational Nuclear Security Ad-ministration The publisher by accepting the article for publicationacknowledges that the United States Government retains a non-exclusive paid-up irrevocable world-wide license to publish orreproduce the published form of the manuscript or allow others todo so for United States Government purposes

REFERENCES[1] Autonomous and ADAS test cars produce over 11 TB

of data per day httpswwwtuxeracomblogautonomous-and-adas-test-cars-produce-over-11-tb-of-data-per-day

[2] Autonomous Car - A New Driver for Resilient Com-puting and Design-for-Test httpsneppnasagovworkshopsetw2016talks15WED20160615-0930-Autonomous_Saxena-Nirmal-Saxena-Rec2016Jun16-nasaNEPPpdf

[3] Autumn model in Udacity challenge httpsgithubcomudacityself-driving-cartreemastersteering-modelscommunity-modelsautumn

[4] Cifar dataset httpswwwcstorontoedu~krizcifarhtml[5] commaairsquos steering model httpsgithubcomcommaairesearch[6] Driving dataset httpsgithubcomSullyChendriving-datasets[7] Epoch model in Udacity challenge httpsgithubcomudacityself-driving-car

treemastersteering-modelscommunity-modelscg23[8] Functional Safety Methodologies for Automotive Applications https

wwwcadencecomcontentdamcadence-wwwglobalen_USdocumentssolutionsautomotive-functional-safety-wppdf

[9] Mnist dataset httpyannlecuncomexdbmnist[10] NVIDIA DRIVE AGX httpswwwnvidiacomen-usself-driving-cars

drive-platformhardware[11] On-road tests for Nvidia Dave system httpsdevblogsnvidiacom

deep-learning-self-driving-cars[12] Rambo httpsgithubcomudacityself-driving-cartreemaster

steering-modelscommunity-modelsrambo[13] Survival dataset httpsarchiveicsuciedumldatasetsHabermanlsquos+Survival[14] Tensorflow Popularity httpstowardsdatasciencecom

deep-learning-framework-power-scores-2018-23607ddf297a[15] Training AI for Self-Driving Vehicles the Challenge of Scale httpsdevblogs

nvidiacomtraining-self-driving-vehicles-challenge-scale[16] Martiacuten Abadi Paul Barham Jianmin Chen Zhifeng Chen Andy Davis Jeffrey

Dean Matthieu Devin Sanjay Ghemawat Geoffrey Irving Michael Isard et al2016 Tensorflow A system for large-scale machine learning In 12th USENIXSymposium on Operating Systems Design and Implementation (OSDI 16) 265ndash283

[17] Rizwan A Ashraf Roberto Gioiosa Gokcen Kestor Ronald F DeMara Chen-YongCher and Pradip Bose 2015 Understanding the propagation of transient errorsin HPC applications In SCrsquo15 Proceedings of the International Conference for HighPerformance Computing Networking Storage and Analysis IEEE 1ndash12

[18] Subho S Banerjee Saurabh Jha James Cyriac Zbigniew T Kalbarczyk and Ravis-hankar K Iyer 2018 Hands Off the Wheel in Autonomous Vehicles A SystemsPerspective on over a Million Miles of Field Data In 2018 48th Annual IEEEIFIPInternational Conference on Dependable Systems and Networks (DSN) IEEE 586ndash597

[19] Mariusz Bojarski Davide Del Testa Daniel Dworakowski Bernhard Firner BeatFlepp Prasoon Goyal Lawrence D Jackel Mathew Monfort Urs Muller JiakaiZhang et al 2016 End to end learning for self-driving cars arXiv preprintarXiv160407316 (2016)

[20] Chun-Kai Chang Sangkug Lym Nicholas Kelly Michael B Sullivan and MattanErez 2018 Evaluating and accelerating high-fidelity error injection for HPCIn Proceedings of the International Conference for High Performance ComputingNetworking Storage and Analysis IEEE Press 45

[21] G Cong G Domeniconi J Shapiro F Zhou and BY Chen 2018 Accelerating DeepNeural Network Training for Action Recognition on a Cluster of GPUs TechnicalReport Lawrence Livermore National Lab(LLNL) Livermore CA (United States)

[22] Matthieu Courbariaux Yoshua Bengio and Jean-Pierre David 2014 Train-ing deep neural networks with low precision multiplications arXiv preprintarXiv14127024 (2014)

[23] Nathan DeBardeleben James Laros John T Daly Stephen L Scott ChristianEngelmann and Bill Harrod 2009 High-end computing resilience Analysis of is-sues facing the HEC community and path-forward for research and developmentWhitepaper Dec (2009)

[24] Jia DengWei Dong Richard Socher Li-Jia Li Kai Li and Li Fei-Fei 2009 ImagenetA large-scale hierarchical image database (2009)

[25] Fernando Fernandes dos Santos Caio Lunardi Daniel Oliveira Fabiano Libanoand Paolo Rech 2019 Reliability Evaluation of Mixed-Precision Architectures In2019 IEEE International Symposium on High Performance Computer Architecture(HPCA) IEEE 238ndash249

[26] Andre Esteva Brett Kuprel Roberto A Novoa Justin Ko Susan M Swetter He-len M Blau and Sebastian Thrun 2017 Dermatologist-level classification of skincancer with deep neural networks Nature 542 7639 (2017) 115

[27] Bo Fang Karthik Pattabiraman Matei Ripeanu and Sudhanva Gurumurthi 2014Gpu-qin A methodology for evaluating the error resilience of gpgpu applicationsIn 2014 IEEE International Symposium on Performance Analysis of Systems andSoftware (ISPASS) IEEE 221ndash230

[28] Michael S Gashler and Stephen C Ashmore 2014 Training deep fourier neu-ral networks to fit time-series data In International Conference on Intelligent

Computing Springer 48ndash55[29] Giorgis Georgakoudis Ignacio Laguna Dimitrios S Nikolopoulos and Martin

Schulz 2017 Refine Realistic fault injection via compiler-based instrumentationfor accuracy portability and speed In Proceedings of the International Conferencefor High Performance Computing Networking Storage and Analysis ACM 29

[30] Jason George Bo Marr Bilge ES Akgul and Krishna V Palem 2006 Probabilisticarithmetic and energy efficient embedded signal processing In Proceedings of the2006 international conference on Compilers architecture and synthesis for embeddedsystems ACM 158ndash168

[31] Jianmin Guo Yu Jiang Yue Zhao Quan Chen and Jiaguang Sun 2018 DLFuzzdifferential fuzzing testing of deep learning systems In Proceedings of the 2018 26thACM Joint Meeting on European Software Engineering Conference and Symposiumon the Foundations of Software Engineering ACM 739ndash743

[32] Suyog Gupta Ankur Agrawal Kailash Gopalakrishnan and Pritish Narayanan2015 Deep learning with limited numerical precision In International Conferenceon Machine Learning 1737ndash1746

[33] Siva Kumar Sastry Hari Sarita V Adve Helia Naeimi and Pradeep Ramachan-dran 2012 Relyzer Exploiting application-level fault equivalence to analyzeapplication resiliency to transient faults In ACM SIGPLAN Notices Vol 47 ACM123ndash134

[34] Simon Haykin 1994 Neural networks Vol 2 Prentice hall New York[35] Kim Hazelwood Sarah Bird David Brooks Soumith Chintala Utku Diril Dmytro

Dzhulgakov Mohamed Fawzy Bill Jia Yangqing Jia Aditya Kalro et al 2018Applied machine learning at Facebook a datacenter infrastructure perspectiveIn 2018 IEEE International Symposium on High Performance Computer Architecture(HPCA) IEEE 620ndash629

[36] Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun 2016 Deep residuallearning for image recognition In Proceedings of the IEEE conference on computervision and pattern recognition 770ndash778

[37] Geoffrey Hinton Oriol Vinyals and Jeff Dean 2015 Distilling the knowledge ina neural network arXiv preprint arXiv150302531 (2015)

[38] SanghyunHong Pietro Frigo Yiğitcan Kaya Cristiano Giuffrida and Tudor Dumi-traş 2019 Terminal Brain Damage Exposing the Graceless Degradation in DeepNeural Networks Under Hardware Fault Attacks arXiv preprint arXiv190601017(2019)

[39] Sebastian Houben Johannes Stallkamp Jan Salmen Marc Schlipsing and Chris-tian Igel 2013 Detection of traffic signs in real-world images The German TrafficSign Detection Benchmark In The 2013 international joint conference on neuralnetworks (IJCNN) IEEE 1ndash8

[40] Jie S Hu Feihui Li Vijay Degalahal Mahmut Kandemir Narayanan Vijaykrishnanand Mary J Irwin 2005 Compiler-directed instruction duplication for soft errordetection In Design Automation and Test in Europe IEEE 1056ndash1057

[41] Sergey Ioffe and Christian Szegedy 2015 Batch normalization Acceleratingdeep network training by reducing internal covariate shift arXiv preprintarXiv150203167 (2015)

[42] Saurabh Jha Subho S Banerjee James Cyriac Zbigniew T Kalbarczyk and Ravis-hankar K Iyer 2018 Avfi Fault injection for autonomous vehicles In 2018 48thAnnual IEEEIFIP International Conference on Dependable Systems and NetworksWorkshops (DSN-W) IEEE 55ndash56

[43] Saurabh Jha Timothy Tsai Subho Banerjee Siva Kumar Sastry Hari Michael Sul-livan Steve Keckler Zbigniew Kalbarczyk and Ravishankar Iyer 2019 ML-basedFault Injection for Autonomous Vehicles A Case for Bayesian Fault Injection In2019 49th Annual IEEEIFIP International Conference on Dependable Systems andNetworks

[44] Kyle D Julian Jessica Lopez Jeffrey S Brush Michael P Owen and Mykel JKochenderfer 2016 Policy compression for aircraft collision avoidance systemsIn 2016 IEEEAIAA 35th Digital Avionics Systems Conference (DASC) IEEE 1ndash10

[45] Zvi M Kedem Vincent J Mooney Kirthi Krishna Muntimadugu and Krishna VPalem 2011 An approach to energy-error tradeoffs in approximate ripple carryadders In Proceedings of the 17th IEEEACM international symposium on Low-power electronics and design IEEE Press 211ndash216

[46] Philipp Klaus Krause and Ilia Polian 2011 Adaptive voltage over-scaling forresilient applications In 2011 Design Automation amp Test in Europe IEEE 1ndash6

[47] Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton 2012 Imagenet classifica-tion with deep convolutional neural networks In Advances in neural informationprocessing systems 1097ndash1105

[48] Yann LeCun Bernhard E Boser John S Denker Donnie Henderson Richard EHoward Wayne E Hubbard and Lawrence D Jackel 1990 Handwritten digitrecognition with a back-propagation network In Advances in neural informationprocessing systems 396ndash404

[49] Guanpeng Li Siva Kumar Sastry Hari Michael Sullivan Timothy Tsai KarthikPattabiraman Joel Emer and Stephen W Keckler 2017 Understanding errorpropagation in deep learning neural network (dnn) accelerators and applicationsIn Proceedings of the International Conference for High Performance ComputingNetworking Storage and Analysis ACM 8

[50] Guanpeng Li Karthik Pattabiraman and Nathan DeBardeleben 2018 TensorFIA Configurable Fault Injector for TensorFlow Applications In 2018 IEEE Interna-tional Symposium on Software Reliability Engineering Workshops (ISSREW) IEEE

313ndash320[51] Guanpeng Li Karthik Pattabiraman Siva Kumar Sastry Hari Michael Sullivan

and Timothy Tsai 2018 Modeling soft-error propagation in programs In 201848th Annual IEEEIFIP International Conference on Dependable Systems and Net-works (DSN) IEEE 27ndash38

[52] Wenchao Li Susmit Jha and Sanjit A Seshia 2013 Generating control logicfor optimized soft error resilience In Proceedings of the 9th Workshop on SiliconErrors in Logic-System Effects (SELSEacircĂŹ13) Palo Alto CA USA Citeseer

[53] Robert E Lyons and Wouter Vanderkulk 1962 The use of triple-modular redun-dancy to improve computer reliability IBM journal of research and development6 2 (1962) 200ndash209

[54] Lei Ma Fuyuan Zhang Jiyuan Sun Minhui Xue Bo Li Felix Juefei-Xu ChaoXie Li Li Yang Liu Jianjun Zhao et al 2018 Deepmutation Mutation testing ofdeep learning systems In 2018 IEEE 29th International Symposium on SoftwareReliability Engineering (ISSRE) IEEE 100ndash111

[55] Andrew LMaas Awni Y Hannun and Andrew Y Ng 2013 Rectifier nonlinearitiesimprove neural network acoustic models In Proc icml Vol 30 3

[56] Marisol Monterrubio-Velasco Joseacute Carlos Carrasco-Jimenez Octavio Castillo-Reyes Fernando Cucchietti and Josep De la Puente 2018 A Machine LearningApproach for Parameter Screening in Earthquake Simulation In 2018 30th Inter-national Symposium on Computer Architecture and High Performance Computing(SBAC-PAD) IEEE 348ndash355

[57] Vinod Nair and Geoffrey E Hinton 2010 Rectified linear units improve re-stricted boltzmann machines In Proceedings of the 27th international conferenceon machine learning (ICML-10) 807ndash814

[58] Nahmsuk Oh Philip P Shirvani and Edward J McCluskey 2002 Control-flowchecking by software signatures IEEE transactions on Reliability 51 1 (2002)111ndash122

[59] Kexin Pei Yinzhi Cao Junfeng Yang and Suman Jana 2017 Deepxplore Au-tomated whitebox testing of deep learning systems In proceedings of the 26thSymposium on Operating Systems Principles ACM 1ndash18

[60] Pranav Rajpurkar Awni Y Hannun Masoumeh Haghpanahi Codie Bourn andAndrew Y Ng 2017 Cardiologist-level arrhythmia detection with convolutionalneural networks arXiv preprint arXiv170701836 (2017)

[61] Prajit Ramachandran Barret Zoph and Quoc V Le 2017 Searching for activationfunctions arXiv preprint arXiv171005941 (2017)

[62] Brandon Reagen Udit Gupta Lillian Pentecost Paul Whatmough Sae Kyu LeeNiamhMulholland David Brooks and Gu-YeonWei 2018 Ares A framework forquantifying the resilience of deep neural networks In 2018 55th ACMESDAIEEEDesign Automation Conference (DAC) IEEE 1ndash6

[63] Joseph Redmon Santosh Divvala Ross Girshick and Ali Farhadi 2016 Youonly look once Unified real-time object detection In Proceedings of the IEEEconference on computer vision and pattern recognition 779ndash788

[64] Joseph Redmon and Ali Farhadi 2017 YOLO9000 better faster stronger InProceedings of the IEEE conference on computer vision and pattern recognition7263ndash7271

[65] Daniel A Reed and Jack Dongarra 2015 Exascale computing and big dataCommun ACM 58 7 (2015) 56ndash68

[66] Shaoqing Ren Kaiming He Ross Girshick and Jian Sun 2015 Faster r-cnnTowards real-time object detection with region proposal networks In Advancesin neural information processing systems 91ndash99

[67] Abu Hasnat Mohammad Rubaiyat Yongming Qin and Homa Alemzadeh 2018Experimental resilience assessment of an open-source driving agent arXivpreprint arXiv180706172 (2018)

[68] Behrooz Sangchoolie Karthik Pattabiraman and Johan Karlsson 2017 One bitis (not) enough An empirical study of the impact of single and multiple bit-fliperrors In 2017 47th Annual IEEEIFIP International Conference on DependableSystems and Networks (DSN) IEEE 97ndash108

[69] Siva Kumar Sastry Hari Radha Venkatagiri Sarita V Adve and Helia Naeimi2014 GangES Gang error simulation for hardware resiliency evaluation ACMSIGARCH Computer Architecture News 42 3 (2014) 61ndash72

[70] Bianca Schroeder and Garth A Gibson 2007 Understanding failures in petas-cale computers In Journal of Physics Conference Series Vol 78 IOP Publishing012022

[71] David Silver Aja Huang Chris J Maddison Arthur Guez Laurent Sifre GeorgeVan Den Driessche Julian Schrittwieser Ioannis Antonoglou Veda Panneershel-vam Marc Lanctot et al 2016 Mastering the game of Go with deep neuralnetworks and tree search nature 529 7587 (2016) 484

[72] Karen Simonyan and Andrew Zisserman 2014 Very deep convolutional networksfor large-scale image recognition arXiv preprint arXiv14091556 (2014)

[73] Marc Snir Robert W Wisniewski Jacob A Abraham Sarita V Adve SaurabhBagchi Pavan Balaji Jim Belak Pradip Bose Franck Cappello Bill Carlson et al2014 Addressing failures in exascale computing The International Journal ofHigh Performance Computing Applications 28 2 (2014) 129ndash173

[74] Nitish Srivastava Geoffrey Hinton Alex Krizhevsky Ilya Sutskever and RuslanSalakhutdinov 2014 Dropout a simple way to prevent neural networks fromoverfitting The Journal of Machine Learning Research 15 1 (2014) 1929ndash1958

[75] Christian Szegedy Sergey Ioffe Vincent Vanhoucke and Alexander A Alemi2017 Inception-v4 inception-resnet and the impact of residual connections onlearning In Thirty-First AAAI Conference on Artificial Intelligence

[76] Christian Szegedy Wei Liu Yangqing Jia Pierre Sermanet Scott Reed DragomirAnguelov Dumitru Erhan Vincent Vanhoucke and Andrew Rabinovich 2015Going deeper with convolutions In Proceedings of the IEEE conference on computervision and pattern recognition 1ndash9

[77] Christian Szegedy Vincent Vanhoucke Sergey Ioffe Jon Shlens and ZbigniewWojna 2016 Rethinking the inception architecture for computer vision InProceedings of the IEEE conference on computer vision and pattern recognition2818ndash2826

[78] Yuchi Tian Kexin Pei Suman Jana and Baishakhi Ray 2018 Deeptest Automatedtesting of deep-neural-network-driven autonomous cars In Proceedings of the40th international conference on software engineering ACM 303ndash314

[79] Jiesheng Wei Anna Thomas Guanpeng Li and Karthik Pattabiraman 2014Quantifying the accuracy of high-level fault injection techniques for hardwarefaults In 2014 44th Annual IEEEIFIP International Conference on DependableSystems and Networks IEEE 375ndash382

[80] Zhaohan Xiong Martin K Stiles and Jichao Zhao 2017 Robust ECG signalclassification for detection of atrial fibrillation using a novel neural network In2017 Computing in Cardiology (CinC) IEEE 1ndash4

[81] Hong-Jun Yoon Arvind Ramanathan and Georgia Tourassi 2016 Multi-taskdeep neural networks for automated extraction of primary site and lateralityinformation from cancer pathology reports In INNS Conference on Big DataSpringer 195ndash204

[82] Ming Zhang Subhasish Mitra TM Mak Norbert Seifert Nicholas J Wang QuanShi Kee Sup Kim Naresh R Shanbhag and Sanjay J Patel 2006 Sequentialelement design with built-in soft error resilience IEEE Transactions on Very LargeScale Integration (VLSI) Systems 14 12 (2006) 1368ndash1378

Abstract

1 Introduction

2 Background and Fault Model

21 TensorFlow and Fault Injection Tool

22 AVs Requirements

23 Fault Model

3 Methodology

31 Error Propagation Example

32 Monotonicity and Approximate Monotonicity

33 Common ML Algorithms and Monotonicity

34 Error Propagation and (Approximate) Monotonicity

35 Binary Fault Injection - BinFI

4 Evaluation

41 Experimental Setup

42 Results

5 Discussion

51 Inaccuracy of BinFI

52 Effect of Non-Monotonicity

53 Implications for Protection

54 Application to Other Areas of HPC

6 Related Work

7 Conclusion

Acknowledgments

References