Concolic Testing for Deep Neural Networks · 2019-09-10 · Intuitively, formula 9x:r expresses that there exists an input x such that r is true. Youcheng Sun, Min Wu, Wenjie Ruan,

Concolic Testing for Deep Neural Networks

Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, MartaKwiatkowska and Daniel Kroening

Arxiv 2018

Youcheng Sun, Min Wu, Wenjie Ruan, Xiaowei Huang, Marta Kwiatkowska and Daniel Kroening Arxiv 2018 (Arxiv 2018)Concolic Testing for Deep Neural Networks 05 May 2018 1 / 49

Table of Contents

1 Introduction

2 Test Requirements

3 Concrete Requirements

4 Overall Design

5 Requirement Evaluation

6 Symbolic Generation of New Concrete Inputs

7 Experimental Results


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Introduction

Major concerns have been raised about the readiness of applying DNNs tosafety- and security-critical systems, where faulty behaviour carries the riskof endangering human lives or potential damage to business.

To address these concerns critical systems implemented with DNNs, orcomprising DNNs components, needs to be thoroughly tested and certified.

None of previous work leverages concolic testing to explore the executionpaths of a program that are hard to cover by other techniques such asrandom testing.

Concolic testing is particularly well-suited for DNNs.

Input space is high-dimensional, which makes random testing difficult.

the number of “execution paths” too large to be completely coveredby symbolic execution.


Introduction

Currently, test requirements for DNNs lack a unified format.

To enable working with a broad spectrum of test requirements, they utiliseQuantified Linear Arithmetic over Rationals (QLAR).

For a given set R of test requirements, we gradually generate test cases toimprove coverage by alternating between concrete execution and symbolicanalysis.

Given an unsatisfied test requirement r , it is transformed into itscorresponding form δ(r) by means of a heuristic function δ.

Then, for the current set T of test cases, they identify a pair (t, δ(r)) oftest case t ∈ T and requirement δ(r) such that t is close to satisfying raccording to an evaluation based on concrete execution.


Introduction

After that, symbolic analysis is applied to (t, δ(r)) to obtain a newconcrete test case t ′, which is then added to the existing test suite, i.e.,T = T ∪ t ′.

This process repeats until we reach a satisfactory level of coverage.

The generated test suite T is given to a robustness oracle, which detectswhether T includes adversarial examples.


Introduction

Current work on the robustness of DNNs can be categorised as offensive ordefensive.

The concrete execution and symbolic analysis interleave until a certainlevel of coverage on program statements, branches, execution paths, etc.,is reached.

The key factor affecting the performance of concolic testing is theheuristics used to select another execution path.


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Test Requirements

Finding a test suite to cover all activation patterns in a DNN is intractable.This is compounded by the fact that a real-world DNN can easily havemillions of neurons.Therefore, they adopt the testing approach to first identify a subset of theactivation patterns according to certain cover criteria or test requirements,and then generate test cases to cover these activation patterns.


Test RequirementsQuantified Linear Arithmetic over Rationals (QLAR)

They adopt QLAR to express the DNN Requirement, abbreviated as DR.

Intuitively, formula ∃x .r expresses that there exists an input x such that ris true.


Test RequirementsSemantics


Test RequirementsSemantics


Test RequirementsTest Criteria


Test RequirementsComputational Complexity


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Concrete Requirements

In this section, they use DR∃,+ formulas to express several importantrequirements for DNNs, including Lipschitz continuity and test criteria.

Given a real number b, they generate a finite set S(DL1 , b) of subspaces ofDL1 such that for all inputs x1, x2 ∈ DL1 , if ||x1 − x2|| < b then thereexists a subspace X ∈ S(DL1 , b) such that x1, x2 ∈ X .

Usually, every subspace X ∈ S(DL1 , b) can be represented with a boxconstraint.


Concrete RequirementsLipschitz Continuity

The computation of cbest is an NP-hard problem and a smaller c cansignificantly improve the performance of verification algorithms.


Concrete RequirementsLipschitz Continuity

The testing of Lipschitz continuity can be guided by having the followingrequirements:

Intuitively, for each X ∈ S(DL1 , b), this requirement expresses the existenceof two inputs x1 and x2 such that N breaks the Lipschitz constant c .


Concrete RequirementsNeuron Cover

The Neuron Cover (NC) is an adaptation of the statement cover insoftware testing.


Concrete RequirementsModified Condition/Decision Cover (MC/DC)

According to Sign-Sign Cover (SSC), each neuron nk+1,j can be regardedas a decision such that these neurons at the precedent layer (i.e., the k-thlayer) are conditions.


Concrete RequirementsNeuron Boundary Cover

The Neuron Boundary Cover (NBC) aims to cover neuron activationvalues that exceed pre-specified bounds.


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Overall Design


Overall Design

requirement evaluation (Line 7), aims to find a pair (t, δ(r)) of input andrequirement which, according to our concrete evaluation, are the mostpromising in finding a new test case t ′ to satisfy the requirement r .

After obtaining (t, δ(r)), symbolic analysis (Line 8), is applied to have anew concrete input t ′.

Then validity check (Line 9) is applied to check if the new input is valid ornot.

The algorithm either if all test requirements in R have been satisfied, i.e.,R = {}, or no further requirement in R can be satisfied, i.e.,S ∪ T = T xR.


Overall Design


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Requirement Evaluation

Given a set of requirements R that have not been satisfied, a heuristic δ,and the current set T of test cases, the goal is to select a concrete inputt ∈ T together with a requirement r ′ = δ(r) for some r ∈ R, both of whichwill be used later in a symbolic approach to find the next concrete input t ′.

The general idea of obtaining (t, δ(r)) is as follows:


Requirement EvaluationHeuristics - Lipschitz Continuity

When a Lipschitz requirement r is unsatisfiable on T , we transform it intoδ(r) as follows:


Requirement EvaluationHeuristics - Neuron Cover- Neuron Boundary Cover

Neuron Cover:When a Neuron Cover requirement r is unsatisfiable on T , we transform itinto δ(r) as follows:

Neuron Boundary Cover:


Requirement EvaluationHeuristics - Sign-Sign Cover

When a Sign-Sign Cover requirement r is unsatisfiable on T , we transformit into δ(r) as follows:

Intuitively, given the decision neuron nk+1,j , it selects the condition that isclosest to the change of activation sign (i.e., smallest |u[x ]k,i |).


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Symbolic Generation of New Concrete Inputs

Given a concrete input t and a transformed requirement r ′ = δ(r), weneed to find the next concrete input t ′ by symbolic analysis.

Thanks to the use of DR, for each symbolic analysis method, itsapplication to different test criteria can be formulated under a unified logicframework.


Symbolic Generation of New Concrete InputsSymbolic Analysis using Linear Programming (LP)

The following linear constraints synthesize a set of inputs that exhibit thesame ReLU behaviour as x .

The symbolic analysis for finding a new input t ′ from a pair (t, r ′) of inputand requirement is equivalent to finding a new activation pattern.

Note that, to make sure that the obtained test case is meaningful, in theLP model an objective is added to minimize the distance between t and t ′.


Symbolic Generation of New Concrete InputsNeuron Cover

The symbolic analysis of neuron cover takes the input test case t andrequirement r ′, let us say, on the activation of neuron nk,i , and it shallreturn a new test t ′ such that the test requirement is satisfied by thenetwork instance N[t ′].

As a result, given N[t]’s activation pattern ap[t], we can build up a newactivation pattern ap′ such that


Symbolic Generation of New Concrete InputsNeuron Cover

This activation pattern specifies the following conditions:

nk,i ’s activation sign is negated: this ensures the aim of the symbolicanalysis to activate nk,i .

In the new activation pattern ap′, the neurons before layer k preservetheir activation signs as in ap[t]. Though there may exist variousactivation patterns that make nk,i activated, for the use of LPmodeling, one particular combination of activation signs must bepre-determined.

Other neurons are irrelevant, as the sign of nk,i is only affected by theactivation values of those neurons in previous layers.

Finally, by applying the LP modeling in [5] to the activation pattern, and ifthere exists a feasible solution, then it will become the new test case t ′,which makes the DNN satisfy the requirement r ′.


Symbolic Generation of New Concrete InputsSS Cover

When it comes to the SS Cover, to satisfy the requirement r ′ we need tofind a new test case such that, with respect to the input t, activation signsof nk+1,j and nk,i are negated, while other signs of other neurons at layerk are kept the same as in the case of input t.

To achieve this, the following activation pattern ap′ is built up for the LPmodeling:


Symbolic Generation of New Concrete InputsNeuron Boundary Cover

In case of the neuron boundary cover, the symbolic analysis aims to findan input t ′ such that the neuron nk,i ’s activation value exceeds either itshigher bound hk,i or its lower bound lk,i .

To achieve this, while preserving the DNN activation pattern as ap[t], weadd one of the following constraints into the LP program:


Symbolic Generation of New Concrete InputsSymbolic Analysis using Global Optimization

The symbolic analysis for finding a new input can also be implemented bysolving the global optimization problem.That is, by specifying the test requirement as an optimization objective,we apply global optimization to find a test case that makes the testrequirement satisfied.

For the Neuron Cover, the objective is thus to find a t ′ such that thespecified neuron nk,i has ap[t ′]k,i = true.

In case of the SS Cover, the optimization objective becomes

When it comes to the Neuron Boundary Cover, the objective offinding a new input t ′ can be one of the two forms: 1) u[t ′]k,i > hk,ior 2) u[t ′]k,i < lk,i .


Symbolic Generation of New Concrete InputsAlgorithms for Lipschitz Test Case Generation

The above problem can be efficiently solved by a novel alternatingcompass search scheme.


Symbolic Generation of New Concrete InputsAlgorithms for Lipschitz Test Case Generation - Stage 1

The above objective enables the algorithm to search for an optimal t1 inthe space of a norm ball or hypercube centred on t0 with radius ∆, suchthat the norm distance of v [t1]1 and v [t0]1 is as large as possible.

To solve the above the problem, we use the compass search method. If itis convergent and we can find an optimal t1 as



If it is convergent at t∗2 , and we still cannot find such a input pair, wemodify the objective function again.



In this case, they return the best input pair we can find, i.e., t∗1 and t∗2 ,and the largest Lipschitz constant.

In summary, they start from the given t0 to search for an image t1 in anorm ball or hypercube (the optimization trajectory on the norm ball spaceis denoted as S(t0,∆(t0)) such that Lip(t0, t1) > c (this step is symbolicexecution).

If they cannot find it, they modify the optimization objective function byreplacing t0 with t∗1 (the best concrete input found in this optimizationtrace) to initiate another optimization trajectory on the space, i.e.,S(t∗1 ,∆(t0)).

This process is repeated until the optimization trace gradually covers thewhole norm ball space S(∆(t0)).


Oracle


A Summary of Coverage-based DNN Testing


Table of Contents

1 Introduction

2 Test Requirements


4 Overall Design





Experimental ResultsComparison with DeepXplore - Neuron Cover


Experimental ResultsComparison with DeepXplore - Adversarial

Figure: It is worth noting that, although DeepConcolic does not impose particulardomain-specific constraints on the original image as DeepXplore does, theconcolic testing procedure automatically generates test cases that resemble“human perception”.


Experimental ResultsConcolic Testing Results on Different Test Criteria


Experimental ResultsExperimental Results for Lipschitz Constant Testing


Documents

Concolic Testing for Deep Neural Networks · 2019-09-10 · Intuitively, formula 9x:r expresses that there exists an input x such that r is true. Youcheng Sun, Min Wu, Wenjie Ruan,