IN DEGREE PROJECT TECHNOLOGY,FIRST CYCLE, 15 CREDITS

, STOCKHOLM SWEDEN 2016

Artificial Neural Network Autonomous VehicleArtificial Neural Network controlled vehicle

WILLIAM BRUCE

EDVIN VON OTTER

KTH ROYAL INSTITUTE OF TECHNOLOGYSCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT

Artificial Neural Network Autonomous Vehicle

Artificial Neural Network controlled vehicle

EDVIN VON OTTERWILLIAM BRUCE

Bacherlor’s Thesis in Mechatronics

Supervisor: Didem GürdürExaminer: Martin Edin GrimhedenApproved:

TRITA: MMKB 2016:26 MDAB087

AbstractThis thesis aims to explain how a Artificial Neural Network algorithm could

be used as means of control for a Autonomous Vehicle. It describes the theorybehind the neural network and Autonomous Vehicles, and how a prototype witha camera as its only input can be designed to test and evaluate the algorithmscapabilites, and also drive using it. The thesis will show that the ArtificialNeural Network can, with a image resolution of 100 × 100 and a training setwith 900 images, makes decisions with a 0.78 confidence level.

iii

Sammanfattning

Denna rapport har som mål att beskriva hur en Artificiellt Neuronnätverk al-goritm kan användas för att kontrollera en bil. Det beskriver teorin bakom neu-ronnätverk och autonoma farkoster samt hur en prototyp, som endast använderen kamera som indata, kan designas för att testa och utvärdera algoritmensförmågor. Rapporten kommer visa att ett neuronnätverk kan, med bildupplös-ningen 100 × 100 och träningsdata innehållande 900 bilder, ta beslut med en0.78 säkerhet.

v

Preface

We would like to thank our supervisor Didem Gürdür for all the feedback andencouragement and all our classmates for all the help and fun during the spring of2016!

William Bruce, Edvin von OtterStockholm, May, 2016

vii

Contents

Abstract iii

Sammanfattning v

Preface vii

Contents ix

Nomenclature xi

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.4.1 Experiment Design . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Theory 52.1 The Artificial Neural Network . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 The Perceptron . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Activation function . . . . . . . . . . . . . . . . . . . . . . . . 62.1.3 Cost function . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.1.4 The Forward-Propagation Algorithm . . . . . . . . . . . . . . 72.1.5 The Back-Propagation Algorithm . . . . . . . . . . . . . . . . 8

2.2 Generaliation and Overfitting . . . . . . . . . . . . . . . . . . . . . . 102.3 Autonomous Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.3.1 State of the technology . . . . . . . . . . . . . . . . . . . . . 112.3.2 Artificial Neural Networks in Autonomous Vehicles . . . . . . 11

3 Demonstrator 133.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Electronics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.2.1 Raspberry Pi 2 Model B . . . . . . . . . . . . . . . . . . . . . 143.2.2 Raspberry Pi Camera Module . . . . . . . . . . . . . . . . . . 14

ix

3.2.3 Pololu DRV8833 Dual Motor Driver Carrier . . . . . . . . . . 143.2.4 Power Sources . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.1 Demonstrator Chassi and motors . . . . . . . . . . . . . . . . 153.3.2 Pololu Housing . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.3 Raspberry Pi Camera Mount System and Case . . . . . . . . 16

3.4 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.4.1 Python Libraries . . . . . . . . . . . . . . . . . . . . . . . . . 173.4.2 The Controller - DriveApp and ANNA . . . . . . . . . . . . . 17

3.5 PythonNeuralNetwork . . . . . . . . . . . . . . . . . . . . . . . . . . 213.6 ANNADrive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Results 234.1 Accuracy for different size of training sets . . . . . . . . . . . . . . . 234.2 Accuracy for different resolutions . . . . . . . . . . . . . . . . . . . . 234.3 Demonstrator Performance . . . . . . . . . . . . . . . . . . . . . . . 23

5 Discussion and conclusions 255.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6 Recommendations and future work 27

Bibliography 29

Appendices

A Additional information 31A.1 Vectorised calculation of forward-propagation . . . . . . . . . . . . . 31A.2 Vectorised calculation of backward-propagation . . . . . . . . . . . . 31

B Software and Code 33B.1 DriverApp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33B.2 ANNA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

B.2.1 Main code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35B.2.2 motorClass used in Main . . . . . . . . . . . . . . . . . . . . 39

B.3 ANNADrive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

x

Nomenclature

SymbolsSymbols Description

W Array of weightsai Output from the i : th perceptronxi Output from the Neural Networkzi Input to activation function

AbbreviationsAbbreviation Description

ANN Artificial Neural NetworksRPi Raspberry PiCSI Camera Serial InterfaceGPIO General Purpose Input/OutputFPS Frames per second

NotationsAbbreviation Description

M Number of training examples usedK Number of input perceptronsJ Number of hidden-layer perceptronsI Number of output-layer perceptronsSuperscriptnumber withinparanthesis,(l)

Parameter corresponding to the l : th network layer

xi

Chapter 1

Introduction

The following chapter describes the background, purpose and scope of the project,which was carried out during the spring of 2016 at the Royal Institute of Technology(KTH) Mechatronics Department.

1.1 Background

As we are well aware of, the human brain is a magnificent computational device,able to handle vast amounts of data. The brain is able to, without much effort,recognize and process patterns in our surroundings, all of which is the basis formany of our decisions, and then learn from the experiences these decisions cause.This is of course something that we take for granted in our everyday life and it hashelped the human species far down the road.

As computers get more powerful, the idea to mimic some of the brains functions andimplement in machines has become a reality. This thesis will focus on the learningaspect, which in computer science is called Machine Learning. Machine learning isthe field of study that gives computers the ability to learn without being explicitlyprogrammed (Mitchell, 2006). There are several Machine Learning algorithms, oneof them are Artificial Neural Networks (ANN). An ANN attempts to do this by us-ing the concept of a brain’s neurons, fed with input from a camera or other sensorsand giving an output in form of, for example, a command that the machine shouldcarry out.

Just as in everyday life, the command will then be judged wrong or right and theANN will get a chance to adjust it self. The more data goes through the network,the more accurate the algorithm will become and the machine will thereby learn,in every sense of the word.

1

CHAPTER 1. INTRODUCTION

1.2 Purpose

The purpose of this project was to build an autonomous vehicle (AV) that usesANN for control. The design of the ANN algorithm is crucial for efficiency andspeed, why this thesis is to answer the following questions:

• How should an Artificial Neural Network algorithm be designed for an au-tonomous vehicle?

• How does the number of training examples affect the accuracy of the ArtificialNeural Network?

• How does the resolution of the images in the training examples affect the ac-curacy of the Artificial Neural Network?

1.3 Scope

In the real world the AV would face hundreds of other situations, but for a firstassessment of the ANN, the AV will be tested for a limited number of situations:

• Driving forward inside a lane.

• Taking corners inside a lane.

• Identifying and stopping at a stop sign.

These situations have been chosen because they are among the most basic trafficsituations and are easy to mimic for testing.

1.4 Method

After an information collecting phase had been carried out, the following methodwas used to answer the research questions:

• Firstly, a Demonstrator was designed and built to be able to collect data (SeeSection 3).

• Secondly, the Demonstrator was driven and data was collected from the situ-ations mentioned in the Scope.

• Lastly, the experiment was conducted and the result was implemented in theDemonstrator.

2

1.4. METHOD

1.4.1 Experiment DesignThis experiment was carried out in the following way: First, the data was processedto get three image resolutions: 25-by-25, 50-by-50 and 100 by 100. Then the datawas processed to get three different sizes of datasets with 1000, 2000 and 3000 100-by-100 images.

The data then was used to evaluate separate ANNs with different settings and theresult was analyzed. The best setting was then used to train a final ANN whichwas then implemented in the Demonstrator.

The optimal number of epochs used for training was determined by plotting theerror on the validation set while training the network on the training set. This gavea good understanding of when the network started to get over-fitted.

The accuracy was measured by using the prediction algorithm with the trainedweights to generate outputs from all examples in the test set. The outputs werethen compared to the target outputs of the examples and the percentage of correctpredictions was calculated.

3

Chapter 2

Theory

This chapter contains the background theory for the ANN and a brief overview ofAVs. The first part of the chapter describes the basis of the ANN and discussessome of the problems with the model, while the second part describes the basicconcepts of AVs.

2.1 The Artificial Neural Network

An ANN is a basic modell of a animal brain. In a brain, a neuron is a cell thatprocess chemical or electrical signals. The neuron is connected to other neurons andcreates a network, which in a human brain contains tens of billions of connectedcells (Awad and Khanna, 2015).

The neuron in a brain has input routes called dendrites, a cell body and output routscalled axons. When the electrochemical signal is transmitted through the dendrites,the neuron is activated. The cell body then determines the weight of the signal andif a threshold is passed, the neuron ”fires” through the axon (Bell, 2015).

2.1.1 The Perceptron

In a ANN, the neurons are named perceptrons and the axon and dendrites are calledlinks.1 The link from perceptron j to perceptron i transmitts the output of j to i.This output has the notation aj . The link also has a corresponding numeric weight,Wj,i, which determines the strength and sign of the connection. (Russel and Norvig,1995). This is displayed in Figure 2.1.

The perceptrons function is two-folded. Firstly, it computes the weighted sums of

1There is no standard naming practice for these objects, but the concept remains the same

5

CHAPTER 2. THEORY

Activationfunction

OutputInputfunction

a0 = 1Bias Weight

W0,i

Wj,i

Inputlinks

aj

Outputlinks

∑zi gai

Figure 2.1: The Perceptron (Russel and Norvig, 1995)

its inputs:

zi =n∑

j=0Wj,iaj . (2.1)

Secondly, it applies an activation function, g, to compute the output from theperceptron (Russel and Norvig, 1995). The output is a numeric value in the rangefrom zero to one.

2.1.2 Activation function

−6 −4 −2 2 4 6

0.5

1

Figure 2.2: The sigmoid function 1/(1 + e(−x)

In order for the ANN to be efficient, the activation function is of great importance.It is designed so that when the perceptron is fed with the correct input the percep-tron becomes ”active”, i.e. the function computes to a value near +1. Vice versa,the perceptron should become ”inactive”, i.e. the function computes to a value near0, when it is fed with the ”incorrect” input.

6

2.1. THE ARTIFICIAL NEURAL NETWORK

This gives us the sigmoid function, also called the logistic function, see figure 2.2:

g(x) = 11 + e−x

. (2.2)

The sigmoid function has the advantage of being differentiable, which is importantwhen using a weight-learning algorithm (Russel and Norvig, 1995).

2.1.3 Cost functionA popular saying is ”You learn from your mistakes.”. This is true in real life, asa bad decision often is followed by an unwanted event. The next time a person isfaced with a similar choice, we are less likely to make that same bad decision again.This is also true for ANNs, where the decision is a prediction which is associatedwith a cost. This cost is calculated with a cost function based on the error ofthe prediction. There are several cost functions used for machine learning, but forthis project, the logistic regression cost function (2.3) is used. It is based on themaximum likelyhood estimation in probability theory. (Raschka, 2015)

J(W ) = − 1M

( M∑m=1

ym log g(xm) + (1 − ym) log(1 − g(xm)))

(2.3)

Since the desired outputs from the training examples ym ∈ {0, 1}, one of the twoterms is always zeroed out. This can then be seen as two cost functions correspond-ing to the possible values of ym, where each assigns a very high cost to a predictionopposite to what the target output is. This cost function was, and is often, chosenfor ANNs because of it’s feature of being a convex function, making it easier tominimise than a non-convex cost function.

2.1.4 The Forward-Propagation Algorithm

The way the ANN makes a prediction is by taking the provided input parame-ters and amplify or weaken the activation of neurons in the network in variousways, yielding some output parameters that are probabilities for each output case.The weights of the links determine how much each perceptron’s activation is to bechanged before being input to the next layer of the ANN. This process is calledforward propagation. (Russel and Norvig, 1995)

The activation for an neuron of the first layer is simply the input parameter forthat neuron. Thus, with a training set of n training examples, the activation iscalculated as

a(1)k = xm,k (add bias unit to a

(1)k ). (2.4)

One amplification (or weakening [find better word]) corresponds to a weighted sumof multiplications of the activation with the appropriate weight according to

7

CHAPTER 2. THEORY

-1

-1

Input layer Hidden layer Output layerW

(1)k,j W

(2)j,ia

(1)k

a(2)j a

(3)i

Figure 2.3: A Multilayered Neural Network.

z(2)j =

n∑j=0

W(1)j,i a

(1)k . (2.5)

The activation for a hidden neuron, i.e of layer two, is the function value of thesigmoid function with the first layer neuron’s ”output” as input according to

a(2)j = g(z(2)

j ). (2.6)

The third ”neuron output” z(3)i and activation a

(3)i are calculated in analogy with

equation 2.4 and 2.5, respectively (Marsland, 2009). The bias units, showed in fig-ure 2.3 set the threshold for the units in that layer.(Raschka, 2015). A vectorisedimplementation of this method can be achieved to reduce the number of heavycomputations. Please visit appendix A.1 for vectorised calculcations of forward-propagation.

2.1.5 The Back-Propagation AlgorithmIn order to minimize the cost of a prediction, we need to take a look at the variablescontributing to calculating the cost. There are three: the inputs, the activationfunction and the weights. Changing the input to get a correct output is out of the

8

2.1. THE ARTIFICIAL NEURAL NETWORK

question since the goal here is to get the network to use an actual image to give anacceptable answer. The activation function is out of our control when the learningis in progress, hence it is the weights that need to be changed to minimize the costMarsland (2009).

In order to change the weights of the network in a way that minimizes the cost ofthe predictions, we need to understand how the sets of weights affect the output.To explain this, we start at the output and go backwards.

The cost of a prediction with respect to the weights is a function J(W ).

To minimize the cost, the jacobimatrix for J(W ) is needed for all weights in the net-work. We can separate this matrix into a jacobimatrix for the input layer, equation2.10 and one for the hidden layer, equation 2.8, since both sets of weights contributeto the overall cost of a prediction.

To run back-propagation for a training example, first a forward propagation is per-formed on that example. This is done in order to calculate the activations of theneurons in the network including those of the output layer. Having these activations,we can calculate the error corresponding to each neuron and get an understandingof how much that neuron was responsible for the error in the output.

The error term for an neuron of the output layer is calculated as

δ(3)i = a

(3)i − yi (2.7)

The derivative of J with respect to W(2)j,i is then calculated as

∂J(W (2)j,i )

∂(W (2)j,i )

= δ(3)i a

(2)i . (2.8)

Now the error term δ(3)i is back-propagated to the hidden layer with equation 2.9.

The hidden layer error term is based on a weighted average of all of the error termsin the output layer. This term is calculated for each neuron as

δ(2)j = W

(2)j,i δ

(3)i h′(z(2)

j ) (2.9)

The derivative of J with respect to W(1)k,j is now calculated as

∂J(W (1)k,j )

∂(W (1)k,j )

= δ(2)j a

(1)k (2.10)

We can now acquire the values of W(1)k,j and W

(2)j,i that minimize the cost J(W )

using a numerical optimization algorithm, making the predictions more accurate

9

CHAPTER 2. THEORY

(Marsland, 2009).

A vectorised implementation of this method can be achieved to reduce the num-ber of computations. With a training set of M training examples, the input is anM ×K matrix where K is the number of units in the input layer of the network. Theweight matrices W (1) and W (2) are of size J × (K + 1) and I × (J + 1) respectively,including the bias units. The activation a1 and a2 matrices are of size M × (K + 1)and M × (J + 1) after the bias units have been added. The output vector and thedesired output vector are of size M × 1. Please visit appendix A.2 for vectorisedcalculcations of back-propagation.

2.2 Generaliation and OverfittingA common problem in machine learning is over-training of the algorithm. Thepurpose of using an ANN is to generalise from the training examples to all possi-ble inputs, rather than matching the training inputs perfectly and not be able tomake good predictions on new inputs. Imagine a 5th order polynomial matching4 datapoints perfectly. The effect of this in an ANN is that the it learns how topredict the training set perfectly, eventually learning the possible noise of the inputsthemselves. To tackle this problem, the dataset is split into three subsets. One fortraining, one for validation and one for testing. While training the network on thetraining set, the cost of the errors on the validation set are plotted for each itera-tion. This will produce a curve of a decreasing cost for the first iterations, until theANN has been over-fitted, where the cost on the validation set will start to increaseagain. With this information, one can set the optimal number of iterations for thelearning algorithm in order for it to be able to generalise satisfyingly. The test setis then used to calculate on how many percent of the examples in the test set theANN predicts correctly (Marsland, 2009).

Another trick used to counter over-fitting is regularization. The regularization term2.11 is added to the cost function and works by penalizing the weights from beingto large. (Raschka, 2015)

λ

2M

((W (1))2 + (W (2))2

)(2.11)

The parameter λ can be changed to vary the impact the regularization should havein a particular algorithm.

2.3 Autonomous VehiclesAn Autonomous Vehicle (AV) is a technology that aims to augment vehicle au-tonomous driving entirely or in part for autonomous and safety purposes. The

10

2.3. AUTONOMOUS VEHICLES

technology it self consists of four fundamental technologies: environment perceptionand modeling, localization and map building, path planning and decision-making,and motion control. (Cheng, 2011)An AV uses machine vision, such as 3D cameras, and other sensors (such as Laser-Imaging Detection and Ranging - LIDAR - and GPS). Input signals from the ma-chine vision and sensor are integrated with stored data by artificial-intelligencesoftware to decide how the vehicle should operate based on traffic rules. (Manyikaet al., 2013)

2.3.1 State of the technologyToday, there are several car manufacturers that offer Driver Assistance and SafetyWarning Systems (DASWS) such as Forward Collision Warning, Adaptive CruiseControl and Lane Departure Warning (Cheng, 2011). However, with no regulatoryframework in place, fully autonomous vehicles were not available for the publicat the time this thesis. There are on the other hand many manufacturers, likeGeneral Motors, Toyota, BMW and Volvo, who are developing and testing theirown autonomous systems (Manyika et al., 2013). Google has demonstrated a fullyautonomous vehicle that has operated in more than 2.5 million kilometers to datewith only a small number of accidents (Manyika et al., 2013).

2.3.2 Artificial Neural Networks in Autonomous VehiclesIn most AVs, road recognition and vehicle driving are two separate modules. Theearly AV systems however, used road images as input for driving commands. Oneexample is the Autonomous Land Vehicle in a Neural Net (ALVINN). This is theCarnegie Mellon Universitys AV that uses a single hidden layer back-propagationnetwork, with the input consisting of a 30×32 two-dimensional video-frame (Cheng,2011).

11

Chapter 3

Demonstrator

In this chapter, the design of the Demonstrator is presented. The chapter willdescribe the capabilities of the Demonstrators components and software, while dis-cussing its limits.

3.1 Problem Formulation

The Demonstrator should be created so that the purpose of the thesis can be fulfilledand the research questions can be answered. The following aspects have been takeninto account when designing the Demonstrator:

• The Demonstrator should be easy to control and use, so that the data collectedfrom the Demonstrator is of high quality.

• The task at hand can be computationally heavy, why there is a demand forprocessing power.

• As visual input will be used, there is need for a camera. Also, pictures will bestored on the Demonstrator, why there needs to be sufficent data storage.

With these aspects, the Demonstrator can be split into three sub-systems: Control,Computations and Visual Input. Worth mentioning is that the project has anbudget of 1000 SEK.

3.2 Electronics

To answer the research questions, the Demonstrator will heavily depend on elec-tronics for input and computations. As there is a budget of 1000 SEK, the cost ofcomponents is critical, since electronics are prone to short-circuit and thereby bedamage beyond repair. The following sections will describe the electronics used inthe Demonstrator.

13

CHAPTER 3. DEMONSTRATOR

3.2.1 Raspberry Pi 2 Model BThe heart of the Demonstrator will be the Raspberry Pi (RPi), which will be usedfor all the sub-systems specified in the Problem Formulation. A Raspberry Pi isa credit card sized, low cost computer, able to run desktop versions of Linux andother operating systems (Raspberry Pi Foundation, 2016a).

The RPi used in this project is the Raspberry Pi 2 Model B. The following specifi-cations are interesting to point out (Raspberry Pi Foundation, 2016a) :

• 900MHz ARM Cortex-A7 CPU

• 1GB of RAM

• Camera Serial Interface (CSI)

• Micro SD card slot

• 40 General Purpose Input/Output (GPIO) pins

The RPi was chosen firstly because of the need of processing power and data storageoptions, but also because of the availability of CSI and GPIO, which are used forVisual Input and Control.

In this project, all of the code written for the Demonstrator is in the programminglanguage Python (see Section 3.4), which the RPi supports.

3.2.2 Raspberry Pi Camera ModuleThe visual input for the Demonstrator will be given by the Raspberry Pi CameraModule, which can be used to take high-definition video, as well as still photographs(Raspberry Pi Foundation, 2016b). The RPi camera module was chosen because ofits integration with the RPi and Python, and because of its variability in resolutionand frames per second.

3.2.3 Pololu DRV8833 Dual Motor Driver CarrierTo be able to control the motors, a Pololu DRV8833 is used, see figure 3.1. ThePololu is a small H-bridge motor driver that has the ability to bidirectionally controltwo motor outputs at 2.7 V to 10.8 V and can supply 1.2 A per channel continuously(Texas Instruments, 2015).

14

3.3. HARDWARE

Figure 3.1: The Pololu DRV8833 Dual Motor Driver Carrier (Texas Instruments,2015)

3.2.4 Power SourcesThe Demonstrator uses two power sources, one for the RPi and one for the motors:

• RPi Power Source: The RPi uses a Linocell Powerbank with a capacity of2200 mAh. The RPi is connected to the power source with a Micro-USB-cableand gets a steady 5V input from the power source.

• Motor Power Source: The motors are connected to 7.4V Li-Po batterywith a capacity of 5000 mAh.

3.3 HardwareEven though electronics play such a vital part in the project, other hardware hasa important function. The hardware part of the Demonstrator takes care of thestructural problems, such as how the motors should be fitted and the camera isangled. This is done with both bought and 3D-printed parts.

Development and 3D-printing of hardware was done with the Autodesk Fusion 360(ľ 2016 Autodesk Inc) - a CAD tool - and Ultimaker 2 - which is a small 3D-printerthat lets the user quickly and easily print smaller objects. (Behövs referens?)

3.3.1 Demonstrator Chassi and motorsThe base for the Demonstrator was the 4WD ATV Mobile Platform kit from ActiveRobots. The kit contains a chassi, 4 wheels and 4 DC-motors. To provide space forall the parts, the chassi was extended with a second floor, taken from a similar kit.

3.3.2 Pololu HousingBecause of the delicacy of the Pololu, there is need to protect the part so that itdoesn’t get short-circuited. This has been done by creating a custom fitted housing,

15

CHAPTER 3. DEMONSTRATOR

displayed in figure 3.2.

Figure 3.2: The Pololu Housing

3.3.3 Raspberry Pi Camera Mount System and Case

To be able to mount the camera to the same place and angle, while also protectingthe camera, three separate parts have been created. Firstly, there is the cameraholder, displayed in figure 3.3a, to hold the camera in place. Secondly, there isthe bottom plate, displayed in figure 3.3b, that holds the holder and camera at aconstant angle. Together with the holder, these make up the Camera Mount System.

(a) (b)

Figure 3.3: The Camera Mount System, with the Holder (a) and the bottom plate(b)

Lastly there is the camera case, designed to protect the RPi camera from [shocks].This part, designed by the user VGer, was downloaded from Thingiverse and wasthen 3D Printed (VGer, 2013).

16

3.4. SOFTWARE

3.4 SoftwareThe software used in the Demonstrator is written in the object-oriented program-ming language Python (Python Software Foundation, 2016). Python is chosen sinceit has great resources and libraries for the Raspberry Pi and for computations.

The software development has been split up in two parts: (1) development of acontroller with user input and (2) development of the ANN-algorithm. These twoare in close relation, the controller is used together with the ANN-algorithm in whenthe Demonstrator drives by itself.

3.4.1 Python LibrariesIn the development of the software used in the Demonstrator, there are a lot of partsthat has been simplified by using already existing functions and objects, developedby other Python developers. The following libraries have been used in the codedescribed below:

• Python Image Library: The library adds image processing capabilities tothe software. It is used to get fast access to and storing of image data. Thislibrary was developed by Fredrik Lundh, Alex Clark and Contributors (Lundhand Clark, 2016).

• SciPy: SciPy gives Python numerical methods. In the SciPy package NumPyis included, which is a package that gives the developer access to linear algebraand helps handle n-dimensional arrays (The Scipy Community, 2015).

• RPIO: Adds the capability to control GPIOs. This library is used mainly forthe PWM control for the motors (Hagen, 2013).

• PiCamera: Gives control over the PiCamera through the CSI. This is usedto access and change the settings of the camera (Jones, 2014).

• PyGame: Used for access to the keyboard on the users computer. Originallydeveloped to make it easy to create multimedia software (pygame.org, 2011).

3.4.2 The Controller - DriveApp and ANNADriveApp collects user input and sends it to the Demonstrator. This is done inthree steps described below and also in figures 3.4 and 3.5:

1. DriverApp collects data: The users input is collected from the keyboard,where the keys ’W’, ’S’, ’A’ and ’D’ are mapped to ’forward’, ’backward’, ’left’and ’right’. When any of the keys are pressed, the program generates a codewhere the pressed keys have the value 1 and the other keys have the value 0(this is done to minimize the size of the code). These values are then put ina specific order (forward, backward, left, right).

17

CHAPTER 3. DEMONSTRATOR

2. ANNA receives the user input: The code generated in step 1 is then sentover wifi to the Demonstrator and the program ANNA. The Demonstratorprocesses the code it received and sets the motor outputs accordingly.

3. ANNA drives and collects data: While driving, the Demonstrator col-lects image data and maps it to the user input. A new image is collectedevery time the user gives a new command or every 0.2 seconds. This gives theminimum frame rate of 5 fpsOne image is a 100×100 grey-scale image reshaped into a 1×10000 row vectorin row-major order (fortran order). This is stored in a n×10000 matrix, whilethe user-input is stored in a n × 1 vector. This is then saved to a ”.mat” filethat later is used for data analysis.

This software is, together with the program PythonNeuralNetwork (see Section 3.5),the foundation of the training of the ANN. It has also been used when testing therobot. For the full implementation and code, see Appendix B.1 and B.2

18

3.4. SOFTWARE

Initializingand setup.

Timer starts

Was there a keyboardevent?

Is the timer > 0.2seconds?

Collect and savekeyboard data

Send latestsaved collectedkeyboard dataand reset timer

Was keyboard data”KeyboardInterupt”?

Send abort toANNA andterminate

all activites

Exit

Data to ANNA

No

Yes

Yes

No

No

Yes

Figure 3.4: Flowchart describing DriverApp

19

CHAPTER 3. DEMONSTRATOR

Initializingand setup

Wait untilnew data hasbeen received

What was the datareceived?

Set motor pa-rameters to right

Set motorparametersto forward

Set motorparameters to left

Set motor pa-rameters to stop

Drive withaccording to themotor parameter

Terminateall activites

Exit

Data from DriveApp

forwardright

leftstop

abort

Figure 3.5: Flowchart describing ANNA

20

3.5. PYTHONNEURALNETWORK

3.5 PythonNeuralNetworkThis is the ANN algorithm itself, developed to work together with ANNA andthe data that has been collected. The software implements the ANN algorithmdescribed in Chapter 2.

3.6 ANNADriveANNADrive is a combination of the ANNA and PythonNeuralNetwork. It takesthe camera and the motor control capabilities from ANNA and the feed-forwardneural network from PythonNeuralNetwork and combines. The camera gives to thefeed-forward neural network, which analyses the image data and gives the motorcontrol the order to drive forward, backward, right or left (See flowchart in figure3.6).

Initializingand setup

Take picture

Run throughFeed-Forward

Neural Network

What was the outputfrom the Neural

Network?

Set motor pa-rameters to right

Set motorparametersto forward

Set motorparameters to left

Set motor pa-rameters to stop

Drive accordingto the motorparameter

forwardright

leftstop

Figure 3.6: Flowchart describing how ANNADrive works.

21

Chapter 4

Results

4.1 Accuracy for different size of training setsTable 4.1 shows the accuracy given the number of images in the training set.

Table 4.1: Number of images in training set and Accuracy

Number of images accuracy180 0.74540 0.76900 0.78

4.2 Accuracy for different resolutionsTable 4.2 shows the difference between the accuracies depending on the image res-olution.

Table 4.2: Image resolutions and accuracy

Image Resolution Accuracy25 × 25 0.7450 × 50 0.78

100 × 100 0.78

4.3 Demonstrator PerformanceThe Demonstrator presented in Chapter 3 could identify lanes and stop signs usingthe camera (see 4.1 for a example image) as its only input to the Artificial NeuralNetwork with good consistency. Reaction on the lanes were immediate and accurate

23

CHAPTER 4. RESULTS

in most cases. There were however times when the Demonstrator faulted and brokethe lanes.

Figure 4.1: Example of image from training set

24

Chapter 5

Discussion and conclusions

5.1 DiscussionTo implement a Artificial Neural Network algorithm as means of control for a Au-tonomous Vehicle worked well. As mentioned in Section 4.3, the Demonstratorperforms well under the limitations stated in the Scope. There are however caseswhen the Demonstrator faults and broke the lanes. This behavior was to be ex-pected, since the algorithm relies on image data provided in a learning stage andnot all situations might have occurred.

It is clear, as seen in table 4.1, that the more images provided in the learning stage,the more accurate the ANN will become. As discussed in the previous paragraph,the more image data provided should give the ANN more situations that it willrecognize and classify accuratly.

More surprisingly is that the image resolution, as shown in table 4.2, seems to havea limited effect on the accuracy. This might be limited to the task at hand; asshown in figure 4.1 the lanes are bright and can easily be identified in images withlow resolutions.

5.2 ConclusionsThe ANN is good algorithm for recognizing patterns in a image, it can with atraining set containing 900 images classify a image with 78 % accuracy. It is notedthat more images in the training set gives a higher accuracy. For the task to classifyimages that then should be used as means of control of a Autonomous Vehicle itcan be shown that the image resolution has little effect on the accuracy in theclassifications.

25

Chapter 6

Recommendations and future work

This project has been wonderfully interesting, and has provided the author of thisthesis with plenty of challenges. An ANN is complex, just as the human brain,and even though the concept is easy to grasp there are a few notations that are ofinterest.

Data collection is key. Just as the human brain, practice makes perfect and thismeans the ANN needs many examples to train with. Therefore, for the demonstra-tor to be able to work even better, more time should be spent on collecting data.

In this thesis the focus has been on the ANN, but in order to build a functioningAV one might add other sensors and make the demonstrator face more situationsthen those limitations stated in the scope.

27

Bibliography

Awad, M. and Khanna, R. (2015). Efficient Learning Machines: teories, Concepts,and Applications for Engineers and System Designers. Apress Media, LLC.

Bell, J. (2015). Machine Learning: Hands-On for Developers and Technical Profes-sionals. John Wiley & Sons, Inc.

Cheng, H. (2011). Advances in Computer Vision and Pattern Recognition. Springer-Verlag London Limited.

Hagen, C. (2013). Welcome to rpio’s documentation! Available from: https://pythonhosted.org/RPIO/ [cited 2016-05-04].

Jones, D. (2013, 2014). picamera. Available from: http://picamera.readthedocs.io/en/release-1.10/ [cited 2016-05-04].

Lundh, F. and Clark, A. (2016). Overview. Available from: https://pillow.readthedocs.io/en/3.2.x/handbook/overview.html [cited 2016-05-04].

Manyika, J., Chui, M., Bughin, J., Dobbs, R., Bisson, P., and Marrs, A. (2013).Disruptive technologies: Advances that will transform life, business, and the globaleconomy. McKinsey Global Institute.

Marsland, S. (2009). Machine Learning: An Algorithmic Perspective. Chapman &Hall/CRC.

Mitchell, T. M. (2006). The Discipline of Machine Learning. Technical ReportCMU-ML-06-108, School of Computer Science, Carnegie Mellon University.

pygame.org (2011). About pygame. Available from: http://www.pygame.org/readme.html [cited 2016-05-04].

Python Software Foundation (2016). What is Python? Available from: https://docs.python.org/3.5/faq/general.html#what-is-python [cited 2016-05-04].

Raschka, S. (2015). Python Machine Learning. Packt Publishing.

Raspberry Pi Foundation (2016a). Raspberry Pi 2 Model B. Available from: https://www.raspberrypi.org/products/raspberry-pi-2-model-b/ [cited 2016-05-03].

29

BIBLIOGRAPHY

Raspberry Pi Foundation (2016b). Raspberry Pi 2 Model B. Available from: https://www.raspberrypi.org/products/camera-module/ [cited 2016-05-03].

Russel, S. and Norvig, P. (2003, 1995). Artifical Intelligence A Modern Approach.Pearson Education, Inc, 2nd edition.

Texas Instruments (2015). DRV8833 Dual H-Bridge Motor Driver. Available from:https://www.pololu.com/file/0J534/drv8833.pdf [cited 2016-05-04].

The Scipy Community (2015). Overview. Available from: https://www.scipy.org[cited 2016-05-04].

VGer (2013). Raspberry pi camera case/enclosure. Available from: http://www.thingiverse.com/thing:92208 [cited 2016-05-08].

30

Appendix A

Additional information

A.1 Vectorised calculation of forward-propagationThe following equations constitute a vectorised implementation of forward propa-gation.

a(1) = x (A.1)

(add bias units to a(1))

z(2) = W (1)a(1) (A.2)

a(2) = g(z(2)) (A.3)

(add bias units to a(2))

z(3) = W (2)a(2) (A.4)

a3 = g(z3) (A.5)

A.2 Vectorised calculation of backward-propagation

δ(3) = a(3) − y (A.6)

δ(2) = δ(3)δ(2). ∗ h′z(2) (A.7)

31

APPENDIX A. ADDITIONAL INFORMATION

(add bias units to z(2) and remove bias units from δ(2))

∂J(W (1))∂(W (1))

= δ(2)a(1)

n+ λ

mW (1) (A.8)

(set bias units to zero in ∂J(W (1))∂W (1) )

∂J(W (2))∂(W (2))

= δ(3)a(2)

n+ λ

mW (2) (A.9)

(set bias units to zero in ∂J(W (2))∂W (2) )

32

Appendix B

Software and Code

B.1 DriverApp

import socke t #Import soc ke t moduleimport pygameimport sysimport time

def initpygame ( ) :pygame . i n i t ( )pygame . key . s e t r e p e a t (100 , 100)s i z e = width , he ight = 320 , 240speed = [ 2 , 2 ]s c r e en = pygame . d i sp l ay . set mode ( s i z e )

def main ( ) :s = socket . socke t ( ) # Create a sock e t o b j e c thost = socket . gethostbyname ( socke t . gethostname ( ) )port = 8081 # Reserve a por t f o r your s e r v i c e .s . bind ( ( host , port ) ) # Bind to the por ts . l i s t e n (5 ) # Now wai t f o r c l i e n t connect ion .print ( ’ L i s t en ing on ’+ str ( host )+ ’ : ’+str ( port ) , ’ . . . ’ )c l i e n t , addr = s . accept ( ) # E s t a b l i s h connect ion wi th c l i e n t .print ( ’Got connect ion from ’ , addr )forward = backward = l e f t = r i g h t = stop = ’ 0 ’r e s p o n s e l i s t b u f = r e s p o n s e l i s t = [ ]initpygame ( )s t a r t = time . time ( )re sponse = ””response = ”” . j o i n ( r e s p o n s e l i s t )

33

APPENDIX B. SOFTWARE AND CODE

while 1 :try :

for event in pygame . event . get ( ) :i f event . type == pygame .QUIT:

c l i e n t . send ( ’ 33333 ’ . encode ( ) )print ( ’ \ nTerminating connect ion . ’ )time . s l e e p ( 1 . 5 )input ( ’ Have you saved the datase t ? ’ )c l i e n t . c l o s e ( ) # Close the connect ionprint ( ’Done ’ )sys . e x i t ( )

i f event . type == pygame .KEYDOWN:r e s p o n s e l i s t b u f = r e s p o n s e l i s ti f event . key == pygame . K w :

forward = ’ 1 ’i f event . key == pygame . K s :

backward = ’ 1 ’i f event . key == pygame . K a :

l e f t = ’ 1 ’i f event . key == pygame . K d :

r i g h t = ’ 1 ’i f event . key == pygame . K LSHIFT :

stop = ’ 1 ’r e s p o n s e l i s t = [ forward , backward , r i ght , l e f t , stop ]i f r e s p o n s e l i s t != r e s p o n s e l i s t b u f :

r e sponse = ””response = ”” . j o i n ( r e s p o n s e l i s t )c l i e n t . send ( re sponse . encode ( ) )print ( re sponse )

i f event . type == pygame .KEYUP:i f event . key == pygame . K w :

forward = ’ 0 ’i f event . key == pygame . K s :

backward = ’ 0 ’i f event . key == pygame . K a :

l e f t = ’ 0 ’i f event . key == pygame . K d :

r i g h t = ’ 0 ’i f event . key == pygame . K LSHIFT :

stop = ’ 0 ’r e s p o n s e l i s t = [ forward , backward , r i ght , l e f t , stop ]re sponse = ””response = ”” . j o i n ( r e s p o n s e l i s t )

34

B.2. ANNA

c l i e n t . send ( response . encode ( ) )print ( re sponse )

i f time . time ()− s t a r t > 0 . 2 :c l i e n t . send ( response . encode ( ) )s t a r t = time . time ( )

except KeyboardInterrupt :c l i e n t . send ( ’ 33333 ’ . encode ( ) )print ( ’ \ nTerminating connect ion ’ )time . s l e e p ( 1 . 5 )user = input ( ’ Have you saved the datase t ? ’ )c l i e n t . c l o s e ( ) # Close the connect ionprint ( ’Done ’ )sys . e x i t ( )

main ( )

B.2 ANNA

B.2.1 Main code

import RPi .GPIO as GPIOimport socke timport sysimport i oimport picameraimport picamera . arrayimport numpy as npimport s c ipy . i o as s i ofrom PIL import Imageimport timefrom datet ime import datet imefrom motorClass import ∗

ProgramStart = time . time ( )camera = picamera . PiCamera ( )camera . r e s o l u t i o n = (100 ,100)

def f l u s h ( s ) :t ra sh = ’ ’for i in range (1 , 10 )

t ra sh = ’ ’ . j o i n ( s . recv ( 5 ) . decode ( ) )

35

APPENDIX B. SOFTWARE AND CODE

def dataAccumulator ( newdata , newinput , dataaccX , dataaccY ) :’ ’ ’This func t i on i s used to reduce the heavy computing that i s done when appending the image data to the”newdata” array . The func t i on accumulates the image data to a l a r g e r accumulator matrix . This i s doneboth f o r the image data and the user−input .

: param newdata : Array with data to be accumulated: param newinput : Array with user−input to be accumulated: param dataaccX : Accumulator o f image data: param dataaccY : Accumulator o f user−input: r e turn : The data accumulators and empty ar rays f o r which to s t o r e new data’ ’ ’

save = ’R ’while save != ’ y ’ and save != ’n ’ :

save = str ( input ( ”Save? Y/N” ) )save = save . lower ( )i f save != ’ y ’ and save != ’n ’ :

print ( ”Wrong input ! ” )

i f save == ’ y ’ :dataaccX = np . vstack ( ( dataaccX , newdata ) )dataaccY = np . vstack ( ( dataaccY , newinput ) )newdata = np . matrix (np . ones ( ( 1 , 1 0 0 0 0 ) ) )newinput = np . matrix (np . ones ( ( 1 , 1 ) ) )print ( ’ ’ )print ( ’−−−−−−−−−Data accumulated−−−−−−−−−− ’ )print ( ’ ’ )

else :newdata = np . matrix (np . ones ( ( 1 , 1 0 0 0 0 ) ) )newinput = np . matrix (np . ones ( ( 1 , 1 ) ) )print ( ’ ’ )print ( ’−−−−−−−−−No data accumulated−−−−−−−−−− ’ )print ( ’ ’ )

return dataaccX , dataaccY , newdata , newinput

def image2matrix ( user input , newdata , newinput ) :’ ’ ’The func t i on takes a image , conver t s i t to g r e y s c a l e and then s t o r e s i t as an array . The image i s mapped tothe user−input

: param user input : The user−input f o r the cur rent image: param newdata : Matrix f o r s t o rage o f image data

36

B.2. ANNA

: param newinput : Matrix f o r s t o rage o f user−input: r e turn : newdata and newinput’ ’ ’

s t a r t = time . time ( )with picamera . array . PiRGBArray( camera ) as stream :

camera . capture ( stream , format = ’ rgb ’ , u s e v i d e o p o r t=True )#p r i n t ( type ( stream . array ))img = Image . fromarray ( stream . array ) . convert ( ’L ’ ) # Convert ing to g r e y s c a l eimgdata = np . matrix (np . asar ray ( img . getdata ( ) ) )newdata = np . vstack ( ( newdata , imgdata ) )newinput = np . vstack ( ( newinput , u se r input ) )print ( time . time ()− s t a r t )return newdata , newinput

def abort ( s ) : # Close connect ion to avoid occupied por tprint ( ’ Shutt ing down ’ )GPIO. cleanup ( )s . c l o s e ( )

def read inputs ( s , dataaccX , dataaccY , newdata , newinput ) : # Read inpu t s from socke tmsg = ’ ’msg = ’ ’ . j o i n ( s . recv ( 5 ) . decode ( ) )print ( ’ Message : ’ ,msg)i f msg == ’ 33333 ’ :

dataaccX , dataaccY , newdata , newinput = dataAccumulator ( newdata , newinput , dataaccX , dataaccY )f i l ename = str ( ” DataSet ”+str ( datet ime . now ( ) . date ())+ ” ”+str ( datet ime . now ( ) . time ( ) ) )f i l ename = f i l ename . r e p l a c e ( ’ . ’ , ’ ’ )s i o . savemat ( f i l ename , { ’X ’ : dataaccX , ’Y ’ : dataaccY } , appendmat=True )abort ( s )sys . e x i t (0 )

e l i f len (msg) != 4 :return ’ 0 ’ , ’ 0 ’ , ’ 0 ’ , ’ 0 ’ , ’ 0 ’ #I f not 5 by tes , s top f o r next t ranmiss ion

else :return msg [ 0 ] , msg [ 1 ] , msg [ 2 ] , msg [ 3 ] , msg [ 4 ]

def main ( argv ) :dataaccX = newdata = np . matrix (np . ones ( ( 1 , 1 0 0 0 0 ) ) )dataaccY = newinput = np . matrix (np . shape ( ( 1 , 1 ) ) )

# Set GPIO pin numbersB1 = 16

37

APPENDIX B. SOFTWARE AND CODE

B2 = 12A2 = 22A1 = 18Drive = Motor (A1 , A2 , B1 , B2)try :

host = str ( argv [ 1 ] )port = int ( argv [ 2 ] )

except IndexError :host = input ( ’ Input host ip : ’ )port = int ( input ( ’ Input host port : ’ ) )

s = socket . socke t ( )s . connect ( ( host , port ) )print (np . shape ( newdata ) )print (np . shape ( dataaccX ) )

while ( 1 ) : #Drive looptry :

forward , backward , r i ght , l e f t , stop = read inputs ( s , dataaccX , dataaccY , newdata , newinput )i f np . shape ( newdata ) [ 0 ] > 100 :

Drive . stop ( )f l u s h ( s )dataaccX , dataaccY , newdata , newinput = dataAccumulator ( newdata , newinput , dataaccX , dataaccY )

i f stop == ’ 1 ’ :Drive . stop ( )time . s l e e p (3 )Drive . forward ( )time . s l e e p (1 )Drive . stop ( )newdata , newinput = image2matrix (4 , newdata , newinput )

e l i f forward == ’ 1 ’ :i f ( r i g h t == ’ 1 ’ ) and ( l e f t == ’ 0 ’ ) : # Go r i g h t

newdata , newinput = image2matrix (3 , newdata , newinput )Drive . r i g h t ( )

e l i f ( l e f t == ’ 1 ’ ) and ( r i g h t == ’ 0 ’ ) : # Go l e f tnewdata , newinput = image2matrix (2 , newdata , newinput )Drive . l e f t ( )

else : # Just forwardnewdata , newinput = image2matrix (1 , newdata , newinput )Drive . forward ( )

e l i f backward == ’ 1 ’ :i f ( r i g h t == ’ 1 ’ ) and ( l e f t == ’ 0 ’ ) : #Backr igh t

Drive . backwardRight ( )e l i f ( l e f t == ’ 1 ’ ) and ( r i g h t == ’ 0 ’ ) : #B a c k l e f t

Drive . backwardLeft ( )

38

B.2. ANNA

else :Drive . backward ( )

else : # Fu l l s topDrive . stop ( )

except KeyboardInterrupt :abort ( s )break

i f name == ” ma in ” :main ( sys . argv [ 0 : ] )

B.2.2 motorClass used in Main

import RPi .GPIO as GPIO

class Motor ( ) :def i n i t ( s e l f , motorPinA1 , motorPinA2 , motorPinB1 , motorPinB2 ) :

s e l f . a in 1 , s e l f . a in 2 , s e l f . b in 1 , s e l f . b in 2 = s e l f . initGPIO ( motorPinA1 , motorPinA2 , motorPinB1 , motorPinB2 )

def initGPIO ( s e l f , A1 , A2 , B1 , B2 ) : #GPIO pins f o r PWMGPIO. setmode (GPIO.BOARD)GPIO. setwarn ings ( Fa l se )GPIO. setup (A1 , GPIO.OUT)GPIO. setup (A2 , GPIO.OUT)GPIO. setup (B1 , GPIO.OUT)GPIO. setup (B2 , GPIO.OUT)a in 1 = GPIO.PWM(A1,400 )b in 1 = GPIO.PWM(B1 ,400 )a in 2 = GPIO.PWM(A2,400 )b in 2 = GPIO.PWM(B2 ,400 )a in 1 . s t a r t (0 )b in 1 . s t a r t (0 )a in 2 . s t a r t (0 )b in 2 . s t a r t (0 )return ain 1 , a in 2 , bin 1 , b in 2

def r i g h t ( s e l f ) :s e l f . a i n 2 . ChangeDutyCycle (0 )s e l f . b in 2 . ChangeDutyCycle (0 )s e l f . a i n 1 . ChangeDutyCycle (1 )s e l f . b in 1 . ChangeDutyCycle (45)

def l e f t ( s e l f ) :

39

APPENDIX B. SOFTWARE AND CODE

s e l f . a i n 2 . ChangeDutyCycle (0 )s e l f . b in 2 . ChangeDutyCycle (0 )s e l f . a i n 1 . ChangeDutyCycle (45)s e l f . b in 1 . ChangeDutyCycle (1 )

def forward ( s e l f ) :s e l f . a i n 2 . ChangeDutyCycle (0 )s e l f . b in 2 . ChangeDutyCycle (0 )s e l f . a i n 1 . ChangeDutyCycle (20)s e l f . b in 1 . ChangeDutyCycle (20)

def backwardRight ( s e l f ) :s e l f . a i n 1 . ChangeDutyCycle (0 )s e l f . b in 1 . ChangeDutyCycle (0 )s e l f . a i n 2 . ChangeDutyCycle (1 )s e l f . b in 2 . ChangeDutyCycle (45)

def backwardLeft ( s e l f ) :s e l f . a i n 1 . ChangeDutyCycle (0 )s e l f . b in 1 . ChangeDutyCycle (0 )s e l f . a i n 2 . ChangeDutyCycle (45)s e l f . b in 2 . ChangeDutyCycle (1 )

def backward ( s e l f ) :s e l f . a i n 1 . ChangeDutyCycle (0 )s e l f . b in 1 . ChangeDutyCycle (0 )s e l f . a i n 2 . ChangeDutyCycle (40)s e l f . b in 2 . ChangeDutyCycle (40)

def stop ( s e l f ) :s e l f . a i n 1 . ChangeDutyCycle (0 )s e l f . b in 1 . ChangeDutyCycle (0 )s e l f . a i n 2 . ChangeDutyCycle (0 )s e l f . b in 2 . ChangeDutyCycle (0 )

B.3 ANNADrive

import RPi .GPIO as GPIOimport picameraimport picamera . arrayimport numpy as npimport s c ipy . i o as s i ofrom PIL import Imagefrom motorClass import ∗

40

B.3. ANNADRIVE

from NeuralNetworkFunctions import ∗

camera = picamera . PiCamera ( )camera . r e s o l u t i o n = (100 ,100)

def image2matrix ( ) :s t a r t = time . time ( )with picamera . array . PiRGBArray( camera ) as stream :

camera . capture ( stream , format = ’ rgb ’ , u s e v i d e o p o r t=True )

# Convert ing to g r e y s c a l eimg = Image . fromarray ( stream . array ) . convert ( ’L ’ )imgdata = np . matrix (np . asar ray ( img . getdata ( ) ) )return imgdata

def main ( ) :

# SetupB1 = 16B2 = 12A2 = 22A1 = 18# Creates a motor o b j e c tDrive = Motor (A1 , A2 , B1 , B2)

data = s i o . loadmat ( ’ Weights1 . mat ’ )theta1 = data [ ’ Theta1 ’ ]theta2 = data [ ’ Theta2 ’ ]print ( theta1 )print ( theta2 )while True :

Drive . stop ( )X = image2matrix ( )Order = nnPredict ( theta1 , theta2 ,X)print ( Order )i f Order == 0 :

Drive . forward ( )e l i f Order == 1 :

Drive . l e f t ( )else :

Drive . r i g h t ( )

main ( )

41

TRITA MMKB 2016:26 MDAB087

www.kth.se