Sound detection with a real e-puck

1

Steve Hottinger, Sebastian Strobl, Gaël GermanoEPFL, 7th June 2016

Table of contents

2

1. Introduction 2. Goals 3. Methodology

• General equationso Phase shift methodo Bearing angle method

• Webots• Real e-puck

4. Implementation on the Webots / real e-puck5. Results6. Difficulties7. Conclusion

Introduction

3

• SIS course project

• Signal processing to localize sound sources• Example of applications:

- To look for people lost in the mountains- To spot mosquitos and kill them with laser (Lenz, 2009)Sound with E-puck

More Information: http://www.e-puck.org/

Speaker

sound wave

Goals

4

• Capture and analyze sound

• Move to the sound source

• Stop when it finds the sound source

• Simulation to real robot transition

General equations to localize sound source

5

Phase shift method

General equations to localize sound source

6

Phase shift method

Signal form:

The phase at microphone i for thesound source fequency (index k):

With B the imaginary part of Fourier coefficientand A the real partPhase shift between microphones:

In this example:

𝑥 𝑡 = $ 𝑅& ∗ cos 2𝜋𝑓& ∗ 𝑡 + 𝜑&012

&34

∆𝜑2,4 = 𝜑2 −𝜑4∆𝜑8,4 = 𝜑8 −𝜑4∆𝜑8,2 = 𝜑8 −𝜑2

∆𝜑8,2 = 𝜑8 −𝜑2 = 0∆𝜑2,4 = 𝜑2 − 𝜑4 > 0

𝜑;,< = atan𝐵𝐴

General equations to localize sound source

7

Bearing angle method

General equations to localize sound source

8

Bearing angle method

Bearing angle: angle between e-puck forward direction and sound source.

cos𝜃 =𝑐∆𝑇𝑑 =

𝑐∆𝜑2𝜋𝑓𝑑 𝑎𝑠∆𝑇 =

∆𝜑𝜔

with : ∆𝜑 = 𝜑< − 𝜑&calculated before

So :θ = cos12 K∆L

8MNO

Methodology - Webots

9

As we can see in the figure 5, the index 4 is the one with

the higher magnitude. Since we have no other source of sound, this corresponds to the sound emitted by the sound source. On the controller, the sound is processed with a Fast Fourier Transform (FFT). The phase is calculated like developed in equation 11. To find the good conditions to give order to the wheels, we follow the scheme from Chapter III. An empirical analysis using Matlab to systematize the procedures is done to find the conditions. The two following conditions must be fulfilled to move the e-puck.

Move Condition 1 Condition 2 Speed left

Speed right

Right 1.1 < 𝜃1,2 < 1.6 1.2 < 𝜃2,0 < 1.7 750 250 Left 0.8 < 𝜃0,1 < 1.3 1.6 < 𝜃2,0 < 2.0 250 750

In front 1.4 < 𝜃1,2 < 1.7 1.7 < 𝜃2,0 < 2.0 1000 1000 Behind 1.2 < 𝜃1,2 < 1.7 1.2 < 𝜃2,0 < 1.6 -250 250

We implement the e-puck so that it stops when it finds the source sound using the proximity sensors 0, 1, 6 and 7. When the proximity sensor values are bigger than 250, the robot stops and the lights start to blink.

VI. IMPLEMENTATION ON THE E-PUCK ROBOT A great part in the implementation of the e-puck robot was

to find the correct index. Indeed the greater difference with Webots implementation is the environment in which we work. It is not as quiet as before. They are several sources of sound, either the sound of the voices, the noise due to the wheels or to the microphone itself. At first, a research to find the good index on the e-puck has been done. We took samples of the sound and plotted them on Matlab in order to see the corresponding frequencies and amplitudes. Plot of results are done with the first one (figure 6) without the e-puck sound, the second one with the e-puck sound (figure 7) and the last one with a zoom on the good pick.

The plot shows that frequencies below indexes 10 are

important and correspond often to voices sounds.

Now amplitudes are much bigger than those of the

previous plot. 4 peaks are visible, one around index 20, one around 30, one around 35 and the last one around 45.

With a zoom, the right frequency index is 21 as we

predicted from the theoretical part in Chapter II. The others are not frequencies from the e-puck sound. The greatest amplitude is due to the sound of the wheels that are rotating.[2] Since they are the nearest to the microphones

Figure 5: indexation of the frequencies

Figure 6: indexation of frequencies of voices

Figure 7: indexation of frequencies

Figure 8: zoom on the correct frequency index

Sampling frequency fs = 33’000Size of the sample, N = 128

𝑓; =;∗NP0

Methodology – e-puck

10

0 5 10 15 20 25 30 35 40 45 500

0.2

0.4

0.6

0.8

1

1.2

1.4

frequency

ampl

itude

Searching index

0 5 10 15 20 25 30 35 40 45 500

20

40

60

80

100

120

frequency

ampl

itude

Searching index

Index k

Index k

𝑓; =;∗NP0

Sampling frequency fs = 16’384Size of the sample, N = 256

Methodology – e-puck

11

they are obviously those that make the most noise. The two others peaks might be due to the microphones themselves or some parasitic sources of sound. An empirical test was done with all these peaks.

After that we found the good frequency, the same implementation as on the Webots simulator has been done. Unfortunately, conditions with bearings angle didn’t work with the real e-puck. We tried to create the conditions by sampling data on each side of the e-puck and make averages and standard deviations for each one of the angle. We found out that there were a lot of NaN values when the e-puck was computing the values. These values come because of the bad quality of the microphones which are unable to get all the frequencies at all time. This is one of the problems that come in real life. We got rid of these NaN values in a Matlab code where we do the average of the non NaN values. A lot of tests were done to unsuccessfully guide the e-puck. We have decided then to use a different method involving the phase shift as explained in Chapter IV.

Move Condition Speed left wheel

Speed right wheel

Left 𝑑𝑝ℎ𝑖1,0 < 0 500 1000

Right 𝑑𝑝ℎ𝑖1,0 > 0 1000 500

Right Else 500 -500

The conditions for the left and the right wheels are explained in Chapter IV. A third condition is used in case the phase shift value is infinite and the robot goes to the right. In this situation we tell the robot to rotate and normally it should enter in one of the first conditions.

VII. RESULTS AND DISCUSSIONS On the Webots simulator, the e-puck searches and finds

always the e-puck sound and very fast. This is due to the fact that we are doing a simulation and we can work with synchronous time. It means that they are no time delays between the different information we give to the robots. It might indeed go even faster than real time if they are not too many robots running in the simulation. Also, we do not have any problem with the speaker and the microphones, which are able to get sound every time.

On the real robot, the e-puck search finds painfully the sound source and with less ease than on the simulator if we use the method of the bearing angles. With the phase shift method it finds well in all directions except when the sound source is behind, where it takes more times to find the source sound. This is due to the fact that it is the same signal as if the sound was in front, it has to wait until the right or left conditions take over.

On the Webots simulator, it is hard to do better. But on the real e-puck, there are some possible improvements to do:

x Run in a quiet room x Use more conditions to move the robot in the case

of the phase shift implementation

x Find condition to running the e-puck search with bearing angle as conditions

x Put better microphones on the e-puck search and a better speaker on the sound emitter.

VIII. CONCLUSION This project was about implementing an algorithm on an

e-puck such that it will find a sound source. We did that on the Webots simulator at first with no real problem since we worked in a neutral environment. The most efficient method was implemented on it (bearing angle method) and it worked quite well. However when we tried to implement the same method on the e-puck, it wasn’t as successful as on the simulation. We decided then to use the phase shift method. This is an easier method but less efficient.

If we had more time, we could try to implement a successful bearing angle method on the e-puck and then try to work in an environment with more obstacles which might create echoes. We could even try to find a way to change the index such that when the e-puck has find the first sound source it can then move on to the next one.

Working with real robots was a complete other experience than working with a simulation, even if it is made to allow the smoother transition possible from virtual to real world. It was sometimes a little bit discouraging, but so satisfying when it finally worked.

REFERENCES [1] Fang, J., & Barry, A. (03.16). Quantitative Methods II. EPFL,

Lausanne. [2] Valin, J. M., Michaud, F., Rouat, J., & Letourneau, D. (2003). Robust

sound source localization using a microphone array on a mobile robot. In 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. (IROS 2003). Proceedings (Vol. 2, pp. 1228–1233 vol.2). http://doi.org/10.1109/IROS.2003.1248813

Matlab code to find conditions

Methodology – e-puck

12

for (m=0; m<NB_SENSORS; m++){ds_value[m] = e_get_prox(sensor[m]);}

if(e_get_prox(0)>300 || e_get_prox(7)>300 || e_get_prox(1)>300 || e_get_prox(6)>300){e_set_speed_left(0);e_set_speed_right(0);

for (i=0;i<8;i++)//8{e_set_led(i,1);wait(10000);e_set_led(i,0);wait(10000); }

}else{

for (i=0;i<8;i++){e_set_led(i,0);}

}

Stopping the e-puck when it finds the sound source

Results Webots

13

Results e-puck

14

Lets show !

Difficulties

15

1. Understand well equations

2. Implementation on the real e-puck• Finding the good frequency• NaN problems• Theta conditions problem

Conclusion

16

• Difference between simulation and real life implementation

• Possibility to find the good frequency by itself

• Possibility to make the difference between an obstacle and the sound source

• Satisfaction when it finally work!

References

17

Lenz, C. (2009). Localization of Sound Sources, Studies on Mechatronics. PhD Thesis, AutonomousSystems Lab, Swiss Federal Institute of Technology Press, Zürich. Retrieved fromhttp://students.asl.ethz.ch/upl_pdf/127-report.pdf

Valin, J. M., Michaud, F., Rouat, J., & Letourneau, D. (2003). Robust sound source localization usinga microphone array on a mobile robot. In 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003. (IROS 2003). Proceedings (Vol. 2, pp. 1228–1233 vol.2). http://doi.org/10.1109/IROS.2003.1248813

Questions ???

18