Air Piano Documentation

Air Piano DocumentationRelease 1.0

Angel Martínez Cañadas

Aug 08, 2017

Contents

1 Installing and configuring Air Piano 31.1 Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Hardware configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Software configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Installing the application and its requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Run the application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Using Air Piano 72.1 Typical usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 Key bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Sources - airpiano Module 9

4 Sources - GameController Class 11

5 Sources - VirtualKeyboard Class 13

6 Sources - Circle Class 15

7 Sources - Notes Class 17

8 Sources - Camera Class 19

9 Sources - RTOpenCV Class 21

10 Sources - ImageProcessor Class 23

11 Indices and tables 25

Python Module Index 27

i

ii

Air Piano Documentation, Release 1.0

Contents:

Contents 1


2 Contents

CHAPTER 1

Installing and configuring Air Piano

Hardware Requirements

• Raspberry Pi Model B or Raspberry Pi 2 Model B.

• SD Card (you can check recomend SD card size an type at https://www.raspberrypi.org/documentation/installation/sd-cards.md).

• Camera Module (https://www.raspberrypi.org/products/camera-module/)

• HDMI cable and USB charger

• Optional: wireless keyboard/mouse

Hardware configuration

Install the camera module, check https://www.raspberrypi.org/help/camera-module-setup/ if you have not done it be-fore.

When possible, use an HDMI cable and connect to a tv or sound system, the quality difference between the analogoutput and HDMI is considerable.

Wire and connect the rest of accesories as usual (wired/wireless keyboard, wifi, etc.). This configurations are notincluded here, as they are aside of this project.

Software configuration

Install Raspbian on the SD. The download page is at https://www.raspberrypi.org/downloads/

I personally recommend using NOOBS to install Raspbian (https://www.raspberrypi.org/downloads/noobs/)

Follow the instructions given in the download page.

3

https://www.raspberrypi.org/documentation/installation/sd-cards.md

https://www.raspberrypi.org/documentation/installation/sd-cards.md

https://www.raspberrypi.org/products/camera-module/

https://www.raspberrypi.org/help/camera-module-setup/

https://www.raspberrypi.org/downloads/

https://www.raspberrypi.org/downloads/noobs/


In order to use Air Piano, the camera module must be enabled and the Pi should boot to graphical desktop. This canbe configured with raspi-config utility. Open a terminal and type:

$ sudo raspi-config

Select this two options and activate the desired options:

• 3 Enable Boot to Desktop/Scratch -> Desktop Log in

• 5 Enable Camera -> Enable

Reboot the Pi

Installing the application and its requirements

1. Open a terminal window.

2. First, update the entire system with this two commands:

$ sudo apt-get update$ sudo apt-get upgrade

3. Get Air Piano source code:

$ git clone https://github.com/amcanadas/raspberry-pi-air-piano

4. Get into the new created folder:

$ cd raspberry-pi-air-piano

5. Then, install the dependencies, starting whith OpenCV, the computer vision library:

$ sudo apt-get install python-opencv

6. More complicated and time consuming is installing the resampling library, be patient. Use this commands todownload/compile:

$ wget http://www.mega-nerd.com/SRC/libsamplerate-0.1.8.tar.gz$ tar xvfz libsamplerate-0.1.8.tar.gz$ cd libsamplerate-0.1.8.tar.gz$ ./configure$ make$ make check

7. At this point, all test should be passed. Finally, actually install the library and its python wrapper:

$ sudo make install$ sudo ldconfig -v$ sudo apt-get install python-setuptools$ sudo apt-get install python-dev$ sudo easy_install scikits.samplerate

8. All requirements installed!. Cleanup:

$ cd ..$ rm -rf libsamplerate

4 Chapter 1. Installing and configuring Air Piano


Run the application

Option 1: from command line.

To run the application from the command line. Type:

$ ./play.sh

Option 2: graphical

Alternativelly you can execute the application directly from the File Manager:

1. Open File Manager (icon on the task bar).

2. Navigate to the folder where Air Piano was downloaded.

3. Double Click on play.sh. If a dialog appears, select Execute.

1.5. Run the application 5


6 Chapter 1. Installing and configuring Air Piano

CHAPTER 2

Using Air Piano

Typical usage

Step 1: run the application

Step 2:

Step 3:

Key bindings

Performance

The followind table shows some tests. Frames per second (fps) refers to frames captured and processed in a second.

Hardware Overclock SpeedRaspberry Pi 2 B no overclock 14 fpsRaspberry Pi B no overclock 1.8 fpsRaspberry Pi B overclock set to High 2.2 fps

Clearly, Raspberry Pi 2 is recommended for this application, since 2 fps makes Air Piano hardly usable in a musicplayout.

7


8 Chapter 2. Using Air Piano

CHAPTER 3

Sources - airpiano Module

Air Piano [email protected]

Dependencies: Python 2.7 NumPy, SciPy, PyGame, Scikits.samplerate, OpenCV

9

mailto:[email protected]


10 Chapter 3. Sources - airpiano Module

CHAPTER 4

Sources - GameController Class

class gamecontroller.GameControllerControls the Piano App (Game) status/behaviour

_draw_background()

_init_calibrating()

static get_instance()

instance = None

static process_image(rtopencv_class, frame)

process_key(key)

redraw()

start_fgsegmentation(status)

stop_fgsegmentation()

test_frame()

11


12 Chapter 4. Sources - GameController Class

CHAPTER 5

Sources - VirtualKeyboard Class

class virtualkeyboard.VirtualKeyboard(bounding_box)A Virtual Piano Keyboard

_generate_test_points()

_key_test_matrix(p1, p2, p3, p4)

_point_between(x1, y1, x2, y2, relative_distance)

pressed(i)

test(mask)

unpressed(i)

13


14 Chapter 5. Sources - VirtualKeyboard Class

CHAPTER 6

Sources - Circle Class

circle.Circle

15


16 Chapter 6. Sources - Circle Class

CHAPTER 7

Sources - Notes Class

class notes.Notes

generate_scale()Given the initial note, middle C, create the rest of the musical scale by resampling.

Returns: Dictionary of musical scale with the key being the name of the note and the value being thecorresponding sound object.

play(note)

17


18 Chapter 7. Sources - Notes Class

CHAPTER 8

Sources - Camera Class

class camera.Camera(size)OpenCv Camera Wrapper

_to_pygame(frame)

close()

get_image(image_format=’bgr’)

get_pygame_image()

19


20 Chapter 8. Sources - Camera Class

CHAPTER 9

Sources - RTOpenCV Class

class rtopencv.RTOpenCV(camera, callback)

static _discard_frames(last_frame)

static _streams()

close()

get_fps()

run()

status_lock = <thread.lock object>

21


22 Chapter 9. Sources - RTOpenCV Class

CHAPTER 10

Sources - ImageProcessor Class

class imageprocessor.ImageProcessor

run()

23


24 Chapter 10. Sources - ImageProcessor Class

CHAPTER 11

Indices and tables

• genindex

• modindex

25


26 Chapter 11. Indices and tables

Python Module Index

aairpiano, 9

27


28 Python Module Index

Index

Symbols_discard_frames() (rtopencv.RTOpenCV static method),

21_draw_background() (gamecontroller.GameController

method), 11_generate_test_points() (virtualkey-

board.VirtualKeyboard method), 13_init_calibrating() (gamecontroller.GameController

method), 11_key_test_matrix() (virtualkeyboard.VirtualKeyboard

method), 13_point_between() (virtualkeyboard.VirtualKeyboard

method), 13_streams() (rtopencv.RTOpenCV static method), 21_to_pygame() (camera.Camera method), 19

Aairpiano (module), 9

CCamera (class in camera), 19Circle (in module circle), 15close() (camera.Camera method), 19close() (rtopencv.RTOpenCV method), 21

GGameController (class in gamecontroller), 11generate_scale() (notes.Notes method), 17get_fps() (rtopencv.RTOpenCV method), 21get_image() (camera.Camera method), 19get_instance() (gamecontroller.GameController static

method), 11get_pygame_image() (camera.Camera method), 19

IImageProcessor (class in imageprocessor), 23instance (gamecontroller.GameController attribute), 11

NNotes (class in notes), 17

Pplay() (notes.Notes method), 17pressed() (virtualkeyboard.VirtualKeyboard method), 13process_image() (gamecontroller.GameController static

method), 11process_key() (gamecontroller.GameController method),

11

Rredraw() (gamecontroller.GameController method), 11RTOpenCV (class in rtopencv), 21run() (imageprocessor.ImageProcessor method), 23run() (rtopencv.RTOpenCV method), 21

Sstart_fgsegmentation() (gamecontroller.GameController

method), 11status_lock (rtopencv.RTOpenCV attribute), 21stop_fgsegmentation() (gamecontroller.GameController

method), 11

Ttest() (virtualkeyboard.VirtualKeyboard method), 13test_frame() (gamecontroller.GameController method),

11

Uunpressed() (virtualkeyboard.VirtualKeyboard method),

13

VVirtualKeyboard (class in virtualkeyboard), 13

29

Documents

Air Piano Documentation