Kinect=002=skeleton tracking 2d

KINECTSkeleton Tracking

2D

02

REVIEW

import SimpleOpenNI.*;

SimpleOpenNI context;

///////////////////////////////////////////////

void setup(){

context = new SimpleOpenNI (this);

context.enableDepth();

context.enableUser();

if(context.isInit() == false){

println("Maybe your Kinect is not connected");

exit();

return;

}

size(context.depthWidth(), context.depthHeight());

}

///////////////////////////////////////////////

void draw(){

context.update();

image(context.userImage(),0,0);

}

Try to

explain ?what does each lines mean



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


}

Import library.

Create an simpleOpenNI object.

Initiate this simpleOpenNI object.

Enable Depth calculation.

Enable User(ID) calculation.

Detect Kinect’s connection

Size of the Window

Update the data

Draw User’s Coloring Image

If you are

IN A HURRY

Open the user example to get it.

But I will

Try to Simplify the code



color[] userClr = new color[]{ color(255,0,0),

color(0,255,0),

color(0,0,255),

color(255,255,0),

color(255,0,255),

color(0,255,255)

};

///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


}

Color for idetifying the users

If you try to compare the differences between

the user example and our previous code.

Here is the first difference:

Which means each of the user identified

by Kinect will has its own skeleton color.



color[] userClr = new color[]{ color(255,0,0),

color(0,255,0),

color(0,0,255),

color(255,255,0),

color(255,0,255),

color(0,255,255)

};

///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


}

But to ME, I don’t need it.

(you can still keep it if you like).



PVector com = new PVector();

PVector com2d = new PVector();

///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


}

Center of Mass// Center of your figure

Second Difference:



PVector com = new PVector();

PVector com2d = new PVector();

///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


}

Center of Mass// Center of your figure

If we want to keep it simple for the first

time, we can still ignore them. (or talk

about it later).

ARE YOU

KIDDING ME?



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();


int [] userList = context.getUsers();

for(int i =0; i<userList.length; i++){

if(context.isTrackingSkeleton(userList[i])){

stroke(255,0,0);

drawSkeleton(userList[i]);

}

}

}

Detect the users.

For every users, they will …

If Kinect have tracked someone…

Draw the skeleton

Here comes

the REAL

stuffs.

But we don’t have

drawSkeleton(userList[i])

function yet!

HEAD

NECK

LEFT_SHOULDERRIGHT_SHOULDER

LEFT_ELBOWRIGHT_ELBOW

LEFT_HANDRIGHT_HAND

TORSO

RIGHT_HIP LEFT_HIP

RIGHT_KNEE LEFT_KNEE

RIGHT_FOOT LEFT_FOOT

void drawSkeleton(int userId)

{

// to get the 3d joint data

/*

PVector jointPos = new PVector();

context.getJointPositionSkeleton(userId,SimpleOpenNI.SKEL_NECK,jointPos);

println(jointPos);

*/

context.drawLimb(userId, SimpleOpenNI.SKEL_HEAD, SimpleOpenNI.SKEL_NECK);

context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER);

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW);

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND);

context.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER);

context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW);

context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND);

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO);

context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);

context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP);

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE);

context.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT);

context.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP);

context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE);

context.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT);

}

Here you are (simply copy from example)

Let’s take a look!




Object

Function

Feed in Data 1:

UserID

Feed in Data 2:

Draw Something

Don’t be afraid of Copy&Paste,Unless you know how to Change.

(Coding is not an memorizing game)

The Last piece of your Skeleton puzzle.

void onNewUser(SimpleOpenNI curContext, int userId)

{

println("onNewUser - userId: " + userId);

println("\tstart tracking skeleton");

context.startTrackingSkeleton(userId);

}

void onLostUser(SimpleOpenNI curContext, int userId)

{

println("onLostUser - userId: " + userId);

}

void onVisibleUser(SimpleOpenNI curContext, int userId)

{

//println("onVisibleUser - userId: " + userId);

}

Lost&Found Users(simply copy from example)



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();





stroke(255,0,0);


}

}

}


{


/*



println(jointPos);

*/
















}

// SimpleOpenNI user events///////////////////////////////////////////////////////////////////////////////////////


{




}


{


}


{


}

THE color: (255,0,0)

Free

As

A

Bird

I want my Head!Let’s create a function!

void drawHead(int userId){

PVector jointPosH = new PVector();

//Create a Vector (to capture the Head joint).

context.getJointPositionSkeleton(userId, SimpleOpenNI.SKEL_HEAD, jointPosH);

//Get the Head Joint in 3D!

PVector jointPosH_Proj = new PVector();

//Create a Vector (for projecting on 2D).

context.convertRealWorldToProjective(jointPosH, jointPosH_Proj);

//Convert 3D Head Joint Vector to 2D screen (for projecting on 2D).

fill(0,255,0);

stroke(255,0,0);

ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50);

// Extract the x and y position from the Projection Head Joint to draw a circle.

}

Put it in the CODE



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();





stroke(255,0,0);


drawHead(userList[i]);

}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,0);

stroke(255,0,0);



}

Let’s reduceSome stuffs



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

context.update();

background(0);

//image(context.userImage(),0,0);




stroke(255,0,0);



}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,0);

stroke(255,0,0);



}

In this Virtual WorldNO One is FAT!

Change the ColorYou Like



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

background(0);

context.update();





strokeWeight(2);

stroke(0,255,255);



}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);

stroke(0,255,255.80);



}

Nude Descending a Staircase No. 2|| Marcel Duchamp'

Duchamp Effect



///////////////////////////////////////////////

void setup(){






exit();

return;

}


background(0);

}

///////////////////////////////////////////////

void draw(){

context.update();





strokeWeight(2);

stroke(0,255,255,80);



}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255,80);

stroke(0,255,255.80);



}

[ If-else ]Again



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

background(0);

context.update();





strokeWeight(2);

stroke(0,255,255,80);



PVector myHead = new PVector();

context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_HEAD, myHead);

PVector myHead_Proj = new PVector();

context.convertRealWorldToProjective(myHead, myHead_Proj);

if(myHead_Proj.x> width/2){

background(255,0,255);

}

else{


}

}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255,80);

stroke(0,255,255,80);



}

if(myHead_Proj.x > width/2)else

But, Dude,

Where is my BODY?

(you should know)



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){

background(0);

context.update();





PVector myHead = new PVector();

context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_HEAD, myHead);

PVector myHead_Proj = new PVector();

context.convertRealWorldToProjective(myHead, myHead_Proj);

if(myHead_Proj.x> width/2){


}

else{


}

strokeWeight(2);

stroke(0,255,255);



}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);

stroke(0,255,255);ellipse(jointPosH_Proj.x, jointPosH_Proj.y, 50, 50);


}


Here you go

[ If-else ]Change

Head to Hand



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////

void draw(){background(0);

context.update();





PVector myRHand = new PVector();

context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_RIGHT_HAND, myRHand);

PVector myRHand_Proj = new PVector();

context.convertRealWorldToProjective(myRHand, myRHand_Proj);

if(myRHand_Proj.x> width/2){


}

else{


}

strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);

ellipse(myRHand_Proj.x,myRHand_Proj.y,50,50);

stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);


}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);



}


GIVE ME

2 HANDS[ no If-else ]



///////////////////////////////////////////////

void setup(){






exit();

return;

}


}

///////////////////////////////////////////////


context.update();









PVector myLHand = new PVector();

context.getJointPositionSkeleton(userList[i], SimpleOpenNI.SKEL_LEFT_HAND, myLHand);

PVector myLHand_Proj = new PVector();

context.convertRealWorldToProjective(myLHand, myLHand_Proj);

strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);


ellipse(myLHand_Proj.x,myRHand_Proj.y,50,50);

stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);



}

}

}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);



}

We got hands!

Please save this file.

Make V-buttons

I will only keep the

“void draw()” part for

now, since there will

not be too many

changes in the rest.


context.update();

//Right button

noStroke();

fill(255,0,255,80);

ellipse(3*width/4,height/2,100,100);

stroke(255,0,255,80);

noFill();


//Left button

noStroke();

fill(255,255,0,80);

ellipse(width/4,height/2,100,100);

stroke(255,255,0,80);

noFill();













strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);



stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);



}

}

}

We need

pre-set

buttons

Keep them transparent, as a default mode

If you move your

hand to “?AREA”,

it will trigger the

buttons.


context.update();

//Right button

noStroke();

fill(255,0,255,80);


stroke(255,0,255,80);

noFill();


//Left button

noStroke();

fill(255,255,0,80);


stroke(255,255,0,80);

noFill();













strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);



stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);



//R_Trigger

if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-

25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-

25){

noStroke();

fill(255,0,255);


stroke(255,0,255);

noFill();


}

//L_Trigger

if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 &&

myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25){

noStroke();

fill(255,255,0);


stroke(255,255,0);

noFill();


}

}

}

}

Let’s

make it

clear.

//R_Trigger

if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25){

noStroke();

fill(255,0,255);


stroke(255,0,255);

noFill();


}

//L_Trigger

if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25){

noStroke();

fill(255,255,0);


stroke(255,255,0);

noFill();


}

Try to explain, please.

(width/4, height/2)

25

25


THINK:

if I want not only

Left hand for Left button

Right hand for Right button

//R_Trigger

if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25

|| myLHand_Proj.x<3*width/4+25 && myLHand_Proj.x>3*width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25)

){

noStroke();

fill(255,0,255);


stroke(255,0,255);

noFill();


}

//L_Trigger

if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25

|| myRHand_Proj.x<width/4+25 && myRHand_Proj.x>width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25

){

noStroke();

fill(255,255,0);


stroke(255,255,0);

noFill();


}

Put “OR” in the code

You are the dancing queen

Please save another file.

Let’s make

Some Noise!!!

Try to combine

an example from

“minim” library

Minim has been

embedded in

Processing.

Yours might look

different than this one,

but sure you can still

find this

“TriggerASample”

file.

import ddf.minim.*;

Minim minim;

AudioSample kick;

AudioSample snare;

void setup()

{

size(512, 200, P3D);

minim = new Minim(this);

// load BD.wav from the data folder

kick = minim.loadSample( "BD.mp3", // filename

512 // buffer size

);

// if a file doesn't exist, loadSample will return null

if ( kick == null ) println("Didn't get kick!");

// load SD.wav from the data folder

snare = minim.loadSample("SD.wav", 512);

if ( snare == null ) println("Didn't get snare!");

}

void draw()

{

background(0);

stroke(255);

// use the mix buffer to draw the waveforms.

for (int i = 0; i < kick.bufferSize() - 1; i++)

{

float x1 = map(i, 0, kick.bufferSize(), 0, width);

float x2 = map(i+1, 0, kick.bufferSize(), 0, width);

line(x1, 50 - kick.mix.get(i)*50, x2, 50 -

kick.mix.get(i+1)*50);

line(x1, 150 - snare.mix.get(i)*50, x2, 150 -

snare.mix.get(i+1)*50);

}

}

void keyPressed()

{

if ( key == 's' ) snare.trigger();

if ( key == 'k' ) kick.trigger();

}

I simplify the code.

The color parts are what we need.

You can run the example, and press the S or K key on your keyboard.

I hope you should know

How to do it!



import ddf.minim.*;

Minim minim;

AudioSample kick;

AudioSample snare;

void setup(){context = new SimpleOpenNI (this);



minim = new Minim(this);

kick = minim.loadSample( "BD.mp3", 512 );

// if a file doesn't exist, loadSample will return null

if ( kick == null ) println("Didn't get kick!");

// load SD.wav from the data folder

snare = minim.loadSample("SD.wav", 512);

if ( snare == null ) println("Didn't get snare!");



exit();

return;

}


}

For

*import libraries,

*declare

*void setup()

//R_Trigger

if(myRHand_Proj.x<3*width/4+25 && myRHand_Proj.x>3*width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25

|| myLHand_Proj.x<3*width/4+25 && myLHand_Proj.x>3*width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25)

snare.trigger();

){

noStroke();

fill(255,0,255);


stroke(255,0,255);

noFill();


}

//L_Trigger

if(myLHand_Proj.x<width/4+25 && myLHand_Proj.x>width/4-25 && myLHand_Proj.y<height/2+25 && myLHand_Proj.y>height/2-25

|| myRHand_Proj.x<width/4+25 && myRHand_Proj.x>width/4-25 && myRHand_Proj.y<height/2+25 && myRHand_Proj.y>height/2-25

){

kick.trigger();

noStroke();

fill(255,255,0);


stroke(255,255,0);

noFill();


}

All you need to do is put the trigger in the button

For

*void draw()

I didn’t put the whole code, it’s too much in one slide.

I suppose you have saved your file all the time

(I save it as “skeleton_Button_Sound”)

Last thing here you have to do:

Get the sound!!!

C:\Users\archgary\Desktop\processing-2.0.3-windows64\processing-

2.0.3\modes\java\libraries\minim\examples\TriggerASample

This is the path

***Everyone might have different path, but just try to find your minim library folder, and

inside the example, there should be the one.

Copy this “data” folder and put it in

your “SAVED” file folder

Ts…Don…

Now you are the drummer

Let’s play a bit pattern

Open the file

With 2 hands

First, get the girds



ArrayList grid;

import ddf.minim.*;

Minim minim;

AudioSample kick;

AudioSample snare;

void setup(){




grid = new ArrayList();



exit();

return;

}


for(int i =0; i<= width; i+=48){

for(int j=0; j<= height; j+=48){

PVector loc = new PVector(i,j,0);

grid.add(loc);

}

}

}

Making

ArrayList for grid


context.update();












for(int j=0; j<grid.size();j++){

PVector nLoc = (PVector)grid.get(j);

strokeWeight(1);

stroke(0,255,0);

noFill();

noFill();

ellipse(nLoc.x,nLoc.y,48,48);

}

strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);



stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);



}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);

stroke(0,255,255);



}

Get the grid point

Draw them

All the skeleton events.


context.update();












for(int j=0; j<grid.size();j++){

PVector nLoc = (PVector)grid.get(j);

strokeWeight(1);

stroke(0,255,0);

noFill();

PVector att = new PVector(myRHand_Proj.x,myRHand_Proj.y,0);

float distance = PVector.dist(nLoc,att);

float d = (map(distance,100,600,2,48));

float c = (map(distance,100,600,0,255));

fill(0,c,0);

ellipse(nLoc.x,nLoc.y,d,d);

fill(myRHand_Proj.x,myRHand_Proj.y,c);

ellipse(nLoc.x,nLoc.y,d-d/5,d-d/5);

}

strokeWeight(2);

stroke(0,255,255);



stroke(0,255,255);

strokeWeight(2);

fill(0,255,0);



stroke(0,255,255);

strokeWeight(2);

fill(255,0,255);



}


{


/*



println(jointPos);

*/
















}



{




}


{


}


{


}










fill(0,255,255);

stroke(0,255,255);



}

Calculate distances

Mapping radius

Draw the patterns

Make Right hand as a attraction point

Mapping colors

Make Your Own

Patterns

Documents

Kinect=002=skeleton tracking 2d