CoreMotion and the Gyroscope.pdf

8/11/2019 CoreMotion and the Gyroscope.pdf

1/46

CoreMotion and the Gyroscope(from Jenga to Augmented Reality)

VTM: iPhone Developers Conference. October 17, 2010.

Jeffrey Powers, Co-Founder, Occipital @jrpowers

Plus: How to build your own

spacetime portal.


2/46

Who am I? Jeff Powers

University of Michigan Bachelors

in EE:Signals, Masters inCS:Artificial Intelligence

Co-founder, Occipital

(computer vision / AR)

@jrpowers


3/46


4/46

" Today: 360 Panorama

" Realtime panorama capture

"

Computer vision based.

This is actually the pano I took during the talk.

452


5/46

What well cover." Demos: What can CoreMotion do?

" The Hardware

" The chips.

"

How accurate are they?

" Coding with CoreMotion.

" The APIs.

" Acclerometer

" Gyro

" Getting started.

" Pitfalls you probably arent aware of.

" Performance analysis and tips.

" Building your own Spacetime Portal

" Coremotion, AVFoundation, OpenGL ES 2.0


6/46


7/46

67$+.8+9% &%:+ *$+: -;,+/% "?@'&* A=>9.B/02C =>9.DE/02F

G7D3/H%*'/B I/0*'30%(:.2;*'-5J

K$ $ */-'L*%-D< M/-' %* NO/&O *O' /*'- /L H%(/B6 $ $K

E8*"8)*#7F*)':"%/7G %"%'#"!$0H IJ@$!);%#)*5'/ #$!)3#'!82

7'0B


26/46

PITFALL

! CMLogItems may come back nil!

! accelData, gyroData, deviceMotion

if([motionManagerisAccelerometerActive]) {

CMAccelerometerData* aData = [motionManageraccelerometerData];

// do something with acceleration aData.acceleration// NOT SAFE!!

}

CMAccelerometerData* aData = [motionManageraccelerometerData];

if(aData != NULL){// do something with acceleration aData.acceleration// SAFE

}


27/46

Accessing the Accelerometer with

CoreMotionRecap

1. Create one (and only one) CMMotionManager

2.

Check sensor availability

3. Set update interval

4. startAccelerometerUpdates

5.

Poll x,y,z values periodically (or get them pushed toyou)


28/46

CoreMotion Axis Conventions

From iOS SDK Documentation

Accelerometer reports acceleration mainly due to gravity, but also

due to user motion.


29/46

CMRotationMatrixrotationMatrixFromGravity(floatx, floaty, floatz){// The Z axis of our rotated frame is opposite gravityvec3f_tzAxis =

vec3f_normalize(vec3f_init(-x, -y, -z));

// The Y axis of our rotated frame is an arbitrary vector perpendicular to gravity // Note that this convention will have problems as zAxis.x approaches +/-1 sincethe magnitude of

// [0, zAxis.z, -zAxis.y] will approach 0 //vec3f_t yAxis = vec3f_normalize(vec3f_init(0, -zAxis.z, zAxis.y));

// For Augmented Reality (AR), we want a different convention. Specifically// we would prefer to ambiguity to occur when zAxis.z approaches +/-1

vec3f_tyAxis = vec3f_normalize(vec3f_init(zAxis.y, -zAxis.x, 0));

// The X axis is just the cross product of Y and Zvec3f_txAxis = vec3f_crossProduct(yAxis, zAxis);

CMRotationMatrixmat = { xAxis.x, yAxis.x, zAxis.x,xAxis.y, yAxis.y, zAxis.y,

xAxis.z, yAxis.z, zAxis.z };

returnmat;

}

From CMTeapot, tweaked for AR scenarios.


30/46

The APIs"

CMMotionManager The central access point for motionsensing.

"

CMAccelerometerData Raw accelerometer readings.

" CMGyroData Raw gyro readings.

" CMDeviceMotion Fusion of all sensors.

" .attitude (CMAttitude)

"

.rotationRate (CMRotationRate)

" .gravity (CMAcceleration)

" .userAcceleration (CMAcceleration)


31/46

Accessing the Gyroscope with CoreMotion

[Same basic setup as Accelerometer]1. [Same] Create one (and only one) CMMotionManager

2. [Same, but Gyro] Check sensor availability

3.

[Same, but Gyro] Set update interval

4. startGyroUpdates

5. Poll gyro values periodically (or get them pushed to

you)


32/46


CMGyroDataTypo is actually from the iOS headers!

/* CMAccelerometerData* Contains a single accelerometermeasurement. */

@interfaceCMGyroData : CMLogItem{ @privateid_internal; }

/* * rotationRate: The rotation rate asmeasured by the gyro. */@property(readonly,nonatomic) CMRotationRaterotationRate;

@end


33/46


CMRotationRate/* * CMRotationRate * * Discussion: * A structure containing3-axis rotation rate data. * * Fields: * x: * X-axisrotation rate in radians/second. The sign follows the right hand *rule (i.e. if the right hand is wrapped around the X axis such thatthe * tip of the thumb points toward positive X, a positiverotation is one * toward the tips of the other 4 fingers). *

y: * Y-axis rotation rate in radians/second. The sign followsthe right hand * rule (i.e. if the right hand is wrappedaround the Y axis such that the * tip of the thumb pointstoward positive Y, a positive rotation is one * toward thetips of the other 4 fingers). *z: *Z-axis rotation rate in radians/second. The sign follows the right hand * rule (i.e. if theright hand is wrapped around the Z axis such that the * tipof the thumb points toward positive Z, a positive rotation is one* toward the tips of the other 4 fingers). */

typedefstruct{doublex;doubley;doublez;} CMRotationRate;

Radians per second around each axis.


34/46

CoreMotion Axis Conventions

G+))

;-"8,W#(

#""-"=&% X JW#( Y ;-"8, Y G+))NK


35/46

The APIs"

CMMotionManager The central access point for motionsensing.

"

CMAccelerometerData Raw accelerometer readings.

" CMGyroData Raw gyro readings.

" CMDeviceMotion Fusion of all sensors.

" .attitude (CMAttitude)

"

.rotationRate (CMRotationRate)

" .gravity (CMAcceleration)

" .userAcceleration (CMAcceleration)


36/46

Accessing DeviceMotion with CoreMotion

[Setup is just like Accel/Gyro]1. [Same] Create one (and only one) CMMotionManager

2. [Same, but deviceMotion] Check availability

3.

[Same, but deviceMotion] Set update interval

4. startDeviceMotionUpdates

5. Poll deviceMotion values periodically (or get them

pushed to you)


37/46

/* * CMDeviceMotion: A CMDeviceMotion object contains basic informationabout the device's *motion. */ @interfaceCMDeviceMotion : CMLogItem{ @privateid_internal; }

/* attitude */

@property(readonly, nonatomic) CMAttitude *attitude;

/* rotationRate */

@property(readonly, nonatomic) CMRotationRate rotationRate;

/* gravity: Returns the gravity vector expressed in the device'sreference frame. Note *that the total acceleration of the device isequal to gravity plus *userAcceleration. */

@property(readonly, nonatomic) CMAcceleration gravity;

/* userAcceleration: Returns the acceleration that the user is givingto the device. */

@property(readonly, nonatomic) CMAcceleration userAcceleration;

@end


CMDeviceMotion


38/46

@interfaceCMAttitude : NSObject { @privateid_internal; }

@property(readonly, nonatomic) doubleroll;@property(readonly, nonatomic) doublepitch;@property(readonly, nonatomic) doubleyaw;

/* rotationMatrix */

@property(readonly, nonatomic) CMRotationMatrix rotationMatrix;

/* quaternion */@property(readonly, nonatomic) CMQuaternion quaternion);

/* multiplyByInverseOfAttitude: * Multiplies attitude by the

inverse of the specified attitude. This gives *the attitudechange from the specified attitude. */

- (void)multiplyByInverseOfAttitude:(CMAttitude*)attitude;

@end


CMAttitude


39/46

Attitude" attitude = (yaw * pitch * roll)T

motionOri =matrix3d::matrix3d_from_z(yaw) *

matrix3d::matrix3d_from_x(pitch) *matrix3d::matrix3d_from_y(roll);

returnmotionOri.inverse();

" Attitude is the rotation from the devices current positionback to its original position. This matrix can be used as-is tokeep a 3D object stationary.

" Rotation matrices above are just rotations about the specifiedaxes. See http://en.wikipedia.org/wiki/Rotation_matrixfortheir definitions.


40/46

Z.%*=) "++) (,%/ )%#$/-/A &%0-8%I+"-+/@

67$+.8+9% #99 J*$%%N *$+: #8$+..#-$

"+ .%% 7#([9-"8,[$+)) #/A)%.


41/46

So, shouldnt I just ALWAYS use

deviceMotion?! CoreMotion isnt free to run.

! Even if you set a low update rate, it will use a good

amount of CPU.


42/46

Performance Considerations

!"#$% '() )*$+,

-.. /0 1. /0

!"#$% 233 !"#$% 233!"#$%"&'(') +,- ./- +,- 0/-

1%%"2"3'4"5"3 ,/- 0,- 6+- ,-

1%%"27893' ,0- 0/- ,/- ,-

0%1 2*3&4

D%0-8%I+"-+/ %#". U\] !"#$"+"#) P;Z #$%'($))+* =9"% $#"%>

.""1 2*3&4H"^. /+" C%8#=.% +* ",% 67$+ -".%)*> G%#&-/A ",% A7$+ -/ #&&-"-+/ "+

",% #88%)%$+:%"%$ &+%. /+" .)+( #/7",-/A &+(/ /+"-8%#C)7>


43/46

Building your own spacetimeportalBonus: No particle physics PhD required


44/46

5",-%+

_#)0% !+*"(#$%

6"7,)*4 5,"#" 3%++$%$*, 8", 5",-%+9


45/46

What it takes to build real-world

Portal with CoreMotion! OpenGL Sample project, plus:

! CoreMotion framework (attitude)

! CoreGraphics framework reference (Images)

!

AVFoundation, CoreVideo framework reference (Camera)

! 360-degree images (for display in AR)

! Matrix and Vector classes from Teapot Sample

! AVFoundation captureOutput session feeding to OpenGL(Ben Newhouse briefly covered this yesterday)

!

Modified OpenGL ES 2.0 shaders to draw the hole


46/46

!9#8%"-:% ;+$"#) D%:+

Documents

CoreMotion and the Gyroscope.pdf