Embed Size (px)
Citation preview
A talk in three acts ...
• Intro to Computer Graphics
• Computer Graphics in OpenGL
• OpenGL and CocoaTouch Playing Nice Together
Intro to Computer Graphics
(a trip down the graphics pipeline)
Build and Tesselate Model
Copyright Steve Jubinville
Pose Model
Copyright Steve Jubinville
Aim Camera
Design Appearance. Light Scene.
Copyright © 2006 Holger Schömann
Copyright © 2006 Florian Wild
Design Appearance. Light Scene.
Copyright © 2006 Jack Qiao
Design Appearance. Light Scene.
Render Scene
Cull & clip model triangles to viewing frustrum.
From models to pixels. Scanline interpolation.
• Depth Comparison
• Apply Appearance - Vertex level. Pixel level.
• Composite Elements
Rasterize
Present Image
Update Model State
Rinse, repeat.
Computer Graphics in OpenGL
OpenGL Cocoa
OpenGL is old school
You are flipping levers on a state machine.
float foo[]
Data Structures are for wimps.
glPushMatrix() and glPopMatrix()
OpenGL is a dragster. Not a Prius. Just buckle up ...
... and enjoy the ride, baby!
iPhone 3GS by the Numbers(via AnandTech http://bit.ly/1pQaJO)
~7M triangles / second
~230,000 realtime triangles (30fps)
iPhone display: 320 * 480 = 153,600 pixels
230,000 / 153,600 = ~1.5 realtime-triangles / pixel
;-)
OpenGL websites are a bit differentthan Cocoa websites ...
Isn’t that sweet ...
Um... WTF?
Can’t you just feel the love?
Dude. I think my eyes are bleeding.
No worries. safari.oreilly.com is your friend
Pose Model
Hierarchical Modeling. Transformation Stacks. Coordinate Frames.
Pose Model
You are already familiar with hierarchical modeling and coordinate frames.
[self.parentView addSubview:childView];
for (UIView *v in self.containerView.subviews) { CGRect bounds = [v convertRect:v.bounds toView:scrollView]; if (CGRectIntersectsRect(bounds, scrollView.bounds)) { NSLog(@"View %d is visible", v.tag); } else { NSLog(@"View %d is hidden", v.tag); } } // for (self.containerView.subviews)
or ...
You are already familiar with hierarchical modeling and coordinate frames.
You are already already familiar with transformations.
- (void)setTransform:(CGAffineTransform)newValue { // Scale along x-axis only CGAffineTransform constrainedTransform = CGAffineTransformScale(CGAffineTransformIdentity, newValue.a, 1.0);
newValue.a = constrainedTransform.a; newValue.b = constrainedTransform.b; newValue.c = constrainedTransform.c; newValue.d = constrainedTransform.d; [super setTransform:newValue]; }
HelloTeapot Demo
Source: http://github.com/turner/HelloTeapot
HelloTeapot Code Walkthrough
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
Pose Model
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
Light Scene
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
Aim Camera ...
Dude, where is the camera ...?
camera teapot
world
wTtwTc
camera teapot
glMatrixMode(GL_MODELVIEW);glLoadMatrixf(_openGLCameraInverseTransform);
?
cTt
camera teapot
world
wTt(wTc)
cTt
cTt wTtcTw *=
-1
cTt wTtcTw *=
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += new_teapot_indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
cTt wTtcTw *=
- (void)placeCameraAtLocation:(M3DVector3f)location target:(M3DVector3f)target up:(M3DVector3f)up;
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
float teapot_vertices [] = { 0.0663, 0.1178, 0.0, 0.0672, 0.1152, 0.0, 0.0639, 0.1178, 0.0178043,
... };
float teapot_normals[] = { -0.987635, -0.156768, 0, -0.902861, -0.429933, 0, -0.953562, -0.156989, -0.257047,
... };
short indicies[] = {// howmany vertices in vertex strip26,// vertex strip indices1122, 1243, 1272, 1242, ... ,1283, 1199, ... };
http://github.com/turner/HelloTeapot/blob/master/teapot.h
Render
glEnableClientState(GL_VERTEX_ARRAY);glVertexPointer(3 ,GL_FLOAT, 0, teapot_vertices);
glEnableClientState(GL_NORMAL_ARRAY);glNormalPointer(GL_FLOAT, 0, teapot_normals);glEnable(GL_NORMALIZE);
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
Render
// Position Camera glMatrixMode(GL_MODELVIEW); glLoadIdentity(); glLoadMatrixf(_openGLCameraInverseTransform);
// Position Light glPushMatrix(); glMultMatrixf(_cameraTransform); glEnable(GL_LIGHT3); glLightfv(GL_LIGHT3, GL_DIFFUSE, spotLight);
glPopMatrix();
// Position Model glPushMatrix();
JLMMatrix3DSetZRotationUsingDegrees(rotation, -45.0f); JLMMatrix3DSetScaling(scale, sx, sy, sz); JLMMatrix3DMultiply(rotation, scale, concatenation); glMultMatrixf(concatenation);
// Render for(int i = 0; i < num_teapot_indices; i += indicies[i] + 1) {
glDrawElements(GL_TRIANGLE_STRIP, indices[i], GL_UNSIGNED_SHORT, &indices[i+1]);
} // for (num_teapot_indices)
glPopMatrix();
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
Render
http://github.com/turner/HelloTeapot/blob/master/Classes/GLViewController.[hm]
red_lightgreen_light
blue_light
push()
pop()
invert( camera_transform )
camera_transformwhite_spotlight
push()
pop()
teapot_transformdraw()
You’ll want to understand this stuff if you intend to do AR* apps like all the cool kids.
*Augmented Reality - See http://layar.com
Design Appearance
Mipmapped textures are your friend.
They will become your workhorse for visual complexity.
Texture Everything
Copyright © Douglass Turner
glGenTextures(1, &_name); glBindTexture(GL_TEXTURE_2D, _name); glTexEnvf(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glTexImage2D(GL_TEXTURE_2D, 0, glColor, _w, _h, 0, glColor, glFormat, data); glGenerateMipmapOES(GL_TEXTURE_2D);
MethodinitWithTextureFile:mipmap:
Classgithub.com/turner/HelloTexture/blob/master/Classes/TEITexture.[h,m]
HelloTexture Demo
Source: http://github.com/turner/HelloTexture
Two Words
Texture. Atlas.
s
t
Texture Space
s
t
Texture Space
HelloParticleSystem Demo
texture atlas hacks
OpenGL ES 2.0 == GLSL
Shaders Take it to Another Level Copyright © Douglass Turner
Shaders Take it to Another Level Copyright © Douglass Turner
Shaders Take it to Another Level ZBrush Modeling Daniel Lieske
Shaders Take it to Another LevelZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf
Shaders Take it to Another LevelZBrush Modeling Daniel LieskeMaterial Design Ralf Stumpf
Shaders Take it to Another Level Copyright: Stephen Molyneaux
GLSL is cool.
Sadly, OpenGL ES 2.0 (iPhone 3Gs) does not support hardware anti-aliasing.
Stick with mipmapped textures until you really know what you are doing.
Or don’t care about aliasing artifacts ...
1 sample per pixel 4 jittered/filtered samples per pixel
Remember the iPhone Imperitive
Your app will be beautiful.
OpenGL and Cocoa Touch Playing Nice Together
MVC
OpenGL is M. Keep V dumb. Lots of chatter between M and V.
• GLGravity | http://bit.ly/2dS8YI• Jeff LaMarche OpenGL ES Template | http://bit.ly/2IvZtV
Touch Sequence & Accelerometer
HelloParticleSystem Demo
Particle system. Touch. Acceleration
Touch Sequence
touchesBegan:withEvent:
touchesMoved:withEvent:
touchesEnded:withEvent:
touchesCancelled:withEvent:
Initialize model state
Evolve model state
Clean up model state
Accelerometer
- (void)accelerometer:(UIAccelerometer*)accelerometer didAccelerate:(UIAcceleration*)acceleration { // Compute "G". Use low-pass filter to attenuate instantaneous acceleration. self.accelerationValueX = acceleration.x * kFilteringFactor + self.accelerationValueX * (1.0 - kFilteringFactor); self.accelerationValueY = acceleration.y * kFilteringFactor + self.accelerationValueY * (1.0 - kFilteringFactor); self.accelerationValueZ = acceleration.z * kFilteringFactor + self.accelerationValueZ * (1.0 - kFilteringFactor); // ParticleSystem particles live in flatland. Use x and y compoments of "G". [ParticleSystem setGravity:CGPointMake(self.accelerationValueX, self.accelerationValueY)];}
UIAccelerometerDelegate
+ (void)setGravity:(CGPoint)gravityVector { // Normalize 2D “G” vector. Only care about direction. float length = sqrtf(gravityVector.x * gravityVector.x + gravityVector.y * gravityVector.y); ParticleSystemGravity.x = gravityVector.x / length; ParticleSystemGravity.y = gravityVector.y / length; // Flip y-component of gravity vector to be consistent with screen space coordinate // system used in ParticleSystem. ParticleSystemGravity.y = -(ParticleSystemGravity.y);}
ParticleSystem - Class State
// dv = dv/dt * dt float dv_x = (ParticleSystemGravity.x * gravityScaleFactor * dt); float dv_y = (ParticleSystemGravity.y * gravityScaleFactor * dt); particle.velocity = CGPointMake(particle.velocity.x + dv_x, particle.velocity.y + dv_y); // dx = dx/dt * dt float dx = particle.velocity.x * dt; float dy = particle.velocity.y * dt; // x = x_old * dx particle.location = CGPointMake(particle.location.x + dx, particle.location.y + dy);
ParticleSystem - Instance State
CoreAnimation:
[myView setNeedsDisplay] & [myView drawRect]
OpenGL:
- (void)startAnimation { animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES];}
iPhone is an Expression Platform
You Must See:Stanford cs 193p Lecture 17 Video | On iTunes: http://bit.ly/PUrGjCreating New Expressive Social Mediums on the iPhone Ge Wang | CCO & CTO | Smule
• Touch• Sound• Proximity• Cloud• Camera• INU (almost)• Geography
iPhone is a general purpose sensing platform
iPhone encourages plausible, hyper-real, extensions of human experience.
Be interesting. Be brilliant.
Resources
• @dugla - search on #iphonemeetup3dtalkacton
• http://theelasticimage.posterous.com
• This presentation on SlideShare: http://bit.ly/4wC4Mu
