1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Lighting in WebGL
Computer Graphics with WebGL © Ed Angel, 2014
2
WebGL lighting
• Application must specify - Normals - Material properties - Lights
• State-based shading functions have been deprecated (glNormal, glMaterial, glLight)
- Replace by vertex attributes and sending uniforms to the shaders
• Compute in application or in shaders
Computer Graphics with WebGL © Ed Angel, 2014
3
Normalization
• Cosine terms in lighting calculations can be computed using dot product
• Unit length vectors simplify calculation
• Usually we want to set the magnitudes to have unit length but - Length can be affected by transformations - Rotation and translation preserve length (rigid body
transformations) - Scaling does not preserve length
• GLSL has a normalization function
Computer Graphics with WebGL © Ed Angel, 2014
2
4
Normal for Triangle
p0
p1
p2
n plane n ·(p - p0 ) = 0
n = (p2 - p0 ) ×(p1 - p0 )
normalize n ← n/ |n|
p
Note that right-hand rule determines outward face
Computer Graphics with WebGL © Ed Angel, 2014
Adding Normals for Quads
5
function quad(a, b, c, d) {
var t1 = subtract(vertices[b], vertices[a]); var t2 = subtract(vertices[c], vertices[b]);
var normal = cross(t1, t2); var normal = vec3(normal); normal = normalize(normal);
pointsArray.push(vertices[a]); normalsArray.push(normal); . . .
Computer Graphics with WebGL © Ed Angel, 2014
6
Specifying a Point Light Source
For each light source, we can set an RGBA for the diffuse, specular, and ambient components, and for the position
var diffuse0 = vec4(1.0, 0.0, 0.0, 1.0);var ambient0 = vec4(1.0, 0.0, 0.0, 1.0);var specular0 = vec4(1.0, 0.0, 0.0, 1.0);var light0_pos = vec4(1.0, 2.0, 3,0, 1.0);
Computer Graphics with WebGL © Ed Angel, 2014
3
7
Distance and Direction
• The source colors are specified in RGBA
• The position is given in homogeneous coordinates - If w =1.0, we are specifying a finite location - If w =0.0, we are specifying a parallel source with the given
direction vector
• The coefficients in distance terms are usually quadratic (1/(a+b*d+c*d*d)) where d is the distance from the point being
rendered to the light source
Computer Graphics with WebGL © Ed Angel, 2014
8
Moving Light Sources
• Light sources are geometric objects whose positions or directions are affected by the model-view matrix
• Depending on where we place the position (direction) setting function, we can
- Move the light source(s) with the object(s) - Fix the object(s) and move the light source(s) - Fix the light source(s) and move the object(s) - Move the light source(s) and object(s) independently
Computer Graphics with WebGL © Ed Angel, 2014
Light Properties
9
var lightPosition = vec4(1.0, 1.0, 1.0, 0.0 );var lightAmbient = vec4(0.2, 0.2, 0.2, 1.0 );var lightDiffuse = vec4( 1.0, 1.0, 1.0, 1.0 );var lightSpecular = vec4( 1.0, 1.0, 1.0, 1.0 );
Computer Graphics with WebGL © Ed Angel, 2014
4
10
Material Properties
• Material properties should match the terms in the light model
- Reflectivities - w component gives opacity
var materialAmbient = vec4( 1.0, 0.0, 1.0, 1.0 );var materialDiffuse = vec4( 1.0, 0.8, 0.0, 1.0);var materialSpecular = vec4( 1.0, 0.8, 0.0, 1.0 );var materialShininess = 100.0;
Computer Graphics with WebGL © Ed Angel, 2014
Using MV.js for Products
11
var ambientProduct = mult(lightAmbient, materialAmbient);var diffuseProduct = mult(lightDiffuse, materialDiffuse);var specularProduct = mult(lightSpecular, materialSpecular);
gl.uniform4fv(gl.getUniformLocation(program, "ambientProduct"), flatten(ambientProduct));gl.uniform4fv(gl.getUniformLocation(program, "diffuseProduct"), flatten(diffuseProduct));gl.uniform4fv(gl.getUniformLocation(program, "specularProduct"), flatten(specularProduct));gl.uniform4fv(gl.getUniformLocation(program, "lightPosition"), flatten(lightPosition));gl.uniform1f(gl.getUniformLocation(program, "shininess"),materialShininess);
Computer Graphics with WebGL © Ed Angel, 2014
12
Front and Back Faces
• Every face has a front and back • For many objects, we never see the back face so we don’t care how or if it’s rendered
• If it matters, we can handle in shader
back faces not visible back faces visible
Computer Graphics with WebGL © Ed Angel, 2014
5
13
Transparency
• Material properties are specified as RGBA values
• The A value can be used to make the surface translucent
• The default is that all surfaces are opaque • Later we will enable blending and use this feature
• However with the HTML5 canvas, A<1 will mute colors - Allows blending other elements onto HTML5 canvas
Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Polygonal Shading
Computer Graphics with WebGL © Ed Angel, 2014
2
Polygonal Shading
• In per vertex shading, shading calculations are done for each vertex
- Vertex colors become vertex shades and can be sent to the vertex shader as a vertex attribute
- Alternately, we can send the parameters to the vertex shader and have it compute the shade
• By default, vertex shades are interpolated across a triangle if passed to the fragment shader as a varying variable (smooth shading)
• We can also use uniform variables to shade each triangle with a single shade (flat shading)
Computer Graphics with WebGL © Ed Angel, 2014
3
Polygon Normals
• Triangles have a single normal - Shades at the vertices as computed by the modified
Phong model can be almost same - Identical for a distant viewer (default) or if there is
no specular component • Consider model of sphere • We want different normals at each vertex even though this concept is not quite correct mathematically
Computer Graphics with WebGL © Ed Angel, 2014
2
4
Smooth Shading
• We can set a new normal at each vertex
• Easy for sphere model - If centered at origin n = p
• Now smooth shading works • Note silhouette edge
Computer Graphics with WebGL © Ed Angel, 2014
5
Mesh Shading
• The previous example is not general because we knew the normal at each vertex analytically
• For polygonal models, Gouraud proposed we use the average of the normals around a mesh vertex
n = (n1+n2+n3+n4)/ |n1+n2+n3+n4|
Computer Graphics with WebGL © Ed Angel, 2014
6
Gouraud and Phong Shading
• Gouraud Shading - Find average normal at each vertex (vertex normals) - Apply modified Phong model at each vertex - Interpolate vertex shades across each polygon
• Phong shading - Find vertex normals - Interpolate vertex normals across edges - Interpolate edge normals across polygon - Apply modified Phong model at each fragment
Computer Graphics with WebGL © Ed Angel, 2014
3
7
Comparison
• If the polygon mesh approximates surfaces with a high curvatures, Phong shading may look smooth while Gouraud shading may show edges
• Phong shading requires much more work than Gouraud shading - Until recently not available in real time systems - Now can be done using fragment shaders
• Both need data structures to represent meshes so we can obtain vertex normals
• Implementation of Gourand is per vertex shading • Implementation of Phong is per fragment shading
Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Per Fragment vs Per Vertex Shading
Computer Graphics with WebGL © Ed Angel, 2014
Vertex Lighting Shaders I
2
// vertex shader
attribute vec4 vPosition;attribute vec4 vNormal;varying vec4 fColor;uniform vec4 ambientProduct, diffuseProduct, specularProduct;uniform mat4 modelViewMatrix;uniform mat4 projectionMatrix;uniform vec4 lightPosition;uniform float shininess;
void main(){
Computer Graphics with WebGL © Ed Angel, 2014
Vertex Lighting Shaders II
3
vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz; vec3 L = normalize( light - pos ); vec3 E = normalize( -pos ); vec3 H = normalize( L + E );
// Transform vertex normal into eye coordinates
vec3 N = normalize( (modelViewMatrix*vNormal).xyz);
// Compute terms in the illumination equation
Computer Graphics with WebGL © Ed Angel, 2014
2
Vertex Lighting Shaders III
4
// Compute terms in the illumination equation
vec4 ambient = AmbientProduct;
float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*DiffuseProduct; float Ks = pow( max(dot(N, H), 0.0), Shininess ); vec4 specular = Ks * SpecularProduct; if( dot(L, N) < 0.0 ) specular = vec4(0.0, 0.0, 0.0, 1.0); gl_Position = Projection * ModelView * vPosition;
color = ambient + diffuse + specular; color.a = 1.0;}
Computer Graphics with WebGL © Ed Angel, 2014
Vertex Lighting Shaders IV
5
// fragment shader
precision mediump float;
varying vec4 fColor;
voidmain(){ gl_FragColor = fColor;}
Computer Graphics with WebGL © Ed Angel, 2014
Fragment Lighting Shaders I
6
// vertex shader
attribute vec4 vPosition;attribute vec4 vNormal;varying vec3 N, L, E;uniform mat4 modelViewMatrix;uniform mat4 projectionMatrix;uniform vec4 lightPosition;
Computer Graphics with WebGL © Ed Angel, 2014
3
Fragment Lighting Shaders II
7
void main(){ vec3 pos = -(modelViewMatrix * vPosition).xyz; vec3 light = lightPosition.xyz;
L = normalize( light - pos ); E = -pos; N = normalize( (modelViewMatrix*vNormal).xyz); gl_Position = projectionMatrix * modelViewMatrix * vPosition;};
Computer Graphics with WebGL © Ed Angel, 2014
Fragment Lighting Shaders III
8
// fragment shader
precision mediump float;
uniform vec4 ambientProduct;uniform vec4 diffuseProduct;uniform vec4 specularProduct;uniform float shininess;varying vec3 N, L, E;
void main(){
Computer Graphics with WebGL © Ed Angel, 2014
Fragment Lighting Shaders IV
9
vec4 fColor; vec3 H = normalize( L + E ); vec4 ambient = ambientProduct; float Kd = max( dot(L, N), 0.0 ); vec4 diffuse = Kd*diffuseProduct; float Ks = pow( max(dot(N, H), 0.0), shininess ); vec4 specular = Ks * specularProduct; if( dot(L, N) < 0.0 ) specular = vec4(0.0, 0.0, 0.0, 1.0); fColor = ambient + diffuse +specular; fColor.a = 1.0; gl_FragColor = fColor;}
Computer Graphics with WebGL © Ed Angel, 2014
4
Teapot Examples
10Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Buffers
Computer Graphics with WebGL © Ed Angel, 2014
2
Buffer
Define a buffer by its spatial resolution (n x m) and its depth (or precision) k, the number of bits/pixel
pixel
Computer Graphics with WebGL © Ed Angel, 2014
3
WebGL Frame Buffer
Computer Graphics with WebGL © Ed Angel, 2014
2
4
Where are the Buffers?
• HTML5 Canvas - Default front and back color buffers - Under control of local window system - Physically on graphics card
• Depth buffer also on graphics card • Stencil buffer
- Holds masks • Most RGBA buffers 8 bits per component • Latest are floating point (IEEE)
Computer Graphics with WebGL © Ed Angel, 2014
Other Buffers
• desktop OpenGL supported other buffers - auxiliary color buffers - accumulation buffer - these were on application side - now deprecated
• GPUs have their own or attached memory - texture buffers - off-screen buffers
• not under control of window system • may be floating point
5Computer Graphics with WebGL © Ed Angel, 2014
Images
• Framebuffer contents are unformatted - usually RGB or RGBA - one byte per component - no compression
• Standard Web Image Formats - jpeg, gif, png
• WebGL has no conversion functions - Understands standard Web formats for texture
images
6Computer Graphics with WebGL © Ed Angel, 2014
3
7
The (Old) Pixel Pipeline
• OpenGL has a separate pipeline for pixels - Writing pixels involves
• Moving pixels from processor memory to the frame buffer
• Format conversions • Mapping, Lookups, Tests
- Reading pixels • Format conversion
Computer Graphics with WebGL © Ed Angel, 2014
Packing and Unpacking
• Compressed or uncompressed • Indexed or RGB • Bit Format
- little or big endian • WebGL (and shader-based OpenGL) lacks most
functions for packing and unpacking - use texture functions instead - can implement desired functionality in fragment
shaders
8Computer Graphics with WebGL © Ed Angel, 2014
Deprecated Functionality
• glDrawPixels • glCopyPixels • glBitMap
9Computer Graphics with WebGL © Ed Angel, 2014
4
10
Buffer Reading
• WebGL can read pixels from the framebuffer with gl.readPixels
• Returns only 8 bit RGBA values • In general, the format of pixels in the frame buffer is different from that of processor memory and these two types of memory reside in different places
- Need packing and unpacking - Reading can be slow
• Drawing through texture functions and off-screen memory (frame buffer objects)
Computer Graphics with WebGL © Ed Angel, 2014
11
WebGL Pixel Function
gl.readPixels(x,y,width,height,format,type,myimage)
start pixel in frame buffer size type of image
type of pixels pointer to processor memory
var myimage[512*512*4];
gl.readPixels(0,0, 512, 512, gl.RGBA, gl.UNSIGNED_BYTE, myimage);
Computer Graphics with WebGL © Ed Angel, 2014
Render to Texture
• GPUs now include a large amount of texture memory that we can write into
• Advantage: fast (not under control of window system) • Using frame buffer objects (FBOs) we can render into
texture memory instead of the frame buffer and then read from this memory
- Image processing - GPGPU
12Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Texture Mapping
Computer Graphics with WebGL © Ed Angel, 2014
2
The Limits of Geometric Modeling
Although graphics cards can render over 10 million polygons per second, that number is insufficient for many phenomena
- Clouds - Grass - Terrain - Skin
Computer Graphics with WebGL © Ed Angel, 2014
3
Modeling an Orange
• Consider the problem of modeling an orange (the fruit) • Start with an orange-colored sphere
- Too simple • Replace sphere with a more complex shape
- Does not capture surface characteristics (small dimples)
- Takes too many polygons to model all the dimples
Computer Graphics with WebGL © Ed Angel, 2014
2
4
Modeling an Orange (2)
• Take a picture of a real orange, scan it, and “paste” onto simple geometric model
- This process is known as texture mapping • Still might not be sufficient because resulting surface
will be smooth - Need to change local shape - Bump mapping
Computer Graphics with WebGL © Ed Angel, 2014
5
Three Types of Mapping
• Texture Mapping - Uses images to fill inside of polygons
• Environment (reflection mapping) - Uses a picture of the environment for texture maps - Allows simulation of highly specular surfaces
• Bump mapping - Emulates altering normal vectors during the
rendering process
Computer Graphics with WebGL © Ed Angel, 2014
6
Texture Mapping
geometric model texture mapped
Computer Graphics with WebGL © Ed Angel, 2014
3
7
Environment Mapping
Computer Graphics with WebGL © Ed Angel, 2014
8
Bump Mapping
Computer Graphics with WebGL © Ed Angel, 2014
9
Where does mapping take place?
• Mapping techniques are implemented at the end of the rendering pipeline
- Very efficient because few polygons make it past the clipper
Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
Texture Mapping 2
Computer Graphics with WebGL © Ed Angel, 2014
2
Is it simple?
Although the idea is simple---map an image to a surface---there are 3 or 4 coordinate systems involved
2D image
3D surface
Computer Graphics with WebGL © Ed Angel, 2014
3
Coordinate Systems
• Parametric coordinates - May be used to model curves and surfaces
• Texture coordinates - Used to identify points in the image to be mapped
• Object or World Coordinates - Conceptually, where the mapping takes place
• Window Coordinates - Where the final image is really produced
Computer Graphics with WebGL © Ed Angel, 2014
2
4
Texture Mapping
parametric coordinates
texture coordinates world coordinates window coordinates
Computer Graphics with WebGL © Ed Angel, 2014
5
Mapping Functions
• Basic problem is how to find the maps • Consider mapping from texture coordinates to a point
a surface • Appear to need three functions
x = x(s,t) y = y(s,t) z = z(s,t)
• But we really want to go the other way s
t
(x,y,z)
Computer Graphics with WebGL © Ed Angel, 2014
6
Backward Mapping
• We really want to go backwards - Given a pixel, we want to know to which point on
an object it corresponds - Given a point on an object, we want to know to
which point in the texture it corresponds • Need a map of the form
s = s(x,y,z) t = t(x,y,z)
• Such functions are difficult to find in general
Computer Graphics with WebGL © Ed Angel, 2014
3
7
Two-part mapping
• One solution to the mapping problem is to first map the texture to a simple intermediate surface
• Example: map to cylinder
Computer Graphics with WebGL © Ed Angel, 2014
8
Cylindrical Mapping
parametric cylinder
x = r cos 2π u y = r sin 2πu z = v/h
maps rectangle in u,v space to cylinder of radius r and height h in world coordinates
s = u t = v
maps from texture space
Computer Graphics with WebGL © Ed Angel, 2014
9
Spherical Map
We can use a parametric sphere
x = r cos 2πu y = r sin 2πu cos 2πv z = r sin 2πu sin 2πv
in a similar manner to the cylinder but have to decide where to put the distortion
Spheres are used in environmental maps
Computer Graphics with WebGL © Ed Angel, 2014
4
10
Box Mapping
• Easy to use with simple orthographic projection • Also used in environment maps
Computer Graphics with WebGL © Ed Angel, 2014
11
Second Mapping
• Map from intermediate object to actual object - Normals from intermediate to actual - Normals from actual to intermediate - Vectors from center of intermediate
intermediate actual
Computer Graphics with WebGL © Ed Angel, 2014
12
Aliasing
• Point sampling of the texture can lead to aliasing errors
point samples in u,v (or x,y,z) space
point samples in texture space
miss blue stripes
Computer Graphics with WebGL © Ed Angel, 2014
5
13
Area Averaging
A better but slower option is to use area averaging
Note that preimage of pixel is curved
pixel preimage
Computer Graphics with WebGL © Ed Angel, 2014
1
1
Introduction to Computer Graphics with WebGL
Ed Angel
WebGL Texture Mapping
Computer Graphics with WebGL © Ed Angel, 2014
2
Basic Stragegy
Three steps to applying a texture 1. Create a texture object
• specify texture parameters • wrapping, filtering
2. Specify the texture • read or generate image • assign to texture object
3. Assign texture coordinates to vertices • Proper mapping function is left to application
Computer Graphics with WebGL © Ed Angel, 2014
3
Texture Mapping
s
t
x
y
z
image
geometry display
Computer Graphics with WebGL © Ed Angel, 2014
2
4
• Specify a texture image from an array of texels (texture elements) in CPU memory
• Use an image in a standard format such as JPEG - Scanned image - Generate by application code
• WebGL supports only 2 dimensional texture maps - no need to enable as in desktop OpenGL - desktop OpenGL supports 1-4 dimensional texture maps
Specifying a Texture Image
Computer Graphics with WebGL © Ed Angel, 2014
5
Define Image as a Texture
gl.texImage2D( target, level, components, w, h, border, format, type, texels );
target: type of texture (gl.TEXTURE_2D) level: used for mipmapping (discussed later) components: elements per texel w, h: width and height of texels in pixels border: used for smoothing (no longer used) format and type: describe texels texels: pointer to texel array
gl.texImage2D(gl.TEXTURE_2D, 0, 3, 512, 512, 0, gl.RGB, gl.UNSIGNED_BYTE, myTexels);
Computer Graphics with WebGL © Ed Angel, 2014
A Checkerboard Image
6
var image1 = new Uint8Array(4*texSize*texSize);
for ( var i = 0; i < texSize; i++ ) { for ( var j = 0; j <texSize; j++ ) { var patchx = Math.floor(i/(texSize/numChecks)); var patchy = Math.floor(j/(texSize/numChecks)); if(patchx%2 ^ patchy%2) c = 255; else c = 0;
image1[4*i*texSize+4*j] = c; image1[4*i*texSize+4*j+1] = c; image1[4*i*texSize+4*j+2] = c; image1[4*i*texSize+4*j+3] = 255; }}
Computer Graphics with WebGL © Ed Angel, 2014
3
Using a GIF image
7
// specify image in JS file
var image = new Image(); image.onload = function() { configureTexture( image ); } image.src = "SA2011_black.gif”
// or specify image in HTML file with <img> tag
// <img id = "texImage" src = "SA2011_black.gif">// </img>
var image = document.getElementById("texImage”) window.onload = configureTexture( image );Computer Graphics with WebGL © Ed Angel, 2014
8
• Based on parametric texture coordinates • Specify as a 2D vertex attribute
s
t 1, 1 0, 1
0, 0 1, 0
(s, t) = (0.2, 0.8)
(0.4, 0.2)
(0.8, 0.4)
A
B C
a
b c
Texture Space Object Space
Mapping a Texture
Computer Graphics with WebGL © Ed Angel, 2014
9
Cube Example var texCoord = [ vec2(0, 0), vec2(0, 1), vec2(1, 1), vec2(1, 0)];
function quad(a, b, c, d) { pointsArray.push(vertices[a]); colorsArray.push(vertexColors[a]); texCoordsArray.push(texCoord[0]);
pointsArray.push(vertices[b]); colorsArray.push(vertexColors[a]); texCoordsArray.push(texCoord[1]);// etc
Computer Graphics with WebGL © Ed Angel, 2014
4
10
Interpolation
WebGL uses interpolation to find proper texels from specified texture coordinates
Can be distortions
good selection of tex coordinates
poor selection of tex coordinates
texture stretched over trapezoid showing effects of bilinear interpolation
Computer Graphics with WebGL © Ed Angel, 2014