Upload
vincent-allen-sims
View
235
Download
1
Tags:
Embed Size (px)
Citation preview
CS361Week 15 - Monday
Last time
What did we talk about last time? Future of graphics
Hardware developments Game evolution Current research
Questions?
Project 4
Student Lecture: Overview of Material up to Exam 1
Week 1: Color and SharpDX
RGB
We will primarily focus on the RGB system for representing color
With Red, Green, and Blue components, you can combine them to make most (but not all) visible colors
Combining colors is an additive process: With no colors, the background is black Adding colors never makes a darker color Pure Red added to pure Green added to
pure Blue makes White RGB is a good model for computer
screens
Luminance
If the R, G, B values happen to be the same, the color is a shade of gray 255, 255, 255 = White 128, 128, 128 = Gray 0, 0, 0 = Black
To convert a color to a shade of gray, use the following formula: Value = .3R + .59G + .11B
Based on the way the human eye perceives colors as light intensities
Brightness and Contrast
We can adjust the brightness of a picture by multiplying each pixel's R,G, and B value by a scalar b b [0,1) darkens b (1,) brightens
We can adjust the contrast of a picture by multiplying each pixel's R,G, and B value by a scalar c and then adding -128c + 128 to the value c [0,1) decreases contrast c (1,) increases contrast
After adjustments, values must be clamped to the range [0, 255] (or whatever the range is)
HSV
HSV Hue (which color) Saturation (how colorful
the color is) Value (how bright the
color is) Hue is represented as
an angle between 0° and 360°
Saturation and value are often given between 0 and 1
Saturation in HSV is not the same as in HSL
SharpDX basics
LoadContent() methodUpdate() methodDraw() methodTexture2D objects Sprites
Week 2: The GPU
Rendering
What do we have? Virtual camera (viewpoint) 3D objects Light sources Shading Textures
What do we want? 2D image
Graphics rendering pipeline For API design, practical top-down problem
solving, and hardware design, and efficiency, rendering is described as a pipeline
This pipeline contains three conceptual stages:
Produces
material to be
rendered
Application
Decides what, how, and
where to
render
Geometry
Renders the final image
Rasterizer
Application stage
The application stage is the stage completely controlled by the programmer
As the application develops, many changes of implementation may be done to improve performance
The output of the application stage are rendering primitives Points Lines Triangles
Important jobs of the application stage
Reading input Managing non-graphical output Texture animation Animation via transforms Collision detection Updating the state of the world in
general
Acceleration
The Application Stage also handles a lot of acceleration
Most of this acceleration is telling the renderer what NOT to render
Acceleration algorithms Hierarchical view frustum culling BSP trees Quadtrees Octrees
Geometry stage
The output of the Application Stage is polygons
The Geometry Stage processes these polygons using the following pipeline:
Model and View Transform
Vertex Shading Projection Clipping Screen
Mapping
Model Transform
Each 3D model has its own coordinate system called model space
When combining all the models in a scene together, the models must be converted from model space to world space
After that, we still have to account for the position of the camera
Model and View Transform
We transform the models into camera space or eye space with a view transform
Then, the camera will sit at (0,0,0), looking into negative z
Vertex Shading
Figuring out the effect of light on a material is called shading
This involves computing a (sometimes complex) shading equation at different points on an object
Typically, information is computed on a per-vertex basis and may include: Location Normals Colors
Projection
Projection transforms the view volume into a standardized unit cube
Vertices then have a 2D location and a z-value
There are two common forms of projection: Orthographic: Parallel
lines stay parallel, objects do not get smaller in the distance
Perspective: The farther away an object is, the smaller it appears
Clipping
Clipping process the polygons based on their location relative to the view volume
A polygon completely inside the view volume is unchanged A polygon completely outside the view volume is ignored
(not rendered) A polygon partially inside is clipped
New vertices on the boundary of the volume are created Since everything has been transformed into a unit cube,
dedicated hardware can do the clipping in exactly the same way, every time
Screen mapping
Screen-mapping transforms the x and y coordinates of each polygon from the unit cube to screen coordinates SharpDX conforms to the Windows standard of
pixel (0,0) being in the upper left of the screen OpenGL conforms to the Cartesian system
with pixel (0,0) in the lower left of the screen
Backface culling
Backface culling removes all polygons that are not facing toward the screen
A simple dot product is all that is needed This step is done in hardware in SharpDX and
OpenGL You just have to turn it on Beware: If you screw up your normals, polygons
could vanish
Rasterizer Stage
The goal of the Rasterizer Stage is to take all the transformed geometric data and set colors for all the pixels in the screen space
Doing so is called: Rasterization Scan Conversion
Note that the word pixel is actually short for "picture element"
More pipelines
As you should expect, the Rasterizer Stage is also divided into a pipeline of several functional stages:
Triangle
Setup
Triangle
Traversal
Pixel Shadin
g
Merging
Triangle Setup and Traversal Setup
Data for each triangle is computed This could include normals
Traversal Each pixel whose center is overlapped by a
triangle must have a fragment generated for the part of the triangle that overlaps the pixel
The properties of this fragment are created by interpolating data from the vertices
These are done with fixed-operation (non-customizable) hardware
Pixel Shading
This is where the magic happens Given the data from the other
stages, per-pixel shading (coloring) happens here
This stage is programmable, allowing for many different shading effects to be applied
Perhaps the most important effect is texturing or texture mapping
Texturing
Texturing is gluing a (usually) 2D image onto a polygon To do so, we map texture coordinates onto polygon
coordinates Pixels in a texture are called texels This is fully supported in hardware Multiple textures can be applied in some cases
Merging
The final screen data containing the colors for each pixel is stored in the color buffer
The merging stage is responsible for merging the colors from each of the fragments from the pixel shading stage into a final color for a pixel
Deeply linked with merging is visibility: The final color of the pixel should be the one corresponding to a visible polygon (and not one behind it)
The Z-buffer is often used for this
Week 3: Programmable Shading
More pipes!
Modern GPU's are generally responsible for the Geometry and Rasterizer Stages of the overall rendering pipeline
The following shows colored-coded functional stages inside those stages Red is fully programmable Purple is configurable Blue is not programmable at all
Vertex Shader
Geometry
ShaderClipping Screen
MappingTriangle Setup
Triangle Traversa
l
Pixel Shader Merger
Programmable Shaders
You can do all kinds of interesting things with programmable shading, but the technology is still evolving
Modern shader stages such as Shader Model 4.0 and 5.0 use a common-shader core
Strange as it may seem, this means that vertex, pixel, and geometry shaders use the same language
Vertex shader
Supported in hardware by all modern GPUs For each vertex, it modifies, creates, or ignores:
Color Normal Texture coordinates Position
It must also transform vertices from model space to homogeneous clip space
Vertices cannot be created or destroyed, and results cannot be passed from vertex to vertex Massive parallelism is possible
Geometry shader
Newest shader added to the family, and optional Comes right after the vertex shader Input is a single primitive Output is zero or more primitives The geometry shader can be used to:
Tessellate simple meshes into more complex ones Make limited copies of primitives
Stream output is possible
Pixel shader
Clipping and triangle set up is fixed in function Everything else in determining the final color of
the fragment is done here Because we aren't actually shading a full pixel, just a
particular fragment of a triangle that covers a pixel A lot of the work is based on the lighting model The pixel shader cannot look at neighboring pixels
Except that some information about gradient can be given
Multiple render targets means that many different colors for a single fragment can be made and stored in different buffers
Merging stage
Fragment colors are combined into the frame buffer
This is where stencil and Z-buffer operations happen
It's not fully programmable, but there are a number of settings that can be used Multiplication Addition Subtraction Min/max
Week 4: Linear Algebra
Vector operations
We will be interested in a number of operations on vectors, including: Addition Scalar multiplication Dot product Norm Cross product
Interpretations
A vector can either be a point in space or an arrow (direction and distance)
The norm of a vector is its distance from the origin (or the length of the arrow)
In R2 and R3, the dot product follows:
where is the smallest angle between u and v
φcosvuvu
Cross product
The cross product of two vectors finds a vector that is orthogonal to both
For 3D vectors u and v in an orthonormal basis, the cross product w is:
xyyx
zxxz
yzzy
z
y
x
vuvu
vuvu
vuvu
w
w
w
vuw
Cross product rules
Also: wu and wv u, v, and w form a right-handed system
)ba)ba
φ
wvwuwvu
uvvu
vuvuw
()((
sin
Matrix operations
We will be interested in a number of operations on matrices, including: Addition Scalar multiplication Transpose Trace Matrix-matrix multiplication Determinant Inverse
Matrix-matrix multiplication
Multiplication MN is legal only if M is p x q and N is q x r
Each row of M and each column of N are combined with a dot product and put in the corresponding row and column element
Determinant
The determinant is a measure of the magnitude of a square matrix
We'll focus on determinants for 2 x 2 and 3 x 3 matrices
100111001110
0100)det( mmmmmm
mmMM
222120
121110
020100
)det(
mmm
mmm
mmm
MM211200221001201102
211002201201221100
mmmmmmmmm
mmmmmmmmm
Adjoint
The adjoint of a matrix is a form useful for transforming surface normals
We can also use the adjoint when finding the inverse of a matrix
We need the subdeterminant dij to define the adjoint The adjoint A of an arbitrary sized matrix M is:
For a 3 x 3:
221202
211101
201000
)adj(
ddd
ddd
ddd
M
Multiplicative inverse of a matrix
For a square matrix M where |M| ≠ 0, there is a multiplicative inverse M-
1 such that MM-1 = I For cases up to 4 x 4, we can use the
adjoint:
Properties of the inverse: (M-1)T = (MT)-1
(MN)-1 = N-1M-1
)adj(11 MM
M
Orthogonal matrices
A square matrix is orthogonal if and only if its transpose is its inverse MMT = MTM = I
Lots of special things are true about an orthogonal matrix M |M| = ± 1 M-1 = MT
MT is also orthogonal ||Mu|| = ||u|| Mu Mv iff u v If M and N are orthogonal, so is MN
An orthogonal matrix is equivalent to an orthonormal basis of vectors lined up together
Homogeneous notation
We add an extra value to our vectors It's a 0 if it’s a direction It's a 1 if it's a point
Now we can do a rotation, scale, or shear with a matrix (with an extra row and column):
1000
0
0
0
222120
121110
020100
mmm
mmm
mmm
M
Translations
Then, we multiply by a translation matrix (which doesn't affect a direction)
A 3 x 3 matrix cannot translate a vector
1000
100
010
001
z
y
x
t
t
t
T
Lines
Explicit form (works for 2D and 3D lines) : r(t) = o + td o is a point on the line and d is its direction
vector Implicit form (2D lines only):
p is on L if and only if n • p + c = 0 If p and q are both points on L then we can
describe L with n • (p – q) = 0 Thus, n is perpendicular to L n = (-(py- qy),(px – qx)) = (a, b)
Planes
Once we are in 3D, we have to talk about planes as well
The explicit form of a plane is similar to a line: p(u,v) = o + us + vt o is a point on the plane s and t are vectors that span the plane s x t is the normal of the plane
Week 5: Transforms
Affine transforms
Adding a vector after a linear (3 x 3) transform makes an affine transform
Affine transforms can be stored in a 4 x 4 matrix using homogeneous notation
Affine transforms: Translation Rotation Scaling Reflection Shearing
List of transforms
Notation Name Characteristics
T(t) Translation matrix
Moves a point (affine)
R Rotation matrix Rotates points (orthogonal and affine)
S(s) Scaling matrix Scales along x, y, an z axes according to s (affine)
Hij(s) Shear matrix Shears component i by factor s with respect to component j
E(h,p,r) Euler transform Orients by Euler angles head (yaw), pitch, and roll (orthogonal and affine)
Po(s) Orthographic projection
Parallel projects onto a plane or volume (affine)
Pp(s) Perspective projection
Project with perspective onto a plane or a volume
slerp(q,r,t) Slerp transform Interpolates quaternions q and r with parameter t
Translation
Move a point from one place to another by vector t = (tx, ty, tz)
We can represent this with translation matrix T
1000
100
010
001
),,()(z
y
x
zyx t
t
t
tttTtT
Rotation matrices
1000
0cossin0
0sincos0
0001
)(φφ
φφφxR
1000
0cos0sin
0010
0sin0cos
)(
yR
1000
0100
00cossin
00sincos
)(φφ
φφ
φzR
Scaling
Scaling is easy and can be done for all axes as the same time with matrix S
If sx = sy = sz, the scaling is called uniform or isotropic and nonuniform or anisotropic otherwise
1000
000
000
000
)(z
y
x
s
s
s
sS
Rotation around a point
Usually all the rotations are multiplied together before translations
But if you want to rotate around a point Translate so that that point lies at the
origin Perform rotations Translate back
Shearing
A shearing transform distorts one dimension in terms of another with parameter s
Thus, there are six shearing matrices Hxy(s), Hxz(s), Hyx(s), Hyz(s), Hzx(s), and Hzy(s)
Here's an example of Hxz(s):
1000
0100
0010
001
)(
s
sxzH
Rigid-body transforms
A rigid-body transform preserves lengths, angles, and handedness
We can write any rigid-body transform X as a rotation matrix R multiplied by a translation matrix T(t)
1000
)(222120
121110
020100
z
y
x
trrr
trrr
trrr
RtTX
Non-commutativity of transforms
This example from the book shows how the same sets of transforms, applied in different orders, can have different outcomes
Normal transforms
The matrix used to transform points will not always work on surface normals
Rotation is fine Uniform scaling can stretch the normal (which should be unit) Non-uniform scaling distorts the normal
Transforming by the transpose of the adjoint always gives the correct answer
In practice, the transpose of the inverse is usually used
Inverses
For normals and other things, we need to be able to compute inverses The inverse of a rigid body transform X is X-1
= (T(t)R)-1 = R-1T(t)-1 = RTT(-t) For a concatenation of simple transforms with
known parameters, the inverse can be done by inverting the parameters and reversing the order:▪ If M = T(t)R() then M-1 = R(-)T(-t)
For orthogonal matrices, M-1 = MT
If nothing is known, use the adjoint method
Euler transform
We can describe orientations from some default orientation using the Euler transform
The default is usually looking down the –z axis with "up" as positive y
The new orientation is: E(h, p, r) = Rz(r)Rx(p)Ry(h) h is head (or yaw), like shaking
your head "no" p is pitch, like nodding your head
back and forth r is roll… the third dimension
Quaternions
Quaternions are a compact way to represent orientations
Pros: Compact (only four values needed) Do not suffer from gimbal lock Are easy to interpolate between
Cons: Are confusing Use three imaginary numbers Have their own set of operations
Operations
Multiplication
Addition
Conjugate
Norm
Identity
),(ˆˆ vvwwvwvwvv rqqr rqrqrqrq
),(ˆˆ wwvv rq rqrq
),(ˆ*wv qqq
2222)̂( wzyx qqqqn q
)1,(ˆ 0i
Vertex blending
If we animate by moving rigid bodies around each other, joints won't look natural
To do so, we define bones and skin and have the rigid bone changes dictate blended changes in the skin
Morphing
Morphing interpolates between two complete 3D models Vertex correspondence▪ What if there is not a 1 to 1 correspondence between vertices?
Interpolation▪ How do we combine the two models?
If there's a 1 to 1 correspondence, we use parameter s[0,1] to indicate where we are between the models and then find the new location m based on the two locations p0 and p1
Morph targets is another technique that adds in weighted poses to a neutral model
10)1( ppm ss
Orthographic projections
An orthographic projection maintains the property that parallel lines are still parallel after projection
The most basic orthographic projection matrix simply removes all the z values
This projection is not ideal because z values are lost Things behind the camera are in front z-buffer algorithms don't work
1000
0000
0010
0001
0P
Canonical view volume
To maintain relative depths and allow for clipping, we usually set up a canonical view volume based on (l,r,b,t,n,f)
These letters simply refer to the six bounding planes of the cube Left Right Bottom Top Near Far
Here is the (OpenGL) matrix that translates all points and scales them into the canonical view volume
1000
200
02
0
002
0
nfnf
nf
btbt
bt
lrlr
lr
P
Perspective projection
A perspective projection does not preserve parallel lines
Lines that are farther from the camera will appear smaller
Thus, a view frustum must be normalized to a canonical view volume
Because points actually move (in x and y) based on their z distance, there is a distorting term in the w row of the projection matrix
Perspective projection matrix
Here is the SharpDX projection matrix
It is different from the OpenGL again because it only uses [0,1] for z
0100
00
02
0
002
0
nffn
nffbtbt
btn
lrlr
lrn
P
IDEA Evaluations
Upcoming
Next time…
Review up to Exam 2
Reminders
Finish Project 4 Due on Friday before midnight