Computer Graphics

Sense from Symbol

Applications of Graphics

Entertainment• consumer enjoyment

Design• of physical objects to be built

Visualization• of data of any kind

Training• of reflexes for critical tasks

Entertainment

Games• interactive virtual worlds

• graphics, sound, physics, AI, rules, networking

Movies• animation

—totally computer generated imagery

• special effects—mix of imagery of the physical and the virtual worlds

Television• cartoons - simple animation

• advertisements - simple special effects

Design

Vehicles• cars, airplanes, etc.

Buildings• architecture & engineering

Appliances• Industrial Design

Electronics• integrated circuits (chips), boards

Coupled with• simulation & automated manufacturing

Visualization

Science & Engineering• simulations

—mostly physics for engineering

• real-world data—weather, oceans, astronomical, planetary

Medical• body sensing

—MRI, CT, ultrasound sonograms

Data Mining• statistical analysis and decision support• economics, business, demographics, etc.

Training

Vehicle piloting• aircraft• ships, tanks• trucks, cars

Team• combat• emergency

Individual skill• soldiers• repair

Graphics Data SourcesDesigned

• computer aided design tool—used by artist or designer

• specialized for each application

Sampled• cameras

• lasers

• magnetic, x-ray, ultra-sound

Algorithms• mathematical functions

—e.g. sphere := x2 + y2 = r2

• algorithms—e.g. fractal terrain

Graphics Systems

SensoryInput

Cognition

MotorOutput

Human Virtual World

Rendering

Modeling

AnimationMotion Tracking

Speech Recognition

Hand Control

Displays

Speakers

Force-feedback

Transducers

Geometric Modeling

Function• Represents the contents of space

— the virtual world at a single moment in time

Constituents• Geometry – represents shape• Energy sources – light, sound and motion• Attributes –

Appearance : properties that effect perceptionMass : properties that effect motion

FoundationsMathematics : continuous functionsData structures : organizing data to facilitate algorithms

Rendering

Function• Generate perception of a virtual world at an instant (or small

interval) of time

Form• Algorithms that take the geometric model as input and generate

output that drives transducers—Displays, speakers, force-feedback

• Based-on physics and perceptual physiology—only as needed to create perception

Motion

Function• updates state of geometric model from one

moment in time to the next

Forms• Interactive input (possibly from a network)

• Animation – designed or sampled• Physics - Newtonian dynamics• Behavior – Artificial Intelligence (AI)

Brain -> GeometryBrain perception

• pre-wired for perception of space (Kant :1724-1804)—presumably for most, if not all, animals

• perception—objects, and motion of objects—relative location/orientation

left/right of, front/behind of, above/below of

Cognition• experiential abstractions (right brain)

—extension, shape, connectedness—location, spatial relations, and movement

• linguistic abstractions (left brain)—space, time, ordering, change—geometry, topology, motion

Geometry

Foundations• space : extension• point : location• geometry : shape • dimension : spatial relation

Euclidian Geometry• points, lines, polygons• angles, length• coincidence, intersection, parallel• similarity, congruence

Geometry & Numbers

Geometry• sets of points• discrete vs. continuous

—discrete means “separate and countable”—continuous means “no gaps”

Numbers• Integers

—discrete—cardinals (counting) & ordinals (ordering)

• Reals—continuous—integer + factional component : 3.14159

Analytic Geometry

Geometry + Numbers• introduced by Rene Descartes (1596-1650)• started the Northern Renaissance

Dimension• numbers : ordering <, =, >• space : left-of, coincident, right-of• multi-dimensional

—multiple independent but simultaneous dimensions

Coordinates• tuples of values

—one value per dimension : ( x,y ) or ( x,y,z )

Functions

Mappings• points -> points• domain -> range

—can differ in dimension & coordinate systems

Equations• explicit : y = x*m + b• implicit : r2 = x2 + y2

• parametric : P = t*P1 + P0

Uses• shape : all points that satisfy some equation

• motion : change in position as a function of time

• transforms : mapping a space onto itself, i.e. domain = range

Digital Numbers

Integers• base 2 string of binary digits : 98 = 1100010• 8-bit (byte), 16-bit (short), 32-bit (int), 64-bit (long)

Reals• fixed point

—5.75 -> 101.110

• floating point—scientific notation -> 2.25 * 103

—fraction (mantissa) + exponent -> 5.75 -> .101110 * 23

—precision– single (float) : 32-bit -> 24-bit mantissa + 8-bit exponent– double : 64-bit -> 56-bit mantissa + 8-bit exponent

Digital Points

Coordinates• number of coordinates == dimension of space

• need one number per coordinate

• number can be any digital number (int, float, etc.)

Java2D Class Constructors• java.awt.Point( int x, int y ) - discrete space

• java.awt.geom.Point2D.Float( float x, float y )

• java.awt.geom.Point2D.Double( double x, double y )

Digital Lines

Lines• 1D, continuous, straight (not curved)• infinite length, unbounded

Line Segments• finite subset of a line• definable by two end-points

Java2D Class• java.awt.geom.Line2D

Line Equations

Explicit• y = mx + b, slope/y-intercept form

Implicit• 0 = Ax + By + C, normal/distance form

Parametric• P = P1 * (1-t) + t * P2, weighted sum form

• P = O + D*t, directed ray form

Transformations

Purpose• make global geometric changes• restricted to use of linear equations in CG

Elementary Transforms• Translation - change in position• Rotation - change in orientation• Scaling - change in size• Shearing - change in perpendicularity

Combinations• any sequence of elementary transformations can be used• order of transforms matters in most cases

Classes of Transformations

Rigid Body• preserves distances and angles• translation & rotation

Affine• preserves parallel lines• all 4 elementary transformations

Projective• preserves straightness/flatness• affine + perspective

Linear• rotation, scaling and shearing

Translation

Geometry• change only in position

—relative to the origin or to other shapes

Vector/Linear Algebra• is vector addition• P’ = P + T, where T is the translation vector

Properties• identity, inverse, closure, commutative

Rotation

Geometry• change in orientation• by some angle A either CCW or CW• about a some point (in 2D)

Properties• identity, inverse, closure, commutative• fixed point

Algebraic form• x’ = x * cos A - y * sin A• y’ = x * sin A - y * cos A

Scaling

Geometry• change in size• change in aspect ratio• change in handedness• projection

Algebra• x’ = x * Sx• y’ = y * Sy• Sx == Sy => symmetric scaling• Sx != Sy => asymmetric scaling

Shearing

Geometry• change in perpendicularity• introduces slant

Algebra• in X

—x’ = x + y * Shx

—y’ = x * Shy + x

Display of Motion

Create visual perception of motion• movies, TV, interactive graphics

• sequence of snap shots : frames

• played back rapidly at a constant frame rate

Variety of frame rates• Film : 24 fps (frames per second)

• S-TV: 30 fps

• HD-TV: 60 fps

• Computers : 60-120 fps

Interactive Animation

Demands• constant display rate• no pre-computed frames

—all images synthesized dynamically—need fast interactive response time : < 50ms

Issues• rendering time varies per frame

—depends on quantity of geometry and sophistication of rendering

• display fps != rendering fps—distance moved per rendered frame should depend upon rendering

time to create illusion of constant rate motion

Interactive Animation

Solutions• Frame Buffers

—holds a single frame as a digital-image—used to drive the display at constant frame rate—decouples rendering rate from display rate

Problems• rendering process will be visible

—buffering clearing—multi-frame rendering times

Double Buffering

Separate Rendering from Display• use two buffers, one “on-screen” and one “off-screen”

• sequence—display from on-screen buffer while rendering into off-screen

buffer

—swap buffers quickly when rendering is completed

Methods• Blt - copy off-screen to on-screen

• Flip - switch which buffer display processor reads

Displays

Monitors• CRT’s - cathode ray tube

—electron beam hits phosphors—image is drawn left-to-right, top-to-bottom

• LCD —light source : backlit or reflected—opacity of surface is modulated

Latest• LED - light emitting diode• Plasma - charged gas• DLP - laser + mirrors

Pixels

Picture Element• smallest area of digital color

displays : light emitting cameras : light sensing

• uses Red-Green-Blue color components—analogue of cones in the retina

Displays & Cameras

Resolutions• displays

— SDTV : 480 x 640— HDTV : 1080 x 1920— computers : adjustable

• cameras — “high res” : 1200 x 1600

Aspect ratio• height / width OR width / height• square pixels only

• displays— SDTV : 3 / 4— HDTV : 9 / 16

Use 2D array of pixels–uniform distribution

Digital Images

2D Array of pixels• pixels are digitally represented in memory• each pixel has red-green-blue components

—numbers -> brightness

Resolution• spatial resolution : width (X) & height (Y)• color resolution (bits per pixel)

—1-bit -> bi-level bitmap (white or black)—3-bits -> corners of color cube—color -> 3 x component-resolution

– 8-bit integer per component -> 24-bit RGB– 32-bit float per component -> 96-bit RGB

Frame BuffersDisplay Image

• used to drive display• holds a single digital

image, i.e. a frame• used to map logical to

physical pixels

Display Refresh- physical pixel output

decays rapidlyneeded to see sequence of

frames

• refresh rates—30 - 120 fps

LightPhotons

• all light is comprised of photons• properties: velocity, wavelength, frequency, polarization

wavelength = velocity / frequency

• interact with matter : transmitted, reflected, absorbed

Spectral Distribution• # of photons at each frequency/wavelength• visible spectrum

Visual Perception

Retina• 2D array of photo receptors; 7 million• rods : low illumination, B&W (grey scale)• cones : medium-high illumination, color

Photo Receptors• measure light intensity (counts photons)• sensitivity is frequency dependent

Cones

Types• “red”, “green”, “blue”

Color Perception

Photo receptor activation level -> brightnessCombinations of cone activation

1. one wavelength -> “rainbow colors”2. two wavelength -> rainbow + purples (hues)3. all cones at equal activation -> white (grey)4. arbitrary spectral distribution

– hue– purity/saturation (complement of whiteness)– brightness/lightness/value

Hue

Hue, Saturation, Lightness/Value

Color Cone

RGB Color Cube

RGB Color Space

MagentaCyan

Yellow

GreenRed

Blue

BlackWhite

Retina

Cell Structure Photoreceptor Distribution

3D Graphics

3D analogues of 2D• Geometry• Transformations• Set Operations• Animation• Interaction

3D only• Viewers• Lights• Shading• Projection

3D Geometry

Coordinates• points : (x,y,z) & vectors : [ x y z]T

• X, Y, Z axes & analogous bases vectors—handedness

Shapes• interior is a solid which has volume

• boundary is a surface—defined piecewise

• surface algorithms more complicated than for curves—e.g. set operations, collision detection

3D Curved Surfaces

Implicits• polynomials define interior, boundary and

exterior• sphere :

x2 + y2 + z2 = r2

Parametrics• many Bezier curves used to define a single

surface• recursive subdivision used to create triangle

mesh approximation

Polyhedra

Boundary• piecewise linear

—each piece is called a “face” or “facet”

• if closed => solid—polyhedra proper are closed—polyhedral meshes may be open

Use• real-time graphics is all polyhedral

—hardware only draws triangular meshes

• can approximate any curved surface arbitrarily well—often combined with “smooth shading”

Polytopes

Dimension independent concept3D polyhedra

2D polygons

1D line segments

0D point

Boundary : recursive definition• d-polytope bounded by set of (d-1)-polytope

—d is the dimension, stops at d == 0

3D Transformations

Categories : same as 2D• types

—rigid body, affine, projective, linear

• elementary—translations, rotation, scaling, shearing

Representations• change of vector basis

P = x * [X] + y * [Y] + z * [Z] + T

• matrices : 4x4

3D Translation

Change in 3D position• T = [ x y z]T

• P’ = P + Tx’ = x + Tx 1 0 0 Tx

y’ = y + Ty 0 1 0 Ty

z’ = z + Tz 0 0 1 Tz

w’ = w 0 0 0 1

3D Scaling

Analogous to 2D• change in size, aspect ratio, projection,

reflectionx’ = x * Sx Sx 0 0 0

y’ = y * Sy 0 Sy 0 0

z’ = z * Sz 0 0 Sz 0

w’ = w 0 0 0 1

3D Rotation

Rotation as change of vector basis• preserve :

—length no change is length of each vector basis

—perpendicularity between every pair of vector bases called orthogonal vector basis

• called orthonormal transformation

3D Rotation

Rotation about a coordinate axisex: rotation about z-axis

x’ = x * cos - y * sin cos -sin 0 0

y’ = x * sin + y * cos sin cos 0 0

z’ = z 0 0 1 0

w’ = w 0 0 0 1

similar for about x-axis or y-axis

3D Rotation

Rotation can be define about any line1. transform arbitrary line (axis of rotation) onto a coordinate axis2. apply desired rotation (now about a coordinate axis)

3. transform line back to original position : invert (1)

AxisAngleClass in Java3D

aa = new AxisAngle4d( Vector3d axis, double angle )m = new Matrix3d() ; m.set( aa );

QuaterionsGood for interpolation between rotations

q == [ axis * sin /2, cos /2 ]q = new Quat4d(); q.set( AxisAngle aa );

3D Shearing

Shearing in Zx’ = x + Shx 1 0 Shx 0

y’ = y + Shy 0 1 Shy 0

z’ = z 0 0 1 0

w’ = w 0 0 0 1

analogous for other axes

Order of 3D Transforms

Translations only or Scalings only• commutative

Rotations• about the same arbitrary axis

—commutative

• about the different axes—not commutative

Shearings• analogous to rotation

Mixed• not commutative

Scene GraphsProvide

• hierarchical organization of scene—for design and for supported features

Support• transformations

—order in which to apply them to geometric primitives

• animation—continuous interpolation of transformations—collision detection

• interaction—control of transforms with input devices—selecting object for modification : picking

• efficiency—find subset of model relevant to current rendering

Java 3D Graphs

Group nodes• Branch

—primarily for grouping together other nodes

• Transform—specifies a transform to be applied to all children

Leaf nodes• Shape3D - Geometry & Attributes

• Lights, Views, Sounds + specialized functions : e.g. Fog, Background

Behaviors for animation & interaction

Java 3D Execution

Definition phase• executes your code that defines the scene graph• no restrictions on what values can be set

Live phase• Java 3D engine processes scene graph for each

frame—updates values controlled by behaviors

• Capability bits—protects what can be modified at runtime