Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is complementary to ray-tracing Lots of different implementations Software implementation

1

Computer Graphics

Graphic pipelineOpenGL, DirectX etc …

2

Computer Graphics

Graphic pipeline

Approach that is complementary to ray-tracing

Lots of different implementations Software implementation e.g. Pixar Renderman

Goal here : speed but also some realistic rendering (very flexible) Do a 2 hours movie in 1 year imposes leaves less than 3 minutes of CPU time

per image … (but it is easily run un parallel) Hardware implementations, e.g. PC graphic cards

Almost realtime : rendering of many millions of triangles per second

In what follows, we will restrict ourselves to an abstraction of an hardware implementation

3

Computer Graphics

Graphic pipeline

Graphic pipeline

Goal here : operations are easily parallelized or vectorized Groups of dedicated GPUs that are able to execute dozens to thousands ot

operations in parallel

→ explains why GPUs are so powerful ( many times that of generic CPUs à 1/5 of the clock speed)

Dedicated high-bandwith memory

4

Computer Graphics

Graphic pipeline

Bus performance (dates back to 2010 but ideas are same today)

Nvi

dia

5

Computer Graphics

Graphic pipeline

GPU specific architecture

Nvi

dia

6

Computer Graphics

Graphic pipeline

Pipeline

Command flow

Transformed geometry

Fragments ( ~ pixels + interpolated data)

framebuffer

Application

vertex processing

Raster conversion

Operations on fragments (fragment processing)

Display

We are here

3D Transformations shading

Compositing, mixing, (shading)

What the user sees

« Langage » standards OpenGL, DirectX etc...

Conversion of primitives into

fragments

7

Computer Graphics

Graphic pipeline

Primitives Points (in space)

Line segments Polylines

Triangles Triangle fans / strips

That’s all ! Curves ? Transformed into polylines Polygones ? Decomposed into triangles Curved surfaces ? Approximated with triangles

The current « trend » is the restrict to a minimal number of primitives Simple, uniform, repetitive → good for vectorial processing

8

Computer Graphics

Graphic pipeline

Command flow : depends on the implementation

e.g. OpenGL 1, DirectX, others

Always somewhat similar See OpenGL labs

OpenGL advantages : Multiplatform, simple, high performance, still evolving, and not attached to a

specific architecture or company. It became a standard and is supported everywhere, even on smartphones

9

Computer Graphics

Graphic pipeline

Pipeline

Command flow



framebuffer

Application

vertex processing

Raster conversion


Display

We are here



What the user sees



fragments

10

Computer Graphics

Graphic pipeline

Pipeline of geometrical transformations

11

Computer Graphics

Graphic pipeline

12

Computer Graphics

Graphic pipeline

Clipping Rasterization expects that primitives are visible on the

screen This is done in the 3D canonical space, after the application of the

perspective projection (see course 2 ), but before the perspective division.

All that is outside the volume limited by :

is discarded

Cut by 6 planes

−wxw−w yw−wzw

13

Computer Graphics

Graphic pipeline

Clipping Basic operation : cut a triangle into 3 by a plane

4 cases : All vertices inside – tr kept

All verticels exteriortriangle discarded

1 vertex in; 2 out → 1 triangle remains

2 vertices in; 1 out → 2 triangles remain

14

Computer Graphics

Graphic pipeline

Hidden face removal We have seen how to geometrically transform the primitives

to the screen

The perspective projection gives a strong hint on depth

Hidden face removal is another strong hint

Allows to draw exclusively what is seen (goal : performance)

15

Computer Graphics

Graphic pipeline

Hidden face removal Backface culling

For closed and opaque shapes, one does not see the inside !

16

Computer Graphics

Graphic pipeline


No need to draw the backward-facing faces

vn

n

v

17

Computer Graphics

Graphic pipeline


No need to draw the backward-facing faces

vn

n

v

v⋅n≤0

18

Computer Graphics

Graphic pipeline


Dépends ont the convention used to compute the normal to the shape

Usually, exterior pointing vector

Computation is easy if triangles are carefully oriented !

s1

n=s1 s2×s1 s3

∥s1 s2×s1 s3∥

s2

s3

19

Computer Graphics

Graphic pipeline

Hidden face removal How to care for this case ?

Painter’s algorithm

Binary space partitionning

Z-buffer

20

Computer Graphics

Graphic pipeline

Pipeline

Command flow



framebuffer

Application

vertex processing

Raster conversion


Display

We are here



What the user sees



fragments

21

Computer Graphics

Graphic pipeline

Raster conversion First stage : enumerate pixels that are covered by a

« continuous » primitive

Second stage : Interpolate values that are known on the vertices of the primitive

Example : the colour known at the vertices of a triangle may be distributed on each pixel covered by the triangle

Other variables may also be interpolated. Normal vectors for instance...

22

Computer Graphics

Graphic pipeline

Raster conversion Transformation of continuous primitives into discrete pixels

Example : drawing of a line

Difficulty : aliasing

23

Computer Graphics

Graphic pipeline

Naïve algorithm Line = unit width rectangle

One specifies beginning and end vertices

Case here : black inside and white outside

24

Computer Graphics

Graphic pipeline

Naïve algorithm Line = unit width rectangle

One specifies beginning and end vertices

Case here : black inside and white outside

25

Computer Graphics

Graphic pipeline

Point sampling One approximates the rectangle by drawing every pixel

whose center is inside the rectangle

26

Computer Graphics

Graphic pipeline

Point sampling One approximates the rectangle by drawing every pixel

whose center is inside the rectangle

Problem : sometimes pixels with more that one adjacency are turned on

27

Computer Graphics

Graphic pipeline

28

Computer Graphics

Graphic pipeline

Bresenham algorithm (midpoint alg.) We will define the thickness with respect to the y axis...

29

Computer Graphics

Graphic pipeline


One turns on only one pixel per column

45° slanted lineswill appearthinner.

30

Computer Graphics

Graphic pipeline



45° slanted lineswill appearthinner.

31

Computer Graphics

Graphic pipeline

32

Computer Graphics

Graphic pipeline

Bresenham algorithm (midpoint alg.) Equation :

Evaluated for everu column


y=mxb

x0x1

0≤m≤1for x = ceil(x0) to floor(x1) y=b+m*x plot(x,round(y))

0

1

2

4

5

6

7

3

y=0.49x−0.01

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

33

Computer Graphics

Graphic pipeline

Optimization Multiply and rounding are rather slow operations (at least on

primitive CPUs)

For each pixel, the only options are E and NE One computes the error :

d > 0.5 decidesE or NE

d=m x1b− y

0

1

2

4

5

6

7

3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

x , y

NE

E

y=mxb mxb− y=0

34

Computer Graphics

Graphic pipeline

Optimization One must only update integer steps along x and y

Exclusive use of the addition (no mult or divide)

d > 0.5 decidesE or NE

d=m x1b− y

0

1

2

4

5

6

7

3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

d=dmd =d −1

35

Computer Graphics

Graphic pipeline

Optimization

0

1

2

4

5

6

7

3

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

x=ceil(x0) // round to ceily=round(m*x+b) // round to nearestd=m*(x+1)+b-yWhile (x<floor(x1)) // round to floor{ If d>0.5 { y=y+1 d=d-1 } x=x+1 d=d+m plot(x,y)}

36

Computer Graphics

Graphic pipeline

Generally, endpoints are given

Implicit form :

y=dydx

⋅x+b x0

y0

x1

y1

dx=x1−x0

dy= y1− y0

G x , y =dy⋅x−dx⋅ydx⋅b

G x , y0

G x , y 0

G x , y =0

≡F x , y=2 dy⋅x−2 dx⋅y2 dx⋅b

37

Computer Graphics

Graphic pipeline

What is the value of F at M ?

: M is under the linego to NE

: M is abovego to E

d i=F x i1, yi1/2=2 dy⋅x i1−2 dx⋅ y i1 /22 dx⋅b

M

Q

NE?

E?

d i0

d i≤0

x i , y i

38

Computer Graphics

Graphic pipeline

First point: what is the value of ?

As belongs to the line :

d 0

d 0=F x01, y01 /2=2 dy⋅ x01−2 dx⋅ y01/22 dx⋅b

=2 dy x0−2 dx⋅y02 dx⋅b2 dy−dx

=F x0, y02 dy−dx

x0, y0 F x0, y0=0

d 0=2dy−dx

39

Computer Graphics

Graphic pipeline

Recursion : what is the value of ?

If then go to E :

Otherwise go to NE :

d i1

d i≤0

x i1 , y i1=x i1 , y i

x i1 , y i1=x i1 , y i1

d i1=F x2, y1/2=d i2 dy

d i1=F x2, y3 /2=d i2dy−2 dx

40

Computer Graphics

Graphic pipeline

Algorithm is valid only for one octant

Bresenham( x1, y1, x2, y2 ){

dx = x2 – x1dy = y2 – y1d = 2*dy – dxplot( x1, y1 )while (x1 < x2 ){

if (d <= 0) // EASTd = d + 2*dy

else // NORTH-EAST{

d = d + 2*(dy-dx)y1 = y1 + 1

}x1 = x1 + 1plot( x1, y1 )

}}

41

Computer Graphics

Graphic pipeline

What to do if the line is not in the right octant ? Exchange x and y

Exchange start and end points

Replace y by -y

42

Computer Graphics

Graphic pipeline

Same type of algorithm do exist for other geometric shapes : e.g. circles (see litterature

What about aliasing ? Algorithm derived from Bresenham that avoid aliasing do exist, see algorithm

of Xiaolin Wu

Oversampling → sampling on a finer grid then averaging on bigger pixels is also an option.

Bresenham, Jack E. "Algorithm for computer control of a digital plotter", IBM Systems Journal, Vol. 4, No.1, pp. 25–30, 1965

Wu, Xiaolin. "An efficient antialiasing technique". Computer Graphics 25 (4): 143–152, 1991.

http://www.research.ibm.com/journal/sj/041/ibmsjIVRIC.pdf

http://portal.acm.org/citation.cfm?id=122734

http://en.wikipedia.org/wiki/Computer_Graphics

43

Computer Graphics

Graphic pipeline

44

Computer Graphics

Graphic pipeline

Interpolation Some variable is known at the vertices of the triangle (color,

normal vector, etc...)

One wish to get a representative value of the same variable along the line for every pixel that is turned on.

A progressive variation would be fine, thus a linear interpolation is the right tool here :

1D :

2D,3D : a is simply the distance ratio to the endpoints.

f x=1− f 0 f 1

=x−x0/ x1−x0

45

Computer Graphics

Graphic pipeline

Interpolation The pixels are not exactly on the line

One defines a projection on the line It is linear

One may useresults obtainedbefore to construct an interpolation

P0

P1

v

=v x / L=v y / L

=v⋅Q−P0/ L

L=v⋅P1−P0

46

Computer Graphics

Graphic pipeline

Interpolation The pixels are not exactly on the line

One defines a projection on the line It is linear

One may useresults obtainedbefore to construct an interpolation

P0

P1

v

=v x / L=v y / L

=v⋅Q−P0/ L

L=v⋅P1−P0

47

Computer Graphics

Graphic pipeline

Alternative meaning d and a are updated from pixel to the next pixel

d tells us at which distance we are to the line

a tells the position along the line

d and a are coordinates in the natural frame of the line

48

Computer Graphics

Graphic pipeline

Alternative meaning The loop means the pixels that we visit

Interpolation of d and a at each pixel

A fragment is emmitted if the pixel’s center is in the band

Interpolationbecomes themain operator

P0

P1

v

u

49

Computer Graphics

Graphic pipeline

Alternative meaning

P0

P1

v

u

x=ceil(x0)y=round(m*x+b)d=m*(x+1)+b-y etc...while (x<floor(x1)){

if (d>0.5){

y=y+1d=d-1 etc ...

} else

{x=x+1d=d+m etc...

}If (-0.5 < d <= 0.5)

plot(x,y, … )}

50

Computer Graphics

Graphic pipeline

Triangle raster conversion Very common case

With a good antialiasing, may be the only case ! Some systems represent lines with two very thin triangles

Triangle represented with 3 vertices

The algorithm has the same philosophy as for lines : One walks from pixel to pixel

Linear operators are evaluated for each step

Those operators allow us to know if a pixel is inside or outside

51

Computer Graphics

Graphic pipeline

Triangle raster conversion

Input :

Three 2D points

Variables to interpolate, at each vertex

Output : a list of fragments, with :

Integer coordinates of pixels

Interpolated variables

x0 , y0 ; x1 , y1; x2 , y2

q00 ,⋯, q0n ; q10 ,⋯, q1 n ; q20 ,⋯ , q2 n

x , y

q0 ,⋯ , qn

52

Computer Graphics

Graphic pipeline

Triangle raster conversion Incremental evaluation of linear functions on a grid of pixels

Functions are defined by the value at the vertices

Use of additionalfunctions to determinethe set of fragments to return back

x0 , y0

q00 ,⋯ , q0n

x1 , y1

q10 ,⋯ , q1n

x2 , y2

q20 ,⋯ , q2n

(x , y)

q0 ,⋯ , qn

Vertex

Fragment

53

Computer Graphics

Graphic pipeline

Incremental linear interpolation A linear affine function in the plane :

Evaluation on a grid is efficient :

q x , y =c x xc y yck

q x1, y=c x x1c y yck=q x , yc x

q x , y1=c x xc y y1ck=q x , y c y

+cx+c

x+c

x

+cy

+cy

.....

.....

54

Computer Graphics

Graphic pipeline

Interpolation of variables known at vertices Determine defining the unique linear function that

gives back the value at the vertices

3 parameters , 3 equations :

Singular system if points are collinear

cx , c y , ck

c x x0c y y0ck=q0

c x x1c y y1ck=q1

cx x2c y y2ck=q2x0 y0 1x1 y1 1x2 y2 1

cx

c y

ck=

q0

q1

q2

55

Computer Graphics

Graphic pipeline

Interpolation of variables known at vertices Translation of the origin to

2X2 linear system

Solution using Cramer’s rule

x0 , y0

q x , y=c x x−x0c y y− y0q0

q x1, y1=c x x1−x0c y y1− y0q0=q1

q x2, y2=c x x2−x0c y y2− y0q0=q2

x1−x0 y1− y0

x2−x0 y2− y0cx

c y=q1−q0

q2−q0

c x= q1 y2− q2 y1/ x1 y2− x2 y1

c y= q2 x1− q1 x2/ x1 y2− x2 y1

56

Computer Graphics

Graphic pipeline

What are the fragments to consider ? Those for which the barycentric coordinates are positive …

Algebraically, on has

Inside iff

Pineda, Juan, " A parallel algorithm for polygon rasterization" Computer Graphics 22 (4): 17-20, 1988

a

c

b

, , p= a b c=1

0 ; 0 ; 0

57

Computer Graphics

Graphic pipeline

Barycentric coordinates are interpolated variables Each barycentric c. yields 1 on a specific vertex and 0 on all others.

They are an implicit representation of the sides of the triangle.

58

Computer Graphics

Graphic pipeline

Pixel-per-pixel raster conversion (Pineda’s algorithm - 1998)

One vists conservatively a superset of the pixels

Use of interpolation of linear functions

Use of barycentric coordinates todetermine whento emit a fragment

x0 , y0

q00 ,⋯ , q0n

x1 , y1

q10 ,⋯ , q1n

x2 , y2

q20 ,⋯, q2n

59

Computer Graphics

Graphic pipeline

Triangle raster conversion Beware of rounding and arbitrary decisions

One has to visit every pixel at least once... (otherwise there may be a hole !)

But not two times ! (Those pixels would take an arbitrary colour that depend in which order things are drawn)

Elegant solution : antialiasing...

60

Computer Graphics

Graphic pipeline

Pipeline

Command flow



framebuffer

Application

vertex processing

Raster conversion


Display

We are here



What the user sees



fragments

61

Computer Graphics

Graphic pipeline

Z-buffer (depth buffer) In many application, sorting with respect to depth is too

costly The order changes with the viewpoint

There exists the BSP tree Viewpoint independent but :

Heavy datastructures (difficult to implement « in silico ») Cutting of primitives Slow building Non-incremental (all data have to be known in advance)

62

Computer Graphics

Graphic pipeline

Solution that is usually favoured : draw in any order and keep track of the closest drawn pixel

Use an additional storage, that stores, for each pixel, the smallest depth to date

When one ought to draw a new pixel, the actual depth is compared to the stored depth and if the latter is greater, then the pixel is drawn and the depth is updated.

63

Computer Graphics

Graphic pipeline

Z-buffer

This is an axemple of a « brute-force » approach, it works because memory is cheap and fast.

Fol

ey e

t al

.

64

Computer Graphics

Graphic pipeline

Z-buffer Evidently limited to bitmap images (not vectorial)

Somewhat more difficult to implement with transparency (alpha channel)

65

Computer Graphics

Graphic pipeline

Z-buffer and alpha channels One separates opaque objects and translucent ones

1) Opaque objects are drawn with an update of the z-buffer.

2) Then , use a BSP tree for partially transparent entities

3) Draw transparent entities using the BSP ordering, while takin the Z-buffer into account, but without updating it

Thus, transparent faces behind opaque ones are not drawn (they are not visible)

66

Computer Graphics

Graphic pipeline

Z-buffer : limited accuracy The supplementary channel is generally encoded as an

integer, as are the color channels

Cause : hardware implementation (simple and fast)

It is possible to have distinct objects seen at the same depth if the value that is stored in the Z-buffer is the same

The accuracy is spread between n (near plane) and f (far plane)

These were used to define the observable volume, see course 2.

0< z*< N −1

67

Computer Graphics

Graphic pipeline

z*=0

z*=N −1

Plane n Plane f

Z-buffer

68

Computer Graphics

Graphic pipeline

Z-buffer Let us choose an integer with b bits (8 or 16...)

What is then the accuracy of the z-buffer?

If the z that is stored is proportional to the actual distance (case of orthogonal projections):

We have N=2b layers for a distance equal to f-n Therefore, the accuracy (independent to the distance!) is

: to maximize the accuracy, f-n must be kept small.

If a perspective projection is used, the z stored in the z-buffer is proportional to the result obtained after the perspective division.

Therfore, the size of the layers depends on depth By how much ?

Δ z=( f −n)

2b

69

Computer Graphics

Graphic pipeline

z*=0 z*

=N −1

Plane n Plane f

Orthographic proj.

Perspective proj.

Z-buffer

70

Computer Graphics

Graphic pipeline

Z-buffer

In the second course, we have computed zc after the

perspective divide :

x , y , z ,1⋅2∣n∣

r−l0 0 0

02∣n∣

t−b0 0

rlr−l

tbt−b

∣ f ∣∣n∣

∣n∣−∣ f ∣−1

0 02∣ f ∣∣n∣∣n∣−∣ f ∣

0

M proj _ persp _ OpenGL

= X ,Y ,∣ f ∣∣n∣∣n∣−∣ f ∣

⋅z2∣ f ∣∣n∣∣n∣−∣ f ∣

,− z

z c=∣ f ∣∣n∣∣ f ∣−∣n∣

2∣ f ∣∣n∣

z ∣ f ∣−∣n∣

z c=f nf −n

2 f n

z f −n

71

Computer Graphics

Graphic pipeline

Z-buffer

The interval for zc is 2 ( -1 to 1) and one still have N layers.

What is the size of the layers ? Need to solve for z (invert)

The largest layer is for z=f :

zc=f nf −n

2 f n

z f −n zc≈

2 z f n

z2 f −n

=2N

z=z2

f −n

N f n zmax=

f f −n

N n

72

Computer Graphics

Graphic pipeline

Z-buffer Example: n = 1 m, f = 100 m, the z-buffer has 8 bits →

N=256, what is the actual size of the layers ?

With n=10 :

z=z2

f −n

N f nΔ zmax=

f ( f −n)

N n=

100⋅99256⋅1

=39 m (0.15m with 16 bits)

zmin=n f −n

N f=

1⋅99256⋅100

=0.0039 m

Δ zmax=f ( f −n)

N n=

100⋅90256⋅10

=3.5 m (0.013 m with 16 bits)

zmin=n f −n

N f=

10⋅90256⋅100

=0.035 m

73

Computer Graphics

Graphic pipeline

Z-buffer For a good accuracy, it is best to increase n and decrease f.

Never ever set n=0 → cancels the z-buffer

Generally, the number of bits b is fixed by the hardware (usually 16, 24 or 32 bits)

The more the better but takes as much memory as the picture (usually 24 bits)

74

Computer Graphics

Graphic pipeline

Interpolation in perspective projection

projection plane

eye

Projections of the endpoints

75

Computer Graphics

Graphic pipeline


projection plane

eye

Projection of the middle of the line ≠ middle point of the projected lineLinear interpolation in screen coordinates ≠ interpolqtion in eye space

p1

p2

p1 p2

2

76

Computer Graphics

Graphic pipeline


projection plane

eye

z1 z2

Projects in the middle ...

z1z2/2


77

Computer Graphics

Graphic pipeline


Projection plane

eye

⏟ equidistant on z


78

Computer Graphics

Graphic pipeline


Projection plane

eye

zc1 z c2

This projects in the middle

zc1z c2/2


79

Computer Graphics

Graphic pipeline


Projection plane

eye

The depth variable that has to be interpolated (at the pixel level is zc (screen depth)

obtained after perspective divide, and not z, the real coordinate

⏟ equidistant on z c (screen depth)

80

Computer Graphics

Graphic pipeline

The perspective correction (use of zc instead of z ) aims to

avoid problems for e.g. textures applied on slanted surfaces

F u=1−u F 0u F 1

F u=

1−uF 0

zc0

uF 1

zc1

1−u1zc0

u1zc1

wik

iped

ia

F0

F1

81

Computer Graphics

Graphic pipeline

Minimal pipeline « Vertex » stage (input : 3D positions / vertices and colors / triangle)

Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion Pixel list Transmission of the color

« Fragment » stage (output : color) Display pixels with right colors on the frambuffer

82

Computer Graphics

Graphic pipeline

83

Computer Graphics

Graphic pipeline

Minimal pipeline with z-buffer « Vertex » stage (input : 3D positions / vertices and colors / triangle)

Position transformation (object space → eye space) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion

Interpolation of zc (z in screen space)

Transmission of the color

« Fragment » stage (output : color, zc)

Display pixels with right colors on the frambuffer and update z-buffer only if zc <

current zc

84

Computer Graphics

Graphic pipeline

85

Computer Graphics

Graphic pipeline

Flat shading Uses the « real » normal to the triangle

Faceted appearance Most realistic view

of the real geometry (as defined)

Foley et al.

86

Computer Graphics

Graphic pipeline

Minimal pipeline with z-buffer and flat shading « Vertex » stage (input : 3D positions / vertices and colors / triangle +

normal) Position transformation (object space → eye space) Color computation (flat shading) with the normal (one color / tri) Position transformation (eye space → screen space) Transmission of the color (no interpolation = constant on the triangle)

Raster conversion

Interpolation of zc (z in screen space)

Transmission of the color



current zc

87

Computer Graphics

Graphic pipeline

Flat shading

88

Computer Graphics

Graphic pipeline

Observer and illumination : close vs. far Phong illumination require some geometric info

Light vector (depends on position)

Obsever vector (depends on position)

Normal to the surface (computed beforehand)

Observer & light vectors are changing Must be computed & normalized for every facet

observer

light

89

Computer Graphics

Graphic pipeline

Observer and illumination : close vs. far Case where observer and source are far away

Almost parallel light rays

Almost orthographic projection

Light & observer vectors do not change much

A frequent optimization is to consider they do not change even though it is generally false

90

Computer Graphics

Graphic pipeline

Directional light (e.g. sun) Light vector is constant

In many cases, it increases dramatically the throughput of the pipeline by simplifying computations

observer

light[ x y z 0 ]

91

Computer Graphics

Graphic pipeline

Observer at the infinite Orthographic projection ?

Constant projection angle

One may also do that for the perspective projection, only for shading computations

The observer vector is considered constant (for instance, it is normal to the image plane)

Yield strange results if a wide angle view is used

Blinn-Phong shading: observer, light, and bisector vector all constant (see course 4)

92

Computer Graphics

Graphic pipeline

Directional light & far observer

Only the normal changes. Means the shading of every facet is the same if the orientation is the same.

observer

light[ x y z 0 ]

[xo yo zo 0]

93

Computer Graphics

Graphic pipeline

Gouraud interpolation One wants a smooth shading eventhough the geometry is

faceted Remember mapping : sensitivity to the normals much greater than the

sensitivity to positions

Idea : colour is computed at the facet’s vertices

Then, an interpolation is performed to get the color everywhere, at same time than the raster conversion.

94

Computer Graphics

Graphic pipeline

Gouraud interpolation

Foley et al.

95

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Gouraud interpolation « Vertex » stage (input : 3D positions / vertices and color per triangle +

normal per vertex) Position transformation (object space → eye space) Color computation made per vertex Position transformation (eye space → screen space)

Raster conversion

Interpolation of zc (z in screen space), rgb color



current zc

96

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Gouraud interpolation

97

Computer Graphics

Graphic pipeline

Gouraud interpolation Normals at vertices

Mathematically undefined

If the tessellation (triangulation) is obtained from a smooth surface (sphere, B-Spline, etc..), one may take the exact normal on that surface at the vertex

If not, just do as if by averaging neigboring triangle normals

N s=

∑i

N i

∥∑i

N i∥N s

N 1

N 2

N 3N 4

N 5

98

Computer Graphics

Graphic pipeline

Gouraud interpolation May be applied to any shading model

Diffuse

Blinn-Phong

Etc ...

However, for specular shading, it does not work well There are strong variations of the shading with respect to the position, even

on a single triangle. A linear interpolation is just too basic to reproduce these variations.

99

Computer Graphics

Graphic pipeline

Blinn shading and with Gouraud interpolation

Foley et al.

100

Computer Graphics

Graphic pipeline

Phong interpolation Why not interpolate normals and compute shading at the

pixel (fragment) level ? As easy as interpolating colors

The shading will be computed are every pixel, from information that have been interpolated (colors, normals, etc...

In the Graphic pipeline, it means that we move the shading computation from the « Vertex » stage to the « fragment » stage.

101

Computer Graphics

Graphic pipeline

Pipeline

Command flow



framebuffer

Application

vertex processing

Raster conversion


Display

We are here



What the user sees



fragments

102

Computer Graphics

Graphic pipeline

Pipeline with z-buffer and Phong interpolation « Vertex » stage (input : 3D positions / vertices , triangle, + normal/color per

vertex) Position transformation (object space → eye space) Position transformation (eye space → screen space)

Raster conversion

Interpolation of zc (z in screen space), rgb color, xyz normal


Shading computation using interpolated data (including normal)


current zc

103

Computer Graphics

Graphic pipeline

Phong interpolation

104

Computer Graphics

Graphic pipeline

OpenGL Some shading models are part of the standard (typically :

Phong, Lambert, Gouraud, Blinn-Phong …), Nothing has to be done .

For specific shading computations, there is an API that allows defining exactly what has to be done at each stage :

Either at the « vertex shader » stage, vertex per vertex

Or, at the « fragment shader » stage, pixel per pixel

Recent graphic cards (and the software driver embedded in the operating system) allow to litterally program in pseudo C-language either the « vertex shader » or the « fragment shader » (see e.g. Nvidia CUDA , OpenCL)

105

Computer Graphics

Graphic pipeline

Programmable !

Not very versatile

106

Computer Graphics

GPGPU

GPGPU Use of computing power of GPUs to do other things that just

shading ...

Recent GPUs are somewhat versatile (but less than generic CPUs)

Branching became possible

Random memory access

But they are designed for vector computing (i.e. same operations applied to different data, SIMD)

There are standards to drive these GPUs : Open Computing Language (OpenCL, open)

Common Unified Device Architecture (CUDA, Nvidia)

Direct Compute (Microsoft)

107

Computer Graphics

GPGPU

OpenCL – pseudo-c language Host program may be in C++, and runs on the CPU

The OpenCL code is stored as a character string (char[] or std::string )

The OpenCL code is then compiled by the graphic card’s « driver » (the CPU does the job), result is uploaded on the GPU by dedicated system calls

To access to the machine code, system calls allow to pass parameters and memory chunks to the GPU

The computation is made on the GPU

Results are given back to the host program using memory transfer.

108

Computer Graphics

GPGPU

OpenCL sample code

__kernel void VectorAdd(__global float* c, __global float* a , __global float* b, __constant float *cst{ // Index of the elements to add unsigned int n = get_global_id(0); unsigned int nl = get_local_id(0); unsigned int gsz = get_global_size(0); unsigned int lsz = get_local_size(0); unsigned int gid = get_group_id(0); // do some math from vectors a and b and store in c __private float res; res=0.; int i; for (i=0;i<100;++i) // not a loop over elements of the arrays { res=a[n]+sin(a[n])+b[n]; if (res>5.5) res=a[n]* (*cst) ; } c[n] = res;}

109

Computer Graphics

GPGPU

In the host program … init

std::string Src ; // Source code (see preceding slide)

std::vector<cl::Platform> platforms;cl::Platform::get(&platforms);

cl_context_properties properties[] = { CL_CONTEXT_PLATFORM,(cl_context_properties)(platforms[0])(),

0};

cl::Context context(CL_DEVICE_TYPE_ALL, properties);std::vector<cl::Device> devices =

context.getInfo<CL_CONTEXT_DEVICES>();cl::Program::Sources

source(1,std::make_pair(Src.c_str(),Src.size()));

cl::Program program = cl::Program(context, source);

program.build(devices); // compilation !

110

Computer Graphics

GPGPU

In the host program … system calls

cl::CommandQueue queue(context,devices[0],0,&err);cl::Buffer GPUOutVec(context, CL_MEM_WRITE_ONLY

,sizeof(float)*SIZE, NULL, &err);float cst=1;cl::Buffer GPUVec1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float)*SIZE,HostVec1,&err);cl::Buffer GPUVec2(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float)*SIZE,HostVec2,&err);cl::Buffer GPUCst1(context,CL_MEM_READ_ONLY|CL_MEM_COPY_HOST_PTR

,sizeof(float),&cst,&err);cl::Event event1;cl::Kernel kernel(program_, "VectorAdd", &err);kernel.setArg( 0, GPUOutVec);kernel.setArg( 1, GPUVec1);kernel.setArg( 2, GPUVec2);kernel.setArg( 3, GPUCst1);

queue.enqueueNDRangeKernel(kernel,cl::NullRange,cl::NDRange(SIZE_TEST),cl::NullRange,NULL,&event1);

event1.wait();// at this point, the computation has been made.

111

Computer Graphics

GPGPU

In the host program … collecting data back

Complete code sample on the website of the course

// at this point, the computation has been made.

cl::Event event2;queue.enqueueReadBuffer(GPUOutVec,CL_TRUE,0,

SIZE_TEST*sizeof(float),HostOutVec, NULL, &event2);event2.wait();

// HostOutVec contains the results

for (int Rows = 0; Rows < SIZE_TEST/32; Rows++){

for (int c = 0; c <32; c++) std::cout << HostOutVec[Rows * 32 + c] << " " ; std::cout << std::endl;}

112

Computer Graphics

GPGPU

OpenCL is a multiplateform paradigm The above code sample may be compiled on any computer (having a

GPGPU enabled driver)

It can even be executed on a ... CPU (when no GPU is available) There exist « dummy » drivers that will simply compile and execute the code on

the same CPU. Useful to debug and benchmark

Performance : factor 2 to 100 in favor of the GPU for vector operations (e.g. on big arrays)

Core i7 with 6 cores 3,33 Ghz vs. Nvidia Quadro FX 580 (not very powerful) : 110 s. vs 55 s.

On a specialized GPGPU graphic card : Nvidia Tesla C2075 : 11 s.

113

Computer Graphics

Graphic pipeline

Painter’s algorithm Idea : display every primitive in the right order

Everything « below » is overwritten, like when painting

114

Computer Graphics

Graphic pipeline



115

Computer Graphics

Graphic pipeline



Amounts to define a topological sorting Find a path in an oriented graph

AB

C

D

EF

A

B

C

D

E

F

ABCDEFABDCFECAEBDF...

116

Computer Graphics

Graphic pipeline

Painter’s algorithm Impossible if cycles are present ...

A

B

C

A

B

C

ABC ???

117

Computer Graphics

Graphic pipeline

Painter’s algorithm Impossible if cycles are present …

Solution : cut all !

a

f

o

b,l cd,g

m

k

ej

i,n

h a

o

l

cg

m

k

ei

h

db

n

118

Computer Graphics

Graphic pipeline

Painter’s algorithm Useful when an order is easy to define

Works with vector graphics

May be very CPU intensive (cut;sort ..)

Fol

ey e

t al

.

119

Computer Graphics

Graphic pipeline

Painter’s algorithm The ordering depends on the point of view

Sorting primitives is costly ( nlog(n) to the best ) and have to be done every time the observer moves

Primitives must be cut if forming cycles (how to detect ?)

A response to these drawbacks is the binary space partition tree (BSP Tree)

120

Computer Graphics

Graphic pipeline

BSP Tree Input data :

Segments in 2D

Triangles in 3D – same idea

a b

c

d

e

121

Computer Graphics

Graphic pipeline

How to build the BSP Tree (1)

a b.1

c

d

e

b.2

c+ -

+

-

→ Take one of the segments and define a line cutting the plane in two separate half-planes→ Classify the other segments with respect to the boundary. If a segment is crossing, partition it and classify its parts→ On each subdomain, if there are more than one segment, repeat the procedure recursively

122

Computer Graphics

Graphic pipeline


a b.1

c

d

e.1

b.2

c

e.2

d

+ -+

-

+- +-

e.1


123

Computer Graphics

Graphic pipeline


a b.1

c

d

e.1

b.2

e.2

c

d b.1

a

+ -+

-

+-

+-

e.1


124

Computer Graphics

Graphic pipeline


a b.1

c

d

e.1

b.2

e.2

c

d b.1

a

+ -+

-

+-

+-

e.1+

-

e.2

b.2


125

Computer Graphics

Graphic pipeline

How to choose the right segment at each step ? A random choice is not that bad...

Complete algorithm :

Let S be a set of line segments (or triangles in 3D)Build(S,BSP){

If (Card(S) <=1) BSP is a tree with only one node that contains the only segment of S or nothingElse{

Use a random segment s belonging to S as a cutting line and cut all the other segments S+ = segments belonging to H+ (« positive » halfspace) (without s)S- = segments belonging to H- (« negative » halfspace) (without s)Call recursively Build(S+,BSP+)Call recursively Build(S-,BSP-)Build a tree with BSP as a root, and BSP+ and BSP- as children. The root contains s.

}}

126

Computer Graphics

Graphic pipeline

Use of the BSP Tree How to scan the tree to get the right display order ?

a

+ -

e.1

b.2

ObserverO

Let O c+ a point (the observer)→ it is clear that entities from c- must be displayed before entities from c, that must be displayed before those of c+.

c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

127

Computer Graphics

Graphic pipeline


a

+ -

e.1

b.2

c- c c+The same remark arises for c- (then c+)O b.1- thus entities of b.1+ must be displayed before those on b.1, and before those in b.1-

1

23

4

5

c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

ObserverO

128

Computer Graphics

Graphic pipeline


a

+ -

e.1

b.2

a b.1 (b.1-) c c+For c+ :O d+ thus the order is d- d d+

1

23

4

5

6 c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

ObserverO

129

Computer Graphics

Graphic pipeline


a

+ -

e.1

b.2

a b.1 (b.1-) c d- d d+

1

23

4

5

6

7

8

9

c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

ObserverO

130

Computer Graphics

Graphic pipeline


a

+ -

e.1

b.2

a b.1 (b.1-) c e.1 d d+For d+ :O e.2+ thus order is e.2- e.2 e.2+

1

23

4

5

6

7

8

9

10101112

c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

ObserverO

131

Computer Graphics

Graphic pipeline


a

+ -

e.1

b.2

a b.1 (b.1-) c e.1 d b.2 e.2 (e.2+)

Final order : a b.1 c e.1 d b.2 e.2

1

23

4

5

6

7

8

9

10101112

c

d b.1

e.2

a b.1

c

d

e.1

b.2

e.2

+

-

+-

+-

+

-

ObserverO

132

Computer Graphics

Graphic pipeline

Recusive scanning algorithmDraw(BSP,ViewPoint){

If BSP is a leaf (no children)Draw the primitives contained in BSP

Else{

Let BSP+ and BSP- - children of BSPIf ViewPoint is in H- (« negative » halfspace){

Call Draw(BSP+,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP-,ViewPoint)

}Else If ViewPoint is in H+ (« positive » halfspace){

Call Draw(BSP-,ViewPoint)Draw the primitives contained in BSPCall Draw(BSP+,ViewPoint)

}Else (we are exactly on the plane...){

(Draw the primitives contained in BSP) / but not necessaryCall Draw(BSP+,ViewPoint)Call Draw(BSP-,ViewPoint)

}}

}

BSP

BSP-BSP+

133

Computer Graphics

Graphic pipeline

The BSP tree is generally not directly used in graphic cards Building a BSP tree is relatively slow (nlogn to the best), and the

datastructure is not well adapted to vector/parallel treatment.

→ Z-buffer : see further

It can however be used in certain cases in software to ease the work of graphic cards

e.g. « FPS » video games where the environnment is mostly fixed « Doom » is an old video game which used this principle.

Il can be used for ray-tracing...

Documents

Graphic pipeline OpenGL, DirectX etc · 2 Computer Graphics Graphic pipeline Approach that is complementary to ray-tracing Lots of different implementations Software implementation