Fall 2002 Interactive Graphical Systems - Uppsala University · PDF fileInteractive Graphical Systems ... • no filled surfaces ... • Fill Area Frame buffer operations are a heavy

Image Generators and DisplaysStefan Seipel

Interactive Graphical Systems Fall 2002

Historical Overview

1960 First random scanned display available 1

1964 A few applications using computer graphics ($100.000) 100

1968 Phosphor storage tubes increasingly available ($5000) 1000

1978 Raster scanned displays become increasingly popular 100.000

1984 First PC with graphical user interface (Macintosh) 1000.000

1994 Graphical user interfaces replaces increasingly text interface 50.000.000

1999 Almost no computer available without GUI 200.000.000

(Source: E.Bengtsson, Uppsala University)

Random scan display

• based on phosphor storage tube• cathode ray is controlled arbitrarily in XY direction• analog control of ray implemented in electronic circuits• sets of vector primitives available (points, lines )

• very smooth lines / high resolution• line drawing only• no filled surfaces• no block transfers• limited time frame to draw graphics • up to 100.000 short vectors per refresh cycle

Random scan display - computer control

• computer keeps a list of graphical primitives

• computer issues rendering and parameters

• very little memory required to store display list

• no management of graphics buffer required

• display/engine and computer strictly separated

Raster scan display

• based on TV technology - short glowing phosphors

• screen is scanned in a line-by-line order

• pixels within a line are switched on/off while line is scanned

• two-dimensional memory matrix (frame buffer) contains pixel value

• filled areas can be drawn

• screen must be refreshed in short intervals

• memory must be read out -> video signal must be generated

Raster scan display - computer control

• computer manipulates values in the frame buffer

• discrete frame buffer matrix -> rasterization

• arbitrary graphical objects can theoretically been displayed

• need for rasterization algorithms

• graphical primitives must be computed

• host computer calculates and accesses frame-buffer

Frame buffer readout - RAMDAC

host DAC(digital to analog converter)

Frame buffer RAM(random access memory)

memoryread

memorywrite

create videosignal

• readout of 1280x1024 pixels at 60 Hz -> memory access time 12.7 nanoseconds !

• “normal” RAM have an average access time of 60ns

• special purpose frame buffer required

• must allow for simultaneous read and write operations

Video Controller

Simple Raster Scan Display Architecture

CPU

SystemMemory

VideoMemory

VideoController

System Bus

other periphery

(frame buffer operations)

Monitor

More or less typical configuration with common graphics cards

Typical frame buffer operations:

2D - Graphics operations

• BitBlt (bit block transfer)

• Draw Pixel

• Draw Line

• Draw Text

• Fill Area

Frame buffer operationsare a heavy burden fora single CPU system !

2D graphics subsystem

CPU

SystemMemory

RasterEngine

VideoController

System Bus

other periphery

Monitor

Frame (color)Buffer

2D graphics subsystem

(2D graphical primitives)

Color buffer standards (common resolutions)

Spatial resolutions:CGA - 320x200EGA - 640x350VGA - 640x480SVGA - 800x600XGA - 1024x768SXGA - 1280x1024 or 1600x1280HDTV - 1900x1200ATC - 2000 x 2000Human eye - 10.000 x 10.000 (not evenly distributed)

Color resolutions:palletized - 256 colors lookup-table (8 bit/pixel)grayscale - 256 shades (8 bit /pixel)grayscale - 4096 shades (12 bit /pixel) e.g. in medical apps.Hi color - 5:5:5 RGB, or 5:6:5 RGB (15 or 16 bit/pixel)true color - 8:8:8 RGB (24 bit/pixel)true color - 8:8:8:8 RGBA (32 bit/pixel)

Memory requirements and signal bandwidths

Spatial Resolution SXGA 1280x1024Color Resolution True color 24 bitScreen Refresh 72 Hz

Video Memory Requirement : 1280*1024*24 bit = 3.75 Mbyte (1.3Mio pixel)

Video readout : 94 Mio./pixel per sec.

Memory readout : 226 Mio. bits per sec. !

-> very special memory readout required(e.g. 240 MHz RAMDAC)

3D graphics adds another dimension

2D color buffer only is not sufficient

The standard 3D graphics pipeline

Geometric processing- thousands of flops- vertex transform- normal transforms- lightning calculation

Rasterization- operates on pixels (framebuffer)- millions of iops- alpha compares- depth buffer test- stencil test- alpha arithmetic- texture addressing

DisplayList

Traversal

ModelingTransfor-mation

ViewingTransfor-mation

Clipping Projection Rasteri-zationLightning

Features of 3D image generators

Geometric processing (per vertex operations):

• one or several floating point geometry engines• perform matrix and vector operations

hardware supported graphics operations:• transformation of vertices• rotation of vectors• normalization of vectors (after scale of object)• calculation of lightning • projection & clipping

Features of 3D image generators

Rasterization (per pixel operations):One or several raster engines feature:

• discrete line drawing and polygon fill

• z-Buffer test (depth buffer test)

• z-Buffer blending (fog)

• blending with alpha-channel (transparency)

• color interpolation (e.g. Gouraud shading)

• texel addressing

• tri/bilinear interpolation of textures

• anti-aliasing of edges

• stencil test

Advanced 3D graphics subsystem

CPU

SystemMemory

System Bus

other periphery

Monitor

Graphics subsystem

GeometryProcessor(s)

RasterEngine(s)

VideoController

Frame Buffer(s)Other Buffers

(Stencil,Depth,Alpha…)

Local Memory(caches, textures)

Buffers configuration example (3D system)

red-color buffer 8 bitgreen-color buffer 8 bit

blue-color buffer 8 bitalpha buffer 8 bit

depth buffer 24 / 32 bit

accumulation buffer 8 b

stencil and/or overlay buffer 8 bit

double buffer(front and back)

Memory required per pixel: 2*24+8+8+32+8 = 128 bit = 16 byte

For a resolution of 1280x1024 -> 20 MB frame buffer

Buffers configuration example (3D stereo system)

red-color buffer 8 bitgreen-color buffer 8 bit

blue-color buffer 8 bitalpha buffer 8 bit

depth buffer 24 / 32 bit

accumulation buffer 8 bit

stencil and/or overlay buffer 8 bit

Memory required per pixel: 4*24+8+8+32+8 = 176 bit = 22 byte

For a resolution of 1280x1024 -> 27,5 MB frame buffer

left right

front

back

Quad-buffer

Depth-Buffer Aliasing

Fixed number of bitsin the z-Buffer limitsresolution of the scene depth

Example:16 bit depth buffer allows only65536 discrete steps in depth

Arithmetical rounding operationof floating point depth valuescauses ambiguous z-values !

Visual artifacts in rendering of objects which are very close toeach other(see picture)

Performance parameters – History (1999)

Pixel Fillrate:• refers to rasterization performance• number of shaded/textured/buffered pixels per second• common: 5-40 million/sec. (low and medium cost game accelerators)• quite good: 100 million/sec. (graphics workstations)• high end: 500-2000 million/sec. (e.g. SGI top of the line)

Geometry Performance:• refers to throughput of graphical primitives• number of 3D shaded triangles/sec. (of certain size e.g. 25 pixel)• number of 3D shaded lines/sec. (of certain length e.g. 10 pixels)• common: <0.5 million triangles/sec. (low cost game accelerators)• quite good: <1-2 million triangles/sec. (graphics workstations)• high end: <50 million/sec. (e.g. SGI Octane)

Nowadays (2002) performance doubled with ten even on PC hardwareThe gap between professional systems and consumer products is closing

Performance parameters - What do they say ?

Problem:• stated performance values are often only achieved under specific circumstances• often these values refer to the most simple rendering modes (no shading, no z-buffer)• performance is often achieved with native implementations• quality of driver implementation is essential• comparability is a tricky issue, since various new technical features

Therefore:• don´t trust vendor supplied specifications !• if you are lucky, you reach 10%-20% of the stated performance !• graphics sub-systems should be tried before bought !

• in the target system (depending largely on CPU configuration)• using your application (depending on typical graphics operations)

• Always check : is there driver support ? are there drivers at all ?

Bottlenecks in 3D graphics systems

Geometry bound systems- too many polygons in the scene- too complex lightning calculation- too complex model transformations

Fill bound systems- too big areas to fill (number of pixels per polygon)- too many large polygons- high overdraw ratio- too many textures in scene- too complex alpha arithmetic (blending, fog)

Depends on your application and graphics system

Example bottleneck evaluation

User requirements for a flight simulation:• terrain model 1000 textured polygons, will fill most of the screen background• airplane model 5000 polygons, average size 200 pixels• for quality reasons, rendering should appear in SXGA resolution• frame-rate >40/sec.

There is a graphics subsystem available:• 60 Mio. textured, lit, shaded pixels/sec.• 2.000.000 triangles/sec. (25 pixels)• cost: 30.000 SEK

Is it advisable to buy this graphics subsystem ?• (1280x1024+5000x200)x40 = 92.428.800 pixels/sec.

• (1000+5000)x2x40 = 480.000 triangles/sec

Fill bound!

API´s (application programmers interface)

Avoid to develop hardware oriented software

Minimize turnaround costs and time

Use well established and standardized 3D API´s

Use graphics accelerators which support those API´s- full functional support- optimized performance for these drivers

The best graphics hardware is worth nothing without appropriate API and driver support !

API´s continued

Different standard 3D API´s:

Many different available today. The most renowned are:

PHIGS and GKS (old DEC machines, not common any longer)

OpenGL (SGI, Microsoft, many others)

QuickDraw 3D (Apple)

Direct3D (Microsoft)

Glide (3Dfx)

Heidi (used by many CAD programs)

Quake (sort of standard for games)

Additional Reading

Roy S. Kalawsky: The Science of Virtual Reality and Virtual EnvironmentsAddison-Wesley Publishing Company, 1993, ISBN 0-201-63171-7

Perception: pages 50-59

Displays techniques: pages 98-107

DisplaysPhysiological Aspects

Spatial Retinal Resolution : 1´

Visual Field : approx. 200o, with 120o binocular overlap

Limits of depth perception from lateral disparity

Temporal Resolution : approx. 50 Hz, increasing with luminance

Visual Displays - Basic Technologies

• Cathode Ray Tubes

• Flat Panel Displays

• Electroluminiscence Displays

• LCD Displays

• Active Matrix TFT

• Light Valves

• Laser Scanners

• Micro Mirror Devices

Basic Technologies - Cathode Ray Tubes (CRT)

CathodeCathode

AccelerationAccelerationAnodeAnode

FocussingFocussing

xx--/y/y--DeflectionDeflection((elstatelstat // magnmagn.).)

Phosphor CoatingPhosphor Coating

Electron gun

Advantages:- high resolution- easy to control- reliable technology- low cost


Operation modes of CRTs:

Random scan mode Raster scan mode

+ no aliasing, smoth lines+ transformation in hardware possible+ little memory required

- only line drawing possible- complex control hardware for the beam- flickers if scene becomes complex

Basic Technologies - Random Scan CRT

How it works:

Display primitives are stored in a display list(ellipses, circles, lines...)

Electron beam is controlled continuously betweenvertice -> analog line drawing

+ uses old TV technology (simple approach)+ filled and shaded surfaces are possible+ guaranteed screen refresh rate

- aliasing problems “staircase effect”- electronics required for raster memory readout - drawing algorithms more complex

Basic Technologies - Raster Scan CRT

How it works:

Display is rasterized into pixels which are stored in a two-dimensional memory array (raster memory)

Electron beam is traversing the screen line by linein a regular time frame and scheme

Content of the raster memory controls beam intensity

Basic Technologies - Raster Scan CRT

Display ControllerDisplay Controller

HostHostHost00000000011100000000000000100010000000001010100000010101000110000000000000000000010010001111100000000000000000000010010100001001000000000000101010001001111110000000000000000101100111100101100000000000000000000000101001010011

Frame buffer

VideoController

VideoController

Converts analog primitiveinto discrete representation Converts discrete frame buffer

into analog video signal

scan line

horizontretrace

verticaretrace


The role of the phosphor coating :- Fluorescence (glowing when hit by electron beam)- Phosphorescence (after glowing while being activated)- Persistency (time until glowing phosphorescence decreases below 10%)typically 5-60 milliseconds.

- Persistency is important. Short persistency requires high update rates otherwiseflicker. Long persistency causes stabile but smeary images.

- Granularity of the phosphor -> spot size, image resolution

- Type of phosphor defines color:p1 : green, average persistencyp4 : white, short persistencyp12 : orange, average persistencyp31 : green, short persistency


How is color accomplished in displays ?

Most usually, colors are mixed by additive composition of base colors

1. Spatially modulated color composition

see page 99, in Kalawsky

2. Temporally modulated color composition

see next slide


Color shutter technology.

- Sequential display of color fields on monochromatic CRT- Synchronization of the fields with color filters

- Filters can be : a) LCD filters (electronically controlled)

b) Optical filters (mechanically coupled)

- Advantages : No color convergence errors

- Disadvantages : High frequent oscillations in the visual field decompose colors

How is color accomplished ?

Parameters for display assessment

- dot size (mm)- dot pitch (mm)- resolution (lp/mm)- brightness (cd/m2)- contrast ratio- display size- addressability- refresh rate- color range- convergence- weight / power consumption

1mm 1mm

1mm

1mm

dot size ?dot pitch?resolution?

Basic Technologies - Typical parameters for CRTs

Screen Diagonal Size (14”-26”)

Shadow masks (Triple holes, Strips)

Dot pitch (0.24 - 0.30 mm)

Video bandwidth (50 -250 MHz)

Horizontal Sync. Frequency (30 - 170 kHz)

Vertical Sync. Frequency ( 48 - 170 Hz)

max. Resolution (1280x1024 - 4800 x 4000)

Basic Technologies - Flat Panel Display - LCD

Liquid Crystal Displays

Light sourceor mirror

hor.pol.filter

vert. pol.filter

vertical electrodes

horizontaltransparentelectrodes

liquid crystallayer

No electric field = light passes through

Basic Technologies - Flat Panel Display - LCD

Addressing of Pixels:

Deposit an electronic charge on intersections between horizontaland vertical electrodes sequentially row by row.

Limited speed, since certain minimum time is required to is deposited (depending on the capacity of the intersections)

When last row has been addressed, first rows have already losttheir electric charge

Poor contrast image

Basic Technologies - Flat Panel Display

Thin Film Transistor Matrix (TFT):

An array of transparent transistors isdeposited on the LCD

Pixels can be switched on and off

Pixel keep their electrical state andoptical properties

=> Significant contrast and intensity enhancements


Color reproduction / Optical Efficiency:

Groups of adjacent pixels are forming one effective color pixel

Sub-pixels are covered with color filters

Common sub-pixels configuration are RGB stripes, triads or quads

Efficient resolution is reduced

Light intensity is diminished significantly when passing throughpolarizes, liquid crystals, and color filters (poor optical efficiency)

(see also page 96)


Electro Luminescence Displays (plasma panel displays):

S

D

t

Strike VoltageStrike Voltage

Discharge VoltageDischarge Voltage

Plasma generates light Plasma generates light

PotentialGas is encapsulated between electrodes

When a certain amount of voltage is applied(striking voltage) the plasma discharges andglows until the potential drops below thedischarge voltage.

Plasma cell keeps luminating for a whilewithout being refreshed.

Active luminance, high intensity display


Electro Luminescence Displays (plasma panel displays):

glas

vertical transparent electrodes

horizontal transparent electrodes

glass

glass substrate with plasma cells (Neon, Argon)

SandwichSandwich--TechniqueTechnique

Basic Technologies - Light Valves

see page 103

Application : Projection Displays

High resolution light modulation (>1600 x 1200)

High refresh rates possible (>130 Hz)

Usable for high light output projection displays

The first choice for stereo projection systems

Tricky problems : Ghost images with slow phosphor

Basic Technologies - Laser Scanner

Used for:

large screen projection displays

direct retinal displays

(very high resolution)

for color displays, several lasersrequired (convergence ?)

Screen

horizontaldeflection

mirror

verticaldeflection

mirror

laser source

Basic Technologies - Micromirror Devices (MMD)

Matrix of micro mirrors

Addressable and electronically controllable

Used for Light Reflection and Projection Display Systems

Extremely high optical efficiency

Basic Technologies - Micro Mirror Devices (MMD)

MMD System Working Principle

Visual Displays - 3D Displays and Optical Systems

3D Projection Systems Front ProjectionRear / Retro Projection

Autostereoscopic DisplaysSlot MaskLenticular / Double Lenticular Arrays

Volume Displays

Other Optical Coupled Displays Fiber coupled displays (HMD see chapter 4.2.1)Lens/Mirror coupled displays (HMD see chapter 4.2.1)

3D Displays and Optical Systems - Projection Systems

3D Projection Display Systems require:

Very high intensity image source• Transmission LCD/TFT Panel + Light Source• Reflection LCD/TFT Panel + Light Source• Projection CRT (specialized high intensity CRT tube )• MMD • Light Valve• (Laser)

Focusing optics/color splitter• Wide / narrow angle optics, fixed or variable

Projection screen• Transparency / Diffusion / Specular Properties

Means of splitting left/right channel• Time Multiplexing / Color Separation / Polarization


3D Front Projection Systems:Image source and observer are located on the same side of the projection surface( - user may interfere with projection beam)

Screen

Image Source + Optics

Observer

Stereo 3D with active shutter glasses• requires very fast image source (>120 Hz)• light valve / projection CRT• extremely expensive• single graphics pipeline• screen with good diffuser properties

Stereo 3D with passive polarizing glasses• requires two image sources• image alignment problems • dual graphics pipeline required• requires special silver screen which

preserves polarization

Stereo 3D with color field separation (red/green)• one color capable projector (cheap)• poor image result• screen with good diffuser properties


3D Rear / Retro Projection Systems:

Image source is positioned behind the projection screen

No interference between user and image source

Requires transparent screen material

No polarized 3D stereo possible sincepolarization is disturbed in transmission

Screen

Image Source + Optics

Observer

Stereo 3D with active shutter glasses• requires very fast image source• light valve / projection crt• expensive• single graphics pipeline

Attention has to paid to mirror effects


3D Rear / Retro Projection Systems: Examples

Caves Virtual Planes Viewing wands

3D Displays / Optical Systems - Autostereoscopic Displays

Autostereoscopy - stereoscopic perception with the “naked” eye

Pixe

l Col

umn

R

Pixe

l Col

umn

R

Pixe

l Col

umn

R

Pixe

l Col

umn

L

Pixe

l Col

umn

L

Pixe

l Col

umn

L

Display surface

Image Splitter (Sanyo)

• Display divided in vertical stripes

• Alternate stripes display left and right image

• Slit-mask is blocking out the view of the left

eye onto the right picture and vice versa

• Only a single user

• Dedicated observer position

• Horizontal resolution decreased

Slit mask


(Double) Lenticular Lens Arrays

• Display divided in vertical stripes

• Alternate stripes display left and right image

• Half-Cylinder shaped lenses project the

stripes to the corresponding eye

• Several viewing zones

• Dedicated observer distance

• Horizontal resolution decreased

LCD-Projektor LCD-Projektor

Double-lenticular retro-projection system

Lenticular flat panel display system

LCD/TFT Panel


Examples (Heinrich-Hertz Institute, Berlin)

• Allows for user movements

• Uses head-tracking

• Screen is automatically positionedcorrectly with a robot arm

• Allows for user movements

• Uses head-tracking

• Lenticular array is shifted

3D Displays / Optical Systems - Volumetric Displays

The display creates a real volumetric representation which is perceived as a 3Dstructure without the need for glasses or other aids.

The idea is to project dynamic images onto oscillating or rotating surfaces in order to createthe sensation of a volumetric object.

Prototypes have been build using:- rotating LED matrices- rotating helical projection surfaces with laser projection laser- lasers projecting into fog- experiments are underway to bring a solid crystal to illumination on addressable positions

All these systems can only show transparent/monochrome objects

Mechanical problems and limits, dead viewing areas

Commercial systems are far ahead

3D Displays / Optical Systems - Volumetric Displays Choice of VR Displays - Evaluation of Requirements

How many observers are watching at the same time ?

What resolution and color fidelity requirements are there ? -> basic display technology

Is wide field of view desirable ?

Is immersion an important issue for the application ?

Is stereoscopic 3D rendering required ?

If yes, decide which type • one screen polarized -> take care for optical properties of the system• one screen time multiplexed -> display must tolerate high refresh rates• dual screen (HMD) -> check for resolution• autostereoscopic ?

Does the application require interaction with haptic stimuli ?

Displays Technologies - Features

C R T L C D T F T M M D L i g h tw a l vA d d r e ssa b i l i t y 4 k P i x e l 2 k P i x e l 2 k P i x e l 1 . 3 k P i x e l 4 k P ix e lC o n t r a s t h ig h lo w h ig h h ig h h ig hC o l o r s ve r y g o o d m e d iu m g o o d ve r y g o o d ve r y g o o dD im e n s i o n s h u g e s m a l l / m e d iu m s m a l l / m e d iu m s m a l l m e d iu mR e f r e sh < 1 8 0 H z 6 0 H z 6 0 H z 6 0 ( 1 8 0 H z ) 1 4 0 H zC o s ts l o w a ve r a g e h ig h h ig h ve r y h i g h


Considerations with regard to stereo image projection

Time-multiplexing with active shutters:• both front and retro projection possible• active glasses are quite expensive (if many are required)• very high speed projector is required (light valve technology, expensive)

Polarized filtered images:• projection screen must preserve polarization (aluminized silver screen)• retro projection not yet possible (no suitable screen material available)• glasses are very cheap• two projectors are required (can cause image alignment problems)

FULLY IMMERSIVE SPHERICAL PROJECTION SYSTEM(THE CYBERSPHERE)

http://www.ndirect.co.uk/~vr-systems/sphere1.htm

Contact:David Trayner or Edwina Orr.RealityVision Ltd. 6 Yorkton St. London E2 8NH, UK: +44 (0)171 7399700F: +44 (0)171 739 9707E: [email protected]

RealityVision´s autostereoscopic displayusing holographic optical elements

Documents

Fall 2002 Interactive Graphical Systems - Uppsala University · PDF fileInteractive Graphical Systems ... • no filled surfaces ... • Fill Area Frame buffer operations are a heavy