CS 352: Computer Graphics Input and Interaction. Interactive Computer GraphicsChapter 3 - 2 What good is Computer Graphics? JS1k canvas examples: 1 2

CS 352: Computer Graphics

Input and

Interaction

Interactive Computer GraphicsChapter 3 - 2

What good is Computer Graphics? JS1k canvas examples: 1 2 3 4 5 Games, visual demos…of what value? Is

there a Christian perspective? Communication of information to the user

Data visualization, simulation, GUIEven a word processor is an "interactive graphics

program" Communication between users

"Collaborative environments" are hot (multi-player games?)

Social networking is transforming the world…

Interaction is an essential component

http://js1k.com/2010-xmas/demo/856






Interaction Much of the fun and utility of graphics

is in interaction: accepting user input that affects the display

Paint programs, modeling, games, word processors, spreadsheets, powerpoint … User initiates input events such as clicking

on a menu or drawing a circle Computer response by changing graphical

display


http://cs.calvin.edu/curriculum/cs/352/files/canvaspaint/CanvasPaint.html


Projects 3 Projects 3 will be a paint program

we'll learn 2-D graphics, interaction, event-loop programming, double-buffering simple animation, basic image processing

Features: Pen, line, and rectangle tools Color, pen size selectors Image processing (sharpen, blur, detect edges) Open, save images Toolbar and menu for controlling application

How to set up an application UI? How to make a menu? How to make a color picker? How to make a toolbar? How to make toolbar buttons pop in

and out?



How to paint? How do I make a colored line follow the

mouse or fingertip?

Interactive prog: Input devices Event handlers Event loop




Input devices Interaction requires handling input

devices Physical: mouse, fingertip, keyboard,

joystick, digitizer, accelerometer, head tracker…

Logical: Text Locator Valuator (slider) Stroke Color picker

How to read?


Input devices Sample mode

Program reads the current state of input device (frequently)

Event mode Each click or motion generates an event

that goes on a queue Program can process events from queue

HTML?

Event-loop programming Events: button press, slider value

change, menu selection, mouse move, keypress

Event loop: Poll (or wait) for events Process events, update state Update display (ideally: wait for rest of frame time to

elapse)



State-driven Typically, the event loop is driven by a

big state table e.g. depressing the paintbrush tool releases

other tools, puts me in "paint mode" Good libraries will handle some of the

bookkeeping for you You still typically have to handle some

state




State diagram

Event loop in HTML/JS HTML/JS provides event queue, support

for many basic events (mousedown, mouseup mouseover, mousemove, keypress, key release, value change, button click, etc.)

You are on your own for higher-level events, e.g. clicking on a toolbar tool

It's also possible to set a function to run every 15 ms, sample input devices



Events for painting? mousedown:

go into paint mode store mouse position draw initial dot

mouseup: exit paint mode

mousemove: if in paint mode

draw a line from old mouse position to current set old mouse position to current position



Event handling with jQuery Binding events to functions

$(cpaint.canvas).bind('mousedown', cpaint.drawStart)

Processing eventscpaint.drawStart = function(ev) { var x, y; x = ev.pageX - $(cpaint.canvas).offset().left; y = ev.pageY - $(cpaint.canvas).offset().top; ev.preventDefault(); cpaint.paintmode = true; cpaint.oldX = x; cpaint.oldY = y;



Looking neat and spiffy How to avoid crinkles in your paint

strokes?



http://caimansys.com/painter/

Looking neat and spiffy How to avoid crinkles in your paint

strokes? Draw connected paths

Or just use line caps



Menus There are many jQuery menu plug-ins…

<ul id="mainmenu"> <li>File <ul> <li id="menuNew">New</li> <li id="menuOpen">Open</li> <li id="menuSave">Save</li>

-------------------------------------

$('#menuNew').bind('click', cpaint.clear); $('#menuOpen').bind('click',cpaint.open); $('#menuSave').bind('click',cpaint.save);



Toolbar

How to make a toolbar?

How should buttons behave?

State diagram?




Buttons Widgets may have several states State should be evident in visual feedback E.g. how exactly does a button work? States? State transition diagram

Most buttons: six states, with six different appearances

neutral neutral-highlighted neutral-depressed selected selected-highlighted selected-depressed

Events: mousedown, mouseup, enter, exit Transitions: what happens in each state under each

event?


Button state diagram (buttons selectable, not unselectable)

S

SHNH

N

NHD SHD

Press

Move

N: neutralH: highlighted (usually mouse over)

S: selectedD: mouse down


Other button considerations Could also consider

Tooltips Whether button merely clicks and releases

or can be selected Whether button can be unselected

(e.g. B/I/U vs. Left, Center, Right) Want consistent appearance, behavior

over whole program – or whole computer

Really need a library implementation and a strict set of UI guidelines…

4-State toolbar buttons: CSS.toolbarCell { background-color:#ddd; width:20pt; height:20pt; border: solid #eee 2px; box-shadow: 2px 2px 2px #666; }

#markerButton { background: url(img/paintbrush.png) no-repeat center center; }

.selected { box-shadow: inset 2px 2px 2px #666; }

.toolbarCell:hover { border:solid #555 2px; }



4-State toolbar buttons: JS$('#markerButton').bind('click', {tool:"marker"},

cpaint.selectTool);

-------------------

cpaint.selectTool = function(ev) { cpaint.tool = ev.data.tool; // get tool name

$('.toolbarCell').each(function(index) { // unselect $(this).removeClass('selected'); // others });

var tool = '#' + cpaint.tool + 'Button'; // get ID $(tool).addClass('selected'); // select }



Paintbrush size selector How?



Color picker Google "jQuery color picker"…

<input type=“color”> works in some browsers



How draw a line? What kind of feedback is normal?

Rubber Banding

Events and actions?



Save & restore Create your own off-screen canvas Copy it back on each mouse movement Events and actions?



Save & Restore event handling Mousedown

enter line mode remember mouse position as startx, starty save screen draw initial dot

Mousemove paste saved screen draw line from startx, starty to current

mouse pos Mouseup

exit line mode



Analysis Drawbacks?

slow – eats memory bandwidth for breakfast

copy smallest possible rectangle? Only points from line?

possible flickering use double buffering?




Double buffering Have two frame buffers, "front" and

"back" Draw into back buffer At vertical retrace time, swap buffers

This is a fundamental graphics technique…

…not built into canvas… …though, some drawing aggregation

seems to happen automatically, behind the scenes;

…not usually necessary in canvas, but browser dependent

Fake Double Buffering in Canvas Create off-screen canvas

canvasBuffer = document.createElement('canvas');canvasBuffer.width = canvas.width;canvasBuffer.height= canvas.height;canvasBufferContext= canvasBuffer.getContext('2d');

Draw into off-screen canvascanvasBufferContext.[drawSomeStuff…]

Copy onto display canvascontext.drawImage(canvasBuffer, 0, 0);



How to erase? Draw background color…



Save, Load? Save

Copy pixels to an image so users can right-click, save-as

Load What are the security risks? Can only load files from the same server Can use a file chooser if it's a local app Security policies are not entirely in place Consider chrome option

--allow-file-access-from-files



Resizing How would you allow the user to shrink,

expand image? What would happen to image

resolution?




Store drawing commands Realistically, to redraw effectively, you

have to save all drawing commandspen color redfill color blueline width 5circle 5 10 12textfont Times 12 text 10 10 "hello world"

Replay commands to redraw scene Could store commands in a file (e.g. Mac PICT

file) For some kinds of drawings, files are smaller

and more accurate than bitmaps


Other toolkits HTML offers basic UI capabilities in a

cross-platform context Writing your own UI extensions is

tedious jQuery plugins extend capabilities Ideally, all UI elements ought to be built

in In the real world, they come in

platform-specific 'toolkits' or 'windowing libraries'

E.g. MFC, QT, OpenLook, Cocoa …

Image processing Image processing examples:

blur, sharpen detect edges enhance contrast noise reduction posterize fisheye redeye reduction find faces …



How to fade an image? Could average pixel colors with white Or just decrease alpha…



How to blur an image?


How to blur an image? Blurring is an averaging process Convolution: apply a convolution kernel, e.g.

Center kernel on pixel of interest Multiply each kernel value by color value

underneath Add results Gaussian smoothing


1/9 1/9 1/9

1/9 1/9 1/9

1/9 1/9 1/9

http://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm

Gaussian smoothing



How to sharpen an image? It's a differentiation process What's Unsharp Masking?


How to sharpen an image? It's a differentiation process What's Unsharp Masking?

Subtract blurred version from original Add back to original


Sharpening Subtract neighbors Like subtracting a blurred version of the

image Unsharp Masking E.g. convolve with a kernel like one of

these:


http://homepages.inf.ed.ac.uk/rbf/HIPR2/unsharp.htm

Edge detection?

Wikipedia


Sobel Edge Operators Look for horizontal and vertical

variation

Could do this at different sales or resolutions

Note: results maybe positive or negative numbers; must normalize


1 2 1

0 0 0

-1 -2 -1

-1 0 1

-2 0 2

-1 0 1

Image Compression What if you created say six different

lower-resolution images and only stored the difference at each resolution?

Note: most of the data is in the highest-frequency component

An early image compression technique


Fourier Transform


The alg that changed imagery…

Discrete cosinetransform (DCT)

(Wikipedia)


http://en.wikipedia.org/wiki/Discrete_cosine_transform

JPEG image compression DCT to convert to frequency domain Perceptual modeling Coefficient quantization Entropy coding


Research in image processing and computer vision How would you clear up a blurry

photograph?

How would you recognize a face in a photograph?


Face recognition One approach: statistically derive a set of

“standardized face ingredients” from many examples, resulting in “eigenfaces”

Represent each face as linear combinationof eigenfaces


Summary Event loop programming Interaction and event handling in HTML State diagrams Painting Rubber banding Double buffering Basic image processing and

convolutions "Photoshop Nano"


Documents

CS 352: Computer Graphics Input and Interaction. Interactive Computer GraphicsChapter 3 - 2 What good is Computer Graphics? JS1k canvas examples: 1 2