If you can't read please download the document
Upload
vuhanh
View
221
Download
1
Embed Size (px)
Citation preview
Chalkboard: Mapping Functions to Polygons
Andy Gill and Kevin Matlage
The University of Kansas
September 23, 2009
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 1 / 19
ChalkBoard
What is ChalkBoard?
An Uncompromisingly Declarative description language,
for two dimensional images,
for the sake of visualizing information and ideas.
ChalkBoard Images are Immutable Values.
... $ fmap (\ x -> if xthen greenelse white)
$ rotate 0.05$ scale 50$ checker
An efficient image generator will allow an efficient animation language.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 2 / 19
Chalkboard Motivation
Why another graphics language?
DSLCapture
FunctionalReactive
ProgrammingGPUs
ChalkBoard
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 3 / 19
Pan Images
ChalkBoards main image structure is inspired by Conal Elliots Pan.
type Image a = (R,R) -> a
This function captures a 2-D plane of values.
Examples from Pan.
... :: Image Bool -- region
... :: Image (R,R) -- location
... :: Image RGB -- Picture
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 4 / 19
Chalkboard Abstract Pan-style Images
Gameplan for Chalkboard
Board, an abstract version of Pans Image.
Build a deep DSL around Board.
Compile Board to an efficient graphics engine.
ChalkBoard IR (CBIR) is a polygon pushing language.
ChalkBoardImage
Specification
DeepDSL
ChalkBoardIR
OpenGL
GPU
DSL Capture
& Compile// ChalkBoard
Back End//
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 5 / 19
Primitives in Chalkboard
Generatorspure :: a -> Board acoord :: Board (R,R)
Moversscale :: Float -> Board a -> Board amove :: (Float,Float) -> Board a -> Board arotate :: Float -> Board a -> Board a
Pointwise
:: (a -> b) -> Board a -> Board b :: Board (a -> b) -> Board a -> Board bover :: (Over a) => Board a -> Board a -> Board a
run
lookup :: Board a -> (R,R) -> a
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 6 / 19
ChalkBoard Example
... = (\ (x,y) -> RGB (x + 0.5) (y + 0.5) 0) coord
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 7 / 19
ChalkBoard Overlay Example
board :: Board RGB
board = unAlpha (sq1 over cir1 over pure (alpha white))
where
cir1 :: Board (Alpha RGB)
cir1 = move (0.2,0.2) $ choose (withAlpha 0.5 green) (transparent white) circle
sq1 :: Board (Alpha RGB)
sq1 = move (-0.2,-0.2) $ choose (withAlpha 0.5 red) (transparent white) square
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 8 / 19
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 9 / 19
ChalkBoard: Conceptual Challenges
Two parts to the creation of a complex board.
Outer (Applicative) Functor Shape of the computation
Small set of primitive combinators (, fmap, scale, etc.)Finite tree (perhaps a DAG) of computation.Capturing this part of the DSL is possible using IO-based observablesharing.
Inner individual points or values.
Infinite number of computations possible over points and values.Capturing these functions requires some new techniques.Still an open problem inside ChalkBoard.
fmap and
fmap :: forall a, b . (a -> b) -> Board a -> Board b
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 10 / 19
Big Idea in ChalkBoard
Optimize what we can, generate what we can not!
Many operations can be given algebraic style interpretations
Draw a unit circle, in red, on white background
fmap (\ x -> if x then red else white) circle
Draw a unit circle, in red, on a green square, over white
fmap (withDefault white) (cir over sqr)wherecir, sqr :: Board (Maybe RGB)cir = fmap (withMask red) circlesqr = fmap (withMask green) square
What we can not optimize, generate an appropriately sized bufferdirectly, and use this in future computations.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 11 / 19
ChalkBoard IR - An Imperative Polygon Pushing Language
Main operations are
Allocate Buffer
Size, depth, background are specified.
Splat Polygon
Takes list of point maps.Splats a polygon from one buffer to another.
Others include deallocation, import/export of images.
Buffers are true first class objects in CBIR.
Buffers can be used as both textures andtargets.
Alpha blending is associative in CBIR.
Source Buffer
Target Buffer
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 12 / 19
Compiling to ChalkBoard IR
Current implementation of our ChalkBoard DSL uses type functions.
data Board a
= PrimFun ((R,R) -> O a)
| Trans Trans (Board a)
| Join (Board a) (Joiner a) (Board a)
| Extras (BoardExtra a)
data Trans = Move (R,R) | Scale (R,R) | ...
data family BoardExtra a
data instance BoardExtra Bool = Polygon_Bool [(R,R)]
data instance BoardExtra RGB = Fmap_Bool_RGB (O Bool -> O RGB) (Board Bool)
| ...
...
We explicitly call out all supported fmaps conversions in the typefunctions (for now).
This is a significant limitation to DSL(requires transliteration from original ChalkBoard code)
We can observe the functions inside these fmaps because they havethe type O a -> O b, where O is an observable value.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 13 / 19
Observable Values
An observable value is a value that can be both interpreted and observed.
data O a = O a E
newtype E = E (Entity E)
data Entity s = Entity String [Expr s]data Expr s = Expr s
| Lit R
We use IO-based observable sharing tricks from Haskell09 here.
Overloading arithmetic.
Use combinators for pattern matching O values.
lifted versions of red, green, etc.
Small first order language can be extracted.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 14 / 19
Compiling to CBIR (continued)
In the compiler
We think of Board Bool as a sequence of imperative True setters,over a False Buffer.
We think of Board RGB as a sequence of (color drawing) imperativesetters, over a specific colored Buffer.
. . .
over becomes sequencing (with conditions).
Transformations like rotate and scale are independent of theseinterpretations, and flow through to the primitives.
We lose the ability to compile efficiently specifications like
circle :: Board Boolcircle = (\ (x,y) -> x*x + y*y
ChalkBoard IR Virtual Machine
We have written two engines for interpreting CBIR.
Reference implementation
Really slow.Uses ChalkBoard itself.
OpenGL based optimized implementation.
Much faster animation within reach.CBIR instructions are sent to an MVar.Instructions are then translated on the fly into a series of efficientOpenGL commands.The engine make appropriate choices about buffer implementation.Various hooks for debugging and monitoring.Basic animation support (frequent redraw).
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 16 / 19
OpenGL pipeline
CommandsVertex
OperationPrimitiveAssembly Rasterization
FragmentOperation
FrameBuffer
TextureMemory
// // // // //OO
We use (the widely supported) Frame Buffer Objects.
This allows the frame buffer to also be a texture map.
This makes it much easier to implement implement splat.
We implement a backwards compatible mode, where a singleframe buffer is re-used over and over, and pixels are copied intotextures automatically.
We also compensate for the built-in alpha blending, making theact of splating associative.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 17 / 19
Status
We have basic graphicals on the screen, via our back end.
We can import images as a Board.
We can also write out popular file formats.
We can write to the screen in real time.
The generation is fast enough for small animations.
Ken Burns style image panning and zooming works.Sprites (.png images)Wiggling lines
Have a small number of libraries on top of ChalkBoard.
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 18 / 19
Future Work and Open Programs
Improve compiler, via algebraic transformations
Improve (or understand) fmap function capture.
Use OpenGL texture functions to implement applicative functors.
OpenCL and GPUs.
Explore timezones inside FRP.
type UI = Doublelifetime :: Behavior UI -- 0 to 1lifetime = Behaviour (\ t -> t)
Fonts, and other libraries.
Movie import (and export).
Release ChalkBoard on hackage.haskell.org
Questions?
Andy Gill and Kevin Matlage Chalkboard: Mapping Functions to Polygons September 23, 2009 19 / 19