Experiences with 2-D and 3-D Mathematical Plots on the Java™ Platform David ClayworthMaplesoft

2

What you will learn> Techniques for writing software that plots

mathematical and scientific data> How to apply Java tools to these tasks:

Java2D for 2D graphs JOGL for 3D graphs

> How to make best use of these toolkits, especially JOGL

3

> Maple is a symbolic math application: Performs math on symbols as well as numbers A huge number of mathematical operations

> User interface is all Java (~1M lines) Outputs math as mathematicians, scientists or

engineers expect it> Previous toolkit lacked speed, quality,

controllability and economy.> Two stage rewrite:2D and then 3D

History

4

Demo> Maple before

5

Requirements> New Features

Interactive annotation Mathematical labels part of the plot Speed and memory improvements Try to use few Java components

> Notable Existing Features Plots embeddable in a worksheet Export to bitmap and vector Run on Windows, Linux, Mac and Solaris

6

DesignOverview

> Maple GUI uses Model-View-Controller> Each symbol, plot, component, axis etc. has a model> Each model has a view> Model update → view layout → view draw> Mapping from plot to pixel coordinates during layout

ViewModel

Controller

7

DesignPlot Atoms> Atoms are small objects that store elements in a

form that is quick to draw> They may store pixel positions, or a Shape object,

or a symbol image and location…> Created at layout time and stored in the MVC view> To draw itself a component just draws all its atoms

8

DesignPlot Atoms interface Atom { void draw(Graphics g);}

class PolygonAtom implements Atom { GeneralPath poly; PolygonAtom(float x[],float y[]) { // create the path… } void draw(Graphics g) { ((Graphics2D)g).fill(poly); }}

9

DesignMath in Labels> Mathematical expressions are drawn directly by

positioning the view within the plot> Annotations use an existing package which draws

over the plot with Java2D

10

Optimizations:float v int coordinates> drawPolyline(int[],int[],int) v draw(Shape)> In theory ints are faster but floats more accurate> You need float coordinates for printing or vector

graphics (e.g. Postscript)> Considered using int-based calls on screen and

float-based for print> Ended using floats throughout: the differences in

both accuracy and performance were small (except for printing).

11

Optimizations:Sprites> Rendering performance was acceptable,

except when drawing many symbols> Sprites are small images which can be drawn to

the screen instead of lines and shapes> Faster to draw an image than even a simple shape> Create a Sprite object which holds an image for

the symbol> Sprites are indexed by symbol, size and colour.

12

Optimizations:Sprites

Backgrounds of Sprites must be transparentYou need to make sure that symbols are symmetric

13

2D PlotsThe results> The 2D plots were released in 2007.> Speeded up by a factor of 7> Memory used less than 1/10th

> The new features were appreciated by customers> Spurred demand for the same features in 3D

14

Requirements3D Project> New Features

Interactive annotation, drawn in Java2D over 3D Mathematical labels part of the plot, embedded at

3D positions Other improvements as before

> Notable Existing Features as before

15

ToolkitsCandidate toolkits for 3D> The choices:

Java3D: a complete scenegraph tool written in Java JOGL: a thin Java layer over OpenGL LWJGL: a Java layer over OpenGL designed for

games A game engine, such as JMonkeyEngine

> LWJGL eliminated as it uses a single window.> Many game engines have LWJGL underneath

16

ToolkitsJOGL v Java3D> Java3D advantages:

A complete OO toolkit Simpler to learn than OpenGL

> JOGL advantages: If you know OpenGL you know JOGL Virtually full OpenGL functionality Known portability Wide use Doesn’t impose it’s own framework

17

ToolkitsJOGL> GLCanvas (heavyweight AWT) or GLJPanel

(lightweight Swing) accelerated components> GLEventListener is attached to the component

render the scene> GL object passed to the listener provides Java

equivalents for OpenGL calls.> You can (and must) use OpenGL documentation.> Each view object has a method to draw itself with

GL calls.

18

DesignMixing 2D labels into 3D> Labels are positioned in 3D space> Math is drawn into an image with a transparent

background> Set a raster position in 3D, then shift it in 2D to get

the alignment right> Image drawn into the scene with JOGL > gl.glRasterPos3fv(arrayXYZ, 0);gl.glBitmap(0, 0, 0, 0,xOff, yOff, null);gl.glDrawPixels(w,h,…, image);

19

DesignMixing 2D labels into 3D

What we expect What we get (without alpha-clipping)

20

DesignMixing 2D labels into 3D (cont.)> When OpenGL writes shape it sets the ‘depth’ at

which it is written (depth buffer)> Subsequent shapes at a greater depth are not

written i.e. surfaces behind a surface aren’t seen.> But the depth buffer is written even for

transparent pixels> We have to use two passes:

Write solid pixels and set the depth buffer Write transparent pixels without the depth buffer

21

DesignMixing 2D labels into 3D (cont.)

gl.glDepthMask(true);gl.glAlphaFunc(GL.GL_GEQUAL,alpha);drawAllComponents();gl.glAlphaFunc(GL.GL_LESS, alpha);gl.glDepthMask(false);drawAllComponents();> OpenGL Programming Guide “Alpha Test”

(p477 Sixth Edition)

22

Rendering MechanismLimitations of GLJPanel> Started with one GLJPanel per plot> A Maple document may have many plots.> But GLJPanels take resources (video RAM)> After creating so many they stop rendering > JOGL Issue 370

23

Rendering MechanismReusing a GLJPanel> Reuse GLJPanel, stealing them from plots that are

offscreen. > But you can’t reposition and draw a GLJPanel in

the same event> There are other problems with writing plots for

export.> A complex and impractical solution

24

Rendering Mechanism GLPBuffers> GLPBuffers allow for drawing hardware

accelerated graphics offscreen> Render 3D to the GLPBuffer, then the image to

the screen> Now easy to draw 2D annotations over the PBuffer

image> We have fewer components> We still need to reuse GLPBuffers since they use

video memory

25

Rendering Mechanism Creating a GLPBuffer GLDrawableFactory factory = GLDrawableFactory.getFactory();

…GLPbuffer buf = factory.createGLPbuffer(…, width,height,…);

…GLContext glContext = buf.createContext(null);

26

Rendering Mechanism Drawing to a GLPBuffervoid draw(Graphics g) { try { glContext.makeCurrent(); GL gl = glContext.getGL(); // make GL calls… BufferedImage img = Screenshot.readToBufferedImage(w,h); g.drawImage(img, x, y null); … } finally { if (GLContext.getCurrent()==glContext) {glContext.release();} }}

27

Rendering Mechanism GLPBuffers (cont)> Not all displays support GLPBuffers: GLDrawableFactory. canCreateGLPbuffer();

> Creation can still fail, because they need video resources

> Catch GLException when creating; and RuntimeException because of JOGL bugs

> Some performance cost

28

Rendering Mechanism Offscreen drawable> There is also a unaccelerated offscreen drawable> This is outside the published interface: it is used

by GLJPanel.> See GLJPanel source

GLDrawableImpl offscreenDrawable = GLDrawableFactoryImpl.getFactoryImpl(). createOffscreenDrawable(glCap,…);

29

Platform issues:> Not all machine configurations will work with JOGL

‘out of the box’.> Working OpenGL doesn’t mean working JOGL> Need to call glGetError() (or use DebugGL)> Catch GLException and process it> Check your display capabilities with GLDrawable.getChosenGLCapabilities();

> Don’t make GL calls when your GLContext is not current

30

Platform issues:Mac> None

31

Platform issues:Windows> Few> ATI drivers: need at least v8.231 (Catalyst 8)> Remote Desktop Connection and some laptop

external monitors: May not be able to use the 2-pass transparency

algorithm Check the supported alpha bits

> Some drivers draw bad plots

32

Platform issues:Linux> Some> Many default Linux installs don’t work with JOGL> Proprietary video drivers needed

ATI Catalyst 8.12 nVidia version 177

> Some drivers crash

33

Demo> The new plots

34

Summary:2D > Favour precision over speed> Store presentation data close to the Java drawing

format> Use sprites for symbols> Java2D provides all you need for this task

35

Summary:3D > JOGL:

Provides all your 3D needs Is good if you already have structure May not be best if you need to create structure

> Programming JOGL is programming OpenGL> Write math as images directly to the 3D scene

Requires transparency handling> Manage video drivers, esp. Linux

36

Summary:3D (cont)> PBuffers:

Simplify 2D/3D mixing Allow many plots in an app Have little performance cost Reduce component count Need to go outside the spec for unaccelerated

images

37

Resources:> Jumping into JOGL:

http://today.java.net/pub/a/today/2003/09/11/jogl2d.html/> JOGL documentatation:

http://download.java.net/media/jogl/builds/nightly/javadoc_public/overview-summary.html http://www.opengl.org/sdk/docs/man/

> JOGL forum: http://www.javagaming.org/index.php/board,25.0.html

> PBuffers: http://today.java.net/pub/a/today/2008/10/30/integrating-glpbuffer-and-graphics2d.html

> OpenGL Programming Guide

38

David ClayworthSenior GUI DeveloperMaplesoft

www.maplesoft.com