July, 2007 GCB SAGE workshop SAGE – Architecture and API

July, 2007 GCB SAGE workshop

SAGE – Architecture and API


Early Tile-Display Software: JuxtaView

• Visualization of 2D datasets with predictive pre-fetching.

• Slow user interaction and inefficient data access

NCMIR – microscopy (2800x4000 24 layers)

Scripps – Bathymetry and digital elevation


Early Tile-Display Software: MagicCarpet

• Smart mip-mapped 2D data access

• Very interactive

• Needs datasets to be pre-processed


Early Tile-Display Software: Vol-a-Tile

• Visualization of 3D datasets• Transfer function and UI


The Sage Pixel Streaming Architecture• Stream uncompressed pixels generated by apps to display walls• Designers tried to achieve a desktop-like environment for tile displays


Adaptive Rendering

• Adapt to the display client– Laptop, high-resolution monitor, tiled display

• Rendering capabilities– Pixel drawing, polygon rendering– Network capacity

• Adapt to output resolution– “Resize” and “move” events– Pixel up-scaling or down-sampling– Increase rendering resolution


Pixel Sources

• Visualization applications– Software rendering– Hardware rendering

• Legacy applications– RDP, ARD, VNC– TeraVision (hardware capture)

• Video streaming


Working in Display-Rich Environments

Remote laptop

High-resolution maps

Live video feeds

3D surface rendering

Volume Rendering

Remote sensing


SAGE Components

• Capture the pixels

• Partitioning of the images

• Routing the pixels

• Layout on the display

• User interaction


Basic Pipeline

• With SAGE we separate the rendering and display.

•Rendering machine is connected to display driven by thin client by fast network


Pipeline Gets Complicated…•More complicated scenarios

•Rendering machine sends pixels to multiple displays.

•There are multiple rendering machines (cluster)

•Assume that the rendering is done remotely across big fat networks

•Provide source (rendering software) with user interaction feedback


Ideal Scenario


Why Raw Pixel Streaming

• Pixels are transmitted raw (without compression) because networks are getting faster/cheaper.

• We don’t spend time compressing and decompressing on the CPU

• New SAGE supports DXT compression


Related Work

• WireGL / Chromium– Streams OpenGL primitives– Slow. User interactive apps suffer.– Not designed for fat long distance networks– Complex configuration

• XDMX– Highly researched. Very flexible.– Streams X primitives– Slow– Not designed for fat long distance networks– No support for mullions


Related Work … contd.

• IBM’s SGE– Hardware switched solution– Streams raw pixels– Not scalable– Expensive and legacy.

• TeraVision– Hardware and open source software solution– Streams raw pixels from any video source– Scalable– Expensive– No support for dynamic routing.


Architecture …• Free Space

manager provides central control between apps, UI and system

• SAGE applications send their streams directly to the display nodes


Free Space Manager

• Central control unit of SAGE.

• Setups the entire system based on configuration files

• Communicates with UI clients

• Window movement and resize require messages to be passed between the Free Space Manager and apps

• Sends SAGE status messages to apps.


SAGE Application Interface Library (SAIL)

• Library for writing SAGE applications

• Interfaces with the Free Space Manager to receive and send system messages

• When a SAGE application gives SAIL a new video frame, it sends pixels directly to the appropriate portions of the display tiles.


More Details• Frame syncing

– Every frame displayed within SAGE is synced with a TCP message.

– So 60 fps = 60 messages per second

• Windowing system– FrenchWindows

• Collaborative features– UI can be launched and used by multiple users at the

same time– New SAGE (v2.x) provides multiple mouse pointers.


Basic Code to ‘SAGEify’ an Existing AppsailConfig scfg; // Setup SAGE structure

scfg.cfgFile = "sage.conf";scfg.appName = “myapp";scfg.rank = 0;

sageRect renderImageMap;renderImageMap.left = 0.0;renderImageMap.right = 1.0;renderImageMap.bottom = 0.0;renderImageMap.top = 1.0;scfg.imageMap = renderImageMap;scfg.colorDepth = 24;scfg.pixFmt = TVPIXFMT_888;scfg.rowOrd = BOTTOM_TO_TOP;

sageInf.init(scfg); // Init SAGEwhile (1) {

.. <render video frame> ..

sageInf.swapBuffer( rgbBuffer ); // send rendered video frame to SAGE }


SAGE UI

• Connects to SAGE over the network.

• Main window is divided into two sections: – top which represents

the display the UI is connected to

– bottom which holds information about the applications currently running.


SAGE UI

• Resize or move application windows like on a desktop

• Change ‘depth’ order of windows


SAGE UI

• Maximize the windows

• Preserve (or not) the aspect ratio of rendering app.


SAGE UI

• Performance monitoring built in

• Bandwidth (Network and Display)

• FPS (Network and Display)

• Number of nodes used by an app


SAGE UI

• Performance graphs

• Built in logging – Global option

accessible via ‘Performance’ menu


SAGE UI

• Session recording and playback


SAGE UI

• Can connect to multiple displays at same time


SAGE UI

• Chat features as a back channel


SAGE Application Model

• Application– Rendering component– User interface component

• Layout on the screen controlled by the FreeSpace Manager

• Pixels captured by SAIL– SAGE Application Interface Library

• Streaming network protocol


SAIL – The SAGE Application Interface Library

• The SAGE programming API.

• Simple interface allows you to specify a SAGE displayand connect to it

• glSwapBuffer like call


OpenGL App SAGEified// headers for SAGE#include "sail.h"#include "misc.h" void reshape(int width, int height){ // blah blah}

//display functionvoid redraw(void){ // draw code glReadPixels(0, 0, winWidth, winHeight,

GL_RGB, GL_UNSIGNED_BYTE, rgbBuffer); sageInf.swapBuffer((void *)rgbBuffer); glutSwapBuffers(); • }•

int main(int argc, char *argv[]){ // application code glPixelStorei( GL_UNPACK_ALIGNMENT, 1 ); sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = "render"; scfg.rank = 0; scfg.ip = argv[2]; scfg.resX = 400; scfg.resY = 400; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg); cout << "sail initialized " << endl; glutMainLoop();}


Write a Native SAGE App• // for SAGE• #include <sail.h>• #include <misc.h>• • // SAGE Stuff• int winWidth, winHeight;• sail sageInf; // sail object• • // initialize SAGE• sailConfig scfg;• scfg.cfgFile = "sage.conf";• scfg.appName = “myApp";

• scfg.rank = rank;• scfg.ip = NULL;• • scfg.resX = (extent.w / extent.zoom);• scfg.resY = (extent.h / extent.zoom);• • sageRect renderImageMap;• renderImageMap.left = 0.0;• renderImageMap.right = 1.0;• renderImageMap.bottom = 0.0;• renderImageMap.top = 1.0;•

•

scfg.imageMap = renderImageMap;scfg.colorDepth = 24;scfg.pixFmt = TVPIXFMT_888;scfg.rowOrd = TOP_TO_BOTTOM;

sageInf.init(scfg); // create zoom adjusted image bufferbuffer = new unsigned char[(extent.w / extent.zoom) * (extent.h / extent.zoom) * (3)];// clear the zoom adjusted image buffermemset(buffer,0,(extent.w / extent.zoom)*(extent.h / extent.zoom)* 3); // Main while loopwhile (!quit){ // Generate pixels and copy into ‘buffer’ // swap buffer sageInf.swapBuffer(buffer);}


Building and Installing SAGEFrom Source

– Dependencies– Compilation– Configuration


SAGE Dependencies

• QUANTA 0.4 (www.evl.uic.edu/cavern/quanta)• Readline (runtime and development packages):

from GNU project, http://cnswww.cns.cwru.edu/~chet/readline/rltop.html

• SDL libraries for the display side, http://www.libsdl.org

• Some test programs need GLUT to compile (atlantis, atlantis-mpi, ...), http://freeglut.sourceforge.net

http://www.evl.uic.edu/cavern/quanta

http://www.evl.uic.edu/cavern/quanta

http://cnswww.cns.cwru.edu/%7Echet/readline/rltop.html

http://cnswww.cns.cwru.edu/%7Echet/readline/rltop.html

http://www.libsdl.org/

http://www.libsdl.org/

http://freeglut.sourceforge.net/

http://freeglut.sourceforge.net/


Compiling SAGE

• Modifying makefiles• Open the appropriate Makefile in the "sage/src"

directory and edit the following lines:– Set QUANTA_DIR to the directory where you built

QUANTA – Set the appropriate flag for your shell preference:– If you use bash, set MYFLAGS –DSAGE_BASH– If you use csh, set MYFLAGS –DSAGE_CSH


Compiling SAGE .. Contd.

• Execute make install in the "sage/src" directory.

• Execute make install in the "sage/src/nwProtocol" directory.

• Execute make install in sage/app and sage/app/atlantis.


Edit Configuration Files

• Go to sage/bin

• Edit fsManager.conf

• Edit sage.conf

• Edit stdtile.conf (or whatever tile config file specified in sage.conf)


Sample sage.confdisplayBinDir SAGE_DIR/binappBinDir SAGE_DIR/bin

appList

render {configName localnodeNum 1 Init 100 100 1000 1000exec 127.0.0.1 render 0 127.0.0.1nwProtocol tvTcpModule.so}

atlantis {configName localnodeNum 1Init 100 100 1000 1000exec 127.0.0.1 atlantis 0 127.0.0.1nwProtocol tvTcpModule.soconfigName UDPnodeNum 1Init 100 100 1000 1000exec 127.0.0.1 atlantis 0 127.0.0.1nwProtocol tvUdpModule.so}

endList


Sample sage.conf … contd.tileConfiguration stdtile.conf

receiverBaseSyncPort 12000receiverBufNum 20receiverStreamPort 21000fullScreen 1

sailBaseSyncPort 11000


Sample stdtile.conf

• For 1 node only :

TileDisplayDimensions 1 1 Mullions 0.625 0.625 0.625 0.625Resolution 1280 1024PPI 90Machines 1

DisplayNode Name localhostIP 127.0.0.1Monitors 1 (0,0)


Stdtile.conf For Multiple NodesTileDisplay

Dimensions 5 3 Mullions 0.625 0.625 0.625 0.625Resolution 1280 1024PPI 90Machines 15

DisplayNode Name yorda1-10IP 10.0.8.121Monitors 1 (0,2)




•







Documents

July, 2007 GCB SAGE workshop SAGE – Architecture and API