July, 2007 GCB SAGE workshop

SAGE Development

July, 2007 GCB SAGE workshop

Architecture …• Free Space

manager provides central control between apps, UI and system

• SAGE applications send their streams directly to the display nodes

July, 2007 GCB SAGE workshop

Writing/Porting SAGE applications

• Modify an existing application. – Source code should be open.– Source should be in C++.– If source code is not easily modifiable we can

try using shared memory. Requires a separate C++ program to read the shared memory and send pixels to SAGE

• Write an application from scratch– Cleanest solution

July, 2007 GCB SAGE workshop

Serial vs Distributed Apps

• Simple SAGE apps run on a single machine. Have one frame buffer– E.g. atlantis

• Distributed SAGE apps can run across multiple machines (cluster). Have many frame-buffers across multiple machines.– E.g. MagicCarpet

July, 2007 GCB SAGE workshop

Distributed Apps

• Use MPI or simple ssh to spawn instances across machines

• Responsible for internal syncing before doing a collective sageSwapBuffer()

• MagicCarpet

• JuxtaView

July, 2007 GCB SAGE workshop

OpenGL Code SAGEified// headers for SAGE#include "sail.h"#include "misc.h" void reshape(int width, int height){       // blah blah} 

//display functionvoid redraw(void){       // draw code       glReadPixels(0, 0, winWidth, winHeight,

GL_RGB,       GL_UNSIGNED_BYTE, rgbBuffer);       sageInf.swapBuffer((void *)rgbBuffer);        glutSwapBuffers(); • }•  

int main(int argc, char *argv[]){     // application code      glPixelStorei( GL_UNPACK_ALIGNMENT, 1 );        sailConfig scfg;      scfg.cfgFile = "sage.conf";      scfg.appName = "render";      scfg.rank = 0;      scfg.ip = argv[2];      scfg.resX = 400;  scfg.resY = 400;       sageRect renderImageMap;      renderImageMap.left = 0.0;      renderImageMap.right = 1.0;      renderImageMap.bottom = 0.0;      renderImageMap.top = 1.0;       scfg.imageMap = renderImageMap;      scfg.colorDepth = 24;      scfg.pixFmt = TVPIXFMT_888;      scfg.rowOrd = BOTTOM_TO_TOP;                         sageInf.init(scfg);      cout << "sail initialized " << endl;      glutMainLoop();}

July, 2007 GCB SAGE workshop

A Native SAGE App• // for SAGE• #include <sail.h>• #include <misc.h>•  • // SAGE Stuff• int winWidth, winHeight;• sail sageInf; // sail object•  • // initialize SAGE• sailConfig scfg;• scfg.cfgFile = "sage.conf";• scfg.appName = “myApp"; 

• scfg.rank = rank;• scfg.ip = NULL;•  • scfg.resX = (extent.w / extent.zoom);• scfg.resY = (extent.h / extent.zoom);•  • sageRect renderImageMap;• renderImageMap.left = 0.0;• renderImageMap.right = 1.0;• renderImageMap.bottom = 0.0;• renderImageMap.top = 1.0;•  

•                    

scfg.imageMap = renderImageMap;scfg.colorDepth = 24;scfg.pixFmt = TVPIXFMT_888;scfg.rowOrd = TOP_TO_BOTTOM;

sageInf.init(scfg);  // Main while loopwhile (!quit){ // Generate pixels and copy into ‘buffer’     ....       // swap buffer       sageInf.swapBuffer(buffer);}

July, 2007 GCB SAGE workshop

TCP vs UDP• Protocol to be used can be specified in

sage.conf– No need to recompile code

• TCP works well locally (within the cluster backplane)– Suffers from performance hits over long fat networks

• UDP is fast but missing packets lead to missing pixels.

• New experimental protocols are used for testing

July, 2007 GCB SAGE workshop

Troubleshooting And Debugging

• Failure to Init ports– fsManager already running in the background

• Failure of SAGE UI to start– Make sure that fsManager is running. Then

make sure that sageui.conf contains the correct IP address and port of the machine that runs fsManager.

July, 2007 GCB SAGE workshop

Troubleshooting and Debugging

• A socket error appears in the console window– It is likely that something caused fsManager to stop

working. Close SAGE UI. And kill sage. Wait for a a few seconds before trying to restart SAGE.  

• Noticeable delay in SAGE UI actions and updates on the SAGE display– The machine running fsManager may be somewhat

loaded and not be as responsive to SAGE UI requests as one would like it to be.

July, 2007 GCB SAGE workshop

The New SAGE• SAGE 2.0• The main differences

– Pixel block streaming– Every app has its own configuration

• UI remains same. User experience is same• Looks better and more stable

PLAY sage-vga.mov

July, 2007 GCB SAGE workshop

What is pixel block streaming?

• SAGE 2.x supports pixel blocks– Frame is broken into tiny blocks– Sets of blocks sent to appropriate tiles– Efficient since the application can send only

the updated blocks and not the entire frame• Good for overlapped windows

July, 2007 GCB SAGE workshop

Developing SAGE application on Windows

July, 2007 GCB SAGE workshop

The Goal

July, 2007 GCB SAGE workshop

Bandwidth (BW) Requirementin Megabits per sec. (Mbps)

• Calculate bandwidth requirements for streaming apps.– Pixels x bpp (bits per pixel) x fps (frames per sec.)

• Image Width: 640, Image Height: 480 = 307200 pixels• Image format: RGB (Red, Green, Blue: 3 colors)• Color Depth: 8 bits per color = 24 bits per pixel• Image frame rate: 30 fps• Bandwidth: 307200 x 24 x 30 = 221Mbps

– If BW > 1Gbps, expect slower frame rate.

July, 2007 GCB SAGE workshop

DirectShow SAGE App• Windows application

– Uses Win32 version of SAIL (sail.lib)

• Streams any video source seen by DirectX to SAGE– Web cams– Files (avi, dv, wmv .. Etc)– HD cameras– Frame grabbers

July, 2007 GCB SAGE workshop

DirectShow SAIL/SAGE AppDSSAGE

• What is DirectShow?• What is COM?• Graph Editor for

visual creation and test video pipeline

• Easy and intuitive for programming

• Not user friendly– Needs a better UI– Just for development

July, 2007 GCB SAGE workshop

DSSAGE: The SAIL/SAGE COM Object• Standard query-able interface • Most Multimedia devices (including files) come as a

COM objects under MS Windows• Can communicate with a device without prior knowledge

of video format. Interface is exposed and agreed upon during initialization

July, 2007 GCB SAGE workshop

DSAGE Main Code StructureDSSAGE :: SetMediaType(){

// Agree with Media type (video format)// e.g: Greylevel, RGB8, RGB555, YUV422, etc.// Populate the SAIL config structure….// Do SAIL/SAGE init with this input media type….

} DSSAGE :: send(buffer){

// Get frame buffer pointer from input DShow COMpImediaSample->GetPointer(&buffer);

    // Using SAIL swap buffer to SAGEsageInf.swapBuffer(buffer);

}

                   

July, 2007 GCB SAGE workshop

Problems Faced• SAGE is developed for Linux and there is

an alpha port for Windows• Linux assumes the program runs in a

terminal• Debugging under Windows is not intuitive.

– Not enough feedback from SAIL– No Console to see messages unless

redirected– Console messages not clear– Configuration files have to be adjusted for

every application added to SAGE. No central registry support.

July, 2007 GCB SAGE workshop

Problem Faced (cont.)

• SAGE 1.4b is unstable– Looking at SAGE 2.x integration with ROCKS– SAIL 3.0 promises to handle error messages

in a better way

• Still is developed mainly for Linux then ported to Windows.

July, 2007 GCB SAGE workshop

Online SAGE resources

• For documentation, papers, latest stable releases:

http://www.evl.uic.edu/cavern/sage/

July, 2007 GCB SAGE workshop

Q and A ?