1. July, 2007 GCB SAGE workshop SAGE Development

2. July, 2007 GCB SAGE workshop Architecture … Free Space manager provides central control between apps, UI and system SAGE applications send their streams directly to the display nodes

3. July, 2007 GCB SAGE workshop Writing/Porting SAGE applications Modify an existing application. –Source code should be open. –Source should be in C++. –If source code is not easily modifiable we can try using shared memory. Requires a separate C++ program to read the shared memory and send pixels to SAGE Write an application from scratch –Cleanest solution

4. July, 2007 GCB SAGE workshop Serial vs Distributed Apps Simple SAGE apps run on a single machine. Have one frame buffer –E.g. atlantis Distributed SAGE apps can run across multiple machines (cluster). Have many frame-buffers across multiple machines. –E.g. MagicCarpet

5. July, 2007 GCB SAGE workshop Distributed Apps Use MPI or simple ssh to spawn instances across machines Responsible for internal syncing before doing a collective sageSwapBuffer() MagicCarpet JuxtaView

6. July, 2007 GCB SAGE workshop OpenGL Code SAGEified // headers for SAGE #include "sail.h" #include "misc.h" void reshape(int width, int height) { // blah blah } //display function void redraw(void) { // draw code glReadPixels(0, 0, winWidth, winHeight, GL_RGB, GL_UNSIGNED_BYTE, rgbBuffer); sageInf.swapBuffer((void *)rgbBuffer); glutSwapBuffers(); } int main(int argc, char *argv[]) { // application code glPixelStorei( GL_UNPACK_ALIGNMENT, 1 ); sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = "render"; scfg.rank = 0; scfg.ip = argv[2]; scfg.resX = 400; scfg.resY = 400; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg); cout << "sail initialized " << endl; glutMainLoop(); }

7. July, 2007 GCB SAGE workshop A Native SAGE App // for SAGE #include // SAGE Stuff int winWidth, winHeight; sail sageInf; // sail object // initialize SAGE sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = “myApp"; scfg.rank = rank; scfg.ip = NULL; scfg.resX = (extent.w / extent.zoom); scfg.resY = (extent.h / extent.zoom); sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = TOP_TO_BOTTOM; sageInf.init(scfg); // Main while loop while (!quit) { // Generate pixels and copy into ‘buffer’.... // swap buffer sageInf.swapBuffer(buffer); }

8. July, 2007 GCB SAGE workshop TCP vs UDP Protocol to be used can be specified in sage.conf –No need to recompile code TCP works well locally (within the cluster backplane) –Suffers from performance hits over long fat networks UDP is fast but missing packets lead to missing pixels. New experimental protocols are used for testing

9. July, 2007 GCB SAGE workshop Troubleshooting And Debugging Failure to Init ports –fsManager already running in the background Failure of SAGE UI to start –Make sure that fsManager is running. Then make sure that sageui.conf contains the correct IP address and port of the machine that runs fsManager.

10. July, 2007 GCB SAGE workshop Troubleshooting and Debugging A socket error appears in the console window –It is likely that something caused fsManager to stop working. Close SAGE UI. And kill sage. Wait for a a few seconds before trying to restart SAGE. Noticeable delay in SAGE UI actions and updates on the SAGE display –The machine running fsManager may be somewhat loaded and not be as responsive to SAGE UI requests as one would like it to be.

11. July, 2007 GCB SAGE workshop The New SAGE SAGE 2.0 The main differences –Pixel block streaming –Every app has its own configuration UI remains same. User experience is same Looks better and more stable PLAY sage-vga.mov

12. July, 2007 GCB SAGE workshop What is pixel block streaming? SAGE 2.x supports pixel blocks –Frame is broken into tiny blocks –Sets of blocks sent to appropriate tiles –Efficient since the application can send only the updated blocks and not the entire frame Good for overlapped windows

13. July, 2007 GCB SAGE workshop Developing SAGE application on Windows

14. July, 2007 GCB SAGE workshop The Goal

15. July, 2007 GCB SAGE workshop Bandwidth (BW) Requirement in Megabits per sec. (Mbps) Calculate bandwidth requirements for streaming apps. –Pixels x bpp (bits per pixel) x fps (frames per sec.) Image Width: 640, Image Height: 480 = 307200 pixels Image format: RGB (Red, Green, Blue: 3 colors) Color Depth: 8 bits per color = 24 bits per pixel Image frame rate: 30 fps Bandwidth: 307200 x 24 x 30 = 221Mbps –If BW > 1Gbps, expect slower frame rate.

16. July, 2007 GCB SAGE workshop DirectShow SAGE App Windows application –Uses Win32 version of SAIL (sail.lib) Streams any video source seen by DirectX to SAGE –Web cams –Files (avi, dv, wmv.. Etc) –HD cameras –Frame grabbers

17. July, 2007 GCB SAGE workshop DirectShow SAIL/SAGE App DSSAGE What is DirectShow? What is COM? Graph Editor for visual creation and test video pipeline Easy and intuitive for programming Not user friendly –Needs a better UI –Just for development

18. July, 2007 GCB SAGE workshop DSSAGE: The SAIL/SAGE COM Object Standard query-able interface Most Multimedia devices (including files) come as a COM objects under MS Windows Can communicate with a device without prior knowledge of video format. Interface is exposed and agreed upon during initialization

19. July, 2007 GCB SAGE workshop DSAGE Main Code Structure DSSAGE :: SetMediaType(){ // Agree with Media type (video format) // e.g: Greylevel, RGB8, RGB555, YUV422, etc. // Populate the SAIL config structure …. // Do SAIL/SAGE init with this input media type …. } DSSAGE :: send(buffer) { // Get frame buffer pointer from input DShow COM pImediaSample->GetPointer(&buffer); // Using SAIL swap buffer to SAGE sageInf.swapBuffer(buffer); }

20. July, 2007 GCB SAGE workshop Problems Faced SAGE is developed for Linux and there is an alpha port for Windows Linux assumes the program runs in a terminal Debugging under Windows is not intuitive. –Not enough feedback from SAIL –No Console to see messages unless redirected –Console messages not clear –Configuration files have to be adjusted for every application added to SAGE. No central registry support.

21. July, 2007 GCB SAGE workshop Problem Faced (cont.) SAGE 1.4b is unstable –Looking at SAGE 2.x integration with ROCKS –SAIL 3.0 promises to handle error messages in a better way Still is developed mainly for Linux then ported to Windows.

22. July, 2007 GCB SAGE workshop Online SAGE resources For documentation, papers, latest stable releases: http://www.evl.uic.edu/cavern/sage/

23. July, 2007 GCB SAGE workshop Q and A ?