1. July, 2007 GCB SAGE workshop SAGE – Architecture and API

2. July, 2007 GCB SAGE workshop Early Tile-Display Software: JuxtaView Visualization of 2D datasets with predictive pre- fetching. Slow user interaction and inefficient data access NCMIR – microscopy (2800x4000 24 layers) Scripps – Bathymetry and digital elevation

3. July, 2007 GCB SAGE workshop Early Tile-Display Software: MagicCarpet Smart mip- mapped 2D data access Very interactive Needs datasets to be pre-processed

4. July, 2007 GCB SAGE workshop Early Tile-Display Software: Vol-a-Tile Visualization of 3D datasets Transfer function and UI

5. July, 2007 GCB SAGE workshop The Sage Pixel Streaming Architecture Stream uncompressed pixels generated by apps to display walls Designers tried to achieve a desktop-like environment for tile displays

6. July, 2007 GCB SAGE workshop Adaptive Rendering Adapt to the display client –Laptop, high-resolution monitor, tiled display Rendering capabilities –Pixel drawing, polygon rendering –Network capacity Adapt to output resolution –“Resize” and “move” events –Pixel up-scaling or down-sampling –Increase rendering resolution

7. July, 2007 GCB SAGE workshop Pixel Sources Visualization applications –Software rendering –Hardware rendering Legacy applications –RDP, ARD, VNC –TeraVision (hardware capture) Video streaming

8. July, 2007 GCB SAGE workshop Working in Display-Rich Environments Remote laptop High-resolution maps Live video feeds 3D surface rendering Volume Rendering Remote sensing

9. July, 2007 GCB SAGE workshop SAGE Components Capture the pixels Partitioning of the images Routing the pixels Layout on the display User interaction

10. July, 2007 GCB SAGE workshop Basic Pipeline With SAGE we separate the rendering and display. Rendering machine is connected to display driven by thin client by fast network

11. July, 2007 GCB SAGE workshop Pipeline Gets Complicated… More complicated scenarios Rendering machine sends pixels to multiple displays. There are multiple rendering machines (cluster) Assume that the rendering is done remotely across big fat networks Provide source (rendering software) with user interaction feedback

12. July, 2007 GCB SAGE workshop Ideal Scenario

13. July, 2007 GCB SAGE workshop Why Raw Pixel Streaming Pixels are transmitted raw (without compression) because networks are getting faster/cheaper. We don’t spend time compressing and decompressing on the CPU New SAGE supports DXT compression

14. July, 2007 GCB SAGE workshop Related Work WireGL / Chromium –Streams OpenGL primitives –Slow. User interactive apps suffer. –Not designed for fat long distance networks –Complex configuration XDMX –Highly researched. Very flexible. –Streams X primitives –Slow –Not designed for fat long distance networks –No support for mullions

15. July, 2007 GCB SAGE workshop Related Work … contd. IBM’s SGE –Hardware switched solution –Streams raw pixels –Not scalable –Expensive and legacy. TeraVision –Hardware and open source software solution –Streams raw pixels from any video source –Scalable –Expensive –No support for dynamic routing.

16. July, 2007 GCB SAGE workshop Architecture … Free Space manager provides central control between apps, UI and system SAGE applications send their streams directly to the display nodes

17. July, 2007 GCB SAGE workshop Free Space Manager Central control unit of SAGE. Setups the entire system based on configuration files Communicates with UI clients Window movement and resize require messages to be passed between the Free Space Manager and apps Sends SAGE status messages to apps.

18. July, 2007 GCB SAGE workshop SAGE Application Interface Library (SAIL) Library for writing SAGE applications Interfaces with the Free Space Manager to receive and send system messages When a SAGE application gives SAIL a new video frame, it sends pixels directly to the appropriate portions of the display tiles.

19. July, 2007 GCB SAGE workshop More Details Frame syncing –Every frame displayed within SAGE is synced with a TCP message. –So 60 fps = 60 messages per second Windowing system –FrenchWindows Collaborative features –UI can be launched and used by multiple users at the same time –New SAGE (v2.x) provides multiple mouse pointers.

20. July, 2007 GCB SAGE workshop Basic Code to ‘SAGEify’ an Existing App sailConfig scfg;// Setup SAGE structure scfg.cfgFile = "sage.conf"; scfg.appName = “myapp"; scfg.rank = 0; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg);// Init SAGE while (1) {.. sageInf.swapBuffer( rgbBuffer );// send rendered video frame to SAGE }

21. July, 2007 GCB SAGE workshop SAGE UI Connects to SAGE over the network. Main window is divided into two sections: –top which represents the display the UI is connected to –bottom which holds information about the applications currently running.

22. July, 2007 GCB SAGE workshop SAGE UI Resize or move application windows like on a desktop Change ‘depth’ order of windows

23. July, 2007 GCB SAGE workshop SAGE UI Maximize the windows Preserve (or not) the aspect ratio of rendering app.

24. July, 2007 GCB SAGE workshop SAGE UI Performance monitoring built in Bandwidth (Network and Display) FPS (Network and Display) Number of nodes used by an app

25. July, 2007 GCB SAGE workshop SAGE UI Performance graphs Built in logging –Global option accessible via ‘Performance’ menu

26. July, 2007 GCB SAGE workshop SAGE UI Session recording and playback

27. July, 2007 GCB SAGE workshop SAGE UI Can connect to multiple displays at same time

28. July, 2007 GCB SAGE workshop SAGE UI Chat features as a back channel

29. July, 2007 GCB SAGE workshop SAGE Application Model Application –Rendering component –User interface component Layout on the screen controlled by the FreeSpace Manager Pixels captured by SAIL –SAGE Application Interface Library Streaming network protocol

30. July, 2007 GCB SAGE workshop SAIL – The SAGE Application Interface Library The SAGE programming API. Simple interface allows you to specify a SAGE display and connect to it glSwapBuffer like call

31. July, 2007 GCB SAGE workshop OpenGL App SAGEified // headers for SAGE #include "sail.h" #include "misc.h" void reshape(int width, int height) { // blah blah } //display function void redraw(void) { // draw code glReadPixels(0, 0, winWidth, winHeight, GL_RGB, GL_UNSIGNED_BYTE, rgbBuffer); sageInf.swapBuffer((void *)rgbBuffer); glutSwapBuffers(); } int main(int argc, char *argv[]) { // application code glPixelStorei( GL_UNPACK_ALIGNMENT, 1 ); sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = "render"; scfg.rank = 0; scfg.ip = argv[2]; scfg.resX = 400; scfg.resY = 400; sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = BOTTOM_TO_TOP; sageInf.init(scfg); cout << "sail initialized " << endl; glutMainLoop(); }

32. July, 2007 GCB SAGE workshop Write a Native SAGE App // for SAGE #include // SAGE Stuff int winWidth, winHeight; sail sageInf; // sail object // initialize SAGE sailConfig scfg; scfg.cfgFile = "sage.conf"; scfg.appName = “myApp"; scfg.rank = rank; scfg.ip = NULL; scfg.resX = (extent.w / extent.zoom); scfg.resY = (extent.h / extent.zoom); sageRect renderImageMap; renderImageMap.left = 0.0; renderImageMap.right = 1.0; renderImageMap.bottom = 0.0; renderImageMap.top = 1.0; scfg.imageMap = renderImageMap; scfg.colorDepth = 24; scfg.pixFmt = TVPIXFMT_888; scfg.rowOrd = TOP_TO_BOTTOM; sageInf.init(scfg); // create zoom adjusted image buffer buffer = new unsigned char[(extent.w / extent.zoom) * (extent.h / extent.zoom) * (3)]; // clear the zoom adjusted image buffer memset(buffer,0,(extent.w / extent.zoom)*(extent.h / extent.zoom)* 3); // Main while loop while (!quit) { // Generate pixels and copy into ‘buffer’ // swap buffer sageInf.swapBuffer(buffer); }

33. July, 2007 GCB SAGE workshop Building and Installing SAGE From Source –Dependencies –Compilation –Configuration

34. July, 2007 GCB SAGE workshop SAGE Dependencies QUANTA 0.4 (www.evl.uic.edu/cavern/quanta)www.evl.uic.edu/cavern/quanta Readline (runtime and development packages): from GNU project, http://cnswww.cns.cwru.edu/~chet/readline/rl top.html http://cnswww.cns.cwru.edu/~chet/readline/rl top.html SDL libraries for the display side, http://www.libsdl.org http://www.libsdl.org Some test programs need GLUT to compile (atlantis, atlantis-mpi,...), http://freeglut.sourceforge.net http://freeglut.sourceforge.net

35. July, 2007 GCB SAGE workshop Compiling SAGE Modifying makefiles Open the appropriate Makefile in the "sage/src" directory and edit the following lines: –Set QUANTA_DIR to the directory where you built QUANTA –Set the appropriate flag for your shell preference: –If you use bash, set MYFLAGS –DSAGE_BASH –If you use csh, set MYFLAGS –DSAGE_CSH

36. July, 2007 GCB SAGE workshop Compiling SAGE.. Contd. Execute make install in the "sage/src" directory. Execute make install in the "sage/src/nwProtocol" directory. Execute make install in sage/app and sage/app/atlantis.

37. July, 2007 GCB SAGE workshop Edit Configuration Files Go to sage/bin Edit fsManager.conf Edit sage.conf Edit stdtile.conf (or whatever tile config file specified in sage.conf)

38. July, 2007 GCB SAGE workshop Sample sage.conf displayBinDir SAGE_DIR/bin appBinDir SAGE_DIR/bin appList render { configName local nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 render 0 127.0.0.1 nwProtocol tvTcpModule.so } atlantis { configName local nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 atlantis 0 127.0.0.1 nwProtocol tvTcpModule.so configName UDP nodeNum 1 Init 100 100 1000 1000 exec 127.0.0.1 atlantis 0 127.0.0.1 nwProtocol tvUdpModule.so } endList

39. July, 2007 GCB SAGE workshop Sample sage.conf … contd. tileConfiguration stdtile.conf receiverBaseSyncPort 12000 receiverBufNum20 receiverStreamPort21000 fullScreen1 sailBaseSyncPort11000

40. July, 2007 GCB SAGE workshop Sample stdtile.conf For 1 node only : TileDisplay Dimensions 1 1 Mullions 0.625 0.625 0.625 0.625 Resolution 1280 1024 PPI 90 Machines 1 DisplayNode Name localhost IP 127.0.0.1 Monitors 1 (0,0)

41. July, 2007 GCB SAGE workshop Stdtile.conf For Multiple Nodes TileDisplay Dimensions 5 3 Mullions 0.625 0.625 0.625 0.625 Resolution 1280 1024 PPI 90 Machines 15 DisplayNode Name yorda1-10 IP 10.0.8.121 Monitors 1 (0,2) DisplayNode Name yorda2-10 IP 10.0.8.122 Monitors 1 (0,1) DisplayNode Name yorda3-10 IP 10.0.8.123 Monitors 1 (0,0) DisplayNode Name yorda4-10 IP 10.0.8.124 Monitors 1 (1,2) DisplayNode Name yorda5-10 IP 10.0.8.125 Monitors 1 (1,1) DisplayNode Name yorda6-10 IP 10.0.8.126 Monitors 1 (1,0) DisplayNode Name yorda7-10 IP 10.0.8.127 Monitors 1 (2,2) DisplayNode Name yorda8-10 IP 10.0.8.128 Monitors 1 (2,1) DisplayNode Name yorda9-10 IP 10.0.8.129 Monitors 1 (2,0)

42. July, 2007 GCB SAGE workshop