1. Jens Krüger (for Tom Fogal)
What Tom’s Been Up To
Jens Krüger (for Tom Fogal)

2. ‘Global’ Tasks / Impact
Infrastructure projects
Efficient parallel rendering in VisIt
“Opening” VisIt: process, community outreach
On the VACET side, everything I do basically boils down to one of two things: making VisIt’s distributed memory rendering (“scalable rendering”) feature more responsive, and various efforts to make the VisIt development process better + engage the community.
I also touch on some VisTrails stuff now and again, but I’m hesitant to claim as much because I feel like I’ve done so little.

3. Past “Future” Work Parallel OpenGL 2.0 Tiled Rendering IceT
Deadlock, transparency bugs
Optimizations: bounds; multi-stage compositing
Tuvok (IV3D)
Multi-GPU distributed memory VR
These are the things I listed I would work on during the Spring all hands meeting. This slide is just an overview/reminder: I only explain things here, and then mention all of the work I’ve done to address these on later slides.
Using OpenGL 2.0 code in VisIt in a parallel setting was difficult. Any code that used OGL 2.0 used GLEW, and the OGL library GLEW used was set at compile time. Compile time is too early for us; if you “mpirun –np 16 engine_par” on a node that has 16 CPUs and 2 GPUs, you can’t know until run time that 14 of those procs should use CPUs and 2 should use GPUs. Thus attempting to do parallel (“SR”) rendering of some plots, without using hardware acceleration, would crash.
Tiled rendering was referring to display walls, but could also refer to desktop-sized images. The idea is to do an image space decomposition, stenciling out pixels that don’t belong to a particular node. In turn, this implies some amount of data overlap; any domain which spans a tile boundary must exist on both tiles. This is a great idea if your rendering is fragment-bound (as it normally is on a display wall)
IceT: IceT would deadlock in some rare cases, and wasn’t given enough information to correctly composite when transparency was involved. As future work, I mentioned giving IceT world coordinate bounds of each domain, which it can project to screen space and therefore use for more efficient compositing; secondly, I mentioned doing multi-stage compositing: composite on a single node (SMP) before compositing over the entire cluster (MPI).
Tuvok: we mentioned bringing it up to production quality, and beginning an integration into VisIt.
Multi-GPU distributed memory volume rendering … well, the bullet really says everything.

4. OpenGL 2.0 in Parallel Then Now www.vacet.org
I’ve recently merged a branch which contains a modified GLEW that loads OpenGL at run time. This fixes those SR mode crashes, for one. It also enables one to do more advanced, cool things, like multi-GPU distributed memory VR via fragment shaders…
The merged code does have some minor issues which I’ve already resolved locally, but need to put in some time to make sure it ends up on the trunk / in VisIt 2.0.
The figure is actually incorrect. We can use GPUs in SR mode (that’s how multi-GPU Tuvok works, among other use cases). Further, even initializing ends up calling GLEW. I’m just trying to keep the slide simple && make the problem obvious.
Then
Now

5. Community Participation
Mesa 7.5+
Watching Mesa dev since ~Jan 2009
Aim: VisIt works with ‘out of the box’ Mesa.
“Mesa developer”
GLEW
Again, working with community; dynamic loading code.
With Jianrong Shu (Utah), contributing back fixes (~1 month away as of this writing).
Getting OGL 2.0 parallel rendering working required some external work. One was updating Mesa in VisIt. I went the extra mile and made sure the upgrade would go well, by working closely with the Mesa development community. Much of my work is now in core Mesa, maintained by that community (admittedly, I still participate). The Mesa “Benevolent Dictator” (my wording) has asked me to get a freedesktop.org account so that I can commit directly.
I debugged / found an issue with how GLEW accesses OGL that effects apps that try to do mixed GPU/CPU parallel rendering, as we do. I implemented a solution that works on Mac / Linux. I’ve since been advising a grad student once / week, as he gets it to work on Windows, and beyond that I’ve had him clean up some code and do some additional testing. This code is on the VisIt trunk already. We’ll be submitting it to the GLEW development community shortly; I would be surprised if round one of that submission did not happen in October 2009.

6. IceT / VisIt Bugs fixed for 1.12
Default compositing mode for nightly tests
Still not default for users … 2.0
Previously mentioned some optimizations:
Tiled rendering
Propagate bounds information
Multi-stage compositing
All stalled / ignored due to lack of time.
IceT: I have done the necessary infrastructure work to fix the deadlock bug. The transparency problem is hard; after discussions with VisIt developers, we decided the best way to get users to adopt this is to make the IceT code path fall back to the older path when transparency is found. I’ve done the infrastructure work required to make this happen; both these fixes are deployed now, in IceT is now our default compositor for nightly regression testing. Unfortunately, I missed the window to force IceT as the default compositor for 1.12, so users still need to explicitly enable it. Missing the window was sort-of on purpose; IceT was our default compositor for nightly regressions for one night before the release, and I wanted to give it more time. Turns out my concerns were unwarranted, though… oh well.
Optimizations: The bounds information might be trivial to pass to IceT, but I haven’t looked at it. Multi-stage compositing is a research topic; IMHO, it’s just barely (a year, maybe two) past ripe in 2009, and I bet someone will have published it by 2010.

7. Tuvok in VisIt
Tuvok integration is underway…

8. Tuvok in VisIt (2) Seriously though… it works
Multi-domain (not resampled) rendering ‘working’ (research code)
Some other fixes need propagation to trunk (e.g. camera)
ToDo: 2D TFs, ClearView, Raycaster
Jens and I have met the promised “ImageVis3D and Tuvok 1.0” milestone. We’ve actually passed that by a wide margin; the 1.2 release contains a significant number of changes aimed at making integrations with other applications much easier. From a changelog I just generated, you might even say that was the focus of the 1.2 release.
There are some issues which are resolved in research code but will require non-trivial engineering to actually get working.

9. Multi-GPU Distributed VR
Basically working, couple minor bugs
Definitely “research code”
Uncovered a lot of VisIt / Tuvok issues
Initial results very promising
Per-process render time, averaged, 16m pixels:
The paper is in progress. We wanted to hit the special CG&A issue, but had no luck. The majority of the effort has been VisIt coding which ends up stalled on me.
The current code for this is definitely “research code.” Despite this, I wish I’d started writing the research code a lot sooner; it’s uncovered a lot of issues much better than the sometimes-sisyphean effort involved to get code production ready. Case in point: sometimes you need to do it wrong first before you can do it right.
Jens and I have done a lot of work to make this smoother, and I’m confident that I can handle any additional issues that may crop up. I’m not nearly as confident on the VisIt side.
Rendering Method
Render Time (s)
OSMesa
NVIDIA 8800 GTX

10. Public (Anonymous) Subversion
VisIt can now be checked out anonymously by those with no affiliation with the VisIt project.
Many users for asking for this; one of the most vocal was Tech-X. Since it took an eternity to get things going, I setup a temporary mirror at SCI for some folks to get immediate access. As of September 17th, though, we’ve now got a more permanent mirror at NERSC.
The idea is that developers (or users) in the community can checkout and test both our trunk and e.g. release candidates. With any luck, they’ll contribute back patches by sending them on the mailing list, and a developer with write access will commit them back to the main repository. Not sure this will really happen in practice, though (hasn’t yet).
Getting this was a complete nightmare.
Art is part GPL, part © Canonical, part mine… don’t copy for public use.

11. Non-LLNL Bug Tracker Setup Gforge, none of us liked it.
Started organizing search for others
Temp. stalled (mea culpa)
Thanks: Cyrus, Jeremy
“Soon”.
I setup a bug tracker at the SciDAC outreach center (based on Gforge). None of us (myself included) liked it. I plodded through setting it up because others wanted something, but when we finally got to the “okay, try it out!” point, we all hated it and decided to look for something better. I started organizing that effort and we got a good ways in, with a huge thanks to Cyrus and Jeremy in that regard. Unfortunately it stalled (because of me) for 1.5 months, which I find pretty embarrassing.

12. User Support Not much to say; title says it all
Spent an exorbitant amount of time on this.
Worth it?
There have been times where I’ve monitored the VisIt lists very closely and been very helpful supporting users or giving my development opinions. I’m starting to doubt the value of this; I almost removed this slide, it seems barely worth mentioning. Yet looking at all my VACET activities individually, I’d bet this one took more time than any other.
I get asked about SR issues from time to time (on the ML); I guess these days, I am the one who has looked at the code most recently. I know about a number of issues, but haven’t had time to investigate…

13. End Future Contact Finish Tuvok integration, + 2d TFs, ClearView
Multi-GPU VR
Dream: large data on large displays (distrib. VR on teslas, view on display wall)
Get bug tracking sorted out
Contact
VACET mailing list, of course