Using Virtual Reality for the Visualization of Developing Tissues J.P. Schulze 2, L.D. Soares 1, J. Weaver 1, A.S. Forsberg 2, S.M. Shim 2, K.A. Wharton 1 Brown University, Providence, RI, Department of Molecular Biology, Cell Biology & Biochemistry 2 Department of Computer Science We have developed VOX (VOlume Explorer), a visualization system to explore image stacks from the laser scanning confocal microscope. We reconstruct the 3D sample by rendering the entire image stack with direct volume rendering. Our system allows the user to interactively view a data set in a Cave, the Brown University’s CAVE-like environment, or at the desktop. In the Cave, head tracking allows the user to walk around the data set and look at it from different sides. A 3D input device (wand) allows interaction with the software application in order to rotate and move the data set, change color intensities, place clipping planes, or display intensity diagrams and histograms. This visualization method is especially well suited for examining samples with considerable 3D complexity. It has allowed us to discover unexpected visual features in new data sets. The main user interface is displayed on the left wall of the Cave. It consists of an array of buttons and dials that control the different visualization parameters. The user interacts with it by pointing to the parameter specific button or dial with a virtual laser beam and clicking on it with the wand. Technical Specifications of the Cave PCs:4 Linux PCs with Intel Xeon CPUs or 48 Linux PCs with dual Xeon CPUs and IBM SGEs Graphics cards: 4 Nvidia Quadro FX 3000G or 48 Nvidia GeForce FX Projectors:4 CRT projectors, active stereo at 120 Hz, resolution 1024x768 or 1280x1024 pixels Software:C++, Cg, OpenSceneGraph Rendering algorithm: Texture hardware-accelerated, slice-based volume rendering (view-dependent slices) We use a hand-held Wanda 3D input device with three buttons and a trackball to interact with the virtual world. Our software also runs on a desktop computer (Windows or Linux) with a slightly reduced set of functions (no 3D input devices, no iso-surfaces, no marking). In this view of a Drosophila egg chamber, nuclei have been labeled with Hoechst (blue) and cell membranes with phalloidin (green). Clipping plane: This function limits rendering to within a plane or on one side of the plane. The clipping plane can be moved with the wand (click-and-drag). This tool allows the user to view unobstructed structures from perspectives that would not otherwise be possible. The image at right shows the anterior surface of a developing oocyte from inside the cell, and the bottom two images show the egg chamber cut along a parallel plane through four nurse cells from two different perspectives. Nuclei have been labeled with Hoechst (blue) and cell membranes with phalloidin (green). VOX allows changing the color intensities of data sets with one to four data channels. The color intensity can be adjusted continuously in real time. The upper image on the right shows a data set with low levels of red; the lower image shows more intense red. This data set shows a Drosophila wing imaginal disc showing nuclei (blue), dpp- lacZ expression (red) and Delta (Dl) expression (green). Altering the intensity of a given channel can allow better visualization of a particular staining with respect to the other channels. This image shows the larval proventriculus with nuclei labeled in green. The markers can be used to count nuclei in a given area/structure. Counting in a Cave environment is much easier because it is possible to see the entire structure at once and thus easily visualize and mark structures that would otherwise be blocked by others in a different plane. The user can interactively place a line in the data set to display an intensity diagram of the data along the line, or a histogram of the values on it. In the image on the right we can see an intensity diagram (bottom) of dpp-lacZ expression in a wing imaginal disc (top). Visualization of an image stack allows the user to better perceive the three-dimensional aspects of a given biological sample. The image on the right shows a wing imaginal disc where the folds inherent to this tissue can be seen and easily manipulated to visualize details from another perspective. A screenshot from such a manipulation using the desktop application is shown below. In addition, visualizing the data set in a Cave environment allows the user to identify the subcellular localization of different components. The user can place cone-shaped markers to point out features.