Page: 1 Sun Confidential: Internal Use and Authorized Partners Only OpenSolaris and the Bio-OpenSolaris Software Project Gilbert Thomas Sun Microsystems, APSTC

Page: 2 OpenSolaris – What is OpenSolaris? – Some Notable Features – OpenSolaris Distributions – Spotlight: ZFS Bio-OpenSolaris Software (BoSS) Project – What is it? – Past and current tasks – Upcoming tasks – Help needed! Agenda

Page: 3 Open source project to build a developer community around Solaris technology Allow developers and administrators to utilize powerful Solaris features for free as well as to contribute back to the project – ZFS – Dtrace Software tools for development are now freely available – Sun Compilers (Sun Studio 11) – Parallel Apps Development Tools (HPC ClusterTools 6) – Portal and Web Deployment (Java Enterprise System) What is OpenSolaris?

Page: 4 Solaris Zones/Containers ( – Allow multiple virtual operating systems to run separately within a single instance of Solaris OS – Applications/processes run on one zone will not affect another – BrandZ – Running a Linux OS on a Solaris zone! Dtrace – Advanced debugging and tuning tool ZFS Some Notable Features

Page: 5 Nexenta – OpenSolaris with GNU software – Uses Debian package system for easy installation of software – UI similar to Ubuntu Belenix – KDE/Xfce Desktop – Open Solaris aiming for as much compatibility with official Solaris Both Distributions come in LiveCD and Installable versions. OpenSolaris Distributions

Page: 6 Nexenta

Page: 7 BeleniX

Page: 8 ZFS : Sun's Next Generation File System

Page: 9 Immense capacity: 256 quadrillion(10^15) ZB (1 ZB = 1 billion TB) Dynamic file system size Flexible storage pool: No more volume Near-Zero administration: No vfstab, dfstab End-to-End data integrity: No more fsck Self healing: Error detection and correction Data security, data compression, data snapshot The Zettabyte File System (ZFS)

Page: 10 ZFS Vs. FileSystem&Volume Model Traditional Volumes Partition/volume for each FS Grow/shrink by hand Each FS has limited bandwidth Storage is fragmented, stranded ZFS Pooled Storage No partitions to manage Grow/shrink automatically All bandwidth always available Pool allows space to be shared Storage Pool Volume FS Volume FS Volume FSZFS

Page: 11 Self-Healing Data in ZFS Application ZFS mirror Application ZFS mirror Application ZFS mirror 1. Application issues a read. ZFS mirror tries the first disk. Checksum reveals that the block is corrupt on disk. 2. ZFS tries the second disk. Checksum indicates that the block is good. 3. ZFS returns good data to the application and repairs the damaged block.

Page: 12 ZFS Administration Pooled storage – no more volumes! – All storage is shared – no wasted space – Grow and shrink are automatic – No more fsck(1M), format(1M), /etc/vfstab, /etc/dfs/dfstab... Unlimited, instantaneous snapshots and clones – Snapshot: read-only point-in-time copy of the data Modified/new data is written in different location to reserve snapshot data – Clone: writable copy of a snapshot

Page: 13 UFS/SVM Administration Creating three filesystems ann, bob, and sue on a two-disk mirror # format... (long interactive session omitted) # metadb -a -f disk1:slice0 disk2:slice0 # metainit d disk1:slice1 d10: Concat/Stripe is setup # metainit d disk2:slice1 d11: Concat/Stripe is setup # metainit d20 -m d10 d20: Mirror is setup # metattach d20 d11 d20: submirror d11 is attached # metainit d disk1:slice2 d12: Concat/Stripe is setup # metainit d disk2:slice2 d13: Concat/Stripe is setup # metainit d21 -m d12 d21: Mirror is setup # metattach d21 d13 d21: submirror d13 is attached # metainit d disk1:slice3 d14: Concat/Stripe is setup # metainit d disk2:slice3 d15: Concat/Stripe is setup # metainit d22 -m d14 d22: Mirror is setup # metattach d22 d15 d22: submirror d15 is attached # newfs /dev/md/rdsk/d20 newfs: construct a new file system /dev/md/rdsk/d20: (y/n)? y... (many pages of 'superblock backup' output omitted) # mount /dev/md/dsk/d20 /export/home/ann # vi /etc/vfstab... while in 'vi', type this exactly: /dev/md/dsk/d20 /dev/md/rdsk/d20 /export/home/ann ufs 2 yes - # newfs /dev/md/rdsk/d21 newfs: construct a new file system /dev/md/rdsk/d21: (y/n)? y... (many pages of 'superblock backup' output omitted) # mount /dev/md/dsk/d21 /export/home/ann # vi /etc/vfstab... while in 'vi', type this exactly: /dev/md/dsk/d21 /dev/md/rdsk/d21 /export/home/bob ufs 2 yes - # newfs /dev/md/rdsk/d22 newfs: construct a new file system /dev/md/rdsk/d22: (y/n)? y... (many pages of 'superblock backup' output omitted) # mount /dev/md/dsk/d22 /export/home/sue # vi /etc/vfstab... while in 'vi', type this exactly: /dev/md/dsk/d22 /dev/md/rdsk/d22 /export/home/sue ufs 2 yes - # format... (long interactive session omitted) # metattach d12 disk3:slice1 d12: component is attached # metattach d13 disk4:slice1 d13: component is attached # metattach d21 # growfs -M /export/home/bob /dev/md/rdsk/d21 /dev/md/rdsk/d21:... (many pages of 'superblock backup' output omitted)

Page: 14 ZFS Administration Creating three filesystems ann, bob, and sue on a two-disk mirror Create a storage pool named “home” # zpool create home mirror disk1 disk2 Create filesystems “ann”, “bob”, “sue” # zfs create home/ann # zfs create home/bob # zfs create home/sue Add more space to the “home” pool # zpool add home mirror disk3 disk4

Page: 15 ZFS: Examples of other Commands zfs create home/mail zfs set mountpoint=/var/mail home/mail zfs create home/ann zfs set quota=50g home/ann zfs set compression=on home/ann zfs set sharenfs=rw home

Page: 16 Resources References > ttp:// > ttp:// > ttp://belenix.sarovar.org/

Page: 17 Bioinformatics-on-OpenSolaris Software Project (BoSS)

Page: 18 BoSS: What is it? An OpenSolaris LiveCD based on APBioKnoppix model. Will include applications compiled with Studio 11, and hence optimized for Solaris platform Will include Sun Grid Engine for an easily deployable cluster grid solution.

Page: 19 BoSS: What is it? Sun Biobox solution already available > Grid Engine Portal > Bioinformatics applications > Grid Engine Portal BoSS is an extension to Biobox > Rapid deployment > Introduction to Solaris OS/Grid Engine without hassle of installing and configuring it.

Page: 20 BoSS: Past and current tasks BLAST, FASTA, FastDNAml and several other applications have been compiled and optimized for OpenSolaris Remastering of LiveCD being done now, first release will be based on BeleniX

Page: 21 BoSS: Upcoming tasks Boot from ISO option for faster boot time. UnionFS to allow persistent user directories and user- installable applications. Integration of a workflow integration system ZFS Compression on the filesystem

Page: 22 BoSS: Help Needed! Lack of manpower is a major issue. Internships are available for APSTC for students to participate in this project. Collaborations with bioinformatics institutes for their expertise. If interested, and start with “BoSS:” in Subject line.

Page: 23 Gilbert Thomas The End