1. www.cs.wisc.edu/condor 1 The Roadmap to New Releases Todd Tannenbaum Department of Computer Sciences University of Wisconsin-Madison http://www.cs.wisc.edu/condor tannenba@cs.wisc.edu

2. www.cs.wisc.edu/condor 2 Stable vs. Development Series › Much like the Linux kernel, Condor provides two different releases at any time:  Stable series  Development series › Allows Condor to be both a research project and a production-ready system

3. www.cs.wisc.edu/condor 3 Stable series › Series number in version is even (e.g. 6.2.0) › Releases are heavily tested › Only bug fixes and ports to new platforms are added on a stable series

4. www.cs.wisc.edu/condor 4 Stable series (cont.) › A given stable release is always compatible with other releases from the same series › Recommended for production pools

5. www.cs.wisc.edu/condor 5 Development Series › Series number in the version is odd (e.g. 6.1.17, 6.3.0) › New features and new technology are added frequently › Versions from the same development series are not always compatible with each other

6. www.cs.wisc.edu/condor 6 Development Series (cont.) › Releases are not as heavily tested › Generally not recommended for production pools  … unless new features are required  … unless we recommend otherwise :^)

7. www.cs.wisc.edu/condor 7 Where is Condor Today? › New stable version 6.2.0 recently released › Previous development series cycle, ver 6.1.*, was far too long › This was our first stable/development series: we're hopefully learning...

8. www.cs.wisc.edu/condor 8 New Ports in 6.2.0 › Full support (with checkpointing and remote system calls):  All current versions of Linux (x86) Kernel: 2.2.* and 2.0.* C Library: glibc-2.[01] and libc-5 NOTE: libc5 support dropped  Solaris 2.7 and 2.8(Sparc and x86)  Irix 6.5

9. www.cs.wisc.edu/condor 9 New Ports in 6.2.0 (cont.) › ”Clipped" support (no checkpointing or remote system calls, but all other functionality is available)  Windows NT  Alpha Linux

10. www.cs.wisc.edu/condor 10 What is New in 6.2.0? › Personal Condor and Grid Support  Flocking  Proof-of-concept support for Globus Job Universe Globus Glide-In › Full, integrated support for Symmetric Multi-Processor (SMP) machines

11. www.cs.wisc.edu/condor 11 What's New in 6.2.0? (cont.) › PVM support › DAGMan (for managing inter-job dependencies) › Use of machine ClassAd attributes in submit file  Example: Executable = foo.$$(ARCH).$$(OPSYS)

12. www.cs.wisc.edu/condor 12 What's New in 6.2.0? (cont.) › Greatly expanded I/O support for STANDARD universe jobs  Condor can automatically buffer I/O requests from jobs  Compression/decompression on-the-fly!  Users get much more information about the kinds of I/O their jobs are performing  Users can "remap" files to alternate locations

13. www.cs.wisc.edu/condor 13 What's New in 6.2.0? (cont.) › CondorVersion and CondorPlatform strings included in all binaries and libraries  Helps identify and avoid problems with having the wrong version installed  Different parts of the Condor protocol automatically check for version incompatibilities

14. www.cs.wisc.edu/condor 14 What's New in 6.2.0? (cont.) › Better accounting  Historical usage logging in collector Use condor_stats web interface called CondorView  Accountant stores usage information per user › Better control over user priorities  "Priority factors"

15. www.cs.wisc.edu/condor 15 What's New in 6.2.0? (cont.) › More powerful administration tools  Setting configuration values remotely  Querying daemons directly for status › Other misc new tools  condor_findhost, condor_hold/release, condor_qedit, condor_userlog, condor_run, … › Lots of performance and bug fixes › A complete list is online in the manual (www.cs.wisc.edu/condor/manual)

16. www.cs.wisc.edu/condor 16 The 6.3 Development Series › Major focus areas for ver 6.3.x Series  Future-aware scheduling for dedicated resources MPI support  Security enhancements –Secure network communication »Kerberos, PKI (GSI/OpenSSL) –Limitations on executable, remote syscall sandbox  Condor-G production version Integrate Bypass technology  Enhanced Windows support

17. www.cs.wisc.edu/condor 17 What else will be added in 6.3.*? › And also several easy-to-add, user- visible features  ”Master agents" - helper programs spawned by the condor_master to aid in administration Retrieving remote log, history and/or configuration files Remote "top", "ps" and other monitoring functions

18. www.cs.wisc.edu/condor 18 What will be added in 6.3.*? (cont.) › Solution for Condor clusters on  Private network  Behind a firewall › New condor_starter and condor_shadow for the VANILLA universe on Unix  "NT version" will be used for Unix, too  Lots of new features, like transferring files automatically for "vanilla" jobs (no need for a shared filesystem)

19. www.cs.wisc.edu/condor 19 What will be added in 6.3.*? (cont.) › condor_startd will enforce resource limits dynamically (like RAM usage) › More logic put into the condor_shadow to  Detect temporary problems with a job's execution, put the job on hold, and notify the user  Enforce job submitting user’s policy › Better propagation of error messages

20. www.cs.wisc.edu/condor 20 The 6.5.* Development Series › 6.5.* will be for adding fundamentally new technology to Condor › Being developed in parallel with 6.3.* › Will ultimately be v7.0 › Will be incompatible with previous versions of Condor

21. www.cs.wisc.edu/condor 21 New Technology in 6.5.* › New version of ClassAds  Conditionals !! if/then/else  Aggregates (lists, nested classads)  Built-in functions String operations, pattern matching, time operators, unit conversions  Clean implementations in C++ and Java  ClassAd collections

22. www.cs.wisc.edu/condor 22 Other changes for 6.5.* › Re-write of the condor_schedd  Performance enhancements and lowered resource requirements (particularly RAM) › Re-write of the checkpoint server  NEST technology infusion  Enhanced support for multiple servers  Will meta-data along with checkpoint files

23. www.cs.wisc.edu/condor 23 Planned Future Ports › Full support  Intel Linux 2.4 kernel (RedHat 7.x) › Clipped support  Windows 2000 / Whistler

24. www.cs.wisc.edu/condor 24 Possible Future Ports  Full Support (I.e. checkpointing, etc) Windows 2000 Alpha Linux DUX 5.x  Clipped-support {free,open,net}BSD MacOS X PowerPC Linux HPUX 11.0 AIX 4.2

25. www.cs.wisc.edu/condor 25 Thank you for coming to Paradyn/Condor Week!