1. “Bookshelf.exe”: Executable Extensions of the Bookshelf Marius Eriksen and Igor Markov University of Michigan, EECS

2. Outline Making the bookshelf more dynamic –Implementations + data = algorithm evaluations –Potentially more than that… Related efforts –SatEx, PUNCH, NEOS, OmniFlow Proposed solution: ``Bookshelf.exe’’ –Services offered: online execution, ASP, sharing of results, flow scripting,… –Interfaces Support for optimization-specific concepts Power versus ease-of-use and modularity –Architecture and implementation details

3. Making the Bookshelf More Dynamic Existing bookshelf: –A static collection of data and implementations –Manual addition of material –No automatic evaluations, reporting of results Those limitations are not fundamental –Artifacts of the current implementation Proposal for improvement: “Bookshelf.exe”

4. Related Efforts SatEx http://www.lri.fr/~simon/satex/satex.php3 –Specialized in satisfiability problems PUNCH http://punch.ece.purdue.edu –Very broad selection of software From StarOffice and gnulot to Capo and Feng Shui –Local to Purdue Univ. NEOS http://www-neos.mcs.anl.gov/neos –Open-source, distributed architecture –Used primarily for linear and non-linear optimization OmniFlow: DAC 2001, Brglez and Lavana –http://www.cbl.ncsu.edu/OpenProjects/OmniFlow –Distributed Collaborative Design Framework for VLSI –GUI-based flow control, chaining of design tools

5. SatEx Continual evaluation and ranking of submitted implementations –Results produced and posted on the Web automatically –Intuitive interface Popular –93,707 hits since March, 2000 –23 SAT provers, 32,610 executions Limited scalability –Runs on one workstation –529 days, 17 hours, 53 minutes and 20 seconds of CPU time –Specialized to one application

6. PUNCH Very general execution framework –Everything from domain-specific algorithm implementations to GUI-based office applications –Custom-designed filesystem (restricts to Purdue hosts) –241,458 executions in 5 years 8,152 on VLSI CAD related applications –20+ publications Only maintainer can add executables No support for evaluation and chaining/flows –No stats for results of runs (cf. “top 20” on SatEx ) –No MIME-like data types (even IE/netscape need them!) Difficult to use when multiple tools are involved

7. NEOS Open-source, distributed framework Wide use, a solid library of implementations Adding new implementations requires maintainer intervention (but less than on PUNCH) –Each new implementation must come with a host –Distributed maintenance Loose data typing –No type system for data and implementations Compare to MIME

8. NEOS (what can be improved) Independent eval. and verification of results –E.g., PUNCH offers a WL-eval. From the bookshelf Real-time on-line reporting of results + stats Provisions for chaining multiple implementations –Scripts to control the execution and evaluation flows Provisions for pairing exec’s with benchmarks –Goal: SatEx-like evaluation, but for multiple data types

9. OmniFlow Context: collaborative VLSI Design –collaboration in terms of computational resources –no provisions for sharing results on the Web Distributed over multiple hosts Provides GUI-based flow control –supports chaining of design tools –several hard-coded conditions for flow control –no support for execution conditional on results –no scripting language; limited by GUI Cannot dynamically add hosts

10. Bookshelf.exe Best of SatEx, PUNCH, NEOS and OmniFlow –Reporting style similar to SatEx (+ alternatives) –The versatility of PUNCH –Scalability and distributed nature of NEOS (or better) –Flow control as in OmniFlow or better New features –MIME-like data types and optimization-specific concepts –Automatic submission of binaries and source code –Chaining of implementations; scripting for flow control –Support for use models with proprietary data or code

11. Bookshelf.exe (2) Scalability –Computation is distributed (unlike in SatEx) –Maintenance is automated (unlike in NEOS) Support for multiple use models –“adapts to users” –Multiple levels of expertise –Multiple levels of commitment –Sharing of public data –Hiding/protection of proprietary data –“Screen-saver” mode, similar to SETI@Home, Entropia, etc

12. Bookshelf.exe (3) Provides basic algorithm evaluation infrastructure to research groups –Avoid large maintenance costs –Just as usable for larger groups

13. End-user Interface Web site –Using CGI/PHP for interactive content and database interaction –Submission through the web site (email is ok) –Dynamic addition of computation hosts –Stores data locally or uploads on demand –Automatic Web-based reports and statistics

14. End-user Interface (2) User accounts (including anonymous) –Every user has private resource-limited “workspace” –Job submissions must be from accounts –Results of jobs stored on accounts –Anonymous jobs possible, but can use fewer resources users are encouraged to register Many policy questions (no roadblocks seen) –e.g., can different owners chain their jobs? Transparent error diagnostics –Greatly improves learning curve

15. End-user Interface (3) Result notification (similar to NEOS) –E-Mail –Web reports Script composer for making run flows without needing to know a scripting language –less flexibility, but better learning curve –GUI implemented as form that POSTs scripts

16. Script Composer HTML Form interface – basic flow with fundamental predicates and functions –All API functions E.g. “run algorithm ‘optimizer 2’ and store the results” –Several common Perl functions –Conditional statements E.g. “verify if quality is more than 60” –Iteration support E.g. “iterate until quality is 100 or the number of iterations is 100”, “run randomized solver 3 timers, report the best result” –Start with one ‘step’ – add an arbitrary number of steps

17. Optimization-specific Concepts in Bookhelf.exe Type system (similar to MIME) for all types of submissions Rules for matching submissions of different types –E.g., match a placer with a LEF/DEF benchmark Types include –Algorithm implementations Deterministic and randomized optimizers, etc. –Input data, results –Common benchmarks

18. Architecture Overview

19. Component Types Where relevant, components also have types Data –Data format E.g. Cadence LEF/DEF, DIMACS CNF Implementations –Architecture (solaris, linux, ‘source code’, etc.) –Class (optimizer, randomized solver, etc.)

20. Component Descriptors Unique id, name and description for each component Owner field Component history –E.g. usage of a certain implementation, piece of data or benchmark Component specific fields –Associated expert for implementation, incremental diff for data Component version Component changelog

21. Data Controller Backing store –Permanent storage of all involved components Backed by database and file system –Data, results suited for database –Implementations suited for file system storage Components may be deferred to local storage –Proprietary, non-disclosed components on a host owned by the proprietor of the component All data stored with complimentary descriptor

22. Job Controller Responsible for communicating with and controlling all agents (and thus jobs and implementations) Initiates jobs Delivers user instructions Handles data retrieval and storage –Interfaces with data controller Keeps track of node load, etc. for proper distribution

23. Host Job Controller Resides on each node in the distributed network –Basic token of presence of DS-OSS Reports statistics to job controller –Load, memory usage Signals when ready to take jobs Launches agents as necessary Performs self upgrades –New software revisions, bugfixes

24. The Agent Responsible for running job on node Provides environment for script to run in –API –Script sandbox/jail For security reasons –Provides common callbacks if not provided by scripts Data storage/retrieval

25. The Agent (2) Input/Output of data from script Job notification and control –Start, stop, termination Maintains local demands –Resource usage –Stop on local demand

26. Scripts In Perl Interfaces directly with an algorithm, or expert handling it API support –Full Perl library –Job control functionality –Data storage and retrieval

27. Scripts (2) Checks for conflicting input data types and algorithm input type –Generates appropriate, understandable errors A number of automatic tasks –Pass thru data storage (e.g. results of a single run) –Can provide all steps automatically, leaving the user to do exactly what he/she wants

28. The Expert Wrappers for proprietary implementation interfaces –Knows how to deal with a particular interface E.g. command line options, configuration files –Translates from well known parameters (specified from the API) to proprietary interface

29. In Conclusion DSS-OS provides features that are missing in current systems –Existing systems are limited by their architecture and thus cannot provide these needed features. Provides an infrastructure that is scalable –Computationally scalable –Scalable to proprietary vendors

30. In Conclusion (2) Scripting support allows for flexible execution flows –Conditional flows, iterative flows Provides an established infrastructure for smaller research groups –Enjoying advantages that only large proprietary, high-maintenance systems previously could provide.