1. “Bookshelf.exe”: Executable Extensions of the Bookshelf Igor Markov University of Michigan, EECS DARPA

2. Outline A three-slide version of the talk –motivation + proposal + how it will help Basic use models –users and interfaces –restrictions Existing VLSI CAD Bookshelf Efforts related to our proposal Details of the proposal

3. Motivation Experiences from education –e.g., undergraduate courses on algorithms and architecture –infrastructure for evaluation: auto-graders Infrastructure for collaborative research –can also benefit from automation –must support sharing, modularity and reuse –must scale, must be industry-compatible Modularity in implementation platforms

4. Bookshelf.exe Dynamic version of the [existing] bookshelf –Implementations+benchmarks = algo evaluations? –Flow composition and high-level scripting Related efforts –SatEx, PUNCH, NEOS, OmniFlow Proposed solution: ``Bookshelf.exe’’ –Application Service Provider –Interfaces Support for optimization-specific concepts Power versus ease-of-use and modularity –High-level scripting and flow composition

5. Why We Need Bookshelf.exe To improve the efficiency of Bookshelf SW maintenance: automation (cf. SourceForge) HW/SW incompatibilities, lack of CPU cycles –Standardization and consolidation Many results cannot be reproduced –Simplified sharing and CAD IP re-use Lack of high-level, large-scale experiments –Web-based scripting, distributed execution –Open-source flows (with or w/o o.-s. components)

6. Basic Use Models Users (over the Web) –“anonymous” –registered (more features) User interfaces –HTML forms (including downloads, uploads, high-level scripting and job control) –email notification –XML-RPC for contributed remote CPUs and data –NFS (where available) for file sharing

7. Restrictions Uploaded files must match existing formats –Interface for adding new formats Execution in a “sandbox” –Cannot make network connections –Visible file-system restricted by the chroot() call –Configurable resource limits (memory, etc) Access permissions –Registered users can keep their results private –Benchmarks or solvers behind firewalls

8. The VLSI CAD Bookshelf IEEE Design and Test, May/June 2002 Contents fairly popular: downloads, pubs –benchmarks, implementations and comparisons –algorithm descriptions and analyses –evaluation methodologies (in English) New slots, benchmarks, codes added regularly ICCAD, DAC, ISPD publications use the bookshelf A static collection –Manual addition of material –No automatic evaluation, reporting of results

9. Other Efforts Related to “.exe” SatEx http://www.lri.fr/~simon/satex/satex.php3 –Specialized in satisfiability problems PUNCH http://punch.ece.purdue.edu –Very broad selection of software (from StarOffice to Capo) –Local to Purdue NEOS http://www-neos.mcs.anl.gov/neos –Open-source, distributed architecture –Used primarily for linear and non-linear optimization OmniFlow: DAC 2001, Brglez and Lavana –http://www.cbl.ncsu.edu/OpenProjects/OmniFlow –Distributed Collaborative Design Framework for VLSI –GUI-based flow control, chaining of design tools

10. SatEx Continual evaluation and ranking of codes –Results produced and posted automatically –Intuitive interface Popular –93,707 hits March, 2000 – September 2001 –23 SAT provers, 32,610 runs – September 2001 Limited scalability –One workstation (2yrs of CPU time) –Specialized to one application

11. PUNCH Very general execution framework –From VLSI CAD to GUI-based office applications –Custom-designed file-system (Purdue hosts only) –241,458 runs in 5 years (8,152 in VLSI CAD ) –20+ publications Only maintainer can add executables No support for eval’n and chaining (flows) –No stats for results of runs (cf. SatEx top 20) –No MIME-like data types –Difficult to use when multiple tools are involved

12. NEOS Open-source, distributed framework Wide use, solid code base Adding new implementations requires maintainer intervention (< on PUNCH) –Each new code must come with a host –Distributed maintenance Loose data typing –No type system for data and implementations Compare to MIME

13. NEOS (what can be improved) Independent eval. and verification of results –e.g., PUNCH offers a WL-eval. From the bookshelf Real-time on-line reporting of results + stats High-level scripting and flow design –Scripts to control the execution and evaluation flows Pairing solvers with benchmarks –SatEx-like evaluation, but for multiple data types

14. OmniFlow Context: collaborative VLSI Design –sharing computational resources, but not results Distributed over multiple hosts Provides GUI-based flow control –supports chaining of design tools –several hard-coded conditions for flow control –no support for execution conditional on results –no scripting language; limited by GUI Cannot dynamically add hosts

15. Bookshelf.exe Best of SatEx, PUNCH, NEOS and OmniFlow –Reporting style similar to SatEx (+ alternatives) –The versatility of PUNCH –Scalability and distributed nature of NEOS (or better) –Flow control as in OmniFlow or better New features –MIME-like data types and optimization-specific concepts –Automatic submission of binaries and source code –Chaining of implementations; scripting for flow control –Support for use models with proprietary data or code

16. Bookshelf.exe (2) Scalability –Computation is distributed (unlike in SatEx) –Maintenance is automated (unlike in NEOS) Support for multiple use models –“adapts to users” –Multiple levels of expertise –Multiple levels of commitment –Sharing of public data –Hiding/protection of proprietary data –“Screen-saver” mode, cf. SETI@Home, Entropia, etc

17. Interface Issues Transparent error diagnostics –Greatly improve learning curve Ownership, privacy, resource limits: sample policy questions –How can different owners chain their jobs? –What jobs can be launched anonymously? Script composer versus programming –Flexibility versus learning curve –GUI implemented using HTML forms (converts clicks and fill-in-the-blanks to scripts)

18. Script Composer Built on top of a scripting language –Based on PERL (PERL functions available) Basic flows designed with HTML forms –Start with one ‘step’ and add more steps –API funcs: e.g.,“run ‘optimizer 2’ + store results” –Support for conditionals and iteration Scripts sent to back-end for execution as jobs Scripts can be saved, posted, reused

19. Language-Level Support Type system for submissions, results and intermediate data –Algorithm implementations Deterministic and randomized optimizers, etc. –Input data, results, status info (runtime, memory,…) –Common benchmarks Rules for matching submission types –E.g., match a placer with a LEF/DEF benchmark –Violations are reported to user as fatal errors

20. Data Models Consistent data models needed for serious data flows and high-level experiments –e.g., integrated RTL-to-layout implementation We plan to use OpenAccess 2.0 –Specs and API published in April 2002 –Implementation and source expected next year Adjustments within bookshelf expected in terms of open-source design flows –E.g., for industry SP&R integration

21. BX: Structure BX front-end: mostly Web-based (+ email) BX back-end –Main server (job scheduling, reporting of results, etc) –Client software on computational hosts Network communications: XML RPC –RPC standard –XML data encoding –HTTP network transport –Compatible with C/C++, Perl, Python, etc.

22. Implementation Status So far main focus on the back-end Back-end ver 0.1 functional on Linux –BX state maintained in a database –Persistence, etc –Simple one-job demo (1 binary & 1 benchmark) Security features and basic policies –Sandbox execution and data type checks Front-end supports one-job demo Next mile-stone: “10X10” demo (cf SAT 2002) –Jobs automatically distributed and results posted

24. Conclusions Bookshelf: popular, but can be improved Bookshelf.exe: executable extensions Goals –Automate routine operations –Create open-source flows –Facilitate high-level, large-scale experimentation We plan to assimilate best features from related works + add new ones Started bottom-up implementation –Basic version of back-end is working

25. If we missed something important, let us know Thank you