1. METRICS: A System Architecture for Design Process Optimization Stephen Fenstermaker*, David George*, Andrew B. Kahng, Stefanus Mantik and Bart Thielges* UCLA CS Dept., Los Angeles, CA *OxSigen LLC, San Jose, CA

2. Motivations How do we improve design productivity ? Does our design technology / capability yield better productivity than it did last year ? How do we formally capture best known methods, and how do we identify them in the first place ? Does our design environment support continuous improvement of the design process ? Does our design environment support what-if / exploratory design ? Does it have early predictors of success / failure? Currently, there are no standards or infrastructure for measuring and recording the semiconductor design process

3. Purpose of METRICS Standard infrastructure for the collection and the storage of design process information Standard list of design metrics and process metrics Analyses and reports that are useful for design process optimization METRICS allows: Collect, Data-Mine, Measure, Diagnose, then Improve

4. Related Works OxSigen LLC (Siemens 97-99) Enterprise- and project-level metrics (“normalized transistors”) Numetrics Management Systems DPMS Other in-house data collection systems  e.g., TI (DAC 96 BOF) Web-based design support  IPSymphony, WELD, VELA, etc. E-commerce infrastructure  Toolwire, iAxess, etc. Continuous process improvement Data mining and visualization

5. Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

6. Potential Data Collection/Diagnoses What happened within the tool as it ran? what was CPU/memory/solution quality? what were the key attributes of the instance? what iterations/branches were made, under what conditions? What else was occurring in the project? spec revisions, constraint and netlist changes, … User performs same operation repeatedly with nearly identical inputs  tool is not acting as expected  solution quality is poor, and knobs are being twiddled

7. Benefits Benefits for project management  accurate resource prediction at any point in design cycle up front estimates for people, time, technology, EDA licenses, IP re-use...  accurate project post-mortems everything tracked - tools, flows, users, notes no “loose”, random data left at project end  management console web-based, status-at-a-glance of tools, designs and systems at any point in project Benefits for tool R&D  feedback on the tool usage and parameters used  improve benchmarking

8. Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

9. METRICS System Architecture Inter/Intra-net DB Metrics Data Warehouse Web Server Java Applets Data Mining Reporting Transmitter wrapper Tool Transmitter API XML

10. METRICS Performance Transmitter  low CPU overhead multi-threads / processes – non-blocking scheme buffering – reduce number of transmissions  small memory footprint limited buffer size Reporting  web-based platform and location independent  dynamic report generation always up-to-date

11. Example Reports hen 95% rat 1%bull 2% donkey 2% % aborted per machine % aborted per task BA 8% ATPG 22% synthesis 20% physical 18% postSyntTA 13% placedTA 7% funcSim 7% LVS 5% LVS convergence time 0100200300400500600 LVS % 88 90 92 94 96 98 100

12. Current Results CPU_TIME = 12 + 0.027 NUM_CELLS (corr = 0.93) More plots are accessible at  http://xenon.cs.ucla.edu:8080/metrics

13. Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

14. METRICS Standards Standard metrics naming across tools  same name  same meaning, independent of tool supplier  generic metrics and tool-specific metrics  no more ad hoc, incomparable log files Standard schema for metrics database Standard middleware for database interface For complete current lists see: http://vlsicad.cs.ucla.edu/GSRC/METRICS

15. Generic and Specific Tool Metrics Partial list of metrics now being collected in Oracle8i

16. Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

17. Testbed I: Metricized P&R Flow Placed DEF QP ECO Legal DEF Congestion Map WRoute Capo Placer Routed DEF CongestionAnalysis Incr. WRoute Final DEF METRICSMETRICS LEF DEF

18. Testbed II: Metricized Cadence SLC Flow DEF Placed DEF QP Pearl METRICSMETRICS QP OptCTGen Incr. Routed DEF WRoute Optimized DEF LEF GCF,TLF Clocked DEF Constraints

19. Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

20. Conclusions Current status  complete prototype of METRICS system with Oracle8i, Java Servlet, XML parser, and transmittal API library in C++  METRICS wrapper for Cadence and Cadence-UCLA flows, front-end tools (Ambit BuildGates and NCSim)  easiest proof of value: via use of regression suites Issues for METRICS constituencies to solve  security: proprietary and confidential information  standardization: flow, terminology, data management, etc.  social: “big brother”, collection of social metrics, etc. Ongoing work with EDA, designer communities to identify tool metrics of interest  users: metrics needed for design process insight, optimization  vendors: implementation of the metrics requested, with standardized naming / semantics

21. http://vlsicad.cs.ucla.edu/GSRC/METRICS