METRICS: A System Architecture for Design Process Optimization Andrew B. Kahng and Stefanus Mantik* UCSD CSE Dept., La Jolla, CA *UCLA CS Dept., Los Angeles, CA

Motivations How do we improve design productivity ? Does our design technology / capability yield better productivity than it did last year ? How do we formally capture best known methods, and how do we identify them in the first place ? Does our design environment support continuous improvement of the design process ? Does our design environment support what-if / exploratory design ? Does it have early predictors of success / failure? Currently, there are no standards or infrastructure for measuring and recording the semiconductor design process

Purpose of METRICS Standard infrastructure for the collection and the storage of design process information Standard list of design metrics and process metrics Analyses and reports that are useful for design process optimization METRICS allows: Collect, Data-Mine, Measure, Diagnose, then Improve

Related Works OxSigen LLC (Siemens 97-99) Enterprise- and project-level metrics (“normalized transistors”) Numetrics Management Systems DPMS Other in-house data collection systems  e.g., TI (DAC 96 BOF) Web-based design support  IPSymphony, WELD, VELA, etc. E-commerce infrastructure  Toolwire, iAxess, etc. Continuous process improvement Data mining and visualization

Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

Potential Data Collection/Diagnoses What happened within the tool as it ran? what was CPU/memory/solution quality? what were the key attributes of the instance? what iterations/branches were made, under what conditions? What else was occurring in the project? spec revisions, constraint and netlist changes, … User performs same operation repeatedly with nearly identical inputs  tool is not acting as expected  solution quality is poor, and knobs are being twiddled

Benefits Benefits for project management  accurate resource prediction at any point in design cycle up front estimates for people, time, technology, EDA licenses, IP re-use...  accurate project post-mortems everything tracked - tools, flows, users, notes no “loose”, random data left at project end  management console web-based, status-at-a-glance of tools, designs and systems at any point in project Benefits for tool R&D  feedback on the tool usage and parameters used  improve benchmarking

Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

METRICS System Architecture Inter/Intra-net DB Metrics Data Warehouse Web Server Java Applets Data Mining Reporting Transmitter wrapper Tool Transmitter API XML

METRICS Performance Transmitter  low CPU overhead multi-threads / processes – non-blocking scheme buffering – reduce number of transmissions  small memory footprint limited buffer size Reporting  web-based platform and location independent  dynamic report generation always up-to-date

Example Reports hen 95% rat 1%bull 2% donkey 2% % aborted per machine % aborted per task BA 8% ATPG 22% synthesis 20% physical 18% postSyntTA 13% placedTA 7% funcSim 7% LVS 5% LVS convergence time LVS %

Current Results CPU_TIME = NUM_CELLS (corr = 0.93) More plots are accessible at 

Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

METRICS Standards Standard metrics naming across tools  same name  same meaning, independent of tool supplier  generic metrics and tool-specific metrics  no more ad hoc, incomparable log files Standard schema for metrics database Standard middleware for database interface For complete current lists see:

Generic and Specific Tool Metrics Partial list of metrics now being collected in Oracle8i

Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

Testbed I: Metricized P&R Flow Placed DEF QP ECO Legal DEF Congestion Map WRoute Capo Placer Routed DEF CongestionAnalysis Incr. WRoute Final DEF METRICSMETRICS LEF DEF

Testbed II: Metricized Cadence SLC Flow DEF Placed DEF QP Pearl METRICSMETRICS QP OptCTGen Incr. Routed DEF WRoute Optimized DEF LEF GCF,TLF Clocked DEF Constraints

Outline Data collection process and potential benefits METRICS system architecture METRICS standards Current implementation Issues and conclusions

Conclusions Current status  complete prototype of METRICS system with Oracle8i, Java Servlet, XML parser, and transmittal API library in C++  METRICS wrapper for Cadence and Cadence-UCLA flows, front-end tools (Ambit BuildGates and NCSim)  easiest proof of value: via use of regression suites Issues for METRICS constituencies to solve  security: proprietary and confidential information  standardization: flow, terminology, data management, etc.  social: “big brother”, collection of social metrics, etc. Ongoing work with EDA, designer communities to identify tool metrics of interest  users: metrics needed for design process insight, optimization  vendors: implementation of the metrics requested, with standardized naming / semantics

Status of METRICS System Current working METRICS systems are installed within Intel and Cadence METRICS system works properly inside a laptop

Flow Metrics Tool metrics alone are not enough  Design process consists of more than one tools  One tool is run multiple times  Design quality depends on the design flow and methodology (the order of the tools and the iteration within the flow) Flow definition  Directed graph G (V,E), V  T  { S, F }, T  { T 1, T 2, T 3, …, T n } (a set of tasks), S  starting node, F  ending node, E  { E s1, E 11, E 12, …, E xy } (a set of edges)  E xy x < y  forward path x = y  self-loop x > y  backward path

Flow Example S T1T1 T2T2 T3T3 T4T4 F Optional task Task sequence: T 1, T 2, T 1, T 2, T 3, T 3, T 3, T 4, T 2, T 1, T 2, T 4 S T1T1 T2T2 F T1T1 T2T2 T3T3 T3T3 T3T3 T4T4 T2T2 T1T1 T2T2 T4T4

Flow Tracking S T1T1 T2T2 F T1T1 T2T2 T3T3 T3T3 T3T3 T4T4 T2T2 T1T1 T2T2 T4T4 Task sequence: T 1, T 2, T 1, T 2, T 3, T 3, T 3, T 4, T 2, T 1, T 2, T 4

Multilevel FM Partitioning Experiment Given: initial partitioning solution, CPU budget and instance perturbations (  I) Find: number of parts of incremental partitioning and number of starts  T i = incremental multilevel FM partitioning  Self-loop  multistart  n  number of breakups (  I =  1 +  2 +   n ) S T1T1 F T2T2 T3T3 TnTn...

Multilevel FM Experiment Flow Setup foreach testcase foreach  I foreach CPU budget foreach breakup I current = I initial S current = S initial for i = 1 to n I next = I current +  i run incremental multilevel FM partitioner on I next to produce S next if CPU current > CPU budget then break I current = I next S current = S next end

Flow Results If (27401 < num edges  34826) and ( < cpu time  ) and (perturbation delta  0.1) then num_inc_parts = 4 and num_starts = 3 If (27401 < num edges  34826) and (85.27 < cpu time  ) and (perturbation delta  0.1) then num_inc_parts = 2 and num_starts = 1... Actual CPU Time (secs) Predicted CPU Time (secs)

Wireload Model Flow WLM flows for finding the appropriate role of WLM  T 1 = synthesis & technology mapping  T 2 = load wireload model (WLM)  T 3 = pre-placement optimization  T 4 = placement  T 5 = post-placement optimization  T 6 = global routing  T 7 = final routing  T 8 = custom WLM generation Post-placement and pre-placement area important steps Choice of WLM depends on the design S T1T1 T2T2 T3T3 T4T4 F T5T5 T7T7 T8T8 T6T6

Datamining Integration Database Datamining Tool(s) Datamining Interface Java Servlet Java Servlet SQL Tables Results DM Requests Inter-/Intranet

Example Applications with DM Parameter sensitivity analysis  input parameters that have the most impact on results Field of use analysis  limits at which the tool will break  tool sweet spots at which the tool will give best results Process monitoring  identify possible failure in the process (e.g., timing constraints are too tight, row utilization is too high, etc.) Resource monitoring  analysis of resource demands (e.g., disk space, memory, etc.)

DM Results: QPlace CPU Time Actual CPU Time (secs) Predicted CPU Time (secs) If (num nets  7332) then CPU time = num cells num nets num pads num fixed cells If (num overlap layers = 0) and (num cells  71413) and (TD routing option = false) then CPU time = num nets num cells num fixed cells - num routing layer...