1. Agile Development & Companies
Harvard Business Review: “Why the lean start-up changes everything”
Agile development applied to entire business of start-ups
Simplest possible prototype!
Iterate!
Image: leadnet.org

2. Case Study:
Case Study #1

3. Background My PhD thesis: new CAD System for FPGAs
Formed company to commercialize
4 people initially
An early customer was Altera
10 months to produce a new “placement and routing” system for Altera’s chips
Aggressive goal: 10X better than current system

4. Place and Route Example

5. Project Management Quick plan Concrete milestones Scared
1 day: estimate all tasks & time
Add time margin (mine: 50%)
Concrete milestones
Meet a milestone every 3 months or don’t get paid!
Scared
Turned out to be helpful!
Avoid any side projects, unnecessary complexity
Get something working, measure & improve
Measure our software vs. Altera’s every day

6. Outcome Hit all milestones, except first (2 weeks late)
Focused!
CAD system exceeded expectations
30X less runtime
38% faster circuits
Altera replaced their P & R engine with the prototype
Started simple, measured where to improve
Some simple algorithms still in current Quartus II  didn’t need more!

7. Case Study 2: (First Version of) Quartus

8. Background Altera had highly successful CAD system: Max+PlusII
Decided to do a complete re-write to a new CAD system
Quartus
Goals:
C++ (not C)
Cleaner, more extensible code
Allow multiple engineers to collaborate on a project easily
Allow fast, “incremental” recompiles

9. Project Management Planned for a year with no coding
Too much waterfall  planning paralysis
Let complexity reign
Fancy user features
Fancy language (object oriented C++ to the max)
No working prototype for ~2 years
Can’t test new features in full system
Didn’t understand & measure key metric
CAD system that optimizes well
And is ready for next-generation chip

10. Outcome Software not ready in time for chip Software rushed to market
Not stable enough, didn’t optimize well enough
Very bad customer experiences
Lost sales: $billions!
Rewrote & renamed later versions:
Now very good!

11. Case Study 3: Hardware + Software (Stratix V)

12. State of the art for 2 years!
Big Project!
More planning?
3 year project
~400 engineers at peak
Yes, plan as far ahead as you can
But don’t paralyze yourself
Accept that plan gets coarse as you go out in time
Key plan: measurable milestones
3 year project  need to break up schedule
Project milestones  goals / milestones per team
Clear must have features
Everything else: nice to have
> $200 M
State of the art for 2 years! +

13. Agile / Iterative Development?
Build prototype / model of HW + SW system
Use to evaluate and test new ideas
Done by smaller team (~10 engineers)
Will never ship to customers!
Prototype shows good results?
Build real hardware & full software
Team grows to 400
Always keep system working as you improve
Always measure results vs. expected (from prototype)
HW & SW both in source code control
Still iterative, but with more structure

14. Communication? Weekly status meeting Meeting style
For sub-teams (~8 engineers)
Managers discuss results for ~8 teams (~64 people)
Directors meet to discuss overall project status
Meeting style
wiki, crisp reporting
Big picture  progress toward milestones

15. Outcome Iterative development of HW + SW
Works much better than pure waterfall
Many companies: out of business by not getting full HW + SW system working
Every 2nd generation: usually late!
Engineers & marketing put in too much complexity
Not scared enough, not enough iterative development!
Late to market is terrible
Competitor has twice the transistors, 1 year earlier
Want to optimize, but on time more important

16. Testing

17. Testing Philosophy “If you didn’t test it, it doesn’t work”
Assume all code broken until proven otherwise
Look at what the program does not what it is supposed to do
Scientist: you are testing hypotheses about program
Think of every case your program should handle  cover them all

18. Testing Time Develop tests as or before you code
Test-driven development
Prototype
Refine
Test & Evaluate

19. Test-Driven Development
The tests are the detailed specification
And now can be executed, not just read
More tests
Fewer specification documents

20. Types of Tests End-to-end (system) tests Tests like an end user
Hard to debug a failure  problem could be anywhere!
Fragile: often test some exact user input would produce some exact output text / file
End to End Tests
Unit Tests
Test small pieces of system (single classes / functions)
Easier to debug failure  start with 1 class / function
Don’t check user interface (I/O): check API / class does what it should with code (testfixture)
Unit Tests

21. Types of Tests Fewer Integration Tests Bigger unit tests
(but not 0!)
End to End Tests
Integration Tests
Bigger unit tests
But now test multiple classes / bigger functionality
Moderate difficulty to debug
Integration Tests
Unit Tests
More

22. Testing Automation You will re-test often System tests: save in files
Every time you change the program
How to speed up?
System tests: save in files
Use file redirection
prog.exe < in.txt
diff out.txt out_good.txt // matched known good result?
Less fragile
Write validity checker instead of checking exact output match
Write scripts to run all tests and check results
> out.txt

23. Unit Tests Want: No! To test individual classes / functions How?
Carefully chosen user input?
Output ...
No!
struct Point {
float x;
float y;
t_point& operator*=(float rhs);
. . .

24. Unit Tests Write new code To put class in right state
To directly send it some input / make some calls
To immediately check the responses
Test driver

25. Unit Tests myCode.cpp struct Point { float x; float y;
t_point& operator*=(float rhs);
. . .
tester.cpp
int test1 () {
Point testme (1, 2);
testme *= 3;
if (testme != Point (3, 6)) {
cout << “uh – oh” << endl;
return (1); }
return (0);
test_main.cpp
g++ myCode.cpp main.cpp –o prog.exe
int main () {
int error = 0;
error += test1();
. . . // Lots more tests
g++ myCode.cpp tester.cpp
test_main.cpp –o test.exe