1. Codeline Management for Evolutionary Development Anders Johnson WIS Technologies

2. Evolutionary Development Traditional (“big-bang”)Evolutionary 100% Time → Completion → 0% 100% Time → Completion → 0% Features Quality

3. So What?  “The literature” says that it leads to: Increased productivity Decreased risk Higher quality  Simultaneously  But be realistic Unlikely to halve the schedule on first try

4. Automated Regression  Yes, you need it!  Near-production test after every mainline change No way you’re going to do that manually! Test at various levels of integration  Don’t need to worry about breaking anything that the tests cover

5. Who Farted?  Sync’ing-in broken changes is extremely counterproductive  Impossible to test unrelated changes  But, you still need to share some changes before they’re functional

6. Branches or Labels?  Branches (a.k.a. “codelines”) are better for evolutionary development  Labels often fail to converge: Time → OK (?) Feature A Broken OK (?) Feature B Broken Feature C Broken L “OK” Label

7. Development Codelines  Contain changes that aren’t expected to work  Multiple contributors can exchange early changes  Use the mainline instead if you require something that actually works  Integrate into mainline when it works

8. Distinguishing Codelines  Each codeline has its own regression (or lack thereof) May or may not be driven by a submit trigger  Each codeline has its own default view Client generation is scripted, such that the view is set automatically Codeline-name@change describes a source configuration definitively

9. Reining in the Chaos  Each workspace is based on exactly one codeline  Each codeline (except the mainline) has exactly one parent codeline  Workspace location corresponds exactly to depot location (relative to codeline root)

10. //depot top file1 dir file2 branch1 top file1 dir file2 branch2 top file1 dir file2 Depot Structure TRUNK branch1/branch2/

11. Extension Fields  Extra fields (such as client’s codeline and branch’s parent codeline) piggy- back onto the “Description” field: Client: anders_a4 Description: Created by anders. ----- Please do not delete these lines from the Descripti on field ----- %Branch: a4/ %Location: //depot/a4/top Root: /home1/anders/anders_a4

12. Codeline Configuration  Each codeline’s regression and default view is stored in a designated location in the file tree CFG/codeline-name/top/ Changes propagate automatically  Regression changes should be reviewed “The regression failed, so I disabled it.”

13. Sequential Regression  If there is a regression, then every submission must be strictly sequential  Submitter’s workspace (including unchanged files) must be up-to-date  Otherwise, the regression might pass even if the depot would be broken  Doing this reliably requires locking

14.  Symbolic link in each workspace directory pointing to workspace root  Portable among workspaces  Can often be used where an absolute path would normally be required.p4top % cd anders_a4/foo/bar; ls –Alo lrwxrwxrwx 1 anders 5.p4top ->../..

15. Staying in Sync  Files can differ even if Perforce thinks that there’s nothing left to merge  For fully-merging codelines: Integrate only between parent and child Child must contain all of parent’s changes Then, p4 integrate -dft and p4 resolve –at into parent  Not for vendor & release codelines

16. Switching Codelines  Changes sometimes wind up in the wrong workspace  To avoid stranding them, need to create and apply patch files

17. Hierarchical Regression  Every codeline can be regressing simultaneously TRUNK (4 hr) Feature A/ (3 min) Development A1/ (no regression) Development A2/ (no regression) Feature B/ (3 min) Feature C/ (3 min) Development C1/ (no regression)

18. Reference Builds  We “cheat” by running regressions inside the submitting workspace  This occasionally causes a false positive or false negative: Environmental dependencies Build system deficiencies Failure to p4 add

19. Reference Builds (contd.)  Periodically clean-build as a pseudo- user with a baseline environment  Regression should be deterministic  Results can be cloned into new workspaces (in conjunction with p4 flush ) to bypass clean-building It’s practical to have tools under Perforce

20. Concurrent Related Projects  SURPRISE: Inter-file branching is not a good general solution for this Relies on merge-at-destination, which is bad because the originator is usually better equipped to merge Results in duplicated merging effort, especially if there are more than 2 projects

21. Concurrent Projects (contd.)  Instead, make all the projects visible in the same tree, and share as much as you can: Symbolic links #ifdef Design patterns Etc.  Works whether or not you branch

22. More Sharing Techniques  Decompose to isolate variations  Override files using a search path  Read or #include from same location  Filter (e.g. Perl) to generate variations  Leave unused code as-is  Make the requirements more similar  Inter-file branching (as a last resort)

23. Concurrent Projects (contd.)  Copy-and-paste is bad code economy, and it’s essentially irreversible  Indiscriminant inter-file branching is only somewhat better  Try sharing first You’ll like it If not, it’s relatively easy to branch later

24. Concurrent Projects (contd.)  Conceptually, you now have a single hierarchically regressed “super-project” TRUNK Project I/ Feature A/ Development A1/ Project II/Project III/ Feature B/Feature C/ Development C1/ Feature X/

25. Interoperability Testing  Requires multiple versions in view  Avoid “mixed” views Difficult to reproduce  Instead, p4 integrate proj/...@1234 \ proj/1234/...

26. Forward Integration Failures  In case of conflicting assumptions, determine which codeline is “at fault” (WARNING: may involve politics), and fix it  In case of complex merging, submit extra merging changes to a grandchild codeline, and integrate back to child

27. A4 – Codeline Automation  Perl, mostly object oriented  Reusable modules P4 form manipulation P4 command result parsing  ~5 person-months, ~22,000 lines 55% Documentation and test code  http://a-4.sourceforge.net http://a-4.sourceforge.net

28. Conclusions  Evolutionary development is good  Use hierarchically regressed codelines to sustain incremental progress  Use a shared code base for concurrent related projects  Use A4 to automate codeline management