CSC444F'05Lecture 71 Source Control & Build

CSC444F'05Lecture 72 Source Control THE most important tool for commercial software development. Source –as opposed to generated intermediate or final files –code + docs + anything else Maintains a central copy of version-controlled files –the “depot” or “repo” Server-class hardware Clients: –sync-out –change locally –check-in Want: –atomic check-in –efficient file storage –low network use –meta-data Server RAID disk storage tape backup Client local copy of depot Client local copy of depot Client local copy of depot depot

CSC444F'05Lecture 73 Want Powerful Branch / Merge Must branch 1000’s of files at once. Should be efficient –Should use COW Imagine find and fix a bug in file,v3 to generate file,v4 Tools required to analyze the delta and automatically suggest an appropriate change to branch/file,v2 –Method 1: use file,v3, file,v4 and branch/file,v2 –Method 2: also use file,v1 and all other changes check-in file, v1file, v2file, v3 branch/file, v1branch/file, v2 branch operation virtual copy check-in physical copy stored as differences with 'file, v1' merge bug fix Δ Δ Δ’Δ’

CSC444F'05Lecture 74 Want Powerful Labeling Allows for a set of files, each at their own revision to be labeled (e.g., as “R3.2.3”) Want to sync to a label Want to retroactively branch at a label

CSC444F'05Lecture 75 Uses For Source Control Repository –always know where to find the most up-to-date versions of any files Structure –Structure of depot defines module structure of software History –Complete history of changes: who, what, when, and how good for tracking down tough bugs good for discovering how code got to be the way it is Control –Management can control what goes into the source –Can see the changes / review them –Can apply check-in policies Collaboration –can sync to latest –can choose not to sync while working –a sub-group can stay in sync Multiple Maintenance Streams –enabled via branch & merge Reproducible System State –can go back to a previous release and test bugs from the field –i.e., if no longer reproducible in most current version what fixed it? want to know! –can issue patch releases Coder/Build Communications –more next slide…

CSC444F'05Lecture 76 Coder / Build Team Communications Want a hard and fast rule: –nothing ever gets shipped without: coder checks in build team checks out Counter-example: –in a very tense moment, the coder has on her desktop the dll that will solve the customer’s problem temptation to “just ship it” to them say they do… –after 24 hrs straight, coder goes home –coder comes back, continues to make changes to the file, then checks it in –1 week later customer has a similar problem –try to reproduce, cannot –try to go back to customer’s exact source code cannot!

CSC444F'05Lecture 77 Codeline Policy To use a sophisticated sccs must establish codeline policies. Codeline policy –specifies the conventions whereby the sccs is used to maintain multiple code streams –specifies the rules regarding who can check things in, when, and what kinds of things. No one codeline policy Here’s mine: –main –maintenance –shipping –private

CSC444F'05Lecture 78 Main Codeline Consistently used for ongoing development, across all releases Any developer may check-in changes The only codeline into which new features are coded –and only at certain times between fork and dcut Defect corrections are coded in at any time –all defect corrections must be coded here –of any priority except when management overrides When sufficiently stable after dcut, and when new feature work must begin on the next release –a maintenance codeline is branched off the main codeline –new feature work may then begin again in main. –timing of this is critical too soon: fix too many defects in one extra place too late: new feature work goes in private codelines that do not incorporate defect corrections – an awkward, time-consuming merge later

CSC444F'05Lecture 79 Maintenance Codelines Branched from main when getting ready to start new features on the next release Only defect corrections All defect corrections into this codeline if active and not retired Any developer may check-in As point releases get closer and recede, codeline policy for defect priority changes: –only higher priority defects as point release gets close –any priority defect after shipping codeline is branched

CSC444F'05Lecture 710 Shipping Codelines Branched from an active maintenance codeline a day or two before a point release is scheduled to go out. Only build team members may check-in changes –no developers allowed –dev team member will fix in main and maintenance –build team member will pull the fix from maintenance and integrate into shipping. Developer will look over their shoulder anxiously! Only the highest priority, most critical, last minute, defect corrections allowed into this codeline Codeline required to promote stability immediately before a shipment –right before ship, a lot of work has gone into validating the build –do not want to negate that work for the sake of low priority fixes may never get it out!

CSC444F'05Lecture 711 Private Codeline Owned by an individual or a sub-group of developers Free to check in new features Free to, but not required to, check in defect corrections May be only a part of the source –the other codelines are the whole source used for feature work occurring outside of fork-dcut will eventually be merged back into main

CSC444F'05Lecture 712 Example main maintenance R1 shipping R1.0 shipping R1.1 maintenance R2 shipping R2.1 retired maintenance R3 shipping R3.0 shipping R1.1 shipping R2.2 activeongoingactive X private big new feature shipping R2.0 R2.2.a

CSC444F'05Lecture 713 Builds & Installs Build: –running build tools against the source files to create executables other release support files (e.g., static database files, conf files, ….) –end result: set of files that may be conveniently used to run the app Nest step: install –starts with the built files + installer sources –results in an installer image –typically burned onto optical media or prep’d for download –one click install / de-install by end-users –either complete releases patches that bring the software from one release to a higher one All files used to support build/install must be in source code control

CSC444F'05Lecture 714 Automation As few manual steps a possible in build or install image creation Creates exposure: –single individuals –can mitigate via documentation, but gets out of date fast and no guarantees of accuracy –should “document” build steps in a script that is used to build! Manual steps are error prone: –lots of details to consider –do no want to mix up files from different codelines –introduces last-minute, untested-for errors Consistent builds: –developers and qa use same build / install steps –eliminates problems due to inconsistent buidls

CSC444F'05Lecture 715 Development Builds A full install is too cumbersome for coders / testers to deal with all the time Therefore use a tiered approach: –tier 1: “dev debug build” that includes a mini “dev debug install”. –tier 2: “dev release build” that includes a mini “dev release install”. –tier 3: “production build” that includes a full “production install”. Dev builds have lots of debug info turned on –never shipped because inefficient –if bug manifests in dev build, generally easy to track down and fix if not, then will in a “dev release” build –same as “dev debug build”, but with certain debug switches turned off –still not a “production build” If not in “dev release” then must resort to cumbersome “full production” –will usually be an installer problem Be builds are very convenient to work with –change code, push button to build/install, test Developers will maintain the dev build / install scripts QA will use dev release builds / install to test on –Automated regression testing also

CSC444F'05Lecture 716 Production Builds Step 1 is usually a dev release build –Carries on to create an installer image –Carries on to create a CD ISO image or downloadable image These are shipped out the door –a final manual release checklist is gone through auto-testing is good, but want humans involved also! Company maintains a list of all future scheduled production builds –“gold” builds intended to go out to customers –intermediate builds leading up to the “gold” List stored in a workflow management system –processes followed –data recorded Weekly company meeting to discuss status and priorities on production builds –what points, what patches, what versions, …

CSC444F'05Lecture 717 Automated Builds Coders should integrate their work as often as possible with that of other coders. –“continuous integration” If working on a stable base for a while (necessary to preserve sanity): –before checking in changes, check out all other’s changes, build and do one last test. –race condition Also, tested on dev platform, not on all versions – may have broken a different version Nightly automated builds –scripted process fires off –deletes all previous source, intermediate, and final files –returns system to pre-install state (e.g., delete any registry entries) –does a complete check out of latest sources –runs a build – records results in a database errors and warnings should be at 0 –does an install – records results in a database –sees if app starts up – records results in a database –across all supported versions on all platforms –across all active codelines Build a web app to report on database Someone should check each morning and fix any build problems immediately, as a top priority Should mercilessly tease developers who “break the build”.