Configuration management Managing the products (code and documentation) of system as it changes

Configuration management New versions of software systems are created Enhancements / Fixes Different stages of development For different machines/OS Less likely, tailored for particular user requirements, offering different functionality Configuration management is concerned with managing evolving software systems … … will keep changes in a separate version than that shipped … will always (after initial version) have different versions in different stages of development – shipped, in testing,and/or in development … must keep track of differences between versions, make sure customers get the right versions, and must make sure changes get into all versions they should get in (and no others) System change is a team activity CM aims to control the costs and effort involved in making changes to a system

Configuration management Involves the development and application of procedures and standards to manage an evolving software product May be seen as part of a more general quality management process When released to CM, software systems are sometimes called baselines as they are a starting point for further development

System families (may be multiple versions that need to be controlled)

CM standards CM should always be based on a set of standards which are applied within an organisation Standards should define how items are identified, how changes are controlled and how new versions are managed Standards may be based on external CM standards (e.g. IEEE standard for CM) Existing standards are based on a waterfall process model - new standards are needed for evolutionary development

CM for evolutionary development (Concurrent development and testing) A time for delivery of system components is agreed A new version of a system is built from these components by compiling and linking them This new version is delivered for testing using pre-defined tests Faults that are discovered during testing are documented and returned to the system developers

Daily system building It is easier to find problems that stem from component interactions early in the process Encourages thorough unit testing - developers are under pressure not to ‘break the build’ Successful use requires stringent change management process to keep track of problems that have been discovered and repaired Produces many versions that must be managed … (deliver components that cause the whole system to fail) … (but a week ago last Thursday version can probably be discarded if there is nothing special about it)

29. 1 Configuration management planning All products of the software process may have to be managed Specifications Designs Programs Test data User manuals Thousands of separate documents are generated for a large software system … possibly project plans as well

CM planning Starts during the early phases of the project Must define the documents or document classes which are to be managed Documents which might be required for future system maintenance should be identified and specified as managed documents

The CM plan Types of documents to be managed Who takes responsibility for CM Policies for change control and version management CM records which must be maintained … Defines the types of documents to be managed and a document naming scheme Defines who takes responsibility for the CM procedures, and for the creation of baselines, and for future submissions (project manager or team leader may be responsible instead of all individuals who develop documents). Possibly, who reviews submissions Defines policies for change control and version management Defines the CM records which must be maintained – records of what has happened (audit trail) <??>

The CM plan Tools for CM CM database used to record configuration information Other … Describes the tools which should be used to assist the CM process, the process to be followed in using the tools, and any limitations on their use Defines the CM database used to record configuration information May include information such as the CM of external software, process auditing, etc.

Configuration item identification Large projects produce thousands of documents Some must be maintained for the software lifetime Document naming scheme needed Hierarchical scheme with multi-level names Large projects typically produce thousands of documents which must be uniquely identified Some of these documents must be maintained for the lifetime of the software – called formal documents or configuration items. Document naming scheme should be defined so that related documents have related names. A hierarchical scheme with multi-level names is probably the most organized approach

Configuration hierarchy <here – top level is project, next level is tools within the project, next level is modules within the tool … In this example, the last level (which will be under every part of the hierarchy) is the different types of things being managed (object descriptions (documentation), code, and tests – the actual files would be stored under these This could be done other ways (e.g. Division into Code, tests, objects could be done at the top, with each having the rest of the structure under it (e.g. Code then has project, then tool, then module, …, as does tests ) Also, Planning for re-use would suggest not organizing by project

The configuration database All CM information should be maintained in a configuration database This should allow queries about configurations to be answered What customers have a particular system version? What platform is required for a particular version? How many versions have been created? What versions are affected by a change to component X? How many reported faults in version T? The CM database should preferably be linked to the version management system for software being managed … purpose – help determine impact of system changes, help management … so that changes to the managed documents can be made to automatically update the DB

29.2 Change management Software systems are subject to continual change requests From users From developers From market forces Change management is concerned with keeping managing of these changes and ensuring that they are implemented in the most cost-effective way Change management starts when materials put into configuration management … and that changes are warranted … – probably at system test when discussing code

The change management process … see upcoming slides … could be duplicate, … or due to misunderstanding … including impacts on other parts of system … when significant changes … see upcoming slide … must be tested – and regression tested to ensure didn’t break anything else … CM team builds new version – not maintainers

Change request form Definition of change request form is part of the CM planning process Records change required, suggestor of change, reason why change was suggested and urgency of change(from requestor of the change) Records change evaluation, impact analysis, change cost and recommendations (System maintenance staff) … or could be predefined by client – e.g. government

Example Change request form Pr j ct : P r t u s / PC L-T ls N m b r: 2 3/ 9 4 re I . S mm rv il l D a 1 /1 8 R q es te d c W n c p is s el cte d f om e, d i ay f le w e i t i y se G ea A ys s da 1 0/ 2 / 9 8 C om po n e ts a f ec t d: D i s p l a y - I c o n. S ect , D sp la Ass oc d m nts : F eTa b le h an ge ss t: R iv y s im en il ta ava . q u r g d i nt ti di ie N o ass oc i ate d c om p ne n t s ar e r eq u r ed . C h an ge ri y : L w im l e me at io n: E st m a f .5 da ys D B 1 5/ 2 / 9 8 B d eci si n da te /2 A cce ha g be en R lea 2. sub mi tt e d t o QA: QA decision Date submitted to CM: C omm e n ts

Change tracking tools A major problem in change management is tracking change status Change tracking tools keep track the status of each change request and automatically ensure that change requests are sent to the right people at the right time. Integrated with E-mail systems allowing electronic change request distribution May be integrated with Configuration Management DB

Change control board Decide based on a cost, strategic and organizational viewpoint Should be independent of project team Representatives from client and contractor staff Changes should be reviewed by an external group who decide whether or not they are cost-effective from a strategic and organizational viewpoint rather than a technical viewpoint Should be independent of project team responsible for system. May include representatives from client and contractor staff. Smaller projects, fewer people (may even just be one person), less formal

Derivation history Record of changes applied to a document or code component Should record, in outline, the change made, the rationale for the change, who made the change and when it was implemented Impact in tracked item should be marked May be included as a comment in code. If a standard prologue style is used for the derivation history, tools can process this automatically … … in a document such as a requirements document or design document, changes may be marked with “Change Bars” or other markings … comments in code can flag the change at the top and at the exact change … see next slide

Component header information Note that the change code used in the change management system is referenced. When I was at IBM, we would mark this header comment with a special symbol and a change number, then any code changes would have the same special symbol and the change number, then tools can track – how many changes in different modules, which modules were impacted by specific changes. And Maintainers can so searches (grep) for the code when searching for what they did on a specific change

29.3 Version and release management Invent identification scheme for system versions Plan when new system version is to be produced Ensure that version management procedures and tools are properly applied Plan and distribute new system releases … releases are versions that will be made available to customers (there are many versions that are not)

Version identification Procedures for version identification should define an unambiguous way of identifying component versions Three basic techniques for component identification Version numbering Attribute-based identification Change-oriented identification … to be discussed in turn

Version numbering Simple naming scheme uses a linear derivation e.g. V1, V1.1, V1.2, V2.1, V2.2 etc. However, actual derivation structure may be a tree or a network rather than a sequence Names are not meaningful. Hierarchical naming scheme may be better

Version derivation structure <SKIP or breeze past>

Attribute-based identification Attributes can be associated with a version with the combination of attributes identifying that version Examples of attributes are Date, Creator, Programming Language, Customer, Status etc. More flexible than an explicit naming scheme for version retrieval; Supports queries when looking for versions Can cause problems with uniqueness Awkward - needs an associated name for easy reference … e.g. MyProd (lang=Java, platform=W2K, date=Dec1999) … and shows (probably) how version was derived – e.g. probably from the Java version in Dec1998 An important advantage of attribute-based identification is that it can support queries so that you can find ‘the most recent version in Java’ etc. <?? – shouldn’t> … biggest problem

System releases Not just a set of executable programs May also include Configuration files defining how the release is configured for a particular installation Data files needed for system operation An installation program or shell script to install the system on target hardware Electronic and paper documentation Packaging and associated publicity Systems are now normally released on CD-ROM or as downloadable installation files from the web

Release problems Release management must not assume that all previous releases have been accepted. Release management must not assume that all previous releases have been accepted. Customer may not want a new release of the system They may be happy with their current system as the new version may provide unwanted functionality Thus, All files required for a release should be re-created when a new release is installed

Release decision making Must plan when to distribute a system release Preparing and distributing a release is expensive Factors: technical quality competition, marketing requirements customer change requests … if serious bugs in the current release, a new release may need to be issued to clear them up … if a competitor has a new version out, a new release may be necessary to compete. If releases are too far apart, the competitors versions will get ahead (on the other hand if new releases are too frequent, customers may skip upgrades) Or marketing may have promised a new release by a certain date. requests for additional functionality (particularly with custom software)

Release creation Collect all files and documentation Configuration descriptions / instructions/ scripts Documented to allow re-creation Release creation involves collecting all files (code, data, documentation) and documentation required to create a system release. Files may be zipped and put in an installation program. Configuration descriptions have to be written for different hardware / O.S. and installation scripts have to be written The specific release must be documented to record exactly what files were used to create it. This allows it to be re-created if necessary. Keep versions of all source code so it can be fixed later. Document what versions of OS, compilers, libraries and other tools were used in the creation

29.4 System building The process of compiling and linking software components into an executable system Different systems are built from different combinations of components Invariably supported by automated tools that are driven by ‘build scripts’ … make (1979) was an early approach

System building Build script tells what files exist and which depend on which Correct versions of code must be obtained Code must be compiled and linked

System building problems Do the build instructions include all required components? Is the appropriate component version specified? Are all data files available? Do the build instructions include all required components? When there are many hundreds of components making up a system, it is easy to miss one out. This should normally be detected by the linker Is the appropriate component version specified? A more significant problem. A system built with the wrong version may work initially but fail after delivery Are all data files available? The build should not rely on 'standard' data files. Standards vary from place to place

System building problems Are data file references within components correct? Is the system being built for the right platform Is the right version of the compiler and other software tools specified? Are data file references within components correct? Embedding absolute names in code almost always causes problems as naming conventions differ from place to place Is the system being built for the right platform Sometimes must build for a specific OS version or hardware configuration. Sometimes, there is code that is “Conditionally compiled” depending on which platform it is being compiled for. If a flag isn’t set correctly for the compile, the built system may be for the wrong platform. Is the right version of the compiler and other software tools specified? Different compiler versions may actually generate different code and the compiled component will exhibit different behaviour. May need old version of compiler to rebuild old release

29.5 CASE tools for configuration management CM processes are standardized and involve applying pre-defined procedures Large amounts of data must be managed CASE tool support for CM is therefore essential Mature CASE tools to support configuration management are available ranging from stand-alone tools to integrated CM workbenches … and attention to detail … workbenches provide 1) help for configuration planning; 2) change management; 3) version management; 4) system building

Change management tools Change management is a procedural process so it can be modelled and integrated with a version management system Change management tools Form editor Workflow system Change database, with query facilities … to support creation of change request forms … to define who does what and to automate information transfer (e.g. handle fill in of forms, sending completed forms to appropriate person after they are completed etc (after approval send them on to the next appropriate person etc)) … that manages change proposals and is linked to a VM system to allow tiying in changes to versions

Version management tools Version and release identification Storage management. Change history recording Independent development VM controls large repository of items (code, documents). Items are “checked-out” to be worked on and checked back in – creating a new version of the item Tools provide: Version and release identification Systems assign identifiers automatically when a new version is submitted to the system Storage management. System stores the differences between versions rather than all the version code (see next slide) Change history recording Record reasons for version creation Independent development Only one version at a time may be checked out for change. Vs Parallel working on different versions, with software trying to ensure that changes don’t conflict when checked back in (tough!)

Delta-based versioning A Backward delta keeps the full copy of the latest version and revisions that were made in moving from version to version – so any older version can be re-created

e.g. Microsoft Visual Source Safe Part of Visual Studio Handles Sharing of Files in a project Check out for making changes Check back in after changes Handles Multiple Versions Saves latest Saves “Backward Deltas” Mostly makes sense with network common area available to whole team … (Like Visual C++, I.e. a separate component, not handled automatically) … check out system ensures that two users aren’t making changes to same file at same time leaving one’s changes to be accidentally lost … so older versions can be recreated (say for debugging problems with released versions) but minimum space used

Microsoft Visual Source Safe VSS keeps a database Inside DB there are Projects Projects include files Users have working folder for when they are working on something Shadow Folder – common, read only versions of current code … administrator sets up, most likely stored on a network … user can set up … user can create and add to projects … outside the DB, could be on users space on network or on own PC … so users making changes can compile the whole project to test their changes

Microsoft Visual Source Safe Check out file – copied to your working folder While checked out, other check-outs are stopped Check back in makes file available to others If not changing file, can Get / View the file (read only) Can cancel checkout if decide not to change file VSS can show differences between versions of a file VSS keeps track of versions using numbers, by date, AND by user-defined labels (names) Can try to merge different versions of a file … Other users can determine who has what they want checked out (so they can run down the hall and ask them to hurry up). inexplicably, the administrator is allowed to set up VSS so that multiple check-outs are allowed – this seems to defeat the purpose … could be useful if change went bad or if developing versions for different platforms … e.g. label – “Final Beta” … if there is a conflict between changes, VSS asks you for help. (Still, this seems dangerous to me – I’m not sure how it checks for conflicts – same section of code, same variables involved …

System building Building a large system is computationally expensive and may take several hours Hundreds of files may be involved System building tools may provide A dependency specification language and interpreter Tool selection and instantiation support Distributed compilation Derived object management … to compile and link all files … making the need for tool support a strong one … allowing to specify component dependencies, so that components only have to be recompiled if they change or a component they are dependent on changes … compilers and other tools needed selected … look for available processors on the network to share the burden of compilation <??p659>

Component dependencies This is so Unix (but then Visual Studio tries to do this for you and is too restrictive) Comp depends on scan.o syn.o etc If scan.c is changed, then – scan.c must be recompiled to produce a new scan.o, and the .o files must be re-linked to produce comp (.exe presumably). No other recompilation must be done