1. Lecture 17 Enterprise Systems Development ( CSC447 ) COMSATS Islamabad Muhammad Usman, Assistant Professor

2. Software Architecture Evaluation

3. Why Analyzing SA Marry your architecture in haste and you can repent in leisure. -Barry Boehm Being the foundation for any software system, Architecture can allow or preclude almost all system’s quality attributes. Software quality cannot be appended late in a project Proposed design may or may not be suitable w.r.t. some qualities of importance –It should be assessed before long term investment Acquiring a large software system

4. Why Analyzing SA Cost Vs Benefits Costs –Staff time –Consumption of experienced designers Benefits At AT&T over last eight years average cost savings due to full architecture review is 10% A large company avoided a multimillion-dollar purchase when architecture of global information system they were procuring was found to be incapable of providing the desired system attributes necessary to support a product line

5. Why Analyzing SA Benefits An architecture review of a retail merchandise revealed early that there would be peak order performance problems that no amount of hardware could fix; a major business failure was avoided. In the architecture review of revision control system, a number of severe limitations in achieving system portability and modifiability were uncovered

6. Why Analyzing SA Benefits –Early detection of problems –Validation of requirements Architectural reviews help uncover conflicts and tradeoffs –Improvement in architecture

7. Quality Attributes Evaluation and Quality attributes Categories –System Quality attributes discernable at runtime –System Quality attributes Not discernable at runtime

8. Quality Attributes System Quality attributes discernable at runtime –Performance –Security –Reliability –Usability

9. Quality Attributes System Quality attributes Not discernable at runtime –Modifiability –Portability –Reusability

10. Analysis/Review techniques Questioning techniques –Scenario based techniques –Questionnaire based techniques –Checklist based techniques Relationship between Scenarios, questionnaires and checklists Measuring techniques –Metrics,Simulation and Prototypes –These techniques provide quantitative results and provide answers to the questions a review team already have about particular qualities. An existing prototype or simulation may be an answer to an issue raised by a questioning technique

11. Analysis/Review techniques Classification of Techniques –Generality General purpose (Questionnaire,Metrics) or domain specific (Scenarios,checklists etc) –Level of detail Coarse (Questionnaire), medium(Scenarios) and fine(Metrics) –Phases Early, middle and post deployment –What is Reviewed Artifact (All) or Process(Questionnaire and checklist)

13. Scenarios Scenarios are brief narratives of expected or anticipated use of a system from both development and end-user viewpoint Quality attributes such as modifiability etc are abstract and vague –No universal measures for these attributes. Context dependent measures are possible –Scenarios are used to represent operational context for a system

14. Scenarios The process of choosing scenarios for analysis forces designers to consider the future uses of and changes to the system –How will this architecture accommodate the following change?

15. Scenarios Benefits of using Scenarios in analysis –Better understanding of requirements –Stakeholder buy-in –Better documentation –Requirements tractability at architectural level Specifically quality attributes

16. Scenario Based Analysis Techniques They are many –Software Architecture analysis method(SAAM) –SAAM for complex scenarios (SAAMCS) –SAAM for evolution and reusability(SAAMER) –Aspectual SAAM (ASAAM) –Architecture level Modifiability Analysis (ALMA) –ATAM,CBAM,ARID,PASA,CBAR….

17. Software Architecture Analysis Method (SAAM)

18. SAAM

19. Scenarios and their Evaluation Develop task scenarios that illustrate the kinds of activities the system must support and the kinds of changes which it is anticipated will be made to the system over time. Scenario evaluations : Direct/Indirect scenarios –For each indirect scenario, list the changes to the architecture that are necessary for it to support the scenario and estimate the cost of performing the change. –A modification to the architecture means that either a new component or connection is introduced or an existing component or connection requires a change in its specification. –For each indirect scenario the impact, or set of changes, that scenario has on the architecture should be described

20. Scenarios and their Evaluation Scenario Interaction –Different indirect scenarios may necessitate changes to the same components or connections Scenarios interact in that component on connector –Scenario interaction measures the extent to which the architecture supports an appropriate separation of concerns –SAAM favors the architecture with the fewest scenario conflicts

21. Overall Evaluation Finally, weight each scenario and the scenario interactions in terms of their relative importance. Determine an overall ranking. This is a subjective process involving all of the stake-holders in the system. The weighting chosen will reflect the relative importance of the quality factors that the scenarios manifest

22. Validation of SAAM Global information system A company was contemplating the purchase of a system as the infrastructure to support applications development for multimedia communication with unlimited conferencing. The purchasing company wanted some assurance that the architecture of the system they purchased was going to provide for the generic satellite-based multi-user applications they anticipated developing in the near and long term. As a result of the analysis, the company decided to not purchase the system, avoiding an investment of tens of millions of dollars.

23. Validation of SAAM Air traffic control This was an investigation of a complex, real-time system against a set of proposed changes to that system. The purpose of the evaluation was to determine whether future development on this system was justified. The change scenarios were intended to represent appropriate manifestations of the abstract qualities of performance and availability. The result of this evaluation was a decision to proceed with the proposed changes

24. Validation of SAAM WRCS This case study was an analysis of a commercial version control/configuration management tool This analysis covers all activities of SAAM and shows all artifacts of a SAAM evaluation that can be produced. The result of the analysis was that significant problems were discovered with the product’s architecture, with respect to the scenarios considered

25. WRCS WRCS provides the functionality to allow developers of projects the ability to create archives, compare files, check files in and out, create releases, back up to old versions of files, and so on. “Project'' in this context means any group of related files that, when linked together appropriately, form a finished product. WRCS keeps track of changes made to these files as they evolve over time It provides capabilities for multiple users to work on the same project within their own private work areas,

26. Applying SAAM Steps Develop Scenarios/Describe Candidate Architecture

27. User: 1. Compare binary file representations. Compare binary files generated by other products. For example, FrameMaker files are stored in a binary representation. But when we are comparing two versions of a FrameMaker file we want to see our editing changes in a human-readable form, and not the changes to the binary codes stored in the files. 2. Configure the product's toolbar. Change the icons and actions associated with a button in the toolbar. Maintainer: 3. Port to another operating system. 4. Make minor modifications to the user interface. Add a menu item, change the look and feel of a dialog box. Administrator: 5. Change access permissions for a project. 6. Integrate with a new development environment. Attach for example to Symantec C++.

28. Perform Scenario Evaluation

29. Reveal Scenario Interaction

30. Fish eye diagram of Scenario Interaction

31. Overall Evaluation Prioritize the scenarios which have been identified as potentially problematic The WRCS analysis identified a number of severe limitations in achieving the extra- functional qualities of portability and modifiability. A major redesign of the system was recommended. Having gone through an analysis procedure such as SAAM before implementation would have substantially contributed to avoiding the problems which the WRCS developers now face

32. Results and Lessons SAAM is for people SAAM and traditional architectural metrics Determining the proper level of architectural description Determining the proper set of scenarios.