1. Software Engineering and Best Practices
Sources: Various.
Rational Software Corporation slides,
OOSE textbook slides, Per Kroll talk, How to Fail with the RUP article, textbooks
Most slides have been modified considerably

2. What is Software Engineering?
The process of solving customers’ problems by the systematic development and evolution of large, high-quality software systems within cost, time and other constraints
Note:
Process, systematic (not ad hoc), evolutionary…
Constraints: high quality, cost, time, meets user requirements

3. Analysis of the Definition:
Systematic development and evolution
An engineering process involves applying well understood techniques in a organized and disciplined way
Many well-accepted practices have been formally standardized
e.g. by the IEEE or ISO
Most development work is evolution
Large, high quality software systems
Software engineering techniques are needed because large systems cannot be completely understood by one person
Teamwork and co-ordination are required
Key challenge: Dividing up the work and ensuring that the parts of the system work properly together
The end-product that is produced must be of sufficient quality
Cost, time and other constraints
Finite resources
The benefit must outweigh the cost
Others are competing to do the job cheaper and faster
Inaccurate estimates of cost and time have caused many project failures

4. Comments: $250 billion annually in US. Over 175,000 projects!
Complexity, size, distribution, importance push our limits.
Business pushes these limits:
Great demands for rapid development and deployment
Incredible pressure to develop applications that are:
On time, Within budget, Meets the users’ requirements
Figures in the late 90s indicated that at most
70% of projects completed
Over 50% ran over twice the intended budget
$81 billion dollars spent in cancelled projects!!
We are doing something wrong as an industry!

5. What Happens in Practice
Sequential activities: (Traditional ‘Waterfall’ Process)
Requirements Design Code Integration Test
Late Design Breakage
Integration Begins
100%
Risk inadequately addressed
Process not receptive to Change
Problems not really ‘seen’
until near delivery date!
Until then, ‘all is well…’
Big Bang approach – full delivery
Long development cycles…
Little user involvement, etc. etc…
Development Progress (% coded)
Original
Target Date
Project Schedule

6. Symptoms of Software Development Problems
These symptoms are readily observable in the real world. Just like in the medical world, it is not sufficient to treat the symptoms. One must address the root causes (as shown on the next page).
An example of treating the symptoms is addressing the second bullet by insisting that no changes to requirements be accepted after a certain date. This is unrealistic. Change will happen to the business environment regardless, and the result may be that users reject the system that is built since it has not adapted to their changing needs.
Encourage student interaction on this slide. One possibility is to ask students for symptoms of problems they have observed, before you show the slide. You could then compare the students’ list to this list. Or you could show the list first and ask for additional symptoms that students have observed.
Inaccurate understanding of end-user needs
Inability to deal with changing requirements
Modules that don’t fit together (integration)
Software that’s hard to maintain or extend (brittle)
Late discovery of serious project flaws (integration)
Poor software quality (architecture, risks unanticipated…)
Process not responsive to Change (Gantt Charts…)
Unacceptable software performance (Risk, architecture, …)
Team members in each other’s way, unable to reconstruct who changed what, when, where, why (software architecture, …
…and we could go on and on…

7. Need a Better Hammer! We need a process that
Will serve as a framework for large scale and small projects
Adaptive – embraces ‘change!’
Opportunity for improvement not identification of failure!
Iterative (small, incremental ‘deliverables’)
Risk-driven (identify / resolve risks up front)
Flexible, customizable process (not a burden; adaptive to projects)
Architecture-centric (breaks components into ‘layers’ or common areas of responsibility…)
Heavy user involvement
Identify best ways of doing things – a better process – acknowledged by world leaders…

8. Best Practices of Software Engineering
Develop Iteratively
Use
Component Architectures
Manage
Requirements
Verify
Quality
Model
Visually
Control Changes
Know these!

9. Addressing Root Causes Eliminates the Symptoms
We don’t expect students to read this chart. It is an abstract view that shows the many to many relationships among the symptoms and root causes, and between the root causes and practices. It is meant to indicate that we have done the mapping, so they don’t have to.
Animation notes:
This animation is automatic. It shows the tracing from symptoms to root causes to the best practices we will discuss.
Symptoms
end-user needs
changing requirements
modules don’t fit
hard to maintain
late discovery
poor quality
poor performance
colliding developers
build-and-release
Root Causes
insufficient requirements
ambiguous communications
brittle architectures
overwhelming complexity
undetected inconsistencies
poor testing
subjective assessment
waterfall development
uncontrolled change
insufficient automation
Best Practices
develop iteratively
manage requirements
use component architectures
model the software visually
verify quality
control changes

10. Practice 1: Develop Software Iteratively
We need to keep the concept of iterative development at a very high level in this module. It is easy to get bogged down in the many ramifications of iterative development.
Try just to get the concept across here. If they would like to have more information, they can take the Rational Unified Process Overview course. Project managers might be interested in the Unified Software Project Management course.
Develop Iteratively
Use
Component Architectures
Manage
Requirements
Verify
Quality
Model
Visually
Control Changes
Considered by many practitioners to be the most significant of the six

11. Practice 1: Develop Software Iteratively
Until recently, we were developing systems under the assumption that most requirements can be identified up front.
The research deconstructing this myth includes work by Capers Jones. (See next slide) In this very large study of 6,700 projects, creeping requirements — those not anticipated near the start—are a very significant fact of software development life, ranging from around 25% on average projects up to 50% on larger ones.

13. Interestingly,
An initial design will likely be flawed with respect to its key requirements. Requirements rarely fully known up front!
Late-phase discovery of design defects results in costly over-runs and/or project cancellation
Oftentimes requirements change – during development!
While large projects are more prone to cost overruns, medium-size/small projects are vulnerable to cancellation.
The key reasons continue to be
poor project planning and management,
shortage of technical and project management expertise,
lack of technology infrastructure,
disinterested senior management, and
inappropriate project teams.”

14. Waterfall Delays Risks
Walker Royce, 1995 R I S K
Requirements
Design
Code
Waterfall risk
Integration
System Test
T I M E

15. Iterative Development
Iteration 1
Iteration 2
Iteration 3
Earliest iterations address greatest risks
Each iteration produces an executable release
Each iteration includes integration and test
Objective Milestones: short-term focus

16. Accelerate Risk Reduction
Walker Royce, 1995 R I S K
Risk reduction
Waterfall risk
Iterative
Iteration
Iteration
Iteration
Iteration
Iteration
T I M E

17. Questions – CIS 4251
How do you go about designing and developing an application?
What do you expect to get? What do you do then?
What’s next?
What is your process?
Repeatable? Predictable? How do you address change?
What does ‘risk’ mean to you?

18. Iterative Development Characteristics
Critical risks are resolved before making large investments
Initial iterations enable early user feedback
Easy to resolve problems early.
Encourages user feedback in meaningful ways
Testing and integration are continuous – assures successful integration (parts all fit)
Continuous testing.
Objective milestones provide short-term focus
Progress measured by assessing implementations
Partial implementations can be deployed
Waterfall method – no delivery
Incremental development? May be some great values in delivering key parts of application. Critical components delivered first?
No big-bang approach!

19. Iterative Lifecycle Graph
In an iteration, you may walk through all disciplines C O N T E S R U
T I M E

20. RUP Iterations and Phases
Executable Releases
Preliminary
Iteration
Architect.
Devel.
Transition
Elaboration
Construction
Inception
An iteration is a distinct sequence of activities with an established plan and evaluation criteria, resulting in an executable release.
(There is a lot of very important ‘key’ terminology used here…
(cycle, iteration, phase, milestones, core disciplines, content of iterations, etc….)

21. Problems Addressed by Iterative Development
Root Causes
Solutions
Enables and encourages user feedback
Serious misunderstandings evident early in the life cycle
Development focuses on critical issues – break it down!
Objective assessment thru testing
Inconsistencies detected early
Testing starts earlier – continuous!
Risks identified and addressed early - via planned iterations!
Insufficient requirements
Ambiguous communications
Brittle architectures
Overwhelming complexity
Subjective assessment
Undetected inconsistencies
Poor testing
Waterfall development
Uncontrolled change
Insufficient automation

22. No Free Lunch - Traps Abound…
Major impacts on Project Managers, though….
Trap: When the initial risks are mitigated, new ones emerge
Do not do just the easy stuff, to look good.
Keep re-planning based on all new information.
Trap: Remember that ‘some’ Rework enables you to enhance
your solution
Accommodate change early in the project
Trap: Iterative development does not mean never to commit
to commit to a solution
Monitor ‘scrap and rework’
Trap: Must Control “requirement creep, ” however… Some
clients will now naturally recognize many ‘musts…’

23. Many Traps in Iterative Development
Here is another trap: Too long initial iteration
Winning is fun. Winning teams work better than loosing teams
Better to have a short initial iteration, than one too long
Cut scope if necessary (much more later)
Avoid ‘analysis-paralysis’ by time-boxing; you can enhance in later iterations (more later)
Establish an even rhythm for project (at least w/i a phase)
Not completing iterations makes you lose most of the benefit
Focus on results and deliverables, not activities

24. Problem: Dangerous Iteration Patterns
Teams not synchronized -> chaos
First iteration too loaded -> panic
Too much overlap -> no feedback loop

25. Problem: Fixed Plans Produced Upfront – Not Real Practical!
Yet, senior management wants firm, fixed plans!
Part of their culture / upbringing/ experience
Necessary for ‘planning’ budgeting, etc. of resources, projects…. BUT:
Trap: Fine-grained planning from start to end
Takes too much time
Frustrating as change occurs (and it will), if plans too fine-grained.
Know that: Projects typically have some degree of uncertainty
This makes detailed plans for the entire project meaningless
Does not mean that we should not plan
Establish milestones and track progress relative to milestones

26. Solution: Plan With Evolving Levels of Detail
Coarse-grained Plan:
Software Development Plan
Fine-grained Plans: Iteration Plans
One For Entire Project
Next Iteration
Phases and major milestones
What and when Iterations for each phase
Number of iterations
Objectives and Duration
Current Iteration
Iterative does not mean less work and shorter schedule
It is about greater predictability

27. Iterations Are Time-boxed
Work is undertaken within an iteration.
The iteration plan defines the artifacts to be delivered, roles and activities.
An iteration is clearly measurable.
Iterations are risk-driven
Generally, initial iterations based on high risk and core functionalities!
This presentation does not attempt to describe in detail the iterative software development process. Good references on iterative development are contained within the Rational Unified Process itself.
Suffice it to say that RUP is an incremental process whereby the overall project is broken down into phases and iterations. The important aspects of iterative development from a plan perspective are:
The iteration plan provides a detailed description of the upcoming phase of work.
It defines the roles, activities and artifacts delivered in that iteration.
It has a very clear set of measurement criteria which can be assessed at the end (and during) the iteration.
An iteration should deliver an executable piece of software that is demonstratable and testable against the project’s requirements and use cases.
An iteration should be time boxed-meaning that it should have a definite duration; Beginning and end.
Iterations are the mechanism that the project manager uses to mitigate risk. Therefore, each iteration should reduce the overall risk to the project.

28. The Iteration Plan Defines….
The deliverables for that iteration.
artifacts
Once you have defined your coarse grained project plan, then an iteration plan is developed at a more detailed level prior to each and every iteration. In short, iteration plans are the roadmap for each iteration. Each phase has a series of milestones associated with it. Each iteration in a phase moves the project towards these milestones. An iteration has a clear objective defined and associated with that objective are the activities and deliverables required to accomplish that objective.
The to do list for the team members

29. Project progress is made against MILESTONES
Each phase is defined by a milestone.
Progress is made by passing milestones.
Milestones measure success
Phases - NOT TIMEBOXED.
Iterations ARE TIMEBOXED.
Secondly, RUP projects measure progress against defined milestones.
Each phase is defined by a milestone; These milestones are defined by the project plan (as opposed to the iteration plan).
Progress on you project is made by passing these milestones. Not passing your defined milestones is also a problem.
Unlike your iterations, phases are not time boxed; There are not defined constraints on time for your four phases.
In the end, milestones measure success! The milestones for a phase provide a focus for the iterations. The milestone helps to plan the objective of the iteration.
For example during the inception phase, one of the phase milestones is the need to understand scope. An iteration within that phase would be structured around the need to understand the scope of the system. The iteration(s) would provide the management framework for the team to explore the system boundary, implications of a possible solution and the size of that solution. The number of iterations would depend on how difficult it will be to define the scope of the project. If the scope was very hard to understand, or it could be grouped into easily defined pieces, then you may need more than one iteration. One iteration for each piece. If, as is normally the case, there is one clear piece of work to understand the scope, then one iteration would be appropriate. Judging the size and number of iterations for a project is described later in this presentation.
Elaboration
Construction
Transition
Major Milestones
Inception

30. Much more about iteration and iteration planning later in the course…
You will see some of these again – and, more importantly, use this information in your own iteration planning.