1. Productive Engineering Teams
Carnegie Mellon University
Software Engineering Institute
Productive Engineering Teams
Watts S. Humphrey
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA
Sponsored by the U.S. Department of Defense
© 2000 by Carnegie Mellon University 1

2. The Improvement Strategy
Management comes first: without sound management
cost and schedule performance is unsatisfactory
quality engineering work is unlikely
With the CMM, management performance improves.
Cost and schedule commitments are met.
Processes are defined and data gathered.
Quality engineering work may or may not be done.
The PSP and TSP provide disciplined engineering.
Engineers use defined processes and data.
With sound management and disciplined practices, quality engineering is standard.

3. Objectives Engineering organizations strive to produce
quality products
on aggressive schedules
for lowest cost
To do this, they need motivated and productive teams.
The methods for building such teams are well known but not obvious.
This talk describes how to build, manage and motivate productive engineering teams.

4. Characteristics of Productive Teams
are attacking important problems
have aggressive but realistic goals
work to defined plans and processes
The team members
are skilled and trained for the job
know their roles and responsibilities
are personally committed to the work
The Team Software Process (TSP) builds productive teams. SM SM
Team Software Process, and TSP are service marks of Carnegie Mellon University.

5. Costs and Benefits - 1 The benefits of the TSP are substantial.
Productivity has more than doubled.
Teams have been consistently on or ahead of schedule.
Testing time was cut by 10 or more times.
Delivered products have been defect free.
Engineering turnover has been zero.

6. Costs and Benefits - 2
Results from three projects in one company show that
Test defects were reduced from an average of 20/KLOC to 1/KLOC.
At an average cost of 12 hours per defect, the engineers paid for the TSP investment with the first 1000 LOC written.
By using TSP, the company estimates they have saved an estimated $5.3 Million in two years.
Customer acceptance test time was reduced from many months to a few weeks.

7. Costs and Benefits - 3 Engineers like working on productive teams:
This really feels like a tight team.
This process forces you to design, to think the whole thing out.
Design time is way up but code time decreased to compensate.
Tracking your time is an eye opener.
Really good teamwork on this project - no duplication of effort.
I’m more productive.
Gives you incredible insight into project performance.
Wonderful to have team members assigned specific roles.
Team really came together to make the plan.
I feel included and empowered.

8. Costs and Benefits - 4 The introduction costs for TSP are significant.
Seminars and courses
executives - 1 1/2 days
managers - 4 days
engineers to 150 hours
instructors/coaches - engineer training plus 2 weeks
team launches, relaunches, and coaching
Culture change
executives - leadership and participation
managers - coaching and support
engineers - commitment and ownership

9. The CMM and the TSP
The SEI has addressed process improvement from two perspectives.
The CMM® establishes the management framework.
policies and practices
systems, methods, and facilities
The TSP shows engineers and their managers how to
use a defined, measured, and planned process
establish the conditions for effective teamwork
manage, track, and report on the work
® Registered with the U.S. Patent and Trademark Office.

10. CMM, TSP & PSP Relationship
CMM - Builds organizational capability
TSP - Builds quality products on cost and schedule
PSP - Builds individual skill and discipline

11. The CMM KPAs and TSP/PSP
Level
Focus
Key Process Areas (KPA)
Software Project Planning
Software Project Tracking
Defect Prevention
Technology Change Management
Process Change Management
Quantitative Process Management
Software Quality Management
Organization Process Focus
Organization Process Definition
Integrated Software Management
Software Product Engineering
Peer Reviews
Indicates the CMM KPAs that are fully or partially addressed at the personal level in the PSP
5 Optimizing
Continuous Process
Improvement
Requirements Management
Software Project Planning
Software Project Tracking
Software Quality Assurance
Software Configuration Management
Software Subcontract Management
Defect Prevention
Technology Change Management
Process Change Management
Quantitative Process Management
Software Quality Management
Organization Process Focus
Organization Process Definition
Training Program
Integrated Software Management
Software Product Engineering
Intergroup Coordination
Peer Reviews
4 Managed
Product and Process
Quality
3 Defined
Engineering Process
Indicates the CMM KPAs that are fully or partially addressed at the team level in the TSP
Requirements Management
Software Quality Assurance
Software Configuration Management
Intergroup Coordination
2 Repeatable
Project Management

12. TSP Teams Need Proper Training
The PSP
Course
The TSP
Launch
The TSP
Process
Project goals
Team roles
Team process
Project plan
Balanced plan
Risk analysis
Team communication
Team coordination
Status tracking
Project reporting
Personal measures
Process discipline
Estimating & planning
Quality management
Team
Management
Engineering
Skills
Team
Disciplines
A Productive
Engineering Team

13. Introducing Engineering Disciplines
Until they try it, most people do not believe disciplined engineering will work for them.
They won’t try it without evidence.
They can’t get evidence without trying it.
The TSP engineering course is called the PSP (Personal Software Process).
Engineers gather data while writing 10 programs.
They start with their current methods.
They advance in steps to the full PSP.
They see from their personal data that sound engineering works. SM SM
Personal Software Process and PSP are service marks of Carnegie Mellon University.

14. The Engineering Course Overview
The course focuses on building engineering disciplines.
to consistently meet commitments
to measure and manage quality
to understand personal performance
The course instructors have all completed the engineering course themselves.
The students must be competent software engineers.

15. Building the Team Environment
Engineers take the PSP/TSP training as teams.
This starts the teambuilding process.
Shortly after completion, the trained teams launch their projects.

16. The Project Launch TSP projects can start on any phase.
Each phase starts with a launch or relaunch step.
The strategy is to
overlap phases
develop in increments
balance team workload
accelerate tasks wherever possible
Postmortem
Requirements
Launch
High-Level
Design
Relaunch
Implementation
Integration
and Test

17. The TSP Launch Every TSP project starts with a launch.
The launch takes 4 days.
It is part of the project.
It is led by a trained team coach.
It immediately follows PSP training.
In the launch, the engineers do essential project work.

18. The Launch Process Meetings
Day 1
Day 2
Day 3
Day 4
1. Establish
Product and
Business
Goals
4. Build Top-
down and
Next-Phase
Plans
7. Conduct
Risk
Assessment
9. Hold
Management
Review
2. Assign Roles
and Define
Team Goals
5. Develop
the Quality
Plan
8. Prepare
Management
Briefing and
Launch Report
Launch
Postmortem
3. Produce
Development
Strategy
6. Build Bottom-
up and
Consolidated
Plans

19. Launch Products
During the launch, the teams produce important products.
documented team goals
defined team-member roles
an overall development strategy
a defined team process
a list of the project’s planned products
size estimates for the planned products
an overall project schedule
a quantitative quality plan
a detailed and balanced next-phase plan
a prioritized evaluation of the project’s key risks

20. Building Motivated Teams
While these products are important, the key launch objective is to build a motivated team.
Motivated teams believe their goals are challenging, important, and achievable.
To believe the goals are achievable, they must have
a plan they are committed to meeting
the skills and resources to do the job
When teams make their own plans, they are committed to meeting them.

21. The Team Roles
The roles distribute team management among the engineers.
These roles establish responsibility for managing the teamworking environment.
The members select their roles during the team launch.
The standard roles cover planning, process, quality, support, user interface, design, implementation, and test.

22. Planning a TSP Project In the TSP, planning is done at three levels.
The team first makes overall project and quality plans.
Then, the team makes a detailed plan for the next project phase.
The engineers next produce detailed personal plans for the following phase.
Finally, the engineers balance their plans to evenly distribute the work and minimize the schedule.

23. TSP Planning Define Process and Tasks Estimate Defects Injected
Produce
Conceptual
Design
Make the
Engineers’
Plans
Balance
Team
Workload
Estimate Tasks
Make Size
Estimates
Estimate
Yields
Estimate
Weekly Time
Balanced
Team
Plans
Conceptual
Design
Overall
Project
Plan
Quality
Plan
Detailed
Engineer
Plans

24. Tracking a TSP Project
The detailed team and individual plans facilitate precise project tracking.
The team members regularly reassess project risks and devise mitigation actions.
In weekly team meetings, the engineers
report on task status
review the key risks
rebalance the workload
The team produces weekly management status reports.

25. Engineer Task, Schedule,
TSP Project Tracking
Enter
Time by
Task
Enter
Week Task
Completed
Product
Summary
Task Status
Engineer A
Updated Team and
Engineer Task, Schedule,
and Quality Plans
Team Task
and Schedule
Summary
Enter Defects
by Component
and Phase
Enter
Size by
Component
Schedule
Status
Engineer A
Quality
Summary

26. Team Leader Support
The Team Leader periodically reports to management on project status and risks.
The team leader supports the team by
obtaining staff and arranging for training
protecting team resources
interfacing with other groups
resolving problems and issues
maintaining process discipline
overseeing process and product quality
sustaining the team’s energy and drive

27. Management Support
The PSP/TSP will not work unless the engineers are fully supported by all management levels.
During the project launch the team reviews its plan with management.
answers questions
resolves issues
explores alternatives
To sustain the team’s motivation, management must
periodically review the project
probe the team’s data
focus on quality

28. Engineer and Team Performance
By learning TSP skills, engineers improve their personal performance.
More important, however, the teams also build a defined process and working framework.
They now have a common language and the ability to manage their work with facts and data.
This combination produces extraordinary team performance.

29. TSP Experience Many TSP projects have been launched.
Project size has ranged from 2 up to 50 team members.
Products have been imbedded systems, real-time controllers, and commercial applications.
Program size has ranged from a few hundred LOC to nearly half a million LOC.
The following charts show some of the results to date.

30. PSP/TSP Change Behavior
Many engineers have now been taught the required PSP/TSP skills.
These skills both provide the foundation for effective teamwork and improve personal performance.
The following data show the results from 298 engineers in PSP courses.
In the course, the engineers write 10 programs.
For program 1, they use their current practices.
By program 10, they are practicing the PSP/TSP disciplines.

31. The PSP Course for Engineers
Cyclic development
Team Software
Process
Teambuilding
Risk management
Project planning and tracking
PSP2.1
Design templates
PSP2
Code reviews
Design reviews
PSP1.1
Task planning
Schedule planning
PSP1
Size estimating
Test report
PSP0.1
Coding standard
Process improvement
proposal
Size measurement
PSP0
Current process
Basic measures

32. Effort Estimation Results
Effort Estimation Accuracy Trend
0.2
0.3
0.4
0.5
0.6
0.7
Estimated Minutes – Actual Minutes/Estimated Minutes
Mean Time Misestimation
PSP Level Average
This chart illustrates the change in average errors for estimating effort across the 10 PSP programs. The figure shows gradual improvement during the course of the training.
The bold line, again, tracks the change across the 10 programs, and the dotted line is the pooled average for each PSP level. 11 10 9 8 7 6 5 4 3 2 1
Program Number
© 2000 by Carnegie Mellon University
Version #
Productive Engineering Teams - page 32

33. Design Time Results Time Invested Per (New and Changed) Line of Code
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Mean Minutes Spent Per LOC
Design
Code
Compile
Test 11 10 9 8 7 6 5 4 3 2 1
Program Number

34. Quality Results Defects Per KLOC Removed in Compile and Test10 20 30 40 50 60 70 80 90
100
110
120
Mean Compile + Test
PSP Level Mean Comp + Test
Mean Number of Defects Per KLOC
We are using the number of defects found and removed in the Compile and Test Phases as one indicator of product quality.
The bold line tracks the changes in average defects per KLOC for the 298 students across each of the 10 PSP assignments. The sample size varies from program to program (not shown in the chart yet). but there is a minimum of 100+ data points reflected in each mean.
The dotted line was drawn by pooling all the individual defect densities by PSP level (program 10 is included in PSP level 2) and then computing an average.
In this chart we see a dramatic decrease in the number of defects per thousand (new and changed) lines of code.
These differences are statistically significant at the PSP level and across the 10 assignments (using a repeated measures Analysis Of Variance). 11 10 9 8 7 6 5 4 3 2 1
Program Number
© 2000 by Carnegie Mellon University
Version #
Productive Engineering Teams - page 34

35. Productivity Results Lines of (New and Changed) Code
Produced Per Hour of Total Development Time 20 22 24 26 28 30
Mean LOC/Hour
Mean LOC/Hour
PSP Level Average
In this first productivity chart, we see that there is no CLEAR AND CONSISTENT change in productivity, compared to the previous figures. Take note of the scale on the vertical 11 10 9 8 7 6 5 4 3 2 1
Program Number
© 2000 by Carnegie Mellon University
© 2000 by Carnegie Mellon University
Version #
Version #
Productive Engineering Teams - page 35
Productive Engineering Teams - page 35

36. Engineering Performance in Industry
From the PSP/TSP course, engineers learn how to
plan and track their personal work
measure and manage product quality
This significantly improves their performance when working alone.
It also has a positive impact on their performance when working on a team.

37. Predictable Schedules
Schedule Deviation Individual Value Control Chart -
Commercial Systems
350
300
250
200
150
% Deviation
100 50
Overrun 112% to 37% to 5%.
Industry data -- average overrun in 1998 was 121%.
-50
01/88
01/89
01/90
01/91
01/92
01/93
01/94
01/95
01/96
01/97
01/98
-100
Pre-CMM
CMM
PSP
-150
Date of Project Start
Individual Data Points
Mean
Upper Natural Process Limit
Lower Natural Process Limit
One Standard Deviation

38. With Less Test Time, Productivity Improves with PSP

39. Team Performance in Industry
Even with the proper training, engineers do not generally practice the TSP methods.
To obtain the benefits of the TSP, organizations need to provide their engineers with
professional training
a defined teambuilding process
a supportive team environment
consistent management and coaching
Under these conditions, engineering teams can be highly productive.

40. TSP Benefits (Boeing Pilot #1) # of Defect Software Size 2.36X more
Sloc count
# of Defect
Detected
75% lower
Defect
Release # 6 Release # 7 Release # 8 Release # 9
PSP/TSP
trained

41. TSP Benefits (Boeing Pilot #1) System Test time 2.36X more Sloc count
41 days
System
Test time
32 days
28 days
2.36X more
Sloc count
94% less time
4 days
Release # 6 Release # 7 Release # 8 Release # 9
PSP/TSP
trained

42. TSP Team Task Hours Task hours directly produce products.
Average Task Hours Per Week 18 16
15.1 14
13.3
12.6 12
Task Hours 10
9.6 8 6 4
Avg. Task Hours - Week
Avg. Task Hours - Phase 2
04/20/1998
04/27/1998
05/04/1998
05/11/1998
05/18/1998
05/25/1998
06/01/1998
06/08/1998
06/15/1998
06/22/1998
06/29/1998
07/06/1998
07/13/1998
07/20/1998
07/27/1998
08/03/1998
08/10/1998
08/17/1998
08/24/1998
08/31/1998
09/07/1998
09/14/1998
09/21/1998
09/28/1998
10/05/1998
10/12/1998
10/19/1998
10/26/1998
11/02/1998
11/09/1998
11/16/1998
11/23/1998
11/30/1998
12/07/1998
12/14/1998
12/21/1998
12/28/1998
01/04/1999
01/11/1999
01/18/1999
01/25/1999
02/01/1999
02/08/1999
02/15/1999
02/22/1999
03/01/1999
03/08/1999
03/15/1999
03/22/1999
03/29/1999
04/05/1999
04/12/1999
04/19/1999
04/26/1999
05/03/1999
05/10/1999
05/17/1999
05/24/1999
05/31/1999
06/07/1999
06/14/1999
06/21/1999
06/28/1999
Task hours directly produce products.
Task hours initially averaged under 10 per week.
Task hours were improved through quiet times, process documentation, more efficient meetings, etc.
© 2000 by Carnegie Mellon University
Version #
Productive Engineering Teams - page 42

43. Unbalanced Team Workload
Engineer E G I 20 40 60
Schedule Weeks

44. Balancing Team Workload
Unbalanced
Engineer E
Balanced G I 20 40 60
Schedule Weeks

45. A TSP Government Project
Air Force flight planning system
6 software engineers
5 were PSP trained
Integration and system test time was reduced from an average of 22% of schedule to 2.7%.
Productivity was 2.33 times the same team’s prior project.
Projects A B C TSP
Size - LOC , , , ,820
Test Days
System test defects/KLOC
Acceptance test defects/KLOC N.A

46. A TSP Industrial Project - 1
Communications test equipment
12 software and 3 hardware engineers
not all software engineers PSP trained
Plan Actual
Size - KLOC
Effort - hours , ,711
Schedule - weeks
Defects per KLOC
Integration test
System test
Field trial

47. A TSP Industrial Project - 2
Comparison with a prior project of 9,500 LOC.
Engineers not PSP trained.
Communications system controller
TSP non-TSP
Size - KLOC
Field testing duration
engineering trial weeks months
acceptance trial weeks months
Trial defects/KLOC

48. A TSP Maintenance Project
A company applied TSP with 2-engineer teams on 14 maintenance projects.
Average results showed.
TSP non-TSP
Time estimate errors % Not tracked
Size estimate errors % %
Defects/page
Review yield % Not tracked
Productivity - pages/hour

49. Getting Started with TSP
Sprinkling a few PSP-trained engineers around an organization will not produce noticeable results.
Installing TSP in an organization requires
a team-based improvement focus
careful planning
senior management involvement and sponsorship
a change in the behavior of both the engineers and their managers

50. The TSP Introduction Strategy
The TSP introduction strategy has these steps:
identify key areas for initial introduction
hold executive kickoff and planning seminar
identify and train affected managers
train the engineers in teams
train and authorize PSP instructors and TSP coaches
establish needed support for pilot projects
conduct PSP/TSP pilot projects
plan and initiate rollout across the organization

51. Introducing the TSP
Installing TSP in an organization can be done in a few months. It requires
substantial training
support from all involved managers
management emphasis on completing PSP course work before the first team launch
Training and transition support
PSP courses are available from the SEI and its transition partners.
Some university PSP courses are now offered.
The TSP is only available from the SEI.

52. Preparation Activities
An executive kickoff and planning seminar takes 1 1/2 days.
Management training is 4 days.
Engineer training: hours of PSP training.
teaches personal project management
engineers learn to measure and manage quality
required for effective team participation
PSP/TSP coaches need additional training.

53. Example Introduction Timeline
Example timeline
based on 9-12 month pilot projects
initial results are available in the first 6-12 months
final results available within months
demonstrates costs and benefits with PSP/TSP
rapidly builds high-performance teams
Task Q Q Q Q Q Q6
Executive training/kickoff session X
Select participants, develop schedule X
Train managers, engineers, instructors X X X
Conduct TSP pilots, train coaches X X
Plan and initiate roll-out X

54. Conclusion The TSP shows engineers how to
plan and direct their own work
meet aggressive schedules
produce superior products
The TSP shows management how to
build world-class teams
lead and support these teams
When properly trained and led, engineers can do extraordinary work.

55. For More Information Visit the PSP or TSP web sites
Contact a PSP transition partner
Contact SEI customer relations
Software Engineering Institute
Carnegie Mellon University
Pittsburgh, PA
Phone, voice mail, and on-demand FAX: 412/

56. TSP Applications - Domain
The PSP/TSP has been used in a wide range of application domains.
TSP programs have also been developed for Windows, Unix, and imbedded systems.
Automotive
Avionics
Communications control
Display systems
Financial
Imbedded
Internet
Office systems
Real time
Spacecraft

57. TSP Applications - Size
The size of TSP systems has varied widely.
The smallest new programs of several hundred to a few thousand LOC have been developed by students.
The largest programs have been for imbedded communications systems and integrated financial applications of 100,000 LOC up to about LOC.
Both small and large modifications have been made to very large legacy systems.

58. TSP Applications - Users
Some of the organizations that are introducing the TSP are
AIS
Allied Signal
Bahn
Boeing
Comnet
EBS
EDS
Erickson
Honeywell
Iomega
JPL
Kaiser
Litton
SAIC
SDRC
Teradyne
Trilogy
United Defense
USAF: Hill AFB
USN: China Lake
USN: NAVOCEANO
Xerox