Top Down View of Estimation Test Managers Forum 25 th April 2007.

Slides:



Advertisements
Similar presentations
Test process essentials Riitta Viitamäki,
Advertisements

GE Healthcare UK Performance Solutions
Web Development Engineering Processes Introduction to Web Development Outsourcing Processes.
Test Automation Success: Choosing the Right People & Process
© 2011 IBM Corporation Improving Reliability and Making Things Cheaper to Run Tuesday 20th September James Linsell-Fraser, Senior Architect & Client Technical.
Alford Academy Business Education and Computing1 Advanced Higher Computing Based on Heriot-Watt University Scholar Materials Waterfall, Scope, Project.
Software Engineering CSE470: Process 15 Software Engineering Phases Definition: What? Development: How? Maintenance: Managing change Umbrella Activities:
RAMIRI2 Prague 2012 Project management and the RI Life Cycle.
1 Title slide Future for Functional Test Automation? TM Forum – April 2006 Susan Windsor Insight Through Intelligence WMHL Consulting Limited, MD.
Stoimen Stoimenov QA Engineer SitefinityLeads, SitefinityTeam6 Telerik QA Academy Telerik QA Academy.
Computer Engineering 203 R Smith Project Tracking 12/ Project Tracking Why do we want to track a project? What is the projects MOV? – Why is tracking.
Software Delivery. Software Delivery Management  Managing Requirements and Changes  Managing Resources  Managing Configuration  Managing Defects 
W5HH Principle As applied to Software Projects
Test Management Challenges. Topics Drivers Value Contribution Focus Areas –Information provision –Estimation –Supplier management –End to end view of.
Week 1 intro to PM Project Management Introduction to Project Management and the Software Development Lifecycle Week 1 Winter quarter 1/7/02 SOS.
1 April 2010 TX SET Timeline Project Conceptualization 11 weeks Market Requirements 12 weeks ERCOT Requirements 12 weeks Conceptual Design 6 weeks Detail.
Unit Slides by UK Versity.  Unit aims:  This unit aims to help the learner with an opportunity to develop their project management and research skills.
COMP8130 and 4130Adrian Marshall 8130 and 4130 Test Management Adrian Marshall.
Embedding Security into a Software Development Methodology April 5 th, 8:30 AM Jonathan Minter Director, IT Development and Engineering Liberty University.
Project Management Basics
Stoimen Stoimenov QA Engineer QA Engineer SitefinityLeads,SitefinityTeam6 Telerik QA Academy Telerik QA Academy.
Readiness Index – Is your application ready for Production? Jeff Tatelman SQuAD October 2008.
1 Performance Auditing  In IT Environment  Evidence Gathering & Analysis Techniques  Computer Assisted Techniques  Use of IDEA.
1 PMIG PUBLIC SECTOR PROCUREMENT BEST PRACTICES & LESSONS LEARNED Kevin James Barrie Kroukamp.
 Contoso is working on Wireless 1xEvDo application to handle high speed 3G application data transfer (voice, video data) in mobile phones while working.
© Mahindra Satyam 2009 Project Metrics QMS Training.
Product Quality, Testing, Reviews and Standards
Test Organization and Management
Dillon: CSE470: SE, Process1 Software Engineering Phases l Definition: What? l Development: How? l Maintenance: Managing change l Umbrella Activities:
Test Roles and Independence of Testing Telerik Software Academy Software Quality Assurance.
Chapter 6 : Software Metrics
BSBPMG405A Apply Human Resource Management Approaches Apply Human Resource Management Approaches Unit Guide C ertificate IV in Project Management
UKSMA 2005 Lessons Learnt from introducing IT Measurement Peter Thomas –
BSBPMG505A Manage Project Quality Manage Project Quality Project Quality Processes Diploma of Project Management Qualification Code BSB51507 Unit.
Chapter 11. Intro  What is Project Management?  Project Manager  Project Failures & Successes Managing Projects  PMBOK  SDLC Core Process 1 – Project.
Corporate Support PPSO Gez Keating, Serco Plc. AGENDA Portfolio Management Role of Corporate PPSO Processes Where to Start Benefits.
© 2001 Change Function Ltd USER ACCEPTANCE TESTING Is user acceptance testing of technology and / or processes a task within the project? If ‘Yes’: Will.
Project Management All projects need to be “managed” –Cost (people-effort, tools, education, etc.) –schedule –deliverables and “associated” characteristics.
Apply Quality Management Techniques Project Quality Processes Certificate IV in Project Management Qualification Code BSB41507 Unit Code BSBPMG404A.
BSBPMG404A Apply Quality Management Techniques Apply Quality Management Techniques Project Quality Processes C ertificate IV in Project Management
Chapter 1: Fundamental of Testing Systems Testing & Evaluation (MNN1063)
Pay for Performance A strategic approach to design Dermot Hand August 2012.
BSBPMG501A Manage Application of Project Integrative Processes Manage Project Integrative Processes Unit Guide Diploma of Project Management Qualification.
The Implementation of BPR Pertemuan 9 Matakuliah: M0734-Business Process Reenginering Tahun: 2010.
Advanced Software Engineering Lecture 4: Process & Project Metrics.
Levers enabling industrialised deployment and a effective multi-vendor eco-system We wish to assess the current position of these levers and the opportunity.
Unit – I Presentation. Unit – 1 (Introduction to Software Project management) Definition:-  Software project management is the art and science of planning.
Continual Service Improvement Methods & Techniques.
Not for Distribution Outside of MBP1 April 11, 2012.
RUP RATIONAL UNIFIED PROCESS Behnam Akbari 06 Oct
Project Life Presented by Chuck Ray, PMP ITS Project Manager.
1 Home Care Support Outcome Based Specification Workshop 26 th November 2009.
BMS4667 Laboratory Leadership and Management Dr. David Ricketts.
BSBPMG503A Manage Project Time Manage Project Time Project Time Processes Part 2 Diploma of Project Management Qualification Code BSB51507 Unit Code.
IS&T Project Reviews September 9, Project Review Overview Facilitative approach that actively engages a number of key project staff and senior IS&T.
Roland Gilbert BSc MRICS; Prince 2 Practitioner
Test Roles and Independence of Testing
12.2 Conduct Procurements The process of obtaining seller responses, selecting a seller and awarding the contract The team applies selection criteria.
Identify the Risk of Not Doing BA
Description of Revision
XRN4361 – UK Link Release 2 - Delivery
Constructive Cost Model
Overall Project RAG Status
Chapter 11 – Project Dashboard
IS&T Project Reviews September 9, 2004.
Risk Management CSCE 489/689 (Software Security) Fall 2018
XRN4361 – UK Link Release 2 Project Status Update
The Imaginist Framework for Benefits Realisation
{Project Name} Organizational Chart, Roles and Responsibilities
Presentation transcript:

Top Down View of Estimation Test Managers Forum 25 th April 2007

Estimating Credentials Risk assessment on fixed price bespoke development projects –If test elements incorrect, the project lost money and my company would go out of business Delivery of managed testing services –Contracted to meet specific and defined goals Management of IBM Test Services –Included estimating testing for major programmes of work, fixed price contracts, and audits of global programmes when things not gone to plan

Topics “Broad Brush” estimating Specialist skills Scope considerations Building the team Creating elapsed time from local factors Metrics to consider

Broad Brush - Considerations How much testing can we afford? –If you’re working to an existing budget, be sure and spend it wisely What is our organisations standard percentage for test effort? –Many organisations have a standard view, even if it’s not documented, “we allocate (30%, 50% 10%) of our budget to testing Do we have any specific time scale considerations? –Typically driven by market or legal demands What are our quality objectives for this project? –Typically driven by business criticality and overall risk drivers

Broad Brush – Programme Parameters Development Phase Percentage Effort Analysis12 Design15 Build15 Static5 Unit5 Link8 Test PhasePercentage Effort System12 Integration20 UAT2 OAT6 Totals100

Specialist Skills - Additional headcount Performance –Not full time, but include planning early on and execution in several phases of testing Security –Not full time, and maybe not as part of your testing project, but has to be included somewhere Reliability –Typically part of operational acceptance, but consider value of including in integration testing on complex projects Automation –Include in the Test Strategy, consider the value for each test phase and manage value contribution carefully

Scope Considerations Environment set up –Who orders, receives and builds the environment? Environment management –Who supports test activity with providing known state of all components? Test data –Who will provide it, and manage it Data migration –How will this be tested? Operational acceptance and UAT –Even if it’s out of scope, what support is required from your team

Building the Team Team structure adds in effort and elapsed time How will the team be managed? How many team leaders? What tasks will Test Manager undertake to contribute to planning and execution effort – if any? What is the scope of test support and who will do this? Where will the skills come from? Will they be available and at the right level?

Building the Team - Example Test Manager Functional Testing Team Leader Undertakes functional testing, support multiple phases, Support for UAT Non-Functional Testing Team Leader Undertakes all non-functional testing Support multiple phases Support for OAT Test Support Oils the wheels of the team

Elapsed Time – Local Factors Test Data –Often difficult to provide data to support testing Development resources available to fix bugs –Do your developers move onto other projects before testing is completed and signed off Environment availability and support –Very few organisations get this right and it impacts test effectiveness greatly Skills availability –Do you have all the skills internally, is training required Meeting culture –Build meeting attendance into your plans Frequency and amount of change –Build in time to review plans in line with the amount of change

Improving Estimating - Considerations Actual against planned effort by phase Track the lost time associated with local factors Monitor level of quality achieved in production Undertake causal analysis Capture the effort required to find and then fix bugs – will require analysis but can be very enlightening

Example Project Percentage of overall Effort Analysis & Design41%7%22%7%18%14% Code & Unit Test41%44%33%49%52%40% System11%26%33%2%20%29% UAT7%22%12%43%9%16% Total Devt percentage82%52%55%56%70%55% Total Test percentage18%48%45%44%30%45% Number of Defects Found Unit20 System UAT Implementation05 Production112 Total Defects Found Total Time to Fix Defects Average Defect Fix Time (HRS) Total Time to Find Defects Average Defect Find Time (HRS)

Improving Estimation – E.G. Cost –Effort expended on test activity by phase of testing – timesheets Quality –Number of defects by severity registered during first 3 months of production –Number of released vs planned releases –Number of criteria met for each quality gate and impact of not having met specific criteria Timescale –Actual against plan for project milestones –Ratio of test effort vs development effort against plan –Number and impact of scope changes by phase –Environment availability to support testing

In Summary Estimating test effort is difficult – no right answer? –Can be more of an art than a science but there are standard industry metrics to help, what’s important is learn from your experience so capture the right metrics and report accurately Test effort is a small part of where the time goes on a project –Remember team structure, how things work in your organisation and lessons learnt from previous projects Testers love to discuss estimation and metrics experience –Network in the industry internet community, attend events, absorb information from all sources, then relate to your own business goals.

Discuss!