1. BT – Managing Complex Systems Ian Johnston & John Palmer BCS Kingston & Croydon Branch presentation 26/02/08

2. © British Telecommunications plc Presentation Objectives Approach to managing e2e systems A standard for application events Business process and component transaction monitoring Order tracking and jeopardy Leveraging the value of monitoring, eg. ASGs, Service and Capacity etc. Managing COTS products eg BEA, Siebel

3. © British Telecommunications plc The BT experience BT architecture – SOA – linked reusable capabilities Our position has been driven from experience in monitoring of complex distributed architecture. The concept of configuring toolsets to monitor e2e is unachievable for large enterprises – maintenance expensive/ impossible. This has led us along the Design route which now parallels ITIL‘s Service Design concepts.

4. © British Telecommunications plc BT’s Matrix Architecture

5. © British Telecommunications plc BT Matrix Architecture Challenges - Service Design –SLM Defining measurements & targets, eg volumes, response times Aligning SLAs with UCs –Capacity Management Procedures to ensure customer targets are met –Business Continuity management Deployment designs to ensure resilience –Availability management Measure e2e availability broken down to capabilities Service Level Management –SLAs aligned to business requirements –BT’s outsourcing strategy Availability –Understanding CE requirements –Response times Capacity Management –Accurate measurement of transaction volumes –Response times broken down by capability IT Service Continuity management –Dynamic deployment in virtualised environments –Physical and geographic resilience

6. © British Telecommunications plc BT Matrix Architecture Challenges - Service Transition Release Management How to monitor the effects of new IT services on capabilities? How do we test the instrumentation and monitoring ready for production for –Events –Performance Deployment Management Can we establish auditable procedures for outsourced support suppliers? How can we ensure that the monitoring tools are fully exploited ?

7. © British Telecommunications plc BT Matrix Architecture Challenges - Service Operation Operational management –How to assess the impact and prioritise application events by business process and IT Service ? Application management –Routing of PRs to the appropriate support groups? –Analysing high volumes of events in log files? Technical management –Pinpointing root-cause across multiple shared capability Metrics –Stepped changes in volumes, errors and response times? –Impact of changes eg trend in error rates –Measuring operational efficiency eg txns vs. failures

8. © British Telecommunications plc BT Matrix Architecture Challenges – E2E Design Place Order Pending Acknowledged Committed Pending Acknowledged Committed Assigned (SMPF ID) Installed SMPF ID RADIUS,B-RAS,VCI,etc Completed Update(SMPF ID,Installation DN etc) Complete Build VC Committed NB:incorporates Flow Stream/Manage/Monitor/Director Status=“Completed” Status=“Completed” To “Close Order” sub-process From “SF– Provide– Progress-pt1” (Place Order) `` `` Create ServiceID Build Port Network Capacity Shortfall Get Tie Cable Mapping Into Error queue for manual processing End Customer To “Close Order” sub-process ` ` ` ` Activation email

9. © British Telecommunications plc BT Approach – Application event standard Application Standard Business Process Business transaction Time Host server Component capability e2e correlation key Event type Business keys

10. © British Telecommunications plc BT Matrix Architecture Solution - Service Design SLM agile design workshop to build in measures to support SLAs Availability Agile capability workshops to build in measures for monitoring of capacity implemented by apis Standardised events for common error conditions such as interface failures IT Service Continuity Dynamic reports of services and deployment profile (host/server distribution)

11. © British Telecommunications plc BT Matrix Architecture Solution - Service Transition Release Management Generate real time and historical reports on capabilities at the level of business and service Standard events can be processed by provided test tools and compared with designs Testing support – performance and resilience (designs and tools) Training programme for support teams (pre go live) Validation of the successful deployment of services

12. © British Telecommunications plc BT Matrix Architecture Solution - Service Operation Operational management Event correlation (by service and transaction identifiers) Impact (problem scenario and guided action) Performance bottlenecks Support group checklists (quick wins) Application management Improved routing of PRs to the appropriate support groups provided by e2e view We can we analyse high volumes of events by restricting the types of events and provision of summarisation Technical management Diagnosis – root cause ( e2e location and standard error) Metrics Summarisation and granularity inherent in standard

13. © British Telecommunications plc BT Application Monitoring Standard

14. © British Telecommunications plc Outsourcing Supplier Contracts 1. Monthly views to identify any stepped changes in –Volumes, Response times, Error rates 2. Weekly views of top 5-10 transactions showing –Distribution of volumes, variance in response times, peaks and spikes –Any worsening trends in errors and thresholds 3. Monthly analysis of error messages showing –Volumes errors, eg aborts, application, business, etc. –Breakdown by business process, IT service and component transaction –Corresponding traps and CR/DRs using AlarmMis 4. Ad-hoc Investigations to review –Loadings and relative performance across servers –Real-time transaction analysis –Drill down diagnostics –COTS, platform and network root cause analysis 5. Service management process to review –Capacity –Supplier’s (eg Siebel, WLS) and applications development group’s CRs and DRs –PRs against remedial activities

15. © British Telecommunications plc What is the BT experience? Key messages Define Standard for Application Events Instrumentation by design built into matrix capabilities Implementation by using agile design workshops Exploitation of toolset supported by supplier contracts Application monitoring standard promotes the effective problem management by integration with the enterprises diagnostic toolsets

16. © British Telecommunications plc Hunter Integration Console Management Frameworks COTS Monitoring definitions, e.g., Seibel, BEA, Oracle System & Application Trap Definitions Remote Operation Business Process & Application txn Monitoring Flexible & agile Uses COTS out-of-the-box Rapid development & deployment Any management frameworks Low maintenance Events Performance