SIP Performance Metrics 66th IETF – Montreal Daryl Malas

Motivation With widespread implementation of SIP the following problems have surfaced: No standard performance terms No standard performance metrics No standard methods No standards between vendors and operators Current reliance on PSTN metrics Operator and vendor problem!!!

Industry Acknowledgment & Relevance SIPPING WG develops standard to benchmark end-to-end SIP application performance BMWG develops standard to benchmark SIP networking device performance SPEC to develop industry-available test code for SIP benchmarking in accordance with IETF’s BMWG and SIPPING standards. ITU-T (Study Group 12) focused on detailed failure/success scenarios and end-to-end benchmark guidance.

SIPPING vs. BMWG -----Original Message----- From: Poretsky, Scott Sent: Thursday, June 22, 2006 8:00 PM To: 'Malas, Daryl'; acmorton@att.com; gonzalo.camarillo@ericsson.com; mary.barnes@nortel.com Cc: vkg@lucent.com; Poretsky, Scott Subject: RE: (BMWG/SippingWG) SIP performance metrics Yes Daryl.  I absolutely agree.  The item posted to BMWG focuses on single DUT benchmarking of SIP performance.  Your work in SIPPING is focused on end-to-end application benchmarking.  It would be great (and I would even say a requirement) that the Terminologies for these two work items remain consistent with each other.   

Proposed Metrics (Solution) Create standard metrics with consistent terms and methods of calculations The metrics: Session Request Delay Measures time from the initiation request to an acknowledgment Session Completion Delay Measures time from a completion request to an acknowledgment Average Hops per INVITE Measures the average hops an INVITE takes to traverse the network Average Hops per Session Measures the average hops of origination messages, associated with a session, taken to traverse the network Session Duration Time Measures time from the initiation message to the completion message Session Establishment Rate Ratio of number of initiation requests to established sessions

Metrics Cont’d Session Defects Ineffective Session Attempts Identifies the number of persistent failures “defects” in dialog processing Ineffective Session Attempts Identify attempts described as ineffective due to failure or congestion Session Disconnect Failures Identify issues causing sessions to fail after they were established Session Completion Rate Measures the number of sessions completed without failure due to timeouts Session Success Rate Identifies the number of successful calls by removing the percentage of Ineffective Session Attempts and Session Disconnect Failures

Feedback and Next Steps Interest in making a working group item? Should end-to-end media performance metrics be included? Should terminology be extracted and included only in BMWG terminology draft? Additional metrics? Reduce number of metrics? SIP MIB incorporation?