1. Testing SIP Services Over IP

2. Agenda  SIP testing – advanced scenarios  SIP testing - Real Life Examples

3. Test Equipment Requirements

4. SIP Testing – the basics  Call simulation  Call monitoring  Signaling analysis  Media analysis  Conformance testing  Load (stress) generation

5. Advanced SIP Testing  Advanced testing  Benchmarking  Performance  Media degradation  Envelope determination  In terms of a large network…  For complex scenarios:  Many users  Many call flows  Different codecs  Different networking configuration  NAT, Firewall…

6. Stress Testing??  Aimed at determining the operational envelope:  Network elements  Service  Certain concept / design  Why stress test:  Vendors: want to determine performance figures before shipping  Service Providers: want to find the problems in the lab

7. Stress Testing! Not just “number of calls”  Emulate a network wide environment:  Large number of users  Many network elements (IPs)  Emulate NAT  Excessive call load  Many profiles / scenarios  Many different services

8. What are we looking for  Predicting the element behavior in real life:  How does the UUT (network element / service) behave in real life (=stress…)  Where is the “breaking point”  What happens at the “breaking point”  How are new / existing calls affected?  Is the media degraded?

9. Stress Factors  Basic call flows (signaling)  RTP streams  Media type (Video, Audio)  Advanced call flows  SIP services

10. Signaling Stress Requirements  Generate and respond to SIP service requests  Simulate User Agent Client / Server communication procedures (RFC 3261)  Emulate behavior of one or many SIP phones  different signaling and media parameters for each

11. Registration Rate  True emulation of user behaviour  Generate calls only when registered  Mega-blast generation of registration events  Full configuration of SIP registration procedures  Mimic real user behaviour

12. Advanced Testing Requirements (1)  Multiple users  100,000’s UACs  Configurable groups each with different behavior profiles  Multiple IPs  1000s of IPs - both media and signaling  Multiple MACs  Multiple VLANs

13. Advanced Testing Requirements -- 2  Full SIP message customization  Video generation  DTMF generation  Different call flows:  Hold, Consultation Hold, Music on Hold, 3-way Conference, Call redirect, Call reject…  Erroneous flows

14. Key Traffic Generation Measurements  Session success rates and GoS  SIP method count and response code  Registration history  Presence and IM message statistics  Incoming and outgoing RTP stream counters

15. Real Life Examples

16. Common pitfalls  Registration properties  Tendency to forget the credentials  Registration rates  Initially takes too long  Server will not withstand load  Non-standard flows  Test signaling & media simultaneously

17. Network Model: Simple Test Lab Test Environment: Multi-service solution Multi-technology: VoIP & PSTN Core 2 Core 1 Call Analysis IP / ATM / FR SIP Stress Generator UUT Call Analysis SIP Stress Generator Analog call generator

18. Network Model: Simple Test Lab (cont.)  SIP Stress Generator:  Generate different levels of stress including RTP  Voice and video simulation  Call Analysis:  Signaling and media analysis  Voice quality (intrusive, non-intrusive and subjective)  RTP transport streams  Analog Call Generator  Connect to PSTN interfaces  Measure voice quality, voice distortion

19. Testing Options  Performance - under different levels of stress:  Signaling and media quality evaluation  Call handling  Media degradation  Interoperability  Different codec models  PSTN and VoIP devices  Reliability  Quality evaluation  Stability testing

20. Firewall, SBC, CSCF Testing  Load scenarios  BHCA – 1,000,000s per hour  CPS – hundreds to thousands  Concurrent calls > 100,000s Public IP Soft-Switch Session Boarder Controller Soft-Switch SP1SP2 SIP Stress Generator Call Analysis Test bed 1 Test bed 2

21. Firewall, SBC & CSCF: Test Lab  SIP Stress Generation requirements:  Different levels of stress  Different services  Different media  Different call flows  QoS analysis:  Voice quality  Transport

22. Firewall, SBC & CSCF: Testing Needs  QoS  Delays, Jitter, Packet loss, MOS  Stability (network elements / services)  Benchmarking and comparisons

23. Simultaneous test bed operation

24. Elegant solution