2008. 4. 2 CJK NGN Test-bed Ad-hoc Group CJK NGN Test-bed Activities Phase I & II Testing Result 2008. 4. 2 CJK NGN Test-bed Ad-hoc Group

Topics Terms of Reference Testing Schedule Testing Scenarios Testing Items Testing Network & Service Configurations Testing Activities Testing Results Next Steps

Terms of Reference

TOR: NGN Test-bed Objectives Interconnecting CJK NGN Test-beds and building a common NGN technology testing infrastructure Feasibility testing prior to NGN commercial deployment Interoperability testing for Standard-based NGN components developed by CJK Providing NGN service development platform especially focusing on inter-country issues Reflection of testing results into corresponding standardization activities periodically Cooperation with other international NGN related testbeds and their activities such as MOONv6, Plugtests, GMI, etc.

TOR: Work Plan Phase 1 (March 2005 ~ March 2006): Preparation Stage Pick up the study items which include our target services for the evaluations on the test-bed Check the participants who are interested in the studies again Plan the schedule of the implementation and evaluation of the test-bed Prepare the test-bed in each SDO Establish each administration’s NOC for each SDO’s test-bed Agree on first and second set of target services and components and test scenarios Build test case suits for the agreed target services including a specific milestone Phase II (April 2006 ~ December 2007): First Interoperability Testing Stage Construct CJK NGN test-bed common operation and management infrastructure Test agreed first set of target services, especially focusing on transport functionality such as QoS, adimission control, traffic monitoring, etc. Especially main interest is in inter-domain/inter-operator issues Phase III (January 2008~ December 2010): Second Interoperability Testing Stage Test agreed second set of target services, especially focusing on application/service functionality and its binding with transport functionality Test other advanced NGN services yet to be defined NOTE: Phase III and later phases are being updated. Refer to Phases III and IV plans.

TOR: Action Plan for Phase I Define Objectives of the CJK interoperability tests for NGN (4th Meeting) Define Roadmap (4th Meeting) Define Network Architecture (Physical, Transport and Service Topologies and underlying technologies (e.g., MPLS, etc.)) of the testbed both intra- and inter-domain scope (4th Meeting) In Intra-domain case, each administration decides its own network (e.g., KOREN or APII in case of Korea) and enhances its functionalities to conform to NGN Define Implementation Agreements for inter-domain connectivity (5th Meeting) Define target services, target components, test scenarios and test case suits (5th Meeting) Recruit participants for equipment and services at each testbed site (5th Meeting) Establish a dedicated reflector for detailed discussion

Testing Schedule

Testing Schedule Phase I: 2006 3rd/4th Quarter Network Connectivity Senario1 only (Feb. 26 ~ Mar. 2) Conducted by CATR and ETRI First Report/Demo during 9th CJK NGN Meeting (April 2007) Phase II: 2007 1st/2nd Quarter Scenario 2 ~ 5 (Feb. 26 ~ Mar. 2) Second Report/Demo during 9th CJK NGN Meeting (April 2007) Phase III: 2007 4th ~ 2010 1st/2nd Quarter Scenario 6 ~ 7 Testing details will further be defined depending on the technology availability status CATR, KDDI, NTT, and ETRI will participate in the testing Third Report/Demo 13th CJK NGN Meeting NOTE: Phase III and later phases are being updated. Refer to Phases III and IV plans.

Testing Scenarios

Test Scenarios Scenario 1: Single Call Server within a single NGN Network Domain without PSTN Scenario 2: Call Servers across multiple IP Network Domains without PSTN Scenario 3: Single Call Server within a single NGN Network Domain with PSTN Scenario 4: Call Servers across multiple NGN Network Domains with PSTN Scenario 5: Call Servers across multiple NGN Network Domains with PSTN & Value Added Services This scenario adds Value Added Services to scenario 4 using Media Servers and Application Servers Scenario 6: Performance Evaluation Scenario 7: Call Servers across multiple NGN Network Domains with PSTN, Value Added Services and RACF Involvement

Scenario 1: Single Call Server within a single IP Network Domain without PSTN SIP H.323 MGCP H.248/Megaco* SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP(-I),H.323 AG SG SG AG Enterprise Access Enterprise Access Residential Access Residential Access Country 1 Country 2

Scenario 1 - Testing Cases Basic Call with the following combinations Access Gateway – Access Gateway SIP phone – Access Gateway SIP phone – SIP phone Subscriber Gateway – Access Gateway Subscriber Gateway – SIP phone Subscriber Gateway – Subscriber Gateway Protocols to test SIP H.323 MGCP H.248/Megaco Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Scenario 2: Call Servers across multiple IP Network Domains without PSTN SIP-I SIP H.323 MGCP H.248/Megaco* SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP(-I),H.323 Enterprise Access Enterprise Access SG AG AG Residential Access Residential Access Country 1 Country 2

Scenario 2 - Testing Cases Basic Call with the following combinations Same cases with scenario 1 Protocols to test SIP H.323 MGCP H.248/Megaco SIP-I Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Scenario 3: Single Call Server within a single NGN Network Domain with PSTN IP Network Domain SG PSTN Sigtran (SCTP/M3UA) Call Server MGCP H.248/Megaco Class4/5 SW TG SIP H.323 MGCP H.248/Megaco* SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP(-I),H.323 Enterprise Access Enterprise Access SG SG AG AG Residential Access Residential Access Country 1 Country 2

Scenario 3 - Testing Cases Basic Call with the following combinations Same cases with scenario 1 Access Gateway – Class4/5 SW Subscriber Gateway – Class4/5 SW SIP Phone – Class4/5 SW Protocols to test SIP H.323 MGCP H.248/Megaco SIGTRAN Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Scenario 4: Call Servers across multiple NGN Network Domains with PSTN IP Network Domain 1 Class4/5 SW IP Network Domain 2 SG SG PSTN TG Call Server 2 Call Server 1 TG SIP-I SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 Enterprise Access Enterprise Access SG SG AG AG Residential Access Residential Access Country 1 Country 2

Scenario 4 - Testing Cases Basic Call with the following combinations Same cases with scenario 1 Access Gateway – Class4/5 SW Subscriber Gateway – Class4/5 SW SIP Phone – Class4/5 SW Protocols to test SIP H.323 MGCP H.248/Megaco SIP-I SIGTRAN Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Scenario 5: Call Servers across multiple NGN Network Domains with Value Added Services & PSTN Back-End Servers Back-End Servers MS MS SIP Application Server SIP Application Server Class4/5 SW Diameter RADIUS Proprietary SIP MGCP H.248/Megaco Diameter RADIUS Proprietary SIP MGCP H.248/Megaco SG SG SIP(-) SIP(-I) PSTN TG TG IP Network Domain 1 IP Network Domain 2 Call Server 2 Call Server 1 SIP-I SIP H.323 MGCP H.248/Megaco* SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP(-I),H.323 Enterprise Access Enterprise Access SG SG AG AG Residential Access Residential Access Country 1 Country 2

Scenario 5 - Testing Cases Basic Call with the following combinations Same cases with scenario 1 Access Gateway – Class4/5 SW Subscriber Gateway – Class4/5 SW SIP Phone – Class4/5 SW Protocols to test SIP H.323 MGCP H.248/Megaco SIP-I SIGTRAN Application Servers Feature Testing SIP Application Server Features: IP conferencing, Voice mail, etc. Media Server Features: Generic conference bridge, Voice mail media processing, etc. Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Scenario 6 – Performance Evaluation Network Performance Testing cases E2E Call Setup Delay Call Completion Rate Each Segment & E2E IPTD/IPDV/IPLR/Path unavailability Call Traffic Accounting Harmonization Testing of Service QoS class differences among CJK User Terminal Performance Testing cases Various User Terminal Performance Testing

Scenario 7: Call Servers across multiple NGN Network Domains with Value Added Services, PSTN, and RACF Involvement Back-End Servers Back-End Servers MS MS SIP Application Server SIP Application Server Class4/5 SW Diameter RADIUS Proprietary SIP MGCP H.248/Megaco Diameter RADIUS Proprietary SIP MGCP H.248/Megaco SG SG SIP(-I) SIP(-I) PSTN TG TG IP Network Domain 2 IP Network Domain 1 Call Server 2 Call Server 1 SIP-I RACF RACF SIP H.323 MGCP H.248/Megaco* SIP H.323 MGCP H.248/Megaco* SIP(-I),H.323 SIP(-I),H.323 Enterprise Access Enterprise Access SG SG AG AG Residential Access Residential Access Country 1 Country 2

Scenario 7 - Testing Cases Basic Call with the following combinations Same cases with scenario 7 SIP phone – RACF Access Gateway – RACF Subscriber Gateway – RACF Application Server - RACF Protocols to test SIP H.323 MGCP H.248/Megaco SIP-I SIGTRAN Rq Application Servers Feature Testing SIP Application Server Features: IP conferencing, Voice mail, etc. Media Server Features: Generic conference bridge, Voice mail media processing, etc. RACF Feature Testing Resource Admission Control Performance NAT/Gate Control Performance Testing Residential & Business Features Call Transfer, Call Waiting, Call Diversion, Calling Numbering and Name Delivery, Three-party conference Centrex, Private numbering plan, DDI/DDO

Testing Items

Detailed Test Items Number Service Combination Test case Basic voice call MGCP IAD–H.248 AG or IAD Normal call Abnormal call- the called party is busy Abnormal call- no answer Abnormal call- the number does’t exist MGCP IAD–SIP phone SIP phone–H.248 AG or IAD

Detailed Test Items PSTN user – SIP phone Normal call Abnormal call- the called party is busy Abnormal call- no answer Abnormal call- the number doesn’t exist Multimedia call SIP phone – SIP phone Fax IAD/AG–IAD/AG Including G.711 and T.38

Testing Item Description Ex. 1 TEST NUMBER:：1 TITLE：Basic Call between SIP user and AG user SUBTITLE：normal call PRE-TEST CONDITIONS： 1、SIP user 1 registered on SS1; AG registered on SS2；User 2 is one of the AG’s Subscribers. 2、The protocol between SS1 and user1 is SIP；The protocol between user2 and SS2 is H.248 TEST CONFIGURATION：SIP user 1－SS1－SS2－AG－user 2 TEST STEP： 1、User 1 calls user2，user2’s phone rings； 2、User2 answers the call； 3、User1 hangs up first,then user2 hangs up,too.

Testing Item Description Ex. 1 EXPECTED SIGNAL SEQUENCE:： Call-setup flow： SIP user1 sends an INVITE to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2. SS2 sends an Add request to creat a new context in AG .AG sends back a Reply message with the RTP port and the stream descriptor. SS2 sends an Modify to AG ,AG sends the ringtone to user2 and monitors user2 hook-off. SS2 sends a 180Ringing response to SS1,SS1 forwards this message to SS2. User2 answers the call ,AG sends a Notify to SS2 and SS2 sends back a Reply. SS2 sends a 200 OK response to SS1 and SS1 forwards to user1. User1 sends back ACK . SS2 sends AG a Modify command to stop ring tone and monitor user2 hook-on.

Testing Item Description Ex. 1 Call-release flow： Caller releases the call, user1 sends a BYE to SS1,and SS1 sends BYE to SS2. SS2 sends a Modify command to AG to send a busy tone to user2. SS2 sends a 200 OK response to SS1.SS1 sends 200 ok to user1. User2 hungs up,and AG sends Notify to AG,then SS2 sends Subtract (TDM) + Subtract (RTP) to AG to delete the context. SS2 sends Modify to AG to monitor hook-oof.

Testing Item Description Ex. 2 TEST NUMBER:：2 TITLE：Basic Call between SIP user and AG user SUBTITLE：abnormal call——the called party is busy PRE-TEST CONDITIONS： 1、SIP user1 registered on SS1; AG registered on SS2；User2 is one of the AG’s Subscribers. 2、The protocol between SS1 and user1 is SIP；The protocol between user2 and SS2 is H.248 3、User2 is busy. TEST CONFIGURATION：SIP user1－SS1－SS2－AG－user2 TEST STEP： 1、User1 calls user2； 2、User1 hears the busytone or voice information； 3、User1 hangs up.

Testing Item Description Ex. 2 EXPECTED SIGNAL SEQUENCE:： SIP user1 sends INVITE to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2. SS2 knows that the called party is busy now, then SS2 sends a 486 response to SS1. SS1 forwards the 486 to user1.

Testing Item Description Ex. 3 TEST NUMBER:：17 TITLE：Basic Call between SIP users SUBTITLE： normal call PRE-TEST CONDITIONS： 1、SIP user 1 registered on SS1; SIP user 2 registered on SS2. TEST CONFIGURATION：SIP1－SS1－SS2－SIP2 TEST STEP： 1、User1 calls user2，user2’s phone rings； 2、User2 answers the call； 3、User1 hangs up first,then user2 hangs up,too.

Testing Item Description Ex. 3 EXPECTED SIGNAL SEQUENCE:： (1) SIP user1 sends an INVITE request to SS1 to setup a session，SS1 responses 100 Trying.SS1 forwards the INVITE request to SS2 then to user2. (6) SIP user1 rings .User 2 sends a 180Ringing response to SS2,SS2 forwards this message to SS1 then to user1. (9) User2 answers the call ,User 2 sends a 200 OK response to SS2 then to User1. (10) User1 sends ACK back. User1 and user2 setups a multimedia session.

Testing Network & Service Configurations

CATR’s CS Lab Configuration SCP SoftX3000 SIP IOSS FTP/FTAM SIP/MGCP /H.248/H.323 U - NICA SIP SIGTRAN Lanswitch INAP USAU SIGTRAN MGCP MRS6100 STP ISUP IP Network SG7000 ISUP SNMP STUN UMSN2000 FE PSTN Eudemon E1 Lanswitch Lanswitch H.248 H.323 SIP MGCP UMG 8900+UA5000 POTS / BRI/ADSL POTS / BRI/ADSL … VedioPhone OpenEye SoftPhone IAD

ETRI’s CS Lab Network Configuration Japan CERNET 155M POS Internet Border Router 2G Ethernet Support Facility GBE Management/ Monitoring System, GW, SIP phones, … GBE, 100BT, 10BT Meeting Room 100BT SBC 100BT(VLAN) L2 Switch 100BT 100BT 100BT 100BT 100BT AS/MS CS SG/TG RACF Packet Monitor E1 TDM 100BT PSTN Network Signaling Link Bearer Link

Testing Services, Venders & Terminals Interworking services: voice and video Interworking venders: CATR (Huawei) and ETRI (Xener) Interworking terminals: ETRI SIP phone and CATR SIP, MGCP IAD and PSTN terminal(H.248 TG) 25 test items

Testing Activities

Testing Processes and Schedules Started at Sept.25, 2006 Four Phases Network connection: Sept.25-30, 2006 Data configuration: Jan.30-Feb.1, 2007 Test: Feb.26-March 2 Test: March 5-9

Testing Result Examples The following several testing result examples provide a few failed cases and successful cases Ethereal was used to capture the SIP messages exchanged in each Call Server site The failures were fixed during the testing process The failures were two kinds: intentional (i.e., abnormal) and non-intentional The former were used to check the correctness of the system behavior. For example, a call attempt was used to non-existing callee The latter were mainly for the message format mismatches and/or IP address misuses

Failed Testing Example 1 This example shows the receiving Call Server doesn’t understand the SIP Invite request message

Failed Testing Example 2 This example shows call attempt to non-existing callee

Successful Testing: Normal

Successful Testing: Busy

Testing Results

Testing Results

Testing Results

Testing Results

Lessons Learned from Testing First successful interconnection of CK NGN Test-beds and build of a common NGN technology testing infrastructure Successful interoperability testing for CJK-driven ITU-T Standard-based NGN components and services (i.e., call server) Provided NGN service development platform especially focusing on inter-country issues First attempt to bridge the standard and implementation by the reflection of testing results into corresponding standardization activities Stimulation of further test-bed activities, for example, RTP/RTCP-based Performance Measurement testing

Next Steps NOTE: Information here is at April 2008. Further updates will be made.

Upcoming Testing Phases Phase I: 2006 3rd/4th Quarter (Completed) Network Connectivity Scenario 2 and 4 only (2 CS and 2 domains) Phase II: 2007 1st/2nd Quarter (Completed) Scenario 1 and 3, (simpler version of 2 and 4, single domain) Phases I & II have been assumed to be achieved their objectives originally planned such as testing the interoperability issues of the SIP signaling. We decided to revise the scenarios of phases III & IV and pursuit them as the high priority study among CJK considering the current NGN requirements. CJK NGN Test-bed Phase III: 2008 1st/2nd Quarter Performance Evaluation of RTP/RTCP-based MoIP (Multimedia Voice/Audio/ Video) over IP) service Network connective for Performance evaluation and Performance Monitoring Scenarios 6-1 and 6-2 Phase IV: 2008 3rd ~ 2009 2nd Quarter Performance Evaluation of RTP/RTCP-based MoIP and IPTV services Performance Monitoring Scenarios 6-3 and 6-4 Initial RACF Interoperability testing : Scenario 7 Testing of RACF and RTP/RTCP-based MPM Interactions IPTV Interoperability testing

Test Scenario 6 & 7 in Phase III & IV Performance Evaluation with RTP/RTCP-based Measurement extensions and initial RACF interoperability testing. Scenario 6-1: Performance evaluation of Network Segments Scenario 6-2: Performance evaluation of MoIP service among CPEs Local testing of MPM (Provided by KDDI R&D Labs.) CPE-to-CPE performance evaluation Scenario 6-3: Performance evaluation of MoIP service with two MPMs CPE – MPM1 – MPM2 – CPE performance evaluation Scenario 6-4: Performance evaluation of MoIP service with three MPMs CPE – MPM1 – MPM2 – MPM3 – CPE performance evaluation Scenario 7: Testing of RACF and RTP/RTCP-based MPM Interactions

CPE terminals and other equipment Scenarios 6 & 7 Details Scenario No. IP Network Service Platform Services CPE terminals and other equipment Evaluations Countries Report Metrics Connections 6-1 C-K: R&D Network C-J: Commercial Network K-J: Commercial Network SIP RTP 　　　　Streams (MoIP: Multimedia over IP) Voice/ Audio/ Video) >>IPTV CPE (VoIP/Video): J (Soft/Hard), K, C Call Server: K, C, J（Emulator) MPM: J (Soft/Hard), K Y.1543 － Local test C,J,K Two MPMs　　 C-J, K-J, C-K Three MPMs C-J-K 13th CJK meeting 2008.09 6-2 RTCP (CPE-CPE) CPE-CPE 6-3 RTCP (CPE-MPM1-MPM2-CPE) CPE-MPM 6-4 RTCP (CPE-MPM1-MPM2-MPM3-CPE) 7 SIP/ IMS-SIP RACF: K, J（1st: ACF, 　　　　　　2nd: RCF) MPM-RACF 14th or 15th CJK meeting

Thanks You CJK NGN Test-bed Ad-hoc Group