Considerations From an IPv6 Product Developer Thomas Narten May 4, 2007, NIST

2 Thomas Narten Background  IBM has long history of supporting IPv6  Active contributors to IETF IPv6 effort  AIX shipped IPv6 product in 1997  Currently ship IPv6 in i5/OS, AIX, z/OS (all have IPv6 Ready Logo Phase I certification)  Significant developer of IPv6 functionality in Linux  Number of our products support IPv6 –  IBM is a strong supporter of IPv6!

3 Thomas Narten Overview  Three flavors of testing  Cost issues for vendors  Product & logistical issues  Harmonization of USG testing efforts  Leverage existing testing programs  Self-certification  Publication of Test Criteria  Ensure adequate accountability

4 Thomas Narten Three Flavors of Testing  USG operated  3 rd Party operated  Self-certification  Questions for each: –How quickly can it ramp up service? –At what rate can it evaluate products? (e.g., number per month?) –Can testing be timely? –What are scaling properties (impacts almost every product)? –Where does product expertise come from? –Who bears cost? –In practice, what will actual cost be?

5 Thomas Narten Significant Money May Be At Stake  Testing not free; someone bears cost (both direct and indirect) –Assumption: cost will fall on product vendors  If cost too high, some vendors will simply opt out –Consequence: reduced product choice for USG  Business built on providing testing service can be self-serving –Predictable revenue stream needed for business plan –“required” testing potentially attractive –Avoid creating a “business” in IPv6 testing

6 Thomas Narten Offsite & Third Party Testing Costs  Requires hardware to be shipped to test site –Not practical for large servers, high-end configurations –Not always trivial to acquire actual hardware –Shipping fees  Direct fee costs to third party –Membership fee –Per-product fee –Facilities space –Third party training (to setup/use/test product)  Travel for testing engineer –Travel cost –Time away from office

7 Thomas Narten Product Considerations  Vendor may have 100s of products  Need to avoid/minimize redundant testing –Many releases of (essentially) same product –Different configurations of same products –Many applications share code –Products may contain OEM components that have already been tested –Not desirable to test/certify each one separately; need way to leverage results of prior testing  Some products are operating system agnostic –Run on top of OS provided by another vendor –Product may be sold independent of underlying hardware/OS

8 Thomas Narten Harmonize USG Testing Requirements  Cannot afford to go through same testing process multiple times for different USG agencies  Ideally, harmonize USG testing requirements...  Even if final profiles differ, 80% of the RFCs overlap  Thus, 80% of testing should also overlap

9 Thomas Narten Leverage/Reuse Existing Testing Programs  IPv6 Ready Logo (Phase I) already covers core IPv6 protocols –RFCs: 2460 (IPv6), 2461 (ND), 2462 (addrconf), 4443 (ICMPv6) (PMTU-D) –Additional Phases as well (e.g., DHCPv6, IPsec, etc.) –For those already certified, what is benefit of additional testing?  Interoperability testing of IPsec has already been done by ICSA or VPN Consortium

10 Thomas Narten Make Test Criteria & Test Suites Publicly Available  Provides transparency w.r.t. details of actual functionality tested  Vendors can test in own labs, as part of product development and test cycle –Facilitates pre-release testing (can be problematical to have outside party test pre-release software) –Significantly reduced cost to vendor  Allows vendor to prepare in advance of an external test (where efficiency is important)  Must be free of IPR concerns  Wide availability of TAHI suites has benefited community

11 Thomas Narten Self-Certification Highly Desirable  Has worked well in practice (e.g., IPv6 Ready Logo, Y2K preparation, all of TCP/IP, etc.)  Increasingly necessary as one moves higher up the stack (e.g., into applications) –Significant application-specific expertise needed to test the product –Infeasible for outside party test the number and range of products  Self-certification to a publicly available criteria –Most efficient –Scales well –Good balance between cost and benefit  Neutral third party can verify claims if needed

12 Thomas Narten Accountability of Testing Program  Any testing/certification suite must provide accountability  IETF defines SHOULD as follows: –“SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.”  Cannot simply require implementation of all SHOULDs  Need a workable process to resolve disagreements between IPv6 tester and product developer  Need an open process to review test criteria  Need an open process to fix “bugs” in test criteria

13 Thomas Narten Questions?