Benchmarking Methodology WG (bmwg) 65th IETF – Dallas, Texas, USA Tuesday, March 21, 2006, 13:00-15:10 (Miro) Chairs: Kevin Dubray (kdubray@juniper.net) Al Morton (acmorton@att.com) If you are not subscribed to the BMWG mailing list and would like to be, put an asterisk (*) by your name when you sign the blue attendance sheet.   When you start, request that folks sign the blue attendance sheet; monitor it’s progress. (You’ll probably have a light turnout given the Monday evening session, but who knows.) Try to conscript a minute taker. (It’s challenging taking good notes and leading the session.) Please record the email addresses from the blue sheet of those folks w/asterisks by their name, send them to me, and I’ll add them to the list. Please note for me then number of people that attended the session. Have the minute taker send the minutes to minutes@ietf.org. This needs to be done before August 15th. Turn the blue list into the secretariat’s/registration desk. (If they are closed down when you get there, you can give the sheets to them on Tuesday.) AD summary: send to randy@psg.com, bwijnen@lucent.com, (and me!) a short, 1 paragraph summary of the results of the WG session. This should be done ASAP, but before the closing Friday.

BMWG Agenda Note-Taker(s), IPR, and Blue Sheets Extended WG Status (Chairs, 10 min) IGP-Dataplane Convergence I-Ds. (Poretsky, 15 min) Core Router Accelerated Life Testing I-Ds (Poretsky, 15 min) Traffic Control Methodology (Poretsky, 15 min) IPsec Benchmarking Terms & New Methods (Kaeo, 15 min) Milestones Status (Chairs, 15 min) New Work (Chairs, 35 min) Protection Bechmarking (Vapiwala) IPv6 Bechmarking (Dugatkin) Kevin gets the last words today… Administrivia: Have the agenda approved. Go over the BMWG activity since the last meeting. (Slide 3) After Kimura presents, please get a sense from the group assembled whether they would support this as a new WG deliverable. If so, say that well have to propose the new work item on the mailing list. When you get to the resource reservation topic, poll the group and see who has read the draft. If none, stop. If some have read the draft, glean from them whether they believe the draft should be forwarded to the Ads…

BMWG Activity AD/IESG Review WG Last Call I-Ds <draft-ietf-bmwg-benchres-term-06.txt> <draft-ietf-bmwg-dsmterm-12.txt>, begin WGC shepherding WG Last Call <draft-ietf-bmwg-igp-dataplane-conv-term-10.txt> LC ended 10/26 <draft-ietf-bmwg-igp-dataplane-conv-meth-10.txt>X-Area Comments <draft-ietf-bmwg-igp-dataplane-conv-app-010.txt> <draft-ietf-bmwg-hash-stuffing-05.txt> WGLC 2/3, Comments <draft-ietf-bmwg-ipsec-term-08.txt> will proceed with Meth I-Ds <draft-ietf-bmwg-ipsec-meth-01.txt> new <draft-ietf-bmwg-acc-bench-term-08.txt> Ready for WGLC? <draft-ietf-bmwg-acc-bench-meth-04.txt> Ready for WGLC? Last Call: you’ll know the results by mtg time; please announce. I-Ds: DSM: Editors believe this might be the last call version, Jerry Perser is there to update folks. The OSPF convergence drafts are the new WG work items from the last meeting. Other: The author of the FIB draft has solicited input on various topics of the draft, with no response. So, we’ll probably move to last call soon. The last call of the Resource Reservation benchmarking terminology draft produced NO input, even though the last call was posted to numerous WGs. Hopefully you can get input at the meeting. The multicast methodology draft expired. The editor has shown signs of life; hopefully, outstanding comments to this draft will be addressed soon and a last call reissued.

BMWG Activity Expected I-Ds New Work proposals. LDP Convergence – expired, but… draft-poretsky-protection-term-00.txt (adopt?) draft-poretsky-mpls-protection-meth-04.txt (adopt?) draft-vapiwala-bmwg-frr-failover-meth-00.txt (adopt?) draft-popoviciu-bmwg-ipv6benchmarking-00.txt (adopt?) Expected I-Ds <draft-ietf-bmwg-bgpcmeth-00.txt> expected soon?

BMWG Activity Expired BMWG I-Ds RFC Editor – none. Charter Update <draft-ietf-bmwg-acc-bench-meth-ebgp-00.txt>, <draft-ietf-bmwg-acc-bench-meth-opsec-00.txt>, <draft-ietf-bmwg-fib-meth-03.txt>, Remove frm Goals <draft-ietf-bmwg-benchres-method-00.txt> Pending term RFC Editor – none. Charter Update Revise Milestones, and more discussion. New RFC: Last Call: you’ll know the results by mtg time; please announce. I-Ds: DSM: Editors believe this might be the last call version, Jerry Perser is there to update folks. The OSPF convergence drafts are the new WG work items from the last meeting. Other: The author of the FIB draft has solicited input on various topics of the draft, with no response. So, we’ll probably move to last call soon. The last call of the Resource Reservation benchmarking terminology draft produced NO input, even though the last call was posted to numerous WGs. Hopefully you can get input at the meeting. The multicast methodology draft expired. The editor has shown signs of life; hopefully, outstanding comments to this draft will be addressed soon and a last call reissued.

Revised Milestones Past Due Milestones Done Net Traffic Control Benchmarking Terminology to AD Review Apr 06 IGP/Data-Plane Terminology I-D to AD Review Apr 06 IGP/Data-Plane Methodology and Considerations I-Ds to AD Review July 06 Hash and Stuffing I-D to AD Review Dec 06 Router Accelerated Test Terminology I-D to AD Review Dec 06 IPsec Device Benchmarking Terminology to AD Review Dec 06 Router Accelerated Test Method. and Consid. I-Ds to AD Review Dec 07 Net Traffic Control Benchmarking Methodology to AD Review Jul 07 Basic BGP Convergence Benchmarking Methodology to AD Review. New Dec 06 IPsec Device Benchmarking Methodology to AD Review Dec 07 Router Accelerated Test Method for EBGP to AD Review Dec 07 Router Accelerated Test Method for Operational Security to AD Review

Recap on Protection Proposal Proposed in 2002, Yokohama Multiple Proposals Emerged, 2003, Vienna Harmonized Proposal, 2005, much interest (off-list) 2006 – More Contributors, More comments on-list, new members joining to work it Room made by progress on other items

Hey, Hey; My, My – 10 Years in the BMWG Kevin Dubray

10 years, so what? bgp control meth bmwg call bgp con term Dsmterm Ethsw Fibmeth fibterm firewall term Framerelay fr term igp conv app igp conv term igp conv meth ippm frame ipsec meth ipsec term lanswitch mcast term framework salt meth bgp salt salt opsec salt term atm meth atm abr benres meth benres term mcast meth lanswitch meth secperf meth router framewk ospf conv cont app ospf conv cont term ospf conv cont meth hash

Lots of I-Ds 35 topics 197 I-Ds (over 5,000 pages of reading material for those sleepless nights!) 18 RFCs (another 6 or so in the pipe) Had 2 RFCs in ‘96

Lot’s of AD’s Alvestrand Bradner Bush Curran Kessens Kostick O’Dell Wijnen Spin-in/Spun out WG! (IPPM) Co-chair – Al Morton in March of ‘03 WG Chairs: Bradner McQuaid Almes Dubray Morton

Lot’s of Support 250-350 subscribers to the BMWG reflector. Over 950 attendees to BMWG meetings. New influx of editor interest!

Lot’s of Spam! 73,000 pieces of spam scraped off the reflector. (estimated since introduction of mailman archive in ’98).

Lot’s of Use! BMWG benchmarks cited in provider RFI/RFP’s. Cited implementation from test instrument vendor community. Used by networking gear vendors or public testing forums to describe device capability.

Lot’s of Challenges! Role of Benchmark Time & Place for Everything A test is not always a meaningful test; A meaningful test is not always a good benchmark. Time & Place for Everything Know when the train is leaving the station. But maybe that place isn’t the BMWG! If you want a more general purpose test forum, start a BOF! Recruit ACTIVE BMWG supporters! Average time to deliver a BMWG RFC: 2.34 YEARS. Couple that w/ a term -> meth turnaround… Shortest 00 -> RFC: LAN Switch Terminology. 1.08 years Longest 00-> RFC: Multicast Methodology. 5.68 years.