1. Benchmarking Methodology WG (bmwg) 65th IETF – Dallas, Texas, USA
Tuesday, March 21, 2006, 13:00-15:10 (Miro)
Chairs:
Kevin Dubray
Al Morton
If you are not subscribed to the BMWG mailing list and would like to be, put an asterisk (*) by your name when you sign the blue attendance sheet.  
When you start, request that folks sign the blue attendance sheet; monitor it’s progress. (You’ll probably have a light turnout given the Monday evening session, but who knows.)
Try to conscript a minute taker. (It’s challenging taking good notes and leading the session.)
Please record the addresses from the blue sheet of those folks w/asterisks by their name, send them to me, and I’ll add them to the list. Please note for me then number of people that attended the session.
Have the minute taker send the minutes to This needs to be done before August 15th.
Turn the blue list into the secretariat’s/registration desk. (If they are closed down when you get there, you can give the sheets to them on Tuesday.)
AD summary: send to (and me!) a short, 1 paragraph summary of the results of the WG session. This should be done ASAP, but before the closing Friday.

2. BMWG Agenda Note-Taker(s), IPR, and Blue Sheets
Extended WG Status (Chairs, 10 min)
IGP-Dataplane Convergence I-Ds. (Poretsky, 15 min)
Core Router Accelerated Life Testing I-Ds (Poretsky, 15 min)
Traffic Control Methodology (Poretsky, 15 min)
IPsec Benchmarking Terms & New Methods (Kaeo, 15 min)
Milestones Status (Chairs, 15 min)
New Work (Chairs, 35 min)
Protection Bechmarking (Vapiwala)
IPv6 Bechmarking (Dugatkin)
Kevin gets the last words today…
Administrivia:
Have the agenda approved.
Go over the BMWG activity since the last meeting. (Slide 3)
After Kimura presents, please get a sense from the group assembled whether they would support this as a new WG deliverable. If so, say that well have to propose the new work item on the mailing list.
When you get to the resource reservation topic, poll the group and see who has read the draft. If none, stop.
If some have read the draft, glean from them whether they believe the draft should be forwarded to the Ads…

3. BMWG Activity AD/IESG Review WG Last Call I-Ds
<draft-ietf-bmwg-benchres-term-06.txt>
<draft-ietf-bmwg-dsmterm-12.txt>, begin WGC shepherding
WG Last Call
<draft-ietf-bmwg-igp-dataplane-conv-term-10.txt> LC ended 10/26
<draft-ietf-bmwg-igp-dataplane-conv-meth-10.txt>X-Area Comments
<draft-ietf-bmwg-igp-dataplane-conv-app-010.txt>
<draft-ietf-bmwg-hash-stuffing-05.txt> WGLC 2/3, Comments
<draft-ietf-bmwg-ipsec-term-08.txt> will proceed with Meth
I-Ds
<draft-ietf-bmwg-ipsec-meth-01.txt> new
<draft-ietf-bmwg-acc-bench-term-08.txt> Ready for WGLC?
<draft-ietf-bmwg-acc-bench-meth-04.txt> Ready for WGLC?
Last Call: you’ll know the results by mtg time; please announce.
I-Ds:
DSM: Editors believe this might be the last call version, Jerry Perser is there to update folks.
The OSPF convergence drafts are the new WG work items from the last meeting.
Other:
The author of the FIB draft has solicited input on various topics of the draft, with no response. So, we’ll probably move to last call soon.
The last call of the Resource Reservation benchmarking terminology draft produced NO input, even though the last call was posted to numerous WGs. Hopefully you can get input at the meeting.
The multicast methodology draft expired. The editor has shown signs of life; hopefully, outstanding comments to this draft will be addressed soon and a last call reissued.

4. BMWG Activity Expected I-Ds
New Work proposals.
LDP Convergence – expired, but…
draft-poretsky-protection-term-00.txt (adopt?)
draft-poretsky-mpls-protection-meth-04.txt (adopt?)
draft-vapiwala-bmwg-frr-failover-meth-00.txt (adopt?)
draft-popoviciu-bmwg-ipv6benchmarking-00.txt (adopt?)
Expected I-Ds
<draft-ietf-bmwg-bgpcmeth-00.txt> expected soon?

5. BMWG Activity Expired BMWG I-Ds RFC Editor – none. Charter Update
<draft-ietf-bmwg-acc-bench-meth-ebgp-00.txt>,
<draft-ietf-bmwg-acc-bench-meth-opsec-00.txt>,
<draft-ietf-bmwg-fib-meth-03.txt>, Remove frm Goals
<draft-ietf-bmwg-benchres-method-00.txt> Pending term
RFC Editor – none.
Charter Update
Revise Milestones, and more discussion.
New RFC:
Last Call: you’ll know the results by mtg time; please announce.
I-Ds:
DSM: Editors believe this might be the last call version, Jerry Perser is there to update folks.
The OSPF convergence drafts are the new WG work items from the last meeting.
Other:
The author of the FIB draft has solicited input on various topics of the draft, with no response. So, we’ll probably move to last call soon.
The last call of the Resource Reservation benchmarking terminology draft produced NO input, even though the last call was posted to numerous WGs. Hopefully you can get input at the meeting.
The multicast methodology draft expired. The editor has shown signs of life; hopefully, outstanding comments to this draft will be addressed soon and a last call reissued.

6. Revised Milestones Past Due Milestones
Done Net Traffic Control Benchmarking Terminology to AD Review
Apr 06 IGP/Data-Plane Terminology I-D to AD Review
Apr 06 IGP/Data-Plane Methodology and Considerations I-Ds to AD Review
July 06 Hash and Stuffing I-D to AD Review
Dec 06 Router Accelerated Test Terminology I-D to AD Review
Dec 06 IPsec Device Benchmarking Terminology to AD Review
Dec 06 Router Accelerated Test Method. and Consid. I-Ds to AD Review
Dec 07 Net Traffic Control Benchmarking Methodology to AD Review
Jul 07 Basic BGP Convergence Benchmarking Methodology to AD Review.
New
Dec 06 IPsec Device Benchmarking Methodology to AD Review
Dec 07 Router Accelerated Test Method for EBGP to AD Review
Dec 07 Router Accelerated Test Method for Operational Security to AD Review

7. Recap on Protection Proposal
Proposed in 2002, Yokohama
Multiple Proposals Emerged, 2003, Vienna
Harmonized Proposal, 2005, much interest (off-list)
2006 – More Contributors, More comments on-list, new members joining to work it
Room made by progress on other items

8. Hey, Hey; My, My – 10 Years in the BMWG
Kevin Dubray

9. 10 years, so what? bgp control meth bmwg call bgp con term Dsmterm
Ethsw
Fibmeth
fibterm
firewall term
Framerelay
fr term
igp conv app
igp conv term
igp conv meth
ippm frame
ipsec meth
ipsec term
lanswitch
mcast term
framework
salt meth
bgp salt
salt opsec
salt term
atm meth
atm abr
benres meth
benres term
mcast meth
lanswitch meth
secperf meth
router framewk
ospf conv cont app
ospf conv cont term
ospf conv cont meth
hash

10. Lots of I-Ds
35 topics
197 I-Ds (over 5,000 pages of reading material for those sleepless nights!)
18 RFCs (another 6 or so in the pipe)
Had 2 RFCs in ‘96

11. Lot’s of AD’s Alvestrand Bradner Bush Curran Kessens Kostick O’Dell
Wijnen
Spin-in/Spun out WG! (IPPM)
Co-chair – Al Morton in March of ‘03
WG Chairs:
Bradner
McQuaid
Almes
Dubray
Morton

12. Lot’s of Support 250-350 subscribers to the BMWG reflector.
Over 950 attendees to BMWG meetings.
New influx of editor interest!

13. Lot’s of Spam!
73,000 pieces of spam scraped off the reflector. (estimated since introduction of mailman archive in ’98).

14. Lot’s of Use! BMWG benchmarks cited in provider RFI/RFP’s.
Cited implementation from test instrument vendor community.
Used by networking gear vendors or public testing forums to describe device capability.

15. Lot’s of Challenges! Role of Benchmark Time & Place for Everything
A test is not always a meaningful test;
A meaningful test is not always a good benchmark.
Time & Place for Everything
Know when the train is leaving the station.
But maybe that place isn’t the BMWG!
If you want a more general purpose test forum, start a BOF!
Recruit ACTIVE BMWG supporters!
Average time to deliver a BMWG RFC: 2.34 YEARS.
Couple that w/ a term -> meth turnaround…
Shortest 00 -> RFC: LAN Switch Terminology years
Longest 00-> RFC: Multicast Methodology years.