1. Benchmarking Methodology WG (bmwg) 93rd IETF
Thursday, July 23rd, 2015
( Central Europe Daylight Time, UTC+2:00)
Chairs:
Al Morton (acmorton(at)att.com)
Sarah Banks (sbanks(at)encrypted.net)
PLEASE MOVE CLOSE TO THE FRONT
If you are not subscribed to the BMWG mailing list and would like to be, please go to
When you start, request that folks sign the blue attendance sheet; monitor it’s progress. (You’ll probably have a light turnout given the Monday evening session, but who knows.)
Try to conscript a minute taker. (It’s challenging taking good notes and leading the session.)
Please record the addresses from the blue sheet of those folks w/asterisks by their name, send them to me, and I’ll add them to the list. Please note for me then number of people that attended the session.
Have the minute taker send the minutes to This needs to be done before August 15th.
Turn the blue list into the secretariat’s/registration desk. (If they are closed down when you get there, you can give the sheets to them on Tuesday.)
AD summary: send to (and me!) a short, 1 paragraph summary of the results of the WG session. This should be done ASAP, but before the closing Friday.

2. Note Well
This summary is only meant to point you in the right direction, and doesn't have all the nuances.
The IETF's IPR Policy is set forth in BCP 79; please read it carefully (next slide)
For further information, talk to a WG chair, ask an Area Director, or review the following:
BCP 9 (on the Internet Standards Process)
BCP 25 (on the Working Group processes)
BCP 78 (on the IETF Trust)
BCP 79 (on Intellectual Property Rights in the IETF)

3. Intellectual Property Rights (IPR) Policy
The IETF intellectual property rights rules are defined in RFC 3979, "Intellectual Property Rights in IETF Technology" (updated by RFC 4879, "Clarification of the Third Party Disclosure Procedure in RFC 3979").
The IETF takes no position regarding the validity or scope of any intellectual property rights or other rights that might be claimed to pertain to the implementation or use of the technology described in any IETF documents or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights.
IETF Contribution: any submission to the IETF intended by the Contributor for publication as all or part of an Internet-Draft or RFC (except for RFC Editor Contributions described below) and any statement made within the context of an IETF activity. Such statements include oral statements in IETF sessions, as well as written and electronic communications made at any time or place, which are addressed to:
* the IETF plenary session,
* any IETF working group or portion thereof,
* the IESG, or any member thereof on behalf or the IESG,
* the IAB or any member thereof on behalf of the IAB,
* any IETF mailing list, including the IETF list itself, any working group or design team list, or any other list functioning under IETF auspices,
* the RFC Editor or the Internet-Drafts function (except for RFC Editor Contributions described below).
Statements made outside of an IETF session, mailing list or other function, that are clearly not intended to be input to an IETF activity, group or function, are not IETF Contributions in the context of this document.
A participant in any IETF activity is deemed to accept all IETF rules of process, as documented in Best Current Practices RFCs and IESG Statements.
A participant in any IETF activity acknowledges that written, audio and video records of meetings may be made and may be available to the public.

4. BMWG Agenda (Any Bashing needed?)
Note-Taker(s), Jabber, IPR, Blue Sheets
1a. New Charter and Milestones
1b. WG Status Presenter: Chairs
2. DataCenter Benchmarking Presenter: Jacob
3. IPv6 Neighbor Discovery Benchmarking Presenter: Ron
4. VNF and Infrastructure Benchmarking Considerations Presenter: Al
5. IPv6 Transition Benchmarking Presenter: Marius
6. Benchmarking for SDN Controller Performance Presenter: Sarah
7. vSwitch Benchmarking in OPNFV Presenter: Maryam Tahhan
8. Benchmarking Meth for Virtual Network Perf Presenter: Gu Rong
LAST. AOB
Administrivia:
Have the agenda approved.
Go over the BMWG activity since the last meeting. (Slide 3)
After Kimura presents, please get a sense from the group assembled whether they would support this as a new WG deliverable. If so, say that well have to propose the new work item on the mailing list.
When you get to the resource reservation topic, poll the group and see who has read the draft. If none, stop.
If some have read the draft, glean from them whether they believe the draft should be forwarded to the Ads…

5. Quick WG Status BGP Dataplane – IESG Approved - MISSREF
Traffic Management – IESG Approved
In-Service SW Upgrade – IESG Telechat August 6
NEW WG drafts on DC Bench & IPv6 ND
WG Adopted Considerations draft:Virtual Network Function and related Infrastructure
New work proposals keep coming

6. BMWG Activity New RFCs: Charter Update none Supplementary BMWG Page
7501 and 7502 SIP Device Benchmarking !!
Charter Update
none
Supplementary BMWG Page
See
UPDATED!
Last Call: you’ll know the results by mtg time; please announce.
I-Ds:
DSM: Editors believe this might be the last call version, Jerry Perser is there to update folks.
The OSPF convergence drafts are the new WG work items from the last meeting.
Other:
The author of the FIB draft has solicited input on various topics of the draft, with no response. So, we’ll probably move to last call soon.
The last call of the Resource Reservation benchmarking terminology draft produced NO input, even though the last call was posted to numerous WGs. Hopefully you can get input at the meeting.
The multicast methodology draft expired. The editor has shown signs of life; hopefully, outstanding comments to this draft will be addressed soon and a last call reissued.

7. Current Milestones
DONE Basic BGP Convergence Benchmarking Methodology to IESG Review
DONE Terminology for SIP Device Benchmarking to IESG Review
DONE Methodology for SIP Device Benchmarking to IESG Review
DONE Draft on Traffic Management Benchmarking to IESG Review
Dec Draft on IPv6 Neighbor Discovery to IESG Review
DONE Draft on In-Service Software Upgrade Benchmarking to IESG Review
Aug Draft on VNF Benchmarking Considerations to IESG Review
Dec Drafts on Data Center Benchmarking to IESG Review

8. Standard “Paragraph” (intro/security)
Benchmarking activities as described in this memo are limited to
technology characterization using controlled stimuli in a laboratory
environment, with dedicated address space and the constraints
specified in the sections above.
The benchmarking network topology will be an independent test setup
and MUST NOT be connected to devices that may forward the test
traffic into a production network, or misroute traffic to the test
management network.
Further, benchmarking is performed on a "black-box" basis, relying
solely on measurements observable external to the DUT/SUT.
Special capabilities SHOULD NOT exist in the DUT/SUT specifically for
benchmarking purposes. Any implications for network security arising
from the DUT/SUT SHOULD be identical in the lab and in production
networks.

9. Work Proposal Summary Matrix
Work Area >
Criteria \/
SDN Controller
IPv6 Transition
vSwitch
OPNFV vSWperf
Proposal Y
In Scope of Charter? (acm)
Draft(s)
Sig. Support at meetings
Sig. Support on List
Dependencies/Notes

10. BMWG Activity Related Active Docs (not working group documents):
Draft name Rev. Status Comments
Not recently active ?
draft-hamilton-bmwg-ca-bench-term
draft-manral-bmwg-power-usage-03.txt
draft-player-dcb-benchmarking