Framework for Metric Composition + Spatial Composition of Metrics Al Morton + many others December 3, 2007

2 draft-ietf-ippm-framework-compagg-05.txt Overall Framework (includes common concepts and definitions) Temporal Aggregation Draft Spatial Aggregation Draft Spatial Composition Draft … and maybe others draft-ietf-ippm-spatial-composition-04

3 Types of Composition  Complete/Sub-Path (or Concatenation in Space)  Aggregation in Time (12x5min stats ->1hr)  Aggregat. in Space

4 Framework 05: Comments from Alan Clark - make the draft a little more generic and apply beyond IPPM  Add that metrics can be composed from dissimilar metrics (see new sentence in 3.4 and Spatial draft Scope of Work 3.1)  Add new definition of Index, e.g., R-factor (see new 3.6)  Add a new requirement in Section 5, to state the (statistical) model of the process being measured (and the validity of the assumption) if applicable (RFC 2330 Sec 12 guides this to some degree).  Describe an “Intermediate Model” in Section 4: capable of representing some key aspect of the performance of a class of applications using metrics on the system of interest. >> This seems different from our composition/aggregation scope. It might take one or more complete path metrics and estimates some reaction at higher layers… <<

5 Recent Comments on 05 Reza Fardid  Section 3.6 R-factor is a valid example of an index as defined here, however, it is composed of both IP performance metrics and analog metrics. On this basis, is it within scope of this draft? Agree some refinement is needed here There is a “packet-performance” subset of the E-model, where all the analog metrics are fixed Loki’s point: Scope is Loss, Delay, and Delay Variation  Another relevant index composed of IP performance metrics is MDI, per rfc4445 Let’s just say “NO” to referencing REF Editor submissions…

6 Recent Comments on 05 Reza Fardid  Section 4.1 Is the comment referring to forecasting of one metric from another, whose temporally aggregated measurements are available? Al believes what was meant here is like weather forecasting – prediction of future  If so, then I fail to see how this is feasible given that the primary metrics have been defined to be orthogonal. Agreed

7 Recent Comments on 05 Reza Fardid  Section 4.5 Higher-order Composition:  Another example will be helpful here.  In particular, if a composed metric is derived from dissimilar metrics, e.g., R-factor or MDI, then can one make a general statement or provide guidance with respect to the order of composition to minimize measurement error?  For instance, is it preferred to compose sub-path R-factor or MDI metrics (assuming their composition functions are defined) derived from primary metrics, or compose the sub-path primary metrics first, followed by R-factor or MDI composition?  Is there statistical or empirical evidence to support either order in general?

8 Recent Comments on 05 Loki Jorgenson  Section there is a reference to "true path" with no entry under Terminology to define it. Is it related to "ground truth" (which is defined)? If it is not intended to be a unique term but only a descriptor, I would suggest changing it to something more like "actual“ (as used in "ground truth").  Section with regard to weighting function and transient vs. intransient measures (see comments further down).  Section second sentence has "would be a the delay"  Section 5. - As I was dealing with IP issues today and aspects of viral licensing, is it appropriate to include in any composition metric description IP references to embedded proprietary metrics and/or use of standards?  Otherwise, I note, most of the requirements in section 5. are technical.  …(a longer discussion on time varying and steady-state categories for ground truth that I haven’t had time to summarize)

9 Status of Framework 05  Next Steps Address the recent comments Consider this text sufficiently stable? If so, Convince more folks to Read and Comment on…

10 Status: Spatial Composition 05  Revisions to Outline to keep in step with Framework Modifications up 05 Statement of Conjecture and Assumptions Added Assumptions in each of the re-titled sections  NEXT STEPS: Need Volunteers to read and comment