Framework for Metric Composition + Spatial Composition of Metrics

Al Morton + many others

December 3, 2007

2

draft-ietf-ippm-framework-compagg-05.txt

Overall Framework (includes common

concepts and definitions)

TemporalAggregation

Draft

SpatialAggregation

Draft

Spatial Composition

Draft

… andmaybe others

draft-ietf-ippm-spatial-composition-04

3

Types of Composition

Complete/Sub-Path (or Concatenation in Space)

Aggregation in Time (12x5min stats ->1hr)

Aggregat. in Space

4

Framework 05: Comments from Alan Clark - make the draft a little more generic and apply beyond IPPM

Add that metrics can be composed from dissimilar metrics (see new sentence in 3.4 and Spatial draft Scope of Work 3.1)

Add new definition of Index, e.g., R-factor (see new 3.6) Add a new requirement in Section 5, to state the (statistical)

model of the process being measured (and the validity of the assumption) if applicable (RFC 2330 Sec 12 guides this to some degree).

Describe an “Intermediate Model” in Section 4: capable of representing some key aspect of the performance of a class of applications using metrics on the system of interest. >> This seems different from our composition/aggregation scope. It might take one or more complete path metrics and estimates some reaction at higher layers… <<

5

Recent Comments on 05

Reza Fardid Section 3.6 R-factor is a valid example of an index as

defined here, however, it is composed of both IP performance metrics and analog metrics. On this basis, is it within scope of this draft? Agree some refinement is needed here There is a “packet-performance” subset of the E-model,

where all the analog metrics are fixed Loki’s point: Scope is Loss, Delay, and Delay Variation

Another relevant index composed of IP performance metrics is MDI, per rfc4445 Let’s just say “NO” to referencing REF Editor

submissions…

6

Recent Comments on 05

Reza Fardid Section 4.1 Is the comment referring to

forecasting of one metric from another, whose temporally aggregated measurements are available? Al believes what was meant here is like weather

forecasting – prediction of future If so, then I fail to see how this is feasible given

that the primary metrics have been defined to be orthogonal. Agreed

7

Recent Comments on 05

Reza Fardid Section 4.5 Higher-order Composition: Another example will be helpful here. In particular, if a composed metric is derived from

dissimilar metrics, e.g., R-factor or MDI, then can one make a general statement or provide guidance with respect to the order of composition to minimize measurement error?

For instance, is it preferred to compose sub-path R-factor or MDI metrics (assuming their composition functions are defined) derived from primary metrics, or compose the sub-path primary metrics first, followed by R-factor or MDI composition?

Is there statistical or empirical evidence to support either order in general?

8

Recent Comments on 05Loki Jorgenson Section 3.2 - there is a reference to "true path" with no entry

under Terminology to define it. Is it related to "ground truth" (which is defined)? If it is not intended to be a unique term but only a descriptor, I would suggest changing it to something more like "actual“ (as used in "ground truth").

Section 4.1 - with regard to weighting function and transient vs. intransient measures (see comments further down).

Section 4.5 - second sentence has "would be a the delay" Section 5. - As I was dealing with IP issues today and aspects

of viral licensing, is it appropriate to include in any composition metric description IP references to embedded proprietary metrics and/or use of standards?

Otherwise, I note, most of the requirements in section 5. are technical.

…(a longer discussion on time varying and steady-state categories for ground truth that I haven’t had time to summarize)

9

Status of Framework 05

Next Steps Address the recent comments Consider this text sufficiently stable? If so, Convince more folks to Read and

Comment on…

10

Status: Spatial Composition 05

Revisions to Outline to keep in step with Framework Modifications up 05 Statement of Conjecture and Assumptions Added Assumptions in each of the re-titled

sections NEXT STEPS:

Need Volunteers to read and comment