1. Ubiquitous Instrumentation Richard Mortier MSR Cambridge mort@microsoft.com

2. Measuring networks Active –traceroute/ping/etc (ICMP) –SNMP MIBs –Non-critical functionality  buggy Passive –NetFlow, IPFIX, sFlow –Port spanning, VLAN spanning –Router/switch only  poor visibility, scalability

3. Claims 1.Routers are just specialized hosts Hardware, protocols, configuration 2.Hosts are part of the network as well Transmit, receive, forward data

4. Claims 1.Routers are just specialized hosts Hardware, protocols, configuration 2.Hosts are part of the network as well Transmit, receive, forward data The strong distinction that has grown between them makes understanding network behaviour difficult –Traffic is opaque to routers (and becoming more so) –Network is black box service to hosts (and becoming more so) Hosts provide inputs –…we should use them to understand demands Routers provide resource –…their configuration implements constraints

5. Networking measurements Host-based measurement – Anemone –Instrumented stack, kernel structures –Per-EXE per-packet live network stats –Unify flows with routing topology Distributed query system – Seaweed –Access to Anemone flow/packet tables –Built over Pastry, highly scalable (10 6 nodes+)

6. Measure for measure Ubiquitous network measurement –Undercarriage of the Knowledge Plane –Infrastructure for autonomic-foo? Concretely… –Combining host & network data – SeaStar Single trust domain – how to expand? –User-visible diagnosis – Constellation What do users want to know about the network?

7. Measure for measure Data gathering and management –Existing tools primitive: tend toward manual analysis –Need performant, type-safe, distributed processing –Theoretical aspects of temporal, spatial sampling Unifying many different datasets –Routing and configuration data –Pcap and netflow and SNMP and … data Mining structure, relationships from data –Machine learning techniques seem a very rich vein –Robust automated processing techniques –Distributed trust, provenance, privacy