Testbeds and the evolving research paradigm Shivendra Panwar Based on discussions with Fraida Fund, Pei Liu and Thanasis Korakis Polytechnic Institute of New York University Brooklyn, NY CCW 2012

How has the research paradigm changed in networking? Major corporations do not dominate research as they once did Puts more of a spotlight on academic researchers This implies the need to push research further out to reality Increases the importance of testbed-based research Even testbed “failures” are often successful! A counter-example, and is this all good?

Platforms for Future Internet Experimentation Fidelity to “real” Internet Programmability of network Simulation (ns-2/ns-3,OPNET) Simulation (ns-2/ns-3,OPNET) Emulation (Emulab, DETER) Emulation (Emulab, DETER) Testbed with Programmable Network (ORBIT, OFELIA, CORNET) Testbed with Programmable Network (ORBIT, OFELIA, CORNET) Collection of Heterogeneous Testbeds (GENI, FIRE projects) Collection of Heterogeneous Testbeds (GENI, FIRE projects) Internet as a Testbed (Seattle) Internet as a Testbed (Seattle) Distributed Testbed (PlanetLab) Distributed Testbed (PlanetLab)

Platforms for Future Internet Experimentation Fidelity to “real” Internet Complex experiments Repeatability Complex experiments Less repeatable Results likely to hold in “real” world Results less likely to hold in “real” world Controlled experiments

Platforms for Future Internet Experimentation Programmability of network Steep learning curve Support clean-slate experiments, fundamental changes Limited ability Technology can be deployed today Easy to learn Good for testing future technology

Shared Design Challenges Future-proof: Design testbeds to study solutions that haven’t been developed yet Flexible use: Support “clean slate” experiments but allow experimenters to re- use existing technologies Open: Simple, open architectures encourage user-contributed tools and resources.

Shared Operational Challenges Support real experimenters: Adapt to the way experimenters actually use testbed, rather than what designers anticipated Clear expectations: Help users understand advantages and limitations of testbed platform and design experiments accordingly