Design of a Scalable Clearing House Architecture Lakshminarayanan Subramanian Chen-Nee Chuah Ramakrishna Gummadi ICEBERG Design Review Jan 12, 2000

Basic Questions in Mind!!!  What is a Clearing House?  What are the basic requirements of the Clearing House?  What are the services it supports?  What are the goals of our design?  What are the assumptions we make in our design?

Clearing House  Coordinates interactions between the various ISPs in the network.  What kind of interactions?  Performs path discovery and resource reservation.  Services wide-area call requests.  Provide QoS guarantees.  Secure billing services.  Support for multicast and mobility.

Present Scenario H1 H3 ISP1 ISP3 ISP2 H2 ISP4

Goals of our design  Scalability- throughput, state maintained  Optimize network utilization  Dynamic call-routing  Continuous path monitoring for QoS  Reduce response time for call requests  Support multicast, mobility and secure billing  Recovery from link,node and packet failures  Security and Privacy

How do we achieve it!!!  Build a logical hierarchy in the network  Distribute state and resources among the nodes in the hierarchy and create a distributed database  Aggregate requests and bound queue size  Hierarchical and dynamic routing of call requests  Continuous monitoring of resources  Separate resource reservation and call-setup

Assumptions  Edge routers can collect traffic statistics and estimate bandwidth requirements  Control and data paths are separated  Clearing House and ISP trust each other  Routers can measure queueing delay statistics  Possible to introduce a monitoring system into existing ISP architecture

Clearing House Structure ISP1 ISP3 ISP2 ISP4 ICH ECH

Clearing House Infrastructure  External Clearing House(ECH) as third party agent to coordinate inter-ISP traffic.  Internal Clearing House(ICH) services intra-ISP traffic and acts as a monitoring agent for external traffic.  ECH organized as a hierarchy of nodes.  ECH stores inter-ISP network state and ICH stores intra-ISP network state in a distributed manner.

Hierarchical Structure  Divide network into non-intersecting basic domains(e.g.. Cluster area codes)  Recursively join physically adjacent domains to form larger logical domains.  Generate a hierarchical tree of domains in the network.  Associate a distributed ECH with every domain in the tree.

Hierarchical Clearing House ISP1 ISP3 ISP2 ISP4 ICH ECH ICH ECH Domain

External Clearing House  Performs hierarchical routing and computes near optimal path for call requests.  Aggregates call requests.  Collects statistics on resource reservations and delay statistics from ISPs.  Performs extra resource reservations for call requests if necessary.  Monitors performance of traffic.  Stores billing prices of ISPs within its domain

Internal Clearing House  Every ISP has an ICH.  Routes intra-ISP calls.  Monitors and predicts incoming and outgoing traffic in edge routers  Performs advanced reservations for predicted traffic and updates ECH.  Determines link reservations in ISP and updates traffic routing table of routers.

Hierarchical Routing 1a 1b 1c Inter-domain and Intra-domain paths Domain 1

Clearing house state  Billing information is present in CH of basic domain.  Each CH maintains aggregated state of its domain.  Calls between two sub-domains of its domain.  Aggregated connectivity graph between domains.  Reservation and delay status along links and nodes in the graph.  Pricing information between domains.

Other Enhancements  Caching  Cache computed inter-domain paths  RxW scheduling  Maximize throughput without affecting response time.  Measuring QoS parameters  Multicast support  Dynamic path routing to support mobility  Secure billing architecture  Fault tolerance

Support for Multicast and Broadcast Trees Nodes up in the hierarchy find inter-domain multicast tree. Local nodes find intra-domain optimal tree. Edge router

Level 0 L1 Moving Object Domain Structure Scalable Infrastructure for supporting Mobility

Strengths  State of network distributed among various CH nodes.  Aggregation of call requests.  Response time depends on locality.  Bounded queue size.  Path discovery is distributed.  Localized billing – makes it real-time.  Core routers do not maintain much state info.  Caching, scheduling improve performance.

Clearing House Design: Resource Reservation Strategies Chen-Nee Chuah ICEBERG Design Review Jan 12, 2000

ISP 1 ISP 2 ISP 3 Example Scenario  Quality of Service? Resource Reservation H2 H3H1

ISP 1 ISP 2 ISP 3 Example Scenario H2 H3H1 SLA SLA: Agreements that describe the volume of traffic exchanged, bandwidth reserved and price

Challenges  How is the SLA between two ISPs established?  How do SLAs reflect dynamic traffic patterns?  What happens when it involves more than 2 ISPs? => Clearing House provides a scalable approach to address these questions

Hierarchical Clearing House source ISP n destination Edge Router CH 1 ICH CH 1 CH 2 ISP2  Distributed database & bandwidth brokering agent reservation status, % link utilization, traffic predictor establish advanced reservation (based on traffic predictor) Updates ISP1 Adapt Reservation ICH

Resource Reservation Infrastructure H1 ISP1 H2 ICH Edge Router ICH Assume the Edge Router  collects traffic statistics  e.g. average aggregate incoming and outgoing traffic volume  estimates dynamic change of bandwidth requirements  statistical techniques (Kalman filter)  sends regular updates to LCH  aggregates reservation requests ISP2

Static & Dynamic Reservations H1 ISP1 H2 ICH Edge Router Internal Clearing House  Maintain intra-ISP reservation status  Establish static reservations based on mean aggregate traffic for different time of the day.  Adapt reservations on a smaller time-scale based on existing reservation and bandwidth predictor.  Send regular update to GCH Static Reservation Dynamic Reservation CH

Properties  Aggregation of Signaling  Resource reservation requests are aggregated at various levels (ER -> ICH, ICH-> CH 1 etc.)  De-couple notifications & reservation requests  notifications: updates on reservation status, % link utilization, traffic predictor  reservation requests: initiation of SLA renegotiations  Hierarchical Approach  Static and Dynamic Reservations  reduce reservation setup time  compensate for the coarse granularity of the notifications

Clearing House Hierarchical Tree Notifications (every  u s) - Reservation status - Link utilization - Bandwidth predictor CH 1 ICH CH 2 CH 1 ICH Adapt Reservations - Advance reservations - Process reservation requests Aggregate reservation requests (T a ) LCH

Int-Serv Approach  End-to-end notifications & reservation requests  ISP 2 notifies next-hop ISPs and negotiate new SLA with them. When all downstream ISPs accept the SLAs, an ISP notifies upstream ISPs and set up new SLAs.  When original SLA is accepted, all SLAs from source to sink are updated. source ISP1 ISP n destination BB ISP2

Diff-Serv Approach  Limited or no notifications  Trade-off end-to-end QoS for scalability source ISP1 ISP n destination Edge Router BB ISP2

Evaluation  Overall Performance Metrics  Trade-offs between scalability, QoS and signaling complexity Effect of aggregation on QoS –e.g. % blocking/dropping Choice of signaling between CHs  Link efficiency  Bandwidth Estimator  How well do the predictors track the traffic fluctuation? Window of measurement?

Clearing House Design: Billing, Security and Privacy Ramakrishna Gummadi ICEBERG Design Review Jan 12, 2000

Basic Goals  Support Scalable, Secure and Correct Billing  Support Trust Management for Traffic Monitoring  Support Privacy Management

Tasks while supporting Secure and Scalable Billing  Must scale to millions of calls per day  Must perform authentication (in both directions), authorization, and correct billing

Approaches to support Secure and Scalable Billing  Use a level of indirection through authorization and billing tickets  Generate these tickets offline  Perform offline settlements with the user and various ISPs  Use aggregation for storing and verifying tickets to reduce storage space  Use X.509 certificates, passwords or Public-key challenge/response for mutual authentication

Notes on Authorization and Billing Tickets  Both used as level of indirection, for achieving scalability, while maintaining high security and requiring little trust  Both like Kerberos, a scalable security service, using tickets for authentication and secrecy  Both acquired by the user once at the beginning, and used as needed

Notes on Authorization and Billing Tickets (contd..)  Authorization tickets used to establish that call corresponds to resources reserved  Billing tickets used to charge the user for time spent on the call  Billing tickets can be returned by the user at end of call, or more can be acquired during duration of call, as needed, to maintain correct billing records

Performance Optimizations  Can use shared-key techniques in using authorization tickets  A lot depends on the degree of trust between the CH and the ISPs (though the ISPs themselves don’t need to trust each other)  If trust possible, we can use shared-key cryptography for billing (no non-repudiation support)  Lots of performance and storage improvement through aggregation

Trust Management  CH can incorporate a Trust Management module to:  Provide a standard, general-purpose mechanism for specifying application security and credentials  Directly authorize security-critical actions, like network monitoring  Bind keys directly to authorization records to perform specific tasks  Describe delegation of trust and subsume the role of public key certificates

Privacy Management  Privacy management very difficult in the current Internet, more so in ICEBERG (because of billing)  Privacy of users and participating ISPs needed  User privacy with respect to participating ISPs achieved by anonymization in the form of indirect authorization and billing tickets