1. 1

2. 2 T-110.6120 Publish/Subscribe Internetworking Helsinki University of Technology, Spring 2010 Lecture 5: Overview of the PSIRP architecture by Arto Karila Slides based on the PSIRP project review slides presented in Brussels on March 3, 2009 and deliverable D2.3: Architecture Definition, Component Descriptions, and Requirements

3. 3 Introduction Project objectives Consortium Working method Development process

4. 4 Project Objectives The key objectives of the PSIRP project are: 1.Define and develop a new inter-networking architecture based on the publish/subscribe (pub/sub) paradigm 2.Develop a reference implementation of the core of the new architecture 3.Analyse and measure the communications efficiency of new paradigm by quantitative parameters 4.Evaluate qualitative parameters of the new architecture with focus on security and business incentives and models 5.Develop migration and deployment strategies Other aspects: Follow the clean-slate design approach – take nothing for granted Consider migration, e.g., via overlay solution Emphasize security and socio-economics Publish the results, including source code, as much and as widely as possible Engage with Future Internet community, e.g., cooperate with FIRE (Onelab2) to test on large scale

5. 5 Consortium The consortium was formed by people that shared a common vision and wanted to work together towards it The partners of PSIRP are: 1.Helsinki University of Technology – Helsinki Institute for Information Technology (TKK-HIIT, FI) 2.RWTH Aachen University (RWTH Aachen, DE) 3.British Telecommunications Plc (BT, GB) 4.Oy L M Ericsson Ab (LMF, FI) 5.Nokia Siemens Networks Finland Oy (NSNF, FI) 6.Institute for Parallel Processing of the Bulgarian Academy of Science (IPP-BAS, BG), 7.Athens University of Economics and Business (AUEB, GR), 8.Ericsson Magyarorszag Kommunikacios Rendszerek K.F.T. (HU)

6. 6 Working Method Iterative working method –Define a pub/sub-based inter-networking architecture –Implement and validate the architecture –Revise the architecture and its implementation Work packages and tasks are allocated to teams that are formed spontaneously, across organizational boundaries –The purpose is to release creativity in a team consisting of people of different companies and nationalities The approach has proven its strength and it will be continued in the proposed Pursuit project

7. 7 Initial Planning Requirements Analysis & Design Implementation Deployment Testing Evaluation Development Process

8. 8 Technical Overview Are the fundamentals still valid? Observation: It’s all about information Hypothesis: Increased information requires information-centric network approaches Approach: Clean-slate with late binding Design methodology Main design principles Information centrism is key Information concepts Grouping information networks Information structures

9. 9 Are the Fundamentals Still Valid? Fundamentals of the Internet Collaboration –Reflected in forwarding and routing Cooperation –Reflected in trust among participants Endpoint-centric services –(mail, FTP, even web) –Reflected in E2E principle  IP, full end-to-end reachability Reality in the Internet Today Phishing, spam, viruses –There is no trust any more! Current economics favor senders –Receivers are forced to carry the cost of unwanted traffic Information-centric services –Do endpoints really matter? –Endpoint-centric services move towards information retrieval through, e.g., CDNs  IP with middle boxes & significant decline in trust in the Internet vs.

10. 10 Observation: It's All About Information Internet Today: In 2006, the amount of digital information created was 1.288 X 10^18 bits 99% of Internet traffic is information dissemination & retrieval (Van Jacobson) HTTP proxying, CDNs, video streaming, … Akamai’s CDN accounts for 15% of traffic Between 2001 and 2010, information will increase 1million times from 1 petabyte (10^15) to 1 zettabyte (10^21) Social networking is information-centric Most solutions exist in silos overlays over IP map information networks onto endpoint networks Internet Today: In 2006, the amount of digital information created was 1.288 X 10^18 bits 99% of Internet traffic is information dissemination & retrieval (Van Jacobson) HTTP proxying, CDNs, video streaming, … Akamai’s CDN accounts for 15% of traffic Between 2001 and 2010, information will increase 1million times from 1 petabyte (10^15) to 1 zettabyte (10^21) Social networking is information-centric Most solutions exist in silos overlays over IP map information networks onto endpoint networks Internet Tomorrow: Proliferation of dissemination & retrieval services, e.g., context-aware services & sensors aggregated news delivery augmented real life Personal information tenfold in the next ten years (IBM, 2008) Increase of personalized video services e.g., YouTube, BBC iPlayer Vision recognized by different initiatives & individuals Internet of Things, Van Jacobson, D. Reed Lack of interworking of silo solutions will slow innovation and development speed Internet Tomorrow: Proliferation of dissemination & retrieval services, e.g., context-aware services & sensors aggregated news delivery augmented real life Personal information tenfold in the next ten years (IBM, 2008) Increase of personalized video services e.g., YouTube, BBC iPlayer Vision recognized by different initiatives & individuals Internet of Things, Van Jacobson, D. Reed Lack of interworking of silo solutions will slow innovation and development speed

11. 11 Hypothesis: Increased Information Requires Information-centric Network Approaches Application developers care about information concepts –Creation of information topologies of various kinds -> Endpoint-centric networking structures are inadequate –Topological network changes too slow in timescale –Topological network boundaries too restrictive –Topological network boundaries often not aligned with information topologies –Overlaying possible but restricted in (developer) scalability -> If it is all about information, why not route on information?

12. 12 Approach Clean-slate design… Question ALL fundamentals Challenge our thinking Take nothing for granted, including industry structures Clear vision …with late binding (to reality) Consider migration and evolvability in separate work items –How to get our design into real deployments, e.g., overlay vs. IP replacement? Even consider necessary evolution of industry (& regulatory) structures –How do industries need to evolve in certain scenarios?

13. 13 Design Methodology Combine bottom-up and top-down –Implementation matters but rationalization is necessary Adaptive design is crucial –Information-centrism is expected to help for areas like metadata & policy Aligns well with cycle- based project approach –Creates micro-cycles of question/remove Choice Goals Principles Design Patterns & Considerations Components Choice Derive Map Specify Implement Instance Question Constraints Remove Add/Remove VISION Deployment Deploy & evaluate Observe SoA

14. 14 Main Design Principles Information is multi-hierarchically organised –Information semantics are constructed as directed acyclic graphs (DAGs) Information scoping –Mechanisms are provided that allow for limiting reachability of information to parties Scoped information neutrality –Within each information scope, data is only delivered based on a given (rendezvous) identifier. The architecture is receiver-driven –No entity shall be delivered data unless it has agreed to receive those beforehand. Communication Model Information Hierarchies Information reachability/ scoping

15. 15 Information Centrism is Key Information is everything and everything is information –Bootstrap other concepts, e.g., identity, policy, …, Scopes build information networks Policy is metadata –So is scope! Producers and consumers need no internetwork-level addressing! FatherFriendSpouseColleague Scope Family Scope Company A Data: Picture Data: Mail Governance policy Scope Friends Governance policy Governance policy

16. 16 Information Concepts Information –Smallest something Information collections –Sets of semantically similar information Information networks –Sets of information under some common governance Information producer –Entity publishing information to a particular network Information consumer –Entity subscribing to information in a particular network

17. 17 Grouping Information Networks

18. 18 RId 2 Information collections alg RId Information Structures RId 1 RId 2 Information items SId 1 Information networks SId 2 alg SId Private Information networks SId 2

19. 19 Architecture High-level architecture Architecture process Architectural entities Identifiers Algorithmic identifiers Architectural processes Architecture overview Components Application considerations Conclusions

20. 20 High-Level Architecture RP: Rendezvous point ITF: Inter-domain topology formation TM: Topology management FN: Forwarding node ITF Topology RP Rendezvous Network Network Architecture Service Model Helper Error Ctrl … Fragmentation Caching TM Forwarding Network Forwarding Network Forwarding Network Forwarding Network FN pub sub Apps Node Architecture

21. 21 Architecture Process The aim of the architecture work is to produce coherent design for a publish/subscribe internetwork –Principles, framework, components, internetworking –Networks of information –Using pub/sub in all parts of the protocol suite The architecture work is iterative –Interaction with implementation and validation workpackages Encompasses both clean-slate and incremental overlay-based solutions In practice work is done in a number of architecture teams with participants from different work packages

22. 22 Architectural Entities Information items –Any data that can be labelled with an identifier –Items can have different identifiers depending on the level of abstraction Application identifier, rendezvous identifier, forwarding identifier Algorithmic identifiers Information networks (or scopes) –Information items grouped into a network of information under a scope –Scopes can be interpreted at various levels of abstraction Information subscribers and publishers Domains –Administrative network areas that can be connected using inter- domain forwarding architecture

23. 23 Identifiers Publish / Subscribe Metadata (source is implementation-dependent) Data Scope Identifiers (SId) Includes... Associated with... Application Identifiers (AId) Rendezvous Identifiers (RId) Forwarding Identifiers (FId) Network Transit Paths Includes... Resolved to... Define...

24. 24 Algorithmic Identifiers Algorithmic identifiers are identifiers that are created by a function from one or more existing identifiers They can be used to determine relationships between information items –Ordering –Composition –Completeness Algorithmic identifiers can also be used to increase security and privacy

25. 25 Architectural processes Rendezvous –The process of resolving higher level identifiers to lower level identifiers within a given scope. –Three simple cases: link-local, intra-domain, inter-domain Topology management and formation –Management of data delivery topologies and forwarding graphs Forwarding –Data delivery within a single administrative domain or across multiple domains. –Temporal forwarding identifiers for each publisher and subscriber are derived via the rendezvous and topology management processes –Various routing and forwarding protocols can be used, for example a new protocol replacing IP or IP-based overlays

26. 26 Architectural processes Helper functions –Extensions to core functionality of the network architecture, such as management and transport Network attachment –Discovery of network attachment points and network configuration

27. 27 SubscriberPublisher Forwarding edge nodes Forwarding node Forwarding node Topology AS Rendezvous Data Forwarding Subscribe Forwarding node Topology AS Rendezvous Forwarding node Publish Create delivery path Configure Forwarding path Architecture Overview AS Topology

28. 28 Component Wheel The network architecture is based on a modular and extensible core called the PSIRP Component Wheel Components may be decoupled in space, time, and context –Layerless protocol suite Typical components include rendezvous, forwarding, helper functions, and caching. Applications may insert or request new components to the wheel at runtime. The components are attached to the local blackboard –Shared components, publications, state –Pub/sub is used to signal changes to blackboard state

29. 29 Component Wheel

30. 30 Compare w/ Haggle Architecture [Sco2006] J. Scott, P. Hui, J. Crowcroft, C. Diot, Haggle: A Networking Architecture Designed Around Mobile Users, IFIP WONS 2006, Les Menuires, France, 2006

31. 31 Component Wheel Interactions

32. 32 Service Model and API We have considered four different classes of network services: 1.A low-level page model that exposes network forwarding and rendezvous/topology formation functions Basic building block on top of raw forwarding API –Pages and no error or congestion control Listen(FiD), Forward(FiD, meta-data, data) Mapping of information items to FiD Pub/sub API towards higher levels 2.A mid-level memory object model Memory pages are mapped to publications 3.A mid-level channel model 4.Various high-level service models including shared state and document models

33. 33 APIs Low-level page API Memory Object API Channel API Higher-level APIs

34. 34 Rendezvous Many faces of rendezvous –Link-local, inter-domain, information services, communal services, … The main requirements for this functionality are: –Scalability to Internet-like networks and data space sizes –Efficiency of operation, measured in signalling overhead and overall latency –Deployability The network is defined in terms of domains and their interconnections –Interconnections between domains include upstream, transit, downstream –Rendezvous networks are units of deployment for the PSIRP rendezvous functionality. The rendezvous networks are formed by rendezvous nodes (RNs), organized as a BGP-like inter-domain hierarchy

35. 35 Rendezvous and Forwarding Rendezvous Forwarding Rendezvous Forwarding R R R Local Intra- domain Inter- domain

36. 36 Rendezvous Networks RN A RN B RN B10 RN B100 N Sub RN C N Pub RP RN C10 RN C100 Interconnection overlay Rendezvous network RN X Rendezvous node End node NXNX RP Sub Pub Rendezvous point Subscriber Publisher Rendezvous signaling

37. 37 Intra- and Inter-Domain Operation Forwarding is configured by the rendezvous system with the help of the topology management function Key challenge is scalability Intra-domain forwarding –Initial work on Bloom-filter based forwarding mechanism indicates that they are useful for sizes up to metropolitan area networks –FiDs identify partial distribution graphs –Minimal state in the routers Inter-domain forwarding –Solution must take performance and policy requirements into account –Initial work on DHT-based overlay and understanding implications for inter-domain routing –Inter-domain topology function

38. 38 Security and Packet Level Authentication (PLA) Direct and indirect cryptographic association using identifiers and rendezvous –Public keys and one-way hash values –Algorithmic identifiers Packet Level Authentication (PLA) is one candidate protocol for securing PSIRP network functions –We assume that per packet public key cryptography operations are feasible in Internet's scale because of new digital signature algorithms and advances in semiconductor technology –PLA is a novel solution for protecting the network infrastructure against various attacks (e.g., DoS) by providing availability –Each packet has a signature (ECC) –Good analogy for PLA is a paper currency: anyone can verify the authenticity of the bill by using built-in security measures like watermark and hologram, there is no need to contact the bank that has issued the bill –The network should be able to fulfill its basic goal: to deliver valid packets of valid users in reliable and timely manner in all situations

39. 39 Prototype Implementation PSIRP Apps Legacy Apps Sockets API Emulator Apps PSIRP Library API Upper layers (possibly Python) Libraries Kernel PSIRP Kernel API Lower layers (C/C++) Ethernet Wi-Fi IP GMPLS Packet Level Authentication (PLA)

40. 40 Application Considerations Application programming interfaces –Inherently information centric –1-to-1 message stream Similar to TCP/IP socket API –1-to-N bidirectional connection Similar to UDP over IP multicast –1-to-N document distribution Similar to CDN and P2P protocols Examples –WWW functionality –BitTorrent-like content distribution

41. 41 Example: Scoping in CS-comms. Using a scope to initiate client-server-like communications

42. 42 Example: Invitation in CS-comms. Using an invitation to initiate client-server-like communications

43. 43 Example: Adding a Node to Tree Simplified signaling for adding a node to a concast-like tree

44. 44 Example: Interactions in the component wheel Interactions in the component wheel

45. 45 zFilter Forwarding Tables

46. 46 Link ID Tag (LIT) Generation

47. 47 LIT Router Model

48. 48 Multicast-Assisted Mobility

49. 49 Conclusions We have outlined an information centric network architecture Publish and subscribe are the basic primitives making multicast the norm Decoupling in space and time Receiver driven (subscriber has control) Rendezvous as the primitive to connect publishers and subscribers across domains on multiple levels –Mapping to forwarding structures Scoping of data into manageable sets Architecture work is iterative Implementation and evaluation are on-going activities