1. Resilient Low Memory Networks: Self-healing plus Compact Routing
Amitabh Trehan
Queen’s University Belfast
Queen’s University Belfast

2. Internet Of Things (IOT)/Overlay/ Peer to Peer networks

3. Distributed Algorithms
A Network
A Printer
Multiple agents in a network
communicating to solve a problem
e.g. they may ask
which one of us will use the printer?

4. Example Problem: Leader Election
Token
Leader
Fundamental problem (Le Lann, 1977)
Massively useful e.g. in large scale networks (P2P(Akamai), ad-hoc, sensor etc.)
Symmetry breaking
Token Ring Networks

5. Distributed Computing: Message Passing model
Each node has a processor with infinite processing power (We’ll revisit this assumption later).
Communication is by sending messages along edges:
Only local knowledge to begin with
Metrics: time (# parallel rounds), # messages exchanged etc.

6. But things are not perfect …
Adsense Mar 2010, Google, May 15, 2009
Twitter, August 6, 2009
Facebook, August 6, 2009
Skype, August 15, 2007
Recent failures in internet systems demonstrate our inability to engineer
robust distributed systems.
- just a few days ago a denial of service attack brought down two of the most popular systems on the internet.
- About a year ago a simple windows update brought down Skype - another popular and important internet system.

7. Rest of the talk
Part 1: Resilient Distributed Architectures by Self- healing
Part 2: Low Memory devices: Compact Routing over Compact Self-healing Networks

8. Here comes the adversary....
An adversary is someone who wants to fail our algorithm!
A way to model faults and attacks!
Allows us to move from static to dynamic (node/edge changes) ...
Fail-Stop faults: self-healing (follows...)

9. How to self-heal?
Brain: component fails, brain rewires and does without it
Computer networks: components fail, network fails until components fixed.
Are these events due to simple engineering failures or are they due to a more general failure in imagining what it means for a system to be robust?
e.g. in natural systems we have this notion of robustness which is missing in engineered systems e.g. in the brain ....

10. An autonomic system Self-managing: Self-configuring Self-healing
Self-optimizing
Self-protecting
Mention what self-healing is.
Source:
Self-Configuring (Increased responsiveness): Adapt to dynamically changing environments.
Self-Healing (Business resiliency): Discover, diagnose and act to prevent disruptions.
Self-Optimizing (Operational efficiency): Tune resources and balance workloads to maximize use of IT resources.
Self-Protecting (Secure information and resources): Anticipate, detect, identify, and protect against attacks.

11. Autonomic Computing IBM’s autonomic computing initiative Self-CHOP
Self-Configuring (Increased responsiveness): Adapt to dynamically changing environments.
Self-Healing (Business resiliency): Discover, diagnose and act to prevent disruptions.
Self-Optimizing (Operational efficiency): Tune resources and balance workloads to maximize use of IT resources.
Self-Protecting (Secure information and resources): Anticipate, detect, identify, and protect against attacks.

12. Resilience and Security @QUB
Project AllScale (€3.4 Million European Grant): Developing (resilient) exascale computing!
PhD project with SAP: Running analytic queries in presence of faulty clusters.
ECIT-2: New Global Science Institute combining Security (CSIT) and Large Scale Scalable data analytics.

13. Self-healing
A self-healing system, starting from a correct state, under attack from an adversary, goes only temporarily out of a correct state.
Under attack from powerful adversary, maintain certain properties within acceptable bounds.

14. Model Start: a distributed network G.
An adversary inserts or deletes nodes .
After each node addition/deletion, we can add and/or drop some edges between pairs of nearby nodes, to “heal” the network.

15. Self-healing illustrationv a

21. Naive?

27. v
And so on...

28. Problem v v a G G 3
Degree(v,G0) = 2
Degree(v,G3) = 5

29. Possible healing topologies: Line graphv a v G G 3
Low degree increase but diameter/ distances blow up

30. Possible healing topologies: Star graphv v a G G 3
Low distances but degree blows up

31. Challenge 1: properties conflict
Low degree increase => high diameter/stretch/ poorer expansion?
Low diameter => high degree increase?

32. Challenge 2: local fixing of global properties
Limited global Information with nodes
Limited resources and time constraints

33. Self-healing (Topological) Goals
Healing should be fast.
Certain (topological) properties should be maintained within bounds:
Connectivity
Degree (quantifies the work done by algorithm)
Diameter/ Stretch
Expansion/ Spectral properties

34. Our SH Algorithms Virtual Non-Virtual Forgiving Tree DASH
Forgiving Graph
Non-Virtual
DASH
(Connectivity, Degree)
+Diameter
Compact
Routing
Low Memory
+Expansion
+Stretch
Xheal
Xheal+
Edge Preserving SH
CompactFT
+Density
DEX
Deterministic Expanders
CompactFTZ

35. Example: Forgiving Tree*
A distributed algorithm, such that, for any network G with max degree ∆, for an arbitrary sequence of deletions,
Graph stays connected
Diameter increases by ≤ log(∆) factor
Degrees increase by ≤ 3 (additive)
Each repair takes O(1) time involving O(∆) nodes.
*Tom Hayes, Navin Rustagi, Jared Saia and Amitabh Trehan, “The forgiving tree: a self-healing distributed data structure”, in Principles of Distributed Computing (PODC), 2008.

36. On a rooted spanning tree of G:
Basic Techniques
On a rooted spanning tree of G:
Replace (non-leaf) v by a Reconstruction Tree (RT) of virtual nodes (in oval). The ‘real’ neighbors are the leaves of the tree.
(leaf deletions) Short-circuiting a redundant virtual node
*Efficient implementation by ‘wills’

37. §2: Compact Self-healing Routing

41. A letter from London

42. Routing the letter

43. Routing the letter

44. Routing the letter

45. Routing the letter

46. Routing the letter

47. Routing the letter

48. Routing the letter

49. Routing the letter

50. Routing the letter

51. Routing the letter

52. Routing the letter

53. Routing the letter

54. Routing the letter

55. But what is there is a sinkhole…..

56. But what is there is a sinkhole…..

57. But what is there is a sinkhole…..

58. But what is there is a sinkhole…..

59. But what is there is a sinkhole…..

60. Problem 1
How to Route a message from a sender u to a receiver v despite failures in the network?

61. Imagine IOT/Overlay/large networks of small devices

62. Problem 2
How to Route a message from a sender u to a receiver v despite node failures in a network of nodes with low memory?

63. Problem 2a
How to Route a message from a sender u to a receiver v despite node failures in a reconfigurable/overlay network of nodes with low memory?

64. Compact/ Bounded memory Self-healing (deletion only) Model
Start: a network G.
Adversary fails nodes in unknown order v1, v2, ..., vn
After each node deletion, we can add and/or drop some edges between pairs of nearby nodes, to “heal” the network
Each node uses less than linear (i.e. o(n)) memory (= Compact)

65. Solution: Intuition v u
A compact Routing Scheme S that successfully routes in absence of failures. v u

66. Solution: Intuition v RS u
A compact Routing Scheme S that successfully routes in absence of failures.
In the event of a node failure, compactly self-heal by replacing node with a reconstruction structure (RS) supporting a Routing Scheme R. v RS u

67. Solution: Intuition v u
A compact Routing Scheme S that successfully routes in absence of failures.
In the event of a node failure, compactly self-heal by replacing node with a reconstruction structure (RS) supporting a Routing Scheme R.
S + R: Transparent integrated compact routing with small additional stretch (equal to max distance in RS). v u

68. Our Implementations/ Algorithms}
A compact Routing Scheme S that successfully routes in absence of failures.
In the event of a node failure, compactly self-heal by replacing node with a reconstruction structure (RS) supporting a Routing Scheme R.
S + R: Transparent integrated compact routing with small additional stretch (equal to max distance in RS).
TZ: Modified Thorup-Zwick
Routing over trees ^ }
COMPACTFT: Compact (O(log n) local memory) version of ForgivingTree*
Binary Search Tree Traversal }
COMPACTFTZ: SH Compact (O(log2 n) local memory) routing
^M. Thorup and U. Zwick. Compact routing schemes. In SPAA, pages 1–10, 2001.
*Tom Hayes, Navin Rustagi, Jared Saia and Amitabh Trehan, “The forgiving tree: a self-healing distributed data structure”, in Principles of Distributed Computing (PODC), 2008.

69. Static Routing
u wants to send a packet to v. Two pieces of information:
Packet Header
Local fields of all nodes on the path p(u,v) (i.e. ‘routing tables’)
TZ: Post-order DFS Interval based routing (easy with large memory!)
v, label(v)
Information…. u v

70. Low Memory Challenges Only O(log n) bits memory = constant # node IDs
Implications:
A node cannot store IDs of its children, list of live ports or know which child ID deleted
A node can store parent’s ID and will portion (in a tree)

71. Part 3: Bringing it all together
CompactFTZ (high level):
Preprocessing (regular memory): BFS spanning tree of the network + DFS labelling and TZ setup + CompactFT data structure
Do forever {
COMPACTFT: On node deletion, neighbours replace deleted node by its RT and maintain SH tree
S + R: TZ on real nodes + BST routing on RTs (virtual nodes) }

72. A quick example 8 1 3 3’ 5’ 8’
Target=8
Information….

73. CompactFTZ properties
A compact self-healing routing scheme using only O(log2n) memory despite node deletions
Repairs done in constant parallel time
packet (u->v) delivered if v has not been deleted. If v deleted, u gets informed and the packet removed from the network.
Additional SH Latency: +O(y log ∆), where y is the number of nodes on path(u,v) in spanning tree.

74. History of ‘provably good’ compact routing
1985
Santoro and khatib
The Computer journal :
1988
Peleg and Upfal
STOC
1999
Cowen
SODA
2001
Thorup andZwick
SPAA
2002
Korman et al
STACS
2013
Chechik
PODC

75. Summary Self-healing model to deal with node failures/insertions
A series of SH algorithms to locally maintain global properties
Low memory highly efficient SH algorithm CompactFT
CompactFTZ: SH compact routing scheme

76. The Future …
Internet of Things: Lot of small low memory devices dynamically connecting in ad-hoc manner
Software Defined Networking: Flexible networks of the future!
Networks and Systems of huge scale: Need Scalable solutions
High Dynamicity: fast changing conditions.

77. Remember, we may be down…

78. but with some help….

79. never out!