Machine Learning for Routing During Congestion Events Kurtis Heimerl

Objectives Congestion Events- Congestion Events are when a router has a saturated out port. This port must drop some packets, as more are entering than exiting. Solution- Use machine learning to determine “important” packets. Route these packets at a higher priority, dropping other, less important packets. Use active testing and killing of services to do the above determination

Current Events Existing solutions to saturated line- Random FIFO packet dropping Advantages- Efficient Problems- Important services are not protected. Other services depend on these, causing unnecessary outages. Examples including the recent Skype outage, and a few Amazon.com outages

Idea! Use machine learning to determine important services to protect. Active testing methodology Interrupt packet streams(TCP connections) to determine these dependencies Route based on the determinations of machine learning DHCP server should be given priority Hadoop master should also be given priority Time server should not Hadoop slave should not Attempt to maximize the number of leaves in the dependency graph we've developed

The Unknown Most of this is fairly simple... except- Can I accurately determine when a service is dead? May have to modify hadoop to die in a particularly easy to detect manner Does this generalize? Need concerted effort to ensure that I don't pick the test bed such that the results are meaningless

The Plan 1-2 Weeks Have virtualized switched network 2-3 Weeks Implement simple death detection in the virtual switch network 3-5 Weeks Implement the active testing and recording methodology 5-7 Weeks Gather data