1. Cross-Layer Scheduling in Cloud Computing Systems Authors: Hilfi Alkaff, Indranil Gupta

2. Motivation Many cloud computing frameworks out there – Batch Processing Framework: Hadoop – Stream Processing Framework: Storm Current applications are not aware of underlying network topology – Might schedule tasks on machines with low bandwidth.

3. Challenges Need to expose underlying network topology efficiently to applications Huge state space to search – Thousands of machines in a cluster – Users demand more interactive jobs Multiple possible data-path representation – Want to have generic schedulers

4. Data-Path: Map-Reduce

5. Data-Path: Stream

6. Proposed Solution Cross-Layer Scheduling Framework – First-level scheduler in application Level – Second-level scheduler in routing level Use Simulated Annealing at each level – Probabilistic framework – Idea: If neighboring state is better, always move there but if it is not, move there with probability P(T) that decreases with time

7. Proposed Architecture Application Master SDN Controller Cross-Layer Scheduling

8. Algorithm: Pre-computation

9. Algorithm: Main

10. Algorithm: genState() Heuristic Too many neighboring states – Not possible to traverse all of them Application Level – Prefer node that has higher # of sink vertices – Prefer node that has higher # of source vertices Routing Level – Prefer paths that have lower number of hops – Prefer paths that have higher amount of available bandwidth

11. Emulab Result: Throughput

12. Simulation Result: Computation Time

13. Simulation Results: CDF

14. Le Questions?

15. Algorithm: Failures Link-Failures – Need to re-allocate flows using that link – Keep a separate hash-table where key=edge, value=flows – Get another path from Topology-Map. Machine-failures – Re-run main algorithm on