Plumbing the Computing Platforms of Big Data Dilma Da Silva Professor & Department Head Computer Science & Engineering Texas A&M University

Where should I put host my big data app? Systems == hardware + software enabled to run applications (+admins) Opportunity for two-way collaboration: –You know which platform you want to use, but your application is not performing well in its target platform Get resources where needed when needed –You are not sure about the right platform for the application –Your app can server as representative workload for evaluation system software research ideas 2/13/15TAMU Big Data Workshop2

Do I need large scale systems ? Different communities, different requirements: –High End Computing / SuperComputing –Scale-out enterprise systems But similar trends ?! 2/13/15TAMU Big Data Workshop3

My app is running, now what? ‘plug’ your app in the system, and it just works –Support for evolution Efficiency: work done without wasting resources Elasticity: grow and shrink to adapt to demand Scale: can handle the ‘big’ part of your problem Agility Heterogeneity Resiliency: failures happens without loss of data or work Problem determination: system helps you find your ‘bugs’, Security, privacy On the cheap 2/13/15TAMU Big Data Workshop4

Computing Industry Offerings (and where CSE people can help) High End Computing (HPC, SC, commodity clusters) –Often application needs to be optimized for the platform Large gap between domain and platform experts Cloud computing platforms –So far optimized for enterprise systems 2/13/15TAMU Big Data Workshop5