1. Overview SCALE14x 2016

2. Agenda/Schedule -Apache Bigtop Overview -Apache Spark Overview/Getting Started -Lunch Break -Apache Ignite -Workshop, tutorial, open time http://workshops.bigtop.rocks (click on Agenda button)

3. What is Bigtop? Setting the standard for testing, packaging and integration of leading big/fast data components

4. and many other… Components as Building Blocks

5. ------------------------------------------------------------------------- Dependency Hell!! hdfs zookeeper hbase kafka spark. mapred oozie hive etc ---------------------------------------------------------- Build all the Things!!!

6. The BOM Build of Materials (BOM) * List of >=1 components * Gradle for build/actions * Produce sets of debs/rpms

7. Bigtop Origins Yahoo!, 2010 Created, fostered early Hadoop community Working on Hadoop 0.20 stack 2011 Yahoo!’s to Cloudera, solving early problems of packaging and maintaining first commercial supported Hadoop distro

8. Early value add Provide a common foundation for proper integration of growing number of Hadoop family components Foundation provides solid base for validating applications running on top of the stack(s) Provide neutral packaging and deployment/config

9. Early Mission Accomplished Foundation for commercial Hadoop distros/services Leveraged by app providers…

10. What now? We are done right?1?!?

11. Industry/Ecosystem Evolution & New Community Needs/Ideas

12. Where should we spend our time?, which users should benefit?

13. Moving beyond oob mapreduce…

14. Lambda/Stream Architectures HDFS + Zookeeper +

15. Get out from the Apache dome

16. New focus and target end users Data engineers vs distro builders Enhance Operations/Deployment Reference implementations & tutorials

17. Laying new foundation with 1.0+ Self-starter, non-kitchen sink building -Making gradle tooling smarter -Jenkins job autogen -leveraging containers for parallelization

18. Data data data… Smarter/Realistic test data -bigpetstore -bigtop-bazaar -weather data gen Tutorial/Learning Data sets -githubarchive.org -more tbd…

19. Deployment/Mgmt Updated puppet modules -newest best practices -next level enhanced security options Wider range of starter deployment topologies Include some handling of test/tutorial data

20. More components…

21. Sounds interesting, how can I help? *Join mailing list, ask questions, suggest features, etc *Contribute (components, tutorials, docs) *Report bugs

22. Thank You, Q&A Nate D’Amico kaiyzen@apache.org @kaiyzen