Introducing – SAS® Grid Manager for Hadoop Cheryl Doninger, SAS Doug Haigh, SAS Copyright © 2010, SAS Institute Inc. All rights reserved.

About the presenter I started with SAS in 1986 and am currently a Senior R&D Director. My teams work on many of the foundation technologies providing the compute capabilities of SAS: SAS Grid, SAS/CONNECT, all host teams, Core, IOM and WorkSpace as well as SAS Environment Manager. I have a Master’s from NC State and hold 2 patents related to SAS Grid Computing. #SASGF Copyright © 2016, SAS Institute Inc. All rights reserved.

A Bit of Background… SAS Grid designed for Workload management High availability Performance Architected to support multiple providers Platform Suite for SAS (LSF, PM) Hadoop (YARN, Oozie)

Why SAS Grid Manager for Hadoop? Co-location of SAS Grid jobs on Hadoop cluster Requires – Integration with YARN Supported enterprise Hadoop distribution Kerberos Spare capacity to accommodate additional workload Nodes architected to be compute nodes

#SASGF Copyright © 2016, SAS Institute Inc. All rights reserved.

#SASGF Copyright © 2016, SAS Institute Inc. All rights reserved.

#SASGF Copyright © 2016, SAS Institute Inc. All rights reserved.

What Behavior Can I Expect? Consistent job submission Grid launched WS servers Grid servers created with SAS/CONNECT Batch submission with SASGSUB Batch submission with Schedule Manager plug-in All SAS Grid integration is the same SAS Grid jobs will run unchanged

What Is Different? Hadoop is not part of the product Monitoring/management via Hadoop interfaces Kerberos is required Need for shared file system is reduced Data will need to be migrated to HDFS Need to understand the capabilities of YARN Each SAS Grid job results in (at least) two YARN containers

Conclusion Gas powered vs. electric

#SASGF Copyright © 2016, SAS Institute Inc. All rights reserved.