Download presentation
Presentation is loading. Please wait.
1
Enterprise grade security in your Hadoop clusters on Azure
Microsoft 2016 4/17/2018 5:35 PM BRK3186 Enterprise grade security in your Hadoop clusters on Azure Saurin Shah Sr. Program Manager © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
2
Session Objectives Key Takeaways
Microsoft Ignite 2016 4/17/2018 5:35 PM Session Objectives Overview of HDInsight – Hadoop and Spark offering on Azure cloud. Configure Perimeter security using Virtual networks Integrate HDInsight with Azure Active Directory Configure multi-user authentication, authorization, auditing using Apache Ranger Encryption of Data at Rest with HDInsight Key Takeaways HDInsight is the solution that will you need if you want enterprise grade security capabilities for your Hadoop environment on cloud © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
3
What is HDInsight? 4/17/2018 5:35 PM
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
4
Demo Quickly create an HDInsight cluster Microsoft Ignite 2016
4/17/2018 5:35 PM Demo Quickly create an HDInsight cluster © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
5
Move to Enterprise Readiness
4/17/2018 5:35 PM Move to Enterprise Readiness © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
6
Hadoop clusters have grown by 60% in the last 2 years
Microsoft Ignite 2016 4/17/2018 5:35 PM Hadoop clusters have grown by 60% in the last 2 years 89% of enterprise users consider Hadoop as opportunity for innovation Forrester report predicts that Hadoop will grow by 33% annually in next five years Hadoop is shifting from a buzzword to a real production service Ownership is shifting from department teams to Central IT. © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
7
Meet Samer, Director of IT
4/17/2018 5:35 PM Meet Samer, Director of IT © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
8
Microsoft Ignite 2016 4/17/2018 5:35 PM CustomerId Name CellPhone
Address City State Zip Credit Card 413707 LUNA PARK 3250 W FOSTER AVE CHICAGO IL 60625 391234 MARIE 4729 N LINCOLN AVE 413751 MANU WORKY 11601 W TOUHY AVE 60666 413708 STEVE BENCH 325 N LA SALLE ST BLDG 60654 … ... CustomerId Reviews Rating 413707 SPICY, YET HEALTHY. WOULD ORDER AGAIN 9.3 391234 HATS OFF TO MAINTAIN PROPER 4.6 413751 AMAZING FOOD PREPARED RIGHT AT 9.4 413708 Decent Food 7.1 … …. Id CustomerId OrdersPlaced Discount Date Revenue 102456 68252 277 $526.30 8/1/2016 $2,243.70 102457 413488 282 $84.60 $2,735.40 102458 250405 134 $281.40 $1,058.60 102459 114533 141 $253.80 $1,156.20 102460 315209 289 $346.80 $2,543.20 … Id Customer ID Time Taken Cost Date 102456 68252 63 $224.00 8/1/2016 102457 413488 65 $235.00 102458 250405 67 $245.00 102459 114533 71 $227.00 102460 315209 72 $213.00 … © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
9
Product demand analysis Delivery and Operations
Microsoft Ignite 2016 4/17/2018 5:35 PM Cluster Admin Product demand analysis Delivery and Operations Developer 2 Data Scientist 2 Developer 1 Data Scientist 1 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
10
HDInsight Security – Rings of Defense
Microsoft Ignite 2016 4/17/2018 5:35 PM HDInsight Security – Rings of Defense Perimeter Level Security Virtual Network Network Security (i.e. Firewalls) Gateway Authentication Kerberos Active Directory Authorization Hive policies HBase policies File and Folder level ACLS Data Security Rest © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
11
Perimeter Security 4/17/2018 5:35 PM
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
12
Using Virtual Network and Gateway Service
Microsoft Ignite 2016 4/17/2018 5:35 PM Using Virtual Network and Gateway Service Perimeter Level Security Virtual Network Network Security (i.e. Firewalls) Gateway © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
13
Gateway HiveServer2 Head Node Ambari Worker node (s) Oozie Services
Microsoft Ignite 2016 4/17/2018 5:35 PM DataScientist Gateway Head Node HDInsight Cluster WASB ADLS VNET HiveServer2 Ambari Oozie Worker node (s) Services © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
14
Demo Setup HDInsight cluster inside a VNET Microsoft Ignite 2016
4/17/2018 5:35 PM Demo Setup HDInsight cluster inside a VNET © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
15
Authentication, Authorization and Auditing
4/17/2018 5:35 PM Authentication, Authorization and Auditing © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
16
Integration with Azure Active Directory
Microsoft Ignite 2016 4/17/2018 5:35 PM Integration with Azure Active Directory Authentication Kerberos Active Directory © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
17
Application and Data-Level Authorization
Microsoft Ignite 2016 4/17/2018 5:35 PM Application and Data-Level Authorization Authorization Hive policies HBase policies File and Folder level ACLS © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
18
Azure VNET to VNET peering
Microsoft Ignite 2016 4/17/2018 5:35 PM Active Directory Domain Services AAD tenant DataScientist 1 DataScientist 2 Domain Credentials VNET VNET Gateway Head Node HDInsight Cluster WASB ADLS Worker node (s) Kerberos AuthN Kerberos Ticket HiveServer2 Ambari Oozie LDAP Azure VNET to VNET peering Services Ranger Ranger DB OAuth Ticket © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
19
Demo Authentication, Authorization and Auditing Microsoft Ignite 2016
4/17/2018 5:35 PM Demo Authentication, Authorization and Auditing © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
20
Secure Endpoints in HDInsight cluster
Access to all users Access to only Cluster Admin HiveServer2 Ambari & Views Ranger SSH WebHCat Oozie
21
Encryption of data @ Rest
4/17/2018 5:35 PM Encryption of Rest © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
22
Transparent Server Side Encryption
Microsoft Ignite 2016 4/17/2018 5:35 PM Transparent Server Side Encryption Data Security Rest © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
23
Transparent Server Side Encryption
Azure Data Lake Storage Windows Azure Storage Blob Public Preview ALWAYS ON transparent encryption All reads/writes are encrypted/decrypted Service managed keys as well as Customer managed keys General Availability ALWAYS ON transparent encryption All reads/writes are encrypted/decrypted Service managed keys
25
Public Preview & Roadmap
4/17/2018 5:35 PM Public Preview & Roadmap © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
26
Public Preview (in next few weeks)
AAD Integration Multi-user Ambari Ranger for Hive All Tooling (Excel, Visual Studio, SDKs)
27
Session Objectives Key Takeaways
Microsoft Ignite 2016 4/17/2018 5:35 PM Session Objectives Overview of HDInsight – Hadoop and Spark offering on Azure cloud. Configure Perimeter security using Virtual networks Integrate HDInsight with Azure Active Directory Configure multi-user authentication, authorization, auditing using Apache Ranger Encryption of Data at Rest with HDInsight Key Takeaways HDInsight is the solution that will you need if you want enterprise grade security capabilities for your Hadoop environment on cloud © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
28
Free IT Pro resources To advance your career in cloud technology
Microsoft Ignite 2016 4/17/2018 5:35 PM Free IT Pro resources To advance your career in cloud technology Plan your career path Microsoft IT Pro Career Center Cloud role mapping Expert advice on skills needed Self-paced curriculum by cloud role $300 Azure credits and extended trials Pluralsight 3 month subscription (10 courses) Phone support incident Weekly short videos and insights from Microsoft’s leaders and engineers Connect with community of peers and Microsoft experts Get started with Azure Microsoft IT Pro Cloud Essentials Demos and how-to videos Microsoft Mechanics Connect with peers and experts Microsoft Tech Community © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
29
Please evaluate this session
4/17/2018 5:35 PM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
30
4/17/2018 5:35 PM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Similar presentations
