1. Protecting a Tsunami of Data in Hadoop
HPE Security – Data Security
March 2017

2. Increased Adoption of Big Data Across Industries
European Telco: Collecting Call Data Records (CDRs) from 27 countries. Used for fault detection, roaming data, network optimization. Includes number of PI data elements
Top 5 Global Car Manufacturer: Collecting sensor data from 5 million cars globally to find defects. Includes GPS location information.
Vast
Quantities Of
PI Data
Top retailer: Analyze customer buying patterns, brand loyalty, detect credit card fraud, etc.
Health Insurer: Analysis sensitive customer health insurance to detect prescription medication fraud, insurance overpayments and to customize their portal.
(C) 2016 Hewlett Packard Enterprise - Confidential

3. Why is securing Hadoop difficult?
Rapid innovation in a well funded open source community
Systems such as Hadoop do not have “Delete” or “Update” functionality
Multiple feeds of data in real time from different sources with different protection needs
Mainframe MQ
RDBMs
XML
Salesforce
Flat
Files

4. Why is securing Hadoop difficult?
Access by many different users with varying analytic needs
Automatic replication of data across multiple nodes once entered into the HDFS data store
Reduced control if Hadoop clusters are deployed in a cloud environment

5. Existing ways to secure Hadoop
Existing IT security
Network firewalls
Logging and monitoring
Configuration management
Enterprise-scale security for Apache Hadoop
Apache Knox: Perimeter security
Kerberos: Strong authentication
Apache Ranger: Monitoring and management
Need to augment these with “data-centric” protection of data in use, in motion and at rest

6. Introducing “Data-centric” security
Threats to
Data
Traditional IT
Infrastructure Security
Data
Ecosystem
Security Gaps
Data and applications
Credential
Compromise
Authentication
Management
Security gap
Middleware
Traffic
Interceptors
SSL/TLS/firewalls
Security gap
SQL injection,
Malware
Databases
Database encryption
Data security coverage
Security gap
Malware,
Insiders
SSL/TLS/firewalls
File systems
Security gap
Malware,
Insiders
Disk encryption
Storage

7. Introducing “Data-centric” security
Threats to
Data
Traditional IT
Infrastructure Security
Data
Ecosystem
Security Gaps
HPE SecureData
Data-centric Security
Data and applications
Credential
Compromise
Authentication
Management
Security gap
Middleware
Traffic
Interceptors
SSL/TLS/firewalls
Security gap
SQL injection,
Malware
Databases
Database encryption
Data security coverage
End-to-end Protection
Security gap
Malware,
Insiders
SSL/TLS/firewalls
File systems
Security gap
Malware,
Insiders
Disk encryption
Storage

8. Format-Preserving Encryption (FPE)
Meet Requirements for Encryption and Pseudonymisation
Format-Preserving Encryption (FPE)
Credit card
SSN/ID
DOB
Full
Partial
Obvious
AZ UYTZ 4321
AZS-UD-2356

9. Before: All applications and users have access to data
HR Application
ETL Tool
Mainframe App
Malware
Analysts
Help Desk
DBAs
Malicious User

10. After: Data is protected at source at “Field Level”
HR Application
ETL Tool
Payments App
Malware
Analysts
Help Desk
DBAs
Malicious User

11. Malicious users, DBAs and malware: only see protected data

12. Help desk and payments apps: operate on partially protected data

13. Analysis on de-identified data
Analysts
Analysts
Analysts

14. Authorized applications access real data
Name
James Potter
Ryan Johnson
Carrie Young
Brent Warner
Anna Berman
Authorized HR Application
Key Management
SS#
Authorized Fraud Analysts
Key Management

15. Architectures for Protecting Data in Hadoop
Hadoop Cluster
HDFS 4
Upstream
Applications 1
Hadoop jobs and analytics
FPE Encrypt
Data
Landing Zone 2 5
Hadoop jobs Hive, MapReduce, etc.
Applications, analytics and data
Upstream
Applications
ETL and batch
FPE Encrypt
Data
FPE Decrypt
Data
Egress Zone 3
ETL and batch
Applications, analytics and data
Upstream
Applications
Hadoop jobs
MapReduce, Sqoop, Flume
HPE Decrypt Data 6
FPE Encrypt
Data

16. Conclusion
Multi-platform enterprises adopting a data lake architecture need a cross-platform solution for protection of sensitive data
Big data partners bring comprehensive security within Hadoop, with core capabilities for authentication, authorization and auditing
Implementing data-centric security across data stores including Hadoop—protecting data at rest, in use and in motion, and maintaining the value of the data for analytics
Together enabling comprehensive security for the enterprise, and rapid and successful Hadoop adoption!

17. Thank you
Contact information