1. Flink Security Enhancements
Eron Wright –
DELL EMC
@eronwright

2. New Security Features Kerberos Authentication Support
Service-Level Authorization
Transport Security (SSL/TLS)

3. Existing Capability Hadoop Delegation Token (DT) Limitations
CLI uses Kerberos to authenticate to HDFS
HDFS provides a DT, which CLI passes to the Flink cluster
Cluster is able to access HDFS files on behalf of the user
Limitations
YARN mode only
Not useful to non-Hadoop services, e.g. Kafka.
Note: Still supported TM
DATA
AKKA JM
CLI
WEB
BROWSER
KAFKA
HDFS ZK
HTTP
Flink
Cluster
delegation token

4. Kerberos Authentication Support
“Cluster-Level Kerberos Identity”
Keytab-based
Shared by all jobs, not job-specific
Enables Kerberos authentication
Data Sources and Sinks (HDFS, Kafka…)
State Backends (ZooKeeper…)
Protects state data
ACL on znodes, files
Supported in standalone and YARN deployment modes TM
DATA
AKKA JM
CLI
WEB
BROWSER
KAFKA
HDFS ZK
HTTP
Flink
Cluster
keytab

5. Service-Level Authorization
“Restrict access to your Flink cluster”
Protects all endpoints:
Akka System (control path)
Intra-Cluster Data Transfer
Web UI
Blob Transfer (JARs…)
Simple shared secret
Configured or generated
Stored on client (~/.flink/…)
Stored in cluster
Supported in standalone and YARN TM
DATA
AKKA JM
CLI
WEB
BROWSER
KAFKA
HDFS ZK
HTTP
Flink
Cluster
keytab
secret

6. Transport-Level Security (SSL/TLS)
“SSL for all connections”
May be enabled on a per-endpoint basis
WebUI is problematic
Supported in standalone and YARN TM
DATA
AKKA JM
CLI
WEB
BROWSER
KAFKA
HDFS ZK
HTTPS
Flink
Cluster
keytab
secret
TLS cert(s)

7. Demo

8. Configuration Configure Kerberos Identity:
security.enabled: true
security.keytab: /path/to/keytab
security.principal:
Configure Service-Level Authorization:
security.cookie: (secret cookie)
Configure Transport-Level Security:
security.ssl.enabled: true
security.ssl.keystore: /path/to/keystore
security.ssl.keystore-password: (password)
security.ssl.key-password: (password)
security.ssl.truststore: /path/to/truststore
security.ssl.truststore-password: (password) TM
DATA
AKKA JM
CLI
WEB
BROWSER
KAFKA
HDFS ZK
HTTPS
Flink
Cluster
keytab
secret
TLS cert(s)

9. Summary

10. Project Status Targeted for: Flink 1.2 Contributors:
Vijay Srinivasaraghavan (Dell EMC)
Suresh Krishnappa (Dell EMC)
Design Doc: Secure Data Access on Google Docs
JIRAs:
FLINK Support for Kerberos Authentication with Keytab Credential
FLINK Implement Service-Level Authorization
FLINK Implement Transport Encryption (SSL/TLS)
FLINK Implement State Backend Security
Code:
Github: