1. Wataru Takase Computing Research Center, KEK 10th February, 2016
Authentication/Authorization extension for ElasticSearch and Kibana platform
Wataru Takase
Computing Research Center, KEK
10th February, 2016
FJPPL Computing Project Annual Workshop

2. Our Goals Kerberos 5 authentication
ElasticSearch API restrictions based on:
Kerberos 5 principals
Index and/or alias names
Index types
Cluster update operations only for certain K5 principals
Kibana 4 tweaking
.kibana.%{krb5_princ} (personal index based on Kerberos 5 principal) read + write
Common .kibana.%{team} (common team index based on K5 principal) read + write (restricted)

3. Search Guard makes us happy
Search Guard is an ElasticSearch plugin
Flexible REST layer access control (User/Role; based on aliases, indices and types)
Also last year I developed an ElasticSearch plugin to limit user’s request
My plugin restricts based on user
All users refer same index.
Plugin intercepts user’s request from Kibana and appends user specific query
Search Guard restricts based on user, role, indices, types, etc.
It’s better to use Search Guard rather than to extend my plugin

4. Development of AppendFilter plugin
My plugin
Development of AppendFilter plugin
Plugin appends user specific query
tenant_id=“xxxxxxxxx”
User gets result related to own tenants
Tenant id query
CERN SSO
Kibana
ES Plugin + ES
Search Query
JSON query
JSON query
User ID
#userid admin_flag tenant_id domain_id
wtakase 0 fcb04d9a-d9f4-4bb c987ce569b9 NULL
...

5. Original JSON query My plugin
{ "sort": [{"_score": { "order": "desc", "ignore_unmapped": true }}], "query": {"filtered": { "query": {"bool": {"should": [{"query_string": {"query": "*"}}]}}, "filter": {"bool": {"must": [{"range": { "to": , "from": }}}]}} }}, "highlight": { "pre_tags": "post_tags": "fragment_size": , "fields": {} }, "size": 500 }

6. Appended Filter to JSON query
My plugin
Appended Filter to JSON query
{ "sort": [{"_score": { "order": "desc", "ignore_unmapped": true }}], "query": {"filtered": { "query": {"bool": {"should": [{"query_string": {"query": "*"}}]}}, "filter": {"bool": {"must": [{"range": { "to": , "from": }}}, {"fquery": {"query": {"query_string": {"query": ]}} }}, "highlight": { "pre_tags": "post_tags": "fragment_size": , "fields": {} }, "size": 500 }

7. @fields.project:"fcb04d9a-d9f4-4bb7-8581-8c987ce569b9"
My plugin
@fields.project:"fcb04d9a-d9f4-4bb c987ce569b9"
{"cell": "", "domain": "Personal", "project": "fcb04d9a-d9f4-4bb c987ce569b9", "project_name": "Personal_wtakase", "source": "nova", "type": "instances", "key": "active", "timestamp": , "val_int": 3} {"cell": "", "domain": "Personal", "project": "fcb04d9a-d9f4-4bb c987ce569b9", "project_name": "Personal_wtakase", "source": "nova", "type": "instances", "key": "error", "timestamp": , "val_int": 0} {"cell": "", "domain": "Personal", "project": "fcb04d9a-d9f4-4bb c987ce569b9", "project_name": "Personal_wtakase", "source": "nova", "type": "instances", "key": "building", "timestamp": , "val_int": 0} {"cell": "", "domain": "Personal", "project": "fcb04d9a-d9f4-4bb c987ce569b9", "project_name": "Personal_wtakase", "source": "nova", "type": "instances", "key": "total", "timestamp": , "val_int": 3} {"cell": "", "domain": "wwww", "project": "wwwwwwww-wwww-wwww-wwww-wwwwwwwwwwww", "project_name": "wwwwwww", "source": "nova", "type": "instances", "key": "active", "timestamp": , "val_int": 5} {"cell": "", "domain": "xxxx", "project": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "project_name": "xxxxxxx", "source": "nova", "type": "instances", "key": "error", "timestamp": , "val_int": 0} {"cell": "", "domain": "yyyy", "project": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy", "project_name": "yyyyyyy", "source": "nova", "type": "instances", "key": "building", "timestamp": , "val_int": 0} {"cell": "", "domain": "zzzz", "project": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz", "project_name": "zzzzzzz", "source": "nova", "type": "instances", "key": "total", "timestamp": , "val_int": 5}
ES returns filtered information

8. Solutions Kerberos 5 authentication
Apache mod_auth_kerb module + reverse proxy
ElasticSearch API restrictions based on:
Kerberos 5 principals
Search Guard supports it
Use REMOTE_USER variable and role
Index and/or alias names
Index types
Cluster update operations only for certain K5 principals
Search Guard may support it
Kibana 4 tweaking
.kibana.%{krb5_princ} (personal index based on Kerberos 5 principal) read + write
Apache configuration + Search Guard
Common .kibana.%{team} (common team index based on K5 principal) read + write (restricted)

9. Kerberos 5 authentication
Apache mod_auth_kerb module + reverse proxy
RewriteRule ^/?es/?(.*)$ [L,P]
kerberos
elasticsearch
authentication
reverse proxy
kerberos auth
httpd
https
reverse proxy
Kibana 4
container
<Location />
AuthType Kerberos
. . .
Require valid-user
</Location>
RewriteRule ^/(.*)$ [L,P]
Only authenticated users can access to ES and Kibana

10. ElasticSearch API restrictions
Search Guard + REMOTE_USER + LDAP
kerberos
ldap
role
elasticsearch
authorization
authentication
search guard
REMOTE_USER
reverse proxy
with REMOTE_USER
Flexible REST layer access control (User/Role; based on aliases, indices and types)
kerberos auth
httpd
https
reverse proxy
Kibana 4
container
Another authorization backend is supported:

11. ElasticSearch API restrictions
Kerberos 5 principals
Index and/or alias names
Index types
Cluster update operations only for certain K5 principals
* Any user with es_admin role can do anything
Index
restriction
public-*
Everyone can read/write
cloud-*
Any user with cloud role can read/write
grid-*
Any user with grid role can read/write
takase-*
User takase can read/write
test1-*
User test1 can read/write
test2-*
User test2 can read/write
role
REMOTE_USER
es_admin
cloud
grid o
takase
test1
test2

12. /etc/elasticsearch/elasticsearch.yaml
##################################################### # Configure the actionrequestfilter to allow or forbid action searchguard.actionrequestfilter.names: ["readonly", "noaccesible"] searchguard.actionrequestfilter.readonly.allowed_actions: ["indices:data/read/*", "*monitor*"] searchguard.actionrequestfilter.readonly.forbidden_actions: ["cluster:admin*", "indices:admin*", "indices:data/write*"] searchguard.actionrequestfilter.noaccesible.allowed_actions: [] searchguard.actionrequestfilter.noaccesible.forbidden_actions: ["cluster:*", "indices:*"]

13. ElasticSearch API restrictions
curl -XPUT ' -d '{ > "acl": [ > { > "__Comment__": "Default is to execute all filters", > "filters_bypass": [], > "filters_execute": ["*"] > }, > "__Comment__": "Any authenticated user on the 'public' index - no filter will be executed", > "indices": ["public-*"], > "filters_execute": [] > "__Comment__": "This means any user with the role cloud can do anything with the cloud index", > "roles" : ["cloud"], > "indices": ["cloud-*"], > "filters_bypass": ["*"], > “__Comment__”: “This means that for the user test2 on index grid only the actionrequestfilter.readonly will be executed, no other", > “users”: [“test2"], > "indices": ["grid-*"], > "filters_execute": ["actionrequestfilter.readonly"] > } >}’

14. Kibana 4 tweaking Problem: Our goal:
1 Kibana instance has only 1 index for storing dashboards
.kibana
All users share same Kibana index
User can read/write others’ dashboards
Our goal:
UserA only can read/write UserA dashboards
Users belonged to GroupB can read/write GroupB dashboards

15. Initial Idea: Overview of Test Environment
kerberos
ldap
Flexible REST layer access control (User/Role; based on aliases, indices and types)
role
elasticsearch
authorization
authentication
search guard
REMOTE_USER
reverse proxy
with REMOTE_USER
kerberos auth
httpd
https
reverse proxy based on REMOTE_USER
Kibana 4
container
port and kibana_index are different
Kibana 4
REMOTE_USER
kibana4 port
kibana_index
takase
15601
.kibana.takase
test1
15602
.kibana.test1
test2
15603
.kibana.test2
es_admin
25601
.kibana.es_admin
cloud
35601
.kibana.cloud
grid
35602
.kibana.grid
container
...
Kibana 4
Doesn’t scale!!
container

16. modify .kibana index request based on REMOTE_USER
Kibana 4 tweaking
For user based restriction
Each user has his/her own index for storing Kibana dashboards
Ex) .kibana.takase
Modify Kibana 4’s request based on REMOTE_USER on httpd
Kibana always tries to store dashboard to .kibana
kerberos
ldap
role
elasticsearch
authorization
authentication
search guard
REMOTE_USER
reverse proxy
with REMOTE_USER
REMOTE_USER
kibana_index
takase
.kibana.takase
test1
.kibana.test1
test2
.kibana.test2
kerberos auth
httpd
modify .kibana index request based on REMOTE_USER
https
reverse proxy
Kibana 4
container

17. Kibana Request Specify ES index in URL or POST data: URL modification
$ curl
$ curl –XPOST --data-binary ‘{“index”: {“_index”: “.kibana”}}’
URL modification
mod_rewrite
POST data modification
mod_ext_filter

18. Kibana 4 tweaking For user based restriction Modify URL
RewriteCond %{LA-U:REMOTE_USER}
RewriteRule .* - [E=TRIMMED_REMOTE_USER:%1]
RewriteCond %{LA-U:REMOTE_USER} !^$
RewriteRule ^/?es/(.*)/\.kibana/(.*)$ [L,P]
RewriteRule ^/?es/(.*)/\.kibana$ [L,P]
RewriteRule ^/?es/\.kibana/(.*)$ [L,P]
RewriteRule ^/?es/\.kibana$ [L,P]
This can be simpler…

19. Kibana 4 tweaking For user based restriction Modify POST data
if REMOTE_USER= takase, then set takase=1
RewriteCond %{LA-U:REMOTE_USER} takase
RewriteRule .* - [E=takase:1]
RewriteCond %{LA-U:REMOTE_USER} test1
RewriteRule .* - [E=test1:1]
ExtFilterDefine takase_kibana_index mode=input cmd="/bin/sed s/\".kibana\"/\".kibana.takase\"/g" enableenv=takase
ExtFilterDefine test1_kibana_index mode=input cmd="/bin/sed s/\".kibana\"/\".kibana.test1\"/g" enableenv=test1
SetInputFilter takase_kibana_index;test1_kibana_index
if takase=1, then execute sed command
Call defined filters
This can be simpler…

20. Kibana 4 tweaking For group based restriction
Each group has own index for storing Kibana dashboards
Ex) .kibana.cloud
Modify Kibana 4’s request based on HTTP_REFERER on httpd
User needs to access group specific URL
Ex)
Avobe URL access is restricted by Apache and LDAP configuration
<Location /cloud>
AuthType Kerberos
. . .
AuthLDAPUrl ldap://ldap01.tgpu01.kek.jp:389/ou=People,dc=tgpu01,dc=kek,dc=jp?krbPrincipalName
Require ldap-group cn=cloud,ou=Groups,dc=tgpu01,dc=kek,dc=jp
</Location>
You can use AuthGroupFile instead of AuthLDAPUrl

21. Kibana 4 tweaking For group based restriction Request URI Proxy to
(kibana4)
<Location /cloud>
AuthType Kerberos
. . .
AuthLDAPUrl ldap://ldap01.tgpu01.kek.jp:389/ou=People,dc=tgpu01,dc=kek,dc=jp?krbPrincipalName
Require ldap-group cn=cloud,ou=Groups,dc=tgpu01,dc=kek,dc=jp
</Location>
<Location />
AuthType Kerberos
. . .
Require valid-user
</Location>
<Location /grid>
AuthType Kerberos
. . .
AuthLDAPUrl ldap://ldap01.tgpu01.kek.jp:389/ou=People,dc=tgpu01,dc=kek,dc=jp?krbPrincipalName
Require ldap-group cn=grid,ou=Groups,dc=tgpu01,dc=kek,dc=jp
</Location>

22. Kibana 4 tweaking For group based restriction
RewriteCond %{HTTP_REFERER} ^https?://kef01.tgpu01.kek.jp/cloud.*$
RewriteRule .* - [E=TRIMMED_REMOTE_USER:cloud]
RewriteCond %{HTTP_REFERER} ^https?://kef01.tgpu01.kek.jp/grid.*$
RewriteRule .* - [E=TRIMMED_REMOTE_USER:grid]
RewriteCond %{LA-U:REMOTE_USER} !^$
RewriteRule ^/?es/(.*)/\.kibana/(.*)$ [L,P]
RewriteRule ^/?es/(.*)/\.kibana$ [L,P]
RewriteRule ^/?es/\.kibana/(.*)$ [L,P]
RewriteRule ^/?es/\.kibana$ [L,P]

23. Kibana 4 tweaking ES index access control is done by Search Guard
restriction
.kibana*
Kibana management user can read/write
.kibana.cloud
Any user with cloud role can read/write
.kibana.grid
Any user with grid role can read/write
.kibana.takase
User takase can read/write
.kibana.test1
User test1 can read/write
.kibana.test2
User test2 can read/write

24. modify .kibana index request based on REMOTE_USER or HTTP_REFERER
Demo
role
REMOTE_USER
es_admin
cloud
grid o
takase
test1
test2
kdc01
ldap01
kerberos
ldap
REMOTE_USER
role
authorization
authentication
group
elasticsearch
Flexible REST layer access control (User/Role; based on aliases, indices and types)
reverse proxy
with REMOTE_USER
search guard
kerberos auth
httpd
https
modify .kibana index request based on REMOTE_USER or HTTP_REFERER
REMOTE_USER
kibana_index
takase
.kibana.takase
test1
.kibana.test1
test2
.kibana.test2
reverse proxy
Kibana 4
container
kef01
HTTP_REFERER
kibana_index
.kibana.cloud
.kibana.grid

25. Need investigation
Data flow from Flume to ElasticSearch under Kerberos/search-guard control
ElasticSearchSink doesn’t support Kerberos auth
Kerberos auth can be avoidable by accessing to
But search-guard may block the request
Access to Kibana/ElasticSearch from command line:
# curl -k --krb --negotiate -u takase
Enter host password for user 'takase’:
How to access by using existing Kerberos ticket
Flume
ElasticSearch
Kibana