Searchlight Lei Zhang Search service for OpenStack Image source: https://www.flickr.com/photos/nasahqphoto/15757310289

Searchlight includes indexing plugins for many of Openstack's core services. This session will focus on: Concept overview Architecture overview Aggregations abilities and example Searchlight UI show Ocata goals

Is managing relationships between things Power off all of Bob’s instances What flavors work for encryption workloads? What resources are tagged with “web”? What am I using in the East availability zone? Managing Clouds Is managing relationships between things

Pre-defined lists for all the things Availability Zones Baremetal Nodes Domains Flavors Floating IPs Host Aggregates Hypervisors Images Instances Key Pairs Metadata Definitions Networks Projects Routers Security Groups Subnets Volumes ... The CLI Pre-defined lists for all the things

Pre-defined lists for all the things The UI Pre-defined lists for all the things

Predefined lists are one way to do it… Details of one thing List of things Details of one thing List of things Details of one thing List of things

What about MY list of things? Cloud Requires Search What about MY list of things?

OpenStack is a set of distributed services Searchlight Based on Elasticsearch Powerful search and aggregation API Full text search on any resource Support search for Nova, Neutron, Cinder, Glance, Swift, Designate resources CLI and Horizon UI OpenStack is a set of distributed services Searchlight Unified Search ~ 3 minutes

List and Query Requests Concept Flow UIs and API Clients Searchlight List and Query Requests plugins Resources indexed: On demand bulk indexing Via notifications Direct Injection Libraries Cloud Services Action Requests ~ 20 minutes. Hand over to Steve

Components

Searchlight Core Engine Plugin architecture Searchlight Core Engine Zero Downtime Bulk Indexing (to [re-]initialize data) Incremental Indexing (ongoing) Rich Policy Based Access Controls Per User Field Level Data Security Resource Plugin Data Mapping Retrieves resources from source service API Transforms event notifications Defines resource RBAC rules Note how it's not necessarily specific to Openstack Examples of RBAC - filter by project, neutron's more complicated rules Keystone (auth) RMQ / Oslo.messaging Nova

Improve the Notifications Notifications in OpenStack are spotty missing events missing data Improve the Notifications avoid api callbacks

Aggregations Provide aggregated data based on a search query Can be composed in order to build complex summaries of the data. Based on the query provided as well as Searchlight RBAC filters.

How many volumes and how many gigabytes does each project use? { "query": {"match_all": {}}, "type": "OS::Cinder::Volume", "limit": 0, "all_projects": true, "aggregations": { "volumes_by_tenant": { "terms": { "field": "project_id" }, "total_volume_gb": { "sum": { "field": "size" } }}}}} { "volumes_by_tenant": { "buckets": [ "doc_count": 2, "key": "a3845ef5d597426086980afeed4838a9", "total_volume_gb": { "value": 4.0 } }, "key": "b1fbc598025841899f63333269abe1d2", "value": 3.0      

Plugging resource types into the interface Web front-end to Searchlight Is a plugin to Horizon Repo: searchlight-ui AngularJS ~2,000 LOC (JS) Uses Neutron Horizon Searchlight UI Plugging resource types into the interface

Ocata list Add more service plugins: Ironic, Murano, Glare, Magnum, Lbaas(Octavia), Heat Provide sort and pagination support for Nova cell v2 Implement pipeline for Searchlight, allow arbitrary publishers can be added into Searchlight Zaqar publisher(Searchlight Zaqar Horizon integration)

More info http://docs.openstack.org/developer/searchlight/ https://blueprints.launchpad.net/searchlight