Prometheus From Berlin to Bonanza

Who am I? One of the four core developers of Prometheus Founder of Robust Perception Contributor to many open source projects Ex-Googler, after 7 years in the Dublin office

A Little History Prometheus started in 2012 by Matt Proud and Julius Volz in Berlin. In 2013 developed within SoundCloud, expanded to support Bazooka (cluster manager/scheduler), Go, Java and Ruby clients.

A Growing Community In 2014 other companies start using it, myself working at Boxever and Johannes at Docker. Project matures: new storage, new text format. In 2015 we "publicly release", adoption increases.

The Open Source Bonanza Today there are 300+ contributors to the core repositories, and 100+ 3rd party integrations. There are 600+ subscribers on our mailing lists, 400+ people in IRC and an estimated 500+ companies using Prometheus in production. Many companies funding Prometheus development.

What is Prometheus? Metrics monitoring system (not logs). A time series database. A query language. Client libraries. An Ecosystem. A Cloud Native approach to monitoring services.

Architecture

Monitoring Approach Service management went from manual to Chef to Kubernetes. Need to do the same for monitoring. Care about what matters to end users, such as latency and error rates. Distracting a human with alerts for everything that's vaguely off only leads to burnout.

Kubernetes Service Discovery Prometheus can discover all the pods, services, containers, and nodes of a Kubernetes cluster. Your Kubernetes labels and annotations can become Prometheus labels. Prometheus automatically picks up changes.

Client Libraries Instrument your code to capture the metrics that matter to you. If upstream libraries are instrumented, you get that for free! Also many exporters, e.g. cAdvisor, MySQL, SNMP, Consul, JMX, HAProxy, Minecraft, Factorio.

Instrumentation in CNCF projects Kubernetes is instrumented with Prometheus, allowing you to monitor the health of the cluster itself. Similarly Linkerd exposes metrics in our format. Interceptors for GRPC and plugins for Fluentd can feed data to Prometheus.

The PromQL Query Language Arbitrary aggregation, joins and slicing all possible. Can calculate how close you'll be to your quota in 4 hours, or the 95th percentile latency across an entire datacenter. If you can graph it, you can alert on it!

Analytics: Top 5 Docker images by CPU topk(5, sum by (image)( rate(container_cpu_usage_seconds_total{ id=~"/system.slice/docker.*"}[5m] )

Alert management Not every alert results in a page. Group similar alerts together, route them to the right team and throttle notifications. Designed to work reliably during network partitions.

What is Prometheus? Metrics monitoring system (not logs). A time series database. A query language. Client libraries. An Ecosystem. A Cloud Native approach to monitoring services.

Suggested Prometheus Talks, In A06 Today 12:35 Counting with Prometheus - Brian 14:40 Alerting in Cloud Native Environments - Fabian 16:20 Grafana is Not Enough: DIY User Interfaces for Prometheus - David Kaltschmidt

Suggested Prometheus Talks, In A06 Tomorrow 11:30 Integrating Long-Term Storage with Prometheus - Julius 12:15 Prometheus: The Unsung Heroes - Alejandro Ramirez 14:00 Configuring Prometheus for High Performance - Björn 14:45 Workshop: Instrumenting Own Applications for Prometheus - Alexandru Somesan & Frederic Branczyk - in C04, registration required

Resources Official Project Website: prometheus.io User Mailing List: prometheus-users@googlegroups.com Dev Mailing List: prometheus-developers@googlegroups.com IRC: #prometheus on chat.freenode.net Robust Perception Blog: www.robustperception.io/blog