Distributed Time Series Database InfluxDB/openTSDB

TSDB Time series database Time series data InfluxDB It is a software system that is optimized for handling time series data, arrays of numbers indexed by time (a datetime or a datetime range) Time series data A time series is a sequence of data points, measured typically at successive points in time spaced at uniform time intervals InfluxDB An open-source distributed time series database with no external dependencies InfluxDB is a time series, metrics, and analytics database Written in Golang (Google programming language) InfluxDB is targeted at use cases for DevOps, metrics, sensor data, and real-time analytics.

InfluxDB KeyFeatures SQL like query language HTTP(S) API and client API (python, ruby, php) Store billions of data points Database managed retention policies for data Built in management interface InfluxDB is schemaless so the series and columns get created on the fly

Design Goals Stores metrics data (like response times and cpu load. i.e. what you’d put into Graphite) Stores events data (like exceptions, user analytics, or business analytics) HTTP(S) interface for reading and writing data. Shouldn’t require additional server code to be useful directly from the browser. Horizontally scalable. Simple to install and manage. Shouldn’t require setting up external dependencies like Zookeeper and Hadoop. Compute percentiles and other functions on the fly. Automatically compute common queries continuously in the background

Reading and writing data Via HTTP API Most of the client libraries use this API. Simply send a POST to /db/<database>/series?u=<user>&p=<pass>. The post data shall be in the JSON format like: [ { "name" : "hd_used", "columns" : ["value", "host", "mount"], "points" : [ [23.2, "serverA", "/mnt"] ] } ] InfluxDB will assign a time and sequence number for every point written.

Data Organization Databases (like in MySQL, Postgres, etc) Time series (kind of like tables) Points or events (kind of like rows)

InfluxDB is distributed, the order of points is only guaranteed by timestamp. [ { "name": "log_lines", "columns": ["time", "line"], "points": [ [1400425947368, "here's some useful log info"] ] } ] The timestamp is a microsecond epoch By default time precision is assumed to be milliseconds.

Login and create DB point your browser to localhost:8083. The InfluxDB HTTP API runs on port 8086 by default

Commandline like mySQL https://influxdb.com/docs/v0.9/introduction/getting_started.html From CLI /opt/influxdb/influx CREATE DATABASE mydb SHOW DATABASES name: databases --------------- name mydb Use mydb INSERT cpu,host=serverA,region=us_west value=0.64 SELECT * FROM cpu INSERT temperature,machine=unit42,type=assembly external=25,internal=37 select * from temperature

openTSDB OpenTSDB is a specialized database to store sequence of data points generated over a period of time in uniform time interval. It uses HBase as the underlying database in order to handle huge amounts of data is designed to handle terabytes of data and still maintain very good performance levels for various types of monitoring needs A typical time series record consists of a metric name, the timestamp and the associated

openTSDB Has three responsibilities Collecting Loading/storing Querying data The main objective of this architecture is to write and read data points into Hbase Properties Scalability Availability and consistency HBase is used for linear scaling, automatic replication and efficient scans.

openTSDB - Architecture How it works? See details from http://opentsdb.net/overview.html