A whirlwind tour Aran Elkington. Open-source Networked / Distributed In-memory  fast Essentially a Key-value data store Optional durability.

Slides:



Advertisements
Similar presentations
Enhancing Productivity with MySQL 5.6 New Features
Advertisements

Connecting to Databases. relational databases tables and relations accessed using SQL database -specific functionality –transaction processing commit.
Configuration management
Databasteknik Databaser och bioinformatik Data structures and Indexing (II) Fang Wei-Kleiner.
Lecture 12: MapReduce: Simplified Data Processing on Large Clusters Xiaowei Yang (Duke University)
COM vs. CORBA.
CpSc 3220 File and Database Processing Lecture 17 Indexed Files.
© Copyright 2012 STI INNSBRUCK Apache Lucene Ioan Toma based on slides from Aaron Bannert
Introduction to Computing Using Python CSC Winter 2013 Week 8: WWW and Search  World Wide Web  Python Modules for WWW  Web Crawling  Thursday:
Building a Database on S3 Matthias Brantner, Daniela Florescu, David Graf, Donald Kossmann, Tim Kraska Xiang Zhang
K ALI ROHAN KOKA AZEEM HIRANI. A Simple database Implementing a Dictionary where keys are associated with values. For e.g.: It can set the key “surname_1987”
SOFTWARE PRESENTATION ODMS (OPEN SOURCE DOCUMENT MANAGEMENT SYSTEM)
1 Chapter 12 Working With Access 2000 on the Internet.
Servlets and a little bit of Web Services Russell Beale.
Data Indexing Herbert A. Evans. Purposes of Data Indexing What is Data Indexing? Why is it important?
Google Bigtable A Distributed Storage System for Structured Data Hadi Salimi, Distributed Systems Laboratory, School of Computer Engineering, Iran University.
© 2006 Pearson Addison-Wesley. All rights reserved13 B-1 Chapter 13 (excerpts) Advanced Implementation of Tables CS102 Sections 51 and 52 Marc Smith and.
Apache : Installation, Configuration, Basic Security Presented by, Sandeep K Thopucherela, ECE Department.
Implementing search with free software An introduction to Solr By Mick England.
C o n f i d e n t i a l Developed By Nitendra NextHome Subject Name: Data Structure Using C Title: Overview of Data Structure.
Introduction To Databases IDIA 618 Fall 2014 Bridget M. Blodgett.
Object Oriented Databases by Adam Stevenson. Object Databases Became commercially popular in mid 1990’s Became commercially popular in mid 1990’s You.
IT 210 The Internet & World Wide Web introduction.
RMG Study Group Session I: Git, Sphinx, webRMG Connie Gao 9/20/
Avro Apache Course: Distributed class Student ID: AM Name: Azzaya Galbazar
COM vs. CORBA Computer Science at Azusa Pacific University September 19, 2015 Azusa Pacific University, Azusa, CA 91702, Tel: (800) Department.
HBase A column-centered database 1. Overview An Apache project Influenced by Google’s BigTable Built on Hadoop ▫A distributed file system ▫Supports Map-Reduce.
Goodbye rows and tables, hello documents and collections.
CHEN Ge CSIS, HKU March 9, Jigsaw W3C’s Java Web Server.
MySQL™: The Open Source Database for Mission-Critical, Heavy Load Applications Kaj Arnö, VP Training, MySQL AB Stuttgart, Germany Belgrad, Serbia.
Information Systems and Network Engineering Laboratory II DR. KEN COSH WEEK 1.
 70s - Database access is hard and depends on the app  80s – Relational databases come on the scene  90s – Object oriented programming and DBs  00s.
Hashing Table Professor Sin-Min Lee Department of Computer Science.
© 2006 Pearson Addison-Wesley. All rights reserved13 B-1 Chapter 13 (continued) Advanced Implementation of Tables.
Róbert Móro. redis In-memory key-value store Master-slave replication Persistence – RDB (snapshots) – AOF (append-only log file) Supported languages –
CE Operating Systems Lecture 3 Overview of OS functions and structure.
MapReduce and GFS. Introduction r To understand Google’s file system let us look at the sort of processing that needs to be done r We will look at MapReduce.
VICTORIA UNIVERSITY OF WELLINGTON Te Whare Wananga o te Upoko o te Ika a Maui SWEN 432 Advanced Database Design and Implementation MongoDB Architecture.
Cloudant & Redis Nikolay Tomitov Technical Trainer SoftAcad Training Center.
240-Current Research Easily Extensible Systems, Octave, Input Formats, SOA.
Apr. 8, 2002Calibration Database Browser Workshop1 Database Access Using D0OM H. Greenlee Calibration Database Browser Workshop Apr. 8, 2002.
Linux Operations and Administration
CIS 250 Advanced Computer Applications Database Management Systems.
CS 540 Database Management Systems
Introduction to Databases Angela Clark University of South Alabama.
Module 6: Administering Reporting Services. Overview Server Administration Performance and Reliability Monitoring Database Administration Security Administration.
Open Map Yamama Dagash & Haitham Khateeb under the supervision of: Benny Daon & Eyal Levin Open Map.
CS 440 Database Management Systems Stored procedures & OR mapping 1.
Orion Contextbroker PROF. DR. SERGIO TAKEO KOFUJI PROF. MS. FÁBIO H. CABRINI PSI – 5120 – TÓPICOS EM COMPUTAÇÃO EM NUVEM
Information Systems and Network Engineering Laboratory I DR. KEN COSH WEEK 1.
Amazon Web Services. Amazon Web Services (AWS) - robust, scalable and affordable infrastructure for cloud computing. This session is about:
( ) 1 Chapter # 8 How Data is stored DATABASE.
SQL Database Management
Neo4j: GRAPH DATABASE 27 March, 2017
Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung
Information Systems and Network Engineering Laboratory II
DURABILITY OF TRANSACTIONS AND CRASH RECOVERY
Data Indexing Herbert A. Evans.
Redis:~ Author Anil Sharma Data Structure server.
Amazon Storage- S3 and Glacier
MongoDB Er. Shiva K. Shrestha ME Computer, NCIT
Spark Presentation.
IS444: Modern tools for applications development
CSE-291 Cloud Computing, Fall 2016 Kesden
MongoDB Distributed Write and Read
IS444: Modern tools for applications development
New Mexico State University
Chapter 2: Operating-System Structures
Advanced Implementation of Tables
Chapter 2: Operating-System Structures
Presentation transcript:

A whirlwind tour Aran Elkington

Open-source Networked / Distributed In-memory  fast Essentially a Key-value data store Optional durability Written by Italian developer Salvatore Sanfilippo Redis is a NoSql database written in ANSI C

Salvatore started Redis in early Deployed in June 2009 rapidly it grew in popularity. Salvatore added features at a very rapid pace and rectified all reports of database corruption, instability, etc. In March of 2010 VMWare hired Salvatore to work full-time on Redis. (Redis itself remains BSD licensed.) Shortly thereafter, VMWare hired Pieter Noordhuis, a key Redis contributor, to help Salvatore. Since June 2010 Redis jobs have increased 524% History

I think that what’s particularly hard with C is not the details about pointers, automatic memory management, and so forth, but the fact that C is at the same time so low level and so flexible. So basically if you want to create a large project in C you have to build a number of intermediate layers (otherwise the code will be a complete mess full of bugs and 10 times bigger than required). This continuous design exercise of creating additional layers is the hard part about C. You have to get very good at understanding when to write a function or not, when to create a layer of abstraction, and when it’s worth to generalize or when it is an overkill. Posted 18th December 2009 at 3:50 pm18th December 2009 Quote from Salvatore

In its outer layer, the Redis data model is a dictionary where keys are mapped to values. In its inner layer these values can be various data structures with different operations you can perform upon them String Redis Strings are binary safe Up to a max size of 512 Megabytes Use Strings as atomic counters using INCR, DECR, INCRBY.INCRDECRINCRBY Append to strings with the APPEND command.APPEND

Lists Simply lists of strings, sorted by insertion order can insert or delete at constant time from head or tail, even with many millions of inserted items Hashes Collection of maps between string fields and string values. With up to one hundred or so pairs you can store millions of hashes in a small Redis instance. Sorted Sets Collections of non-repeating elements ordered by a floating-point number called “score”. While members are unique, scores may be repeated. Sets Collections of non-repeating unsorted elements. It is possible to add, remove, and test for existence of members in constant time regardless of Set size

List Operations

Set Operations

Sorted Set Operations

Hashes Operations

Replication Redis supports master-slave replication. Data from any Redis server can replicate to any number of slaves. A slave may be a master to another slave. This allows Redis to implement a single-rooted replication tree If you set up a slave, upon connection it sends a SYNC command. The master collects all new commands received that will modify the dataset. And transfers the database file to the slave To configure replication is trivial: just add the following line to the slave configuration file: slaveof

The County of Los Angeles actively promotes and is committed to ensure a work environment that is free from any discriminatory influence be it actual or perceived. As such, it is the County's expectation that our manufacturers, suppliers and contractors make a concentrated effort to ensure that any equipment, supplies or services that are provided to County departments do not possess or portray an image that may be construed as offensive or defamatory in nature. One such recent example included the manufacturer's labeling of equipment where the words "Master/Slave" appeared to identify the primary and secondary sources. Based on the cultural diversity and sensitivity of Los Angeles County, this is not an acceptable identification label. We would request that each manufacturer, supplier and contractor review, identify and remove/change any identification or labeling of equipment or components thereof that could be interpreted as discriminatory or offensive in nature before such equipment is sold or otherwise provided to any County department. Thank you in advance for your cooperation and assistance. Joe Sandoval, Division Manager Purchasing and Contract Services Internal Services Department County of Los Angeles

Persistence RDB Performs point-in-time snapshots at specified intervals. AOF Logs every write operation received using Redis protocol in an append-only fashion that can be replayed. Advantage Compact single-file point-in-time Perfect for backups transferred to far data centers for DR Maximizes Redis performances Allows faster restarts Durable and Flexible: fsync every second, not at all or every query. Append only no seeks, no corruption Log easy to understand and parse Disadvantage Data loss possible in crash if between snapshots Needs to fork() often which be time consuming if the dataset is big Files bigger than RDB files Can be slower then RDB depending on the exact fsync policy. RDB provides more guarantees about maximum latency

Example App Code: github Used F# and ServiceStack.Redis client on a.Net stack Tried to stick with a schema. "object-type:id:field“ – “file:1000:name". – Use dots for multi-words fields, like in "comment:1234:reply.to".

API’s and Clients Loads of Clients: C, C++, C#, Clojure, Erlang, Go, Haskell, Java, Lua, Node.js, Objective-C, Perl, PHP, Python, Ruby, Scala – Store Strings or Serialize objects Object-hash mapping library for Redis – Inspired from Ruby’s ohm ॐ (Object-hash mapping) created by soveran ( Michel Martens – Argentina) soveran – JOhm for Java, created by xetorthio JOhm – Lohm for Lua, created by slact Lohm – Nohm for Node.js, created by maritz Nohm – Redisco for Python, created by iamteem Redisco

Conclusion Redis is: – Fast with lots of persistence options – Flexible data structures – Lots of operations run on the server – Distributed [master -> slave (master) -> slave] – Easy to program with in almost any language I may add I have thoroughly enjoyed this exercise and look forward to using Redis in the future

Links Official Redis Documentation at redis.io – The Little Redis Book– free ebook – Redis, from the Ground Up – ground-up.html#heading_toc_j_0 ground-up.html#heading_toc_j_0 Thanks to Chris Petrilli for pics: