Redis And Python PyCon India, 2011 (Work in Progress) Sunil Arora

Raising Hands... How many of you have used Redis before ? How many of you have got a laptop here ?

Who Sunil Arora Work for ShopSocially ( Interests: Backend, Frontend, scaling, internet technologies in general, startups... all kind of shit

Who Sunil Arora Work for ShopSocially ( Interests: Backend, Frontend, scaling, internet technologies in general, startups...

Today's talk What is Redis How it works and what you can do with it Real life use-cases

What is Redis ? Its between lot of stuff, so difficult to categorize it precisely

What is Redis ? I see Redis definitely more as a flexible tool that as a solution specialized to solve a specific problem: his mixed soul of cache, store, and messaging server shows this very well -Salvatore Sanfilippo Picture by herzogbr

Brief History of Redis Released in March 2009 by Salvator Sanfilippo Open source, BSD licensed VMWare sponsored its development in March, 2010

A few fundamentals Written in C (25K LOC) Uses memory as main storage Single Threaded Uses disk for persistence Screamingly fast performance 50K read/write operations per seconds 200K read/write ops per second on a regular EC2 instance

Installation $ git clone $ cd redis $ make $./src/redis-server

redis-py The most popular python client library Andy McCurdy Github: easy_install redis OR pip install redis Optional: easy_install hiredis or pip install hiredis

Lets get started... $ cd redis $./src/redis-server >>> from redis import Redis >>> redis_client = Redis() >>> redis_client.keys() >>> help(redis_client)

Redis Keys Not binary safe. Should not contain space or newline character A few rules about keys: Too long keys are not a good idea Too short keys is also not a good idea “object-type:id:field” can be a nice idea, i.e. “user:1001:name”

Operations on Keys KEYS EXISTS DEL EXPIRE OBJECT PERSIST RANDOMKEY RENAME TYPE TTL EXPIREAT MOVE

Lets play with keys >>>redis_client.keys() >>>redis_client.exists('key') >>>redis_client.delete('key') >>>redis_client.type('key') >>>......

Data Structures Strings Lists Sets Sorted Sets Hashes

Strings SET GET MSET MGET SETEX INCR INCRBY DECR DECRBY

Strings – with redis client >>> redis_client.set('key', 'value') >>> redis_client.get('key') >>> redis_client.delete('key')

Fetch multiple keys at once mget/mset redis_client.mset({'key1': 'val1', 'key2': 'val2'}) redis_client.mget('key1', 'key2',......)

Expiration Set a value with expire >>>redis_client.setex('key', 'value', 2) #key to expire in 2 secs >>>redis_client.expire('key', 2) >>>redis_client.get('key') >>>None

Uses To store transient states in your web application

Uses Who is online?

Uses Redis as LRU cache ( as-LRU-cache.html)

Atomic Increments >>>help(redis_client.incr) >>>help(redis_client.decr) >>> >>> redis_client.incr('counter', 1) >>>1 >>> redis_client.incr('counter') >>>2 >>> redis_client.incr('counter') >>>3

Uses High Speed counters (views/clicks/votes/likes..)

Uses API Rate Limiting

Uses Generating unique IDs

Lists Ordered list of binarysafe strings Doubly linked list Memory footprint optimized for smaller list O(1) insertion/deletion at both ends

Lists - operations LPUSH RPUSH LSET LRANGE LPOP BLPOP BRPOP BRPOPLPUSH LINSERT RPOP RPOPLPUSH LPUSHX RPUSHX

Uses Web apps are full of lists :)

Uses Capped List

Uses Normal Queue Real time message Queue

Uses Social Activity Streams

Sets An unordered collection of distinct byte strings Nothing different from the data type in python

Sets - Operations SADD SCARD SDIFF SDIFFSTORE SINTER SINTERSTORE SISMEMBER SMEMBERS SMOVE SPOP SRANDMEMBER SREM SUNION SUNIONSTORE

Uses Picking random items from a set using SRANDMEMBER

Uses Primitive to construct filtering mechanism on items

Which of my friend's are online right now? Uses

Sorted Sets ZADD ZCARD ZCOUNT ZINCRBY ZINTERSTORE ZRANGE ZRANGEBYSCORE ZRANK ZREM ZREMRANGEBYRA NK ZREMRANGEBYSC ORE ZREVRANGE ZREVRANGEBYSCO RE ZSCORE ZUNIONSTORE

SORT SORT KEY SORT key BY pattern (e.g. object-type:*:age) SORT key LIMIT 0 10 SORT key GET user:*:username SORT key BY pattern STORE dstkey

Uses Generating sorted transient views

Publish/Subscribe A simple and efficient implementation of publish/subscribe messaging paradigm Client can subscribe/psubscribe to receive messages on channels (key patterns)

Publish/Subscribe PSUBSCRIBE PUBLISH PUNSUBSCRIBE SUBSCRIBE UNSUBSCRIBE

Uses Many to Many message passing

Uses Web Chat

Questions