Azure Table Storage Ivan

breakdown What is Table Storage? Why use Table Storage? The Basics Designing a Table Accessing tables with .NET Example

What Is table storage NoSQL DataStore service for large amounts of unstructured data When would I use it? Datasets that don’t require complex joins, keys or storeprocs Datasets that can be denormalized for fast access Datasets that need to scale big as demand increases

Why use table storage? Cheap. From US$0.12 per GB to as little as US$0.045c per GB (R1.42 to R0.54) Price is based on redundancy and total size. Fast. (With a little forethought). LINQ is supported (somewhat).

The Basics Entries into the table have two keys: PARTITION and ROW (and also contain a TIMESTAMP) Row Key is unique, Partition Key groups rows. Query operations are limited. Supported: From, Where, Take (<1000). Not supported: GroupBy, OrderBy, Math functions, Distinct, Join, Count, Contains… and many more. Very fast to get by Row and Partition key, moderately slow otherwise.

Designing a partition/row schema Be aware of LIMITS so you don’t lose data! Account limit – 20 000 entities/second Partition limit – 2 000 entities/second Read Queries that do a “table scan” count each entity towards this limit whether or not it is returned. Aim to never have to do a table scan. Don’t use Table Storage if you will need to make complex queries. Ideally, only ever query on Partition Key or Row Key (or groups thereof) Create your own indexes by duplicating data.

How azure structures your data Partitions served from one partition server Partition server has a rate limit, and speed will be throttled down when it becomes “hot”. For small partitions, sequential partition keys may be served from the same server, allowing faster range lookups (e.g. 0001, 0002, 0003, …).

Rule - If the entity has one key property, use it as the partition key. If the entity has two key properties, use one as partition, one as row. If there are more potentially dominant queries, duplicate the data to provide an ad- hoc index.

Inserting into tables with .net CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connString); CloudTableClient cloudTableClient = storageAccount.CreateCloudTableClient(); CloudTable cloudTable = cloudTableClient.GetTableReference(tableName); ElasticTableEntity elasticEntity = ElasticTableEntity.From(data); TableOperation tableOperationContext = TableOperation.InsertOrMerge(elasticEntity); cloudTable.Execute(tableOperationContext);

Example – logging ACCOUNTS – environments TABLES – products & index tables PARTITION – index value ROW – reverse timestamp+GUID

…

References & questions https://msdn.microsoft.com/en-us/library/azure/hh508997.aspx https://msdn.microsoft.com/en-us/library/azure/dd135725.aspx http://stackoverflow.com/questions/15809078/design-of-partitioning-for-azure- table-storage http://www.troyhunt.com/2013/12/working-with-154-million-records-on.html