Nivo 300 Windows Azure Storage Bojan Vrhovnik Microsoft

Nivo 300 Storage Infrastructure Windows Azure Storage is just a REST service running in Windows Azure. Each item in storage is stored in triplicate. Scalable, durable, and available Anywhere at anytime access Only pay for what the service uses

Nivo 300 Windows Azure: Skladišče

Nivo 300 With client library Automatically referenced in a cloud project Supplies a nice.NET API on top of the REST API You will need the following: Account Name - movieconversion Account Shared Key – NOcqFlqUwJPLlm… Endpoint - blob.core.windows.net -or- UseDevelopmentStorage=true

Nivo 300 BLOB Addresses Blob ContainerAccount contoso images PIC01.JPG videos VID1.AVI / Pages/ Blocks Block/Page PIC02.JPG

Nivo 300 Partitions Partitions are used by the storage fabric as the unit of scale and redundancy Each storage type defines a partition differently Queue -> One queue = One Partition Table -> One Table Partition = One Partition Blob -> One blob = One Partition

Nivo 300 Blob Containers Multiple Containers per Account Special $root container Blob Container A container holds a set of blobs Set access policies at the container level Associate Metadata with Container List the blobs in a container Including Blob Metadata and MD5 NO search/query. i.e. no WHERE MetadataValue = ? Blobs Throughput Effectively in Partition of 1 Target of 60MB/s per Blob

Nivo 300 Working with Containers

Nivo 300 Demo kontajnerji

Nivo 300 Two Types of Blobs Under the Hood Block Blob Targeted at streaming workloads Each blob consists of a sequence of blocks Each block is identified by a Block ID Size limit 200GB per blob Optimistic Concurrency via ETags Page Blob Targeted at random read/write workloads Each blob consists of an array of pages Each page is identified by its offset from the start of the blob Size limit 1TB per blob Optimistic or Pessimistic (locking) concurrency via Leases

Nivo 300 Uploading a Block Blob 10 GB Movie Windows Azure Storage Block Id 1 Block Id 2 Block Id 3 Block Id N blobName = “TheBlob.wmv”; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); ………… PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1,…,blockIdN); blobName = “TheBlob.wmv”; PutBlock(blobName, blockId1, block1Bits); PutBlock(blobName, blockId2, block2Bits); ………… PutBlock(blobName, blockIdN, blockNBits); PutBlockList(blobName, blockId1,…,blockIdN); TheBlob.wmv

Nivo 300 Page Blob – Random Read/Write 0 10 GB 10 GB Address Space

Nivo 300 Working with BLOBs

Nivo 300 Uploading a BLOB Other Options UploadFile UploadText UploadFromStream

Nivo 300 Container & BLOB Metadata

Nivo 300 Demo upload blob

Nivo 300 Downloading a BLOB Other Options DownloadText DownloadByteArray DownloadToFile

Nivo 300 Demo prenos bloba

Nivo 300 Snapshotting Pins a version of your BLOB that is read only.

Nivo 300 Demo snapshots

Nivo 300 Shared Access Signatures Fine grain access rights to blobs and containers Easiest way to allow an untrusted client to upload files to your blob account Sign URL with storage key – permit elevated rights Revocation Use short time periods and re-issue Use container level policy that can be deleted

Nivo 300 Sample Use

Nivo 300 Demo SAS

Nivo 300 Content Delivery Network Scenario Frequently accessed blobs Accessed from around the world Windows Azure Content Delivery Network (CDN) provides high-bandwidth global blob content delivery

Nivo 300 How does it work? !

Nivo 300 Demo CDN

Nivo 300 Cloud Drive Client Library Sample CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount"); //Initialize the local cache for drives mounted by this role instance CloudDrive.InitializeCache(localCacheDir, cacheSizeInMB); //Create a cloud drive (PageBlob) CloudDrive drive = account.CreateCloudDrive(pageBlobUri); drive.Create(1000 /* sizeInMB */); //Mount the network attached drive on the local file system string pathOnLocalFS = drive.Mount(cacheSizeInMB, DriveMountOptions.None); //Use NTFS APIs to Read/Write files to drive … //Snapshot drive while mounted to create backups Uri snapshotUri = drive.Snapshot(); //Unmount the drive drive.Unmount();

Nivo 300 What is the Table Service? Simple Highly scalable Cost effective Non-relational Hierarchical

Nivo 300 Table Structure Storage Account: MovieData Star Wars Matrix Fan Boys Star Wars Matrix Fan Boys Table Name: Movies Mike Collier Mr. Anderson Bill Gates Mike Collier Mr. Anderson Bill Gates Table Name: Customers Account Table Entity

Nivo 300 Sample Table Partition Key Row KeyDeliveryAddressOrderAmount KirkNC Kobayashi Maru Drive $35, KirkNC170 1A 409 Tiberious Road$98.99 KirkNC170 1B 1024 Priceline Ave$4,523,765, Archer Sliders Blvd$5, Archer11405 Suliban Road$55.39 Archer234DS9, Level 29, Suite 782 $1.58 Bender1900 South Futurama Drive $87, VIN KJUDH

Nivo 300 Inherit or add the Required Fields

Nivo 300 Create a Context Class WCF Data Services needs a context class. This represents your connection to the table.

Nivo 300 Adding an Entity

Nivo 300 Deleting an Entity Keep the entity, or use a lightweight copy for better performance.

Nivo 300 Batching Use batching to increase performance

Nivo 300 Queries Queries less than 5 seconds return the results. Queries between 5 and 30 seconds return a continuation token so you can fetch pages of results. Queries over 30 seconds are cancelled. When more than 1,000 entities are found, a continuation token is returned.

Nivo 300 Demo Tables

Nivo 300 Performance

Nivo 300 What is a queue? Queues are used to store messages They are FIFO, and one way A queue name must be lower case name, and URL friendly

Nivo 300 Messages A queue can hold an unlimited number of messages Messages must be serializable as XML Limited to 8KB in size Commonly use the work ticket pattern The queue garbage collector runs once a week

Nivo 300 Queue Terminology

Nivo 300 Message lifecycle QueueQueue Msg 1 Msg 2 Msg 3 Msg 4 Worker Role PutMessagePutMessage Web Role GetMessage (Timeout) RemoveMessageRemoveMessage Msg 2 Msg 1 Worker Role Msg 2

Nivo 300 Working with Queues Create a CloudQueueClient

Nivo 300 Add a message

Nivo 300 Peek a message Peeking lets you see the contents without ‘getting’ the message. Handy when trying to determine how to handle a message without consuming it.

Nivo 300 Get a message The visibility is 30 seconds and can be as long as 2 hours.

Nivo 300 Deleting a Message The Message Id and the Pop Receipt must match for a successful delete.

Nivo 300 Polling Most queues are polled in an infinite loop Make your processes idempotent

Nivo 300 Demo Queues

Nivo 300 VPRAŠANJA? Po zaključku predavanja, prosimo, izpolnite vprašalnik. Vprašalniki bodo poslani na vaš e-naslov, dostopni pa bodo tudi preko profila na spletnem portalu konference Najlepša hvala!