NoSQL on Azure: An Introduction to DocumentDB Nilabja Ball Premier Field Engineer Microsoft India

Nilabja Ball Premier Field Engineer with Microsoft India 10 years of SQL Server Experience Speaker in SQL UG group, Microsoft Tech-Ed Closely works with Mission Critical Tier 1 Microsoft Customers

What is NoSQL? A Database that doesn’t use SQL Language Replacement of RDBMS Storing and retrieving data other than tabular relations All of the Above 3 | 9/22/2018 | NoSQL on Azure: Introduction to DocumentDB

Database for modern apps 9/22/2018 Database for modern apps DocumentDB is good fit if you … Why documentdb?   A database that keep up with the modern application. Modern application known as 3V . velocity, volume and variety If you have agile development and you have multiple version of application running on mobile, web and desktop version. So you need a platform\database that can support all schemas at once The code iteration needs fast and deploy in production quickly. User have different types of query pattern and requirement changes rapidly. So, the existing schema needs to be altered with appropriate indexes. Elasticity : The solution should be scalable and should support application load without making much code changes and minimizing regression testing © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Data Technologies A summary TechReady12 9/22/2018 Azure Data Technologies A summary Operational Data Analytical Data Document Store (DocumentDB, MongoDB, …) NoSQL Technologies Key/Value Store (Tables, Riak, …) Big Data Analytics (HDInsight, Hadoop) Column Family Store (HBase, Cassandra …) SQL Technologies Data in two category. One is operational data like sales data and then we save and analyze they become analytical data. Yellow color are managed services offered in Azure. The white are traditional systems that can run on Azure such as IaaS. Various categories One of them is document store DocumentDB : managed document store . Other document store that runs on azure is MongoDB Tables: Key value services that azure provides. Riak Hbase: Columh Family store as a service Managed service provided by Azure Relational Database (SQL Database, SQL Server, Oracle, MySQL, …) Relational Analytics (SQL Server, Oracle, MySQL, …) Software that can run in Azure virtual machines © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

DocumentDB in Azure SQL Server in a VM SQL Database Tables Blobs > fully featured RDBMS fully managed, scalable, queryable, schemafree JSON document database service for modern applications > transactional processing > rich query Managed as a Service< SQL is the query language of choice for many developers. DocumentDB provides a SQL-like query language which includes hierarchical querying and the ability to execute JavaScript. Scale-out and High Availability DocumentDB is architected with High Availability and Scalability as a primary goal. DocumentDB is built upon an incredibly well battle-tested distributed systems framework This system allows DocumentDB to focus on building a solid database system rather than the intricacies of distributed systems. Transactions DocumentDB supports ACID compliant transactions. All queries are executed as a transaction. Semantics which aren't easily conveyed via the query language can be written in JavaScript and executed either directly, as a stored procedure, a User-Defined Function, or as a trigger. JavaScript Integration The JavaScript based application logic executes directly on the storage partitions (in the same memory space as the buffer pool) and the execution is fully transactional. Secondly, there is no impedance mismatch between the type system of the programming language and the store - JSON is the common type system shared between the JavaScript programming language runtime and the store. The result is a very simple yet powerful model for transactional execution of JavaScript application logic. JavaScript developers get “durable” programming model while still using their familiar language constructs and library primitives. DocumentDB provides highly resource governed multi-tenant execution of JavaScript logic. RESTful API DocumentDB provides a RESTful interface over HTTP. Their .NET library is a fairly thin wrapper around the HTTP API. Queries must include an authorization header.  For a complete reference on the REST APIs, please refer to: http://msdn.microsoft.com/en-us/library/dn781481.aspx elastic scale < schema-less data< arbitrary data formats < internet accessible http/rest < Microsoft Confidential

SQL Database A relational data service Tables SQL Query char Name ID int date LastUse Country Age Application 1 3 2 7 Column Name Column Type Primary Key Data

DocumentDB A document store Collections Document 1 { "name": "John", "country": "Canada", "age": 43, "lastUse": "March 4, 2014" } { "name": "Eva", "country": "Germany", "age": 25 } Document 2 Request Application {…} { "name": "Lou", "country": "Australia", "age": 51, "firstUse": "May 8, 2013" } Document 3 { "docCount": 3, "last": "May 1, 2014" } Document 4 If you look at this example , there is no schema. The document 2 don’t have the lastuse column and document1 and document3 has different column last use and first use. The final document is a summary. This doesn’t conform to any schema and each document can be schematically completely different than other. The data is returned as JSON format. Paste { "name": "John", "country": "Canada", "age": 43, "lastUse": "March 4, 2014" } "name": "Eva", "country": "Germany", "age": 25 "name": "Lou", "country": "Australia", "age": 51, "firstUse": "May 8, 2013" "docCount": 3, "last": "May 1, 2014"

DocumentDB A more complete picture Database Collections Documents Application RESTful Access Methods Libraries: .NET, JavaScript, Java, Python, … Stored Procedures Triggers UDFs How do you access documentDB ? The basic CRUD operation can be be formed using RESTful API. There is also query language like an extension or subset of SQL. This is not a full fledged TSQL . Instead of building a new language, lets reuse. You can create stored procedure, triggers and UDFs using javscript. DocDB stores data as JSON. It is a native javascript environment. Your client can work on .NET,Java,Python,node.js and these language\platform provides library that can exchange data using JSON format. If you want to use transaction, you have to use stored procedure and triggers. In documentDB,

Data Model - JSON 9/22/2018 JavaScript program Schema-First Relational Database incompatible type-system Requires explicit Schema/Indices (Infoset Agnostic, Schema Free) Key-Value Database Opaque values Requires Secondary Indices/Attribute Projections extraneous to the value (JSON Infoset Aware, Schema Free) DocumentDB Deep commitment to JSON infoset No secondary indices/schema hints needed for indexing the values © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Resource Model All entities uniquely addressable by a logical URI. 9/22/2018 Resource Model All entities uniquely addressable by a logical URI. Partitioned for scale out and replicated for HA DocumentDB accounts scale out through the addition of capacity units with associated DB storage and throughput Azure DocumentDB manages data through well-defined database resources. These resources are replicated for high availability and are uniquely addressable by their logical URI. DocumentDB offers a simple HTTP based RESTful programming model for all resources. The DocumentDB database account is a unique namespace that gives you access to Azure DocumentDB. Before you can create a database account, you must have an Azure subscription, which gives you access to a variety of Azure services. All resources within DocumentDB are modeled and stored as JSON documents. Resources are managed as items, which are JSON documents containing metadata, and as feeds which are collections of items. Sets of items are contained within their respective feeds. A database account consists of a set of databases, each containing multiple collections, each of which can contain stored procedures, triggers, UDFs, documents, and related attachments. A database also has associated users, each with a set of permissions to access various other collections, stored procedures, triggers, UDFs, documents, or attachments. While databases, users, permissions, and collections are system-defined resources with well-known schemas - documents, stored procedures, triggers, UDFs, and attachments contain arbitrary, user defined JSON content. /udfs/{id} © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Why JavaScript Object Notation (JSON) JSON is a lightweight data-interchange format Support for rich queries and transaction support. Native Java Script support Supports programmability features such as user defined functions, stored procedures, and triggers.

Demo DocumentDB Open Visual Studio on your computer. Select New from the File menu, and choose Project. In the New Project Dialog, select Templates / Visual C# / Console Application, name your project, and then click Add. In the Solution Explorer, right click on your new console application, which is under your Visual Studio solution. Then without leaving the menu, click on Manage NuGet Packages... On the left most panel of the Manage NuGet Packages window, click Online / nuget.org. In the Search Online input box, search for DocumentDB Client Library. Within the results, find Microsoft Azure DocumentDB Client Library and click Install. The package ID for the DocumentDB Client Library is Microsoft.Azure.DocumentDB Open the solution in visual studio, ensure that you have a breakpoint after DB create Press f10 to step through and stop before deleting the database. Go back to portal and review it.

Indexing Automatically indexes all document properties Uses write optimized, lock free, log structured index maintenance techniques. The DocumentDB indexing subsystem is designed to support:   · Efficient, rich hierarchical and relational queries without any schema or index definitions. · Consistent query results while handling a sustained volume of writes. For high write throughput workloads with consistent queries, the index is updated incrementally, efficiently, and online while handling a sustained volume of writes. · Storage efficiency. For cost effectiveness, the on-disk storage overhead of the index is bounded and predictable. · Multi-tenancy. Index updates are performed within the budget of system resources allocated per DocumentDB collection. For most applications, you can use the default automatic indexing policy as it allows for the most flexibility and sound tradeoffs between performance and storage efficiency.

What Does DocumentDB Cost? Standard pricing tier with hourly billing S1, S2 and S3: good, better, best Each collection includes 10GB of SSD storage Limit of 100 collections for each account – can be lifted as needed

Each complete feedback form gives you a chance at the raffle We want your feedback Each complete feedback form gives you a chance at the raffle

Microsoft Corporation Our Gold Sponsors Microsoft Sri Lanka Pluralsight Microsoft Corporation Brought to you by: