Developer Intro to Cosmos DB

About Me Brandon Brockhoeft BIO Brandon is a career software engineer working primarily with Microsoft solutions with over 15 years of professional experience. He has provided development and architecture technology solutions in the finance, state government, energy, and aerospace industries. His functional expertise includes: full-stack development, distributed architecture, real-time communication, DevOps and agile transformation, & mentoring. Brandon Brockhoeft Principal Consultant @bbrockhoeft22 brandon-brockhoeft-55777a1

What is Cosmos DB? Globally distributed, multi-model DB Azure service Schema-less (Code is the schema!) Transparently replicates data Multi-master writes Consistency choices Automatic indexing SQL and NoSQL APIs Changefeed (CEP)

DB Usage Comparison Cosmos DB SQL Server Scale Horizontal Vertical Data Geo-Distributed / Eventual Consistency Integrity and Consistency Focused Cost Throughput and Storage-based CPU-based Tooling Microsoft and Dev Community Microsoft and Commercial Products

Cosmos DB Setup Containers and Partitions Partition Keys Request units: item size, indexing, property count, consistency, query patterns RU/s = $ Cost Insufficient funds = throttled requests

Indexing Cosmos DB automatically indexes every property for all items in your container Indexing done synchronously when data is written Indexes are written as trees used to map to property paths: /locations/1/country: "France" Customize indexing by setting its indexing mode & include/exclude property paths Order by multiple properties requires custom composite indexes Index transformation done online and in-place: impacts query consistency

Tools Cosmos DB Data Migration Tool Build / Download: https://github.com/azure/azure-documentdb-datamigrationtool Documentation: https://docs.microsoft.com/en-us/azure/cosmos-db/import-data Manage and automate Cosmos DB resources: PowerShell Azure CLI Azure Resource Manager templates Work with data (using SQL) Cosmos DB Playground: https://aka.ms/cosmos-db-playground Document Explorer - Azure or Local Emulator

Data Modeling Code-first at all times Performing w/o a net: little constraints, no cascading, few DB types, no sizes, or defaults Have well-defined and tested validation patterns Mistakes not as easily corrected* (as SQL Server) Self-joins only in SQL API Unique key constraints only within partition Implies document ID is not the partition ID Can be single property or composite key Cannot be changed; requires new container + data migration Modeling techniques: Embed data (denormalized) Reference data (normalized) Hybrid

Data Modeling - Embed All within one JSON document - allows single read/write Duplication of data is okay Typically better read performance Useful when: Contained relationships between entities One-to-few relationships between entities Embedded data that changes infrequently Embedded data that will not grow without bound Embedded data that is queried frequently together { "id": "1", "firstName": "Thomas", "lastName": "Andersen", "addresses": [ "line1": "100 Some Street", "line2": "Unit 1", "city": "Seattle", "state": "WA", "zip": 98012 } ], "contactDetails": [ {"email": "thomas@andersen.com"}, {"phone": "+1 555 555-5555"} ]

Data Modeling - Reference No foreign-keys / weak links only Requires multiple database trips Don’t create join containers Typically better write performance Consider data growth for relationship Useful when: Representing one-to-many relationships Representing many-to-many relationships Related data changes frequently Referenced data could be unbounded Author documents: {"id": "a1", "name": "Thomas Andersen", "books": ["b1, "b2", "b3"]} {"id": "a2", "name": "William Wakefield", "books": ["b1", "b4"]} Book documents: {"id": "b1", "name": "Azure Cosmos DB 101", "authors": ["a1", "a2"]} {"id": "b2", "name": "Azure Cosmos DB for RDBMS Users", "authors": ["a1"]} {"id": "b3", "name": "Learn about Azure Cosmos DB", "authors": ["a1"]} {"id": "b4", "name": "Deep Dive into Azure Cosmos DB", "authors": ["a2"]}

Data Modeling - Hybrid Author documents: Book documents: { "id": "a1", "firstName": "Thomas", "lastName": "Andersen", "countOfBooks": 3, "books": ["b1", "b2", "b3"], "images": [ {"thumbnail": "https://....png"} {"profile": "https://....png"} {"large": "https://....png"} ] }, "id": "a2", "firstName": "William", "lastName": "Wakefield", "countOfBooks": 1, "books": ["b1"], } Book documents: { "id": "b1", "name": "Azure Cosmos DB 101", "authors": [ {"id": "a1", "name": "Thomas Andersen", "thumbnailUrl": "https://....png"}, {"id": "a2", "name": "William Wakefield", "thumbnailUrl": "https://....png"} ] }, "id": "b2", "name": "Azure Cosmos DB for RDBMS Users", {"id": "a1", "name": "Thomas Andersen", "thumbnailUrl": "https://....png"}, }

Sample Azure Data Explorer SQL Query examples - see CosmosDemo/CustomerContact/docs/SQL_Samples.md Customer Contact Demo application with C# SDK (v3) ASP.NET sample app with form data

More Info Conference video on Cosmos DB by Jimmy Bogard https://headspring.com/2018/05/30/video-from-sql-to-azure-cosmos-db/ Microsoft documentation (Used for presentation content) https://docs.microsoft.com/en-us/azure/cosmos-db/ Microsoft dotnet SDK reference https://docs.microsoft.com/en-us/dotnet/api/overview/azure/cosmosdb?view=azure-dotnet Query cheat sheets https://docs.microsoft.com/en-us/azure/cosmos-db/query-cheat-sheet

Session Evaluations Session Evaluations are ONLINE ONLY. Your feedback is valuable! SQLSatBR.com/Sessions/SessionEvaluation.aspx

Changefeed Event Driven Architecture by default Lambda Architecture with low TCO Use Azure Functions without infrastructure concerns Observe container changes sorted within partition key Consumer parallel processing available by partition key Use soft markers for data operation filtering (insert/update/delete) Consider use of TTL and start time options

Data Modeling - Mixed Types Book and Review documents in one container: { "id": "b1", "name": "Azure Cosmos DB 101", "bookId": "b1", "type": "book" }, "id": "r1", "content": "This book is awesome", "type": "review" "id": "r2", "content": "Best book ever!", }