History Many teams have made great efforts to use relational databases in projects only to have to supplement them in order to meet performance demands.

Common 3-tier Architecture

Physical Architecture Web Farm Caching Server CDN - Media Application Server Search Engine Server Database Server SANS or Disk Array

In the 90 s Microsoft development best practices, code samples and designs started with designing the database first Eventually ORMs NHibernate 2004, Linq to SQL Nov 2007, and Entity Framework August 2008 The Path: Education, Experiences, Vendor Influences

ORMs solve the problem of interacting with the database by mapping domain objects to relational tables (CRUD) Effects with NHibernate focus more on domain (EF eventually follows) We have taught BAs about CRUD and other technology terms to cope with constraints

Performance of SELECT mapping over 500 iterations - POCO serialization MethodDurationRemarks Hand coded (using a SqlDataReader) 47ms Dapper ExecuteMapperQuery 49ms ServiceStack.OrmLite (QueryById) 50ms PetaPoco 52msCan be faster BLToolkit 80ms SubSonic CodingHorror 107ms NHibernate SQL 104ms Linq 2 SQL ExecuteQuery 181ms Entity framework ExecuteStoreQuery631ms

Performance of SELECT mapping over 500 iterations - typical usage MethodDurationRemarks Linq 2 SQL CompiledQuery 81ms Not super typical involves complex code NHibernate HQL 118ms Linq 2 SQL 559ms Entity framework 859ms SubSonic ActiveRecord.SingleOrDefault3619ms

Common Performance Tweaks Indexes De-normalizing Tables Pinning Tables Full Text Search Scaling Up or Out? Scale Up with SQL Server is cheaper than scaling out (licensing) SQL Azure

Our Efforts ORMs learning curve 30 to 40 % of the project for saving data In pursuit of Normalization / Normal Form Normalization in reality isn’t normalization at all! First step in performance: Caching, caching, and more caching Why did we do this again?

Changing your perspective Simpler design tends to be better MVC is a successful UI pattern because it is a simpler solution than say web forms What are we doing today? What is caching? A key/value database Taking advantage of caching

Document Database is one solution Mongo, CouchDB, RavenDB etc… Changing your Model to deal with performance Is this the answer? No silver bullet Great to have in the tool belt One view one model

Advantages of document database Craigs List – scheme change took over a month to process using MySQL Farm Question: How is the data being used and what is the data being used for? Event based architecture – events happen in a system capture the changes

CQRS - Command Query Separation Separating your reads from your writes At its heart is a simple notion that you can use a different model to update information than the model you use to read information. This simple notion leads to some profound consequences for the design of information – Martin Fowler

Writes (Commands) Are commands Service Bus Handlers on different machines Publish event to db Publish event to cache Publish event to search engine Events stored in event store A true audit log Reply events to populate cache Reply events to find exact state of user!

Events as a Storage Mechanism Save data as binary in SQL Server or JSON in document database From Jonathan Oliver’s Event Store CREATE TABLE [dbo].[Commits] ( [StreamId] [uniqueidentifier] NOT NULL, [StreamRevision] [int] NOT NULL CHECK ([StreamRevision] > 0), [Items] [tinyint] NOT NULL CHECK ([Items] > 0), [CommitId] [uniqueidentifier] NOT NULL CHECK ([CommitId] != 0x0), [CommitSequence] [int] NOT NULL CHECK ([CommitSequence] > 0), [CommitStamp] [datetime] NOT NULL, [Dispatched] [bit] NOT NULL DEFAULT (0), [Headers] [varbinary](MAX) NULL CHECK ([Headers] IS NULL OR DATALENGTH([Headers]) > 0), [Payload] [varbinary](MAX) NOT NULL CHECK (DATALENGTH([Payload]) > 0), CONSTRAINT [PK_Commits] PRIMARY KEY CLUSTERED ([StreamId], [CommitSequence]) );

Events as a Storage Mechanism Snapshots to lower the level of mining on multiple events From Jonathan Oliver’s Event Store CREATE TABLE [dbo].[Snapshots] ( [StreamId] [uniqueidentifier] NOT NULL, [StreamRevision] [int] NOT NULL CHECK ([StreamRevision] > 0), [Payload] [varbinary](MAX) NOT NULL CHECK (DATALENGTH([Payload]) > 0), CONSTRAINT [PK_Snapshots] PRIMARY KEY CLUSTERED ([StreamId], [StreamRevision]) );

Events as a Storage Mechanism Aggregate: A collection of objects that are bound together by a root entity, otherwise known as an aggregate root. The aggregate root guarantees the consistency of changes being made within the aggregate by forbidding external objects from holding references to its members. Example: When you drive a car, you do not have to worry about moving the wheels forward, making the engine combust with spark and fuel, etc.; you are simply driving the car. In this context, the car is an aggregate of several other objects and serves as the aggregate root to all of the other systems.

Reads (Query) Reads on cache machines Reads on search indexes Sharding available on some cache solutions Transactions available on some solutions

Effects Effects on business BAs – now talking about business Simpler development – more productivity Need scalability add machine for reads, writes, or bus Audit log – exactly what the user was doing Build cache Build development environment Task Based UI for writes

Physical Parts Service Bus Caching Web Server Servers for Handlers Database

Task Based User Interface Single URL application More complicated Pub / Sub with Node or SignalR

Resources Gregg Young Udi Dahan Jonathan Oliver Rinat Abdullin