Stored Procedures for Web Search Engine Dejan Sunderic

Solutions to Performance Problems Splitting resultset Denormalization

Previous Situation Three types of queries (quick, advanced, full)quickadvanced full Small number of concurrent users Typical query run at least 5 to 10 sec Workarounds

Example Query example Queryexample ERD example ERDexample Resultset

Technical issues Join on many tables Table scans (indexes, Like) Complete resultset sent to the client (B)locking Guid as nvarchar Sorting

How Did We Get Here? normalization OLTP v. DSS OO design Code generators

Available options (pro/contra) Denormalization Limiting resultset Splitting resultset Indexing Stored procedures v. ad-hock queries Search typestypes

Splitting Result Web server Temporary table Static table(s)

Denormalization Table structure (denormalized v. normalized)denormalizednormalized Surrogate index fields Process execution Process

Quick Queries Stored procedure Sorting Result splitting Top vs. Rowcount Next batch Search types Count

Advanced Query Dynamically assembled queries Dynamically queries Criteria Batch size Table joins Arrays in TSQL

Performance improvements Page rendering under 1 sec. IOs reduced more then 3 times (B)locking avoided 100s of concurrent users Network traffic reduced

What next ? Advanced queries with fixed params Reduce change delay ? Resultset length hard-coded Link with Capability Profiles Full-text search

Call to Action Always do splitting of a result Evaluate denormalization for searching/reporting Contact