1. OLAP Council APB-1 OLAP Benchmark Release II
November 1998
Database Lab.
Kwon,Sook-Kyo
3/30/2000

2. CONTENTS INTRODUCTION APPLICATION ENVIRONMENT DATABASE STRUCTURE
DATA FILES
QUERIES / QUERY DISTRIBUTION
IMPLEMENTATION RULES
FULL DISCLOSURE AND AUDITING

3. INTRODUCTION (1) OLAP Council
Sponsor the development of an Analytical Processing Benchmark
Successful OLAP Application must provide ‘Just-In-Time-Information’.
Goal of APB-1
Measure a server’s overall OLAP performance rather than specific tasks.
Include common business operations
Bulk loading of data from internal or external data source
Incremental loading of data from operational systems
Aggregation of input level data along hierarchies
Calculation of new data based on business models
Time series analysis
Queries with high degree of complexity
Drill-down through hierarchies
Ad hoc queries
Multiple on-line session

4. INTRODUCTION (2) Judging Server’s Ability
Represent complex business relationships
Respond to changing business requirements
Standard benchmark metric : AQM (Analytical Queries per Minute)
Number of analytical queries processed per minute
Data loading performance, calculation performance, query performance
Calculated as
Total time in seconds to perform the incremental data load +
Total time in seconds to perform batch computation, if required +
Total time in seconds to execute all queries =
Total time in seconds for AQM measurements.
AQM = Total number of queries executed * 60 /
Total time in seconds for AQM measurement

5. Application Environment
Contains a set of business operation that exercise basic functionality
Sales and Marketing analysis system
Synthesis of general business practices
Analyze product sales to customers through distribution channels over time:
unit sold, dollar sales, costs, margins are tracked by actual, budget,
and forecast scenarios.
Included items in the computation of AQM
calculated values are preprocessed
calculated values are stored in database
calculated values at query time
Updated incrementally at a specific time interval (weekly, or monthly)
Not included items in the computation of AQM
historical data loading

6. Database Structure (1) Six dimensions Product
Time, scenario, measure,
3 aggregation dimensions that define database size (product, customer, and channel)
Product
10 times the number of members in customer dimension
minimum number of members : 10,000
7 levels : Top, Division, Line, Family, Group, Class, Code
Customer
100 times the number of members in channel dimension
minimum number of members : 1,000
3 levels : Top, Retailer, Store

7. Database Structure (2) Channel fewest members Time
minimum number of members : 10
number of members in channel is input parameter to GEN program
2 levels : Top, Base
Time
dimension : two years (1995 and 1996) of monthly members
Julian calendar is used
hierarchy : quarterly, yearly, year-to-date aggregations
current month in the benchmark : June, 1996
aggregation : summation except;
Inventory - ending balance calculation
Cost factor measures - average balance
Percentage measure - cannot be aggregated

8. Database Structure (3) Scenario
3 base values : actuals and budget (from data files), forecast (modeled scenario )
Variance values : Budget vs. Actuals, Forecast vs, Actuals
Measures
Input
Unit sold varies by product,customer, channel, time, scenario
Dollar Sales varies by product, customer, channel, time, scenario
Inventory varies by product, customer, time
Product Cost varies by product, time, scenario
Shipping Cost varies by customer, time, scenario
Calculated
Average Price = Dollar Sales /Unit Sold
Cost, Margin, Margin Percent, Smoothed Sales

9. Data Files (1) Two set of data files
Initialize the database, incremental processing
APB1GEN program generates all data files
Files area presented in both tree and level formats
Product Hierarchy - Level Format Tree Format
Column Width Filed Column Width Field
Code Member
Class Parent
Group Level
Family
Line
Division

10. Data Files (2) Hierarchy data files : product, customer, channel
Reference scenario members by the names
: ACTUAL, BUDGET, FORECAST
Reference members of time dimension at the month level
Not in any sorted order
Sorting time will be included in the calculation of AQM

11. Queries / Query Distribution
Ad hoc and very dynamic
The time periods, products, customers, and channels must be dynamically generated from their respective hierarchies.
To best simulate a realistic operating environment,
each of the queries will have substitution parameters.
Query 1 - Channel Sales Analysis : 10 %
Query 2 - Customer Margin Analysis : 10 %
Query3 - Product Inventory Analysis : 15 %
Query 4 - Time Series Analysis : 3 %
Query 5 - Customer Budget : 5 %
Query 6 - Product Budget : 5 %
Query 7 - Forecast Analysis : 15 %
Query 8 -Budget performance : 20 %
Query 9 - Forecast Performance : 15 %
Query 10 - Ad Hoc : 2 %

12. Implementation Rules (1)
Software used to implement
Must be generally available
Should not be engineered specially
Use of alpha, beta, limited release, non full-production is not accepted.
Database design restriction
All data must be stored on the server
All calculation must be performed on the server
APB-1 is executed in six steps:
1. Execution of APB.EXE program to produce hierarchy / historical data
2. Database initialization and historical data load and optional pre-calculation
3. Execution of the APB.EXE program to produce incremental data files
4. Incremental data load and optional pre-calculation
5. Execution of APB.EXE program to produce query data files
6. Query execution

13. Implementation Rules (2)
All processing must be done on the server.
The execution time for step begins when the first user initiates
the first query and ends when the last user ends the last query.
The results of each query must be written to disk on the client
platform to be available for auditing.

14. Full Disclosure and Auditing (1)
Full disclosure is required by anyone publishing APB-1
benchmark results so that others can duplicate the results
same documentation, settings, products and environment.
The disclosure includes the information ;
an evaluator needs to judge a server’s processing performance
its suitability to the task of analytical processing
Disclosure Items;
Audit Report, Database schema, Any code, scripts, programs and / or
interactive user processes used for the dat loading……..
Total number of users, data file size, AQM metric, OLAP software
Server / Client hardware, network S/W H/W configuration...

15. Full Disclosure and Auditing (2)
The audit of the APB-1 benchmark will consist of the
following six tasks :
. Verifying that benchmark was run in the prescribed order as detailed
in the implementation rules section
. Examining the programming code on both the client and the server
to verify that :
The client’s sole role is to pass the queries to and receive the
results from the server
The server performs all calculations

16. Full Disclosure and Auditing (3)
. Verifying that the results returned are correctly calculated. Thirty queries
(three of each query type) must be randomly chosen and must be audited to
Ensure data accuracy
Determine that the dimensions members returned are correct
. Ascertaining that all disclosure items are correctly stated in the
disclosure document.
. Verifying that the benchmark implementation conforms completely
to the OLAP council APB-1 benchmark specification, addenda and
errata. Auditor must also specify which revision of the APB-1
benchmark specification and which version of APB.EXE program
was used.
.Producing a report detailing the audit steps and results.