Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 1 Henrik Nordberg, June 1998 Query Estimator Henrik Nordberg Lawrence Berkeley National Laboratory

Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 2 Henrik Nordberg, June 1998 Purpose of the Query Estimator Provide estimates of how “big” a query is Execute a query Primary purpose: Also needs to handle: Indexing of large amounts of data Simultaneous access

Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 3 Henrik Nordberg, June 1998 Query Estimator Functions Build bit-sliced index, and tag index Accept multiple asynchronous query requests Quick query estimate - use bitmap in-memory index Full query estimate & execute - use tag index Invoke Query Monitor for execution Act on “query abort” and “query done”

Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 4 Henrik Nordberg, June 1998 Quick Estimate (use bit-sliced index) (index in memory) no_of_events: (min,max) -- nearest bin boundaries no_of_files to be cached %_of_events_in_files that qualify for a query (max) Full Estimate - Execute (use tag index) (Index on disk) precise list_of_events that qualify set of (file: event_list) total_MBs_to_be_moved no_of_events_in_cache (also files_in_cache) time_to_process_query Estimation

Scientific Data Management Research Group National Energy Research Scientific Computing Center, L B N L 5 Henrik Nordberg, June 1998 MDC-1 RAM Requirements Assumptions: 2  100,000 events 20 properties (tags) 2 bytes per property 8 MB per index = 16 MB 2  10,000 average number of hits 100 concurrent queries 8 bytes per query 16 MB for queries Total = (code) = 42 MB