New Technologies for Storage and Display of Meteorological Data Dale Paynter Operations Management Group May, 2002 NUMUG Conference

10 May 2002OMG2 Benefits of Technology? zMove to modern technology path that will be developed and supported zMake initial installation, maintenance & upgrades easier yChange is the name of the game xExternal requirements & standards xInfrastructure changes xOrganizational re-alignment yMeet rising expectations xSimple systems interfaces xCapitalize on staff knowledge & skills zLet Meteorologists manage, analyze and use the data

10 May 2002OMG3 New Technologies What are they? zData Driven System Architecture zRelational Databases zWeb Applications

10 May 2002OMG4 Data Driven Architecture What is it? Write code to be independent of specifics Make system configurable to define: yInput stream yParameters for internal calculations yReports yInterfaces to other systems

10 May 2002OMG5 Data Driven Architecture What are the benefits? zRespond to new requirements and standards zReduce dependency on vendors & IT zGet out of program change queue zReduce change orders & site visits zGive users control through configuring system

10 May 2002OMG6 What does this let you do easily? zAdd new sensors with no code changes zCreate new list reports zChange corporate information zRevise alarm / flag limits zAdd or decommission sampling locations zInterface to different systems

10 May 2002OMG7 So how do we do this? Parameters, parameters, parameters... zMake data base ‘self-descriptive’ yStore ‘what’ is being stored as well as ‘values’ yEliminate need to reorganize / reload data to accommodate changes zMake variables for all ‘constants’ yTower names yTime zones yDaylight saving time yMeasurement codes yReport titles & headings yData to display / order yReport field formats yConversion formulae

10 May 2002OMG8 Adding a new sensor zTraditional “Programming” Approach yUser configures or reprograms datalogger yProgrammer adds new field to file formats. Then dumps existing data, reloads and verifies yProgrammer changes data loading to process new input layout & update new field yProgrammer changes updating to display / edit new field yProgrammer changes reporting to display new field ySystem administrator installs final programs on all affected computers z Data Driven Architecture Approach yUser configures datalogger yUser defines sensor in database yUser tests and verifies results ySystem administrator copies configurations to production system

10 May 2002OMG9 Relational Databases What are they? zData is stored in Tables made up of rows of fields (columns) zData is referenced through name of the field zSubsets of data may be selected zTables may be joined to retrieve related records

10 May 2002OMG10 Relational Databases Usage comparison example zSequential file - 30’ wind speed is in columns 26 to 31 (format F6.2) zTypical relational database - 30’ wind speed is stored in a field named WS30 zRelational database using data driven architecture - 30’ wind speed is stored in a field named Value where the field MeasCode contains ‘WS’ and Height contains ‘30’

10 May 2002OMG11 Relational Databases What are the the general benefits? zBuilt-in security zSupports simultaneous accesses zBuilt-in data recovery techniques zBuilt-in data retrieval algorithms & optimization zStandard database interaction (SQL) zIn-house expertise can be leveraged

10 May 2002OMG12 Relational Databases And... for meteorological data? zConcurrent loading, updating and reporting zPermits non-sequential loading for legacy data and data recovery zHandles data retention and archiving requirements zAd hoc queries zTime-stamping data changes zKeep original values as well as edited values

10 May 2002OMG13 Web Applications In general... zUser requests report and submits parameters through browser zWeb application generates report on server and sends to browser zUser views / prints report through the desktop browser Benefits? zRemoves need to run / maintain specific report programs on desktop zJust ask your LAN Administrator…

10 May 2002OMG14 Web Server Configuration Meteorologist’s Workstation running Updater Meteorological Monitoring System Server MMS Database Intercept TM & Loader Web Server Reporter Interface Files Other Servers Meteorologist’s & Users’ Workstations running Web Browsers Sensors Zenos Zeno ® s

10 May 2002OMG15 Web Technology Internet or Intranet Server zData manipulation and report generation performed on server zOnly results sent to browser zAutomatic report refresh option zAdditional security levels possible

10 May 2002OMG16 How does it perform? Even running on a notebook: zCurrent reports are instantaneous zHistorical reports take a few seconds zFrequency Distribution produces a 1 year report in 12 seconds zData Capture Summary produces a 1 year report in 55 seconds

10 May 2002OMG17 Benefits of Technology zMove to modern technology path that will be developed and supported zGet out of the program change queue zFlexibility to respond to change zLet Meteorologists manage, analyze and use the data