Inter-RPO Emissions Inventory Warehouse System Update

Slides:



Advertisements
Similar presentations
ASIAES Project Overview Satellite Image Network for Natural Hazard Management in ASEAN+3 region Pakorn Apaphant Geo-Informatics and Space Technology Development.
Advertisements

Integration of Applications MIS3502: Application Integration and Evaluation Paul Weinberg Adapted from material by Arnold Kurtz, David.
Data Warehousing: Defined and Its Applications Pete Johnson April 2002.
Chapter 10 Publishing and Maintaining Your Web Site.
Chapter 9 Collecting Data with Forms. A form on a web page consists of form objects such as text boxes or radio buttons into which users type information.
ISOWare Presentation January 2009 ISOWARE is a management tool, that simple and efficient describes and communicates Business Processes. ISOWARE is also.
This presentation will guide you though the initial stages of installation, through to producing your first report Click your mouse to advance the presentation.
Electronic Reporting for Clean Air Act NSPS and NESHAP Regulations
The purpose of this Software Requirements Specification document is to clearly define the system under development, that is, the International Etruscan.
About Dynamic Sites (Front End / Back End Implementations) by Janssen & Associates Affordable Website Solutions for Individuals and Small Businesses.
Tutorial 10 Adding Spry Elements and Database Functionality Dreamweaver CS3 Tutorial 101.
Open Emissions Model(OPEM) Emissions Model for Models3/CMAQ and CAMX Mark Janssen – LADCO/MWRPO Zion Wang – UC Riverside
AQS Web Quick Reference Guide Changing Raw Data Values Using Maintenance 1. From Main Menu, click Maintenance, Sample Values, Raw Data 2. Enter monitor.
1 Web Server Administration Chapter 1 The Basics of Server and Web Server Administration.
MSF Requirements Envisioning Phase Planning Phase.
PostalOne! / FAST Data Exchange - Vision 02/15/05.
NMED 3850 A Advanced Online Design January 12, 2010 V. Mahadevan.
TSS Project Update WRAP Technical Analysis Forum San Francisco, CA October 11, 2007.
Chapter 9 Publishing and Maintaining Your Site. 2 Principles of Web Design Chapter 9 Objectives Understand the features of Internet Service Providers.
Emission Inventories and EI Data Sets Sarah Kelly, ITEP Les Benedict, St. Regis Mohawk Tribe.
Carolina Environmental Program 1 UNC Chapel Hill A New Control Strategy Tool within the Emissions Modeling Framework Alison M. Eyth Carolina Environmental.
FP6 IT System 1 ELECTRONIC PROPOSAL SUBMISSION SYSTEM.
TSS Database Inventory. CIRA has… Received and imported the 2002 and 2018 modeling data Decided to initially store only IMPROVE site-specific data Decided.
1 Emissions Modeling Survey Results Marc Houyoux, US EPA Presentation to the RPO National Workgroup November 4-6, 2004 St. Louis, Missouri.
Emission Inventory Input Format Eastern Canadian Premiers/New England Governors Conference Data Exchange Workgroup Gregory Stella U.S. Environmental Protection.
1 Emissions Inventory Overview-Part 2 Melinda Ronca- Battista, ITEP/TAMS Center.
TSS Project Update WRAP Technical Analysis Forum Boise, ID May 22, 2007.
WRAP Activities Overview §308 Technical Work Outline June 3, 2003.
Customer Billing Services Electronic Workflow and Online New Service/ Service Disconnect Application Information Technology Services Financial Services/Customer.
E-commerce Architecture Ayşe Başar Bener. Client Server Architecture E-commerce is based on client/ server architecture –Client processes requesting service.
Advanced Higher Computing Science
KEEPS – a system for UELMA preservation and security
Intermountain West Data Warehouse - Western Air Quality Study
Make-Up Testing/Undo Student Test Submissions
Architecture Review 10/11/2004
Western Regional Technical Projects 2011 through 2013
Chapter 8 Environments, Alternatives, and Decisions.
Software Application Overview
Shared Services with Spotfire
Web Development Web Servers.
Introduction and Principles
EPA Tools and Data Update
Introduction to Operating Systems
ELECTRONIC PROPOSAL SUBMISSION SYSTEM
Overview of Emissions Processing for the 2002 Base Case CMAQ Modeling
MANE-VU Emissions Inventory Update
Flanders Marine Institute (VLIZ)
Dissemination of Haze Data, Data Products and Information
Publishing and Maintaining a Website
PHP / MySQL Introduction
Division of Air AirCom DARM’s New Compliance and Enforcement Database and Field Inspection Tool.
Preparation of Fine Particulate Emissions Inventories
Intermountain West Data Warehouse
EPA Office of Air Quality Planning and Standards
Information Technology Ms. Abeer Helwa
LO2 - Be Able to Design IT Systems to Meet Business Needs
Course: Module: Lesson # & Name Instructional Material 1 of 32 Lesson Delivery Mode: Lesson Duration: Document Name: 1. Professional Diploma in ERP Systems.
Issues on Ozone Planning in the Western United States
Serpil TOK, Zeki BAYRAM. Eastern MediterraneanUniversity Famagusta
ETS – Air Data submission Training
SMOKE Improvements in version 2.0
The ultimate in data organization
SDLC Phases Systems Design.
This presentation document has been prepared by Vault Intelligence Limited (“Vault") and is intended for off line demonstration, presentation and educational.
E.H. Pechan & Associates, Inc.
Fire Emissions Joint Forum February 9, 2005
Technical Review Workshop
Regional Modeling for Stationary Source Control Strategy Evaluation
Emissions Discussion Group November 6, 2003
Inter-RPO Projects Update
Presentation transcript:

Inter-RPO Emissions Inventory Warehouse System Update Megan Schuster, MARAMA RPO Technical Meeting June 10, 2005

Technical Oversight Committee WRAP – Tom Moore CENRAP – Kathy Pendleton, Annette Sharp VISTAS – Pat Brewer, Greg Stella MRPO – Mark Janssen MANE-VU – Megan Schuster, Jung-Hun Woo, Serpil Kayin

Solution requirements Design of warehouse solution to meet RPO needs Outline Introduction Web-based solution Solution requirements Design of warehouse solution to meet RPO needs

Introduction and Background Inventory Development/Sharing Source types: point, area, nonroad, onroad, and biogenic sources RPOs have developed 2002 Baseline Photochemical Modeling Inventory PM2.5, Regional Haze and Ozone State Implementation Plans (SIPs) RPOs also developing future year and typical year inventories A lot of information Each RPO has develop an inventory for 2002 and will be developing future year inventories too.

Inventories are continually updated Challenge Inventories are continually updated Each RPO has several versions of each inventory Making the most up-to-date, quality assured inventory available to large user community is difficult Main issue – each RPO has several versions of each inventory that has been/ will be developed Difficult to get our data out to users in a timely fashion

Inter-RPO Emissions Inventory Warehouse Warehouse Solution Inter-RPO Emissions Inventory Warehouse Web-based application to store and share inventory and associated data

Warehouse Requirements Easy and accessible data upload and download Allow for several acceptable formats Versioning - Track versions of inventories Security - Secure Data Storage and Transmittal Maintenance - Low system operating and maintenance costs Database Storage Supporting Data Listserv Reports Low costs for system to survive for extended period of time

Required Data Formats and Files NIF Records (all sources) RPO Data Exchange Protocol Files Mobile emissions files Continuous emissions monitoring (CEM) files Temporal allocation and profile assignment files Spatial surrogate files Chemical speciation files Growth and control factor files Basically there a lot of data types that need to shared across the RPOs and with the general public, and we need a system that is large enough to handle all the file types necessary

Required Data Formats and Files Nonroad model inputs MOBILE6 model inputs NMIM inputs System is also required to handle Nonroad, M6 and NMIM inputs

Web Site Map ERG and Environ helped the Inter-RPO TOC design a system to meet all of our requirements. The system design is outlined in this website map. There will be 1 main homepage, with 4 pages under that. I will show screen shots of each page to give a taste

Web Site Functionality Four Main Components of Site Importing Data RPO assigned importer responsible for: Verifying the content of the files Determining if and when higher quality data are available and should replace Designating which files and info are associated with a specific version of the inventory Determining when a set of inventory files is complete and of sufficient quality to share with the public Follow naming convention for inventory files to be parsed into the database Required by EPA Content and Format Checker

Web Site Functionality Downloading Data Public users can search for and download files stored and managed in the system

Web Site Functionality 3. Reporting Public users Summarize and retrieve inventory data stored in the database Reports include Emissions summaries Top 10 sources Threshold Reports: sources that exceed pollutant and stack thresholds. Mobile VMT and speed data reports Reports can be generate for a subset of an inventory based on source type, geographic location, SCC level 1 or SCC level 2, pollutant, and stack parameters Emission summaries - user can choose inventory name, category – typical actual, source type – point, version, geographic location, (RPO, state, tribe, county) , SCC level, pollutant Top 10 sources – Point source reports will be at facility level, non-point at SCC level. Threshold report – inv version, category, year, source type, geo location, pollutant, pollutant threshold criteria, stack threshold criteria (for point source only), stack height threshold, stack diameter threshold, stack flowrate threshold. Point – at facility level, non-point at SCC level (at the level reported) Mobile vmt and speed data reports – inv version, category, year, geo location, roadway type, vehicle clssification, report type, vmt summary or avg speed, and time period: hourly or daily

Web Site Functionality Maintenance Manage users and modify limited portions of the site content

Web Site Functionality Three databases will be supported Working database – inventories actively being worked on by RPOs Public Database – publicly accessible data Archive Database – store older versions

Few clips of how the system is designed to work. Users will be required to enter a user name and password. Each user must have a valid email address. Listserv will be created based on login email addresses. Updates will be sent via a listserv.

As said earlier, a requiremetn of this system was that it need to track version of the inventory. One of the greatest challenges to the RPO community was managing and tracking inventory versions. The Inter-RPO Emissions Inventory Warehouse will automate the process of assigning versions to inventory data. Specific inventory version ‘names’ will be added to the database. Each RPO will have access to an upload area for their data files only. As RPO users upload data, they will be required to designate the inventory version for the file. Each RPO will be able to upload as many updates to an inventory as desired until a predetermined number of days before the inventory release date.

Public will be able to specify what geographic area, year, inventory version, source and file type they require. Depedning on size of the file – it may be placed on an FTP site or emailed to the user

As described earlier one of the requiremetns of this system was that it store a great amount of data. Uploading, storing and providing access to, and allowing users to download several versions of a nationwide inventory and ancillary modeling files each year requires a tremendous amount of server storage and database resources. To reduce the requirements for database storage, only those files and data needed for reporting purposes will be stored in the database. All other files will be stored in their native format, compressed if appropriate, on a file management server. To speed upload and download times, file transfer via http and ftp will be enabled. One of the challenges of using an open-source database tool is the potential impacts on the speed of real-time, on-line reporting when working with extremely large data sets such as those that will be stored in the Inter-RPO Emissions Inventory Warehouse. The RPO TOC decided to address this challenge in two ways: 1) after users select report criteria, the report will be scheduled and generated off-line; the user will be able to download the completed report at a later time, and 2) the speed of off-line report generation will be improved through the use of pre-aggregated data tables that can support most reporting requirements. Stored procedures will be used to populate these pre-aggregated tables after each version of an inventory is finalized and designated as ready for release

Open Source Solution Database: PostgreSQL Web Site Architecture Open Source Solution Database: PostgreSQL www.postgresql.org Procedural Language: PL/pgSQL Web interface: Java Server Pages (JSP) Servers: Linux, Apache Considering the requirement to provide Internet access to the data and files managed by the Inter-RPO Emissions Inventory Warehouse and the significant file/database server resource requirements, the licensing fees, hosting fees, and the costs of staff with specialized expertise required to prepare and maintain proprietary database technology such as Oracle or SQL Server could be prohibitive. To balance these competing needs, the RPO TOC chose to implement the website using reliable, proven, open-source technologies that are capable of handling large volumes of data. Additional details on the selected open source technologies and site architecture are provided later in this paper.

Draft Design document complete Final Deign Document due June Status and Next Steps Draft Design document complete Final Deign Document due June Require RPO TOC approval for implementation Begin Implementation June/July Testing in Fall Scheduled Final Implementation Date January 2006

Project Website: http://www2.ergweb.com/projects/interrpo/rpo/index.htm user name: rpo password: warehouse