State of the Practice for ITS Data Archiving: Regional Data Archive and Performance Measures Robert L. Bertini Portland State University North American Travel Monitoring Exhibition & Conference San Diego, California  June 30, 2004

2 Objectives  Introduction to Data Archiving Project  National ITS Architecture and ADUS  Introduce Portland Regional ADUS  Describe Architecture  Describe Database Processing and Storage  Data Analysis and Visualization  Conclusions and Next Steps

3 Introduction Guiding Principles “Data are too valuable to only use once.”

4 Introduction Guiding Principles “ Management of the transportation system cannot be done without knowledge of its performance.”

5 National ITS Architecture Major Components  Travel and Traffic Management  Public Transportation Management  Electronic Payment  Commercial Vehicles Operations  Emergency Management  Advanced Vehicle Safety Systems  Information Management  Maintenance and Construction Management

6 Overview: Relational Diagram of ITS Architecture Source: Guidelines for Developing ITS Data Archiving Systems

7 Archived Data User Service (ADUS) is Born ITS Architecture 1999  USDOT Vision for ADUS: “Improve transportation decisions through the archiving and sharing of ITS generated data.”  Principles of ADUS to Achieve Vision  All ITS deployments should consider data archiving  Archive data to maximize integration with other data sources and systems  Archive data in a way that eases retrieval for those with access  Provide information that is integral to transportation practice

8 Archived Data User Service (ADUS) is Born ADUS Standards  Operational Data Control: by managing operations data integrity  Data Import and Verification: through historical data  Automatic Data Historical Archive: with a permanent data archive  Data Warehouse Distribution: which integrates the planning, safety, operations and research communities and processes data for these communities  ITS Community Interface: by providing a common interface to all ITS users for data products specification and retrieval

9 Who Can Use Archived ITS Data Stakeholders  Transportation Planning  Transportation System Monitoring  Air Quality Analysis  MPO/State Freight and Intermodal Planning  Land Use/Growth Management Planning  Transportation Administrators and Policy Analysis  Traffic Management  Transit Management  Construction and Maintenance  Safety Planning and Administration  Commercial Vehicle Operations  Emergency Management  Transportation Research  Private Sector

10 Data Resources Potential Archiving Applications  Traffic Surveillance  Fare/Toll Systems  Incident Management  Traffic Video  Environmental  CVO  Traffic Control  Highway/Rail  Emergency Response ITSDataArchivesITSDataArchives  Performance Monitoring National reporting Performance-based planning Evaluations Public Reactions  Long Range Planning TRANSIMS IDAS Four step models Transit routes  Operations Planning Incident management ER deployment Signal timing Transit service  Travel Time Forecasting Customized route planning ATIS Advisories  Other Stakeholder Functions Safety Land use Air quality Maintenance management

11 ADUS Architecture: Implementing a Successful Data Archive A Few Examples of Existing Systems  California PeMS - only statewide system  Puget Sound (WSDOT/TRAC) - started small and successfully expanded  San Antonio, TX TransGuide, Datalink system Lessons Learned from Existing Systems  Begin with a single data source  Provide data through the web or CD subscription  Create a user friendly interface  Save raw data  Make aggregate data available to users  Implement data quality control measures  Create adequate documentation of system and metadata

12 Portland ADUS PSU Designated Regional Archive Center

13 Portland ADUS PSU Designated Regional Archive Center  Through regional cooperation, Portland State University is the regional center for collecting, coordinating and disseminating variable sources of transportation data and derived performance measures.  Portland ITS data will be warehoused at their most raw level.  A user interface for extracting relevant performance measures in real time and historical data is being implemented.

14 Portland ADUS Data Sources  Oregon Department of Transportation (ODOT)  Freeway loop detector data  Incident data  National Oceanic and Atmospheric  Administration (NOAA)  Weather data  Portland Metropolitan Transit Agency (Tri-Met)  Automatic vehicle location (AVL) poll data  Bus dispatch system (BDS) data  City of Portland  Traffic signal count/speed data  Washington State Department of Transportation (WSDOT)  ODOT Weigh-in-motion data

15 Portland ADUS Portland Regional Infrastructure  77 CCTV Cameras  18 Variable Message Signs (VMS)  436 Inductive Loop Detectors  118 Ramp Meters  TriMet Automatic Vehicle Location (AVL) System and Bus Dispatch System (BDS)  Extensive Fiber Optics Network

16 ADUS Architecture Fiber Optic Data Connection Rich Johnson, City of Portland

17 Portland ADUS Architecture Data Flows

18 Portland ADUS Architecture Database Back End/ Web Front End  Database Back End  SQL relational database  200 MB per day of raw data  75 GB per year  High capacity disk array  5 MB compressed for download  Off-site backup server  Daily backups  Uninterruptible power  Web-based interface  Easily accessible  User-friendly  PHP language

Database Processing and Storage Web Interface is Up and Running

20 Database Processing and Storage  Data Fidelity  Loop health analysis  Proven techniques to prevent propagation of error  Aggregation Levels  Raw 20-second loop detector data collected  Useful for research  Large data sets  Previously difficult to manage and organize  High resolution aggregates  5 and 15 minute aggregates  Flow, speed and occupancy characteristics retained  VMT, VHT, travel time, delay added  Useful for research and planning purposes  Lower resolutions aggregates  Hourly or Daily  VMT, VHT, travel time, delay  Increased processing speed for long term overviews

Data Analysis and Visualization Contour Plots

Data Analysis and Visualization Time Series Plots

Data Analysis and Visualization Data Fidelity

24 Next Steps Expanding Archive Functionality  Data from:  TriMet  Washington State DOT  City of Portland  ODOT WIM Data  Additional processing tools and performance measures

Acknowledgments  National Science Foundation  Oregon Department of Transportation  City of Portland  TriMet  Portland State University  Oregon Engineering and Technology Industry Council

26 Feedback Your Opinions and Suggestions Please let us know what you think about existing or future:  Data sources  Performance measures  Data visualizations and summaries Contact us: