U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Comprehensive Safety Analysis CSA 2010 National Rollout and Implementation CVSA Fall Conference September 21, 2009 U.S. Department of Transportation Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Presentation Overview CSA 2010 Defined Op-Model Test Results National Implementation Panel Discussion State Programs Impacts Roadside Uniformity Update SMS Methodology Details | 2

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Defined Gary Woodford, FMCSA CSA 2010 Program Manager

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Comprehensive Safety Analysis 2010 CSA 2010 is a pro-active initiative to improve the efficiency and effectiveness of FMCSA’s enforcement and compliance program. Uses ALL roadside inspection results and crash reports to identify safety deficiencies Employs a wider array of interventions tailored to problems instead of solely the time-intensive Compliance Review process Enables more carriers to be contacted earlier Requires sustained accountability of carriers AND increases accountability of drivers | 4

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 New Operational Model: 3 components | 5 1. New Safety Measurement System (SMS) Improved ability to identify demonstrated safety problems 2. New intervention process Employs an array of interventions instead of the single option, labor-intensive compliance review 3. New approach to the Safety Fitness Determination (SFD) SFD tied to current safety performance; not limited to acute/critical violations from a Compliance Review SFD requires rulemaking, not necessarily part of rollout in July 2010

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 New Safety Measurement System CSA 2010 introduces a new safety measurement system (SMS) that… Uses crash records and ALL roadside inspection safety- based violations to determine carrier/driver safety Weights time and severity of violations based on relationship to crash risk Triggers the intervention process (eventually would feed the proposed Safety Fitness Determination) Calculates safety performance based on 7 Behavior Analysis and Safety Improvement Categories (BASICs) | 6

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 SMS BASICs SMS BASICs focus on behaviors linked to crash risk 1.Unsafe Driving (Parts 392 & 397) 2.Fatigued Driving (Hours-of-Service) (Parts 392 & 395) 3.Driver Fitness (Parts 383 & 391) 4.Controlled Substances/Alcohol (Parts 382 & 392) 5.Vehicle Maintenance (Parts 393 & 396) 6.Cargo Related (Parts 392, 393, 397 & HM) 7.Crash Indicator | 7

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 SafeStat vs SMS Today’s Measurement System: SafeStatCSA 2010 SMS Organized by four broad categories - Safety Evaluation Areas (SEAs): Accident, Driver, Vehicle, and Safety Management Organized by seven specific Behavior Analysis Safety Improvement Categories (BASICs) Identifies carrier for a compliance review (CR)Identifies safety problems to determine who to investigate and where to focus the investigation From roadside inspections, uses only out-of- service (OOS) and moving violations Emphasizes on-road safety performance, using all safety-based road-side inspection violations No impact on safety ratingUsed to propose adverse safety fitness determination based on carriers’ current on-road safety performance (future) Violations are not weighted based on relationship to crash risk Violations are weighted based on relationship to crash risk Assesses carriers onlyTwo distinct safety measurement systems-one for individual carriers and one for individual commercial motor vehicle (CMV) drivers

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 New Intervention Process The New Intervention Process addresses the… WHAT Discovering violations and defining the problem (similar to current model), but also expanding to include the why and how WHY Identifying the cause or where the processes broke down HOW Determining how to fix it/prevent it through use of Safety Management Cycle and Safety Improvement Resources | 9

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Safety Management Cycle | 10

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 New Intervention Tools New intervention tools reach more carriers and influence safety compliance earlier Roadside Inspections Warning Letters Investigations −Offsite Investigations −Onsite Investigations – Focused −Onsite Investigations – Comprehensive Follow-on corrective actions −Cooperative Safety Plan (CSP) −Notice of Violation (NOV) −Notice of Claim (NOC) −Operations Out-of-Service Order (OOS) | 11

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Current vs CSA 2010 Intervention Process Current CR ProcessCSA 2010 Intervention Process Broad one-size fits-all investigation regardless of extent or scope of safety deficiencies Array of interventions can be tailored to address extent and scope of specific safety deficiencies Resource intensive for agency and time consuming for carrier/fewer carriers contacted Less resource intensive for agency and less time consuming for carrier/more carriers contacted Focuses on broad compliance based on rigid set of acute/critical violations Focuses on improving behaviors that are linked to crash risk Discover what violations existDiscover what safety problem(s) are and why they exist, to facilitate corrective action Major safety problems result in fines (Notice of Claim (NOC)) When problems found, major focus on carrier proving corrective action; significant problems continue to result in fines Focuses on carrierExpands focus to include investigating individual drivers

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Operational Model Test Results Bryan Price Office of Enforcement & Compliance Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Operational Model Field Test Began February 2008 Planned completion June 2010 Designed to test validity, efficiency and effectiveness of new model Evaluation to be conducted by independent 3 rd party Original test states include: Colorado, Georgia, Missouri, New Jersey | 14

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Test Spring 2009 –Added Montana and Minnesota 100% of the State participates in CSA 2010 –Offers a more accurate picture of efficiencies, capabilities and benefits –Tests integration with national program goals and Congressional mandates –Provides more data to evaluate test including workload and workforce analyses Fall 2009: Two additional states: Kansas and Maryland | 15

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Preliminary OM Test Results (10/1/08-8/31/09) Warning letters are having a positive impact: About 3,000 sent since Feb % of recipients logged in to view safety scores Feedback from test states indicate that carriers appreciate the early alert “…carrier officials thanked us for notifying them of their safety problems… once carrier officials understand that the new system enables them to identify their problem drivers, a light goes on. They see CSA 2010 as a tool that they can use to stress the importance of roadside inspections with their drivers, to hold their drivers accountable for their on-road safety performance, and to thereby improve their companies’ overall safety performance.” - Daniel Drexler, Division Administrator in Minnesota | 16 WARNING LETTERS TestNon-Test CSA Warning Letters / PRISM Warning Letters951535

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Preliminary OM Test Results (10/1/08-8/31/09) Reaching its goal of contacting more carriers –One objective of CSA 2010 was to conduct more investigations Employing the full array of investigations to achieve efficiency and effectiveness –Investigations in test states have been done in the following proportions: Onsite Investigations – Comprehensive (~25%) Onsite Investigations – Focused (~45%) Offsite Investigations (~30%) | 17 Test GroupNon-Test Group Resources (Safety Investigator: Full Time Equivalents)2539 INVESTIGATIONS Offsite Investigations294 (9 hours per)Not Applicable Onsite – Focused431 (13 hours per)Not Applicable Onsite - Comprehensive / Compliance Review (CR)221 (19 hours per)1,181 (20 hours per) Total Carrier Investigations9461,181 Total Carriers Investigated9311,117 Carrier Investigations per investigator per month3.52.8

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Preliminary OM Test Results (10/1/08-7/31/09) | 18 FOLLOW-ON ACTIONS Test GroupNon-Test Group Notice of Claim (NOC) From Offsite Investigations19Not Applicable From Onsite–Focused43Not Applicable From Onsite- Comprehensive/Compliance Review (CR)64348 Notice of Violation (NOV) 12No Data Available Enforcement (NOV, NOC) per investigator per month Cooperative Safety Plan (CSP)337Not Applicable Total Follow-On Actions Total Investigations with Follow-On Action469 (50%)348 (29%)

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Preliminary OM Test Results (10/1/08-7/31/09) Emphasizing Red Flag driver review with every carrier investigation | 19 DRIVER COMPLETED ACTIVITIES INTERVENTIONSTest GroupNon-Test Group Red Flag Driver Reviews 200No Data Available Driver Notice of Violation (NOV) 2No Data Available Driver Notice of Claim (NOC) 3549 Driver Enforcement (NOV, NOC) per investigator1.51.3

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 National Implementation Plans Bryan Price Office of Enforcement & Compliance Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Rollout Strategy Summer 2010: Nationwide rollout SMS replaces Safestat BASICs sent to roadside Summer – December 2010 : State by State rollout Roll out interventions Send warning letters after Division/state training | 21

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 What is Changing? The way FMCSA assesses carrier safety –Identifies unsafe carrier and driver behaviors that lead to crashes –Uses all safety-based roadside inspection violations –Evaluates/tracks driver performance individually How FMCSA addresses carrier safety issues –Reaches more carriers earlier and more frequently –Improves efficiency of investigations Focuses on specific unsafe behaviors Identifies root causes Defines and requires corrective actions How FMCSA promotes safety –Forces carriers/drivers to be accountable for their safety performance Demands and enforces safe on-road performance –Makes more complete safety performance assessments publicly available | 22

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 What Can Carriers Do To Prepare Now? Learn more about CSA: –Understand the BASICs –Check the site for implementation schedule –Sign up for latest news: RSS/listserv Check and update records –Motor Carrier Census (Form MCS -150) –Inspection and crash reports Ensure compliance –Review inspections and violation history over the past 2 years –Address safety problems now –Educate drivers about how their performance impacts their own driving record and the safety assessment of the carrier | 23

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Implementation: The State Partner Perspective A Panel Discussion

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 State Considerations Topic AreaThings to Consider TrainingNew measurement system, new interventions, new IT tools ITBASICs replace SEA values at roadside (ISS) Impact on related state systems Hardware or software changes and training OutreachRoadside inspectors (ISS viewchanges) Helping to spread word to industry Workforce PlanningJob or role changes? New or different resources needed? Administrative and/or DataQ increase? Policy/GuidanceIntrastate process State-specific legislation or policy FundingMore funding needed to implement? If so, when will I get information on how to apply ? | 25

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Implementation: State Program Impacts Tom Keane Chief, State Programs Division Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Overview 1.Overall Impacts to State MCSAP Programs 2.Specific Changes to CVSP Requirements 3.Other Changes – Data Quality 4.Specific Changes to FMCSA State Programs (SP) Policy Memos | 27

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 “Change is more about culture than cost” CR production may change – new interventions CR processes – still asked to follow eFOTM How you prepare CVSPs Volume of Data Qs challenges ( (11)) Questions/phone calls from industry 1. Overall Impacts-State MCSAP Programs | 28

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, Specific Changes to CVSP Will ask States to identify goals for new CSA2010 interventions as part of CR National Program Element CVSP National Program Element Changes –Driver/Vehicle Inspections –Traffic Enforcement –Compliance Reviews Add CSA 2010 interventions here –Public Education & Awareness –Data Collection | 29

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 May require updates to FY2010 CVSP budget –i.e., training & travel costs State-Specific Objectives should remain unchanged –Specific strategies & monitoring plan may change Maintenance of Effort (MOE) levels should remain unchanged 2. Specific Changes to CVSP – Continued | 30

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Data quality under new system is critical –How violations are treated by SMS –All safety-based violations count (not just OOS) –Roadside violations may be used to assign ratings Dependent upon SFD rulemaking Data Qs challenges will likely increase –Inspection Uniformity will be critical –Requests for inspection reports will increase –Impacted by Driver Info Resource (DIR) tool 3. Other Changes – Data Quality | 31

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, Specific Changes to MCSAP SP Memos Prior to Implementation – May update SP Memo – Funding Eligibility of Compliance Reviews (SP SA) –May update Inspection Selection System (ISS) Memo (SP GE ) After Implementation –Update to Cost Eligibility (Overtime) Memo (SP CE) –Update to MOE Memo (SP GE) | 32

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 FY2010 will be transition year - Be flexible –CSA 2010 Rollout –FMCSA grants program changes –Reauthorization (indirectly) Please articulate your concerns & challenges We will work with you during implementation! Primary Keys to Successful Implementation | 33

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 For more information please contact: Tom Keane, (202) | 34

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Roadside Data Uniformity Ad-hoc Status Report Mark Savage, Colorado State Patrol

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Objectives of CVSA Data Uniformity Ad-hoc Objectives of CVSA Data Uniformity Ad-hoc Consistent documentation of roadside inspection and violation data Standardized processes for challenging data Increased awareness and understanding that all inspections (good and bad) must be uploaded Uniform inspection selection processes | 36

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Ad-hoc Structure At the spring meeting the structure of the Data Uniformity Ad-hoc group changed –Much of the work was divided between the various committees Programmatic issues to PIC Roadside data violations to individual committees This format shows that CVSA is committed to the goals of the Ad-hoc for the long term | 37

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Consistent Documentation of Roadside Inspection and Violation Data Why? CSA 2010 Dependent upon SFD rulemaking, roadside data may be used to rate carriers as opposed to its current use which is to prioritize carriers for compliance reviews New safety measurement system uses more than just OOS violations in carrier measurement Program Integrity Data Reciprocity CVSA Strategic Goals | 38

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Consistent Documentation of Roadside Inspection and Violation Data Last spring CVSA and the Ad-hoc submitted a grant to FMCSA to fund travel costs associated with work on the Aspen violation lists. This work has been completed and a work-product will be presented at the various committees. The objective of the work groups was to: – “ To develop operational policies or guidance that will promote and/or mandate the consistent documentation of roadside inspection and violation data.” | 39

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Consistent Documentation of Roadside Inspection & Violation Data-Goals & Objectives The actual finished product is a consistent and uniform listing of violations that will appear in ASPEN/Safetynet –Thank you to the subject matter experts from the HM, Driver-Traffic Enforcement, Vehicle and Passenger Carrier committees –Goal is to correctly classify the data as consistently as possible across jurisdictional lines Operational guidance for roadside inspector | 40

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Consistent Documentation of Roadside Inspection and Violation Data-Plan The work product in the form of suggested violations list will be presented to the full committees at this meeting Next, the suggestions will go to Info Systems Committee and for review and eventual submission to FMCSA as the final authority for review and consideration as a change to ASPEN/Safetynet –How could IT support the guidance being developed? Simultaneously, the same suggestions will be submitted to the training committee for review for training needs to the roadside inspectors or data entry specialists –How could training support the guidance being developed? | 41

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Standardized Processes for Challenging Data As for the second task, standardizing the data challenge process, FMCSA has formed a working group that is currently addressing these issues. Several CVSA members are on this group The group is developing procedural guidance on the Data Q’s process –Sub-committee of this group is working on a national model for standardizing due process among jurisdictions | 42

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Uniform Inspection Selection Processes Increased awareness and understanding that all inspections (good and bad) must be uploaded Uniform inspection selection processes Clean inspections are just as valuable as inspections with multiple violations –Documents carrier improvement –Impacts resource usage and carrier safety measurement | 43

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Program Management Issues-Uniform Inspection Selection Processes Issue request has been submitted to CVSA and routed to PIC for review. The issue request suggests a change to Operational Guidance #5 Inspection Selection –Currently CVSA operational guidance in this area is to follow your jurisdiction’s policies –Amendment suggests that jurisdictions develop and implement inspection selection policies based on several principles | 44

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Program Management Issues-PIC Program-wide management of inspectors and the inspection process Examine possible metrics for measuring roadside data accuracy and uniformity Promote integrity of our processes throughout the inspection from initial contact to completion ―Inspection/Screening A final outcome is to provide operational guidance to program managers in the form of best practices | 45

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Questions? For more information please contact: Mark Savage, Mike Wilson, | 46

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 SMS: Methodology Details, Role in SFD NPRM, & Violation Severity Weightings Bryan Price Office of Enforcement & Compliance Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Safety Measurement System (SMS) Objectives SMS Methodology Details SMS Role in Safety Fitness Determination (SFD) Development of Violation Severity Weights | 48

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Entities Two measurement systems for CSA 2010: Carrier Safety Measurement System (CSMS) Driver Safety Measurement System (DSMS) Potential to add additional measurement systems in the future HM Shipper | 49

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Methodology Overview 1.Obtain on-road safety event data (e.g., inspections, crashes) and attribute to entity to create a safety event history 2.Place each entity’s violations/crashes into a BASIC 3.Convert BASIC data to quantifiable measure/rate (Safety Fitness Determination would be based on absolute performance) 4.Based on each entity’s BASIC measure, develop rank and percentile for each entity’s BASIC performance Safety Events By Entity BASIC Data BASIC Measures Percentile | 50

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Safety Events BASIC DATA SAFETY EVENTS BASIC MEASURES PERCENTILE Safety Event Data Attributed to Entity Carrier Safety Measurement System (CSMS) – Includes 24 months of carrier on road safety performance ~6.6 Million inspections ~290 K crashes ~690 K carriers Driver Safety Measurement System (DSMS) – Includes 36 months of driver on road performance ~9.6 Million inspection records ~440 K crash records ~3.6 Million drivers | 51

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 BASIC Data Safety Event Data Sorted by BASIC PERCENTILEBASIC DATA BASIC MEASURES SAFETY EVENTS Unsafe Driving (Parts 392 & 397) Fatigued Driving (HOS) (Parts 392 & 395) Driver Fitness (Parts 383 & 391) Controlled Substances /Alcohol (Part 392) Vehicle Maintenance (Parts 393 & 396) Cargo Related / HM (Parts 392, 393, 397 & HM) Crash Indicator | 52

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 BASIC Measures Convert BASIC Data into Quantifiable Measure Considerations Time Weighting / Time Frame - More recent events more relevant Severity Weightings - Increase weighting of violations that have been shown to create a greater risk of crash involvement Normalizing - Based on exposure: use of number of inspections and power units Single Inspection Cap – limit violation weight of single poor inspection Violation Cap – Cited section number only counts once per inspection BASIC MEASURES PERCENTILEBASIC DATA SAFETY EVENTS | 53

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Based on each BASIC measure, develop percentile indicating entity's BASIC performance –Provides a relative assessment of performance –Allows for prioritizing intervention resources by behavior Considerations: –Peer Grouping - compare measures of entities with similar levels of exposure –Data Sufficiency standards – define events/exposure necessary to generate a robust measure –Current Inspection and Crash Data – assignment of percentile dependent on age and result of most recent inspection (12 months) SAFETY EVENTS BASIC DATA BASIC MEASURES PERCENTILE Percentile | 54

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Peer Grouping Create percentile based on measure for carrier with similar exposure (same peer group) | 55 Peer Group BASICs  Unsafe Driving  Controlled Substances/Alcohol  Crash Indicator  Fatigued Driving (HOS)  Driver Fitness  Vehicle Maintenance  Cargo Related 10 < PU <= 55 – 10 Inspections (3-10 Fatigued) 25 < PU <= 1511 – 20 Inspections 315 < PU <= 5021 – 100 Inspections 450 < PU <= – 500 Inspections 5500 < PU501+ Inspections

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Data Sufficiency Minimum number of inspections with applicable violations required for percentile to be assigned Assists in identification of patterns of carrier behavior- note safety problems across multiple inspections | 56 BASIC Number of Inspections Unsafe Driving3 Fatigued Driving (Hours-of-Service)3 Driver Fitness5 Controlled Substances / Alcohol1 Vehicle Maintenance5 Cargo Related5 Crash Indicator2 Crashes

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 BASIC PassengerHMOther Unsafe Driving, Fatigued Driving (HOS) Crash Indicator 50%67%72% Driver Fitness Drugs / Alcohol Vehicle Maintenance Cargo Related 55%72%77% CSA 2010 OM Test Phase 2 Training Carriers’ BASIC percentile results are used to trigger carriers for interventions in Op-Model Test CSMS BASIC Thresholds | 57

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 How does a carrier move below the intervention thresholds? “Good” Inspections “Get Well” Rules –Violation time weight diminishes –Unsafe Driving and Controlled Substances /Alcohol BASICs No percentile assigned if no inspections with a violation in these BASICs in the last year –Crash Indicator No percentile assigned if no crashes in last year –Fatigued Driving (HOS), Driver Fitness, Vehicle Maintenance and Cargo Related BASICs No percentile assigned if: –No inspections with a violation in that BASIC within the past year; and –Most recent relevant inspection does not have a violation of that BASIC | 58

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Driver Safety Measurement System (DSMS) DSMS quantifies commercial motor vehicle (CMV) driver performance in terms of BASICs, using available roadside performance data During the Operational Model Test: SIs examine drivers who have been cited for severe driver violations, in conjunction with carrier interventions May result in driver Notice of Violation or Notice of Claim based on driver violation history across current and previous employers Beyond the Operational Model Test: DSMS may be used to identify the “worst of the worst” drivers so that interventions may be done directly with drivers, independent of carrier interventions | 59

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Commonly Asked Question About Driver Data When Will Carriers Have Access to Driver Data for Employment Decisions? FMSCA’s Driver Information Resource (DIR) attributes roadside inspection and crash data to individual CMV drivers “Driver Profiles” from DIR that contain inspection and crash histories for individual drivers will be made available through FMCSA’s Commercial Driver Pre-employment Screening Program (PSP) later in 2009; drivers would authorize release of profiles FMCSA is negotiating with 3 rd party vendors to provide access to PSP data for carriers and drivers | 60

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Example of SafeStat vs SMS The following slides provide examples of key differences between SafeStat and the new SMS | 61

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Carrier A Measurement: SafeStat Results | 62

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Carrier A Measurement: SMS Results | 63

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Violation Details Provided in SMS | 64

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Further Drilldown in SMS | 65

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 The following slides provide an example of why we are moving to propose through rulemaking changes to the formal safety rating process that is currently tied to the on-site compliance review Moving to a new SFD process | 66

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Carrier B Safety Rating | 67

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Carrier B Measurement: SMS Results | 68

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Limitations of Current Rating Process Current Ratings: Can only be issued or downgraded with an on-site review – resource intensive Represent a snapshot of carrier compliance at the moment of the most recent compliance review Do not consider roadside driver inspection performance Are based only on violations deemed “critical” or “acute” and vehicle out-of-service violations Generally require multiple areas of deficiency for adverse rating Only issued to small portion of carrier population | 69

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Objectives of Proposed CSA 2010 SFD Methodology Make carriers accountable for sustained unsafe operations and performance Assess larger portion of carrier population Move away from agency “seal of approval” –Carrier can continue to operate until deficiency identified, focus is on removing high risk carriers from road vs. identifying “good” carriers Maximize use of data collected by inspection program –~3 million inspections performed annually | 70

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 SFD Methodology Two major components considered in determining SFD for a carrier: 1.On Road Performance - Violations identified during roadside inspections and crash data AND 2.Intervention Results – Violations identified and data collected during investigations | 71

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 SFD – Roadside Data Role of On Road Performance 24 months of violation data used to evaluate a carrier in the following BASICs: –Unsafe Driving –Fatigued Driving (Hours-of-Service) –Driver Fitness –Vehicle Maintenance –Cargo Related (1) Crash Indicator and (2) Controlled Substances /Alcohol BASICs cannot fail based on roadside data alone Measure exceeding established “absolute” thresholds – results in failed BASIC | 72

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 SFD – Intervention Data Role of Intervention Results Essential Safety Management Violations –Tied to BASICs –Analogous to “critical” violations of current rating process –Discovery of at least 10% of the records checked would result in failed BASIC Fundamental Violations –Discovery of a single instance during an intervention would result in proposed Unfit –Largely in line with New Entrant Rule Accountable Crashes and VMT –Determined onsite during investigation by SI –Rate may result in failed Crash BASIC | 73

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 SFD Methodology Failed BASICs result from roadside and investigation data SFD methodology –Classifies BASICs as “Stand Alone” or “Non Stand Alone” according to their demonstrated relationship with carrier crash risk –Driven by the carrier’s failed BASICs Have any BASICs failed? How Many? Which One(s)? –Would result in three potential SFDs Continue to Operate Marginal Unfit | 74

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 CSA 2010 Safety Fitness Determination Methodology Being Considered | 75 SFD Methodology Being Considered Number of BASICs: Measure equals or exceeds BASIC Failure Threshold or Essential Safety Management Violations Discovered Discovery of Fundamental Violation? Resulting Proposed SFD Stand Alone BASICs: Unsafe Driving Fatigued Driving (HOS) Non Stand Alone BASICs: Driver Fitness Cargo Related Crash Indicator Controlled Substances / Alcohol 1Unfit 0 >1Unfit Marginal 000 Continue to Operate

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Example: Marginal SFD (Roadside data- driven)

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Example: Proposed Unfit SFD (Investigation-driven) | 77

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Safety Fitness Determination Today vs. CSA 2010 Methodology Adverse SFD could be issued with a single area of deficiency NTSB Recommendation: H Adverse rating generally only issued with multiple areas of deficiency 3 SFD “labels”: Unfit, Marginal, Continue to Operate 3 rating labels: Unsatisfactory, Conditional, Satisfactory SFD would be based on violations of all safety- based regulations and evaluation in 7 BASICs NTSB Recommendation: H-07-3 Rating based on violations deemed “critical and acute” and vehicle out-of-service violations from inspections Adverse SFD could be made based on roadside driver inspection performance alone Rating does not consider roadside driver inspection performance Safety fitness would be evaluated on a monthly basis Rating is a snapshot of compliance on date of compliance review SFD would change based on roadside data aloneRating only issued or changed with on-site review CSA 2010 Safety Fitness Determination (SFD) Under Consideration Existing Safety Fitness Rating Process

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Violation Severity Weighting Bryan Price Office of Enforcement & Compliance Federal Motor Carrier Safety Administration

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Background Study: Establish Association Between BASIC Violations and Crash Involvement Identified drivers with 7+ inspections, excluding post-crash inspection, over 3-year period for DIR data. Grouped drivers by crash involvement: For each crash level – –Calculated average violation rate (no severity weights) for each BASIC | 80

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Association Between BASICs And Crash Involvement | 81

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Violation Severity Overview Objective of SMS is to identify patterns of behavior assessing all violations However, all violations do not have the same relationship to crash risk Therefore, preliminary severity weights have been assigned to each violation to reflect relevance to crash risk Severity weights based on quantitative analysis supplemented with Subject Matter Expert input | 82

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Violation Severity Overview Because SMS assesses each BASIC independently, severity weights are used to differentiate crash risk for violations classified within that BASIC only ― 1 to 10 scale where 1 represents lowest crash risk ― Can NOT compare weights across different BASICs (e.g. 5 in one BASIC is not equivalent to a 5 in another BASIC) | 83

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Derivation of Preliminary Severity Weights – 6 Step Process 1.BASIC Mapping All safety-related roadside violations mapped to appropriate BASIC 2.Violation Grouping Violations in each BASIC placed in groups of like violations Allows rarely cited violations to be used in statistical analysis Helps ensure similar violations receive same severity weight | 84

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Derivation of Preliminary Severity Weights – 6 Step Process 3.Crash Occurrence Analysis Used DIR data of crash occurrences for drivers to analyze relationship between violation rates in each violation group within a BASIC, and crash involvement Applied multivariate regression: ―Showed a vast majority of violation groups yielded statistically significant relationships between high violation rates and increased crash occurrence ―Provided relative strength of the relationship between each violation group and crash occurrence within a BASIC Served as basis for preliminary violation group severity weights | 85

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Derivation of Preliminary Severity Weights – 6 Step Process 4.Crash Consequence Analysis Incorporated findings from the Violation Severity Assessment Study (VSAS) to address crash consequence (severity of outcome) 5.Subject Matter Expert (SME) Review Enforcement SMEs reviewed purely statistical based results Modifications were made based on SME input | 86

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, Carrier Safety Measurement System (CSMS) Effectiveness Test 1.Performed a simulated CSMS run that calculates carrier percentile ranks for each BASIC using historical data 2.Observed each carrier’s crash involvement over the immediate 18 months after the simulated CSMS timeframe, and 3.Calculated the relationship between the percentile ranks in each BASIC and the subsequent post-CSMS carrier crash rates. Iterative process used to optimize the ability of CSMS to identify high risk carriers Derivation of Preliminary Severity Weights – 6 Step Process | 87

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 Ongoing Review of Severity Weights in Op-Model Test Use of SMS throughout OM Test –Ongoing review and feedback on SMS design through Federal and State Working Group, Safety Investigators and SMEs involved in testing the CSA 2010 Operational Model –SMS methodology has been continuously improved based recommendations and experiences | 88

U.S. Department of Transportation Federal Motor Carrier Safety Administration CVSA Annual Conference, September 21, 2009 For more information see