The FAST Act required the National Research Council of the National Academies of Science (NAS) to conduct a study of the Compliance, Safety, Accountability (CSA) program, specifically the Safety Measurement System (SMS).
The FAST Act outlined several elements for NAS to analyze and/or consider, including:
- The accuracy with which the Behavior Analysis and Safety Improvement Categories (BASIC) identify high risk carriers; and predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers, including the high risk carriers;
- The methodology used to calculate BASIC percentiles and identify carriers for enforcement, including the weights assigned to particular violations and the tie between crash risk and specific regulatory violations, with respect to accurately identifying and predicting future crash risk for motor carriers;
- The relative value of inspection information and roadside enforcement data;
- Any data collection gaps or data sufficiency problems that may exist and the impact of those gaps and problems on the efficacy of the CSA program;
- The accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault;
- Whether BASIC percentiles for motor carriers of passengers should be calculated separately from motor carriers of freight;
- The difference in the rates at which safety violations are reported to FMCSA for inclusion in the SMS by various enforcement authorities, including States, territories, and Federal inspectors;
- How members of the public use the SMS and what effect making the SMS information public has had on reducing crashes and eliminating unsafe motor carriers from the industry;
- Whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers;
- Whether alternatives to the SMS would identify high risk carriers more accurately; and
- The recommendations and findings of the Comptroller General of the United States and the Inspector General of the Department and independent review team reports, issued before the date of enactment of the FAST Act.
The NAS report included six recommendations for FMCSA’s consideration and provided specific information to be considered in some of the recommendations. Generally, the recommendations are:
- FMCSA should investigate a new statistical model, within the existing structure of SMS, over the next 2 years. If the model is then demonstrated to perform well in identifying motor carriers that engage in unsafe practices, FMCSA should implement it.
- FMCSA should continue to collaborate with states and other agencies to improve the quality of Motor Carrier Management Information System (MCMIS) data in support of SMS, focusing on carrier exposure and crash data.
- FMCSA should investigate ways of collecting data that could benefit the recommended methodology for safety assessment. This includes data on carrier characteristics—including information on driver turnover rate, type of cargo, method and level of compensation, and better information on exposure.
- FMCSA should structure a user-friendly version of the data file used as input to SMS without any personally identifiable information to facilitate its use by external parties, such as researchers, and by carriers. In addition, FMCSA should make user-friendly computer code used to compute SMS elements available to individuals in accordance with reproducibility and transparency guidelines.
- FMCSA should undertake a study to better understand the statistical operating characteristics of the percentile ranks to support decisions regarding the usability of public percentiles.
- Given that there are good reasons for both an absolute and a relative metric on safety performance, FMCSA should decide on the carriers that receive SMS alerts using both the SMS relative percentile ranks and the SMS absolute measures, and the percentile ranks should be computed both conditionally within safety event groups and over all motor carriers.
NAS posted a copy of the Correlation Study.
For further information about the SMS Correlation Study, see the National Academies of Sciences.
In accordance with the FAST Act, FMCSA will complete a review of the report and address its recommendations. A response will be submitted to Congress and the Office of the Inspector General (OIG) within 120 days of the report being provided to Congress. Copies of the letters to Congress and the OIG will be posted to this website upon completion.
Public Availability of SMS Alerts and Percentiles
As required by Section 5223 of the FAST Act, FMCSA has removed alerts and relative percentiles for property carriers from the public display of the SMS. FMCSA is prohibited from publishing this information until the SMS Correlation Study is complete, and all reporting requirements and certification requirements under the FAST Act are satisfied. The delivery of the NAS report does not change the public display of the SMS.
A report to Congress containing study findings; study findings should also be published on a publicly accessible Web site. - Complete
February 2016: Contract awarded - Complete
June 2017: Report to Congress; report published on publicly accessible Web site - Complete
October 2017: Response to recommendations transmitted to Congress and the OIG
FY 2016: $971,519
Project is on schedule.
For information about the Correlation Study, please contact Ms. Theresa Rowlett, Senior Policy Advisor, Office of Enforcement, FMCSA, 1200 New Jersey Avenue, S.E., Washington, DC 20590, Telephone (202)360-2924 or by email at Theresa.Rowlett@dot.gov.
National Research Council of the National Academy of Sciences