8.0 Results of Feedback from FMT Technologies: U.S. Phase

8.1 Copilot ® (PERCLOS), SafeTRAC ® , and AP+ ® truck outcomes: U.S. Phase

8.1.1 Analyses of PERCLOS (from Copilot ®) during night driving ≥ 30 mph

PERCLOS (percent slow eyelid closure) obtained from the Copilot ® technology during night driving above 30 mph, was a primary outcome variable for hypothesis testing. Tables 18-21 provide the descriptive analyses of changes in four values of Copilot ® estimates of PERCLOS during night hours. Weighting factors for all outcomes analyzed are shown in Table 22 and in Table 30 (for analyses based on night driving). The doubly weighted mixed model analysis of variance found that the average median PERCLOS was reduced from 3.47 with NO FEEDBACK to 2.64 with FEEDBACK (Tables 20 and 21). The mean change in these medians was -0.83 (SE = 0.31). The null hypothesis that the average change in median PERCLOS is equal to zero was rejected with t = 2.70, df = 8, p = 0.027 (Table 21). Nearly identical results were observed when attention was restricted to nighttime driving (Table 28) and when the average value was measured by distribution medians (Tables 21 and 29) . Thus, the U.S. study phase provided evidence that the FMT feedback resulted in shifts toward lower levels of sleepiness as reflected in smaller values of PERCLOS. The use of median values, which are less influenced by skewness or outliers in the driver and condition-specific distributions, produced results suggesting decreased sleepiness with use of the FMT FEEDBACK relative to the NO FEEDBACK condition. The systematic skewness in the PERCLOS distributions detailed in Data Quality Tables 26 and 27 suggested that the median values may better reflect typical values. Therefore, analyses involving the non-parametric summaries were interpreted as the most important.

8.1.2 Analyses of "Driver Alertness" (from SafeTRAC ® ) during driving ≥ 30 mph

The second primary outcome variable used in hypothesis testing was obtained from the SafeTRAC ® technology during all driving above 30 mph. This was the SafeTRAC® output labeled "Driver Alertness" as estimated by a proprietary algorithm involving stable lane tracking. The weighted mean SafeTRAC® "driver alertness" estimate during the NO FEEDBACK and FEEDBACK conditions were 69.84 and 75.89, respectively (Table 19). The weighted estimate of the mean increase from the NO FEEDBACK to FEEDBACK change in SafeTRAC alertness was 6.06 (SE = 2.30). The increase in mean alertness was statistically significant (t = 2.63, p = 0.030). Nearly identical results were observed when attention was restricted to nighttime driving (Table 28) and when the average value was measured by distribution medians (Tables 21 and 29). Specifically, the nighttime driving weighted median values during the NO FEEDBACK and FEEDBACK conditions were 71.36 and 77.27, respectively (Table 29). The weighted estimate of the mean increase in median SafeTRAC alertness from the nighttime NO FEEDBACK to nighttime FMT FEEDBACK conditions was 5.91 (SE = 2.21). This increase in the mean median value of alertness was statistically significant (t = 2.67, p = 0.028). Considering both the mean and median methods of estimation, unweighted and weighted, all driving or only nighttime driving, virtually all SafeTRAC results reflected a statistically significant increase in driver alertness due to the FEEDBACK intervention. Therefore, the positive effects of FMT FEEDBACK on estimates of driver alertness using the SafeTRAC® indication of "alertness" were consistent with the positive effects of FMT FEEDBACK on Copilot® indicators of reduced sleepiness (PERCLOS) during night driving in the U.S. study phase.

8.1.3 Analyses of Lane Tracking Variability (from SafeTRAC ® ) during driving ≥ 30 mph

The third primary outcome measure used in hypothesis testing was Lane Tracking Variability obtained from the SafeTRAC ® technology during all driving above 30 mph . The crude lane tracking measure was converted into lateral distance. Two measures of variability in lateral distance were examined, the standard deviation (SD; Tables 18 and 19) and the interquartile range (IQR; Tables 20 and 21). The lateral distance interquartile range decreased by about 20% from 47.99 to 38.40 (Table 21). In mixed model weighted analyses, the estimated mean change in the lateral distance IQR was -9.59 (SE = 6.25). This difference did not reach statistical significance (t = -1.53, p = 0.164) for all driving time. The average unweighted IQR (Table 28) during the NO FEEDBACK and FEEDBACK conditions were 45.78 and 38.22, respectively. The mean change (SD) in the IQR values was -7.56 (9.37) with minimum and maximum values of -30.0 and 2.0, respectively. The descriptive t-test suggested a statistically significant feedback effect (p = 0.042). Based on the unweighted data, the standardized effect size was ES = -7.56/9.37 = 0.81. A sample size of n = 15 is necessary to achieve at least 80% power to reject the null hypothesis that the mean difference in the lateral distance IDQ is equal to 0 for effect sizes of 0.81 or greater. The results from weighted mixed model analyses restricted to nighttime driving (Table 29) were similar, but showed only a trend (p = 0.083). Although only a statistical trend, lane tracking variability improved with FMT FEEDBACK during night driving in the U.S. study phase, consistent with the effects observed of PERCLOS and SafeTRAC estimates of drivers' sleepiness and alertness, respectively.

8.1.4 Analyses of Steering Wheel and Front Wheel Movement Variability (from AP+ ® ) during driving ≥ 30 mph

A fourth class of outcomes also evaluated relative to the primary hypothesis were s teering wheel mean variability and front wheel movement variability obtained from the AP+ ® system during all driving above 30 mph. As Tables 18-21 indicate, there were only 7 drivers evaluated for these variables after data cleaning-as with the Canada study phase, we suspect that problems with steering sensors contributed to loss of reliable steering and front wheel movement data in the U.S. study phase. As Table 19 reveals, significant increases in the mean standard deviation (SD) were observed for both steering wheel movements (weighted mixed model p = 0.001) and front wheel movements (p = 0.008). Very similar results emerged when analyses were restricted to night driving. However, as Table 21 reveals, no significant differences were observed for the IQR measures (p = 0.553 and p = 0.352, respectively, for steering wheel and front wheel movements). In fact, the results for front wheel movements changed direction with a smaller mean IQR observed with FEEDBACK (Table 21). Upon inspection of the distribution details (Data Quality Tables 19-23 in Appendix C-1), there were extreme outliers in 3 instances (1 case in the NO FEEDBACK and 2 cases in the FEEDBACK condition) which likely influenced the results. Consequently, given the inconsistency between the SD and the IQR results, it appears that no definitive conclusions can be made in the U.S. study phase on the basis of changes in steering wheel and front wheel movement variability.

8.1.5 Analyses of Truck Motion Variables (from AP+ ® ) during driving ≥ 30 mph

For completeness, the other AP+ ® vehicle parameters were subjected to the same analyses. These included the truck motion variables (vehicle speed, engine rotation, longitudinal acceleration [X], lateral acceleration [Y]), and ambient light. These variables were not a priori hypothesized to be different between NO FEEDBACK and FEEDBACK conditions. This was the case for all these variables in both the Canada and U.S. study phases ( Tables 18-21) .

8.2 Comparison of U.S. Phase and Canada Phase for primary driving outcomes relative to FMT FEEDBACK

Tables 33 to 47 were constructed to facilitate general comparisons between the results from the Canadian and U.S. study phases relative to the effects of FMT FEEDBACK versus NO FEEDBACK. These Tables have the same structure as the primary analysis Tables, reflecting each of the four components of the data analyses hierarchy: (1) unweighted analysis of parametric distribution, (2) doubly weighted mixed model analysis of parametric distribution, (3) unweighted analysis of non-parametric distribution, and (4) doubly weighted mixed model analysis of non-parametric distribution. There are sets of Tables for all driving (Tables 33 to 40) and driving during the nighttime (Tables 41 to 48). Even-numbered Tables show the results for each study phase (e.g., Table 33 contains results from Tables 2 and 18), as well as comparisons between the p values for NO FEEDBACK vs FEEDBACK comparisons within each study phase (at the bottom of the Tables). Odd-numbered Tables display the NO FEEDBACK vs FEEDBACK comparisons for each of the primary driving outcomes for the two study phases combined (i.e., n = 18 to 27).

Composite Table 47 and Table 48 summarize the primary results of NO FEEDBACK versus FEEDBACK comparisons for the U.S. and Canada study phases during night driving, using non-parametric distributional summaries (medians and interquartile ranges). This is the most succinct summary of study results because in most cases the results were very similar to those obtained using parametric distribution summaries (means and standard deviations) and in cases where there were qualitative differences, the non-parametric results made more sense, and the doubly weighted, mixed model analysis of variance was considered the most definitive for comparisons. The non-parametric distributional summaries are used (medians and interquartile ranges) because in most cases the results were very similar to those obtained using parametric distribution summaries (means and standard deviations), and in cases where there were qualitative differences, the non-parametric results made more sense given high inter-subject variability. Tables 47 and 48 are reprinted for ease of viewing.

As can be seen in Table 47, in both the U.S. and Canada study phases, the average median values of Copilot® recordings of PERCLOS decreased, the average median values of the SafeTRAC® "alertness" score increased, and the average value of SafeTRAC® measure of lane tracking variability (lateral distance interquartile ranges) decreased during nighttime driving in the FMT FEEDBACK condition relative to the NO FEEDBACK (control) condition. In the U.S. study phase-where the vast majority of driving was at night-these changes were statistically significant for Copilot® PERCLOS and SafeTRAC Alertness measures, and marginally significant for the SafeTRAC® measure of lane tracking variability. However, the changes were not statistically reliable in the Canada study phase.

Table 47: PERCLOS, SafeTRAC, and AP+ Outcomes during Night Driving
Mixed Model ANOVA Comparisons Based on Doubly Weighted‡ Medians and IQR's

Although statistical power is generally low for tests of interaction, the differences between study phases in the changes in median PERCLOS were not statistically significant (see bottom of Table 47). Similarly, study differences between the mean changes in median

SafeTRAC alertness was also not significant. Mean reductions in the SafeTRAC® measure of lane tracking variability were statistically significantly larger in the U.S. study phase compared to the Canada study phase (-6.97 vs. -1.25, p = 0.039). Comparisons between studies in steering wheel movements and front wheel movements were not meaningful because of differences in the scales of the metrics.

Table 48 summarizes composite results for night driving from pooling data from the two study phases. This is presented for descriptive purposes only. It is noteworthy that when data are combined from the two study phases, FMT FEEDBACK had the following positive effects relative to NO FEEDBACK: Reduction in slow eyelid closures as measured by Copilot® scores for PERCLOS (t = -3.24, p =0.004); increased alertness as measured by SafeTRAC® scores for "driver alertness" (t = 3.49, p =0.002); and reduced variability in lane tracking as measured by SafeTRAC® (t = -2.96, p =0.007). The convergence of these measures showing improvements in fatigue indices during nighttime driving with FMT Feedback is striking. It suggests that with a focus on night driving, when fatigue and sleepiness would be expected to have a higher probability of occurrence, a larger sample size would reveal that FEEDBACK from fatigue management technologies (Copilot, SleepWatch, SafeTRAC, HPCS) has the potential to reduce slow eyelid closures (PERCLOS), increase alertness (as measured by lane tracking), and decrease lane tracking variability in experienced truck drivers.

8.3 Psychomotor Vigilance Task (PVT-192) performance outcomes: U.S. Phase

As in the Canada study phase, U.S. study phase drivers were provided with a portable psychomotor vigilance task (PVT-192) test device while on the road, to provide information on their behavioral alertness as assessed by reaction-time (RT) based vigilance performance at the midpoint and end of each driving workday. It was hypothesized that relative to the NO FEEDBACK condition, FMT FEEDBACK would reduce PVT performance lapses, improve median RT performance, and reduce subjective sleepiness (as measured by a visual analog scale [VAS] drivers completed at the end of each PVT task trial).

8.3.1 PVT-192 performance variables

PVT data were available for all 12 drivers in the U.S. study phase. The total number of PVT trials in the NO FEEDBACK condition during the day, evening, and night times-of-day were 37, 12, and 178, respectively. Similarly, in the FEEDBACK condition, there were 32, 3, and 174 total trials during the day, evening, and night time intervals. Table 23 displays PVT variables and the results of comparisons between FEEDBACK and NO FEEDBACK conditions (see Appendix D-2 for complete PVT results). Consistent with the high amount of night driving in the U.S. study phase, the vast majority of PVT trials were conducted at night (352 night trials vs. 84 day and evening trials for all 12 drivers).

Table 48: AP+ and SafeTRAC Outcomes at Night
Mixed Model ANOVA Comparisons Based on Doubly-Weighted Medians or IQR's

As in the Canada study phase, the total number of PVT lapses (RT ≥ 500ms), median response time, and subjective sleepiness by VAS (post-PVT trial) were the primary PVT outcome variables. The remaining variables were analyzed as secondary outcome variables. Although the originally planned mixed modeling approach examined the statistically significance of time-of-day interaction on FEEDBACK condition effects, the very few numbers of non-night trials prevented meaningful analyses in this regard. Therefore, interpretations of the main effects of the FEEDBACK condition and analyses restricted to nighttime trials were emphasized in the U.S. study phase.

8.3.1.1 Mixed model analyses of PVT-192 responses: Raw lapses (RT ³ 500 ms)

The interaction model (i.e., feedback condition, time-of-day, time-of-day by feedback condition) was used to compute an adjusted intraclass correlation (ICC). The ICC is the proportion of total variance explained by systematic differences among drivers after accounting for time-of-day and feedback condition effects. Assessment of ICC was taken as a quality control procedure and to document the ability of this study to obtain reliable PVT performance assessments in the field.

The intraclass correlation for PVT raw lapses in the U.S. study phase was 0.303 (p = 0.014), indicating that 30% of the variance among the number of vigilance lapses was attributable to systematic differences among drivers after accounting for time-of-day effects and FMT condition effects. Consequently, multiple testing reliability, although statistically significant, was somewhat smaller than the ICC = 0.473 value found in the Canadian study phase. During night trials, the model predicted more lapses in the FEEDBACK condition compared to the NO FEEDBACK condition (3.12 vs. 4.59; t = 2.83, df = 11, p = 0.016). There was an increase in total PVT performance lapses per trial under the FEEDBACK condition relative to NO FEEDBACK condition.

For comparison purposes, in the Canadian study, the expected numbers of lapses per trial were 2.54 and 2.34 during the nighttime with NO FEEDBACK and FEEDBACK, respectively (p=0.332). The U.S. study finding of an increase contradicts the small non-significant decrease previously observed. On the other hand, in the Canadian study, PVT lapses were significantly elevated in the daytime and evening in the FEEDBACK condition (relative to the NO FEEDBACK condition). It appears therefore that PVT lapses were elevated in each study in the FEEDBACK condition, relative to the NO FEEDBACK (control) conditions, and the increase occurred during the portion of the 24-hr day in which drivers most often were driving (i.e., daytime for the Canada drivers, and nighttime for the U.S. drivers). Increased PVT lapses are a sign of reduced behavioral alertness. While the increases engendered in both study phases are not large, they are statistically reliable, and must be explained. Possible reasons for why PVT lapses were increased in the FMT FEEDBACK conditions in both study phases are not clear, but the findings suggest that there may be a fatigue-related "cost" to the added effort (in attention and compensatory behaviors) required to respond to the feedback from the FMT devices, and that effort may manifest itself when performing a demanding vigilance-based reaction time task while not driving. Other explanations are also possible (see below).

8.3.1.2 Mixed model analyses of PVT-192 responses: Median reaction times

The ICC for PVT median response time was even smaller than or lapses (ICC = 0.246; p = 0.015) in the U.S. study phase compared to the value observed in the Canadian study phase (ICC = 0.709). It is unclear as to the cause of this apparent reduced reproducibility. As shown in Table 23, similar to total lapses, during night PVT trials in the U.S. study phase, the model predicted median PVT response time was higher in the FEEDBACK condition compared to the NO FEEDBACK condition (243 ms vs. 258 ms; t = 5.14, df = 11, p < 0.0001). In the Canada study phase, PVT median values during the nighttime were 256 ms and 255 ms with NO FEEDBACK and FEEDBACK, respectively (p = 0.851). Again, however, as with PVT lapse frequency (above), the significant increase of nighttime median reaction times on the PVT found in the FEEDBACK phase of the U.S. study phase-while modest in size-were consistent with the statistically significant increases in PVT median reaction times in the FEEDBACK phase of the Canada study phase (in the day and evening driving conditions).

Since FEEDBACK was also associated with elevations in PVT lapses (long reaction times) in both studies, the concurrence on reliable increases in median RTs further supports a "cost" to behavioral alertness associated with the FEEDBACK condition. An alternative explanation is that the small but statistically significant changes observed in PVT performance in both study phases (during the most common driving time-of-day for each) may reflect a "letting down" phenomena, in which drivers reduced their motivation on the PVT task ever so slightly with the view that they were in the final weeks/days of the study and did not need to try as hard. Since the FEEDBACK and NO FEEDBACK conditions were deliberately not counterbalanced to prevent drivers from using FMT devices (had the NO FEEDBACK condition followed the FEEDBACK condition), we cannot be certain that "motivation" wasn't the factor producing the PVT results. On the other hand, if the PVT changes found during FEEDBACK in both study phases were due to motivation, one would not expect to find drivers ratings to be elevated in the FEEDBACK condition relative to the NO FEEDBACK condition. The next section presents these results.

8.3.1.3 Mixed model analyses of PVT-192 responses: Post-PVT Sleepiness Rating

Table 23 reveals that the intraclass correlations for the U.S. study phase post-PVT sleepiness ratings (by visual analog scale) were larger than for PVT lapses and median response time (ICC = 0.429; p = 0.012), and in this case, larger than the value observed in the Canadian study phase (ICC=0.289). During the nighttime PVT tests in the U.S. study phase, the expected subjective sleepiness was significantly lower during the NO FEEDBACK condition compared to the FEEDBACK condition (3.29 vs. 5.33; t = 6.63, df = 11, p < 0.0001). This result is in the opposite direction to that found in the Canadian study phase where the expected values at night were 7.56 for the NO FEEDBACK condition and 6.18 for the FEEDBACK condition (p = 0.009, Table 12). In the U.S. data, the finding of increased subjective sleepiness observed for the post-PVT test sleepiness VAS rating was also observed for the pre-PVT subjective sleepiness rating (p < 0.0001, see bottom of Table 23). Thus, in terms of subjective sleepiness, the FEEDBACK condition appeared to increase drivers' self-rated levels of sleepiness at night in the U.S. study phase. While this finding is contrary to that found in the Canadian study phase, it is fully consistent with the U.S. study phase results for PVT lapses and median reaction times, as well as other PVT parameters (fastest 10% RTs, slowest reciprocal RTs-see Table 23). This convergence of subjective and objective PVT results suggesting greater fatigue during the nighttime PVT test bouts in the FEEDBACK condition of the U.S. study phase makes it unlikely that driver motivation accounted for the results. Instead, it lends further support to the possibility that FMT FEEDBACK in drivers who operate primarily at night may have alertness-promoting benefits during driving (Table 21), but such feedback may also create a modest "cost" to the added effort (in attention and compensatory behaviors) required to respond to the feedback from the FMT devices. That "cost" may manifest itself as slightly worse performance and greater subjective sleepiness when performing a demanding vigilance-based reaction time task such as the PVT (while not driving).

8.3.1.4 Mixed model analyses of PVT-192 responses: Secondary PVT outcomes

Results for the secondary PVT outcomes are also summarized in Table 23. As noted above, these other PVT performance variables supported the findings of the primary PVT outcomes (lapses, median RT and post-PVT sleepiness). During the night driving schedule, U.S. study phase drivers performed less well on the PVT in the FEEDBACK condition relative to the NO FEEDBACK condition. Thus, the results from PVT testing consistently showed worse performance under the FEEDBACK condition in the U.S. study phase, which was the opposite of the findings of reduced sleepiness suggested by the reduced median PERCLOS, increased mean SafeTRAC Alertness, and improved driving performance suggested by the reductions in mean lateral distance IQR.

8.4 SleepWatch ® ( Actigraphy) and Sleep Management Model ou tcomes: U.S. Phase

Mixed model ANOVA comparisons of actigraphy variables between the NO FEEDBACK condition and FEEDBACK condition are summarized in Table 24 from the U.S. study phase. Random effects including intraclass correlations are summarized in Table 25. ICC values adjusted for feedback condition were relatively large and statistically significant for almost all actigraphy outcomes demonstrating consistency within driver over days. Table 24 reveals that there was a significant increase in the number of sleep episodes in the FEEDBACK condition relative to the NO FEEDBACK. The mean per day value increased from 1.87 in the NO FEEDBACK condition to 2.11 in the FEEDBACK condition (p = 0.045).

8.5 Daily diary outcomes: U.S. Phase

As in the Canada study phase, U.S. drivers were provided a daily diary (Appendix B-1) to record driving conditions (weather, slow traffic, hilly roads, crosswinds, waiting); work activities (loading and unloading, deliveries, etc.); rest breaks and naps; days off; reactions to FMT devices; and day and night activities (work, rest, and sleep). Diary Tables 1 to 25 in Appendix E-2 provide per driver quantitative summaries of the diary data for 10 U.S. drivers. This subset of drivers did not coincide exactly with that of the "cleaned analysis samples" of the AP+ and PVT analyses above. Two drivers were excluded from these analyses because their diaries were not accurately maintained.

There were three types of daily diary variables summarized. Data were tabulated a number of ways, according to type of variable. The first was the proportion of days in which at least one event of a specific type was reported (e.g., long delays for traffic). Proportions were summarized by FMT condition (FEEDBACK vs. NO FEEDBACK). The second type of variable was the number of events per day. The descriptive diary Tables summarize the distributions over days for each driver separately for the NO FEEDBACK and FEEDBACK conditions. The third type of variable was the cumulative duration for the events summarized by frequency per day. These are also summarized in Tables in Appendix E-2.

Tables 31 and 32 show the results of descriptive analyses comparing the NO FEEDBACK condition to the FEEDBACK condition for the mean and median cumulative duration variables (Table 31) and for the mean and median frequency per day variables (Table 32). No systematic differences between conditions were found (see also Table 17). Although there were no formal statistical analyses performed to assess differences between the Canadian and U.S. study phases, it was obvious that the sample of drivers from each study phase had different work activities and driving chores-there were generally more events reported by the Canadian drivers because they had more loading chores. However, in terms of lack of diary differences between the NO FEEDBACK and FEEDBACK conditions, the U.S. study phase results agree with those from the Canada study phase. Thus, there was no evidence from drivers' daily diaries to support the hypothesis that FMT FEEDBACK resulted in increased sleep time relative to NO FEEDBACK. But diary measures appeared to be too limited in scope to effectively discern the differences found between workdays and non-workdays in sleep duration, and the effects of FEEDBACK on non-workday sleep durations based on actigraphy (see Section 9).

Foot Notes

† IQR - interquartile range (75 th percentile minus 25 th percentile)

‡ Doubly weighted-In these analyses, the statistics used to summarize sleepiness and performance for each driver within each experimental condition (No Feedback and Feedback) were made to optimally reflect 'typical' performance by weighting observed values proportionally to their observed record duration time by replicating records prior to computing statistics from the driver/condition specific distributions. These summary statistics included mean, median, standard deviation, and interquartile range values. Then, inference to the population with regard to differences between the no feedback and feedback conditions were made optimally efficient by giving greater weight to the above summary statistics that were estimated on the basis of larger cumulative AP+ recording times.

^ From mixed model with factor for country added.