5.7  Spectrograms using data containing behavioral transitions

Now comes the fun part.  By combining the additional information obtained from the high sampling rate and high filter cutoff, with noise removal, filter compensation, scaling, image processing, and the tight behavioral correlation obtained from the experiment, we are able to view what may be a newly discovered phenomenon.

Now that we are familiar with spectrograms derived from data during test periods devoid of errors of omission due to drowsiness, we will continue analysis by examining data segments containing failures due to excessive drowsiness and possible sleep.  Data for the next set of spectrograms was taken from tests taken during the "drowsy" test conditions (sleep deprived) and represent typical findings observed in all test subject's who experienced failures on their visual test.

The first example shown in Figure 5.7.1L (cia01) was created from a 9.7 minute data segment which contains a transition from good test performance to failure (inability to respond to the visual signal OR failure to perform the test by eye closure which we can attribute to fatigue since the test subject knows that this should not be done intentionally).  We will examine this first data segment in more detail than the others since it illustrates various aspects of test performance and the power shifts with characteristics that are common to all of the examples that follow.

Figure 5.7.1L  Low frequency - behavioral transition

The vertical marks above the plot (Figure 5.7.1L) mark the time of appearance of the visual stimulus events during testing.  If we examine the marks, we are able to see how the control computer interacted according to changes in the subject's performance (simple feedback).  The first two events of this segment were hits as indicated by the solid lines.  The third event appeared 57.04 seconds (random appearance mode) after the last hit and this event was missed by the subject (dashed line).  The computer then switched to continuous appearance mode which was triggered by the missed event and produced events every 8.01 seconds from the time the previous event was illuminated until two consecutive responses were received.  The next two events were hits (solid lines).

Video analysis reveals the subject looking fatigued and having their eyes closed momentarily during the third event (first missed event) of the segment.  The subjects eyes then re-opened without prompting and the next two signals were hit.

The computer switched back to random mode with the next event appearing after 71.05 seconds.  Since this event was also a hit, another event was presented after 29.02 seconds and was missed, thereby causing the computer to switch back to generating consecutive events with uniform spacing.

Video analysis shows the subject looking extremely tired during this event with their eyes only partially open during the seventh event (which is the second missed signal in this sequence).

Following this missed event, the subject misses a long string of events (dashed lines) until awakened by the test operator by speaking to the subject over the intercom.  After operator intervention, the subject immediately began responding to the signals and the computer switched back to random appearance mode for the remainder of the segment.  The Appendix contains an excerpt from the automated performance report (with some additional comments) that corresponds to this data segment.

The low frequency spectrogram (Figure 5.7.1L) shows a very clear band of alpha activity throughout most of the test.  During the portion of the test where the subject continuously failed to respond to the signals, the band of alpha activity lost intensity and activity in the lower frequency theta band increased in energy which indicates possible entry into stage 1 sleep.

As expected from previous analysis using the individually selected PSD plots, the high frequency spectrogram reveals noticeable changes in activity.  It can be seen in Figure 5.7.1H that high frequency power drops in the region of data that corresponds to the string of missed events.  The high frequency region shows a relatively steady reduction in power toward lower frequencies over time (as indicated by the appearance of the darker blue and red colors).  After the subject is awakened/alerted, activity immediately returns to normal levels and the red color is replaced by the typical yellow and blue colors which indicate normal levels of alertness.

Figure 5.7.1H  High Frequency - behavioral transition

Because most of the pixels in the high frequency image (Figure 5.7.1H) reside in a small subrange of values (blue), a histogram equalization routine can be used to spread the pixel distribution so that each of the pixel values contains an approximately equal number of members.  The procedure is to first use a histogram function to generate the density distribution of the image.  This distribution is then integrated to obtain the cumulative distribution function.  Finally, the function is normalized so that its maximum element has a value of 255.  Care needs to be taken when using this procedure because it can obscure detailed features in the image which only occupy a small number of pixels.  Because our interest is in the relative power shifts over a large range, this procedure can be used to enhance the visual information content of the image.  Figure 5.7.2H is generated by using the histogram equalization procedure on Figure 5.7.1H.

Figure 5.7.2H  High frequency - enhanced

The histogram equalization routine enhances the power intensity changes over time measured relative to the power in this region of the spectrogram.

Consistent with the averaged PSD plots analyzed previously, the power in the high frequency spectrum decreases as frequency increases (white/yellow across bottom changing to blue and red and frequency increases).  It is evident, especially with enhancement, that during failures, power shifts from the highest frequencies downward as the total time spent in the failure increases.  We can infer that the temporal nature of the power change is due to the increasing depth of sleep/sleepiness once a failure has begun.  Certainly if the failure were allowed to proceed without operator intervention, we would expect the subject to experience the various stages of sleep (S1, S2, etc.) over time.  In order to take the fullest advantage of the testing opportunity, the subjects were not allowed to remain asleep.  Sleep information is gathered during the sleep phase after the test.  We can observe that once the failure episode has finished and the subject begins performing the test adequately, the power and energy distribution levels return back to normal levels.

Although Figure 5.7.1H is representative of the changes in the high frequency band which occur during extended failures, three additional examples have been included in this section.  For all examples, the low frequency, high frequency, and enhanced high frequency spectrograms have been included.  Only a few words will be included for each example shown.

The example shown in Figures 5.7.3L, 5.7.3H, and 5.7.4H (cvj01) have changes in low and high frequency similar to those just described.  We can observe that in this example, the most dramatic changes appear after the subject finally stops responding to all signals.  When the subject misses lone events, the changes are not as discernable on these plots.  But, the overall trends are extremely obvious when viewed under these conditions.


Figure 5.7.3L  Low frequency - behavioral transition

Figure 5.7.3H  High frequency - behavioral transition

 

Figure 5.7.4H  High frequency - enhanced

The next example is shown in Figures 5.7.5L, 5.7.5H, and 5.7.6H (cvj01).  Again, the transitions in both the low and high frequencies are very consistent with the previous examples.  Although in this example, we can see that the power shift occurs much more rapidly (faster transition) in the high frequency band than in the previous examples.

Figure 5.7.5L  Low frequency - behavioral transition

Figure 5.7.5H  High frequency - behavioral transition

Figure 5.7.6H  High frequency - enhanced

Finally, the final example is shown in Figures 5.7.7L, 5.7.7H, and 5.7.8 (cia01).  Very slight changes are observable in the high frequency band when there is only a small failure, i.e. the subject responds spontaneously after only a few missed events.  When the subject experiences a larger (extended number) failure , the shifts in the high frequency band become much more noticeable.  This plot, as well as those given previously, shows noticeable but small changes that occur in the high frequency band that accompanying "smaller" failures, where changes in the low frequency range are not perceptible.  This strengthens our assertion that the high frequency components may be more suitable for detection of "smaller" transient failures and thereby good indicators of incipient drowsiness.

Figure 5.7.7L  Low frequency - behavioral transition

Figure 5.7.7H  High frequency - behavioral transition

Figure 5.7.8H  High frequency - enhanced

<PREVIOUS

NEXT>