5.6  Spectrograms

In Section 5.5, the data segments used to compare hits and misses were adequate for gaining an initial understanding of the changes which take place between hit and missed events under conditions of extreme drowsiness and in the identification of a new frequency range that appears to be potentially useful in drowsiness detection.  Unfortunately, the events previously examined were temporally-disconnected and only represent the extreme end-points of the drowsiness spectrum encountered during this visual test (according to the data-inclusion criteria set forth in section 5.5).

In order to gain a broader understanding of the nature of transitions from hits to misses, spectrograms will now be employed.  When referring to the nature of transitions, we are particularly interested in the sensitivity of various frequency bands, whether energy shifts are smooth or transition rapidly (jumps), whether the energy shifts behave linearly or nonlinearly, and how different frequency bands compare according to these criteria.  Spectrograms provide a convenient computational framework for generating various frequency bands and for gaining an improved visual understanding of the phenomena involved.

Briefly, spectrograms (also called waterfall plots) are a sequence of power spectral density plots (PSD's).  The PSD's are computed using the windowed Fast-Fourier Transform as before by taking into account that the EEG is a non-stationary signal.  PSD's can be computed on contiguous non-overlapping segments of data or on overlapping data segments with the overlap percentage specified.  The PSD's are typically arranged as three-dimensional plots with time across the x-axis, frequency across the y-axis, and spectral amplitude (power) across the z-axis.

For the spectrograms displayed here, two-dimensional raster projections with the z-axis represented by changes in color intensity according to a predefined scale are used in place of the typical three-dimensional plots.  The 2-D plots tend to be easier to read and interpret on complex datasets and allows us to treat the spectrogram as an image and use image processing techniques for visual enhancement.

As an example, a test signal has been generated which is the pointwise sum of two frequency modulated sine waves of 65536 points each.  The first sine wave increases from 0 to 60Hz while the second sinusoid increases from 0 to 33Hz.  Figure 5.6.1 shows two views of a 3-dimensional waterfall plot from this example signal which uses 2048-point data segments with a 50% (1024pt) overlap rate.  Changes in the viewing perspective allow us to distinguish different features.  For example, the top plot is easier to distinguish the x- and y-coordinate values, whereas the bottom plot gives improved z-value interpretation.

Figure 5.6.1  Examples of Standard 3-D Waterfall Plots

Figure 5.6.2 Spectrogram and color table displayed as 2-D raster image
of frequency modulated test signals

In general, the 2-dimensional raster images, although not always as visually attractive as the 3-dimensional plots, tend to be easier to read and interpret on the datasets being analyzed especially because of the large fluctuations in power at various scales.  From these pictures, exact z-axis information can be extracted on the workstation interactively using a mouse to place a cursor over the area of interest to read the numerical intensity.

5.6.1  Spectrograms of Data

By using spectrograms, we are now able to examine larger temporally-continuous datasets in more detail than before.  Because an extremely large body of literature currently exists detailing those changes which occur in the 0-30Hz frequency range, this section will focus on those changes which occur outside of this range. Information on the 0-30Hz band is shown for comparison purposes.  By creating raster images from the spectrogram data, we are able to overlay various color tables to represent intensity such as the example shown in Figure 5.6.2(a).  Choosing the appropriate color table can help clearly highlight those features being emphasized.

The color palette used for the remaining plots in this section is shown next in
Figure 5.6.3.

General info. about the spectrograms shown in this section

The individual PSD plots are arranged vertically with frequency across the y-axis and time across the x-axis (time bins).  For the spectrogram examples in this section, time bins of 2048 points each were sequentially transformed with a 50% data overlap (1024 points).  A Hanning windowing function was applied along with a standard double-precision FFT algorithm.

After spectral transformation, all 60Hz and harmonic components were removed by the methods described previously (Section 5.5).  The spectral data is then transformed with a natural log function to close the scale of the data due to the presence of multiple order-of-magnitude differences in power from the low frequencies to the high frequencies.

To improve the print quality of the printed plots, a 3-point smoothing function (boxcar) was applied to the image to selectively average the elements within a moving window (horizontally across time).  The window traverses the input image element by element from the upper left to the lower right.  Smoothing lowers the spatial frequency components in the image (obscures trends), and can soften sharp transitions from one color to another in the color table.

Finally the data is re-gridded to expand the number of elements in the image by interpolating values at specified intervals using nearest neighbor interpolation in both the x and y-directions, intensity values are scaled between 0 and 255 for the color table mapping, and transformed into a Sun raster image for display.

First spectrogram example

At this point we can examine our first extracted data segment to gain a qualitative understanding of the data.  A quantitative interpretation of the data will be explored in chapter 6.

The first data segment to be examined is shown in Figure 5.6.4 (kal01) which represents a typical data segment (9.4 minute duration) from the occipital EEG channel of a test subject performing the visual test.  Because all of the event markers above the spectrogram are solid lines, we know that the subject is performing the test normally with no missed events.

Figure 5.6.4  Full spectrum 0-475Hz

In this figure we can view the entire spectrum from 0 to 475Hz (Nyquist).  As we expect, the highest power occurs at the low frequencies (white and yellow) and the power decreases sharply (roughly proportional to 1/f) as we increase in frequency (blue to red to black).  Because we would like to further explore energy shifts in the high frequency range (beyond 30-40Hz), the fact that the power in the high frequency band is already at the bottom of the intensity scale will hinder our ability to take a detailed look at its behavior.

Another interesting point which has not yet been explored comes out of the previous analysis using individual PSD plots along with examining the frequency response of the anti-aliasing filters used in this experiment: the attenuation of the high frequency activity closely follows the filter rolloff characteristics in that range.  Of course, this effect is expected and desirable (that's the reason an anti-aliasing filter is used in the first place), but the high frequency portion of the EEG signal may not naturally be losing power (rolloff) as sharply as observed if it were not for the attenuation of the lowpass filter.  Therefore, in order to determine if the changes in high frequency energy are the result of physiological changes, anti-aliasing lowpass filter compensation will be implemented.

Anti-aliasing filter compensation

Because we are interested in examining the frequency content of the EEG signal up to the Nyquist frequency, the anti-aliasing filter attenuates a large portion of potentially useful signal content.  In an attempt to gain a better understanding of the high frequency portion of the signal, a filter compensation routine has been developed.

Because the frequency response of the 100Hz lowpass filter used in the data collection is known (experimentally), we can develop a function to compensate for portions of the signal attenuation in the form of a non-causal frequency-domain filter.  This filter will be applied to the data (after data collection) in the post-processing analysis phase.

The frequency response data of the anti-aliasing lowpass filter is converted from dB attenuation to percent amplitude attenuation.  Missing frequency points were interpolated and the data was fit to a polynomial function.  The attenuation amplitude at each frequency point (according to the frequency range and resolution of the transform) is squared to compute the percent power attenuation, which is then inverted to define a sequence of power multipliers for the entire frequency range.  This array of multipliers can then be applied directly to a power spectrum (PSD) to compensate for some of the signal attenuation introduced by the anti-aliasing filters as demonstrated in Figure 5.6.5.  The compensator is implemented in a general purpose function which accepts the frequency points from any of the PSD transform functions and creates a compensation array corresponding to the particular frequencies and resolution being used.

Figure 5.6.5  Block diagram of filter compensation implementation

In Figure 5.6.6, an example of the filter compensation multiplier is shown for the frequency range of 0 to 475Hz.  This function can be used to compensate for the original attenuation of a PSD which occurred during the data collection phase.

Figure 5.6.6  Lowpass filter compensation function from 0 to 475Hz

We can now compare the compensated and uncompensated PSD's of the data being analyzed.  Figure 5.6.7(a) shows the uncompensated PSD using an example EEG signal (shown previously in Figure 5.4.4) from 31 to 475Hz and Figure 5.6.7(b) shows the same signal with the filter compensation function applied to the spectrum over the same frequency range.  Small amplitude power components, which were barely visible in the standard PSD, are quite easily detected in the compensated spectrum.  We also observe that the steep rolloff which is visible on the original PSD is primarily a function of the lowpass filter's attenuation and not necessarily the behavior of the input signal itself.

Figure 5.6.7  Power spectrum from 31 to 475Hz: (a) Uncompensated
(standard) spectrum, (b) Compensated spectrum

Finally, we can compare the activity from 100 to 475Hz with and without compensation in Figure 5.6.8.

Figure 5.6.8  Power spectrum from 100 to 475Hz: (a) Uncompensated
(standard) spectrum, (b) Compensated spectrum

As a final note on the filter compensation: of course the most desirable (and technically correct) option is to sample the input time signal fast enough that the anti-aliasing filter cutoff is sufficiently high so as to avoid attenuating the portion of the signal under analysis.  This filter compensation routine is both a salvage operation to try and extract as much information as possible from the EEG signals and provides a means of making this analysis independent of individual EEG amplifier filter characteristics.

Figure 5.6.9 was generated by applying the filter compensation function of Figure 5.6.6 to the data shown in Figure 5.6.4.  We now see that the high frequency portion of the signal has a much more uniform power distribution throughout the entire high frequency range and this should provide a more accurate (or at least closer) representation of the original (pre-filtered) signal.  Also, the high frequency portion of the spectrum falls in the middle of the color table (instead of at the bottom) which will provide improved visual resolution to the changes in amplitude which occur in this range.  The filter compensation function will be used throughout the remainder of this analysis unless specified otherwise.

Figure 5.6.9  Full spectrum with filter compensation

Split scales

There are primarily two distinct regions of the spectrogram which are of interest, the image will be split (according to frequency) into two separate spectrograms as shown in Figures 5.6.10L and 5.6.10H.  The spectrogram of Figure 5.6.10L focuses on the traditional low frequency range (0-60Hz) and Figure 5.6.10H focuses on the higher frequency portion (60-475Hz).  The ordinate of the graphs lists the frequency range in parenthesis.  Because the power contained in the low frequency components is several orders of magnitude larger than at the higher frequencies, by separating out the low frequency components, the higher frequencies can be analyzed at their natural scale which might otherwise be obscured by the larger (low frequency) components.  Also, the low frequency data which can be used to gain a traditional understanding of the data (sleep stages, etc.) is contained in a narrow range of frequencies (approximately 0-30Hz), and this information becomes obscured when viewed at a full spectrograms wide bandwidth (475Hz).  Refer to the bottom of Figure 5.6.9.

Figure 5.6.10L  Low frequency portion 0-60Hz

Figure 5.6.10H  High frequency portion 60-475

Features evident in spectrogram of good performance

Although not particularly interesting visually, Figures 5.6.10L and 5.6.10H show a subject performing the visual test on their alert day (full night of sleep) and performing well.  The low frequency plot shows bands of activity in the DC, Alpha, and Beta bands which are consistent with wakefulness.  The high frequency band shows an overall uniform power distribution with a few vertical streaks indicating larger magnitude activity.  Generally, these streaks are caused by motion/movement artifact which results in broadband noise seen through the entire spectrum.  The streaks can be created by even a small number of samples containing artifact, and although they are higher in magnitude than the background activity, they typically are not of sufficient magnitude or duration so as to obscure much of the useful data in these plots.  Also, there are other occasions where streaking cannot be directly attributable to artifact.  For example, when a technician speaks to the subject during testing there appears to be an increase in high frequency activity from the relaxed/resting baseline.

Another example of a well-rested subject performing the visual test without failures is shown in Figures 5.6.11L and 5.6.11H (cia01) which was created from 9.9 minutes of data.  The conditions of the testing, EEG channel selected, signal and image processing are identical to those in Figure 5.6.10.  The low frequency plot shows predominant energy in the DC and alpha bands.  This subject never produced a distinctive beta band of activity during either of their tests, but we can see that there is elevated energy through the beta region.  Once again, the high frequency plot shows generally uniform power distribution with high amplitude streaking as observed in the previous example.

Figure 5.6.11L  Low frequency - good performance

Figure 5.6.11H  High frequency - good performance

Finally, Figures 5.6.12L and 5.6.12H (10.3 minutes duration) is a continuation of the data used to create Figures 5.6.11L and 5.6.11H.  Again, we see the same general features as seen in the previous two examples.

Figure 5.6.12L  Low frequency - good performance

Figure 5.6.12H  High frequency - good performance