Ch 10: The Dot Product and Convolution

This chapter goes over the math / meaning of the Dot Product and Convolution, critical concepts that make up the basis of a lot of analysis, such as FFT

10.1 Dot Product

Convolution is an extension of Dot Product, so it is important to learn the Dot Product first. You likely know of some interpretations of the Dot Product. Purely mathematically it is a function that takes in 2 vectors of equal length (the 2 vectors can be in any order), multiplies like-element of each together, and returns the sum of all those results, outputting a single value.

In practice, this can be thought of in a few ways:

• Sum of elements in one vector weighted by the elements of another vector - signal-processing interpretation
• A measure of covariance / similarity between 2 vectors - statistical interpretation
• A mapping between 2 vectors scaled by the cosine angle between them - geometric interpretation

In the geometric sense, you can think of the dot product of the length of the "shadow" of a vector projected / "shadowed" onto another.

This example is given in 2D space, but mathematically it can be done in any number of dimensions. Ergo, you can take an EEG signal time-domain set of 640 points and write it as a single vector of 640 dimensions, and use this for taking dot products.

Further details about the dot product can be found here.

Be SURE you understand how dot products are computed before continuing, otherwise you WILL get lost!

10.2 Convolution

Broadly speaking, a convolution is an operation taken on 2 functions to product a 3rd. In our context of EEG analysis, a convolution in the time-domain is when the dot product is repeatedly computed over time. There are various ways of thinking about this:

• A time series of one signal weighted by another shorter signal that slides along the first signal - signal-processing interpretation
• A cross variance / similarity between 2 vectors over time - statistical interpretation
• A time series of mappings between 2 vectors - geometric interpretation
• A frequency filter
  • More on this later in chapter 11

In EEG signal processing, we have names for the 2 input vectors used in a convolution. The input EEG signal is simply called the "signal" and the the time-series acting upon it (AKA sliding along it as will be discussed in the next section) is called the kernel. Different kernels can be used to analyze various properties, and itself is the basis of many analysis methods.

10.3 How does Convolution work?

Convolution can be thought of as 3 general steps

• Take the kernel, and flip it time-wise (along the time axis)
• Slide the kernel along the data (one step at a time)
• Take the dot product and output it at each step
  • you will notice as you slide along, you use the whole kernel every time but a slightly different slice of the signal

The end result will look like you "dragged" the kernel along the data. Essentially, the output shows what the kernel and that segment of data had in common (because again, dot product can be used to determine how alike 2 vectors are). 

There is technically another step. Since you slide the kernel along the points as a full set, if you think about it, the output must have fewer points than the input signal. To avoid this, we actually at the start align the right most of the kernel to the left most of the signal. Since this would leave the rest of the kernel "dangling", we 0-pad the signal with (length of kernel - 1) points on the left side. We also do the same thing on the right side as well. 

However, now we actually have too many points in the output convolution on either end. To bring the result back to the same length of the signal, we remove (length of kernel / 2) points from the start, and ((length of kernel / 2) + 1) points.

In math terms, convolution is often denote by the asterisk, such that the convolution of a and b will be represented by a*b . Beware not to confuse this with multiplication!

To compare the convolution with the original signal, its best to scale the convolution by the sum of kernel points. Keep in mind that doing this, however, is not the same as mean-centering the kernel before doing the convolution, since the latter can result in negative numbers.

10.4 Convolution versus Cross-Covariance

Convolution and Cross-Covariance are similar, except that Cross-Covariance does not include the step of flipping the kernel. Ergo the result of Cross-Covariance will be different, except in the case of when the kernel can be reflected upon itself X-wise, in which case the Convolution would of course equal the Cross-Covariance.

10.5 The Purpose of Convolution for EEG Data Analyses

Convolution is used to isolate frequency band specific activity and localize it in time. This can be done by convolving wavelets (time-limited sine waves) with EEG data. If you do this to a signal many times with the wavelet kernel done at a different frequency each time, you can build a time-frequency representation. This then leads us the the Fourier Transform...