Nyquist and sampling

Due to one thing and another (mostly the one thing commonly referred to as my thesis, although I have some more personal terms for it), I haven't found the time to keep this little corner of the blog-o-sphere up to date lately. However, the blighter is just about finished (the light is most definitely in sight), which means normal service should be resumed in the near future.

In the mean time, there are a couple of things of late that have caught my attention, causing my little grey cells of curiosity to spark into life. So, with luck, I hope to take up some of the time off I'm allowing myself over Easter (ssh, don't tell the supervisors) to catch up on a little bit of blogging.

First amongst the musings I wanted to talk about is a few thoughts I had whilst looking up some information on Harry Nyquist. Anyone who has studied physics, maths, or engineering of some description must has come across him and his famous Nyquist Theorem (although, properly it should probably be called the Nyquist-Shannon Theorem).

For anyone who hasn't come across this particular idea, it is one of the fundamental principles of Information Theory (something which Nyquist and Shannon both contributed heavily toward developing). Regardless of whether you've ever studied this subject at all, it will have played a huge part in your life. Information theory is fundamental to the way the all those electronic gizmos (from the telephone to the computer and the microwave) we have got so used having loitering around the place. The fundamental principles of information theory are what govern the way the information is communicated, both within a electronic system (e.g., signals between the CPU and graphics card in your computer), and between them (e.g., telephone signals). While subtle nuances in the theory allows this information to be transferred faster and more efficiently, therefore making your computer or tv work faster and better.

At the core of this is something called Sampling Theorem; i.e., the Nyquist-Shannon Theorem. This basically covers the way analogue signals are translated into digital ones (and vice versa). This is easiest explained if we consider an example, say music. Lets consider we're sat in a quiet room strumming a guitar. The sound produced by the guitar spreads out through the room as a series of pressure waves. If we were to think about how these pressure waves are heard by a human ear, we would get a continuous line of varying amplitude; an analogue signal (see figure below).

However, if we want to record this sound so we can replay it on a computer we have to digitize it. Computers can't store analogue signals since they, by definition, have an infinite number of amplitude values. Thus, analogue signals are digitized, which means we describe the analogue signal as a series of amplitude values (see figure above). Thus, the sound of the guitar becomes a series of numbers that represent the amplitude of the sound (pressure wave) at a moment in time.

This digitization of the analogue signal is where the sampling theorem comes in. We have all got music stored on our computers which we can listen to. In order for this to sound the same as if were were sat in the same room as the real instruments playing, the sound has to be sampled at short enough time steps so as to convince the human ear that it is not a series of descrete samples. Basically, it's the same way as animations work. An animation consists on c. 30 individual drawings which are flashed onto the screen every second. The human eye can't tell the difference between them, and so is tricked into thinking it is a moving picture. With the digitization of the sound from our guitar, we're doing exactly the same process.

Every electrical gadget in your home works with these digital signals. What the Nyquist-Shannon Theorem describes, is the minimum time step at which a signal has to be sampled in order to convey a certain set of information. Engineers, mathematicians, and Physicists think of this information in terms of it's frequency, which is what the theorem describes; the Nyquist Frequency.

I deal with the Nyquist theorem pretty much every day as it also plays a fundamental part in signal processing, which is essentially what I do. However, what I didn't know was just how successful Nyquist and his colleagues at AT&T Bell Labs who pioneered information theory (amongst other things) were. The total of 11 researchers won 6 Nobel prizes between them! That is an incredible success rate, and probably one which will never be replicated anywhere else. Kinda puts into perspective everything us little people are struggling with!