Development
With the decoder now working I started looking for solutions to some of the shortcomings that I had identified. These ranged from simple changes to a complete rethink of the methodology!
The limitations fell into 2 main categories
Waveform processing
Variations in CW speed and quality
User interface
These are discussed in this order in the following section.
Waveform Processing
The main issue with the decoder was that whilst it worked well on the sample 20wpm practice video that I had used in development, it struggled with other examples. I guessed that the Discrete Fourier Transform (DFT) algorithm was mainly to blame as this had a fixed centre frequency of 600Hz. To compensate it had a relatively wide bandwidth (around 300Hz) but this was quite wide for filtering out noise and too narrow for some of the carrier frequencies in use.
I looked at different sampling speeds and numbers of samples combined with a calibration routine to identify the best centre frequency. This helped but wouldn't adapt to a change of carrier frequency without a recalibration. I looked at sampling adjacent centre frequencies whilst waiting between samples but this was not very successful and took too long. I even looked at other techniques including monitoring the amplitude of the carrier directly or timing the carrier cycles to pick up the presence or absence of signal. Both of these worked to an extent but I eventually concluded that the benefits of the DFT approach far outweighed the limitations.
As a diversion whilst pondering how to improve the DFT solution, I built a simple interface to bypass the microphone and connect the computer headphone output directly to the decoder. I did this primarily to remove the background household noises and to avoid my wife's complaints of "annoying" tones from the computer. However, I was pleasantly surprised to find that it also solved most of the variation in signal level issues. I believe that the small room where I was doing the testing with the microphone was echoing and/or resonating certain frequencies and resulting in changes in level as I moved about and altered the resonances.
So, I decided the best way forward was to run the Analogue to Digital Converter (ADC) measurements and DFT calculations continuously with a variable sampling rate that adapted to the carrier frequency. I also decreased the DFT sampling rate and increased the number of samples to narrow the bandwidth of the target frequency. This would normally result in the sampling taking too long so I ran 4 DFT calculations in parallel with the outputs spaced evenly in time. With some experimentation and tuning this technique provided a much better solution but it also introduced a lot of new timing constraints that meant that I had to completely restructure the coding - more on this later.
CW speed and quality variations
With a direct audio input and the revised DFT processing algorithm, the CW signal output was now much cleaner. The large variations in high state amplitude were no longer apparent, the noise level was reduced and the shape of the pulses much better defined.
However, it was still necessary to adapt to the CW speed to ensure that the dots and dashes could be discriminated and the inter symbol spaces correctly identified. This was made more difficult as the low state bit time could be as little as half the high state bit time or as large as twice the high state bit time. Similarly, some of the inter character spaces could be greater than 3 low state bit times and the inter word spaces less than 7 low state bit times.
My initial calibration approach was not that useful for this issue and a more adaptive system was necessary. This will be described in more detail later.
User Interface
A simple 16 character x 2 line LCD display was chosen to display user messages and the decoded CW characters. As with the OLED display used for the oscilloscope project, the available libraries were slow and took up a lot o memory with features that are not needed for a simple implementation. However, the advantage of the LCD display over the OLED is that the character mapping is held in the LCD memory so it is not necessary to define fonts and send every pixel to the device - just the required character code. It also has a scroll mode such that each new character moves the display across automatically.
With reference to the datasheet and some trial and error, I wrote a short setup routine to initialise the LCD display and set up the relevant parameters. This allowed me to display a character in around 90 μs compared with around 1.6 ms to display a character on the OLED display.
I used the 4 bit data transfer mode to save data pins on the Arduino although this made the data transfer a little more involved. I intended to use both lines of the display but this would have meant I couldn't use the automatic scrolling in the way I wanted so I reverted to a single line display.
Programming
As hinted above, the latest version of the software moves away from the simple process flow described in the part 2 and is now orientated around the need to sample the waveform continuously and sufficiently fast enough to measure the frequency of the carrier. All the other processing had to be slotted in between the sampling without affecting the timing.
The sampling rate needed to be as fast as practical in order to allow the carrier frequency to be measured with sufficient accuracy. Assuming a starting carrier frequency of 600Hz, I chose to run the ADC at 19,200 samples per second which gives a nominal frequency measurement accuracy of about 5%. However, this gives a 52 μs period in which to do the sampling and any other processing before taking the next sample. This is somewhat challenging to say the least!
The other advantage of continuous sampling is that the DFT process can now be run continually. This allows multiple DFT processes to be run in parallel to produce an output at regular intervals whilst still capturing sufficient samples to narrow the bandwidth and improve the signal quality.
I used every 8th ADC measurement as an input to the Goertzel DFT algorithm which gave 2,400 samples per second. Using 32 samples for each Goertzel output, this gave a much more focussed 75Hz bandwidth at a rate of 75 output measurements per second. This would allow just 4 DFT values within a "bit" period at 20wpm so I ran 4 DFT processes in parallel giving 300 output measurements per second which should be adequate up to around 60wpm.
There was another good reason for choosing these sampling values for the Goertzel algorithm: for a 600Hz carrier, the coefficient used in the calculation becomes zero. This means that the calculation is much simpler and does not require floating point arithmetic which reduces the processing time by an order of magnitude. This was essential in order to fit the processing into the 52 μs period dictated by the ADC measurement cycle.
This still leaves the problem of adjusting the DFT sampling for carrier frequencies other than 600Hz. I will address this in part 4 and also describe how the other software processes are squeezed into the timeslots between the ADC measurements.
Reference
The 1602 LCD datasheet can be found at:
More information on Discrete Fourier Transforms and the Goertzel algorithm can be found in the links at the bottom of part 1.
Comments