New CTS Client

I’m doing some changes lately on my CTS board firmware and I started by cleaning up and rewriting some parts of the code. Got rid of some unused code and the “windowed” mode and managed to shrink entire code by 10% even after adding some features.

Anyway, since one of the new feature is to increase the colour gamut for tracking from 12 bits/pixel to 18 bits/pixel I had to revamp my CTS Client too. I hate switching from Windows to Mac and back every time so I decided to write it on Java. Considering the fact is the first time I’m writing anything on Java… err…

Anyway, I am using “RXTXComm.jar” for serial communication, which is available on Linux, Windows and Mac, so is supposed to work fine on all platforms, i guess :-)

Screen Shot 2013-11-27 at 10.44.55 pm


Writing the application was quite fun, especially now when I got it working. I added support for controlling servos in real-time, set the ranges, set PID values, put back the colour selection monitoring.

I will be adding support for other features available on the new firmware version: real-time tracking with servo control and streaming, upload camera configuration over serial port (gamma and saturation profiles, AWB configuration, resolution settings), etc.


8 thoughts on “New CTS Client

  1. Hello, congratulations for your project, its very well done and documented. Here i leave you a link to a similar project of mine with STM32F1.

    I must say that in my particular case using a discrete algorithm for PID worked much better than the one that you use ( K=kp*err+ki*ierr*T+kd*derr*T). It might be useful to you as well, it improved a lot my tracking system. The algorithm is here, but it lacks the derivative part, it is only a PI, but i can calculate the complete algorithm with the derivative part if you need it.
    U_[k] =U_[k-1] +E_[k-1] ×(Ki×T/2-Kp)+E_[k] ×(Ki×T/2+Kp)
    Best of lucks!

    • Hi Diego,
      Nice project you have there! Thank you for your feedback and I have few questions if you don’t mind:
      – you mention object tracking, is it the object or the colour you are tracking?
      – hearing the servos and looking at the movement seems you are refreshing the position a bit too slow. Did you tried to call the PID every 10 ms instead? I had a similar problem before when I was calling the PID every new frame, however, the results improve significantly once I changed it to 10 ms (you get 2 extra cycles to adjust the position to your last known destination).
      – any idea on the frame rate you are getting with this setup? The image on LCD seems somewhere around 15 fps, am i right? If so, is this caused by the processing speed or because of LCD? You know you can just update the LCD in the same time you read the data from your FIFO, this will speed-up the entire system…

      Anyway, nice project, well done! Any webpage with details of your project?

      • Hi, thanks for the comment, regarding your questions:
        – This is a color tracking
        -The PID is called every frame, this happens every 115ms approximately, but i refresh the servos each 20ms with a filter that only have the function of smooth a little bit it’s movement. The reason i done this is because PID shouldn’t recalculate with the same measurement, this in theory, in practice i tried to update PID values each 20ms, it was stable but not better. Reading your comment i think i should try updating PID every 50ms, that might improve the system, but technically i don’t see why.
        – The frame rate is governed by how much the microcontroller takes to make all processing, the result is a new frame each 115ms, that would be like 9fps. The LCD do show the camera pixels as they are read from the FIFO, but the time increases significantly because i’m using a 320×240 resolution which is a big amount of data to be processed and RGB565 format in order to show the image in the LCD without any format transformation. But i do make a transformation to each pixel to get its chromatic value and make a more accurate segmentation.
        I’m trying to see if there is any page interested in publishing documentation of the project, i will keep you updated if i got some luck. Thanks for the comment and i must say that i tried different controllers, then i read your PID post and i tried that one, it was good, but then i used that discretized PID and it just went a lot better, give it a try.

      • Ok, few things:
        – To have servos moving smooth you should do a refresh at least every 20 ms. I am processing the entire frame in a bit under 30 ms. If i call PID only then I have to choose between having t0 move the servos very fast and risk to overshoot for the following frame or following to slow. Going for 10 ms refresh rate I get 3 cycles to reach to last known COG instead of 1 and i get a very smooth move. Also, when you are using FIFO it may be a 1 frame delay between real life and what your MCU sees unless you do it right. Read AL422 data sheet for details.

        – I am using YUV instead of RGB, makes my life easier and i get a faster colour indexing. Also, i get 4 bytes for 2 pixels to process (Y, U, Y1, V) rather than a “scaled down” and noisy RGB565 where I need to do bit shifting and masking to get the colours. I also find YUV less affected by lighting condition.

        – If you go for RGB you can just put a bus latch and a bus buffer to “widen” your bus to 16 bits. Use the RDCLK (divided) to load the latch for first byte/clock, then on second byte/clock enable the latch and the buffer and you get the 16 bits you need for your LCD ;-) Can send other data between frames, check OV7xxx data sheet, there are 115 lines between each frame.

        There are lots of papers online on colour segmentation but most of them are too high level, however, you may want to take a look on this one (

      • Could you have used HSV instead of YUV?

        HSV color space is independent of illumination and makes the processing of images easy too.

  2. I read a lot of documents, but this one is REALLY useful thanks, often i think my project is good and code efficient and then i find out there are a lot of ways to improve, this happened a lot over the months. I think main weakness is that i can only make a color segmentation of one color at a time, reading this paper gave me a lot of ideas.
    Your algorithm to extract the centroid and region information of different colors is such a good one, how do yo manage to do that, how do you get the centroid and the surrounding box?, I can barely get a good centroid and only with one color…

    • Well, you may need to read more then :-)
      Not sure how efficient is your code, you are using a Cortex M3 72 Mhz with 48 Kbytes RAM. You should be able to get your code blazing fast. You have enough space to store a QQVGA frame (38400) then do entire processing on it then stream it out as a whole.

      I was able to do 1 colour tracking at 30 FPS on ATEGA8 running at 24 Mhz with only 1 Kbyte of RAM and 8 Kbytes flash. You will need to start reading your MCU user manual, understand how much each instruction take in MCU cycles, set up your peripherals correctly, get your own optimised libraries, etc. Worse come to worse: process every other pixel instead of every pixel, will double the speed and will still work pretty well.

      I tried to optimised my code, and I am not an expert on this, then I posted sometime ago a diagram on the timing I used to read 4 bytes and process 2 pixels looking for colours and update my blob details:

      I could process in average a pixel every 418 ns, basically enough to process 31 frames in QVGA. I did play with the MCU configuration and internal clocks and I got another increase of 20-30% and I still think is possible to get it even faster (store an entire frame in SRAM using DMA then process the image using optimised algorithms, etc).

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s