Camera Tracking System (Camera Vision)

NOTE: If you are interested in purchasing these CTS boards , you can contact me directly using “comments” section.

This page focus on a low-budget solution that implements Colour Segmentation to do real-time tracking of one or multiple objects of a selected colour using a cheap OV7670 camera sensor from ebay and a regular micro-controller, Atmega644P. Initial version was designed and worked well using an Arduino board based on Atmega328P.

Able to track one colour at up to 30 fps, can provide real-time blob details and attached to any existing robotic project using any micro-controller that can communicate over UART port.

Details on how to communicate with the module, sample code on the host side and other information are available bellow.

About CTS module

Some story about this project.

Features

CTS module was design primarily to be able to offer a low power and cheap solution for tracking. Due limited processing speed a Colour Segmentation solution was used to implement tracking. Some of the features of this implementation are:

• Small size (30 x 30 mm or 50 x 25 mm);
• Low power consumption;
• Working with OV7670/OV7725 cameras available cheap on eBay;
• Some CTS models have on board cameras;
• Real-time Streaming and Tracking over UART/serial port at very high-speed (up to 921,600 baud);
• Windows-based client available for GUI setup (free for download, VB source available on request);
• Simple serial interface to allow command line configuration for all settings;
• Built-in help for commands when using command line;
• Configuration can be stored in EEPROM;
• Standard ISP/JTAG headers available on board for future firmware upgrades.

Some details about tracking:

• Tracking done in real-time;
• Up to 8 colour classes can be created for tracking (using YUV format);
• Virtually can track up to 256 blobs;
• Fast Tracking mode: up to 30 FPS (1 colour/1 blob);
• Colour Tracking:  up 8 blobs, 1 blob for each classes of colour defined;
• Single Colour Tracking: up to 32 non-connected blobs of a selected colour class;
• Streaming and Tracking can be done simultaneously using one of the following resolutions: 128 x 120, 160 x 120, 256 x 240, 320 x 240;
• Streaming using YUV422 (2 bytes per pixel), YUV211 (1 byte per pixel) or B&W (4 bits per pixel);
• Detailed tracking data for each blob is available in ASCII or HEX format (COG, weight, bounding box, etc);
• Sample code available for AVR, Processing, ARM, etc.

Hardware details

When I started working on this project , I thought of keeping everything within 30 x 30 mm, same size with the OV7670 FIFO camera module, to keep it as compact as possible. Another reason is that most of the pan-and-tilt brackets used for various UAV projects are designed to hold 30 x 30 mm camera module boards. To achieve this, everything on the board is SMD technology, except the connectors, which are through-hole components.

Entire project is designed around ATmega644P and AL422B (3 Mbit FIFO). Data is continuously stored in FIFO chip that can accommodate up to one VGA frame (640 x 480 pixels, 2 bytes per pixel). MCU will retrieve the image from FIFO and process the data line by line.

During initial testing, I decided to use YUV format instead of RGB as seems less affected by light variation and get better results during tracking.

A basic serial terminal is implemented, allowing the board to be configured manually using a simple serial console or through applications running other microcontrollers. A Windows-based client application that help accessing and configuring the module it is also available for download. Details on how to use, set and control together with data format is detailed in Communicating with CTS module section.

There are two versions:

CTS 5.1

30 x 30 mm, no camera on board, can take any OV7670 camera modules from eBay, also available separately, a compact version using OV7670 camera phone module (see pictures).

Standard 6 pin AVR ISP header and FTDI UART header in 0.1″ pitch (for pinout details check here).

CTS 5.2

Measuring only 50 x 25 mm, is same hardware with CTS 5.1 but also come equipped with a OV7670 camera and the electronics required to drive the camera. While keeping the size down due the low profile camera, there is no compromise in image quality.

Also standard 6 pin AVR ISP and FTDI UART headers are available for access.

Both boards will run at 5 Volts.

Pictures with real boards can be seen here.

Specifications

Both AVR modules have the same specs beside the physical differences:

• MCU: ATmega644P/ATmega1284P;
• FIFO: AL422B (3 Mbit FIFO);
• Camera: OV7670/OV7725;
• UART speeds: 38,400, 57,600, 115,200, 230,400, 460,800, 921,600;
• Tracking: up to 30 FPS in FastTracking mode (1 colour, 1 blob) or 15 FPS multiple colours/blobs using Colour Segmentation;
• Streaming: 128 x 120, 160 x 120, 256 x 240, 320 x 240 YUV 422, YUV 211, B&W (4 bit/pixel).

For ARM version, specs are:

• MCU: LPC1343;
• EEPROM: 4/8 Kbytes on board;
• FIFO: AL422B (3 Mbit FIFO);
• Camera: OV7670/OV7725;
• UART speeds: 38,400, 57,600, 115,200, 230,400, 460,800, 921,600;
• Tracking: up to 30 FPS in FastTracking mode (1 colour, 1 blob) or 15 FPS multiple colours/blobs using Colour Segmentation;
• Streaming: 128 x 120, 160 x 120, 256 x 240, 320 x 240 YUV 422, YUV 211, B&W (4 bit/pixel).

Specs are pretty much the same for both AVR and ARM (LPC1343) boards.

CTS 0.5 boards, from left to right: CTS 0.5.1 with AT644P, CST 0.5.2 with AT1284P and OV7725, CTS 0.5.2 with LPC1343, CTS 0.5.2 with LPC1769 and OV7670.

CTS 7

The new CTS boards (CTS 0.7) are already here and working fine. Are still based on LPC1343 and LPC1769, however, there are lots of changes:

• Code is smaller and faster;
• Serial firmware upgrade option;
• Serial updates for camera’s initial configuration registries;
• Serial updates for various camera configurations (e.g.: GAMMA, White Balance, contrast, brightness, saturation);
• Hardware and software to control Pan and Tilt servos in Fast Tracking mode up to 30FPS;
• Access to servo configuration and control through command line (e.g.:  min/max ranges, positions, etc);
• PID controller for servos for faster tracking speed (details here);
• Real-time streaming option up to 30FPS in FastTracking mode;
• Fine-tuning AWB for OV7670 cameras;
• Many small changes and fixes.

CTS 0.7 boards, with LPC1343 and LPC1769

And a small demo for the new CTS 0.7 (LPC1343) boards:

CTS 0.7 board controlling Pan and Tilt servos @30 FPS and real-time streaming

Communicating with CTS module

How to communicate with CTS module module (connectors pinout, data format, commands to control image or tracking or configure the module, etc.).

Using CTS module with various micro-controllers

Code sample for connecting Camera Tracking System module other platforms or micro-controllers (e.g.: Processing, Propeller, Arduino, AVR, ARM, etc).