Tracking control commands

Commands used to control tracking. Please note the setting will change between frames.

Basic Help

A basic Help feature is implemented by default on CTS for every command. To get help type the command followed by ‘h’:

h
Available commands (use '<command>h'for more details).
- Tracking/capture:
    Frame dumping:       F...
    Tracking:            T...
    YUV Class:           C...
- Image control:
    AWB:                 a...
    Brightness:          b...
    Contrast:            c...
    GAMMA:               g...
    Orientation:         f...
    Resolution:          r...
    Saturation:          s...
    Sharpness:           S...
- System control:
    Configuration:       M...
    Version info:        v...
    UART:                u...

Dump/Capture new frame

Capture a frame and dump it on the serial port, used for streaming the image to PC for various purposes (e.g.: debug, streaming, select tracking colours, lens calibration, change image settings, etc).

Capture control

To list all available options:

Fh
Select Frame Dumping mode for each frame.
  Use Windowed mode: Fy<c>
  Use Normal mode:   FY<c>
    YUV422:          F<m>4
    YUV211:          F<m>2
    B&W:             F<m>b

Windowed or Normal mode?

By default, OV7670/OV7725 cameras can capture images in different formats (e.g.: VGA, QVGA, QQVGA, CIF, etc) which pretty much covers all the standard formats. However, when handling images using limited amount of memory, each byte counts, so I figure it out that reducing the size of the image from 160 to 128 or from 320 to 256 will reduce by 20% the amount of data manipulated while still having most of the information required.

Also, using 128 and 256 pixel line width will allow an AVR MCU to store and handle at least 2-4 lines @ 2 bytes per pixels in QQVGA mode in only 1 Kbyte of RAM while still have enough space to handle various image processing algorithms. Also, will reduce the amount of data sent for each blob package.

This was important at the time when this project was initiated, as I was using ATmega8 then later ATmega328 (Arduino) chips where SRAM memory was a problem.

This option is still available for both Tracking and Capturing modes. To differentiate between Normal and Windowed modes, the second character is in capitalised for Normal mode and non-capitalised for Windowed mode.

Example: To Capture a frame in B&W using Windowed mode use:

Fyb

To Capture a frame in YUV422 using Normal mode use

FY4
NOTE: Using Windowed mode can improve performance for streaming by up to 20%. Streaming QQVGA, B&W, Windowed mode @921600 baud, CTS ARM using Unix/Processing can reduce from 150 ms per frame to 120 ms per frame.  

YUV422

To capture and dump a YUV422 frame in Normal mode over serial port can be done using:

FY4

Once this command is issued, CTS will capture a frame in YUV422 format (data format is explained here and sample code to process YUV422 format and display it in RGB format is detailed here) and send it line by line over the serial port.

NOTE: Capture commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

YUV211

To capture and dump a YUV211 frame in Windowed mode over serial port can be done using:

Fy2

Once this command is issued, CTS will capture a frame in YUV211 format (data format is explained here and sample code to process YUV211 format and display it in RGB format is detailed here) and send it line by line over the serial port.

NOTE: Capture commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

Black and White (B&W)

To capture and dump a B&W frame in Windowed mode over serial port can be done using:

Fyb

Once this command is issued, CTS will capture a frame in B&W format (data format is explained here and sample code to process and display it is detailed here) and send it line by line over the serial port.

NOTE: Capture commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

Tracking

Capture a frame, apply basic color segmentation algorithms, detect blobs of predefined colors based on the YUVClass Tracking Table (see YUV Class section for details) then send information over the serial port. Several modes are available including image streaming and tracking,  Fast Tracking, debug mode, etc.

Tracking control

To list all available options:

Th
Select Tracking mode for each frame.
  Fast Tracking:     TF
  Use Windowed mode: Td<c>
  Use Normal mode:   TD<c>
    YUV + blobs:     Ty
    B&W + blobs:     T<m>d
    Debug + blobs:   T<m>f
    Blobs (hex):     T<m>t
    Blobs (info):    T<m>T

Windowed or Normal mode?

See the above explanation in Capture section for details.

Fast Tracking

There is a Fast Tracking mode available on CTS module. This mode will allow tracking up to 30 FPS in QQVGA mode, only one color will be used for tracking (first color defined in YUVClass Tracking Table, see the YUV Class section bellow for details).

To start Fast Tracking for the next frame use:

TF

In Fast Tracking mode CTS will return basic tracking details in following format if a blob of predefined colour is detected with weight over 1%:

Byte1:     Start byte (0x42)
Byte2:     Start byte (0x00)
Byte3:     COG X MSB
Byte4:     COG X LSB
Byte5:     COG Y MSB
Byte6:     COG Y LSB
Byte7:     Weight (in percentage)
Byte8:     End byte (0x00)
Byte9:     End byte (0xFF)

If no blob detected or weight bellow 1%, the data returned have the following format:

Byte1:     Start byte (0x42)
Byte2:     Start byte (0x00)
Byte3:     0x00
Byte4:     End byte (0x00)
Byte5:     End byte (0xFF)
NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

Windowed, FastTracking mode, stream captured with Processing.

YUV and Blob data

In this mode, CTS will return the captured frame in YUV211 format and Blob details for all the blobs detected within the frame. YUV211 format is detailed here, the Blob package format will be explained bellow.

To start YUV Tracking in Normal mode in the next frame use:

TDy

After YUV211 frame, CTS will return Blob tracking details in following format if at least one blob of predefined colours is detected with weight over 1%:

Byte1:                    Start byte (0x42)
Byte2:                    Start byte (0x00)
Byte3:                    Number of blobs detected(n)
... 
Byte(4+((0 to n)*15)+0):  Blob ID
Byte(4+((0 to n)*15)+1):  COG X MSB
Byte(4+((0 to n)*15)+2):  COG X LSB
Byte(4+((0 to n)*15)+3):  COG Y MSB
Byte(4+((0 to n)*15)+4):  COG Y LSB
Byte(4+((0 to n)*15)+5):  COG Y MSB
Byte(4+((0 to n)*15)+6):  Weight in percentage
Byte(4+((0 to n)*15)+7):  Top X MSB
Byte(4+((0 to n)*15)+8):  Top X LSB
Byte(4+((0 to n)*15)+9): Top Y MSB
Byte(4+((0 to n)*15)+10): Top Y LSB
Byte(4+((0 to n)*15)+11): Bottom X MSB
Byte(4+((0 to n)*15)+12): Bottom X LSB
Byte(4+((0 to n)*15)+13): Bottom Y MSB
Byte(4+((0 to n)*15)+14): Bottom Y LSB
...
Byte(4+n*15+15):          End byte (0x00)
Byte(4+n*15+16):          End byte (0xFF)

If no blob detected or weight bellow 1%, the data returned have the following format:

Byte1:     Start byte (0x42)
Byte2:     Start byte (0x00)
Byte3:     0x00
Byte4:     End byte (0x00)
Byte5:     End byte (0xFF)
NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.
 

CTS tracking boxes and colours

Windowed, YUV Tracking mode, captured with Processing. 

B&W and Blob data

In this mode, CTS will return the captured frame in B&W format and Blob details for all the blobs detected within the frame. B&W format is detailed here, the Blob package format is already explained above, in YUV Tracking section.

To start B&W Tracking in Windowed mode in the next frame use:

Tdd

After B&W frame, CTS will return Blob tracking details if at least one blob of predefined colours is detected with weight over 1%, similar with YUV Tracking mode.

NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

Windowed, B&W Tracking mode, stream captured with Processing

Debug and Blob data

In this mode, CTS will return the captured frame in YUV211 format, where only selected blobs can be seen, and Blob details for all the blobs detected within the frame. YUV211 format is detailed here, the Blob package format is already explained above, in YUV Tracking section.

To start B&W Tracking in Windowed mode in the next frame use:

Tdf

After YUV211 frame, CTS will return Blob tracking details if at least one blob of predefined colours is detected with weight over 1%, similar with YUV Tracking mode.

NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

 CTS tracking boxes

Windowed, Debug Tracking mode, captured with Processing

Blob hex format

In this mode, CTS will return only Blob details for all the blobs detected within the frame in hex in  format is already explained above, in YUV Tracking section.

To start Blob (hex) Tracking in Normal mode in the next frame use:

TDt

CTS will return Blob tracking details in following format at least one blob of predefined colours is detected with weight over 1%, similar with YUV Tracking mode.

NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

Blob Info

In this mode, CTS will return only Blob details for all the blobs detected within the frame in a simple to read, one line per blob, format explained bellow.

To start Blob (info) Tracking in Normal mode in the next frame use:

TDT

CTS will return Blob tracking details in following format at least one blob of predefined colours is detected with weight over 1% using following format:

Byte1:                    Start byte (0x42)
Byte2:                    Start byte (0x00)
Byte3:                    Number of blobs detected(n)
... 
<blob id> <COG X> <COG Y> <Weight> <Top X> <Top Y> <Bottom X> <Bottom Y>0x0A
...
Byte(last-1):          End byte (0x00)
Byte(last):            End byte (0xFF)

A line will be generated for EACH blob, data is in DEC format with a space character (‘ ‘) as separator, and CR as end of line.

If no blob detected or weight bellow 1%, the data returned have the following format:

Byte1:     Start byte (0x42)
Byte2:     Start byte (0x00)
Byte3:     0x00
Byte4:     End byte (0x00)
Byte5:     End byte (0xFF)
NOTE: Tracking commands will not change or set any image parameters, this can only be done using Image Control Commands detailed here.

YUV Class

For tracking I decided to use YUV instead of RGB for a number of reasons, one of them is the fact that RGB555 and RGB565 data generated by OV7670 is quite noisy plus YUV give better results when light varies for same colour.

In order to improve the speed, the YUVClass Table is implemented. This will allow to check the pixel colour against Y, B and V channels in fewer steps. 

YUV Class control

To list all available options:

Ch
List or set Y, U and V tracking values.
  Copy EEPROM to SRAM: Cc
  Dump SRAM   (hexa) : Cd
  List EEPROM (ascii): Ce
  List SRAM   (ascii): Cr
  Set values to SRAM : Cs<YY>
  Verification       : Cv
  Write SRAM to EEPROM: Cw

Copy EEPROM data to SRAM

To reload saved YUV Class data stored in EEPROM into SRAM use:

Cc
 Note: There is no confirmation returned by CTS, use ‘Cv‘ to verify SRAM and EEPROM are in sync, see bellow. 

DUMP SRAM (hex)

When accessed by a different system,/application, current YUV Class data stored in SRAM can be dumped over serial port in hex format, explained bellow.

To start this, use:

Cd

Returned data use following format:

Byte1:                Start byte (0x43)
...
Byte(2+(index*4)+0):  YUV Class Index
Byte(2+(index*4)+1):  Y information
Byte(2+(index*4)+2):  U information
Byte(2+(index*4)+3):  V information
...
Byte66:               End byte (0x00)
Byte67:               End byte (0xFF)

Where index is from 0 to 14 and data is generated in hex format.

List EEPROM (ascii)

To command will list the content stored in EEPROM when issued:

Ce
EEPROM
00 00 00 00
01 00 01 00
...
0e 08 03 00
0f 00 00 00

List SRAM (ascii)

To command will list the content stored in SRAM when issued:

Ce
SRAM
00 00 00 00
01 00 01 00
...
0e 08 03 00
0f 00 00 00

Update values in SRAM

For easy update of the values using a serial console, this command will allow entering line by line the YUV Class details:

Cs<UU>

Where:

  • ii – is YUV Class line index in hex (2 ascii characters)
  • YY – is Y value in hex  (2 ascii characters)
  • UU – is U value in hex  (2 ascii characters)
  • VV– is Y value in hex  (2 ascii characters)
NOTE:
  • this command can be used from a terminal, through a custom script/application from a host system or using CTS Client provided. 
  • There is no confirmation returned by CTS, use ‘Cr‘ to check. 

Verification

To check if SRAM and EEPROM are in sync use:

Cv

This command will return OK if both are in sync or ERR if are not. 

Write data from SRAM to EEPROM

This option is used to save the YUV Class Tracking values between RESTART cycles:

Cw
Note: There is no confirmation returned by CTS, use ‘Cv‘ to verify SRAM and EEPROM are in sync, see above. 
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s