Using 24 Mhz crystal will affect the speed, too many communication errors and maximum speed I can achieve is 230,400 bauds which is not really enough for a decent streaming. Changed the crystal to 22.1184 Mhz and now I can get 921,600 bauds with no errors, allowing me to stream 2 RGB444 frames per second and an “outstanding” 5 fps B&W 4 bits per pixel (128 x 120), all this is using an Atmega16 microcontroller.
I just managed to buy a pair of Atmega1284P chips today and got one of them installed on a prototype board, did some small changes in the code and got it up and running @ 921,600 bauds without issues. Will continue to work on my code and hope to get the first demo soon. So far, can do tracking at 15 fps in pooling mode (host initiate tracking within a frame), however, will implement a full speed mode, that will allow continuos tracking at higher speed, targeting 30 fps.