Visual Object Recognition with Tracked Position Averaging Filter

In this commercial, the basketballs are isolated in the HSV colorspace. Ball centroids are output and marked with red circles, however position is lost / degraded due to the basketballs are out of the camera's field of view, briefly covered by the British people throwing them, or one of the bloke's round heads being recognized instead of the basketball. During these instances, the position is erroneously recorded as in the top-left corner (0,0).

The goal of using the Kalman Filter is to estimate the position(s) of the basketballs in such a way that their changes in position are averaged, and thus less affected by these outlier (bad) measurements. However, the source video was a bad choice, because of how often the basketballs leave the FOV, and the thresholding picking out a chap's head over the basketball.

The Kalman Filter state transition matrix accounts for position, velocity, and acceleration of the x and y components of the tracked objects:

[1 1/6.3 .5*(1/6.3)^2 0 0 0;
0 1 1/6.3 0 0 0;
0 0 1 0 0 0;
0 0 0 1 1/6.3 .5*(1/6.3)^2;
0 0 0 0 1 1/6.3;
0 0 0 0 0 1]

with states [x x' x'' y y' y'']' for the two components of position and their time derivatives. Note 6.3 = Frames per second (FPS) for a sample rate of 1/6.3.

Because of the rapid changes in motion of this video, such a simple model does not suffice for spot-on tracking, and therefore the green, filtered position markers appear to lag quite a bit, and are still somewhat highly affected by bad measurements as described above. To keep the filtered estimates closer to the actual position, the process noise covariances should be increased, and the measurement noise should be lowered. However, to better demonstrate the averaging effect of the Kalman filter, I kept a process vs measurement covariance ratio (wk/vk) of 2:1.

Changing the variances for individual states, such as more weight on position vs velocity and acceleration has no noticeable effect for this example, since the framerate is so low (6.3 FPS). A better video would have made for an even better demonstration.

Category:

Education

Tags:

• USNA
• ES453
• kalman filter