Transcript Motion Detection And Analysis

Motion Detection And Analysis
Michael Knowles
Tuesday 13th January 2004
Introduction



Brief Discussion on Motion Analysis and its
applications
Static Scene Object Tracking
Motion Compensation for Moving-Camera
Sequences
Applications of Motion Tracking

Control Applications




Object Avoidance
Automatic Guidance
Head Tracking for Video Conferencing
Surveillance/Monitoring Applications



Security Cameras
Traffic Monitoring
People Counting
Two Approaches

Optical Flow


Compute motion within region or the frame as
a whole
Object-based Tracking


Detect objects within a scene
Track object across a number of frames
My Work



Started by tracking moving objects in a
static scene
Develop a statistical model of the
background
Mark all regions that do not conform to
the model as moving object
My Work



Now working on object detection and
classification from a moving camera
Current focus is motion compensated
background filtering
Determine motion of background and
apply to the model.
Static Scene Object Detection and
Tracking




Model the background and subtract to
obtain object mask
Filter to remove noise
Group adjacent pixels to obtain objects
Track objects between frames to develop
trajectories
Background Modelling
Background Model
After Background Filtering…
Background Filtering

My algorithm based on:
“Learning Patterns of Activity using Real-Time Tracking” C.
Stauffer and W.E.L. Grimson. IEEE Trans. On Pattern
Analysis and Machine Intelligence. August 2000

The history of each pixel is modelled by a
sequence of Gaussian distributions
Multi-dimensional Gaussian
Distributions

Described mathematically as:
  X t ,  ,  

1
2 
n
2

More easily visualised as:
(2-Dimensional)
1
2
e

1
 X t   t T  1  X t   t 
2
Simplifying….


Calculating the full Gaussian for every
pixel in frame is very, very slow
Therefore I use a linear approximation
How do we use this to represent a
pixel?


Stauffer and Grimson suggest using a
static number of Gaussians for each pixel
This was found to be inefficient – so the
number of Gaussians used to represent
each pixel is variable
Weights




Each Gaussian carries a weight value
This weight is a measure of how well the
Gaussian represents the history of the pixel
If a pixel is found to match a Gaussian then the
weight is increased and vice-versa
If the weight drops below a threshold then that
Gaussian is eliminated
Matching



Each incoming pixel value must be
checked against all the Gaussians at that
location
If a match is found then the value of that
Gaussian is updated
If there is no match then a new Gaussian
is created with a low weight
Updating


If a Gaussian matches a pixel, then the
value of that Gaussian is updated using
the current value
The rate of learning is greater in the early
stages when the model is being formed
Colour Spaces



If RGB is used then the background
filtering is sensitive to shadows
The use of a colour space that separates
intensity information from chromatic
information overcomes this
For this reason the YUV colour space is
used
Colour Spaces

Background and Frame:

Channel Differences:
Isolate Objects



Groups of object pixels must be grouped
to form objects
A connected components algorithm is
used
The result is a list of objects and their
position and size
Track objects

Objects are tracked from frame to frame
using:




Location
Direction of motion
Size
Colour
The Story So Far…



Basic principle of background filtering
Stages necessary in maintaining a
background model
How it is applied to tracking
Moving Camera Sequences

Basic Idea is the same as before


Detect and track objects moving within a
scene
BUT – this time the camera is not
stationary, so everything is moving
Motion Segmentation



Use a motion estimation algorithm on the
whole frame
Iteratively apply the same algorithm to
areas that do not conform to this motion
to find all motions present
Problem – this is very, very slow
Motion Compensated Background
Filtering

Basic Principle


Develop and maintain background model as
previously
Determine global motion and use this to
update the model between frames
Advantages

Only one motion model has to be found



This is therefore much faster
Estimating motion for small regions can be
unreliable
Not as easy as it sounds though…..
Motion Models


Trying to determine the exact optical flow
at every point in the frame would be
ridiculously slow
Therefore we try to fit a parametric model
to the motion
Affine Motion Model
 u   a0   a1
      
 v   a3   a4


a2  x 
 
a5  y 
The affine model describes the vector at each
point in the image
Need to find values for the parameters that best
fit the motion present
Minimisation of Error Function

If we are to find the optimum parameters we
need an error function to minimise:
E ( x, y)  I t ( x, y)  I t 1 ( x  u, y  v)

But this is not in a form that is easy to
minimise…
Gradient-based Formulation

Applying Taylor expansion to the error function:
 u  I
I
I
I
E ( x, y )  u  v   I   
x
y
t
 v  t

Much easier to work with
Gradient-descent Minimisation


If we know how the error changes with
respect to the parameters, we can home
in on the minimum error
Various methods built on this principle:
Applying Gradient Descent
E
an

We need:

Using the chain rule:
E E u

an u an
Robust Estimation


What about points that do not belong to
the motion we are estimating?
These will pull the solution away from the
true one
Robust Estimators

Robust estimators decrease the effect of
outliers on estimation
Error w.r.t. parameters

The complete function is:
ER ER E u

an
E u an
Aside – Influence Function

It can be seen that the first derivative of the
robust estimator is used in the minimisation:
Pyramid Approach



Trying to estimate the parameters form
scratch at full scale can be wasteful
Therefore a ‘pyramid of resolutions’ or
‘Gaussian pyramid’ is used
The principle is to estimate the
parameters on a smaller scale and refine
until full scale is reached
Pyramid of Resolutions

Each level in the pyramid is half the scale of
the one below – i.e. a quarter of the area

Out pops the solution….

When combined with a suitable gradient
based minimisation scheme…
Problems with this approach

Resampling the background model:



Model cannot be too complex
Resampling will bring in errors
Motion model is only an estimate of what
is really happening

Can lead to false object detection –
particularly close to boundaries
Background Model Design


The background model needs to be robust
to these problems
We need some way to differentiate
between genuine object detections and
false ones from motion model and
background model errors
My Approach


Rather than updating model values with
current, matched values replace them
In this way resampling errors are not
allowed to accumulate
Aside – ‘Real Time’

The ability to process a sequence in realtime is dependent on THREE key factors:



The speed of the algorithm
The frame rate required
The number of pixels
Recap





Introduction to motion analysis
Principles of background modelling
Example of a static scene tracker
Discussion of motion estimation
Shortcomings when applied to background
filtering