Transcript Foreground Background detection from video

Foreground Background
detection from video
‫אבישג אנגרמן‬:‫מאת‬
The goal: dection of moving object
Why we need this?




video surveillance.
traffic monitoring.
Human detection.
video editing.
Fusing Complementary Operators to
Enhance Foreground/Background
Segmentation
combine two probabilistic approaches:
1. Mixture of Gaussians Algorithm.
2. Statistical Background Disturbance Technique
Mixture of Gaussians Algorithm

Pixel processes –

At any time, t, what is known about a particular
pixel, {x0; y0}, is its history
Scatter plots of the red and green values of
a single pixel from the image over time.
The Algoritem





Model the values of a particular pixel as a mixture of
Gaussians.
We determine which Gaussians may correspond to
background colors-Based on the persistence and the
variance of each of the Gaussians.
Pixel values that do not fit the background
distributions are considered foreground until there is
a Gaussian that includes them.
Update the Gaussians.
Pixel values that do not match one of the pixel's
“background” Gaussians are grouped using connected
components.
Model the values of a particular pixel
as a mixture of Gaussians.

At time t we have k distributions of Gaussian for each
pixel- determined by the available memory and
computational power (Currently, 3-5 are used).

For each Gaussian we have:
-is an estimate of the weight of the ith Gaussian

in the mixture at time t -(the portion of the data
accounted for by this Gaussian)

.
- is the mean value of the ith Gaussian in the
mixture, at time t.

- covariance matrix of the ith
Gaussian in the mixture at time t.
This assumes that the red, green, and blue pixel valuesm are
independent and have the same variances.
Gaussian probability density

function.
The probability of observing the current pixel
value is:
Update the mixture model
Stage 1


Every new pixel value, Xt, is checked against the
existing K Gaussian distributions until a match
is found.
A match is defined as a pixel value within 2.5
standard deviations of a distribution.
Stage 2-No match


If none of the K distributions match the
current pixel value, the least probable
distribution is go out.
A new distribution with the current value as its
mean value, an initially high variance, and low
prior weight, is enter .
Stage 3

The prior weights of the K distributions at time
t are adjusted as follows:
The learning rate
1 for the model which
matched and 0 for the
remaining models.
Stage 4


The and parameters for unmatched
distributions remain the same.
The parameters of the distribution which
matches the new observation are updated as
follows:
The last mean
The value of the new pixel
The last variance
The distance of the new
pixel from the updated
mean.
Background Model Estimation


determine which of the Gaussians of the
mixture are most likely produced by background
processes.
We are interested in the Gaussian distributions
which have the most supporting evidence =
and the least variance.
Why?


For “background” distributions when a static,
persistent object is visible
hige weight
and relatively low variance.
New object occludes the background object
creation of a distribution or the increase in the
variance of an existing distribution
the
variance of the moving object is expected to
remain larger than a background pixel until the
moving object stops
low weight and relatively hige variance.
Background Model Estimation



the Gaussians are ordered by the value of
Then, the first B distributions are chosen as the
background model, where
T is a measure of the minimum portion of the
data that should be accounted for by the
background.
Statistical Background Disturbance
Technique

The idea- separates the brightness from the
chromaticity component.



Ei represents an expected color of a given ith
pixel, in the reference or background image.
Ii represents the color value of the pixel in a
current image.
The line OEi called expected chromaticity line.

-brightness distortion- obtained by
minimizing

represents the pixel's strength of brightness
with respect to the expected value.
Less than 1 if it 1 if the brightness of the pixel
is darker
in the current image is the same
as in the reference image
greater
than 1 if it
brighter

CD-Color Distortion- The distance between
the observed color and the expected
chromaticity line.
Background Subtraction



Background modeling - constructs a
reference image representing the background.
Threshold selection - determines appropriate
threshold values used in the subtraction
operation to obtain a desired detection rate.
pixel classication - classies the type of a given
pixel, i.e., the pixel is the part of background
(including ordinary background and shaded
background), or it is a moving object.
Background Modeling


A reference background image computed over a
number of static background frames.
Each pixel is modeled by a 4-tuple <Ei; si; ai; bi>
the expected color value.
Are the arithmetic means of the ith pixel's red, green, blue
values computed over N background frames.
standard deviation of the ith
pixel's red, green, blue values
computed over N frame of
the background frame.
Cameras typically have different sensitivities to different colors, in
order to make the balance weights on the three color bands we
normalized the pixel color by its standard deviation.
Pixel Classication or Subtraction
Operation

Original background (B): Brightness and
chromaticity similar to those of the same pixel in the
background image.

Shaded background (S): Similar chromaticity
but lower brightness.

Highlighted background (H): Similar
chromaticity but higher brightness.

Moving foreground object (F): Chromaticity
different from the expected values in the background
image.
the variation of the
brightness distortion
of ith pixel.
the variation of the
chromaticity distortion
of the ith pixel
Pixel Classication


Different pixels yield different distributions of
and CDi
In order to use a single threshold for all of the
pixels, we need to rescale the
and CDi
0
S
B
H
If pixel from a moving object contains very low RGB values, This
dark pixel will always be misclassied as a shadow.
0
F
S
B
H
Automatic Threshold Selection



First, a histogram of the
and
are constructed.
The histograms are built during background learning period.
The total sample would be NXY.



The thresholds are now automatically selected
according to the desired detection rate r.
A threshold for chromaticity distortion,
, is
the normalized chromaticity distortion value at
the detection rate of r.
In brightness distortion
is the
value at
that detection rate, and
is the
value at
the (1-r) detection rate.
Clustering Detection Elimination

The problem- those pixels have very small
variation in chromaticity distortion i.e. , very
small bi.
too big, and likely to exceed the threshold


Hence, limit the value of bi to default minimum
bi.
an optimization process is performed.

The process:
1. assigning a default minimum bi value to all
bi that are smaller than the default.
2. performing the detection on all frames.
3. compare the error rate of pixels that have bi
bigger than the default value against the
error rates of those pixels that have the
default bi value.
4. A search is performed to find the default
minimum bi that yields a balanced error
rates.
Results
http://lecturer.it.kmitl.ac.th/thanarat/research/bgs.html
Demo
Speed-Up Technique Used in Our
Implementation

Reduce number of operations at run-time:

Global S vs local si:

Screening Test:

Parallel Processing: By dividing the images into segments
and performing the operations independently on each segment.
2 Problems


New objects deposited into the scene and
become part of the background scenecan be coped with by adaptively update the
background model.
Highly specular surfaces (such as mirror, steel,
or water surface) when the color of a point on
such surfaces can change non-deterministically
Fusing Complementary Techniques
Extending the Mixture of Gaussians to Remove Shadows
and Highlights:
Our first improvement is to extend the mixture of
Gaussians approach to remove highlights and shadows.
The Enhanced Foreground/Background Selector
The extended mixture of Gaussians algorithm is joined
with the SBD technique by ANDing the results of
both operators.
Dynamic Gaussian Background Distributions
Large size for the background distributions
More background pixels but less accuracy of identifying the moving object
pixels.
Small background distribution sizes
Maintain the accuracy of the extracted moving object but will
not have the ability to contain all the background pixels




The algorithm first specifies two distribution
sizes : small and large.
Used with the small distribution size.
The pixels identified as moving objects are
tested using the statistical background
disturbance technique.
If the algorithm doesn’t agrees ,the background
distributions will be enlarged
REFERENCES

Al-Mazeed, A. H., Nixon, M. S. and Gunn, S. R. (2003)
Fusing Complementary Operators to Enhance Foreground/Background Segmentation.
In: British Machine Vision Conference 2003, 2003, Norwich.

Learning Patterns of Activity Using Real-Time Tracking C. Stauffer and W. Grimson,
IEEE TPAMI, 22(8):747–757, 2000.

A statistical approach for real-time robustbackground subtraction and shadow
detection. T. Horprasert, D. Harwood, and L. Davis. In Proceedings IEEE ICCV’99,
pages 1–19, 1999.
The end…