Transcript Vehicle Tracking - University of Haifa

Vehicle Movement Tracking
Written by Goldberg Stanislav
for Vision Topics Seminar
Present Day Traffic Management
1) Magnetic Loop Detectors
2) Video Monitoring Systems
Magnetic Loop Detectors
http://micro.magnet.fsu.edu/electromag/java/detector/index.html
Magnetic Loop Detectors
Pros:
 Accurate counting
 Stable under different lighting and traffic conditions
Cons:
• Costly : require digging up the road surface
• Unable to provide additional traffic parameters
Video Monitoring Systems
Video Monitoring Systems
Pros:
 Vehicle counts and speeds
 Vehicle classification: { Bus, Truck, Bike, Car }
 Lane changes
 Acceleration/Deceleration
 Queue length for traffic jams
 Less costly to install then magnetic loop detectors
Cons:
• Problems with congestion ( vehicles occlusion )
• Long shadows linking vehicles together
• Transition between day and night
Tracking Requirements
1) Automatic segmentation of a vehicle from a
background and other vehicles so there can be a
unique track associated with each vehicle
2) Deal with variety of vehicles – motorcycles, passenger
cars, buses, construction equipment, trucks, etc.
3) Deal with a range of traffic conditions – light midday
traffic, rush-hour congestion, varying speeds in
different lanes.
4) Deal with variety of lighting conditions – day, evening,
night , sunny, overcast, rainy days.
5) Real-time operation of the system
Vehicle Tracking Approaches
1) 3D Model based
2) Region based
3) Active contour based
4) Feature based
3D Model based tracking
Image view is aligned with a detailed 3D model of each
vehicle
Pros:
 Easy vehicle classification { Bus, Truck, Bike ..}
Cons:
• Memory and processing consuming approach
• Unrealistic to expect to be able to have detailed
models for all vehicles that are found on the roadway
Region based tracking
Every connected region in the image – “a blob”
associated with each vehicle and then tracked over time
using cross correlation. The “blobs” are found by the
means of background extraction .
Pros:
 Works well in free flowing traffic conditions
Cons:
• Partial occlusion under congested traffic
conditions leads to grouping of several vehicles
into one “blob”
Active contour based tracking
Representing vehicle by bounding contour of
the object and dynamically update it during
the tracking
Pros:
Reduced computational complexity
compared to region based approach
Cons:
• Partial occlusion is still a problem.
Feature based tracking
Tracking not a object as a whole, but subfeatures such as distinguished points or lines on
the object.
Pros:
Partial occlusion is not a problem: some of the
sub futures remains visible
Cons:
• Additional problem to solve : which set of sub
features belong to one object (grouping)
Motion Based Grouping
• Based on a common motion constraint aka
“common fate” : sub-features that are moving
rigidly are grouped together into a single
vehicle.
• The grouping must be sensitive enough to pick
up even slight acceleration or lane drift to
distinguish a vehicle from the neighbors
• Spatial proximity (sub-features spatially close
one to another ) must be taken into
consideration
Motion Based Grouping:
Why it’s good for vehicle tracking?
• For congested traffic vehicles are constantly
changing their velocities to adjust to near by
traffic, thus giving the grouper the information
it needs to perform the segmentation.
• For free flowing traffic, vehicles more likely to
maintain constant speed with almost no lane
drift, making “common fate” grouping less
useful, but there is more space between
vehicles.
The Algorithm
( David Beymer et al )
1)
2)
3)
4)
5)
6)
Off-line camera definition
Features detection
Features tracking
Features grouping
Obtaining traffic parameters
Vehicle Classification
The Algorithm
Off-line camera definitions (1)
Line correspondence for the homography
A projective transform H , or homography, is used to map from image
coordinates (x,y) to world coordinates (X,Y)
Off-line camera definitions (1)
Line correspondence for the homography
H – linear transformation { rotation, scaling, shear, reflection }
H
X
Y
cos
sin
0
x cos
y cos
sin
cos
0
x0
y0
1
y sin
x sin
x0
y0
(x,y) rotated clockwise by
theta angle and translated
by (x0,y0), no scaling
Off-line camera definitions(2)
Detection regions
Stop Detection
Area
Start Detection
Area
Off-line camera definitions (3)
Fiducial points for camera stabilization
Features detection
• Corner features are chosen as sub-features
• Corner detector is based on image gradient
Features detection
• Horizontal ( x axis) differentiation can be
approximated by
• Vertical ( y axis )
© stolen from Hagit’s Image Processing slides
Features detection
Features tracking:
Kalman Filter Predictor
The position and velocity of the vehicle is described by the linear state
space:
We assume that between the (k − 1)th and kth timestep the vehicle
undergoes a constant acceleration of ak that is normally distributed, with
mean 0 and standard deviation σa. From Newton laws of motion we
conclude that:
xk
xk
1
xk t
xk
xk
1
ak t
ak t2
x
2
v
x0
vt
v0
at
a t2
2
Features tracking
• Kalman filtering predicts the area where to
search for each corner feature
• Corner feature is found
• The distance between predicted and
measured feature is computed
• If the distance is above threshold the track is
rejected
Features tracking
Grouping
• The central principle: common motion
• When sub-feature is detected , it initially connected
to all neighboring tracks within certain radius
• For all joined pairs of tracks pa(t), pb(t), relative
displacement d(t)= pa(t)- pb(t) is calculated
• For each frame d value is computed for each edge
and edge is broken if either
maxtdx(t)- mintdx(t)> thresholdx
maxtdy(t)- mintdy(t)> thresholdy
• Shadow sub-features tend to be unstable over time,
so grouping eliminates them.
Grouping
Grouping : problems
• Oversegmentation
one vehicle is grouped into several segments
• Overgrouping
several vehicles are grouped into one segment
Real Time: hardware
Real Time: hardware
PC – Intel Pentium 150 Mhz
 160 MFLOPS, 260 MIPS
C40 – TMS320C40, Texas Instruments Floating-Point
Digital Signal Processor
 40 MFLOPS, 20 MIPS
Present PC - Intel Core i7-920 Quad Core Processor
 63000 MFLOPS, 76000 MIPS
Results
Length in min:sec
G – number of reported vehicles (by algorithm )
N – number of actual vehicles (counted by human)
Results
• True match: A one to one match between ground truth and a group
• False negative: An unmatched ground truth
• Oversegmentation: A ground truth that is matched by more then one
group
• False positive: An unmatched group
• Overgrouping: A group that matches more then one ground truth
Results
Start
End
Results: flow scatter plot
Results: velocity scatter plot
The end