Transcript Document

Astrophysics Applied
What to do with Images and Wavelets
Plato's Republic, Book vii:
Socrates: Shall we set down astronomy among the subjects of study?
Glaucon: I think so, to know something about the seasons, the months and the years
is of use for military purposes, as well as for agriculture and for navigation.
Socrates: It amuses me to see how afraid you are, lest the people should accuse you
of recommending useless studies.
Socrates goes on to say that the use of astronomy is not to add to the vulgar comforts
of life, but to assist in raising the mind to the contemplation of things which are to be
perceived by the pure intellect alone.
What this is about – a history
 1984: Started a small hi-tech venture – to offset the stress
of the academic job situation (!). The venture focused on
CCTV imaging on the then-new PC platform.
 1988: BBC TV Tomorrow’s World
 1995 started work in earnest after winning UK government
contracts
 2003 developed leading video acquisition, storage and
retrieval system based on wavelet methodology, modern
statistical methods and nonlinear image processing
algorithms.
 Currently happily spend a week per month at the Kapteyn
Institute in Groningen: mind-to-mind resuscitation.
28th. June 2006
Bernard60 - Valencia, Spain
2
Achievements
 Best real-time video compression algorithm based
on adaptive wavelet technology: up to 9 days on
one DVD!
 Sophisticated video event detection through
adaptive scene analysis.
 Automated video synopses based on event
detection allows searching terra-bytes in minutes
 Earning a reasonable living without having to
suffer the slings and arrows of arbitrary high
level funding decisions – independence!!
 Even more stress and anxiety
28th. June 2006
Bernard60 - Valencia, Spain
3
Image processing
Classical
Modern
 Static images
 Moving image sequences
 Weiner filters
 Nonlinear reconstruction
 Fourier transforms
 Wavelet transforms
Astronomy
TV Imaging
 Static images (generally)
 Dynamic images
 Huge dynamic range
 Limited dynamic range
 Measured quantities have
important physical meaning
 Data is not photometric
28th. June 2006
Bernard60 - Valencia, Spain
4
CCTV – Closed circuit television
 The UK has some 8x106 CCTV cameras
–
–
–
–
–
10% of world total
Main detector is small CCD (mostly colour)
Sensitivity down to 10-5 lux (mono cameras)
Resolution at best 576x704
Output is analogue
starlight
0.00005 lux
Moonlight
1 lux
Office light
100 lux
midday
100,000 lux
(1 footcandle = 10.76 lux)
 Future is to go to HDTV and digital IP delivery
– Resolution 1024x768 (p) or 1280x1024 (i)
– Data format MPEG-4 (ideally AVC – level 10)
Note that even the new HDTV has nowhere near the 5 megapixel resolution of a modest still digital camera.
28th. June 2006
Bernard60 - Valencia, Spain
5
The CCTV data nightmare
Some numbers …
Traditional strategies for handling this …

One camera generates 30 Megabytes of data
per second.

A small 100 camera system delivers 3
Gigabytes per second or 1 Petabyte per year

All CCTV cameras in the UK generate about
100 Exabytes per year

There is no way of storing all this, let alone
looking at it or searching for anything.

Cut down on frame rate (time-lapse)

Save only 1-2 weeks

Compress while trying to retain quality

Save only “interesting” data

Such strategies may compromise data
integrity if data is eventually to be
used as part of a legal investigation.
G=109, T=1012,P=1015, E=1018,Z=1021,…,W=1030
Library of Congress = 10 Tbytes
While acquiring the images
Improved search methodology:
1.
Do far better compression
1.
2.
Analyze the data
Cross-correlate what is seen on
different cameras
3.
Enhanced real time scene analysis
2.
Learn from previous searches
4.
Store data synopses
3.
Provide a search query mechanism
28th. June 2006
Bernard60 - Valencia, Spain
6
Acquiring video pictures to PC
Analogue to digital
converters
High performance
digital signal processor
(VLIW multi-core
4 billion instructions
per second)
Analogue video in
(16 channels)
One or more of
these sit inside a
“PC” running Linux
We design and
manufacture these.
This is the 6th in 20
years.
Digital data output
Sent to PC processor
The video frame grabber converts analogue video signals into digital data array that can be
processed. Much depends on the quality of this device: this one supports broadcast quality.
28th. June 2006
Bernard60 - Valencia, Spain
7
Typical site monitoring
28th. June 2006
Bernard60 - Valencia, Spain
8
One dimensional wavelet filter
Reconstruction of data S0 from multiresolution analysis {Sj, Dj, Dj-1}.
Schematically:
The yellow bits
at level j
Scaling function
at level j
(orthogonal to
the wavelet)
f ( x)   s j [k ]  j , k ( x)
k
Levels used in
reconstruction

L j
d
jL
[k ]  j  L , k ( x)
k
The red bits.
Contributions
come from all
levels, L
Wavelet Function at
level L
You don’t have to use the same wavelets at all levels, and nor do you have to keep all the {dj}.
The choices can be made to depend on image content and dynamics. This is a key trick.
28th. June 2006
Bernard60 - Valencia, Spain
9
Multi-resolution hierarchy
FINISH
3
SS
3
SD
2
Level 3
3
DS
3
SD
DD
1
2
SD
2
SS
SD
2
Level 2
2
2
DS
DD
2
DS
DD
1
1
SS
SD
1
DS
1
DD
Level 1
1
Level 0
START
28th. June 2006
1
DS
Original Image
DD
The image is shrunk by linear factors
of two. The residuals that would
allow the picture at any level to be
reconstructed from the higher levels
are stored alongside.
The shrinking process is important,
but need not use wavelets.
Bernard60 - Valencia, Spain
10
Image representation - wavelets
Rien van de Weygaert
The scene analysis might as well be done at the
same time as the censorship and compression of
the data. That way the compression can adapt
locally to scene content.
Censorship of
coefficients for
compression
Two level wavelet
transform
Images have been enhanced for clarity
28th. June 2006
Bernard60 - Valencia, Spain
11
Wavelet compression vs. MPEG
Wavelet compressed frame
from CCTV video sequence.
MPEG-1 compressed frame from same
video sequence. Note the blocky artefacts
and colour contouring.
The goal is to achieve high levels of compression without sacrificing image quality.
This is particularly difficult when there is a lot of movement in the scene.
Using our locally adaptive wavelets makes a huge difference over normal wavelets.
28th. June 2006
Bernard60 - Valencia, Spain
12
Sunlight, shadows & trees

This is full-screen video event
detection. No areas have been
excluded.

Note the moving shadows and
trees, together with varying
illumination. These do not
produce alarms.

The first detection is in fact
behind the window before walking
through the door.

The picture turns grey on
detection for demo purposes only:
we need to show colour blocks.

This is an MPEG-1 version of an
Astraguard wavelet sequence
The main issue is to detect what you want to detect without missing anything,
while at the same time avoiding things you do not want like sunlight, shadows and
trees. This sequence demonstrates some successes. There are also some failures
which can in fact be handled with a little more algorithm refinement.
28th. June 2006
Bernard60 - Valencia, Spain
13
Application Areas
The goal is to provide equipment
to handle video streams in
situations where an understanding
of what is taking place in the
scene, or has taken place at an
earlier time, is relevant.
The system must be easy to
configure and even easier to use.
Application areas are often
mission-critical, so close to 100%
reliability is essential.
 Large government and
industrial sites
 Hospitals
 Police Work
 Fire detection on gas rigs
 Oil Pipelines
 Power line monitoring
 High value premises
Systems range from one camera to many hundreds of cameras. Upper limit
is in the 1000’s of cameras.
28th. June 2006
Bernard60 - Valencia, Spain
14
What next?
 Retire to a vineyard in Australia?
 Write the book that has taken over 10 years to
not finish.
 Continue to share in the excitement of cosmology.
 Keep healthy if not fit
28th. June 2006
Bernard60 - Valencia, Spain
15
“frequency”
amplitude
Signal recognition using wavelets
Time
The same signal (top) is analyzed using two different wavelets.
The Gabor wavelet is capable of localizing the signal in both frequency and time.
28th. June 2006
Bernard60 - Valencia, Spain
16
“frequency”
amplitude
Signal recognition using wavelets
Time
The same signal (top) is analyzed using two different wavelets.
The Gabor wavelet is capable of localizing the signal in both frequency and time.
28th. June 2006
Bernard60 - Valencia, Spain
17