Transcript Hardware Modelling Coding Styles

HW-SW Codesign
Lab Course Setup
Peter Tummeltshammer,
Martin Delvai
http://ti.tuwien.ac.at/ecs/teaching/courses/hwswcode_lu
1
Overview

ECS Group
Face recognition

How does it work

Lab course approach

Implementation details

Further Steps
Tummeltshammer & Delvai
HW-SW Codesign - Folie 2
Facial Recognition System

Computer-driven application

Identifies a person from a digital image

Identification vs. Verification


Identification: Comparing to a single image

Verification: Comparing to a database
Not to confuse with face detection

ECS Group
Locate a face inside an image
Tummeltshammer & Delvai
HW-SW Codesign - Folie 3
Known FR-Algorithms


2D Face Recognition

Eigenfaces

Correlation filters

Hidden Markov Model

Dynamic Link Matching
3D-Face recognition
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 4
Image Correlation

2D FFT / IFFT

Point-wise multiplication

Peak to Sidelobe Ratio (PSR) calculation
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 5
Input Images

32x32 pixels

8 bit greyscale

No/only little background

Consistent lightning

High contrast
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 6
2D FFT decomposition

Can be decomposed into 1D FFTs

1. 1D 32-point FFT of all 32 lines

2. 1D 32-point FFT of all 32 columns

Overall 64 32-point FFTs
…
…
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 7
2D FFT result


Result: Complex 32x32 Matrix
Problem for fixed-point precision:
big numerical range
real
ECS Group
Tummeltshammer & Delvai
imag
HW-SW Codesign - Folie 8
2D FFT



Decomposes image data into frequency
domain
Representation of facial feature
transitions
Constant component at (0,0)
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 9
Pointwise Multiplication

Complex point-wise multiplication

(a+bi) * (c+di)=ac+adi+bci-bd

Four Multiplications for each pixel

Overall 4096 Multiplications
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 10
2D inverse FFT (IFFT)

Similar to 2D FFT (conjugated complex)

64 32-point 1D IFFTs

Only real part necessary for PSR
calculation
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 11
Peak to Sidelobe Ratio


PSR is a measure for correlation
peak  m ean
PSR 
s tan dard deviation

5
5
21
21
32


32
ECS Group
Tummeltshammer & Delvai
1. Find Peak
2. draw 5x5 and 21x21
rectangles (sidelobe region)
3. calculate µ and σ inside
sidelobe region
HW-SW Codesign - Folie 12
Peak to Signal Ratio

High PSR = high correlation

PSR > Θ -> Person identified

Θ is application- and filter specific
PSR=38
ECS Group
Tummeltshammer & Delvai
PSR=3
HW-SW Codesign - Folie 13
Correlation in Face Recognition

Offline Generation of reference filter

One correlation filter for each person

Out of N training images

Filters are stored in frequency domain
Online
Offline
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 14
Verification Example
VerifyImage(image, filterdatabase) {
fftimage = FFT(image);
for (i=0; i<size(filterdatabase); i++) {
multimage = fftimage * filterdatabase[i];
ifftimage = ifft(multimage);
PSR = computePSR(ifftimage);
if (PSR > Θ) {
PSR 1 ≤ Θ
PSR 2 ≤ Θ
…
PSR N > Θ
return i;
}
}
return -1;
}
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 15
MACE Filter Design

Minimum Average Correlation Energy
Filter [Savvides et al.]

High illumination invariance

Generates sharp peaks in PSR analysis

Filter generation performed offline

For details: [Savvides et al.] or MATLAB
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 16
Design Path





MATLAB Model

Algorithmic
approach

Floating point
precision


CImplementation
Fixed point
precision
Provided by us



HWSW-CD
Partitioning
Timing
optimization
Todo by you
Provided by us
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 17
MATLAB Model

Available on HW-SW CD Homepage

Optional (only if interested)

Workspace (facerec.mat)


Images (10 example images)

Filter (single correlation Filter)
Functions
ECS Group

computePSR

generateFilter
Tummeltshammer & Delvai
HW-SW Codesign - Folie 18
MATLAB Model

Example command:
computepsr(real(ifft2(fft2(double(Images(:,:,:,i))) .* filterN)));



Computes PSR of filterN and Image i inside
Vector “Images“
Ifft2 and fft2 are MATLAB functions that need
double precision input
Computepsr returns PSR of given matrix
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 19
C Implementation

Available on HW-SW CD Homepage

Mandatory download


Calculates PSR for given image and
filter
Uses 16 bit fixed point representation
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 20
C Implementation


Uses fixed point representation
Challenges:




ECS Group
Most arithmetic operations can cause an
overflow
Requires scaling
Fixed scaling -> loss of precision
Pro: dynamic scaling won‘t affect the PSR
calculation due to division by standard
deviation
Tummeltshammer & Delvai
HW-SW Codesign - Folie 21
C Filestructure


Filter generated from
images 2, 4 and 9 (same
as MATLAB)
Syntax:
ECS Group

./frconsole image1.txt filter_2_4_9.txt

./frconsole image2.txt filter_2_4_9.txt

…
Tummeltshammer & Delvai
/facerec
frconsole.c
fix_fft.c
Makefile
filter_2_4_9.txt
image1.txt
image2.txt
…

Compile with make (gcc)
image10.txt
HW-SW Codesign - Folie 22
C Implementation

FFT taken from web (http://www.jjj.de/)

FFT scales automatically


PW multiplication and PSR calculation
scale dynamically
Images and filter are scaled to 16 bit
integer [-32768, 32767]
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 23
HW-SW CD Partitioning

Implemented on ALTERA FPGA, using
Softcore SPEAR

16 bit Instruction & Data Memory

No FPU, no Multiplication


ECS Group
Integration of custom HW-modules by
memory mapping
Concept presentation on 14.5.07
Tummeltshammer & Delvai
HW-SW Codesign - Folie 24
HW-SW CD Partitioning

Timing optimization by



ECS Group
Analyzing code for repetative and timeconsuming function calls
Implementing HW-modules for these
functions
Implementing whole blocks (i.e. like 2DFFT) in HW
Tummeltshammer & Delvai
HW-SW Codesign - Folie 25
Further Steps

1.: Download and understand C-Code

2.(optional): Download MATLAB design



3.: Compile C-code for SPEAR (SPEAR
concept presentation: 14.5.07)
4.: Analyze time consumption
5.: Implement HW modules and partition
design
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 26
Questions?
ECS Group
Tummeltshammer & Delvai
HW-SW Codesign - Folie 27