Transcript The Bayesian Image Retrival System,PicHunter

The Bayesian Image Retrieval
System,PicHunter
Theory, Implementation, and
Psychophysical Experiments
Introduction
Relevance feedback
—— users give additional information
Main idea:
With an explicit model of a user’s
actions, assuming a desired goal,
PicHunter uses Bayes’ rule to predict
the goal image, given their actions
Nature of search
Target-specific search (Target search)

exact match
Category search

same category is ok
Open-ended search (browsing)
Bayes’ formula
P( Fj | E ) 
P( E | Fj ) P( Fj )
n
 P( E | F ) P( F )
i
i
i 1
Fj – hypothesis (Target image is j)
E – experiment (user’s response behavior)
Show us how the correctness of a hypothesis
change after carrying out an experiment
How to model P(E|Fj)?
Theoretical basis for
PicHunter
During each session
a set Dt of ND images, Action At
H t ----History of the session
PT  Ti | H t  

PH t | T  Ti P(T  Ti )
P( H t )
PH t | T  Ti P(T  Ti )
 PH
n
j 1
t
| T  T j P(T  T j )
User Model:Assessing Image
similarity
Key term:
P(At|T=Ti,Dt,U)
U:specific user
Purpose:update the probability of each
Ti being target
Relevance feedback
e.g. 2AFC (two-alternative forced-choice)
Given two image, user need to choose which
one is similar to target
P(E|Fj)  P(A=1|X1,X2,T=Ti)



1
0.5
0
if d(X1,Ti) < d(X2,Ti)
if d(X1,Ti) = d(X2,Ti)
d(X1,Ti) > d(X2,Ti)
Another one is relative distance
Relative distance measure
using the pictorial features distance
as the form of the probability
When ND=2, At=1 or 2
Psigmoid(A=1|X1,X2,T)
=
1
1  exp(d ( X 1 , T )  d ( X 2 , T ) /  )
Pictorial features
HSV-HIST

Hue, Saturation, Value histogram
HSV-CORR
RGB-CCV

Color histogram
Display Updating Model
Most-Probable Display Updating Model

Give the most similar one for user to
choose
Most-informative Display Updating
Model
n
C[P(T)]   P(T  Ti ) log P(T  Ti )
i 1

Give both similar and dissimilar images for
use to choose
Results
Cox formulated an experiment XYZ

X - with memory or with out



Use all the response or just response in one iteration
Y - with using relative / absolute distance measure
Z – use pictorial or semantic measure
Benchmark - how many images need to be
displayed before target is found
MRS is the best

With memory, use relative distance and semantic
measure