Transcript PPT

Advance Information Retrieval
Topics
Hassan Bashiri
Information Filtering
Agenda
•
•
•
•
Information filtering
Automatic profile learning
Social filtering
Training Strategies
Information Need
Information Access Problems
Different
Each Time
Stable
Retrieval
Data
Mining
Stable
Collection
Filtering
Different
Each Time
Indexing and Complexity
Agenda
• Inverted indexes
• Computational complexity
Inverted File
A
B
AI
AL
BA
BR
C
D
F
G
J
L
M
N
O
P
Q
T
TH
TI
Term
Doc 1
Doc 2
Doc 3
Doc 4
Doc 5
Doc 6
Doc 7
Doc 8
An Example
Postings
aid
all
back
brown
come
dog
fox
good
jump
lazy
men
now
over
party
quick
their
time
0
0
1
1
0
0
0
0
0
1
0
0
1
0
1
1
0
4, 8
2, 4, 6
1, 3, 7
1, 3, 5, 7
2, 4, 6, 8
3, 5
3, 5, 7
2, 4, 6, 8
3
1, 3, 5, 7
2, 4, 8
2, 6, 8
1, 3, 5, 7, 8
6, 8
1, 3
1, 5, 7
2, 4, 6
0
1
0
0
1
0
0
1
0
0
1
1
0
0
0
0
1
0
0
1
1
0
1
1
0
1
1
0
0
1
0
1
0
0
1
1
0
0
1
0
0
1
0
0
1
0
0
0
0
0
1
0
0
0
1
0
1
1
0
0
1
0
0
1
0
0
1
0
0
1
0
0
1
0
0
1
0
0
0
1
0
1
0
0
1
0
0
1
1
0
0
1
0
0
1
0
0
1
0
0
1
0
1
0
0
0
1
0
0
1
0
0
1
1
1
1
0
0
0
The Finished Product
Inverted File
A
B
AI
AL
BA
BR
C
D
F
G
J
L
M
N
O
P
Q
T
TH
TI
Term Postings
aid
all
back
brown
come
dog
fox
good
jump
lazy
men
now
over
party
quick
their
time
4, 8
2, 4, 6
1, 3, 7
1, 3, 5, 7
2, 4, 6, 8
3, 5
3, 5, 7
2, 4, 6, 8
3
1, 3, 5, 7
2, 4, 8
2, 6, 8
1, 3, 5, 7, 8
6, 8
1, 3
1, 5, 7
2, 4, 6
Cross-Language
Information Retrieval
Agenda
• Cross-language IR
– Controlled vocabulary
• Automatic indexing
– Free text
– Evaluation
– User interface design
What is CLIR?
Users enter their query in one
language and the search engine
retrieves relevant documents in
other languages.
English Query
French Documents
Retrieval System
11
Cross-Language Text Retrieval
Query Translation
Document Translation
Text Translation
Controlled Vocabulary
Vector Translation
Free Text
Knowledge-based
Corpus-based
Ontology-based Dictionary-based
Term-aligned Sentence-aligned Document-aligned Unaligned
Thesaurus-based
Parallel
Comparable
11
Retrieval System Interfaces
Agenda
•
•
•
•
Query interface
Selection interface
Examination interface
Document delivery
Retrieval System Model
User
Query
Formulation
Detection
Selection
Index
Examination
Indexing
Docs
Delivery
Query Formulation
User
Query
Formulation
Detection
Index
Starfield
NLP in IR
The Different Levels of
Language Analysis
1-Phonetic or Phonological Level
2-Morphological Level
3-Syntactic Level
4-Semantic Level
5-Discourse Level
How Information Retrieval
Works
Step 1: Document Processing
Step 2: Query Processing
Step 3: Query Matching
Step 4: Ranking & Sorting
Intelligent Information
Retrieval
or
Knowledge Based IR
What Is Different From IR?
•
•
•
•
IR is more concerned with words and concepts.
IIR or KBIR is more concerned about relations.
Most of IR models assume term independence.
IIR or KBIR acknowledges existence of
relationships.
• IR more suitable for large scale and general retrieval
• IIR or KBIR more suitable for domain specific tasks.
Knowledge Based IR
IIR-KBIR
•
•
•
•
•
•
•
Expectation or Interaction With User
Objects
KB
Relation Between the objects
Reasoning
Learning
Relation Extraction
Experiments in Farsi Retrieval
Retrieval Models Investigated
• Fuzzy Logic
– MMM, Paice
•
•
•
•
Vector Space
Probabilistic, BM25
N-Grams
Combinational