Transcript PowerPoint

Discussion Class 11
Click through Data as Implicit
Feedback
1
Discussion Classes
Format:
Question
Ask a member of the class to answer.
Provide opportunity for others to comment.
When answering:
Stand up.
Give your name. Make sure that the TA hears it.
Speak clearly so that all the class can hear.
Suggestions:
Do not be shy at presenting partial answers.
Differing viewpoints are welcome.
2
Question 1: Training data
(a) Why is training data important in tuning a search
engine?
(b) Explain the difference between the use of "explicit
relevance data" and "implicit feedback."
(c) What is "click through data" and why might it be
useful?
3
Question 2: Eye tracking
(a) What is eye tracking?
(b) Define "fixation" and "saccade"?
(c) How are they measured?
4
Question 3: Explicit relevance
judgments
The method for obtaining explicit relevance judgments was very
different from the TREC methodology. Explain the difference in:
(a) Selection of assessors
(b) Documents reviewed by assessors
(c) Task required of assessors for each document
(d) Technique for ensuring consistency between assessors
Are these differences suitable for this experiment?
5
Question 4: Regular Google ranks
How do the fixations differ from the click?
6
Question 5: Relevance, ranking, and
click through data
(a) When the rankings were reversed, how did the user
behavior change?
i) Average rank of pages clicked
ii) Number of pages clicked
(b) The evidence that user's click through behavior is
influenced by trust in Google was studied by a
statistical experiment.
i) What was the hypothesis?
ii) What was the methodology?
iii) What was the conclusions?
7
Question 6: Clicks as relative
relevance judgments
Section 5.3 looks at a number of relative measures:
click > skip above
last click > click above
click > click earlier
last click > click previous
click > no-click next
(a) What are these tests attempting to measure?
(b) Explain the rationale behind: click > skip above.
8
Question 7:
9
Question 8:
10
Question 9:
11