Transcript Recommender Systems and Search engines – two sides of the
R
ECOMMENDER
S
YSTEMS AND
S
EARCH ENGINES
–
TWO SIDES OF THE SAME COIN
!?
Bracha Shapira Lior Rokach Department of Information Systems Engineering Ben-Gurion University
C
ONTENT
Introduction Applications Methods Recommender Systems vs. search engines
A
RE
Y
OU
B
EING
S
ERVED
?
What are you looking for?
Demographic – Age, Gender, etc.
Context Casual/Event Season Gift Purchase History Loyal Customer What is the customer currently wearing?
Style Color Social Friends and Family Companion
4
R
ECOMMENDER
S
YSTEMS
A recommender system (RS) helps people that have not sufficient personal experience or competence to evaluate the, potentially overwhelming, number of alternatives offered by a Web site.
In their simplest form RSs recommend to their users personalized and ranked lists of items Provide consumers with information to help them decide which items to purchase
E
XAMPLE APPLICATIONS
6
W
HAT BOOK SHOULD
I
BUY
?
7
W
HAT MOVIE SHOULD
I
WATCH
?
• The Internet Movie Database (IMDb) provides information about actors, films, television shows, television stars, video games and production crew personnel.
• Owned by Amazon.com since 1998 • 796,328 titles and 2,127,371 people • More than 50M users per month.
abcd The Nextflix prize story
In October 2006, Netflix announced it would give a $1 million to whoever created a movie-recommending algorithm 10% better than its own.
Within two weeks, the DVD rental company had received 169 submissions, including three that were slightly superior to Cinematch, Netflix's recommendation software After a month, more than a thousand programs had been entered, and the top scorers were almost halfway to the goal But what started out looking simple suddenly got hard. The rate of improvement began to slow. The same three or four teams clogged the top of the leader-board.
Progress was almost imperceptible, and people began to say a 10 percent improvement might not be possible.
Three years later, on 21 st of September 2009, Netflix announced the winner.
24.04.2020
8
9
W
HAT NEWS SHOULD
I
READ
?
10
W HERE SHOULD I SPEND MY VACATION
?
Tripadvisor.com
I would like to escape from this ugly an tedious work life and relax for two weeks in a sunny place. I am fed up with these crowded and noisy places … just the sand and the holiday … it should not be to expensive. I prefer mountainous places… not too far from home. Children parks, easy paths and good cuisine are a culture. I would like to be fascinated by the people and learn to look at my life in a totally different way.
Usage in the market/products Recommendation State-of-the-art solutions
Method Collaborative Filtering Content-Based Techniques Knowledge-Based Techniques Stereotype-Based Recommender Systems Ontologies and Semantic Web Technologies for Recommender Systems Hybrid Techniques Ensemble Techniques for Improving Recommendation Context Dependent Recommender Systems Conversational/Critiquing Recommender Systems Community Based Recommender Systems and Recommender Systems 2.0
Commonness Examined Solutions
Jinni Taste Kid Nanocrowd Clerkdogs Criticker IMDb Flixster Movielens Netflix
v v v v v v v v v v v v v v v v v v v v v v
Shazam Pandora LastFM YooChoose Think Analytics Itunes Amazon
v v v v v v v v v v v v v v v v v v v v v v v v v v v v v v future v v v v v v v v v
24.04.2020
11
C OLLABORATIVE F ILTERING
Collaborative Filtering
The method of making automatic predictions (filtering) about the interests of a user by collecting taste information from many users (collaborating). The underlying assumption of CF approach is that those who agreed in the past tend to agree again in the future.
1 Collaborative Filtering kNN - Nearest Neighbor SVD – Matrix Factorization Similarity Weights Optimization (SWO) 24.04.2020
13
C
OLLABORATIVE
F
ILTERING
abcd The Idea
Trying to predict the opinion the user will have on the different items and be able to recommend the “best” items to each user based on: the user’s previous likings and the opinions of other like minded
users Negative Rating ?
Positive Rating
24.04.2020
14
How collaborative filtering works?
“People who liked this also liked…”
User to User
abcd How it works
Item to Item
abcd User-to-User
Recommendations are made by finding users with similar tastes. Jane and Tim both liked Item 2 and disliked Item 3; it seems they might have similar taste, which suggests that in general Jane agrees with Tim. This makes Item 1 a good recommendation for Tim.
This approach does not scale well for millions of users.
Item-to-Item
Recommendations are made by finding items that have similar appeal to many users. Tom and Sandra are two users who liked both Item 1 and Item 4. That suggests that, in general, people who liked Item 4 will also like item 1, so Item 1 will be recommended to Tim. This approach is scalable to millions of users and millions of items.
24.04.2020
15
K
NN - N
EAREST
N
EIGHBOR
Current User 1 1 st item rate 0 Dislike 1 ?
Like Unknown User Model = interaction history We are looking for the Nearest Neighbor. The one with the 0 1 1 1 1 lowest Hamming 0 distance.
0 1 1 0 1 1 This user did not The prediction rate the item. We was made based will try to predict a rating according neighbor.
to his neighbors.
There are other users who rated the same item. We are interested in the Nearest Neighbors.
after Richard Hamming. In information theory, the Hamming distance between two strings of equal length is the number of positions at which the corresponding symbols are different.
14 th item rate Hamming distance 5 6 6 5
abcd
Nearest Neighbor 4 8 24.04.2020
16
IMPORTANT ISSUES
Cold Start Implicit/Explicit Rating Sparsity Long Tail problem - many items in the Long Tail have only few ratings Portfolio Effect: Non Diversity Problem It is not useful to recommend all movies by Antonio Banderas to a user who liked one of them in the past Beyond Popularity Gray sheep problem Iformation Security Misuse Privacy
C ONTENT -B ASED R ECOMMENDER S YSTEM
19
C
ONTENT
-B
ASED
R
ECOMMENDATION
In content-based recommendations the system tries to recommend items that matches the User Profile. The Profile is based on items user has liked in the past or explicit interests that he defines.
A content-based recommender system matches the profile of the item to the user profile to decide on its relevancy to the user.
S
IMPLE
E
XAMPLE
Read update User Profile
20
New books Match User Profile Recommender Systems recommendation
C ONTEXT BASED RECOMMENDER SYSTEMS
Context-Based Recommender Systems
abcd Overview
The recommender system uses additional data about the context of an item consumption.
For example, in the case of a restaurant the time or the location may be used to improve the recommendation compared to what could be performed without this additional source of information.
A restaurant recommendation for a Saturday evening when you go with your spouse should be different than a restaurant recommendation on a workday afternoon when you go with co-workers 24.04.2020
22
Context-Based Recommender Systems
Motivating Examples
Recommend a vacation Winter vs. summer Recommend a purchase (e-retailer) Gift vs. for yourself Recommend a movie To a student who wants to watch it on Saturday night with his girlfriend in a movie theater.
24.04.2020
23
Context-Based Recommender Systems
Motivating Examples
Recommend music The music that we like to hear is greatly affected by a context, such that can be thought of a mixture of our feelings (mood) and the situation or location (the theme) we associate it with.
Listen to Bruce Springteen "Born in USA" while driving along the 101.
Listening to Mozart's Magic Flute while walking in Salzburg.
24.04.2020
24
Information Discovery: Example “Tell me the music that I want to listen NOW"
abcd Musicovery.com
abcd Details
An Interactive personalized WebRadio A mood matrix propose a relationship between music and mood.
Ethnographic studies have shown that people choose music peaces according to their mood or mood change expectation.
24.04.2020
25
Context-Based Recommender Systems
What simple recommendation techniques ignore?
What is the user when asking for a recommendation?
Where (and when) the user is ? What does the user a product)?
(e.g., improve his knowledge or really buy Is the user Are there or with other ?
products to choose or only ?
24.04.2020
26
Context-Based Recommender Systems
What simple recommendation techniques ignore?
What is the user when asking for a recommendation?
Where (and when) the user is ? What does the user a product)?
(e.g., improve his knowledge or really buy Is the user Are there or with other ?
products to choose or only ?
Plain recommendation technologies forget to take into account the user context.
24.04.2020
27
Context-Based Recommender Systems
abcd Major obstacle for contextual computing
Obtain sufficient and reliable data describing the user context Selecting the right information, i.e., relevant in a particular personalization task Understand the impact of contextual dimensions on the personalization process Computational model the contextual dimension in a more classical recommendation technology For instance: how to extend Collaborative Filtering to include contextual dimensions?
24.04.2020
28
Context-Based Recommender Systems
abcd Item Split - Intuition and Approach
Each item in the data base ( ) is a candidate for splitting Context defines ( ) all possible splits of an item ratings vector We test all the possible splits – we do not have many contextual features We choose one split (using a single contextual feature) that maximizes an impurity measure and whose impurity is higher than a threshold 24.04.2020
29
S OCIAL ( TRUST ) BASED RECOMMENDER SYSTEMS
Social Based (Trust based) Recommender Systems
abcd Overview
.
Intuition – Users tend to receive advice from people they trust, i.e., from their friends.
Trusted friends can be defined explicitly by the users or inferred from social networks they are registered to.
24.04.2020
31
T
RUST
-
BASED
C
OLLABORATIVE
F
ILTERING
Active users’ trusted friends
Active user
Rating prediction
T
RUST METRICS
Global metrics: computes a single global trust value for every single user (reputation on the network) 1 b 3 Pros: Based on the whole community opinion a Cons: Trust is subjective (controversial users) 2 c 3 3 d
T
RUST METRICS
(
CONT
.)
Local metrics: predicts (different) trust scores that are personalized from the point of view of every single user Pros: More accurate Attack resistance Cons: Ignoring the “ wisdom of the crowd ” 1 b a ?
2 c 5 3 d
S EARCH ENGINES AND RECOMMENDER SYSTEMS
S EARCH E NGINES VS . R ECOMMENDER S YSTEMS –
Search Engines
Goal – answer users ad hoc queries Input – user ad-hoc need defined as a query Output - ranked items relevant to user need (based on her preferences???) Methods - Mainly IR based methods
Recommender Systems
Goal – recommend services or items to user Input - user preferences defined as a profile Output - ranked items based on her preferences Methods – variety of methods, IR, ML, UM
N
EW TRENDS
…
“Understand” the user actual needs from her context Personalize results according to the user preferences
Search engines may use some recommender systems methods to achieve these goals
S EARCH E NGINES P ERSONALIZATION ADOPTED FROM RECOMMENDER SYSTEMS M ETHODS Collaborative filtering User-based - Cross domain collaborative filtering is required???
Content-based Search history Collaborative content-based Collaborate on similar queries Context-based Little research – difficult to evaluate Locality, language, calendar Social-based Friends I trust relating to the query domain Notion of trust, expertise