Mining and Summarizing Customer Reviews

Transcript Mining and Summarizing Customer Reviews

KDD-2012 Summer School, August 10, 2012, Beijing, China

Modeling Opinions and Beyond in Social Media

Bing Liu University Of Illinois at Chicago [email protected]

Introduction



Why are opinions so important?

 Opinions are key influencers of our behaviors.  Our beliefs and perceptions of reality are conditioned on how others see the world.   Whenever we need to make a decision we often seek out others’ opinions.  True for both individuals and organizations It is simply the “human nature”   We want to express our opinions We also want to hear others’ opinions Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 2

Topics of this lecture

 Sentiment analysis and opinion mining  It has been studied extensively in the past 10 years. A large number of applications have been deployed.  We will define/model this task and introduce some core research and challenges.  Going beyond: comments, discussions/debates  Beyond expressing our opinions in isolation, we also like to comment, argue, discuss and debate.  They involve user interactions.

 These are opinions too but of a slightly different type  We will try to model some of these interactive forums Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 3

Roadmap

  

Sentiment Analysis and Opinion Mining

 Problem of Sentiment Analysis      Document sentiment classification Sentence subjectivity & sentiment classification Aspect-based sentiment analysis Mining comparative opinions Opinion spam detection Beyond Sentiments  Modeling review comments  Modeling discussions/debates Summary Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 4

Sentiment analysis and opinion mining

   Sentiment analysis or

opinion mining

 computational study of opinions, sentiments, appraisal, and emotions expressed in text.  Reviews, blogs, discussions, microblogs, social networks Its inception and rapid growth coincide with those of the social media on the Web  For the first time in human history, a huge volume of opinionated data is recorded in digital forms.

core technology

for social media analysis  Because a key function of social media is for people to express views & opinions Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 5

A fascinating and challenging problem!

 Intellectually challenging & many applications .

   A popular research topic in NLP, text and Web mining (Edited book: Shanahan, Qu, & Wiebe, 2006; Book Chapters: Liu, 2007 & 2011; Surveys: Pang & Lee 2008; Liu, 2012) It has spread from computer science to management science and social sciences (Hu, Pavlou & Zhang, 2006; Archak, Ghose & Ipeirotis, 2007; Liu et al 2007; Park, Lee & Han, 2007; Dellarocas et al., 2007; Chen & Xie 2007).

> 350 companies working on it in USA.

 Almost no research before early 2000.

 Either from NLP or Linguistics (no data?)  Potentially a major technology from NLP.  But it is very hard!  People grossly underestimated the difficulty earlier.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 6

Roadmap

   Sentiment Analysis and Opinion Mining  Sentiment Analysis Problem      Document sentiment classification Sentence subjectivity & sentiment classification Aspect-based sentiment analysis Mining comparative opinions Opinion spam detection Beyond Sentiments  Modeling review comments  Modeling discussions/debates Summary Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 7

Abstraction (1): what is an opinion?

  Find a structure from the unstructured text.

Id: Abc123 on 5-1-2008

ago. It is such a nice

“

I bought an phone. The iPhone a few days touch screen is really cool . The voice quality is clear too. It is much better than my old Blackberry . However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive , …”

 One can look at this review/blog from    Document level , i.e., is this review + or -? Sentence level , i.e., is each sentence + or -? Entity and feature/aspect level Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 8

Entity and feature/aspect level



Id: Abc123 on 5-1-2008

ago. It is such a nice

“

 What do we see?

    Opinion targets: entities and their features/aspects Sentiments: positive and negative Opinion holders: persons who hold opinions Time: when opinions are given Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 9

Two main types of opinions

(Jindal and Liu 2006; Liu, 2010)  Regular opinions : Sentiment/opinion expressions on some target entities   Direct opinions :  “The touch screen is really cool.” Indirect opinions :  “After taking the drug, my pain has gone.”   Comparative opinions: Comparisons of more than one entity.  E.g., “iPhone is better than Blackberry.” We focus on regular opinions in this talk, and just call them opinions. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 10

Basic Definition of an Opinion



Definition

: An

opinion

is a quadruple,  (

target

sentiment

holder

time

)  This definition is concise, but is not easy to use in many applications.

  The target description can be quite complex.

E.g., “

I bought a Canon G12 camera last week. The picture quality is amazing

.” 

Target

picture quality?

(not quite) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 11

A More Practical Definition

(Hu and Liu 2004; Liu, in NLP handbook, 2010) 

An

opinion

is a quintuple

(

e j

a jk

so ijkl

h i

t l

),      

e j a jk

is a target entity.

is a feature/aspect of the entity

e j

so ijkl

is the sentiment value of the opinion of the opinion holder

h i so ijkl

on aspect

a jk

of entity

e j

at time

t l

is +ve, -ve, or neu, or a more granular rating. .

h i t l

is an opinion holder. is the time when the opinion was expressed. Still a simplified definition (see Liu, 2012 book) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 12

Structure the unstructured

 Objective : Given an opinion document,   Discover all quintuples (

e j

a k

so ijkl

h i

t l

) , Or, solve some simpler forms of the problem  E.g., sentiment classification at the document or sentence level.   With the quintuples ,  Unstructured Text  Structured Data  Traditional data and visualization tools can be used to slice, dice and visualize the results.

 Enable qualitative and quantitative analysis . The definition/model is widely used in industry Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 13

Abstraction (2): Opinion Summary



With a lot of opinions, a summary is necessary.

 A multi-document summary task  Different from traditional summary of facts  1 fact = any number of the same fact  Opinion summary has a quantitative side  1 opinion  any number of the same opinion  The quintuple representation provides a basis for opinion summarization.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 14

….

(Aspect)Feature-based opinion summary

(Hu & Liu, 2004)

Feature Based Summary of iPhone :

“

I bought an iPhone a few days ago. It is such a nice phone. The touch screen is really cool . The voice quality is clear too. It is much better than my old Blackberry , . However, my mother was mad with me as I did not tell her before I bought the phone. She also thought the phone was too expensive, …”

Feature1

Touch screen

Positive : 212 

The touch screen was really cool

. 

The touch screen was so easy to use and can do amazing things.

…

Negative : 6  The screen is easily scratched.

 I have a lot of difficulty in removing finger marks from the touch screen .

… Feature2

voice quality …

Note: We omit opinion holders

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 15



Opinion observer - visualization

(Liu et al. 05) + Summary of reviews of Cell Phone 1  Comparison of reviews of Cell Phone 1 Cell Phone 2 _

Voice

+ _

Screen Battery Size Weight

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 16

Feature/aspect-based opinion summary

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 17

Google Product Search

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 18

Not just ONE problem

 (

e j

a jk

so ijkl

h i

t l

),         

e j

- a target entity: Named Entity Extraction (more)

a jk

- a feature/aspect of

e j

: Information Extraction (more)

so ijkl

is sentiment: Sentiment Identification

h i

is an opinion holder: Information / Data Extraction

t l

is the time: Information/ Data Extraction Coreference resolution Synonym match (voice = sound quality) … A multifaceted and integrated problem!

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 19

Roadmap

Document sentiment classification

 Classify a whole opinion document (e.g., a review) based on the overall sentiment of the opinion holder (Pang et al 2002; Turney 2002, …)   Classes : Positive, negative (possibly neutral) Neutral or no opinion is hard. Most papers ignore it.   An example review : 

“I bought an iPhone a few days ago. It is such a nice phone, although a little large. The touch screen is cool. The voice quality is clear too. I simply love it!”

 Classification : positive or negative?

Classification methods: SVM, Naïve Bayes, etc Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 21

Assumption and goal

 Assumption : The doc is written by a single person and express opinion/sentiment on a single entity. 

Goal

: discover (

), where e, a, h, and t are ignored  Reviews usually satisfy the assumption .  Almost all papers use reviews  Positive: 4 or 5 stars, negative: 1 or 2 stars  Forum postings and blogs do not   They can mention and compare multiple entities Many such postings express no sentiments Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 22

Features for supervised learning

 The problem has been studied by numerous researchers  Probably the most extensive studied problem  Including domain adaption and cross-lingual, etc.  Key: feature engineering. A large set of features have been tried by researchers. E.g.,      Terms frequency and different IR weighting schemes Part of speech (POS) tags Opinion words and phrases Negations Syntactic dependency, etc Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 23

Domain adaptation (transfer learning)

  Sentiment classification is sensitive to the domain of the training data .  A classifier trained using reviews from one domain often performs poorly in another domain.   words and even language constructs used in different domains for expressing opinions can be quite different. same word in one domain may mean positive but negative in another, e.g., “

this vacuum cleaner really sucks .

” Existing research has used labeled data from one domain and unlabeled data from the target domain and general opinion words for learning (Aue and Gamon 2005; Blitzer et al 2007; Yang et al 2006; Pan et al 2010; Wu, Tan and Cheng 2009; Bollegala, Weir and Carroll 2011; He, Lin and Alani 2011).

Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 24

Cross-lingual sentiment classification

 Useful in the following scenarios:   E.g., there are many English sentiment corpora, but for other languages (e.g. Chinese), the annotated sentiment corpora may be limited. Utilizing English corpora for Chinese sentiment classification can relieve the labeling burden.

 Main approach: use available language corpora to train sentiment classifiers for the target language data. Machine translation is typically employed  (Banea et al 2008; Wan 2009; Wei and Pal 2010; Kim et al. 2010; Guo et al 2010; Mihalcea & Wiebe 2010; Boyd-Graber and Resnik 2010; Banea et al 2010; Duh, Fujino & Nagata 2011; Lu et al 2011) Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 25

Roadmap

Sentence subjectivity classification

  Document-level sentiment classification is too coarse for most applications. We now move to the sentence level.  Much of the early work on sentence level analysis focuses on identifying subjective sentences .

  Subjectivity classification: classify a sentence into one of the two classes (Wiebe et al 1999)  Objective and subjective. Most techniques use supervised learning as well.  E.g., a naïve Bayesian classifier (Wiebe et al. 1999).

Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 27

Sentence sentiment analysis



Usually consist of two steps

 Subjectivity classification  To identify subjective sentences  Sentiment classification of subjective sentences  Into two classes, positive and negative  But bear in mind  Many objective sentences can imply sentiments  Many subjective sentences do not express positive or negative sentiments/opinions  E.g.,”I believe he went home yesterday.” Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 28

Assumption



Assumption

: Each sentence is written by a single person and expresses a single positive or negative opinion/sentiment.   True for simple sentences , e.g.,  “I like this car” But not true for compound and “complex” sentences , e.g.,   “I like the picture quality but battery life sucks.” “Apple is doing very well in this lousy economy.” Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 29

Roadmap

We need to go further

 Sentiment classification at both the document and sentence (or clause) levels are useful , but  They do not find what people liked and disliked.

 They do not identify the targets of opinions, i.e.,  Entities and their aspects  Without knowing targets, opinions are of limited use.  We need to go to the entity and aspect level.



Aspect-based opinion mining and summarization

(Hu and Liu 2004) .  We thus need the full opinion definition.

Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 31

Recall an opinion is a quintuple



An

opinion

is a quintuple

(

e j

a jk

so ijkl

h i

t l

), where     

e j

is a target entity.

a jk

is an aspect/feature of the entity

e j

so ijkl

is the sentiment value of the opinion of the opinion holder

h i

on feature

a jk

of entity

e j

at time

t l

so ijkl

is +ve, -ve, or neu, or a more granular rating.

h i

is an opinion holder.

t l

is the time when the opinion is expressed. Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 32

Aspect-based sentiment analysis

 Much of the research is based on online reviews  For reviews , aspect-based sentiment analysis is easier because the entity (i.e., product name) is usually known  Reviewers simply express positive and negative opinions on different aspects of the entity.  For blogs , forum discussions , etc., it is harder:   both entity and aspects of entity are unknown, there may also be many comparisons, and  there is also a lot of irrelevant information. Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 33

Aspect extraction

  Goal : Given an opinion corpus, extract all aspects A frequency-based approach likely to be true aspects (Hu and Liu, 2004) : nouns (NN) that are frequently talked about are (called frequent aspects) .  Pruning based on part-of relations and Web search, e.g., “camera has” (Popescu and Etzioni, 2005).

  Supervised learning , e.g., HMM and CRF (conditional random fields) (Jin and Ho, 2009; Jakob and Gurevych, 2010).

Using dependency parsing + “opinion has target” (Hu and Liu 2004, Zhuang,Jing and Zhu, 2006; Qiu et al. 2009) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 34

Extract Aspects & Opinion Words

(Qiu et al., 2011)  A d

ouble propagation

(DP) approach proposed  Use dependency of opinions & features to extract both features & opinion words.

  Knowing one helps find the other.

E.g., “

The rooms are spacious

”  It bootstraps using a set of seed opinion words, but no feature seeds needed.

 Based on the dependency grammar.  It is a domain independent method!

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 35

Rules from dependency grammar

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 36

Aspect-sentiment statistical models

 This direction of research is mainly based on topic models:  pLSA : Probabilistic Latent Semantic Analysis (Hofmann 1999)  LDA : Latent Dirichlet allocation (Blei, Ng & Jordan, 2003; Griffiths & Steyvers, 2003; 2004)  Topic models:  documents are mixtures of topics  a topic is a probability distribution over words.  A topic model is a document generative model Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 37

Aspect-sentiment model

(Mei et al 2007)  This model is based on pLSA (Hofmann, 1999).  It builds a topic (aspect) model, a positive sentiment model, and a negative sentiment model.  A training data is used to build the initial models.  Training data: topic queries and associated positive and negative sentences about the topics.  The learned models are then used as priors to build the final models on the target data.  Solution: log likelihood and EM algorithm Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 38

Multi-Grain LDA to extract aspects

(Titov and McDonald, 2008a, 2008b)  Unlike a diverse document set used for traditional topic modeling. All reviews for a product talk about the same topics/aspects. It makes applying PLSA or LDA in the traditional way problematic.  Multi-Grain LDA (MG-LDA) models global topics and local topics (Titov and McDonald, 2008a).  Global topics are entities (based on reviews)  Local topics are aspects (based on local context, sliding windows of review sentences) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 39

Aspect-rating of short text

(Lu et al 2009)  This work makes use of short phrases, head terms (w h ) and their modifiers (w m ), i.e.

 (w m , w h )  E.g., great shipping, excellent seller  Objective: (1) extract aspects and (2) compute their ratings in each short comment.

 It uses pLSA to extract and group aspects  It uses existing rating for the full post to help determine aspect ratings. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 40

MaxEnt-LDA Hybrid

(Zhao et al. 2010) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 41

Graphical model

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China    y d,s,n  indicates Background word   Aspect word, or Opinion word MaxEnt is used to train a model using training set   d,s,n  x d,s,n feature vector u d,s,n indicates  General or  Aspect-specific 42

Topic model of snippets

(Sauper, Haghighi and Barzilay, 2011)  This method works on short snippets already extracted from reviews.  “battery life is the best I’ve found”  The model is a variation of LDA but with seeds for sentiment words as priors,  but it also has HMM for modeling the sequence of words with types (aspect word, sentiment word, or background word).

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 43

Semi-supervised model

(Mukherjee and Liu, ACL-2012)   Unsupervised modeling is governed by “higher order co occurrence” (Heinrich, 2009), i.e., based on how often terms co-occur in different contexts.

It results in not so “meaningful” clustering because conceptually different terms can co occur in related contexts e.g., in hotel domain

stain

shower

walls linens

pillows

in aspect in aspect

Maintenance Cleanliness

;

bed

, , are equally probable of emission for any aspect.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 44

Semi-supervised model (contd.)

 Semi-supervised modeling allows the user to give some seed aspect expressions for a subset of aspects (topic clusters)  In order to produce aspects that meet the user’s need.

 Employ seeds to not by “guide” model clustering, “higher order co-occurrence” alone.

 Standard multinomial => 2-level tree structured priors Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 45

Graphical model

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 46

Aspect sentiment classification

   For each aspect, identify the sentiment or opinion expressed about it. Classification based on sentence is insufficient. E.g.

 “The battery life founder is

small

and picture quality ( )”. are

great

(+), but the view   “ Apple (+) is doing well in this bad economy (-) .” “ Standard & Poor downgraded Greece's credit rating (-) ” Classification needs to consider target and thus to segment each sentence  Lexicon-based approach (e.g., Ding, Liu and Yu, 2008)  Supervised learning (e.g., Jiang et al. 2011) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 47

Aspect sentiment classification

 Almost all approaches make use of opinion words and phrases. But notice:   Some opinion words have context independent orientations, e.g., “good” and “bad” (almost) Some other words have context dependent orientations, e.g., “small” and “sucks” (+ve for vacuum cleaner)  Lexicon-based methods  Parsing is needed to deal with: Simple sentences, compound sentences, comparative sentences, conditional sentences, questions, etc  Negation (not), contrary (but), comparisons, etc.  A large opinion lexicon, context dependency, etc.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 48

A lexicon-based method

(Ding, Liu and Yu 2008)    Input : A set of opinion words and phrases. A pair (

), where

is an aspect and

is a sentence that contains

. Output : whether the opinion on

is +ve, -ve, or neutral. Two steps:   Step 1: split the sentence if needed based on BUT words (but, except that, etc). Step 2: work on the segment

s f

containing

. Let the set of opinion words in

s f

1 , ..,

w n

. Sum up their orientations (1, -1, 0), and assign the orientation to (

) accordingly. 

n i

 1

w i

o d

(

w i

) where

w i .o

is the opinion orientation of

w i

(

w i

) is the distance from

w i

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 49

Sentiment shifters

(e.g., Polanyi and Zaenen 2004)  Sentiment/opinion shifters (also called

valence shifters

are words and phrases that can shift or change opinion orientations.  Negation words like

not

never

cannot

, etc., are the most common type.  Many other words and phrases can also alter opinion orientations. E.g., modal auxiliary verbs  (e.g.,

would

should

could, etc

) “The brake could be improved.” Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 50

Sentiment shifters (contd)

 Some presuppositional items also can change opinions, e.g.,

barely

and

hardly

 “It hardly works.” (comparing to “it works”)  It presupposes that better was expected.  Words like

fail

omit

neglect

behave similarly,  “This camera fails to impress me.”  Sarcasm changes orientation too  “What a great car, it did not start the first day.”  Jia, Yu and Meng (2009) designed some rules based on parsing to find the scope of negation. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 51

Basic rules of opinions

(Liu, 2010)  Opinions/sentiments are governed by many rules, e.g., 

Opinion word or phrase, ex: “I love this car”

P P ::= a positive opinion word or phrase  N ::= an negative opinion word or phrase

Desirable or undesirable facts, ex:

“After my wife and I slept on it for two weeks, I noticed a mountain in the middle of the mattress” ::= desirable fact N ::= undesirable fact Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 52

Basic rules of opinions



High, low, increased and decreased quantity of a positive or negative potential item

, ex: “The battery life is long.” PO ::= no, low, less or decreased quantity of NPI | large, larger, or increased quantity of PPI NE ::= no, low, less, or decreased quantity of PPI | large, larger, or increased quantity of NPI NPI ::= a negative potential item PPI ::= a positive potential item Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 53

Basic rules of opinions



Decreased and increased quantity of an opinionated item, ex:

“This drug reduced my pain significantly.” PO ::= less or decreased N | more or increased P NE NE ::= less or decreased P  | more or increased N

Deviation from the desired value range

: “This drug increased my blood pressure to 200.” PO ::= within the desired value range ::= above or below the desired value range Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 54

Basic rules of opinions



Producing and consuming resources and wastes, ex:

“This washer uses a lot of water” PO ::= produce a large quantity of or more resource NE | produce no, little or less waste | consume no, little or less resource | consume a large quantity of or more waste ::= produce no, little or less resource | produce some or more waste | consume a large quantity of or more resource | consume no, little or less waste Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 55

Opinions implied by objective terms

(Zhang and Liu, 2011)  For opinion mining, many researchers first identify subjective sentences and then determine if they are positive/negative. 

This approach can be problematic

  Many objective sentences imply opinions/sentiments E.g., “After sleeping on the mattress for one month, a valley is formed in the middle.” Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 56

Roadmap

Comparative Opinions

(Jindal and Liu, 2006) 

Gradable



Non-Equal Gradable

: Relations of the type

greater

less than



Ex: “optics of camera A is better than that of camera B”



Equative

: Relations of the type

equal to

 Ex: “

camera A and camera B both come in 7MP

” 

Superlative

: Relations of the type

greater

less than all others

 Ex: “

camera A is the cheapest in market

” Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 58

Analyzing Comparative Opinions



Objective

: Given an opinionated document

, Extract comparative opinions : (

1 ,

2 ,

po, h, t

), where

1 and

2 are the entity sets being compared based on their shared features/aspects

is the preferred object set of the opinion holder

, and

is the time when the comparative opinion is expressed.  Note: not positive or negative opinions. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 59

Roadmap

Opinion Spam Detection

(Jindal et al, 2008, 2010 and 2011) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 61

Supervised learning (fake reviews) Training data

Same userid, same product Different userid, same product Same userid, different products Different userid, different products  The last three types are very likely to be spam!

 Other reviews, non-spam  Build a supervised classification model (Jindal and Liu 2008)  (Ott et al., 2011) and (Li et al., 2011) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 62

Finding Unexpected Behavior Patterns

(Jindal and Liu 2010)  Opinion spam is hard to detect because it is very difficult to recognize fake reviews by manually reading them.  i.e., hard to detect based on content  Let us analyze the behavior of reviewers  identifying

unusual review patterns

which may represent suspicious behaviors of reviewers.  We formulate the problem as finding

unexpected rules and rule groups

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 63

Finding unexpected review patterns

 For example,

if a reviewer wrote all positive reviews on products of a brand but all negative reviews on a competing brand

…

 Finding unexpected rules,  Data:

reviewer-id

brand-id

product-id

, and a

class

 Mining: class association rule mining  Finding unexpected rules and rule groups, i.e., showing atypical behaviors of reviewers. Rule1: Reviewer-1, brand-1 -> positive (confid=100%) Rule2: Reviewer-1, brand-2 -> negative (confid=100%) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 64

The example (cont.)

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 65

Confidence unexpectedness

Rule: reviewer-1, brand-1  positive [sup = 0.1, conf = 1]  If we find that on average reviewers give brand-1 only 20% positive reviews (expectation), then reviewer-1 is quite unexpected.

(

v jk



c i

)  Pr(

c i

v jk

) 

(Pr(

c i E

(Pr(

c i

v jk

)) |

v jk

))

(Pr(

c i

v jk

v gh

))  Pr(

c i

Pr(

c i

) 

m r

 1 |

v jk

Pr(

c r

) Pr(

c i

v gh

) |

v jk

) Pr(

c r

v gh

) Pr(

c r

) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 66

Support unexpectedness

Rule: reviewer-1, product-1 -> positive [sup = 5]  Each reviewer should write only one review on a product and give it a positive (negative) rating (expectation).  This unexpectedness can detect those reviewers who review the same product multiple times, which is unexpected.  These reviewers are likely to be spammers.

 Can be defined probabilistically as well.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 67

Detecting group opinion spam

(Mukherjee, Liu and Glance, WWW-2012)  A group of people who work together to promote an product or to demote another product.  The algorithm has two steps  Frequent pattern mining: find groups of people who reviewed a number of products. These are candidate spammer groups.  A relational model is then formulated to compute a ranking of candidate groups based on their likelihood being fake. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 68

Roadmap

Modeling Review Comments

(Mukherjee and Liu, ACL-2012)     Online reviews by consumers evaluate products and services that they have used.

While certainly useful, reviews only provide part of the story: evaluations and experiences of the reviewers.

    Hidden glitches: Reviewer may not be an expert.

Misuses a product.

Doesn’t mention some product aspects of consumer interest.

Reviewer can be an opinion spammer writing fake reviews.

Clearly, there is a room for improvement of the online review system.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 70

Review Comments

   To improve the reviewing system, popular review hosting sites (e.g., Amazon, Epinions, Wired.com, etc.) support reader-comments on reviews.

Comments on review are a richer way of “review profiling”, rather than just clicking whether the review is helpful or not.

Many reviews receive a large number of comments.

(e.g., hundreds of them)  Reading them all to get a gist of them is not easy.

 Some kind of summary will be very useful.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 71

What to model?

 Topics/aspects and different types of comments      

Thumbs-up

(e.g., “review helped me”)

Thumbs-down

(e.g., “poor review”)

Question

(e.g., “how to”)

Answer acknowledgement

clarifying”). (e.g., “thank you for

Disagreement

(

contention

) (e.g., “I disagree”)

Agreement

(e.g., “I agree”).

 They are collectively called,

C-expressions

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 72

Summary and usefulness

    Extracted topics and C-expressions from comments are quite useful in practice: Enable more accurate classification of comments, e.g., evaluating review quality and credibility. Help identify key product aspects that people are troubled with in disagreements and in questions.

    Facilitate comments summarization. Summary may include but not limited to: % of people who giving a thumbs-up or thumbs-down % of people who agree or disagree with the reviewer Disagreed (contentious) aspects (or topics) Aspects that people often have questions with Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 73

A graphical model – generative process

For each C-expression type For each topic

, draw For each comment post 𝜑 𝑇 𝑡 𝑒 , draw ~𝐷𝑖𝑟 𝛽 𝑇 𝑑 ∈ {1 … 𝐷} : 𝜑 𝑒 𝐸 ~𝐷𝑖𝑟(𝛽 𝐸 ) ii.

iii.

Draw 𝜃 𝐸 𝑑 ~𝐷𝑖𝑟 𝛼 𝐸 Draw 𝜃 𝑇 𝑑 ~𝐷𝑖𝑟 𝛼 𝑇 For each term 𝑤 𝑑,𝑗 , 𝑗 ∈ {1 … 𝑁 𝑑 } : a.

Draw 𝜓 𝑑,𝑗 ~𝑀𝑎𝑥𝐸𝑛𝑡 𝑥 𝑑, 𝑗 Draw 𝑟 𝑑,𝑗 ~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝜓 𝑑,𝑗 ) c.

if ( 𝑟 𝑑,𝑗 Draw = 𝑒 // 𝑤 𝑑,𝑗 is a C-expression term 𝑧 𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃 𝐸 𝑑 ) else Draw // 𝑧 𝑟 𝑑,𝑗 𝑑,𝑗 = 𝑡 , 𝑤 𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜃 𝑇 𝑑 ) is a topical term Emit 𝑤 𝑑,𝑗 ~ 𝑀𝑢𝑙𝑡(𝜑 𝑟 𝑑,𝑗 𝑧 𝑑,𝑗 ) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 74

The graphical model in plate notation

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 75

Roadmap

Modeling Online Discussions/Debates

(Mukherjee and Liu, KDD-2012)  A large part of social media is about discussion and debate.

 A large part of such contents is about social, political and religious issues.

 On such issues, there are often heated discussions/debates, i.e., people argue and agree or disagree with one another.

 We can model such interactive social media.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 77

The Goal

   Given a set of discussion/debate posts, we aim to perform the following tasks.   Discover expressions often used to express Contention/Disagreement (e.g., “I disagree”, “you make no sense”) and A greement (e.g., “I agree”, “I think you’re right”). We collectively call them

CA-expressions

 Determine contentious topics. First discover discussion topics in the whole collection,  then for each contentious post, discover the contention points (or topics). Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 78

Joint modeling of debate topics and expressions (JTE)

 We jointly model topics and CA-expressions  Observation: A typical discussion/debate post mentions a few topics (using semantically related topical terms) and expresses some viewpoints with one or more CA-expression types (using semantically related contention and/or agreement expressions).

 The above observation motivates the model  Posts are represented as random mixtures of latent topics and CA-expression types. Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 79

A graphical model – generative process

(the same as that for comments) 1.

For each C-expression type For each topic

, draw For each comment post 𝜑 𝑇 𝑡 𝑒 , draw ~𝐷𝑖𝑟 𝛽 𝑇 𝑑 ∈ {1 … 𝐷} : 𝜑 𝑒 𝐸 ~𝐷𝑖𝑟(𝛽 𝐸 ) ii.

iii.

Draw 𝜃 𝐸 𝑑 ~𝐷𝑖𝑟 𝛼 𝐸 Draw 𝜃 𝑇 𝑑 ~𝐷𝑖𝑟 𝛼 𝑇 For each term 𝑤 𝑑,𝑗 , 𝑗 ∈ {1 … 𝑁 𝑑 } : a.

Draw 𝜓 𝑑,𝑗 ~𝑀𝑎𝑥𝐸𝑛𝑡 𝑥 𝑑, 𝑗 Draw 𝑟 𝑑,𝑗 ~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(𝜓 𝑑,𝑗 ) c.

JTE in plate notation

(the same as that for comments) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 81

JTE-R: Encoding reply relations

  

Observation

: Whenever a post

replies to the viewpoints of some other posts by quoting them, and the posts quoted by

should have similar topic distributions.

Let

q d

be the set of posts quoted by post

q d

is observed.

Key challenge: - constrain 𝜃 𝑑 𝑇 𝜃 𝑇 𝑑 , where 𝑑 during inference while the topic distributions of both 𝜃 𝑑 𝑇 to be similar to and 𝜃 𝑇 𝑑 , 𝑑 are latent and unknown

apriori

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 82

Exploiting Dirichlet distribution

 A simple solution: exploit the following salient features of the Dirichlet distribution:  Since 𝜃 𝑇 𝑑 ~𝐷𝑖𝑟(𝛼 𝑇 ) , we have 𝑡 suffices that 𝜃 𝑇 𝑑 𝜃 𝑇 𝑑,𝑡 = 1. Thus, it can act as a base measure for Dirichlet distributions of the same order.

 Also, the expected probability mass associated with each dimension of the Dirichlet distribution is proportional to the corresponding component of its base measure  𝐸 𝑋 𝑖 = 𝛼 𝑖 𝛴𝛼 𝑖 . Thus, 𝐸 𝑋 𝑖 ∝ 𝛼 𝑖 Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 83

Exploiting Dirichlet distribution

(contd)  We need functional base measures    Thus for posts that quote:  we draw 𝜃 𝑇 𝑑 ~𝐷𝑖𝑟(𝛼 𝑇 𝒔 𝒅 ) , where 𝒔 𝒅 = 𝑑 ′ ∈𝑞 𝑑 𝜃 𝑇 𝑑 ′ (the expected topical distribution of posts in 𝑞 𝑑 ). |𝑞 𝑑 | For posts that do not quote any other post,  we simply draw 𝜃 𝑑 𝑇 ~𝐷𝑖𝑟(𝛼 𝑇 ) . The Gibbs sampling is, however, an approximation (see the paper for detail) Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 84

JTE-R in plate notation

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 85

JTE-P

: Encoding Pair Structures



Observation

: When authors reply to others’ viewpoints,  they typically direct their topical viewpoints with contention or agreeing expressions to those authors.  Such exchanges can go back and forth between author pairs.  The discussion topics and CA-expressions emitted are thus caused by the author pairs’ topical interests and their nature of interactions.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 86

The approach

   Let 𝑎 𝑑 be the author of a post be the list of

target authors

replies to or quotes in 𝑑 .

𝑑 , 𝑏 𝑑 = [𝑏 1…𝑛 ] to whom 𝑎 𝑑 The pairs of the form 𝑝 = ( 𝑎 𝑑 , 𝑐 ),

∈ 𝑏 𝑑 essentially shapes both the topics and CA expressions emitted in

as contention or agreement on topical viewpoints are almost always directed towards certain authors.

Thus, it is appropriate to condition 𝜃 𝐸 over author-pairs.

𝜃 𝑇 and Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 87

The approach

  To generate each term 𝑤 𝑑,𝑗 , a target author, 𝑐~𝑈𝑛𝑖(𝑏 𝑑 ) , is chosen at uniform from a pair 𝑝 = ( 𝑎 𝑑 , 𝑐 ).

𝑏 𝑑 forming Then, depending on the switch variable 𝑟 𝑑,𝑗 , a topic or an expression type index 𝑧 is chosen from a multinomial over topic distribution 𝜃 𝑝 𝑇 or CA-expression type distribution 𝜃 𝑝 𝐸 , where the subscript 𝑝 denotes the fact that the distributions are specific to the author-target pair 𝑝 which shape topics and CA-expressions.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 88

JTE-P graphical model

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 89

Roadmap

Summary

Tutorial @ Sentiment Analysis Symposium, May 7-8, 2012, New York City 90

Summary

 We first introduced some basics of sentiment analysis and opinion mining  Current solutions are still inaccurate.     Every sub-problem is hard General NL understanding is probably hopeless in near future But can we understand this restricted aspect of semantics?

Endless applications due to the human nature  We also discussed the problem of modeling interactive social forums, such as review comments and debates/discussions .

 There is a lot of future work, e.g., linguistic knowledge.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 91

References

All references are in the

New Book

 Bing Liu.

Sentiment Analysis and Opinion Mining

Morgan & Claypool Publishers. May 2012.

Bing Liu @ KDD-2012 Summer School, Aug 10, 2012, Beijing, China 92