Transcript Schedule of Personalized TV Advertisements

A Cultural Sensitive Agent for
Human-Computer
Negotiation
Galit Haim,
Ya'akov Gal, Sarit Kraus
and Michele J. Gelfand
1
Motivation

Buyers and seller across geographical and
ethnic borders
–
–
–
electronic commerce:
crowd-sourcing:
deal-of-the-day applications:
Interaction between people from different
countries
 to succeed, an agent needs to reason about
how culture affects people's decision making

2
Goals and Challenges
Can we build an
agent that will
negotiate better than
the people in each
countries?
Can we build
proficient negotiator
with no expert
designed rules?
Culture sensitive agent?
3
The approach
1. Collect data on each country
2. Use machine learning
3. Build influence diagram
Sparse Data
Noisy Data
The Colored Trails (CT) Game


An infrastructure for agent design,
implementation and evaluation for open
environments
Designed in 2004 by Barbara Grosz and Sarit
Kraus (Grosz et al AIJ 2010)
CT is the right test-bed to use
because it provides a task analogy
to the real world
4
The CT Configuration
7*5 board of colored squares
 One square is the goal
 Set of colored chips
 Move using a chip in the
same color

5
CT Scenario

2 players

Multiple phases:
communication: negotiation
(alternating offer protocol)
– transfer: chip exchange
– movement
–

Complete information
Agreements are not enforceable
Complex dependencies

Game ends when one of the players: reached the goal or


6
did not move for three movement phases
Scoring and Payment




7
100 point bonus for getting to goal
5 point bonus for each chip left at end of game
10 point penalty for each square in the shortest
path from end-position to goal
Performance does not depend on outcome for
other player
Personality, Adaptive Learning
(PAL) Agent
machine
learning
Human
behavior
model
Data from
specific
country
Decision
Making
Take action
8
8
Learning People's Reliability
Predict if the other player will keep its promise
9
Learning how People Accept Offers
10
Accept or reject the proposal?
Feature Set

Domain independent feature:
–
–
–
–

Domain dependent feature:
–
11
Current and Resulting scores
Offer generosity
Reliability: between 0 (completely unreliable) to
1(fully reliable)
Weighted reliability: over the previous rounds in the
game
Round number
How to Model People's Behavior

For each culture:
–
–


12
Use different features
Choose learning algorithm that minimized error using
10-fold cross validation
In US and Israel - we only used domain
independent features
In Lebanon we added domain dependent
features
Data Collection with Sparse Data

Sources of data to train our classifiers:
–
–
–
13
222 game instances consisting of people playing a
rule-based agent
U.S. and Israel: collect 112 game instances of people
playing other people
Lebanon: collect 64 additional games
The Lebanon people in
 “Nasty agent”: less reliable when fulfilling its
this data set almost
agreement
always kept the
agreements and as a
result, PAL never kept
agreements
People Learned Reliability
0.8
0.7
0.6
0.5
Lebanon
U.S.A
Israel
0.4
0.3
0.2
0.1
0
People learned reliability: Dependent case
14
Experiment Design

3 countries: 157 people
–
–
–




15
Israel: 63
Lebanon: 48
U.S.A: 46
30 minutes tutorial
Boards varied dependencies between players
People were always the first proposer in the
game
There was a single path to the goal
Decision Making
There are 3 decisions that PAL needs to make:
 Reliability: determine the PAL transfer strategy
 Accepting an offer: accept or reject a specific offer
proposed by the opponent
 Propose an offer
Use backward induction
over two rounds…
16
Success Rate: Getting to the Goal
17
Performance Comparison: Averages
250
200
150
PAL
Human
100
50
0
U.S
18
Lebanon
Israel
Example in Lebanon



2 chips for 2 chips; accepted  both sent
1 chip for 1 chip; accepted
PAL learned that people in Lebanon were highly
reliable PAL did not send, the human sent
games were
relatively
shorter
19
people were very
reliable in the
training games
Example in Israel





20
2 chips for 2 chips; accepted  only PAL sent
1 chip for 1 chip; accepted  the human only sent
1 chip for 1 chip; accepted  the human only sent
1 chip for 1 chip; accepted only PAL sent
1 chip for 3 chips; accepted only the human
sent
people were less
games were
reliable
in
the
relatively
training games
longer
than in Lebanon
Conclusions


PAL is able to learn to negotiate proficiently with
people across different cultures
PAL was able to outperform people in all
dependency conditions and in all countries
This is the first work to show
that a computer agent can
learn to negotiate with
people in different countries
21
Colored trails is easy to use
for your own research






22
Open source empirical test-bed for investigating
decision making
Easy to design new games
Built in functionality for conducting experiments with
people
Over 30 publications
Freely available; extensive documentation
http://eecs.harvard.edu/ai/ct (or Google ”colored trails”)
THANK YOU
[email protected]