Transcript lecture12

Today’s Topics
• Playing Deterministic (no Dice, etc) Games
– Mini-max
– - pruning
– ML and games?
• 1997: Computer Chess Player (IBM’s Deep
Blue) Beat Human Chess Champ (Kasparov)
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
1
Mini-Max Algorithm
• Generate Search Tree of ALL Possible Games
• Score Leaves (Assume Higher Better for ‘You’)
eg, +1 you win, -1 you lose, 0 if a draw (ie, a tie)
• Assume Opponent Scores ‘Board Configurations’
the Same Way You Do (more on this later)
• Propagate Info from Leaves to Root of Search Tree,
then Choose Best Move
– Your Turn:
choose MAX score
– Opponent’s Turn: choose MIN score (which is best for opponent)
• Only Choose NEXT Move; When Your Turn Again, Repeat
– opponent might not have done what you expected
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
2
Tic-Tac-Toe
(https://en.wikipedia.org/wiki/Tic-tac-toe)
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
3
What If Game More Complex?
Chess: approx 35 moves per term and
50 moves/player/game, so O(35100) possible
board configurations!
(Common) Abuse of big-O notation
Better estimate:
1047 en.wikipedia.org/wiki/Shannon_number
Number of atoms in the Earth: 1050 www.fnal.gov/pub/science/inquiring/questions/atoms.html
Solution: project ahead as far as possible, then
use a ‘static board evaluator’ (SBE) to score
in-progress board configurations
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
4
SBE’s
• No Perfect SBE Exists for Chess
(otherwise we’d know if black can always win, etc)
• But a lot of Domain K can be put in the SBE
• You Might Have Learned “Piece Values” in Chess
SBE =
1  (#myPawns - #theirPawns) + …
+ 10  (#myQueens - #theirQueens)
+ points for ‘center of board’ control + etc
• Can Run Mini-Max with SBE
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
5
Saving Unnecessary
Calculations: - Pruning
• Can Prune Away Subtrees by Realizing no Matter
their Score, they won’t Impact Choice
• We Won’t Cover the Full - Pruning Algorithm,
Since its Applicability Across AI is Limited
• But We’ll Cover some Simple Cases of the
Two Types of Pruning
• Ask: Would it Matter if this Node had Value  or
Value -? If Decision Same in Both Cases, No
Need to Compute Node’s SBE Value
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
6
An -Cutoff Example
(view in Slide Show mode to see animation)
Tip: Always
fully compute
the left-most
subtree of root
move
2
Us (max)
1
2
Them (min)
2
7
1
big
subtree
?
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
If good for Us,
Opponent will
not go here.
If bad for Us,
we will take
left branch
from root.
7
A -Cutoff Example
(view in Slide Show mode to see animation)
Recall: Always
compute the
left-most
subtree of root
move
7
Them (min)
9
7
Us (max)
2
7
9
big
subtree
?
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
If good for Us,
Opponent will
take left
branch from
root.
If bad for Us,
we will take
left branch.
8
Note
We project ahead as far as we
can in the time we have, but only
choose our next move
Next time it is our turn, we repeat - calc
(possibly repeating some work, but
too much to store)
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
9
Performance Improvement
Best case: - can double search depth
considered is fixed amount of time
(doubling depth exponentially
increases boards considered!)
Worst case: no savings
Note: - and mini-max return SAME answer
(- simply avoids wasted work)
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
10
Some Built-in Assumptions
• Projecting Ahead k Moves will Lead to Better
SBE Estimates than Simply Applying SBE to
the Legal Next Boards
– seems reasonable, but isn’t guaranteed
• Our Opponent Thinks the Same Way We Do
– no consideration of ‘traps’ and tricking opponent
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
11
Horizon Effect
Defn: Since We Usually Stop before
Game Ends, ‘Danger’ might be
Lurking just Beyond our Search Depth
One (Partial) Soln: Once Best Move
Chosen, Look Ahead a Few Moves More
if turns out bad, spend more time
searching this turn
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
12
Chess: The ‘E. coli’ of AI?
• Chess Originally Thought to be a
Great Testbed for Advancing AI
• Has had Moderate Impact on AI Progress
• Not Much ‘Technology Transfer’ to AI Tasks
• Played Minor Role in ‘ML Revolution’
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
13
ML and Games
• ML Led to World-Class
Backgammon Player Decades Ago
• ML Produced World-Class
Poker Player Over Last Few Years
• Had Less Success at Chess and Go,
but Recent Promise Shown with ‘Deep
[Neural] Networks’
• Deep Networks Recently Learned to Play
Well without being given Games’ Rules
10/6/15
CS 540 - Fall 2015 (Shavlik©), Lecture 12, Week 5
14