Transcript www.ailab.si
CG 2006
Computer analysis of World Chess Champions
Matej Guid
and
Ivan Bratko
Introduction
Who was the best chess player of all time?
Chess players of different eras never met across the chess board.
No well founded, objective answer.
Computers...
Were so far mostly used as a tool for statistical analysis of players’ results.
High quality chess programs...
Provide an opportunity of an objective comparisson.
Statistical analysis of results do NOT reflect:
true strengths of the players, quality of play.
I Wilhelm Steinitz, 1886 - 1894
Related work
Jeff Sonas, 2005:
rating scheme, based on tournament results from 1840 to the present, ratings are calculated for each month separately, player’s activity is taken into account.
Disadvantages
Playing level has risen dramatically in the recent decades.
The ratings in general reflect the players’ success in competition, but NOT directly their quality of play.
II Emanuel Lasker, 1894 -1921
Our approach
computer analysis of individual moves played determine players’ quality of play regardless of the game score the differences in players’ style were also taken into account calm positional players vs aggresive tactical players a method to assess the difficulty of positions was designed
Analysed games
14 World Champions (classical version) from 1886 to 2004 analyses of the matches for the title of “World Chess Champion” slightly adapted chess program Crafty has been used III Jose Raul Capablanca, 1921 -1927
The modified Crafty
Instead of time limit, we limited search to fixed search depth. Backed-up evaluations from depth 2 to 12 were obtained for each move.
Quiescence search remained turned on to prevent horizont effects .
Advantages
complex positions automatically get more computation time, the program could be run on computers of different computational powers.
Obtained data
best move and its evaluation, second best move and its evaluation, move played and its evaluation, material state of each player.
IV Alexander Alekhine, 1927 -1935 and 1937 - 1946
Average error
average difference between moves played and best evaluated moves basic criterion
Formula ∑|Best move evaluation – Move played evaluation| Number of moves
“Best move” = Crafty’s decision resulting from 12 ply search
Constraints
Evaluations started on move 12.
Positions, where both the move suggested and the move played were outside the interval [-2, 2], were discarded.
Positional players are expected to commit less errors due to somewhat less complex positions, than tactical players.
V Max Euwe, 1935 - 1937
Average error
V Max Euwe, 1935 - 1937
Blunders
Big mistakes can be quite reliably detected with a computer.
We label a move as a blunder when the numerical error exceeds 1.00.
VI Mikhail Botvinnik, 1948 - 1957, 1958 - 1960, and 1961 - 1963
Complexity of a position
Basic idea
A given position is difficult, when different “best moves”, which considerably alter the evaluation of the root position, are discovered at different search depths.
Assumption
This definition of complexity also applies to humans.
This assumption is in agreement with experimental results.
Formula ∑ |Best move evaluation – 2nd best move evaluation| best i ≠ best i - 1
VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity = 0.00
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
Euwe-Alekhine, 16th World Championship 1935
0.00 +
(1.30 – 1.16)
VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
complexity =
0.14
Euwe-Alekhine, 16th World Championship 1935 VII Vasily Smyslov, 1957 - 1958
Complexity of a position
depth
2 3 4 5 6 7 8 9 10 11 12
1st
Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc2 Qc1 Qc1 Qd4
eval
-0.09
+0.24
+0.08
+0.35
+0.07
+0.57
+0.72
+0.96
+1.30
+1.52
+4.46
2nd
Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc1 Qc2 Qc2 Qc1
eval
-0.17
+0.16
+0.00
+0.30
+0.02
+0.55
+0.60
+0.87
+1.16
+1.26
+1.60
Euwe-Alekhine, 16th World Championship 1935
3.00
+
(4.46 – 1.60)
VII Vasily Smyslov, 1957 - 1958
Complexity of a position
VII Vasily Smyslov, 1957 - 1958
Average error in equally complex positions
How would players perform if they faced equally complex positions?
What would be their expected error if they were playing in another style ?
60 50 40 30 20 10 0 0,1 0,3 average 0,5 0,7 complexity Capablanca 0,9 Tal 1,1 VIII Mikhail Tal, 1960 - 1961
Percentage of best moves played
It alone does NOT reveal true strength of a player.
IX Tigran Petrosian, 1963 - 1969
The difference in best move evaluations
X Boris Spassky, 1969 - 1972
Percentage of best moves played...
... and the difference in best move evaluations
XI Robert James Fischer, 1972 - 1975
Material
70 5 60 0 50 20 -15 10 31 31 41 41 51 51 61 61 71 71 81 81 91 XII Anatoly Karpov, 1975 - 1985
Credibility of Crafty as an analysis tool
By limiting search depth we achieved automatic adaptation of time used to the complexity of a given position.
Occasional errors cancel out through statistical averaging (around 1.400
analyses were applied, altogether over 37.000 positions).
Using another program instead of Crafty...
An open source program was required for the modification of the program.
Analyses of “Man against the machine” matches indicate that Crafty competently appreciates the strength of the strongest chess programs.
Deep Blue 0.0757
Deep Fritz 0.0617
Deep Junior 0.0865
FritzX3D Hydra 0.0904
0.0743
New York, 1997 Bahrain, 2002 New York, 2003 New York, 2003 London, 2005 Kasparov Kramnik Kasparov Kasparov Adams XIII Garry Kasparov, 1985 - 2000
Conclusion
Slightly modified chess program Crafty was applied as tool for computer analysis aiming at an objective comparison of chess players of different eras.
Several criteria for evaluation were designed: average difference between moves played and best evaluated moves rate of blunders (big errors) expected error in equally complex positions rate of best moves played & difference in best moves evaluations A method to assess the difficulty of positions was designed, in order to bring all players to a “common denominator”.
The results might appear quite surprising. Overall, they can be nicely interpreted by a chess expert.
XIV Vladimir Kramnik, 2000 -
XIV Vladimir Kramnik, 2000 -