Monday, January 14, 2013

Chess and luck

In previous posts, I argued about how there's luck in golf, and how there's luck in foul shooting in basketball.

But what about games of pure mental performance, like chess?  Is there luck involved in chess?  Can you win a chess game because you were lucky?


Start by thinking about a college exam.  There's definitely luck there.  Hardly anybody has perfect mastery.  A student is going to be stronger in some parts of the course material, and weaker in other parts.

Perhaps the professor has a list of 200 questions, and he randomly picks 50 of them for the exam.  If those happen to be more weighted to the stuff you're weak in, you'll do worse.

Suppose you know 80 percent of the material, in the sense that, on any given question, you have an 80 percent chance of getting the right answer.  On average, you'll score 80 percent, or 40 out of 50.  But, depending on which questions the professor picks, your grade will vary, possibly by a lot.

The standard deviation of your score is going to be 5.6 percentage points.  That means the 95 percent confidence interval for your score is wide, stretching from 69 to 91.

And, if you're comparing two students, 2 SD of the difference in their scores is even higher -- 16 points.  So if one student scores 80, and another student scores 65, you cannot conclude, with statistical significance, that the first student is better than the second!

So, in a sense, exam writing is like coin tossing.  You study as hard as you can to learn as much as you can -- that is, to build yourself a coin that lands heads (right answer) as often as possible.  Then, you walk in to the exam room, and flip the coin you've built, 50 times.


It's similar for chess.

Every game of chess is different.  After a few moves, even the most experienced grandmasters are probably looking at board positions they've never seen before.  In these situations, there are different mental tasks that become important.  Some positions require you to look ahead many moves, while some require you to look ahead fewer.  Some require you to exploit or defend an advantage in positioning, and some present you with differences in material.  In some, you're attacking, and in others, you're defending.

That's how it's like an exam.  If a game is 40 moves each, it's like you're sitting down at an exam where you're going to have 40 questions, one at a time, but you don't know what they are.  Except for the first few moves, you're looking at a board position you've literally never seen before.  If it works out that the 40 board positions are the kind where you're stronger, you might find them easy, and do well.  If the 40 positions are "hard" for you -- that is, if they happen to be types of positions where you're weaker -- you won't do as well. 

And, even if they're positions where you're strong, there's luck involved: the move that looks the best might not truly *be* the best.  For instance, it might be true that a certain class of move -- for instance, "putting a fork on the opponent's rook and bishop on the far side of the board, when the overall position looks roughly similar to this one" -- might be a good move 98 percent of the time.  But, maybe in this case, because a certain pawn is on A5 instead of A4, it actually turns out to be a weaker move.  Well, nobody can know the game down to that detail; there are 10 to the power of 43 different board positions. 

The best you can do is see that it *seems* to be a good move, that in situations that look similar to you, it would work out more often than not.  But you'll never know whether it's 90 percent or 98 percent, and you won't know whether this is one of the exceptions. 


It's like, suppose I ask you to write down a 14-digit number (that doesn't start with zero), and, if it's prime, I'll give you $20.  You have three minutes, and you don't have a calculator, or extra paper.  What's your strategy?  Well, if you know something about math, you'll know you have to write an odd number.  You'll know it can't end in 5.  You might know enough to make sure the digits don't add up to a multiple of 3. 

After that, you just have to hope your number is prime.  It's luck.

But, if you're a master prime finder ... you can do better.  You can also do a quick check to make sure it's not divisible by 11.  And, if you're a grandmaster, you might have learned to do a test for divisibility by 7, 13, 17, and 19, and even further.  In fact, your grandmaster rating might have a lot to do with how many of those extra tests you're able to do in your head in those three minutes.

But, even if you manage to get through a whole bunch of tests, you still have to be lucky enough to have written a prime, instead of a number that turns out to be divisible by, say, 277, which you didn't have time to test for.

A grandmaster has a better chance of outpriming a lesser player, because he's able to eliminate more bad moves.  But, there's still substantial luck in whether or not he wins the $20, or even whether he beats an opponent in a prime-guessing tournament.


On an old thread over at Tango's blog, someone pointed this out: if you get two chess players of exactly equal skill, it's 100 percent a matter of luck which one wins.  That's got to be true, right? 

Well, maybe you're not sure about "exactly equal skill."  You figure, it's impossible to be *exactly* equal, so the guy who won was probably better!  But, then, if you like, assume the players are exact clones of each other.  If that still doesn't work, imagine that they're two computers, programmed identically. 

Suppose the computers aren't doing anything random inside their CPUs at all -- they have a precise, deterministic algorithm for what move to make.  How, then, can you say the result is random?

Well, it's not random in the sense that it's made of the ether of pure, abstract probability, but it's random in the practical sense, the sense that the algorithm is complex enough that humans can't predict the outcome.  It's random in the same way the second decimal of tomorrow's Dow Jones average is random.  Almost all computer randomization is deterministic -- but not patterned or predictable.  The winner of the computer chess game is random in the same way the hands dealt in online poker are random.

In fact, I bet computer chess would make a fine random number generator.  Take two computers, give them the same algorithm, which has to include something where the computer "learns" from past games (otherwise, you'll just get the same positions over and over).  Have them play a few trillion games, alternating black and white, to learn as much as they can.  Then, play a tournament of an even number of games (so both sides can play white an equal number of times).  If A wins, your random digit is a "1".  If B wins, your random digit is a "0". 

It's not a *practical* random number generator, but I bet it would work.  And it's "random" in the sense that, no human being could predict the outcome in advance any faster than actually running the same algorithm himself. 

Labels: ,


At Monday, January 14, 2013 3:04:00 AM, Blogger j holz said...

Just read The Success Equation, eh? The author does use checkers as his example instead of chess, possibly because it is "solvable".

At Monday, January 14, 2013 8:20:00 AM, Blogger Phil Birnbaum said...

Actually, I *did* read that, a while ago ... had to go back and refresh my memory!

I would see success in high-level chess as requiring more luck than checkers, because checkers seems easier to "solve." I wonder, when two high-level checkers players compete, is it usually a draw?

At Monday, January 14, 2013 7:59:00 PM, Anonymous Anonymous said...

Liked this post but not the example at the end. Randomly one chess computer could learn quicker than the other and it would be come self reinforcing such that you wouldn't want this to be your rnadom number generator. A bias would develop over time.

At Monday, January 14, 2013 8:14:00 PM, Blogger Phil Birnbaum said...

The two computers have exactly the same algorithm and the same database of games to learn from, so they should be identical.

At Monday, January 14, 2013 9:11:00 PM, Blogger Don Coffin said...

You'd need to factor draws into that. In fact, given your set-up the expectation might be a very high percentage of draws.

Also, this is, to some extent testable from records of human play and ratings, isn't it? (Not that I'd want to have to figure out exactly how to structure the statistical test...)

Finally, in chess there is the ongoing issue of, um, cheating (about which you can read more than you'd probably like to know here:

At Monday, January 14, 2013 10:40:00 PM, Blogger Phil Birnbaum said...

Thanks, Doc! Coincidentally, I saw that cheating post today, which is the first I've heard of it ... 12 hours after I posted about chess! I wish it had more information about their statistical tests.

Hmmmm ... do identical computers play to a lot of draws? Be interesting to find out! I wonder if Google knows ...


Post a Comment

<< Home