Saturday, September 15, 2012

More "hot hand" thoughts

There are two ways to define (or measure) a "hot hand": 

(A) you can check if a hit is more likely than normal to be followed by a hit, and if a miss  is more likely that normal to be followed by a miss.

(B) you can check if a *high-probability opportunity* is more likely than normal to be followed by another high-probability opportunity, and if a low-probability opportunity is more likely to be followed by a low-probability opportunity.

The measure in Type A is usually what fans mean.  After a team wins, say, six consecutive games, are they more likely than average to win the next one?

The measure in Type B is a bit more obscure.  Type B is saying something like, "are there streaks where a .500 team is really a .550 team, regardless of whether they win or lose during that streak?"

------

Now, you can always convert Type B to Type A.  That is, suppose your model says you get 100 consecutive chances at 40%, and then 100 more at 60%.  That's a hot hand in terms of probabilities, in terms of Type B.  Indeed, it's a *very* hot hand: with only one exception, a 40 is always followed by a 40, and a 60 is always followed by a 60.

But, in terms of Type A, a hit being followed by a hit, not so much.  If you work it out, you get

-- After a hit, the chance of a hit is 52%.
-- After a miss, the chance of a hit is 48%.

That is: if we assume a "hot hand" of type Type B that's absolutely huge -- 20 percentage points -- the corresponding Type A effect is much smaller, at only 4 percentage points.

With lesser effects, it's even worse.  Suppose that the Toronto Blue Jays show type Type B streakiness in probabilities.  If the weather is warm, they play .510 ball.  If the weather is cold, they play .490 ball.  That's a streaky pattern because temperature is streaky -- warmer in the summer and colder in spring and fall.

If you didn't know about this weather thing, would you be able to observe any streakiness?  No way.  After a win, the Jays are a .5002 team.  After a loss, they're a .4998 team.  You wouldn't notice that even after 1,000,000 games.  Seriously.  We're talking two games every 10,000.

Even if Type B is reasonably large, Type A is likely still small.

-------

That's what we saw in the previous post.  In his paper, Dan Stone showed a model where the "Type B" effect was 10 percent of the difference from .500.  Then, he added a bunch of randomness.  And, what happened?  The equivalent "Type A" effect was almost zero:

-- after a hit, the chance of a hit is 75.049 pct.
-- after a miss, the chance of a hit is 74.875 pct.

If the academic standard for this kind of research is a "Type B" model, I wish authors would also show us the "Type A" effect, so we can really see what we're dealing with, as fans.  In this case, we're dealing with one extra hit every 575 attempts, which is barely a "hot hand" effect at all.

-------

Of course, all this partially depends on what you're looking for.  If you just want to discover an effect -- any effect -- then it doesn't matter how big it is.  But, in a way, proving "existence" is unnecessary.  We *already know* there's a "hot hand" effect -- because of home field advantage.  

Players do better at home than on the road.  Therefore, a hit is more likely to be a home hit than a road hit, which means a hit is more likely after a hit (probably home) than after a miss (probably road).

Now, that effect is very, very small.  If you assume a .750 foul shooter is .751 at home, but .749 on the road, then the probability of a hit, given that the prevous attempt was successful, is .750001333.  Barely there at all -- but it's there, and we know it's there!

So, if you're looking just to prove the *existence* of an effect, there's no need.  I just did it.

And, of course, there certainly must other effects, of some magnitude.  I'm sure if a player's wife is mean to him one morning, he'll play differently that day.  Maybe better, probably worse.  It would be silly to insist that doesn't happen, that a .750 shooter is still a .7500000000 shooter no matter what's preoccupying him that day. 
  
But ... the effect might be very, very small.  If you want to prove it's there, good luck.  As I said, if he's .751 on good days and .749 on bad days, the Type A effect is one shot in 750,000.

Even if he's .760 on good days, but .740 on bad days -- which would be pretty significant, in my book -- he'd be .750133 after a success.  That's one shot in 7,519.

------

So it's not about existence -- it's about size.  We have evidence that *some* streakiness exists, albeit in very, very small amounts.  What the skeptics have to show is that it exists in some *significant* amount, in the real-world, "type Type A" sense. 

And, even that's probably not really enough.  Because, "hot hand" implies more than just correlation.  Suppose a baseball team trades for a bunch of superstars in August.  That  team will definitely show a "hot hand" effect.  If you check when it won three straight, it was most likely after the trade than before.  And, therefore, the chance of it winning the fourth game is higher than its average for the season.

But is that really a "hot hand," in the sense that people mean?  I doubt it.  I think they're talking about psychology, about how the three consecutive wins convinced the team it could compete, and that confidence and momentum extends into the fourth game and lifts them higher.  I think fans won't call it a "hot hand" if it's just caused by extra superstars, something they know is already there. 





Labels: ,

6 Comments:

At Monday, September 17, 2012 5:49:00 PM, Blogger Unknown said...

Hi Phil
Sorry we haven't been able to see eye to eye on this yet. Actually i agree partly with your initial A vs B example - fans are interested in the probability of a team winning the next one. My point is if you just look at wins/losses (0s/1s), you can't identify if team is hot. But fans have more information - strength of win etc. So fan judgments are not comparable to statistical analysis of made/missed shots.

I still don't know what you're referring to with this 1 extra hit every 575 shots. I analyze much bigger hot hands in my paper (eg, as we've discussed, situations where 50% shooters become 70% shooters). I know you say that it's implausible. That's fine - only strenthens my point - which is that even when the hot hand is implausibly large, it's really hard to identify.
Best, Dan

 
At Tuesday, September 18, 2012 2:50:00 PM, Blogger Phil Birnbaum said...

Hi, Dan,

I don't see how fans would have information on whether or not a certain game has above- or below-average probability. Unless you're talking about Vegas odds. If so, you can check for hot hands just by looking at the odds, can't you?

Or if you mean that fans can see whether it was an underdog win or an overdog win, then, sure. But that only helps a little bit.

1 extra hit every 575 shots means that, in the 575 shots each after a hit, the player will hit one more shot than he would in the 575 shots each after a miss.

 
At Wednesday, September 19, 2012 11:48:00 AM, Blogger Zach said...

Before reading any of the research on hot hands I saw it as more of the NBA Jam "He is on fire!" than the idea making one shot increases the likelihood of making another. I saw it as if a guy is shooting really well for a game it is evidence that there might be something different about the circumstances in this game that is leading him to play above his normal level. It could be a personnel or style mis-match, or his CPAP machine worked correctly the night before, or the localized gravity fit his shot better here than anywhere else.

Of course, evidence doesn't mean he actually is playing above his level, it could just be random variation. I guess I'd like to see some analysis of first half/second half and whether the variation is explained by randomness or if there is evidence of a possible effect.

 
At Thursday, September 20, 2012 6:14:00 PM, Blogger Unknown said...

A statistician only sees whether a shot was made or missed - a 0 or 1

Yes - i mean ex post not ex ante information - A fan sees not only whether it was an underdog win or not (ie the quality of the opponent), but the margin of victory, home advantage, statistics (whether the win was lucky or not) etc

What I meant re 1/575 is I don't know why you're using probabilities for my model that generate such a small effect, when I look at different probabilities, which would yield larger effects, in my paper. (although I'm also not 100% what kidn of parameterization of the model you're referring to) I don't think it's fair/appropriate to criticize my work for only applying/referring to small effects like that

 
At Thursday, September 20, 2012 6:21:00 PM, Blogger Phil Birnbaum said...

OK, fair enough. I did go with the smallest of the effects you used, mostly because I found them to be the most plausible. But I'll repeat for the larger effects.

 
At Friday, September 21, 2012 9:49:00 AM, Blogger Unknown said...

Thanks Phil. Don't mean to create work for you :) If you don't have time to pursue this further don't worry about it

 

Post a Comment

<< Home