tag:blogger.com,1999:blog-31545676.post3570946466997940489..comments2019-10-14T15:40:55.203-04:00Comments on Sabermetric Research: Does previous playoff experience matter in the NBA?Phil Birnbaumhttp://www.blogger.com/profile/03800617749001032996noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-31545676.post-33591947273871545082017-11-08T15:26:12.255-05:002017-11-08T15:26:12.255-05:00OK, I have a better answer to Tango.
There are tw...OK, I have a better answer to Tango.<br /><br />There are two issues when you use a formula to estimate team talent. First, is your estimate unbiased (as likely to be too high as too low)? Second, how accurate is it (how big is your confidence interval)?<br /><br />In the NBA, as Tango mentions, you need only add 12 games of .500 ball to the record to get an estimate of talent. But that means only that adding 12 games is how you get *the unbiased estimate*. If you add more than 12 games, you're going to wind up with more estimates being too close to the mean. If you add fewer than 12 games, you're going to wind up with more estimates being too far from the mean.<br /><br />But: the "add 12 games" has nothing to do with the *variance* of your estimate, with your accuracy, with the confidence interval around the estimate. That confidence interval is still based on binomial luck. If you use 82 games of performance, your confidence interval will be a certain size. If you use 164 games of performance, your confidence interval will be only .7 times as big (1 divided by root 2).<br /><br />So, more games is always better, and always better in the SAME RATIO, if your estimate is unbiased. No matter what sport you use, you cut your confidence interval in half by using four times as many games.<br /><br />So, the fact that NBA adds only 12 games does not, in any way, mean that your confidence interval is narrower than in MLB when you add 70 games.<br /><br />------<br /><br />HOWEVER: the confidence interval for the NBA is already a bit lower than for MLB. That's because, in the NBA, you have lots of mismatched games. The SD of the outcome is proportional to the square root of p(1-p), where p is either team's probability of winning. In the NBA, it's common to have 4:1 favorites. Root [p(1-p)] equals 0.4 for those games. For an even game, it's 0.5. <br /><br />If you assume that the "average" favorite in the NBA is a 3:1 shot, and in MLB it's a 1.5:1 shot, the SD difference is 0.43 to 0.49, so the NBA confidence interval is about 12 percent smaller. But, still, you gain the same proportion of accuracy by increasing the proportional number of games, in both sports. <br /><br />So, the argument stands: when you look at previous playoff experience in the NBA, you're getting strong evidence of how the team did in the past. That would also apply to MLB. But, actually, it's much more valuable in the NBA, because how far a team went in the playoffs is much more correlated to skill (ie, much less luck) than in MLB. So last year's playoff experience gives you a LOT of information on how good the team was last year. Probably not as good as their actual regular season record, but much closer than it would be in MLB.Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-13382356299918519152017-11-03T16:03:44.858-04:002017-11-03T16:03:44.858-04:00Guy makes a very good point regarding individual p...Guy makes a very good point regarding individual player talents vs team talent. I believe, for team sports like baseball, basketball etc, any model that is based solely on individual player talent will perform much better than Elo models, or even a combination of Elo and player.<br /><br />So, let's say you have the ZiPS/Steamer projections for all the players in both teams of an MLB game. Is there *any* advantage in considering the team records when calculating the win probability? I cannot imagine how team record can add anything but noise here.<br /><br />Basketball *is* somewhat different than baseball, in that team synergy may have a larger impact. But I am still willing to bet any model that completely ignores team records, but only considers player talent will be better.jonashttps://www.blogger.com/profile/01032950586923321304noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-70724741889746954622017-11-02T06:49:12.740-04:002017-11-02T06:49:12.740-04:00I think there is a problem with the experience var...I think there is a problem with the experience variable, even if the decay rate is perfect. Even if Elo is perfectly calibrated, and has wrung all possible predictive power from the team's W-L and points scored/allowed, couldn't our predictions still be improved by adding a separate measure of *individual* player talent? If we had a perfect Elo for baseball, wouldn't knowing each player's career WAR still provide useful predictive power? And surely playoff minutes is highly correlated with player talent in the NBA.<br /><br />Indeed, we don't have to guess about this. When 538 predicts game outcomes, it uses a hybrid measure combining Elo with their estimates of player talent ("CAMELO"). That does a better job than Elo alone. So I think we know that there is a need to control for players' individual talent, beyond Elo.Guynoreply@blogger.comtag:blogger.com,1999:blog-31545676.post-58870849606597860592017-11-01T21:57:38.981-04:002017-11-01T21:57:38.981-04:00Elo weirdness could be part of it, but I think pla...Elo weirdness could be part of it, but I think playoff rotations play a role as well. Say the Cavaliers were a 1600 Elo in the regular season, but we know they have a lot of playoff experience in their top guys. When you get to the playoffs and they heavily/only play those top guys, they might really be a 1700 Elo team. If alternate reality Cavs were a 1600 team but without a lot of playoff experience, their best players probably aren't as good as "Cavs prime". So even when they shorten their bench in the playoffs, maybe they're only a 1650.Alexnoreply@blogger.comtag:blogger.com,1999:blog-31545676.post-48428318396677831912017-11-01T19:17:21.541-04:002017-11-01T19:17:21.541-04:00Also: a 12 percent weighting implies that you give...Also: a 12 percent weighting implies that you give a 50% weighting to the last 27 games, and a 50% weighting to everything before that.<br /><br />Specifically, 50% to the last 27 games, 38% to the previous 54 games, and 12% to everything before that.<br /><br />So, each of the last 27 games gets about 2.5 times the weight of each of the first 54 games of the season.<br /><br />That can't be right, can it? Can it be that team talent -- ON AVERAGE -- changes so much that the 70th game gives you 2.5 times as much information about the team's talent as the 28th game?<br /><br />That's also easy to check. Try to predict the last third of the season from the first two thirds. Then, try to predict the last third of the season from the sum of (first third + 2 * second third). If the second way gives you a larger standard error than the first way, you know your weighting is wrong.<br /><br />Even better: try to predict the last game of the season from (4 * the last 27) + (2 * the middle 27) + (the first 27). I bet you get a lot weaker a result than if you weight them equally and just use the results of the first 81.<br /><br />This is still an argument to your gut, for now.Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-21198440563192810242017-11-01T17:08:06.512-04:002017-11-01T17:08:06.512-04:00You could find out by running a regression to pred...You could find out by running a regression to predict this season based on (1) last season, and (2) the season before. That should give you the relative weights. I don't have the data handy, but I'm guessing the coefficient of the T-2 season will have a better than a 9:91 ratio to the coefficient of the T-1 season.<br /><br />Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-25654873625043966632017-11-01T17:04:39.146-04:002017-11-01T17:04:39.146-04:00OK, I take that back. I think the argument does i...OK, I take that back. I think the argument does indeed depend on the appropriate decay rate. I still think a decay rate that weights the previous season at 9% is too high. Tango, do you have any argument otherwise?Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-89395000947361012902017-11-01T16:59:31.558-04:002017-11-01T16:59:31.558-04:00There are two sources of error in estimating talen...There are two sources of error in estimating talent from outcomes:<br /><br />1. Random luck going from talent to outcome; and<br />2. Changes in talent over the sample you're using to estimate.<br /><br />When you decide to make your sample bigger, by, say, adding the previous year's outcomes, you're doing so because the improvement in accuracy by reducing random error (1) is greater than the loss of accuracy due to changes in talent that you can't see (2).<br /><br />As you say, in the NBA, (1) is smaller than other sports. So, you get less improvement by adding data. Which means that if you use data from too far back, your losses from (2) more quickly overcome the gains from (1). And that's why, for the NBA, the decay rate has to be higher. <br /><br />If you were flipping coins, where (2) is zero, the decay rate would be zero. And if you're measuring a child's height, where (1) is close to zero, the decay rate would be close to infinite.<br /><br />Now, what's the appropriate decay rate for the NBA? You could be right that because (1) is low, (2) should be high, and a 9% weight for the previous year (after the adjustment for personnel) is pretty good.<br /><br />You may be right, but I still think it's way too low.<br /><br />But: I now realize that doesn't matter that much here. It's sufficient for my argument that 9% is too low, but it's not necessary. <br /><br />Let me check on my dinner cooking and I'll be back.<br /><br />Phil Birnbaumhttps://www.blogger.com/profile/03800617749001032996noreply@blogger.comtag:blogger.com,1999:blog-31545676.post-76889965151803808272017-11-01T15:15:13.655-04:002017-11-01T15:15:13.655-04:00I think I'm ok with a rapid "decay" ...I think I'm ok with a rapid "decay" rate. <br /><br />For example, in baseball, we know the regression point is around 70 games. So, playing 162 games means that 162/(162+70) = 70% of the weight is from the performance. The other 30% is regression, or if you had earlier performance, it can come from there too.<br /><br />In basketball, the regression point is only 12 games IIRC. So, playing 82 games means that the weight comes 87% from the performance. There's just not much reason to go earlier than this season.<br /><br />I think.<br />Tangotigernoreply@blogger.com