Does the Wonderlic test predict QB performance?
The Wonderlic is a kind of intelligence test, used to screen potential employees. Apparently the NFL has been using it as part of scouting the draft.
Does a player's Wonderlic score predict how well he'll perform in the NFL? According to this study, by Arthur J. Adams and Frank E. Kuzmits, the answer is no. They found no statistically-significant correlations between Wonderlic score and measures of draft position, salary, QB rating, or games played.
However, a subsequent study by a testing company called "Criteria" *did* find an effect. Criteria looked at the 61 QBs drafted between 2000 and 2004. For performance measures, they looked at total passing yards and TD passes (which you would think would be so highly correlated with each other that they would be measuring almost the same thing, but never mind). They found a mild positive correlation of about .2.
But: when they included only quarterbacks with at least 1,000 yards – effectively eliminating the ones that didn't get any playing time – the correlation jumped to .5. As Brian Burke points out on his own blog, .5 is a HUGE correlation in a study of this type.
Now, it should be mentioned that Criteria has a vested interest in making aptitude tests look effective. If this is a trial and I'm on the jury, I'm going to assume that they tried a bunch of possible cutoffs and methods and chose the one that makes the test look the most predictive. Brian notes that if you change the cutoff from 1,000 yards to 2,000, and take Tom Brady (high Wonderlic, very high NFL performance) out of the study, the correlation drops to about .26 (still pretty significant).
I'm even more skeptical than Brian. Well, I am and I'm not. First, I would be willing to bet a lot of money that there IS a significant correlation between the Wonderlic and QB effectiveness. There's no doubt that QB requires good intelligence and quick decision-making skills, which is partly what the Wonderlic measures. In fact, I'd bet that Vince Young's supposed score of 6 out of 50 is a mistake – I don't believe that anyone with Wonderlic skills that low could have been a successful college quarterback, never mind a top-ranked one. (See these sample questions for yourself – supposedly Vince Young would only score 2 out of 15.)
So, yes, of course intelligence (or whatever you want to call what the Wonderlic measures) is important to making a good quarterback. But that's really not the question. The question is, *given what you already know* about the quarterback, in four years of extremely competitive college play, does the Wonderlic give you any *additional* useful information about the player?
On that, I'd bet that it doesn't.
Look at it this way: switch to baseball for a minute, and suppose I ask you to predict what Albert Pujols will do in the remainder of this season. You'd have a pretty good idea, and would make a fairly accurate estimate. Now, suppose I boot on over to St. Louis, give Pujols the Wonderlic test, and tell you his score. Will that affect your estimate? No, of course not. Whatever effect Pujols' Wonderlic skills have on his performance has already shown itself in several seasons of major league play. No matter how much Wonderlic correlates with having a high OPS, if you know the player's skill in achieving OPS, no intelligence test is going to give you a better estimate.
Where the Wonderlic would help you is if you didn't know what real-world skills an applicant has. It would be much more useful to know the Wonderlic scores of presidential candidates than football players, because, for instance, we don't have a good idea of their president skills -- we haven't seen Obama or Hillary play president for four years in college. It would be less useful for George W. Bush, because we have strong, first-hand observations on what he's like as president. Did he achieve his Bushness with a high Wonderlic, a medium Wonderlic, or a low Wonderlic? That might be interesting trivia, but shouldn't affect how we evaluate his potential, because we've actually seen him in action.
And the same is true for drafting quarterbacks. The question we *really* want to answer, in the NFL context, is this: given what you know about the player, and have discovered in the NFL combine and from watching him play college football for four years, does the Wonderlic tell you anything *additional* to that?
That is: you have two identical quarterback candidates, with identical college records playing for identically-skilled college teams against identical opponents. At the combine, they look exactly the same in every respect, except the Wonderlic. One of them has a Wonderlic score of 30, and one has a Wonderlic score of 20 (24 is average for QBs). Should you expect a difference in NFL performance?
It's possible to study that question, but I think you'd have trouble finding enough data, and adjusting for the different environments in which quarterbacks play, to be able to make any significant conclusions.
And intuition suggests that they'd probably be about the same. If A and B had equal performance, despite a being significantly "smarter" than B, it follows that B must have had *other* compensating aptitudes that made up for it. Doesn't it?
As for this study, if you believe the results, all we have is a conclusion that better and faster thinkers make better quarterbacks. Well, duh.
Hat tip: NFL Stats (Brian Burke)