The NFL draft is starting Thursday night. Given the huge amount of attention that the draft gets every year, every aspect of the player scouting process is picked over ad nauseam.
Reminder: The NFL’s Wonderlic aptitude test is totally worthless
That includes the Wonderlic — a timed 50-question aptitude test that players take at the scouting combine in February. (You can take a sample test here.) Wonderlic scores are supposed to be secret, but every year, a few players’ scores get leaked by the press and obsessed over.
Low scores cause fans to worry that the players won't be smart enough to make it in the pro game. High scores relieve fans' concerns about a player's ability to absorb a playbook. This year, Johnny Manziel's 32 (out of a possible 50) has earned praise; last year, Tavon Austin's low score of 7 caused some to doubt his long-term potential.
But we’re here to tell you what a growing number of NFL executives already know — the Wonderlic is totally worthless.
Studies have repeatedly shown that there’s no correlation between a player’s score and his NFL success. Some even believe that the test is racially biased — part of the reason that the league introduced a new aptitude test last year to be taken alongside the Wonderlic.
There’s no good evidence the Wonderlic predicts NFL success
Hall of Fame QB Dan Marino scored a dismal 15 on the Wonderlic. RHONA WISE/AFP/Getty Images
The written test was developed in the 1930s by psychologist Eldon F. Wonderlic to help companies select potential hires on the basis of their math, vocabulary, and reasoning skills. After being used for decades by the Navy, it was introduced to the NFL by Dallas Cowboys coach Tom Landry in the 1970s. Eventually, it became standard practice to have every player take the test at the scouting combine.
All this despite the fact that there's no known research establishing the fact that the Wonderlic's questions — which involve arithmetic, analogies, and knowing which month comes eighth in the year — have anything to do with success on the football field. In recent years, researchers have looked into the matter and consistently found they don't.
A 2009 study of 762 players from three draft classes found that for all but two positions, there was simply no correlation between a player’s Wonderlic score and his future NFL performance. For the two exceptions — defensive back and tight end — there was a negative correlation. In other words, a low score was more strongly correlated with eventual NFL success than a high score.
Quarterbacks’ Wonderlic scores, in particular, have gotten a lot of scrutiny, because playing the position is so strategically complex. But a 2008 study that looked specifically at 84 quarterbacks drafted between 1989 and 2004 found there was no correlation for QBs’ scores either. A 2011 study that judged how NFL teams evaluate quarterbacks as a whole found, again, a weak negative correlation between Wonderlic scores and future success.
The caveat with all these studies is that Wonderlic scores are supposed to be kept secret — so they all rely on leaked scores that could be inaccurate. Mike Callans, a VP at Wonderlic, Inc., has used this point to dispute the studies. But neither he nor anyone else has released research using verified data that comes to the opposite conclusion.
The Wonderlic might discriminate based on race
Elite cornerback Patrick Peterson scored a 9 on the test. Photo by Christian Petersen/Getty Images
Last year, the league introduced a new aptitude test to be taken along with the Wonderlic at its rookie scouting combine. The new test came at the urging of Cyrus Mehri — the attorney who co-founded the Fritz Pollard Alliance, an organization that's worked to promote the hiring of minorities for coaching and front office jobs in the NFL.
Mehri didn’t specifically mention race as a reason for the new test, but did note that it “levels the playing field from a socio-economic point of view.”
That the Wonderlic might have a built-in racial bias — like many standardized tests — is a common criticism. Studies have consistently found disparities in scores between white test takers and minorities, both among NFL players and people who are assigned the test for real-world jobs: the 2011 quarterback study, for instance, found that African-American QBs average a score of 20.2, while white QBs average 27.7.
It’s unclear whether this sort of disparity is caused by an unfair test with questions that are inherently harder for minorities. But a 2012 study did find that NFL teams, on the whole, seem to interpret Wonderlic scores differently based on race: for a white player, a test score that’s ten points higher leads to getting drafted 14.7 spots earlier, while for a black player, it only means a rise by 6.4 spots.
Companies other than the NFL that administer the Wonderlic to screen potential hires have repeatedly been sued on the grounds that the test is discriminatory. In 1971, as part of a case against Duke Energy, the Supreme Court ruled that because the test effectively discriminated on the basis of race and had no relationship with job performance, it couldn’t be used.
Even the NFL is over the Wonderlic
The NFL’s new test, developed by industrial psychologist Harold Goldstein, is designed to eliminate judging players on the basis of pre-existing knowledge, which can reflect race and socioeconomic factors. Instead, the computer-based test — which closely resembles a test commonly given to firefighters — looks at players’ ability to quickly learn new concepts and analyze situations under pressure.
The idea is that these qualities are more important for success on the field than knowledge of vocabulary words and comparison of geometric shapes. Of course, using a computer screen to simulate the pressure of an NFL game and the kind of quick, instinctive response often needed to succeeded in it is extremely difficult, and it’s unclear whether this new test will predict NFL success any better than the Wonderlic.
But one thing that is clear: the Wonderlic itself is useless. It’s time to stop worrying about players’ scores. Johnny Football might make a great NFL QB, but not because he’s good at math.
See More:
