Today, Nate Silver of FiveThirtyEight posted a long explanation of how his Senate forecasting model works this year — check out Vox’s new forecasting hub to see how it compares to several other models. But in today’s post, Silver mentions that he has particular objections to the design of one competing model. That’s Princeton neuroscientist Sam Wang’s model, which in recent weeks has given Democrats more rosy projections than any others, showing them with a clear Senate advantage. Silver writes:
Nate Silver: Sam Wang’s model showing Democratic Senate advantage ‘is wrong’
Fred Lee/Disney-ABC/Getty
Andrew Prokop is a senior politics correspondent at Vox, covering the White House, elections, and political scandals and investigations. He’s worked at Vox since the site’s launch in 2014, and before that, he worked as a research assistant at the New Yorker’s Washington, DC, bureau.
That model is wrong - not necessarily because it shows Democrats ahead (ours barely shows any Republican advantage), but because it substantially underestimates the uncertainty associated with polling averages and thereby overestimates the win probabilities for candidates with small leads in the polls. This is because instead of estimating the uncertainty empirically - that is, by looking at how accurate polls or polling averages have been in the past - Wang makes several assumptions about how polls behave that don’t check out against the data.
Silver specifically argues that, by building so little uncertainty into his model, Wang’s projections don’t present an accurate estimate of the chances that the polls may be wrong — either in certain races or overall. He argues that Wang’s 2010 model estimated absurdly low probabilities for events that actually ended up happening — Sharron Angle losing to Harry Reid and the GOP picking up 63 seats in the House.
One of Silver’s broader points is that his model makes extensive use of historical data about how much polls have been off by in the past — because it’s not at all infrequent for polls to miss the mark. Wang’s model, though, doesn’t take this into account. “There are many elections in which all or almost all polls are biased in the same direction,” Silver writes, adding that Wang “underestimates the chance for the underdog to win because of systematic errors in the polls, better-than-expected turnout, and so forth.”
Update: Check out a response from Wang here, courtesy of the Washington Post’s Greg Sargent.
See More:
