Nate Silver tweeted yesterday that the “Democratic Party is rigged … in Sanders’s favor,” referencing an article he co-authored with Harry Enten from FiveThirtyEight.com.
— FiveThirtyEight (@FiveThirtyEight) May 26, 2016
I nearly spat my tea out.
As an admirer of Silver’s work, I was very curious to read the article to understand the logic behind this assertion. But I was dumbfounded when I realized that the Silver’s and Enten’s analysis doesn’t hold water. I was so disappointed that I exploded on Twitter.
— Salam Morcos (@SalamMorcos) May 26, 2016
Washington and Nebraska “Beauty Contest” Primaries
The article points to the two contests were both a caucus and a primary were conducted. This is such a waste of tax-payer money because only the results of the caucus matter. In both Washington and Nebraska, Sanders overwhelmingly won the caucus contests, while Clinton won the “beauty contest” primaries where no delegates would be allocated to either candidate. Silver and Enten used the primary results in their model.
This is absurd!
In fact, there were no exit polls in these “beauty contest” primaries to understand the demographics of who voted for them. It is very conceivable that many of those voters are lifelong Democrats who tend to be older in age. Younger voters who overwhelmingly support the Vermont Senator are known for being unreliable voters. How many of those bothered to vote in primaries where they know the result wouldn’t matter? I don’t know. Nate Silver doesn’t know. No one knows!
But to use the primary results as a predictor of how the result would have been if the states held a primary is absolutely ridiculous.
In fact, there are other indicators that prove the opposite of what Silver and Enter asserted. While it’s unfortunate that nearly no polls were conducted in Washington and Nebraska, there is an old poll that we can have a quick look. Back in May of 2015, a Gravis poll showed that Hillary Clinton was leading Bernie Sanders by just 9 points. That’s only 3 points more than the result of the Washington primary itself. Remember, this was back in May of 2015 soon after Sanders announced his candidacy for the Democratic nomination. According to Real Clear Politics, Clinton was leading Sanders by 57 points nationally at that point.
So if Nate Silver’s argument was valid, that Clinton would have won Washington State by 6 points if it were a primary, then he’s arguing that Sanders was only able to cut Clinton’s lead by 3 points in the State despite cutting the national lead by over 50 points. Now is that logical?
Evaluating the Model
All forecasting models rely on factors and formulas, some of which are based on historical data, others on estimates and predictions. But in every case, to assess the reliability of a model, statisticians and analysts rely on other reliable tests to see if the output of the model is reasonable.
A simple check would be to compare the results of the model to that of opinion polls. It’s very sensible to assume that the results from the primary would have been similar to some certain extent to that of opinion polls. If that’s not the case, Nate Silver should probably find himself another job.
Unfortunately, pollsters don’t seem very keen on conducting opinion polls in caucus states. The only exceptions are the first in the nation caucuses: Iowa and Nevada. Now let’s compare the polling numbers to the projections developed by Five Thirty Eight’s model.
As the graph shows, the results of the caucuses in Iowa and Nevada are very close to what the polls predicted. However, the projections predicted by Five-Thirty-Eight are off the charts! It predicts that Clinton would have performed better than the polls by 20 points in Iowa and 27 points in Nevada.
With all due respect to Nate Silver and Harry Enten, this is nonsense!