Myths and the reality of election forecasts
It is difficult to comment on the predictions of most Indian pollsters as they do not publish their methodology
Richard Charnin’s 2010 book, Proving Election Fraud: Phantom Voters, Uncounted Votes, and the National Exit Poll, investigated possible irregularities in Democrat John Kerry’s defeat in the 2004 United States presidential election.
The final national exit poll was the real “Smoking Gun”. The difference between the vote shares of John Kerry and George W. Bush was 6.5% more than the actual.
- Interestingly though, such was the credibility of exit polls even one and a half decades ago. Take the example of the Venezuelan recall referendum of 2004 to determine whether Hugo Chávez, then President, should be recalled from office. A huge discrepancy with the exit poll created a massive uproar worldwide.
- The “Orange Revolution” in Ukraine during late November 2004 to January 2005 was also in the immediate aftermath of the presidential election. The allegations of electoral fraud were strengthened by several exit polls exhibiting a substantial lead for Viktor Yushchenko. Similarly, widespread protests over the disputed parliamentary elections of 2003 triggered the “Rose Revolution” in Georgia, and culminated in the ousting of President Eduard Shevardnadze.
- One wonders if polls predictions are still so trustworthy. Certainly not. In 2020, U.S. (President-elect) Joe Biden won the election, and not many people are looking back to the predicted margin in nation-wide popular vote share which was huge (mostly within 8-12%), but never materialised. Also the credibility of pollsters must have been undermined quite a bit in between. People possibly learned to listen to them, but not to keep much faith in their predictions.
- In reality, the difference in popular vote shares in this U.S. election is just about 3%. And, in most of the States, the difference between the predicted percentage of votes for Mr. Biden and U.S. President Donald Trump compared to Mr. Biden’s final lead is more than 3%.
The Bihar mandate
- Pollsters performed miserably in Bihar in 2015 in one of their worst performances in Indian elections. Although the opinion polls mostly predicted a victory by the National Democratic Alliance this time, very few of them could guess the quantum.
- The exit polls, on the other hand, mostly predicted a Rashtriya Janata Dal+ victory. A funny thing is that the minimum and maximum predictions for the RJD+ were 76 seats (a range of 71-81) and 180 seats (a range of 169-191), out of 243 seats.
- One wonders whether the RJD+ securing anything less than 71 or more than 191 was at all practicable. Thus, there will be some pollster or other to claim a perfect or near-perfect prediction.
Surveys and lessons learnt
- Yes, poll predictions have failed miserably on many historical occasions including some which were in the developing process. Before the 1936 U.S. presidential elections, a reputed magazine, the Literary Digest, conducted an opinion poll survey with a massive 24 lakh samples, and predicted 57% vote share for Republican Alfred Landon, and 43% for President Franklin Roosevelt. In reality, Roosevelt got 62% votes against 38% favouring Landon.
- This episode was a lesson about the importance of ‘selection bias’, as the individuals under study were affluent people having a telephone, club memberships and magazine subscription in that era; Roosevelt had less support among such people.
- Also, people understood the importance of ‘non-response bias’ as only 24% of the people who were approached actually responded to the survey.
- Similarly, the severe failure of Gallup’s poll prediction of the 1948 U.S. election taught pollsters about the importance of ‘random’ sampling. In 2016, Mr. Trump supporters were under-counted.
- Certainly, the winner of opinion/exit polls is not necessarily the winner in the election. Hillary Clinton, Ed Miliband or Atal Bihari Vajpayee could vouch for that. Most of the opinion polls of the 2009 and 2014 Lok Sabha elections either failed to project the winner or to foresee the margin of victory.
- A small State like Delhi has caused misery for pollsters, be it in 2015 or 2020. In the U.K., for example, no major survey could predict the victory of Conservatives in 2015. And the history of the Brexit referendum is a part of folklore, for sure.
- Over decades, pollsters around the world have transformed poll prediction into a funny game. When Warren J. Mitofsky conducted an exit poll in a local election in Kentucky in 1967 for CBS News and also conducted the first national exit poll in 1972, he never envisioned such a future within half-a-century, for sure.
- The Literary Digest never attempted to predict the outcome of the U.S. elections after its 1936 debacle. That was never its main task. However, unless strong regulations enforce today’s pollsters to publish their methodology and summary statistics across different variables, poll predictions may not regain the trust of the people.