The result of the US election came as a revelation to many. Wednesday morning, analysts and pollsters had a difficult day to face as most of them had to explain how they got it so wrong. Right before Election Day, most polls and forecasts were projecting Democratic candidate Hillary Clinton as the winner of the presidential race. As one state after another reported a win for Republican candidate Donald J. Trump, analysts and political pundits were stunned. Many were left wondering where they went wrong in their analysis. The general public was left wondering whether they could believe the findings of big data.
Results at the Polls Take Analysts by Surprise
Election night was a rough night for big data. Forecasts and polls failed to accurately predict the outcome of the election. It is the second time a popular vote takes pollsters and analysts by surprise. The day after the referendum in the UK on leaving the European Union was equally difficult for number crunchers. The result at the polls took everyone by surprise. In its aftermath, the Brexit was called a black swan that nobody could have anticipated. Trump’s win seems to have had the same unexpected quality to it. But some are starting to wonder if maybe we’re not overestimating the ability of data to predict events.
In various fields of human activity from business to politics and from sports to academia we have become increasingly reliant on big data. We hope that by using artificial intelligence to sift through large volumes of data we will reach the right conclusions about many different decisions. But recent events have shown us the limitations of big data. They have also brought home the point that we may have set our expectations too high.
Predictive analytics and especially election forecasting is still a young science. Big data analysis is not the equivalent of having a crystal ball. Despite having almost every major forecast declare her the winner, Hillary Clinton lost the election. But many of the forecasts lacked sufficient context that would have explained that there was a wide margin of error to those forecasts.
With the Spotlight On, Big Data Fails
The business of election prediction is just one small slice of the pie that is big data. A change that has far-reaching implications has taken place in many industries that have started to put a strong emphasis on data. The promise of data is finding insights that will save costs and drive profits up. So, data mining is something that more and more businesses are integrating into their activities. It is a behind-the-scenes technology that permeates business decisions from ad campaigns to billion-dollar acquisition deals.
However, data science is a new technology that comes with trade-offs. It can uncover things that would otherwise have gone unnoticed. But it can also be a blunt instrument that misses context and nuance. Companies, as well as institutions, use big data currently. But they keep a low profile about it. This election gave everyone the opportunity to see how the use of data works. And how great it is the possibility that it could fail.
Image source: here.