How Data Science became an integral part of Lok Sabha elections 2019

The former Apple retail chief, Angela Ahrendts, said that last year: “Consumer data will be the biggest differentiator… Whoever unlocks the reams of data and uses it strategically, will win.” Apple’s focus may have been to augment the customer base, however, the same logic may be applied to the electoral process to a great extent, if the data is harnessed properly.

In the past few years, the modus operandi of reaching out to the masses as part of the electoral process has changed, which has happened not as much at the front, rather, at the backend - a previously unchartered territory.

The change has been triggered with over half a billion internet users, a figure, estimated to reach 627 million this year according to a report by Economic Times. Growing in proportion is the number of social media users, a figure that has already crossed the 350 million mark.

Impact Factor

How significant this phenomenon is can be gauged by how government agendas are shaped. When a controversial statement promoting toilets over temples created an uproar in social media, the concerned communications team huddled into damage control, channelling the sentiments in a new version - 'sanitation for all.' Similarly, due to public resentment, a result of social media uproar and feedback from surveys, a step was taken to waive off a certain amount of loans taken by farmers of UP.

Data Science & Lok Sabha Elections 2019

Data Science is about mapping datasets and making sense out of them. With 725 districts in India, advanced analytics had made it possible to study their individual patterns and relevant local issues. Political parties engage the top of the line digital experts for a lot of innovative work that happens behind the scenes. The invisible hand has guided the system towards a data-centric approach with a high precision predictive analysis.

The 2019 Lok Sabha elections saw an exceptionally accurate exit-poll analysis, thanks to over eight lakh voters surveyed in over 540 constituencies. According to a polling organisation, the sample size for each exit poll varied from 40,000 voters to two million. 

India Today-Axis My India post-poll again proved to be the most accurate survey, availing the capabilities of their robust mechanism of data collection, reporting, and analysis techniques.

The key here was a reasonably large sample size with as much diversity in the subjects as possible; this had to be corroborated with data cleaning, a fundamental step in using data science to derive precise analysis. Using appropriate technology is critical not only to collect data but also to ensure that the data is devoid of redundancies and sampling errors.

A good starting point for a Data Analyst is access to publicly available data, in this case, the data with the Election Commission and commissioned data from various other government websites, for instance, the census data from Ministry of Home Affairs and government-sponsored initiatives that amass user profiles. The next logical step is to mash those datasets with the ones collected from social media platforms, mobile apps, Geographic Information Systems (GIS) data, and telecommunications data. Where available, banking, demographic and socio-economic data may be added to drill down further and obtain a higher quality analysis.

There are numerous party-based apps, often for constructive reasons; then, there are sentimental videos, clips, news items - both pro and against a party or ideology - that are downloaded, viewed, clicked, liked and shared by millions of users.

Data Science can help political parties track positive or negative comments of masses, and hence, their level of loyalty towards particular parties based upon their online behaviour as it says a lot about personal political inclination. A poll strategist who worked with the Bharatiya Janata Party (BJP) in Manipur and Tripura revealed that BJP had over 25,000 WhatsApp groups in the northeast states which enabled them to penetrate their ideology at the grassroots, leaving Congress way behind as an influential force.

The use of Data Science in psephology has gained such significance that in the US, premier educational institutes like MIT have created the Election Data Science Lab. Its primary purpose is to provide robust Data Science for collecting and analyzing data and sharing its findings, trends, and forecasts. As good as it gets in terms of precision, such labs do, what they call, sentiment analysis or opinion mining, broadly referred to as data mining. Traditional methods of opinion polling - landline subscriber surveys - which were an important tool for conducting surveys, are all waning, considering the doubt on their accuracy.

The Future

How a simple interaction can be used to profile us is one thing; taking that one step further is the China model whereby over $20 billion were spent to install CCTV cameras to monitor its citizens’ movements. Were you aware that there is a lexicon constituting 25,000 words that interpret tweets, emojis, and even social media reactions?

The x-factor of the Indian constitutional canvas is the humongous set of data accessible to the political groups. The more data that is available to them, the more they have an edge in assessing the public mood. The insights gathered can then be used to catapult positive sentiments to votes or undertake remedial measures to invert negative trends.

The Election Commission has regulations on campaigns and speeches prior to polling dates, however, the online marketing tools continue to operate, raising questions on their restriction in future. What may be considered as inappropriate, controversial or inciteful in the real world, manages to find its way to limelight in the digital world. Hence, laws need to encompass the deeds or misdeeds of those with vested interests.

From an analyst’s point of view, the future is very bright. What can be a greater testimony to this than the fact that Harvard Business Review has labelled Data Scientist as the “The Sexiest Job of the 21st Century.” With the advent of the Internet of Things, better known as IoT, our interaction with smart devices has become a source of churning out tons of data by the second. By 2020, it is estimated that 1.7MB of data will be created every second for every person on earth. Imagine the impact on democratic processes, and the very future of nations and societies, if data analysts can harness all this information to swing votes to the ‘left’ or the ‘right’!

About the author

Schooled in India and the US, Dr Yavar Ehsan has completed his graduation and post-graduation from Temple University's Fox School of Business, Philadelphia. He later completed his PhD from Jamia Millia University, Delhi. Prior to settling down in the academia in India, he has worked in Michigan, Pennsylvania and New Jersey. This has helped him gain a rich exposure with various international corporations as well as experience in multi-cultural workforce management. Currently, he is the Associate Professor of Management at Indian School of Business & Finance, Delhi.