Big Data Revolutionizes Journalism: Crunching Numbers for Predictions is the New Norm

For businesses, the 20^th century was like a two edged sword; all due to the fortunate and unfortunate events that put the world into overdrive. The great depression followed by the collapse of market due to the world war latched unbreakable shackles on industries and organizations. But the economic expansion after the war, which was hailed as the Golden Age of Capitalism by many brought the global market back on its feet. Amid the fall and rise of many businesses, there was one industry that survived all—the Press.

With the coming of Big Data in the 21^st century, data has become the priority - regardless of the industry and in its wake the old age journalism has passed on the mantle to the more precise data journalism. People as always, like a good story but now with the advent of new technologies and methods, they demand for more accurate reports—and who is to say they don’t deserve it? Data journalism is all about identifying stories other reporters can't. It is a journalism specialty reflecting the increased role that numerical data plays in the production and distribution of information in the digital era. It reflects the increased interaction between journalists and several other fields such as design, computer science and statistics. From the point of view of journalists, Data Journalism represents an overlapping set of competencies drawn from disparate fields. Data journalism has been widely used to unite several concepts and link them to journalism. Some see these as levels or stages leading from the simpler to the more complex uses of new technologies in the journalistic process. Chief amongst them are computer assisted reporting, data visualization, interactive visualization and Serious Games.

Stay ahead of the industry with exclusive feature stories on the top companies, expert insights and the latest news delivered straight to your inbox. Subscribe today.

The problem with data journalism though, is that there is a shortage of data professionals. The issue lies in the education systems. Colleges offer computer science, statistics and business as distinct subjects. But these are all skills that a data journalist or a data scientist in general must possess and there are hardly a handful of institutions that offer programs that include all the three subjects. This means organizations are falling short of options and have to build teams of differently skilled workers rather than a specialist. There are even cases where a candidate might have ample knowledge about two of them but a professional, well drilled and adept to all the three are few and far between.

Although there is a concern regarding the work force in this field, there are a few solutions one can apt for, in particular online courses. There are a handful of organizations which have recognized the issue now, at its infancy, and are providing online lessons - on each subject in particular or data journalism itself as a whole. There are still some organizations, which hire candidates who are proficient in two of those subjects and send them back to school to learn the third. Although it is an option, it is advisable and in the good will of both the parties that they go with the former, as the employee would still be at the firm, working whilst learning.

Even when the workforce problem has been dealt with, there is always an issue with Big Data itself. Renowned statistician Nate Silver is one of the first to successfully use Big Data to make successful predictions. He has witnessed forecasts go horribly wrong for others and explains why Big Data might be an issue in itself. According to Nate, Big data can mean more distraction, more false positives, more bias and more reliance on machine-learning - all of which can falsify analytical results and interpretation and lead to potentially disastrous decisions. When there is more data, people have the opportunity to cherry-pick the results they want to see and unless they are careful, it produces more bias in the end, which is a disaster. But there is a catch. Nate, whose predictions have been precise to a great degree, makes use of three principles from Bayes’ theorem to pave its victory lap: ‘Know where you're coming from; think probabilistically; try, err and try again.’

Firstly, acknowledging bias up front can help businesses be more aware of their blind spots, and be more open to novel results in the data, which often offer the best opportunities for competitive advantage. Second, adding probability to data and analytics - helps businesses prepare and avoid potential disasters. And third is trial and error. For every two steps forward, there’ll be a step backward and the data journalists would do well to learn from the mistakes at each step and rectify them as they go.

It can thus be deduced that regardless of the drawbacks, big data is shedding its shadow on journalism as well and Data Journalism is slowly but definitely gaining momentum. Number crunching, data and statistics seem to be the way forward.