Can Big Data Solve Universal Problems?

By CIOReview | Friday, April 11, 2014

FREMONT, CA: Big Data is in the news for various reasons and is growing in popularity. It is being collected, analyzed, used to make money, and also has the power to solve just about any problem that may exist in the universe like analyzing a zillion queries for predicting flu outbreaks, or to check millions of restaurants for the best gourmet food, or to even check on millions of airlines for the best time to buy tickets, Big Data is there to help, reports Gary Marcus and Ernest Davis for New York Times.

Big Data combined with the intensity of modern computing, can solve virtually any problem, be it related to crime, health, travel, and weather just by crunching numbers.

Peter Tucker in his book on Big Data, “The Naked Future,” states, “In the next two decades, we will be able to predict huge areas of the future with greater accuracy than ever before.”

Big Data can be used to detect correlations between any related or unrelated topic; however, it does not identify which correlation is more meaningful or accurate.

Despite its usefulness, Big Data can never be the replacement for scientific inquiry. For example, molecular biologists, using Big Data, would like to deduce the 3D structure of proteins from their underlying DNA sequence; however, no scientist will think that number crunching and statistical data alone can solve the problem. They prefer to rely on an analysis of physics and biochemistry.

There are innumerable tools which are developed as solutions and applications based on Big Data and even cater to educational institutions as well for grading. For example, the applications developed for grading students essays have algorithms and coding which use measures such as the length of the sentence, word sophistication, and sentence structure, which is found to be in correlation with scores given by human graders. Though Big Data is useful, fast and efficient, it has its flip side. Once the students identify the way the program works, they will tend to write essays with long sentences, weird and obscure words, rather than learn how to write clear and logical sentences, to earn better grades.

At times the results of Big Data analysis, even if they are not tweaked, turn out to be less robust than they actually are. For example, Google Flu Trends, was considered to predict accurately and quickly flu-related search queries than the Center for Disease Control and Prevention. However, later, after a few blunders, it is found that it has made more inaccurate predictions. The reason why Google Flu Trends faltered is because of the fact that it takes into account all the key words which people enter in the Google search with regards to the flu. There maybe those people who are just wanting to read about the flu outbreak in a particular area and those who are actually suffering from the flu. Both sets of keywords were taken into account by Google Flu Trends, due to which inaccurate predictions were made and which was the main reason for its downfall. 

Big Data is useful and may also be the only resource for anyone analyzing large amounts of data; it has its flaws due to the incorrect data interpretation and inaccuracy of data drill down.