Foca nitu v4

Marcel Foca and Ionel Nițu

Bucharest, October 18th 2013

A new challenge in

data analysis:

the Big Data phenomenon


Abstract

Information life cycle

If the structured data (like databases with aggregation, searching and processing rules) are relatively easy to integrate, data analysis needs to include more and more unstructured data.

This is a change in the paradigm of the world of information technology and intelligence, one we must fully understand in order to be able to adapt to it.

We must be smart, prospective and pro-active, we must anticipate change instead of just reacting to stimuli.

Collect Manage

Analyze

Present

Instead of introduction – three short examples

1. Predicting the winner of the Eurovision singing competition (May 2013)

2. Accurately anticipate the flu propagation model for the H1N1 virus

3. Predicting that Obama will be re-elected

These three examples are using different logics, but they all show that the analysis can be digitalized at least up to a certain point.



The volume of internet data is growing exponentially

Facts- in 2011 - 2.1 billion users, and it will rapidly reach the benchmark of 50% of the world’s population- around 7 million new sites/month (2-3 new websites per second)- around 116 billion messages are sent and received daily (around 20 messages per person every day)- the volume of data generated on a daily basis doubles every 40 months- in Romania, at the beginning of 2013, the number of Internet users was estimated to 9.6 million.

Big Data means more than just the Internet, it also means billion of data accumulated in their own data bases (let’s only imagine the volume of information accumulated by American agencies such as NASA or the NSA, or the historical data archived with the NGA, mainly as images or satellite videos).


Diversity creates new challenges

Smartphones, sensors (whether terrestrial, aquatic or on satellites), video cameras, smart meters and other connected devices generate huge volumes of data added to the already gigantic stock of data from traditional sources.

UAVs and drones replace airplanes, where self-driven cars and smart highways are about to become reality.

Many aspects of our daily life can be digitized and transformed into data, and soon everything may become digitized. From food to political preferences, from professional history to hobbies and vices, attitude, character, skills, strong or weak points – everything could become part of one giant database.

(Big Data: A revolution that will transform the way we live, Viktor Mayer-Schönberger and Kenneth Cukierne)


How do we cope with an information tsunami?

- a first challenge is generated by the fact that by now it is no longer an issue of generating the data/information we need, but how to eliminate the information we don’t need- many data on the Internet cannot be dated or certified, as they are anonymous and/or false- we are witnessing a change in the relevance or impact of the Big Data phenomenon.

Companies like Facebook and Twitter are becoming non-state actors.

These globalization realities are overwritten by the need to make decisions in real time, which involves ever-growing and ever-more refined analysis capabilities.Data analysis will grow in importance and it will require evolved software (allowing for relevant searches in the Big Data and making correlations between information).Marcel Foca and Ionel Nițu

Knowledge is power, and knowledge is provided by analysis and analysts

- the FBI approached the Big Data in 1999- in Boston, in less than 24 hours after the explosions, the FBI processed around 10 terabytes of information.Since we are discussing Big Data and changes, we can add that the need for real time decisions often implies real time analysis, with interactive IT software that can be updated in real time. We are talking about concepts such as on-line dispatch centers and command & control decision systems (C2), where the role of GIS information and geospatial analysis makes the difference.In the GIS world it was always about Big Data, given the enormous data volumes and diversities (also the diverse sources of information) and the need to make quick decisions.The greatest challenge for GIS is standardization for interoperability. Standardization of geospatial information facilitates real time data processing, cooperation and exchange.


Knowledge is power, and knowledge is provided by analysis and analysts


Instead of conclusions

The Big Data is part of our day-to-day life. It is associated with clouding, and our lives (professional, personal or commercial) are slowly but surely switching over to the Internet.

Confidentiality issues (that is, intimacy risks) meet security issues, in an age where the greatest challenge – the cyber threats – target the very essence of Big Data. The latest trends show us an increase in the likelihood and impact these threats have on security and business environments.

Information analysis is more and more important, as it helps decision-making in real time, but it need to be automated to be effective, because an analogue (manual) analysis on large data volumes is completely unfeasible.

One of the greatest challenges will be the specialization of human resources.



Romania does not lack people talented in IT and mathematics, as proven by the many awards gained at international contests. The problem is we seem to be unable to use this important resource for increasing our country’s competitiveness worldwide, and these young people are not to blame for this.

We feel more than ever the need to support the IT field, which seems to have turned into the new grounds of economic competition all over the world. If Romania lost the big prizes in other areas, then maybe in the IT area and in particular in the Big Data Analysis area (which requires new thinking patterns, creativity and innovation) we stand a chance to make it up to the rest of the EU.



In the end worth to mention that big governmental IT projects seem to be lost somewhere (abandon? stand-by?): e-Romania, the law of national registries, electronic ID cards, governmental intranet, internet for all schools.It might not be too late to have a Big Data Analysis division at the central government level with a double role:- to process and to analyze Big Data information (as a support for the strategic development policies)- to monitor the implementation of the strategic IT decisions (to make Romania passing over the “IT consumer” phase).


Technology

Foca nitu v4