BIG DATA CHALLENGES - FCW.com

BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING

SIZING UPBIG DATA: 4 WAYS TO SUCCEED

BIG DATA’S SECURITY IMPERATIVE

BIG DATA HAS BIG POTENTIAL IN THE CLOUD

BIG DATA GOES MOBILE

VISUALIZATION DRIVES HOME BIG DATA’S VALUE

P2 P4 P5 P8 P9

ONLINE REPORTSPONSORED BY:

CDW-G and EMC

Special Report

FCW SPONSORED REPORT 2

Big data has been around long enough that there have been very notable successes – and resounding failures – related to its use.

Today 81 percent of IT executives surveyed say that big data is a top-� ve IT priority for 2013 and only 6 percent of IT shops do not have big data on their top 10 priorities list. But 55 percent of respondents also say that they have already had big data failures – projects that were not completed or fell short of their objectives.

There are myriad reasons why big data projects fail, according to a recent industry report. The top two are a lack of expertise to connect the dots and a lack of business context around data. But nearly every organization faces similar challenges, including � nding the right tools, dealing with time constraints, understanding the platforms and training staff.

The good news is that there are things that IT executives can do ahead of time to improve their success rate and end user satisfaction related to big data. Here the top four strategies that can help organizations get the most out of their big data efforts.

1) Get the scope rightAccording to the report, 58 percent of respondents say inaccurate scope is responsible for their failed big data projects. Alex Rossino, principle research analyst at Deltek, says that the bigger and more unlimited the mission of an organization is, the more complex its data requirements will be – and consequently the more work it will take to get the scope right.

“For example, if you think about traf� c enforcement in Suffolk County, N.Y. a big data project might require making sure that data collected on red light cameras, ticket issues and any related data is what is needed, and the rest would theoretically be discarded,” he says. “However, agencies run into trouble when they start with

one outcome in mind and it spirals past what they have already decided is mandatory to collect and analyze.”

Rossino suggests talking to everyone who might be affected by the analysis, from the secretary or commissioner of an agency down to the heads of of� ces and programs. “It needs to be discussed and then kicked over to the CIO to determine the resources that are needed to make it happen and put in writing so that no one is expecting more out of a project than has previously been discussed.”

That’s not to say a project’s scope might not change at a future date. But by sticking to the agreed upon scope – at least in the beginning – agencies and organizations are more likely to � nd success.

2) Get the business users involvedIn the case of big data, success hinges on producing information that is of value. So it only makes sense to involve the people who will be using that information.

“IT must recognize that big data means something different to every business and IT user,” says Evan Quinn, a senior principle analyst with research � rm Enterprise Strategy Group (ESG). “The � rst question every agency needs to ask itself is, ‘What am I trying to get out of this. What is the value?’”

The value should be re� ected in the queries that people make, said Mukul Krishna, director for digital media at Frost and Sullivan, a market research � rm. People might run ad hoc queries, but on the whole the payoff comes from asking the right questions.

It’s also important to have someone who can sift through the results and understand what it means. Some agencies might � nd themselves adding a chief analytics of� cer that has both business and IT knowledge so they can understand how to turn raw data into speci� c deliverables.

“Data is only worth something if IT can work with

SIZING UP BIG DATA: 4 WAYS TO SUCCEED

Special Report

www.fcw.com/2013BigDataChallenges>>>

BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


someone who understands the agency’s mission and can explain why it even needs a big data project to begin with,” Krishna said.

3) Hire the right talentIn an August 2012 1105 Government Information Group survey, more than half of nearly 200 government agencies reported that they are having dif� culty � nding and keeping knowledge workers and data scientists for their big data efforts. This challenge will only get worse as private and public organizations expand their big data efforts.

The dif� culty, says David Loshin, president of Knowledge Integrity Inc., a consultancy that focuses on business data management advice and guidance, is that big data is a departure for most IT professionals. “There is a major learning curve to understand the opportunities that big data affords,” he said.

That’s not to say every agency will need to hire new data scientists or mathematicians. Loshin suggests identifying the trainable staff within your current organization by looking for existing skills such as a love for statistics and training in computer science. A security background is also important since some of the data that your teams will be using may be classi� ed or contain personal identi� cation.

4) Size the infrastructure rightThose agencies and organizations that are doing big data analysis on-site will need to make sure that they have the necessary storage and compute power, says Loshin.

“Tiered storage may work well for some since you have the capacity to � ow data between disk and share memory,” Loshin said. “The most important thing is to make sure that, if you’re pulling data from multiple sources, performance does not lag.” Evaluating network

bandwidth should also be on your to-do list since low latency streaming will be key to end user satisfaction.

Many organizations may end up underestimating the amount of storage they will need, said Mike Gualtieri, an analyst with Forrester Research. “You may know how much data you have, but you might not realize that you need to duplicate everything in an analytical sandbox for those who want to do advanced analysis,” he said.

Gualtieri suggests that clustered systems can help. Still, many organizations may � nd that cloud-based services are the easiest way to provide the scaling and elasticity needed when dealing with big data.

“For very large data sets it might be more economic, especially if you’re launching something completely new,” he says. “You won’t have to invest millions of dollars if you’re keeping everything in the cloud.”

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


When big data began catching on, the tool of choice for many organizations was Hadoop, an open source programming framework that supports the

processing of large data sets in a distributed computing environment.

Some IT experts were concerned about the security risks that came with Hadoop. Still, from the beginning companies such as Facebook and eBay turned to the technology to do what nothing else could do: Collect, aggregate, analyze, and share structured and unstructured data from a variety of disparate sources.

In fact, Hadoop’s bene� t was its biggest Achilles’ heel: When bringing together data the security clearance for one set might be completely different than another’s. With big data being made available to multiple stakeholders and, in some cases, constituents, there was no way to protect its sources.

Today, while Hadoop has more security features – mostly software add-ons from other developers and service providers – security remains a serious concern for any IT executive considering a big data project. They must consider the security both of data within a cluster of servers and the cluster itself.

IT is charged with setting up encryption for the data and authentication or identity management for the clusters. In addition, especially for those organizations allowing mobile or web-based access, they must ensure the security of the applications and of the data that is produced. In particular, IT must make sure that crucial and classi� ed data doesn’t end up being downloaded on to a mobile device, which can be lost or stolen at any time.

It’s everywhereAnother problem, says Forrester analyst Mike Gualtieri, is that IT might not know which data is important for a speci� c query. “So when the line of business or end users say, ‘Give me all you’ve got,’

IT may do so without making sure the appropriate security standards are in place.”

Gualtieri recounts a client that had attempted to “anonymize” medical data that was being used in predictive analytics. However, although identifying data had been scrubbed from the sets, users were still able to infer and inadvertently exposed some information that never should have been allowed to come into the public view.

“The challenge becomes you want to give access to as much data as possible, but you still need to protect privacy,” he says.

Mobile data is even more problematic because it is not only real-time, but also attached to GPS positioning. “You’ve got to do a lot more governance when you’re dealing with so much personally identi� able information, says Gualtieri.

Cloud installations are also giving IT pause, says Evan Quinn, a senior principal analyst with ESG. When data is on premise, you have more control over the data. Once it is put into the public cloud, IT must worry about whether or not the data is on a shared resource, if the servers are inside or outside of this country, and whether or not the cloud provider’s security and governance practices could withstand an audit.

“You need to go to the cloud provider and ask point blank to see a checklist of its practices,” he said. He also suggests ensuring that employees and, in some cases, constituents are accessing data only via secure connections.

And � nally, IT executives need to think carefully about the accessibility of big data query results. Sometimes, employees are so excited about what they’re seeing that they extend access to others in their department or those who are working with them on projects. “You don’t want to get any surprises and � nd out that you’ve just distributed data to 1,000 people who were never cleared to see it in the � rst place,” said Quinn.

BIG DATA’S SECURITY IMPERATIVE

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


This past January, IT heavyweights from the National Institute of Standards and Technology (NIST) as well as other government agencies, industry, and

academia, got together to discuss the critical intersection of big data and the cloud. Although government agencies have been slower to adopt new technologies in the past, the event underscored the fact that the public sector is leading – and in some cases creating – big data innovation and adoption.

Cloud is a multiplier when used with other technology, and is capable of big things when it comes to big data, according to U.S. CIO Steven VanRoekel. Combining big data with cloud delivery and compute power might help create new industries and provide bene� ts to every citizen.

“The government is sitting on a treasure drove of data,” he said during a speech at the two-day NIST Joint Cloud and Big Data Workshop event. “We’re opening data, and looking at what we can do. We can greatly impact the lives of every American by just unlocking simple prices of data.”

He also pointed to the formation of companies built and founded completely on government data.

Nowhere to go but upResearch and consulting � rm Deltek Inc. says cloud environments are “optimal” for using analytics since cloud providers are investing in the best analytical and visualization tools available today. In addition, big data projects require processing speed and the most up-to-date technologies, two other cloud provider specialties.

But most important, said Richard Blake, senior technical advisor in the Enterprise GWAC Division of the Integrated Technology Service at the General Services Administration’s Federal Acquisition Service, is that big data in the cloud enables something that is crucial for innovation, analysis and return-on-investment: The

ability to share resources between agencies. Evan Quinn, a senior principle analyst with research

� rm ESG agrees. “Big data is an incremental learning process and when you can share expertise and resources you can score more small wins and progress more quickly than you may have on your own,” he says.

As highlighted in Deltek’s Federal Big Data Outlook 2012-2017, data is available from a wide variety of sources, including agency data such as data logs, space telescopes, reconnaissance, citizen information, and mission critical apps. However, data is also being sourced from the outside as well — in the form of email, text messages, pictures, embedded sensors, social media, GPS data, purchase data, traf� c cams and market research. With the cloud, it becomes easier to store, analyze, and access this information.

In addition, the same data can be analyzed and used in different applications and analytic projects, since one agency no longer “owns” that data. With these tenets in mind going forward, every piece of data will be examined as a potential resource and used as the basis of future applications, explained VanRoekel during the conference.

One promising approach, he said, is to ensure that data is machine-readable – that is, that it can move from system to system without requiring human intervention or translation.

“Government agencies are ordered to look at the data they produce, catalog data, start to publish data, and think about machine-readable as the new default inside government,” VanRoekel explained. “Any time we’re building a new system, or amending a system, we focus on machine readability both on the collection, as well as getting agencies to develop [application programming interfaces] around their data.”

However, as promising as it is, agencies do not want

BIG DATA HAS BIG POTENTIAL IN THE CLOUD

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


to rush into putting big data in the cloud. Although it might mean that IT no longer has to worry about buying, maintaining, and managing those databases and associated storage, big data is a very specialized workload, says ESG’s Quinn. Agencies must ensure that their cloud providers can handle big data and that they understand the public sector’s security and compliance requirements, he said.

The other issue is expertise. Most if not all agencies are currently doing some form of business intelligence (BI) on their data, but what works for most organizations with BI is not going to work for big data. Expectations on a whole must be realistic and in line with what is currently technologically possible, said Quinn.

“If any agency thinks it’s going to go in and put $1 million and six months of work into a big data cloud project and change how the agency works from the inside out they are misinformed,” he said. “For most organizations this is a completely new discipline so they need to think in terms of small wins.” This past January, IT heavyweights from the National Institute of Standards and Technology (NIST) as well as other government agencies, industry, and academia, got together to discuss the critical intersection of big data and the cloud. Although government agencies have been slower to adopt new technologies in the past, the event underscored the fact that the public sector is leading – and in some cases creating – big data innovation and adoption.

Cloud is a multiplier when used with other technology, and is capable of big things when it comes to big data, according to U.S. CIO Steven VanRoekel. Combining big data with cloud delivery and compute power might help create new industries and provide bene� ts to every citizen.

“The government is sitting on a treasure drove of data,” he said during a speech at the two-day NIST Joint Cloud and Big Data Workshop event. “We’re opening

data, and looking at what we can do. We can greatly impact the lives of every American by just unlocking simple prices of data.”

He also pointed to the formation of companies built and founded completely on government data.

Nowhere to go but upResearch and consulting � rm Deltek Inc. says cloud environments are “optimal” for using analytics since cloud providers are investing in the best analytical and visualization tools available today. In addition, big data projects require processing speed and the most up-to-date technologies, two other cloud provider specialties.

But most important, said Richard Blake, senior technical advisor in the Enterprise GWAC Division of the Integrated Technology Service at the General Services Administration’s Federal Acquisition Service, is that big data in the cloud enables something that is crucial for innovation, analysis and return-on-investment: The ability to share resources between agencies.

Evan Quinn, a senior principle analyst with research � rm ESG agrees. “Big data is an incremental learning process and when you can share expertise and resources you can score more small wins and progress more quickly than you may have on your own,” he says.

As highlighted in Deltek’s Federal Big Data Outlook 2012-2017, data is available from a wide variety of sources, including agency data such as data logs, space telescopes, reconnaissance, citizen information, and mission critical apps. However, data is also being sourced from the outside as well — in the form of email, text messages, pictures, embedded sensors, social media, GPS data, purchase data, traf� c cams and market research. With the cloud, it becomes easier to store, analyze, and access this information.

In addition, the same data can be analyzed and used

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


in different applications and analytic projects, since one agency no longer “owns” that data. With these tenets in mind going forward, every piece of data will be examined as a potential resource and used as the basis of future applications, explained VanRoekel during the conference.

One promising approach, he said, is to ensure that data is machine-readable – that is, that it can move from system to system without requiring human intervention or translation.

“Government agencies are ordered to look at the data they produce, catalog data, start to publish data, and think about machine-readable as the new default inside government,” VanRoekel explained. “Any time we’re building a new system, or amending a system, we focus on machine readability both on the collection, as well as getting agencies to develop [application programming interfaces] around their data.”

However, as promising as it is, agencies do not want to rush into putting big data in the cloud. Although it might mean that IT no longer has to worry about buying, maintaining, and managing those databases and associated storage, big data is a very specialized workload, says ESG’s Quinn. Agencies must ensure that their cloud providers can handle big data and that they understand the public sector’s security and compliance requirements, he said.

The other issue is expertise. Most if not all agencies are currently doing some form of business intelligence (BI) on their data, but what works for most organizations with BI is not going to work for big data. Expectations on a whole must be realistic and in line with what is currently technologically possible, said Quinn.

“If any agency thinks it’s going to go in and put $1 million and six months of work into a big data cloud project and change how the agency works from the

inside out they are misinformed,” he said. “For most organizations this is a completely new discipline so they need to think in terms of small wins.”

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


For CIOs in the public and private sectors looking for ways both to innovate and save money, the combination of big data and mobile is increasingly

appealing solution. For example, Westminster, U.K., recently posted

online all of its geospatial data related to a state-run, London-based bicycle rental — where the city’s bike racks were, how many bikes were available for rental in real time and other mapping and bike path data. Within days of the release, one of the agency’s constituents had taken that information and built a mobile app to help people plan future bicycle journeys. Today, it’s easier for citizens to get around and the app is also helping reduce pollution, boost public health, and make more room on public transportation.

This is just one example of why CIOs are so interested in big data and mobility, according to venture capital � rm Sierra Ventures, which recently released a study titled “Seizing Opportunity: The Transition from Legacy to Innovation in Enterprise IT.”

According to that report, which was conducted in conjunction with its CIO Advisory Board, when it comes to innovation many CIOs are making big data and mobility an even higher priority than cloud computing and social media.

Meanwhile, a TechAmerica Foundation study in January found that 75 percent of federal IT of� cials surveyed think that real-time data – something that mobile devices can enable – is already “helping government improve the quality of citizens’ lives.”

Another study from IDC Government Insights forecasts that this year, 35 percent of new Federal and state applications will be mobile.

Where the toys areThe reason that big data is intersecting with mobile is simple: Reach.

In order for big data to be successful there needs to be

a common way to disseminate information to a wider audience. Mobile in particular has also been a huge data sources as billions of data points have been collected by carriers, and those data points will grow exponentially as cell phone usage continues to grow.

Today, 78 percent of people between the ages of 12 and 17 own cell phones and 47 percent of those own smartphones, according to a Pew Research report released in March.

Likewise, the number of devices per U.S. household with online access is up to 5.7 – almost a half a percentage point growth in three months, according to the NPD Group’s “Connected Home Report.” During the same three-month period, the installed base of tablets grew by almost 18 million, with almost 60 percent of U.S. households now owning a tablet device. Combined, connected devices now number about a half a billion and growing.

The challenge for government IT executives is how to extend access to big data initiatives without compromising the security or quality of the data or other agency resources.

In fact, the TechAmerica Foundation’s report found that “privacy and policy concerns” trumped “demonstrating the level of return on investment” by a wide margin at the state level – 40 percent versus 22 percent, respectively. At the federal level, privacy and policy was also the top concern, cited by 47 percent of federal IT of� cials as opposed to 42 percent of of� cials who cited ROI.

The concept of combining big data and mobility is still evolving. But despite its limitations, the combination, when done successfully, can provide a big payoff, as Westminster found out. That might be why, at the end of last year, David Willetts, minister for universities and science in the U.K., announced big funding for big data, with more than £189 million going to fund research related to the technologies. After all, when you can harness data and disseminate it in a way that democratizes it, everyone wins.

BIG DATA GOES MOBILE

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING


Analytics and business intelligence is the fastest growing software market today, according to research � rm Gartner. The global market for BI is

expected to reach $13.8 billion this year and shoot to more than $17 billion in 2016. Big data and “data-as-a-service” are the growth factors for the segment’s upward trajectory.

It might stand to reason then that all it takes to start a big data project is some basic analytics software and a few good on-staff experts. However, experts say basic analytics is only part of the equation.

Another essential part of that equation is visualization software, which translates raw data into a graphical presentation, which humans can understand more intuitively.

“Big data needs an integrated approach — one that combines analytics and visualization software,” explains Alex Rossino, a principle research analyst at Deltek. “You can’t just have analytics and expect it to solve all your problems.”

Picture thisVisualization is important because data, in its raw form, can be dif� cult to analyze, even for the most savvy IT and knowledge workers. In addition, traditional BI tools are designed for traditional data that’s stored in relational databases. Visualization provides a framework for working with the wider range of data types that might be combined in big data applications.

For example, an application could take a county map and lay speci� c data points over it: incidents of driving-while-intoxicated fatalities, socioeconomic data, the locations and coverage of local law enforcement. Presented visually, the results might help government of� cials determine to beef up policing or where to install extra safety measures, such as guard rails or impact-softening barriers.

Agencies also might extend visualization applications to constituents, says David Loshin, president of Knowledge Integrity Inc, a consultancy that focuses on business data management advice and guidance.

For example, an application might be created to show how the U.S. House of Representatives is voting. “People complain that lobbyists have that information, and they do because they’re on the � oor, but you could take that voting data and make a graphical representation so that constituents were as much in the know as lobbyists.”

Visualization also helps IT and its users uncover what might not be obvious to the untrained eye. The Department of Homeland Security’s Cyber Security Research and Development Center uses visualization to identify patterns in network � ow data and metadata that could signal denial of service attacks and other cyber attacks and criminal behavior.

Once people can see information in graphical format, the message of data can really hit home, says Ed Parsons, a fellow of the Royal Geographical Society.

Parsons worked on a project about with the U.K.’s Hadley Centre, a government agency charged with tracking climate change. At the time, there was no way to take the data that had been collected and visualize it in an impactful way, he explained at the Aspen Ideas Festival session, “Visualizing our Future through the Lens of Big Data.” Few people watched the YouTube clips of the climate change announcement or went to the website detailing the news. However, only a few years later he was able to use big data and visualization to demonstrate at the street level the impact and water level after the 2011 Japanese earthquake and tsunami.

“Often our heart is more powerful than our brain when we really want to drive home a message,” Parsons said at the conference. People, whether c-level executives, constituents or voters, are more moved by what they can see than what they can hear. •

VISUALIZATION DRIVES HOME BIG DATA’S VALUE

Special Report


BIG DATA CHALLENGES

BEST PRACTICESFOR

MEETING

+

Conquer the chaos with storage management solutions from CDWG. Utilizing our partnership with EMC, our solution architects can help design and implement a custom storage solution that meets your needs. And once all of the excess has gone, your data center can achieve perfect harmony.

Get organized at CDWG.com/emc

SOLVED.

EVErybODy WantS tO StOrE EVErythinG EVEryWhErE fOrEVEr.

©2013 CDW Government LLC. CDW®, CDW•G® and PEOPLE WHO GET IT™ are trademarks of CDW LLC.

124542_CDWFed_SnapBigData_043013.indd 1 3/20/13 2:30 PM

Documents

BIG DATA CHALLENGES - FCW.com