Business white paper

Harvest your Big DataTransforming CSPsrsquo Big Data to knowledge and revenue

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Table of contents

Executive summary

Introduction

Understanding the four Vs that define Big Data

Strategic priority

A complete Big Data platform

From data to knowledgemdashbetter business decisions

Use cases

Whatrsquos your next move Consider these seven questions

The HAVEn ecosystem

HP Software Services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Executive summaryThe game has changed Leading organizations are driving fundamental shifts in business models through insights derived from Big Data Communications service providers (CSPs) have a choice Be a leader or be forced to follow To join the leaders yoursquoll need the ability to capture manage and analyze the deluge of Big Data everything from business transactions to sensor data to tweets

Big Data is an opportunity for CSPs to create the intelligence for operating network more efficiently to analyze the success of the services that CSPs are offering and to create a better personal experience for their customers Chief marketing officers vice-president network operations and line of business managers are equally eager to exploit the large amount of information to achieve better business decisions They expect their chief marketing officer to provide end-to-end analytic solutions based on the data available in their IT and network infrastructure

This white paper analyzes the complete value chain that can transform CSPsrsquo data to knowledge and revenue It covers the sources of information the data collection tools the analytic platforms providing quick data access and finally the business intelligence use cases with the presentation and visualization of the results and predictions Introducing the HP HAVEn platform we bring together everything you need to profit from Big Data Encompassing hardware software services and business transformation consulting HAVEn helps organizations make the critical business transformation to connected intelligence and analytics-driven decision making

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

IntroductionCSPs know an inordinate amount of personal information about their customers such as who their contacts are and their phone numbers addresses (home work email) the Internet usage applications downloaded travel history even how long it takes them to commute to work each morning A customerrsquos smartphone usage becomes a snapshot of their daily lives data that any social media company would love to possess

Over the years your large communication network and their associated switches billing systems and service departments may have generated hundreds of millions of individual call details records (CDRs) daily Terabytes of dynamic customer data will continue to grow exponentially as carriers add new services and as IP-based traffic increases

Why nowWhat we have seen in the last five years is an evolution of the technology that has turned the data explosion into an opportunity to transform and monetize The analytics equation described in figure 1 has made possible solutions and approaches that were impossibly expensive and complex just a few years ago

bull The cost of processing data and doing analytics on it in real time has gone down making it increasingly possible for live data to trigger system actions rather than just generate static reports for human consumption

Figure 1 The analytics equation

Cheap processing

Commoditizedaccess

Cheapstorage

Urge toshare

Belief thatdata explainsbehavior

Opportunityto transformto monetize

+ + + + =

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull The commoditized access of the information from multiple systems network elements sensor data that could outstrip all others and the Internet data that have equal value to internal data has facilitated the explosion of the data you can analyze

bull The cost of data storage has fallen dramatically This coupled with the application of new data processing techniques is enabling far larger datasets to be captured stored and analyzed

bull New sources of user data and urge to share from smartphones and social networks have come on stream potentially enabling a multidimensional view of the customer

bull Data can explain behavior interestmdashfor this belief many operators apply statistical techniques complex relationships across data and graphs social network analysis and their own business problems

From a strategic point of view you want to be able to reconnect with customers with a full understanding of their needs detect business trends and opportunities timely detect fraud and comply with regulatory requirements

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Table of contents

Executive summary

Introduction

Understanding the four Vs that define Big Data

Strategic priority

A complete Big Data platform

From data to knowledgemdashbetter business decisions

Use cases

Whatrsquos your next move Consider these seven questions

The HAVEn ecosystem

HP Software Services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Executive summaryThe game has changed Leading organizations are driving fundamental shifts in business models through insights derived from Big Data Communications service providers (CSPs) have a choice Be a leader or be forced to follow To join the leaders yoursquoll need the ability to capture manage and analyze the deluge of Big Data everything from business transactions to sensor data to tweets

Big Data is an opportunity for CSPs to create the intelligence for operating network more efficiently to analyze the success of the services that CSPs are offering and to create a better personal experience for their customers Chief marketing officers vice-president network operations and line of business managers are equally eager to exploit the large amount of information to achieve better business decisions They expect their chief marketing officer to provide end-to-end analytic solutions based on the data available in their IT and network infrastructure

This white paper analyzes the complete value chain that can transform CSPsrsquo data to knowledge and revenue It covers the sources of information the data collection tools the analytic platforms providing quick data access and finally the business intelligence use cases with the presentation and visualization of the results and predictions Introducing the HP HAVEn platform we bring together everything you need to profit from Big Data Encompassing hardware software services and business transformation consulting HAVEn helps organizations make the critical business transformation to connected intelligence and analytics-driven decision making

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

IntroductionCSPs know an inordinate amount of personal information about their customers such as who their contacts are and their phone numbers addresses (home work email) the Internet usage applications downloaded travel history even how long it takes them to commute to work each morning A customerrsquos smartphone usage becomes a snapshot of their daily lives data that any social media company would love to possess

Over the years your large communication network and their associated switches billing systems and service departments may have generated hundreds of millions of individual call details records (CDRs) daily Terabytes of dynamic customer data will continue to grow exponentially as carriers add new services and as IP-based traffic increases

Why nowWhat we have seen in the last five years is an evolution of the technology that has turned the data explosion into an opportunity to transform and monetize The analytics equation described in figure 1 has made possible solutions and approaches that were impossibly expensive and complex just a few years ago

bull The cost of processing data and doing analytics on it in real time has gone down making it increasingly possible for live data to trigger system actions rather than just generate static reports for human consumption

Figure 1 The analytics equation

Cheap processing

Commoditizedaccess

Cheapstorage

Urge toshare

Belief thatdata explainsbehavior

Opportunityto transformto monetize

+ + + + =

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull The commoditized access of the information from multiple systems network elements sensor data that could outstrip all others and the Internet data that have equal value to internal data has facilitated the explosion of the data you can analyze

bull The cost of data storage has fallen dramatically This coupled with the application of new data processing techniques is enabling far larger datasets to be captured stored and analyzed

bull New sources of user data and urge to share from smartphones and social networks have come on stream potentially enabling a multidimensional view of the customer

bull Data can explain behavior interestmdashfor this belief many operators apply statistical techniques complex relationships across data and graphs social network analysis and their own business problems

From a strategic point of view you want to be able to reconnect with customers with a full understanding of their needs detect business trends and opportunities timely detect fraud and comply with regulatory requirements

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Executive summaryThe game has changed Leading organizations are driving fundamental shifts in business models through insights derived from Big Data Communications service providers (CSPs) have a choice Be a leader or be forced to follow To join the leaders yoursquoll need the ability to capture manage and analyze the deluge of Big Data everything from business transactions to sensor data to tweets

Big Data is an opportunity for CSPs to create the intelligence for operating network more efficiently to analyze the success of the services that CSPs are offering and to create a better personal experience for their customers Chief marketing officers vice-president network operations and line of business managers are equally eager to exploit the large amount of information to achieve better business decisions They expect their chief marketing officer to provide end-to-end analytic solutions based on the data available in their IT and network infrastructure

This white paper analyzes the complete value chain that can transform CSPsrsquo data to knowledge and revenue It covers the sources of information the data collection tools the analytic platforms providing quick data access and finally the business intelligence use cases with the presentation and visualization of the results and predictions Introducing the HP HAVEn platform we bring together everything you need to profit from Big Data Encompassing hardware software services and business transformation consulting HAVEn helps organizations make the critical business transformation to connected intelligence and analytics-driven decision making

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

IntroductionCSPs know an inordinate amount of personal information about their customers such as who their contacts are and their phone numbers addresses (home work email) the Internet usage applications downloaded travel history even how long it takes them to commute to work each morning A customerrsquos smartphone usage becomes a snapshot of their daily lives data that any social media company would love to possess

Over the years your large communication network and their associated switches billing systems and service departments may have generated hundreds of millions of individual call details records (CDRs) daily Terabytes of dynamic customer data will continue to grow exponentially as carriers add new services and as IP-based traffic increases

Why nowWhat we have seen in the last five years is an evolution of the technology that has turned the data explosion into an opportunity to transform and monetize The analytics equation described in figure 1 has made possible solutions and approaches that were impossibly expensive and complex just a few years ago

bull The cost of processing data and doing analytics on it in real time has gone down making it increasingly possible for live data to trigger system actions rather than just generate static reports for human consumption

Figure 1 The analytics equation

Cheap processing

Commoditizedaccess

Cheapstorage

Urge toshare

Belief thatdata explainsbehavior

Opportunityto transformto monetize

+ + + + =

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull The commoditized access of the information from multiple systems network elements sensor data that could outstrip all others and the Internet data that have equal value to internal data has facilitated the explosion of the data you can analyze

bull The cost of data storage has fallen dramatically This coupled with the application of new data processing techniques is enabling far larger datasets to be captured stored and analyzed

bull New sources of user data and urge to share from smartphones and social networks have come on stream potentially enabling a multidimensional view of the customer

bull Data can explain behavior interestmdashfor this belief many operators apply statistical techniques complex relationships across data and graphs social network analysis and their own business problems

From a strategic point of view you want to be able to reconnect with customers with a full understanding of their needs detect business trends and opportunities timely detect fraud and comply with regulatory requirements

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

IntroductionCSPs know an inordinate amount of personal information about their customers such as who their contacts are and their phone numbers addresses (home work email) the Internet usage applications downloaded travel history even how long it takes them to commute to work each morning A customerrsquos smartphone usage becomes a snapshot of their daily lives data that any social media company would love to possess

Over the years your large communication network and their associated switches billing systems and service departments may have generated hundreds of millions of individual call details records (CDRs) daily Terabytes of dynamic customer data will continue to grow exponentially as carriers add new services and as IP-based traffic increases

Why nowWhat we have seen in the last five years is an evolution of the technology that has turned the data explosion into an opportunity to transform and monetize The analytics equation described in figure 1 has made possible solutions and approaches that were impossibly expensive and complex just a few years ago

bull The cost of processing data and doing analytics on it in real time has gone down making it increasingly possible for live data to trigger system actions rather than just generate static reports for human consumption

Figure 1 The analytics equation

Cheap processing

Commoditizedaccess

Cheapstorage

Urge toshare

Belief thatdata explainsbehavior

Opportunityto transformto monetize

+ + + + =

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull The commoditized access of the information from multiple systems network elements sensor data that could outstrip all others and the Internet data that have equal value to internal data has facilitated the explosion of the data you can analyze

bull The cost of data storage has fallen dramatically This coupled with the application of new data processing techniques is enabling far larger datasets to be captured stored and analyzed

bull New sources of user data and urge to share from smartphones and social networks have come on stream potentially enabling a multidimensional view of the customer

bull Data can explain behavior interestmdashfor this belief many operators apply statistical techniques complex relationships across data and graphs social network analysis and their own business problems

From a strategic point of view you want to be able to reconnect with customers with a full understanding of their needs detect business trends and opportunities timely detect fraud and comply with regulatory requirements

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull The commoditized access of the information from multiple systems network elements sensor data that could outstrip all others and the Internet data that have equal value to internal data has facilitated the explosion of the data you can analyze

bull The cost of data storage has fallen dramatically This coupled with the application of new data processing techniques is enabling far larger datasets to be captured stored and analyzed

bull New sources of user data and urge to share from smartphones and social networks have come on stream potentially enabling a multidimensional view of the customer

bull Data can explain behavior interestmdashfor this belief many operators apply statistical techniques complex relationships across data and graphs social network analysis and their own business problems

From a strategic point of view you want to be able to reconnect with customers with a full understanding of their needs detect business trends and opportunities timely detect fraud and comply with regulatory requirements

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Understanding the four Vs that define Big DataWhile ldquobigrdquo is part of the data challenge it doesnrsquot tell the whole story Volume is just one part of the problem Three additional Vs complete the picture variety velocity and vulnerability To manage Big Data effectively you must be able to

bull Gather store and manage large amounts of datamdashup to millions of times more data than what you handle today (volume)

bull Collect and store all relevant data structured semi-structured and unstructured (variety)

bull Analyze and react to it as quickly as itrsquos created (velocity)

bull Keep it secure and compliant with regulatory requirements at all times (vulnerability)

Volume Every day we collectively create 41000 times the amount of information in every book ever written The digital universe doubles in size every two years1 Our challenge lies not only in collecting and storing massive amounts of diverse data but also managing and governing existing legacy data

Variety Around the world at work and at home we are connecting with each other through email social media websites and freely sharing information and sentiments Sensors smart devices Web feeds event logs all constantly generate data Your enterprise needs a way to not only capture all of this diverse information in a common format but also to interpret synchronize understand and use it to make better business decisions

Velocity Thanks to technology our world has become immediate and instantaneous Your customers expect answers from you about your products and services much faster than evermdashideally in real time The goal is to harness insight from information at the speed of business

Vulnerability How do you secure a vast and growing volume of critical information Protecting this datamdashand being able to search and analyze it to detect potential threatsmdashis more essential than ever As the platforms supporting this data move to hybrid IT environments managing their security and availability becomes a Big Data challenge in and of itself requiring continuous diagnostics and monitoring1 ldquoExtracting value from Chaosrdquo IDC June 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Strategic priorityAccording to a survey of over 140 senior Telco managers just 54 percent of operators say that Big Data was a current strategic priority in their organizations 24 percent said they did not know and 22 percent said it definitively wasnrsquot2 This means that a short majority of CSPs are ready to embrace Big Data as more executives understand the possibilities it provided

So why are Big Data and analytics tools so important for CSPs Analytics improves multiple aspects of CSPsrsquo operations such as

bull Enable new business model as CSPs are starting to realize that the information they have is an untapped asset Choose the potential offered by new revenues stream as the biggest opportunity (see figure 2)

bull Provides business optimization capability that helps to increase revenue through more-targeted marketing and to reduce expenses by identifying cost and revenue leakages

bull Improve the customer experience by launching new innovative services quickly and gaining deep customer insights to personalize offerings

bull Create differentiation from competition by embracing future market opportunities as connected devices machine-to-machine (M2M) devices and oriented sensors are proliferating at an incredible growth rate fueling the amount of data collected

bull Reduce churn with the ability to better segment subscribers to provide more-targeted marketing spend with the insight to predict churn cross-sell opportunities the quality of customer experience and the value of a customer

bull Decrease cost with the provision for product managers of a better understanding for which services are most profitable the impact of competitive offerings and the impact of ldquocannibalizationrdquo caused by a new service launch

bull Improve operational efficiencies allowing network operationsrsquo ability to predict capacity issues and the impact of a new service launch

2 ldquoEuropean Communications surveyrdquo European Communication magazine March 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Figure 2 What is the biggest opportunity that Big Data presents to operators

Improving the customer experience

Creating differentiation from competitors

Reducing churnimproving ARPUupselling

Reducing CAPEXOPEX

Reducing strategic operational risks

Creating new revenue stream

35302520151050

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

A complete Big Data platformThe HAVEn technology stack includes proven technologies from HP Software including HP Autonomy HP Vertica and HP ArcSight And HAVEn extends HP technologies with key industry initiatives such as Hadoop Together these solutions provide the capability to handle 100 percent of enterprise datamdashstructured unstructured and semi-structuredmdashand securely deliver actionable intelligence from that data in moments

bull Hadoop The HP open strategy supports all leading Hadoop distributions

bull HP Autonomy IDOL Seamless access to 100 percent of enterprise data whether human or machine generated

bull HP Vertica A massively scalable database platform custom-built for real-time analytics on petabyte-sized datasets Vertica supports standard SQL- and R-based analytics and offers support for all leading business intelligence (BI) and Extract Load Transform (ELT) vendors

bull HP ArcSight (enterprise security) Provides real-time collection and analysis of logs and security events from a wide range of devices and data sources leveraging Big Data to bridge both operational intelligence and security intelligence

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

bull ldquonrdquo applications HP is already bringing the strength of HAVEn to our own software portfolio so as CSPs you can take advantage of connected intelligence to

ndashUnderstand how your network is utilized so that you can identify where to invest the new capacity when to make traffic-boosting offers and how to serve customers most efficiently with Subscribers Network Usage

ndashDeliver actionable intelligence about service health with advanced analytics capabilities for consolidated IT datamdashincluding machine data logs events topologies and performance statistics and analyze all of your important information to diagnose problems and determine root cause with HP Operations Analytics

ndashUnderstand your net promoter score and increase loyalty with proactive customer experience management as you need to ensure that you donrsquot simply invest more and more in the network without proving your value to your subscribers You can know about your subscribers with Multiple channel customer analytics

ndashAddress the customer usage behavior and interests with targeted products and marketing offers that personalize the user experience according to actual customer needs with Mobile experience personalization and Ad experience personalization

ndashProtect your critical assets by creating total visibility into activity across the customerrsquos IT and IP network infrastructuremdashincluding external threats such as malware and hackers with Big Data for security and internal threats such as data breaches and fraud with Fraud prevention

ndashCreate new business models by connecting with over the top players and transform from a provider of network infrastructure to value-added content provider for third-party advertisers and verticals with Big Data as a service and Machine to machine

Additionally the ecosystem around HAVEn can grow very large over time but the technologies today are delivering solutions that give enterprises greater power over their Big Data intelligence HAVEn incorporates the HP strength as the leader in enterprise-class hardware and services Converged infrastructure and services offerings from HP and its partners will be part of this growing pan-HP ecosystem

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP HAVEnmdashthe Big Data platform

Social media ITOT ImagesAudioVideoTransactional

dataMobile Search engineEmail Texts

Documents

HadoopHDFS HP Autonomy IDOL HP Vertica Enterprise Security nApps

Catalog massive volumes of distributed data

Process and index all information

Analyze at extreme scale in real time

Collect and unify machine data

Powering HP Software + your apps

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

From data to knowledgemdashbetter business decisionsTo achieve better business decision CSPs need to consider the complete value chain that can transform their data to knowledge bringing all components together in one end-to-end solution It includes (see figure 3)

bull Data sources From network information billing systems subscriber profile devices or social network

bull Data collections Including different technology such as network probe that captures the data

bull Data management and structuring The heart of your business knowledge with analytic databases (ADBs) providing fast access to that data

bull Data access Enabling query sessions to be truly interactive

bull Business intelligence The analytics process applied in a number of CSP-specific use cases

bull Presentation and visualization Proposing predictions and results to staff in a usable format

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

2 Data collections3 Data management and structuring

4 Data access

5 Business intelligence

6 Presentation and visualization

Letrsquos analyze each of these components in more detail

Figure 3 Big Data value chain

Click on each icon to read more

1 D

ata

sour

ces

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data sourcesSuccess for a Big Data strategy lies in recognizing the different types of Big Data sources using the proper mining technologies to find the treasure within each type and then integrating and presenting those new insights appropriately according to your unique goals to enable your organization to make more effective steering decisions The different sources of information for CSPs include

Network usage Such as CDR or Internet Protocol Detail Record (IPDR) and information from business support systemsoperational support systems

Sensors The global market for wireless sensor devices used in end vertical applications totaled $532 million USD in 2010 and $790 million USD in 2011 This market is expected to increase at a 431 percent compound annual growth rate (CAGR) and reach an estimated $47 billion USD by 20163

Connected devices There are nine billion connected devices at present and by 2020 that number is going to explode to 24 billion devices according to new statistics released by GSMA4

Mobile devices The total number of mobile connected devices doubles from 6 billion today to 12 billion by 20205

Apps Information from the number of applications available and the marketing around the number of apps expected to rise The number of apps likely to be downloaded worldwide in 2016 is expected to reach to 44 billion raising the information and the marketing around them Subscriber profiles come from network systems such as home location registerhome subscriber server (HLRHSS) or provisioning or CRM

Services Where we are engaging with a service to buy something sell something and others ultimately creating an opportunity to analyze behavior target a pattern and so on

3 ldquoGlobal markets and technologies for wireless sensorsrdquo report code IAS042A BCC Research February 2012

4 5 ldquoInternet of things will have 24 billion devices by 2020rdquo GigaOM Pro October 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Log intelligence IT needs to augment their current monitoring strategy to now be able to collect data including machine data and logs from devices all over the environment Since they donrsquot necessarily know what bit of information might prove useful in todayrsquos complex world they need to collect everythingmdashlogs machine data performance metrics and others

Business discovery The ability to quickly analyze customer interactions in real time is critical to your businessrsquo success It requires the following information

bull Customer feedback allows you to understand the comments in survey verbatim and other direct feedback and sorts them by concepts

bull Clickstream allows you to understand visitor behavior beyond the simple click counts and other pre-determined success measures

bull Social network profiles taps user profiles from Facebook LinkedIn Yahoo Googletrade and specific-interest social or travel sites to cull individualsrsquo profiles and demographic information and extend that to capture their hopefully like-minded networks

bull Speech analytics organizes and understands customer conversations automatically so that you can take advantage of the rich insights in phone conversations

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data collectionsNetwork probe is a technology that decodes protocols extracts information embedded in the traffic or transmitting over the traffic Then it delivers this information in the form of metadata and content feeds to an application developed by the user that leverages the information provided by the network probe

The probe makes an acquire specifying what information is required The network probe delivers this information in a tabular format just like in a database to a storage engine This technology can also deliver packets and packets contents The process of extracting and delivering the information from the network is done in real time and scale up to 10 gigabit per second Different protocols are supported such as network protocols application protocols such as webmail email database or any kind of network application For each protocol tens of metadata are delivered which make at least thousands of metadata in your application These protocols are regularly updated and new protocols are added to protocol plug-in library

Network probe intelligence is designed to be embedded into your application so you can rely on the real-time visibility provided by network probe to develop application processing the traffic information or storing this information for some reporting or traffic shaping

The HAVEn platform has 400 ready-to-use connectors to extract a wide variety of data and human information from over 70 repositories as well as 300 connectors from HP ArcSight connectors targeted at collecting and managing machine data from a wide variety of log-generating sources HP Autonomy also addresses the necessary tools to extract file content and metadata from over 1000 file types

In addition each of the components of HAVEn (HP Autonomy HP Vertica and HP ArcSight) also provides additional data connector frameworks and tools to help you build custom connectors The HAVEn platform supports popular frameworks such as Hadoop Flume and Chukwa and it is open to all ELT frameworks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Internet Usage Manager (IUM) is the industry-established real-time charging convergent mediation solution that supports a huge variety of billing and charging services HP IUM handles data for convergent mediation real-time charging as well as IP-Multimedia Subsystem (IMS)-based voice and data services across wireline wireless cable and broadband networks HP IUM has a vast array of collection techniques that support both real-time and batch event collection HP IUM provides retrieval mechanisms such as FTP SCP HTTP GTP FTAM Radius Diameter SIP CSG Cisco SCE (P-Cube) and local file collection techniques All of these components are configurable entities in the application and IUM customers get the entire spectrum with the product which is immediately ready to use into the HP Vertica platform

HP Smart Profile Server (Data Orchestrator) implements an ELT process It is connected to network elements (for example switches home location registers home subscriber servers deep packet inspection and others) and business systems (such as CRM) in the CSP ecosystem and harvests structured data such as network data usage data and profile data as well as unstructured data like customer complaints and support tickets logs The raw data is cleaned aggregated correlated and sessionized prior to being passed to the upper layer for warehousing and analysis The ELT process can be executed in batch micro-batch as well as real-time modes (with session servers) and can scale to cope with several hundreds of network elements Finally it supports major protocols and interfaces and offers the appropriate environment to develop custom plugins which is key to adapt to various network types (fixed cable GSM CDMA and others) and to upcoming 4GLTE networks

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data management and structuringADBs provide fast access to that data and allow the logical progression of the communications analyst to drive much deeper into root cause analysis and deliver more in their analysis window than would be permissible with a row-based database Many analytic databases differ in the way the data is stored on disk In those that have adapted a ldquocolumnarrdquo orientation disk files are occupied by the values of a single column instead of complete rows This physical division allows more frequently used data to be assigned to faster storage tiers It also ensures that columns not pertaining to a query are excluded from access This ldquocolumn eliminationrdquo results in improved (sometimes dramatically improved) performance for an important class of query in an enterprise The clustering of similar values in a column also makes columnar databases much more disposed to a new class of compression capabilities Column elimination and compression help to alleviate the IO problems that have been plaguing analytic queries for years Techniques include

bull Run-length encoding whereby column values that repeat across consecutive rows can be stored once

bull Dictionary compression which abstracts the real values and stores only tokens in the record

bull Delta compression which stores only offsets from a set value

In addition some analytic databases such as the one HP Vertica offers are ldquohybridrdquo in the way they store data allowing for multiple columns to be stored in a single disk file When you access columns together you could place them in the same disk file This would streamline the process at the end of the query where the result set columns are brought together for presentation When a table has a large number of columns like CDRs do it is frequently a candidate for effective multiple-column disk files

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Vertica HP Vertica Real Time Analytics Platform is a powerful highly scalable analytics engine ideal for solving todayrsquos Big Data challenges Compared to traditional databases and data warehouses HP Vertica drives down the cost of capturing storing and analyzing data while producing answers 50 to 1000 times faster This enables the iterative conversational approach to analytics you need Enterprise customers choose HP Vertica because it produces answers to business queries in record time at a lower cost than alternative solutions

Click on each icon to read more

HP Verticarsquos high-performance analytics capabilities bring to HAVEn

Bl

azin

g-fa

st a

naly

tics

Massive scalabilit

y

Open architecture Enhanced data storage Real-time loading and querying Advanced in-database analytics

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP AutonomyHP Autonomy combines a purpose-built Intelligent Data Operating Layer (IDOL) technology with an extensive portfolio of applications designed to manage and analyze data to meet specific needs IDOL recognizes concepts and meaning in unstructured human information which falls into two categories

1 Unstructured text data 2 Unstructured rich media

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP ArcSightHP ArcSight is a universal log management solution that unifies searching reporting alerting and analysis across any type of enterprise log data It is unique in its ability to collect analyze and store massive amounts of machine data generated by modern networks HP ArcSight supports multiple deployments such as an appliance software virtual machine and within the cloud in both Microsoftreg Windowsreg and Linux environment The unified data can be stored in any format you have (NAS DAS SAN and so on) through a high compression ratio of up to 101 helping eliminate the need for database administrators

HadoopHadoop complements HP technologies and is a strategic part of HAVEn as a way to cost-effectively store massive amounts of data from virtually any source Hadoop has received significant attention due to its scalable architecture based on commodity hardware a flexible programming language and strong support from the open-source community But enterprises using Hadoop face two key challenges in processing Big Data how to efficiently bring data into Hadoop file systems and how to process unstructured data using Hadoop

Addressing these two challenges is where HP Autonomy and HP Vertica come in The Hadoop distributed file system (HDFS) allows for the distributed processing of large data sets across clusters of computers using simple programming models It is designed to scale up from single servers to thousands of machines each offering local processing and storage Rather than rely on hardware to deliver high-availability the system is designed to detect and handle failures at the application layer delivering a highly available service on top of a cluster of computers HP Vertica and Hadoop are highly complementary systems for Big Data analytics as well HP Vertica is ideal for interactive real-time analytics and Hadoop is well suited for batch-oriented data processing

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Data accessThe vendorrsquos usual statement for analytic query performance seems to be ldquosoldier onrdquo or that the users are not fully exploiting their existing technology The only real answer to the problem if you are sticking with the stack is more hardware However a reality is that most systems are already fully optimized for the hardware in place If there was a way to attack query performance in business intelligence without resorting to hardware it would allow revolutionary leaps in information dissemination in multiple ways

bull Enable query sessions to be truly interactive not limited by poor performance that causes analysis to stop at three interactive queries instead of 10 20 or 100 to achieve actionable insight into business

bull Facilitate rollout of the analytic environment to all knowledge workers of the company as well as customers supply chain partners and broad potential users of the data

bull Allow for years of history to be kept knowing that high volume data can be queried

bull Add the possibilities for CDR text data clickstream and other volume-intensive data to be kept at the detail level for analysis

bull Add the possibilities for analysis of complex data types such as flat files XML graphics and spreadsheets

HP AutonomyHP IDOL forms a conceptual and contextual understanding of all content in an enterprisemdashautomatically analyzing any piece of information from over 1000 different content formats IDOL can perform over 500 operations on digital content including hyperlinking creating agents summarization taxonomy generation clustering education profiling alerting and retrieval

In addition IDOL enables you to benefit from automation without sacrificing manual control This means you can combine automatic processing with a variety of human controllable overrides never forcing you to make an ldquoeitherorrdquo choice IDOL integrates with all known legacy systems

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Smart Profile ServerHP Smart Profile Server (SPS) is a ground-breaking software product for Subscriber Profile Management and Analytics designed especially for CSPs and built to satisfy CSP business needs It proposes a data exposure layer that provides access to analytics insights in a secure and controlled fashion It enables CSPs to adopt new business models by sharing their subscribersrsquo profile (for example interests preferences and such) with third parties such as advertisers The exposure layer offers a multitenant view of subscribers and smart attributes via REST and SOAP APIs The access is subject to robust authentication and authorization mechanisms based on Security Assertion Markup Language (SAML) and Open Mobile Alliance (OMA) In addition it features opt-inout flexible identity and privacy management functions to hidealias identifiers (MSIDN public emails) and provide anonymity of the shared attributes if needed Finally it provides a data cache based on an in-memory database to support northbound real-time queries while protecting the analytics layer from security attacks and unexpected loads

HP SPS comes with a lot of integrated analytic services ready to use such as

bull Categorization and scorings l    Recommendation and Next Best Action (NBA) engine

bull KPI engine l    Predictive engine

bull Network and applications QoE and KPI l    Sentiment analysis (future release)

R integrationThe integration of R into the HP Vertica Analytics Platform provides developer with data mining and statistics with the database With the R integration using standard SQL-99 syntax developers and end users can quickly implement new data mining algorithms into their applications because they are already familiar with SQL This makes using the algorithms much easier as they are embedded within the database itself Developers and users can now access the algorithms using their favorite GUI and BI tools The integration of R into the HP Vertica Analytics Platform enables a wide range of data analytics including regression testing statistical modeling linear spatial models time series forecasting geo-statistical and text mining

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Business intelligenceAnalytics brings together statistics operational research and computational knowledge to solve business challenges by analyzing data held within information systems From patterns in data you can use statistical methods to create models and algorithms that forecast future events The analytics process can be described as two distinct activities modeling and prediction

The modeling starts with the process of mining data to identify patterns and relationships with the intention of explaining a set of actions or of looking for anomalies that identify a sector or cluster After patterns and relationships have been identified a model is created that describes the behavior for example the likelihood of churning the impact of the video on network bandwidth the likelihood of services adoption through new bundles

The frequency with which the model is run depends on the application computational power the amount of data and the complexity of the model There may also be limitations on how quickly new data is fed from source data points With the advent of more-powerful database analytics technology such as HP Vertica the time required to run an analytics model is decreasing Figure 4 Analytics use case for CSPs6

Sales and marketing Network Products andservices

Supplier Customermanagement

Operational and resource management

Enterprise

bull Partner analysisbull Channel analysisbull Campaign analysisbull Customer insight analyticsbull Sales analyticsbull Social network analyticsbull Customer segmentationbull Customer network analysisbull Customer churn analysisbull Customer cross-sell and upsellbull Customer lifetime valuebull Customer profitabilitybull Customer acquisition

bull Capacity managementbull Performance

management

bull Performance analysisbull Margin analysisbull Pricing impact and

stimulationbull Cannibalization impact

of new servicesbull What-if analysis for new

product launchbull Impact of supply chain

bull Cost and contribution analysis

bull Quality controlbull Regulatory requirementsbull Fulfillment analysisbull Quality analysisbull Roaming analyticsbull Settlements

bull Customer churn (retention)

bull Customer experiencebull Service-level

agreementsbull Credit scoringbull Customer retention

analysisbull Customer problem

case analysis

bull Operational analysis of key processes

bull Mean time from order to cash

bull Trouble ticketing resolution

bull Performance management

bull Facility profitability analytics

bull Fraud management and prevention

bull Revenue assurance security

6 ldquoDefining analytics optimizing business processes with existing data in CSPsrdquo Analysis Mason January 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Presentation and visualizationPresentation and visualization functions allow your staff and automated systems that require predictions and results to receive them in a usable format Traditionally delivery has been to a staff member who then acts on it manually But this is increasingly being automated as actions are required in near real time including the use of customer data for real time personalized on-device customer interaction You will demonstrate how you are strengthening customer intimacy enhancing the experience and opening new revenue streams through direct engagement on mobile business intelligence such as HP Anywhere or HP Executive Scorecard

Figure 5 Analytics visualization through customer mobile experience personalization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Also the rich content built into HP ArcSight helps you perform complex searches and create comprehensive drill-down reports In addition you can rely on real-time alerts to use machine data for IT security governance risk and compliance (GRC) IT operations security information and event management (SIEM) solutions and log analytics

TableauThe Tableau Professional Desktop solution is based on breakthrough technology from Stanford University that lets you drag and drop to analyze data You can connect to data in a few clicks then visualize and create interactive dashboards with a few more This drag-and-drop tool that lets you see every change as you make it Work at the speed of thought without ever taking your eyes off the data Anyone comfortable with Microsoft Excel can get up to speed on tableau quickly Business leaders use it to see and understand many facets of their business at a glance Scientists use it to create sophisticated trend analysis Marketers use it to make data-driven decisions that drive ROI

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use casesClick on each icon to read more

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 1 Subscriber network usageIn this first case we would consider a communications service provider who wants to collect CDRs or IPDRs from the different operating system support in the network The first challenge is the quantity of information a service provider may produce about 2 billion CDRs per day

CDRs and IPDRs are still the core of the customer profile CDRs are an important resource for a CSP and the complete record must be stored and made available across the company CDRs and IPDRs provide comprehensive information on each and every call occurring on the data circuits including complete signaling information categorization of the call disruptive alarms and errors occurring during the call detailed voice band event information or main Internet protocols information (email webmail Web browsing file sharing peer-to-peer sharing chat and others)

An analytic database is ideal for analyzing CDR data because the data is characterized by two key properties

1 Massive quantity of data Calls produce readings at regular ratesmdashsometimes thousands of times per second over long time periods Communications devices are ldquoalways onrdquo generating data Consider the packets in a streaming video going to and from the device

2 Need for scan queries A common use of CDR data is to study historical trends and compare time periods and regions For example region managers may want to query all the data in their region and surrounding regions This requires a series of aggregate queries over large amounts of historical data

CSPs will have to rely on fast complex analysis of CDR and IPDR data for implementing critical functions such as

bull Customer relationship managementmdashanalyzing behavioral data to optimally target services and reduce churn

bull Billingmdashensuring complete and accurate billing and avoiding fraud

bull Revenue assurancemdashmodeling call behavior

bullNetwork performancemdashoptimizing network operations using operations management programs

Real-life examplesKDDI Japanrsquos second largest mobile operator relies on constant access to call data to ensure a high level of customer service But when the company launched its first 4G network they were challenged to keep pace with increasing volumes of call data KDDI deployed a HP Vertica-based solution and drove its data analysis time from 3 minutes to 10 seconds and enabled 24x7 high-speed data loading with a compression rate of up to 90 percent7

7 KDDI streamlines data warehouse to improve mobile services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Subscriber network usage componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 2 HP Operations AnalyticsAs CSPs adopt new technologies in data centers and networksmdashsuch as software data network network functions virtualization or cloud servicesmdashIT and OSS organizations no longer know or control all the technologies in their environment and need analytics for predicting the occurrence of problems and identifying issues before they occur Hundreds of devices and applications emit large amounts of data that contain information pertinent to the performance of the CSP network and critical for debugging issues Add this volume to the amount of events IT operations and OSS receives on a daily basis from their proactive performance monitoring tools and IT quickly enters into an unstructured Big Data nightmare

The combination of customerrsquos existing monitoring strategies combined with predictive analytics plus log management and analysis is what we call operational analytics The triggers

bull Need to determine the cause of recurring system or network issues

bull Need to integrate fragmented IT operations for a single view of the infrastructure

bull Want to improve ROI by streamlining IT operations adding efficiency

bull Need to distinguish between performance and functional quality issues and security threats

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

If you only store it for a few hours or a day you might miss out on critical historical information that may point to the root cause of the issues So now you need to be able to store everything including machine data logs events topologies alarms and performance statistics

Also you need to be able to analyze the information to debug the issues HP Operations Analytics delivers actionable intelligence about service health with advanced analytics capabilities for consolidated network and IT data

But just collecting and analyzing logs is not enough IT and network operations need to continue doing their proactive monitoring and also have predictive analytics capabilities so that they can not only resolve any issue but also be able to predict and prevent issues in the first place

By making it possible to collect store and analyze operational data HP Operations Analytics lets you analyze all of your important information to diagnose problems and determine root cause

8 Vodafone Ireland implements world-class service excellence with HP BSM

Real-life examplesVodafone Ireland transformed from reactive to proactive IT operations with HP solutions The success rate for identification of major incident root-cause jumped from 40 percent to 90 percent and Vodafone achieved a 300 percent return on investment in the first year of the HP solutionrsquos deployment based on operational savings8

HP Operations Analytics

Powerful searchAcross logs events topology and performance metrics

Guided troubleshooting

Anomaly and outlier detection and suggestions

Visual analytics

Intuitive visual depiction of relationships and impacts

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 2 HP Operations AnalyticsAs CSPs adopt new technologies in data centers and networksmdashsuch as software data network network functions virtualization or cloud servicesmdashIT and OSS organizations no longer know or control all the technologies in their environment and need analytics for predicting the occurrence of problems and identifying issues before they occur Hundreds of devices and applications emit large amounts of data that contain information pertinent to the performance of the CSP network and critical for debugging issues Add this volume to the amount of events IT operations and OSS receives on a daily basis from their proactive performance monitoring tools and IT quickly enters into an unstructured Big Data nightmare

The combination of customerrsquos existing monitoring strategies combined with predictive analytics plus log management and analysis is what we call operational analytics The triggers

bull Need to determine the cause of recurring system or network issues

bull Need to integrate fragmented IT operations for a single view of the infrastructure

bull Want to improve ROI by streamlining IT operations adding efficiency

bull Need to distinguish between performance and functional quality issues and security threats

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

If you only store it for a few hours or a day you might miss out on critical historical information that may point to the root cause of the issues So now you need to be able to store everything including machine data logs events topologies alarms and performance statistics

Also you need to be able to analyze the information to debug the issues HP Operations Analytics delivers actionable intelligence about service health with advanced analytics capabilities for consolidated network and IT data

But just collecting and analyzing logs is not enough IT and network operations need to continue doing their proactive monitoring and also have predictive analytics capabilities so that they can not only resolve any issue but also be able to predict and prevent issues in the first place

By making it possible to collect store and analyze operational data HP Operations Analytics lets you analyze all of your important information to diagnose problems and determine root cause

8 Vodafone Ireland implements world-class service excellence with HP BSM

Real-life examplesVodafone Ireland transformed from reactive to proactive IT operations with HP solutions The success rate for identification of major incident root-cause jumped from 40 percent to 90 percent and Vodafone achieved a 300 percent return on investment in the first year of the HP solutionrsquos deployment based on operational savings8

HP Operations Analytics

Powerful searchAcross logs events topology and performance metrics

Guided troubleshooting

Anomaly and outlier detection and suggestions

Visual analytics

Intuitive visual depiction of relationships and impacts

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

If you only store it for a few hours or a day you might miss out on critical historical information that may point to the root cause of the issues So now you need to be able to store everything including machine data logs events topologies alarms and performance statistics

Also you need to be able to analyze the information to debug the issues HP Operations Analytics delivers actionable intelligence about service health with advanced analytics capabilities for consolidated network and IT data

But just collecting and analyzing logs is not enough IT and network operations need to continue doing their proactive monitoring and also have predictive analytics capabilities so that they can not only resolve any issue but also be able to predict and prevent issues in the first place

By making it possible to collect store and analyze operational data HP Operations Analytics lets you analyze all of your important information to diagnose problems and determine root cause

8 Vodafone Ireland implements world-class service excellence with HP BSM

Real-life examplesVodafone Ireland transformed from reactive to proactive IT operations with HP solutions The success rate for identification of major incident root-cause jumped from 40 percent to 90 percent and Vodafone achieved a 300 percent return on investment in the first year of the HP solutionrsquos deployment based on operational savings8

HP Operations Analytics

Powerful searchAcross logs events topology and performance metrics

Guided troubleshooting

Anomaly and outlier detection and suggestions

Visual analytics

Intuitive visual depiction of relationships and impacts

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Operations Analytics componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 3 Multichannel customer analyticsFor most CSPs despite claiming to be customer centric obtaining customer insight is no trivial task Maximizing value from customer analytics requires the ability to draw insight from data to accurately understand and predict customer behaviors and to act appropriately

Yet mature organizations are struggling to capture fundamental basics such as

bull Information from every customer touch point

bull Past behaviors current interactions and likely future actions such as customer churn

bull Information from sources that have only recently come online such as social network

bull Larger market or views that contextualize information about individuals

Customer profiling requires the ability to derive data from behavioral context and individual needs in addition to transactional historical and contractual information therefore data needs to be garnered from unstructured as well as structured sources

Increasingly CSPs are looking to sources of information that do not rely on them collecting the right data and asking the right questions They need to proactively understand and improve their net promoter score at the individual customer level by encapsulating 100 percent of the subscriberrsquos relevant promoter data garnered from surveys blogs social network Web clickstream customer feedback and contact center voice recording

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Autonomy Explore our multichannel analytics solution removes barriers between multichannel interaction data to give you a real-time unified view of consumer behavior by

Leveraging direct survey verbatim Automate the process of understanding verbatim comments which allows you to fully leverage the rich data contained within these surveys

Making sense of customer activity beyond clicks and visits Track how users interact online as they tag pages jump from page to page post reviews and more It is based on the goal of determining the failure of an online program based on a pre-determined success metric

Understanding opinions and sentiment on social media Understand social conversations from over 200 million daily social media and broadcast news mentions from across the globe from sites such as Facebook Twitter various news channel YouTube and Google+mdashin more than 50 languages

Mining rich insights from contact center interactions Analyze elements such as topics discussed speaker identity and gender emotions contained in the conversation and excessive silence or crosstalk

Enriching your data with insights from customer interactions Make your data intelligent for example filter all net promoter score customer surveys and then group the verbatim responses according to concepts to identify emerging themes

Understanding the voice of the customer Get a comprehensive view of all customer interactionsmdashcall center Web email chat and social mediamdashto reveal the concerns and sentiments of your market

Real-life exampleNASCAR Fan and Media Engagement Center (FMEC) is facilitating real-time response to traditional digital broadcast and social media This enables the company to better position itself and drives results for sponsors and media partners HP Autonomy technology and supporting applications give NASCAR access to a complete analysis of all key forms of media including print television radio video images and social media Unlike other monitoring and measurement systems that focus on only social or digital media the FMEC platform gives NASCAR a unique perspective that combines several different types of media9

9 Take a look at what NASCAR has accomplished with HAVEn technology

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 3 Multichannel customer analyticsFor most CSPs despite claiming to be customer centric obtaining customer insight is no trivial task Maximizing value from customer analytics requires the ability to draw insight from data to accurately understand and predict customer behaviors and to act appropriately

Yet mature organizations are struggling to capture fundamental basics such as

bull Information from every customer touch point

bull Past behaviors current interactions and likely future actions such as customer churn

bull Information from sources that have only recently come online such as social network

bull Larger market or views that contextualize information about individuals

Customer profiling requires the ability to derive data from behavioral context and individual needs in addition to transactional historical and contractual information therefore data needs to be garnered from unstructured as well as structured sources

Increasingly CSPs are looking to sources of information that do not rely on them collecting the right data and asking the right questions They need to proactively understand and improve their net promoter score at the individual customer level by encapsulating 100 percent of the subscriberrsquos relevant promoter data garnered from surveys blogs social network Web clickstream customer feedback and contact center voice recording

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Autonomy Explore our multichannel analytics solution removes barriers between multichannel interaction data to give you a real-time unified view of consumer behavior by

Leveraging direct survey verbatim Automate the process of understanding verbatim comments which allows you to fully leverage the rich data contained within these surveys

Making sense of customer activity beyond clicks and visits Track how users interact online as they tag pages jump from page to page post reviews and more It is based on the goal of determining the failure of an online program based on a pre-determined success metric

Understanding opinions and sentiment on social media Understand social conversations from over 200 million daily social media and broadcast news mentions from across the globe from sites such as Facebook Twitter various news channel YouTube and Google+mdashin more than 50 languages

Mining rich insights from contact center interactions Analyze elements such as topics discussed speaker identity and gender emotions contained in the conversation and excessive silence or crosstalk

Enriching your data with insights from customer interactions Make your data intelligent for example filter all net promoter score customer surveys and then group the verbatim responses according to concepts to identify emerging themes

Understanding the voice of the customer Get a comprehensive view of all customer interactionsmdashcall center Web email chat and social mediamdashto reveal the concerns and sentiments of your market

Real-life exampleNASCAR Fan and Media Engagement Center (FMEC) is facilitating real-time response to traditional digital broadcast and social media This enables the company to better position itself and drives results for sponsors and media partners HP Autonomy technology and supporting applications give NASCAR access to a complete analysis of all key forms of media including print television radio video images and social media Unlike other monitoring and measurement systems that focus on only social or digital media the FMEC platform gives NASCAR a unique perspective that combines several different types of media9

9 Take a look at what NASCAR has accomplished with HAVEn technology

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HP Autonomy Explore our multichannel analytics solution removes barriers between multichannel interaction data to give you a real-time unified view of consumer behavior by

Leveraging direct survey verbatim Automate the process of understanding verbatim comments which allows you to fully leverage the rich data contained within these surveys

Making sense of customer activity beyond clicks and visits Track how users interact online as they tag pages jump from page to page post reviews and more It is based on the goal of determining the failure of an online program based on a pre-determined success metric

Understanding opinions and sentiment on social media Understand social conversations from over 200 million daily social media and broadcast news mentions from across the globe from sites such as Facebook Twitter various news channel YouTube and Google+mdashin more than 50 languages

Mining rich insights from contact center interactions Analyze elements such as topics discussed speaker identity and gender emotions contained in the conversation and excessive silence or crosstalk

Enriching your data with insights from customer interactions Make your data intelligent for example filter all net promoter score customer surveys and then group the verbatim responses according to concepts to identify emerging themes

Understanding the voice of the customer Get a comprehensive view of all customer interactionsmdashcall center Web email chat and social mediamdashto reveal the concerns and sentiments of your market

Real-life exampleNASCAR Fan and Media Engagement Center (FMEC) is facilitating real-time response to traditional digital broadcast and social media This enables the company to better position itself and drives results for sponsors and media partners HP Autonomy technology and supporting applications give NASCAR access to a complete analysis of all key forms of media including print television radio video images and social media Unlike other monitoring and measurement systems that focus on only social or digital media the FMEC platform gives NASCAR a unique perspective that combines several different types of media9

9 Take a look at what NASCAR has accomplished with HAVEn technology

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Multichannel customer analytics componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 4 HP Mobile Experience PersonalizationHarvesting customer data provides you with the opportunity to strengthen your customer relationships and gain a competitive advantage Designed to promote a highly personalized customer experience HP Mobile Experience Personalization solution is targeted at reducing churn intelligently upselling telecom products and services and creating new revenue streams

HP Mobile Experience Personalization implementation includes four functional areas

1 Drawing subscriber profiles usage events and demographic information and identity from all sources of CSP operation home location registerhome subscriber server (HLRHSS) usage detail record (UDR) CRM location network traffic provisioning and charging

2 Collecting these events into a power analytics environment such as HP Vertica

3 Applying real-time profiling and analytics on data across IT network and Web sources to build an enriched actionable subscriber profile and insight

4 Exposing the smart profile through CSP mobile portal to enable personalization and subscriber interaction in the areas of self care social media updates personalized advertising and contentmdashpremium and news

The Mobile Experience Personalization implementation is about enabling CSPs personalizing the service experience to develop subscriber intimacy and stickiness resulting in reduced churn relevant recommendations increased revenue and uptake of data usage and services and personalized advertising channels

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Mobile experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use Case 5 HP Ad Experience PersonalizationYou decide to book a plane ticket to New York on the Internet Few clicks later when reading the newspaper online an advertisement displays an attractive offer for a car rental in New York Itrsquos not a coincidence it is a mechanism for targeted advertisement

The business model for many leading over-the-top companies like Internet search engines or social media is based on a subscription apparently ldquofreerdquo to the user but predominantly if not exclusively funded by advertisements This delivery model for services backed up by advertisements has become almost the norm so that the user subscribing for these free services receives an increasing amount of advertisements Advertising is therefore a strategic issue for all the major players in the digital world For service providers too it is a challenge that they need to address Being able to deliver targeted advertisement as possible to the end-user profile behavior and preferences is a mandatory path to increasing their positioning in the value chain and to the prevention of becoming simple commodities

Over the past years CSPsrsquo advertising systems have reached various levels of maturity However there is an increasing demand for introducing personalization in these advertising systems and the HP Ad Experience Personalization solution provides you with an answer to this new challenge by

bull Building a rich end-user profile by collecting data from across the operatorrsquos network

bull Analyzing the profile and enrich it by inferring behavior preferences or additional inclinations the purpose is to make the inferred data actionable to deliver personalized advertisements

bull Enabling targeted campaign triggers by allowing searching filtering queries and maintaining confidentiality toward third parties when required

By integrating the power of the HP Vertica Analytics Platform the HP Ad Experience Personalization solution adds a unique knowledge and intelligence to the simple delivery of advertisements It brings you into the new dimension of targeted advertising with tailored offering having unequaled high acceptance rates

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HAVEn

Ad experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 4 HP Mobile Experience PersonalizationHarvesting customer data provides you with the opportunity to strengthen your customer relationships and gain a competitive advantage Designed to promote a highly personalized customer experience HP Mobile Experience Personalization solution is targeted at reducing churn intelligently upselling telecom products and services and creating new revenue streams

HP Mobile Experience Personalization implementation includes four functional areas

1 Drawing subscriber profiles usage events and demographic information and identity from all sources of CSP operation home location registerhome subscriber server (HLRHSS) usage detail record (UDR) CRM location network traffic provisioning and charging

2 Collecting these events into a power analytics environment such as HP Vertica

3 Applying real-time profiling and analytics on data across IT network and Web sources to build an enriched actionable subscriber profile and insight

4 Exposing the smart profile through CSP mobile portal to enable personalization and subscriber interaction in the areas of self care social media updates personalized advertising and contentmdashpremium and news

The Mobile Experience Personalization implementation is about enabling CSPs personalizing the service experience to develop subscriber intimacy and stickiness resulting in reduced churn relevant recommendations increased revenue and uptake of data usage and services and personalized advertising channels

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Mobile experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use Case 5 HP Ad Experience PersonalizationYou decide to book a plane ticket to New York on the Internet Few clicks later when reading the newspaper online an advertisement displays an attractive offer for a car rental in New York Itrsquos not a coincidence it is a mechanism for targeted advertisement

The business model for many leading over-the-top companies like Internet search engines or social media is based on a subscription apparently ldquofreerdquo to the user but predominantly if not exclusively funded by advertisements This delivery model for services backed up by advertisements has become almost the norm so that the user subscribing for these free services receives an increasing amount of advertisements Advertising is therefore a strategic issue for all the major players in the digital world For service providers too it is a challenge that they need to address Being able to deliver targeted advertisement as possible to the end-user profile behavior and preferences is a mandatory path to increasing their positioning in the value chain and to the prevention of becoming simple commodities

Over the past years CSPsrsquo advertising systems have reached various levels of maturity However there is an increasing demand for introducing personalization in these advertising systems and the HP Ad Experience Personalization solution provides you with an answer to this new challenge by

bull Building a rich end-user profile by collecting data from across the operatorrsquos network

bull Analyzing the profile and enrich it by inferring behavior preferences or additional inclinations the purpose is to make the inferred data actionable to deliver personalized advertisements

bull Enabling targeted campaign triggers by allowing searching filtering queries and maintaining confidentiality toward third parties when required

By integrating the power of the HP Vertica Analytics Platform the HP Ad Experience Personalization solution adds a unique knowledge and intelligence to the simple delivery of advertisements It brings you into the new dimension of targeted advertising with tailored offering having unequaled high acceptance rates

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HAVEn

Ad experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Mobile experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use Case 5 HP Ad Experience PersonalizationYou decide to book a plane ticket to New York on the Internet Few clicks later when reading the newspaper online an advertisement displays an attractive offer for a car rental in New York Itrsquos not a coincidence it is a mechanism for targeted advertisement

The business model for many leading over-the-top companies like Internet search engines or social media is based on a subscription apparently ldquofreerdquo to the user but predominantly if not exclusively funded by advertisements This delivery model for services backed up by advertisements has become almost the norm so that the user subscribing for these free services receives an increasing amount of advertisements Advertising is therefore a strategic issue for all the major players in the digital world For service providers too it is a challenge that they need to address Being able to deliver targeted advertisement as possible to the end-user profile behavior and preferences is a mandatory path to increasing their positioning in the value chain and to the prevention of becoming simple commodities

Over the past years CSPsrsquo advertising systems have reached various levels of maturity However there is an increasing demand for introducing personalization in these advertising systems and the HP Ad Experience Personalization solution provides you with an answer to this new challenge by

bull Building a rich end-user profile by collecting data from across the operatorrsquos network

bull Analyzing the profile and enrich it by inferring behavior preferences or additional inclinations the purpose is to make the inferred data actionable to deliver personalized advertisements

bull Enabling targeted campaign triggers by allowing searching filtering queries and maintaining confidentiality toward third parties when required

By integrating the power of the HP Vertica Analytics Platform the HP Ad Experience Personalization solution adds a unique knowledge and intelligence to the simple delivery of advertisements It brings you into the new dimension of targeted advertising with tailored offering having unequaled high acceptance rates

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HAVEn

Ad experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use Case 5 HP Ad Experience PersonalizationYou decide to book a plane ticket to New York on the Internet Few clicks later when reading the newspaper online an advertisement displays an attractive offer for a car rental in New York Itrsquos not a coincidence it is a mechanism for targeted advertisement

The business model for many leading over-the-top companies like Internet search engines or social media is based on a subscription apparently ldquofreerdquo to the user but predominantly if not exclusively funded by advertisements This delivery model for services backed up by advertisements has become almost the norm so that the user subscribing for these free services receives an increasing amount of advertisements Advertising is therefore a strategic issue for all the major players in the digital world For service providers too it is a challenge that they need to address Being able to deliver targeted advertisement as possible to the end-user profile behavior and preferences is a mandatory path to increasing their positioning in the value chain and to the prevention of becoming simple commodities

Over the past years CSPsrsquo advertising systems have reached various levels of maturity However there is an increasing demand for introducing personalization in these advertising systems and the HP Ad Experience Personalization solution provides you with an answer to this new challenge by

bull Building a rich end-user profile by collecting data from across the operatorrsquos network

bull Analyzing the profile and enrich it by inferring behavior preferences or additional inclinations the purpose is to make the inferred data actionable to deliver personalized advertisements

bull Enabling targeted campaign triggers by allowing searching filtering queries and maintaining confidentiality toward third parties when required

By integrating the power of the HP Vertica Analytics Platform the HP Ad Experience Personalization solution adds a unique knowledge and intelligence to the simple delivery of advertisements It brings you into the new dimension of targeted advertising with tailored offering having unequaled high acceptance rates

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HAVEn

Ad experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

HAVEn

Ad experience personalization componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 6 Big Data for securityThe volumes of event data that are reported to your security information event management (SIEM) solution is growing exponentially and attacks are every day more sophisticated It becomes critical to enrich your security events with the source asset user and data-intelligent situational awareness In addition real-time correlation of this information with event flows needs to be accommodated with a storage infrastructure that can handle these millions of data points organizations and devices without hitting a brick wall in expanding the intelligence of your security The requirements for obtaining true situational awareness go far beyond simple event flow analysis

Todayrsquos SIEMs require real-time analytics that identifies changes in usage behavior and dynamically adjusts risk based on source reputation and asset risk as well as the data applications network and database activity that relates to it Real-time analytic is a critical component of attack detection and Big Data architectures need to be implemented to accommodate

bull The support pattern-based intelligence that enables visualization and investigation of collusive or criminal networks activities and behaviors

bull The improvement system performance as opposed to business users trying to solve overall security problems such as theft of intellectual property or financial assets

bull Continuous improvement necessary to alleviatemdashconstantly moving targets

Real-time analytic allows you to collect store and analyze any log or event data from any system It then correlates system events flow information and user and application activity to help answer the who what and when of everything that has happened or is currently happening in your organization

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Examples of the applicability of Big Data for security include

Account take over An incoming transaction is using the same device from the same location associated with this network of suspect activity previously identified in the Big Data analysis and triggers a high alert

Insider threats An organizational analyst then mines the information to find unusual building access (off hours) by a system administrator who uses a company desktop to access sensitive files that other employees in his job category have not accessed before

DDoS attack mitigation Detect patterns of DDoS attacks so that they can deny future Internet traffic using these patterns

Security detection at the edge of the network Using already-installed network devices to perform data collection and minimizing monitoring costs The fast detection of abnormal events helps minimize potential damage by allowing security teams to act more quickly

The security detection and response are supported by the HP Vertica Analytics Platform and HP ArcSight Logger Within these solutions data gathered are correlated with other infrastructure data to provide additional insight into infrastructure events and to provide the security operations center with the actionable information they need to respond to events

HP ArcSight is supported by the revolutionary CORR Engine which delivers industry-leading correlation speeds with significant storage requirement decreases from prior versions The HP Vertica Analytics Platform engine both stores and analyzes these enormous amounts of data allowing the security team to use it to parse historic activity HP Vertica also supports very fast query performance therefore the security team doesnrsquot experience long lags when they use the dashboard to load and query data and generate useful reports It helps security team better understand how application eco-systems and networks interact and communicate by gaining direct insight into how its protocols affect applications

Real-life examplesHP Vertica Analytics Platform is an integral component of Lancope StealthWatch because it enables the tool to monitor and analyze HP networkrsquos 450000 flowssecond providing comprehensive actionable information on network activity The combination of continual network flow monitoring and analyticsmdashprovided by Lancope StealthWatch a partner network monitoring tool that leverages the HP Vertica Analytics Platformmdashand the monitoring and intrusion protection capabilities that HP ArcSight and HP TippingPoint offer enable the HP Cyber Security team to respond to events quickly and effectively HP Verticarsquos analytical capabilities and fast query speeds help detect network security issues as quickly as possible which provides the HP network security team a cost-effective yet powerful way to monitor and analyze the network traffic10

10 Read about HP network

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Big Data for security componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 7 Fraud preventionDetecting fraud is a game of pattern recognition and the patterns always change CSPs are impacted as they continue to see an increase in volume and variety of financial transactions and data transiting through their network and billing solution

Hackers are thwarted only to come back through a different security hole and novel scams are being thought up for loopholes and exceptions in business workflows And the playing field keeps growing as volume and velocity of the transactions in your network keep increasing with the source and type of transactions Your proprietary systems canrsquot keep up as it is expensive to scale for todayrsquos ldquoBig Datardquo real-time transaction streams and it is difficult to modify legacy analytic code and architectures to keep up with changing patterns

Therefore comprehensive monitoring is critical to recognizing patterns You need to consider a dual approach of real-time monitoring and right-time analytics to stop fraudsters while gaining the enhanced knowledge you need to implement effective preventive measures

CSPs need a fraud prevention strategy that

bull Gives a comprehensive view of the problems and opportunities

bull Evolves with future complexity scales with future growth

bull Streamlines decision making throughout the revenue chain to accelerate action

Most CSPs fraud solutions are limited to developing profiles on usage at the individual account level and within a given application which limits visibility into low-volume fraud executed through multiple accounts Extension of fraud management tools to include intelligence capabilities enables companies to perform data mining for better risk management

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Fraud prevention componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 8 Big Data as a serviceBig Data clearly presents a big opportunity for service providers especially for service providers who already have infrastructure-as-a-service and storage-as-a service offerings It also presents challenges as moving into this space requires new technologies and skills The challenge is to pick the right kind of services and technologies from the available options

The Big Data-as-a-service market is in its early stages and there is still relatively little market analysis data available However you can gain some indication of the market size by examining what percentage of all workloads is moving from enterprise premises to public or virtual private cloud service providers and applying those trends to the Big Data market figures By 2016 public IT cloud services will account for 16 of IT revenue12

Provided that the Big Data drives rapid changes in infrastructure and $232 billion USD in IT spending through 2016 the Big Data-as-a-service market will therefore be 16 percent of that figure or $37 billion USD With an estimated Big Data market of about $88 billion USD in 2021 the Big Data-as-a-service market at 35 percent of that or about $30 billion USD13

There are multiple ways to address the Big Data market with as- a-service offerings These can be roughly categorized by level of abstraction from infrastructure to analytics software CSPs must base their decisions on their customersrsquo demands their own skill base and the relative technology strengths and weaknesses of their partner ecosystem

bull Hosted data model Hardware software data infrastructure and business intelligence are dedicated to single customer on the service providerrsquos premise

bull SaaS Business Intelligence SaaS BI (also known as on-demand BI or cloud BI) involves delivery of BI applications to end users from a hosted location while the data infrastructure remains on the customer premises This model is scalable and makes start up easier

bull Cloud BI broker Service providers become a cloud business intelligence broker and uses cloud-based tools and data from different sources that include the customer data CSPs subscriber data and social media sites that best serve the business customer purposes

Real-life exampleKansys provides event-monitoring solutions for CSPs The company needed to streamline and speed up the processing of massive amounts of service-provider data and also increase the speed of reporting and ad-hoc querying HP Vertica delivered a solution that gives Kansys dramatically faster performance as well as massive scalability11

11 Read about Kansys12 Worldwide and Regional Public IT Cloud

Services 2012ndash2016 Forecast IDC August 201213 Big Data drives rapid changes in infrastructure and

$232 billion in IT spending through 2016 Gartner October 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 7 Fraud preventionDetecting fraud is a game of pattern recognition and the patterns always change CSPs are impacted as they continue to see an increase in volume and variety of financial transactions and data transiting through their network and billing solution

Hackers are thwarted only to come back through a different security hole and novel scams are being thought up for loopholes and exceptions in business workflows And the playing field keeps growing as volume and velocity of the transactions in your network keep increasing with the source and type of transactions Your proprietary systems canrsquot keep up as it is expensive to scale for todayrsquos ldquoBig Datardquo real-time transaction streams and it is difficult to modify legacy analytic code and architectures to keep up with changing patterns

Therefore comprehensive monitoring is critical to recognizing patterns You need to consider a dual approach of real-time monitoring and right-time analytics to stop fraudsters while gaining the enhanced knowledge you need to implement effective preventive measures

CSPs need a fraud prevention strategy that

bull Gives a comprehensive view of the problems and opportunities

bull Evolves with future complexity scales with future growth

bull Streamlines decision making throughout the revenue chain to accelerate action

Most CSPs fraud solutions are limited to developing profiles on usage at the individual account level and within a given application which limits visibility into low-volume fraud executed through multiple accounts Extension of fraud management tools to include intelligence capabilities enables companies to perform data mining for better risk management

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Fraud prevention componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 8 Big Data as a serviceBig Data clearly presents a big opportunity for service providers especially for service providers who already have infrastructure-as-a-service and storage-as-a service offerings It also presents challenges as moving into this space requires new technologies and skills The challenge is to pick the right kind of services and technologies from the available options

The Big Data-as-a-service market is in its early stages and there is still relatively little market analysis data available However you can gain some indication of the market size by examining what percentage of all workloads is moving from enterprise premises to public or virtual private cloud service providers and applying those trends to the Big Data market figures By 2016 public IT cloud services will account for 16 of IT revenue12

Provided that the Big Data drives rapid changes in infrastructure and $232 billion USD in IT spending through 2016 the Big Data-as-a-service market will therefore be 16 percent of that figure or $37 billion USD With an estimated Big Data market of about $88 billion USD in 2021 the Big Data-as-a-service market at 35 percent of that or about $30 billion USD13

There are multiple ways to address the Big Data market with as- a-service offerings These can be roughly categorized by level of abstraction from infrastructure to analytics software CSPs must base their decisions on their customersrsquo demands their own skill base and the relative technology strengths and weaknesses of their partner ecosystem

bull Hosted data model Hardware software data infrastructure and business intelligence are dedicated to single customer on the service providerrsquos premise

bull SaaS Business Intelligence SaaS BI (also known as on-demand BI or cloud BI) involves delivery of BI applications to end users from a hosted location while the data infrastructure remains on the customer premises This model is scalable and makes start up easier

bull Cloud BI broker Service providers become a cloud business intelligence broker and uses cloud-based tools and data from different sources that include the customer data CSPs subscriber data and social media sites that best serve the business customer purposes

Real-life exampleKansys provides event-monitoring solutions for CSPs The company needed to streamline and speed up the processing of massive amounts of service-provider data and also increase the speed of reporting and ad-hoc querying HP Vertica delivered a solution that gives Kansys dramatically faster performance as well as massive scalability11

11 Read about Kansys12 Worldwide and Regional Public IT Cloud

Services 2012ndash2016 Forecast IDC August 201213 Big Data drives rapid changes in infrastructure and

$232 billion in IT spending through 2016 Gartner October 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Fraud prevention componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 8 Big Data as a serviceBig Data clearly presents a big opportunity for service providers especially for service providers who already have infrastructure-as-a-service and storage-as-a service offerings It also presents challenges as moving into this space requires new technologies and skills The challenge is to pick the right kind of services and technologies from the available options

The Big Data-as-a-service market is in its early stages and there is still relatively little market analysis data available However you can gain some indication of the market size by examining what percentage of all workloads is moving from enterprise premises to public or virtual private cloud service providers and applying those trends to the Big Data market figures By 2016 public IT cloud services will account for 16 of IT revenue12

Provided that the Big Data drives rapid changes in infrastructure and $232 billion USD in IT spending through 2016 the Big Data-as-a-service market will therefore be 16 percent of that figure or $37 billion USD With an estimated Big Data market of about $88 billion USD in 2021 the Big Data-as-a-service market at 35 percent of that or about $30 billion USD13

There are multiple ways to address the Big Data market with as- a-service offerings These can be roughly categorized by level of abstraction from infrastructure to analytics software CSPs must base their decisions on their customersrsquo demands their own skill base and the relative technology strengths and weaknesses of their partner ecosystem

bull Hosted data model Hardware software data infrastructure and business intelligence are dedicated to single customer on the service providerrsquos premise

bull SaaS Business Intelligence SaaS BI (also known as on-demand BI or cloud BI) involves delivery of BI applications to end users from a hosted location while the data infrastructure remains on the customer premises This model is scalable and makes start up easier

bull Cloud BI broker Service providers become a cloud business intelligence broker and uses cloud-based tools and data from different sources that include the customer data CSPs subscriber data and social media sites that best serve the business customer purposes

Real-life exampleKansys provides event-monitoring solutions for CSPs The company needed to streamline and speed up the processing of massive amounts of service-provider data and also increase the speed of reporting and ad-hoc querying HP Vertica delivered a solution that gives Kansys dramatically faster performance as well as massive scalability11

11 Read about Kansys12 Worldwide and Regional Public IT Cloud

Services 2012ndash2016 Forecast IDC August 201213 Big Data drives rapid changes in infrastructure and

$232 billion in IT spending through 2016 Gartner October 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 8 Big Data as a serviceBig Data clearly presents a big opportunity for service providers especially for service providers who already have infrastructure-as-a-service and storage-as-a service offerings It also presents challenges as moving into this space requires new technologies and skills The challenge is to pick the right kind of services and technologies from the available options

The Big Data-as-a-service market is in its early stages and there is still relatively little market analysis data available However you can gain some indication of the market size by examining what percentage of all workloads is moving from enterprise premises to public or virtual private cloud service providers and applying those trends to the Big Data market figures By 2016 public IT cloud services will account for 16 of IT revenue12

Provided that the Big Data drives rapid changes in infrastructure and $232 billion USD in IT spending through 2016 the Big Data-as-a-service market will therefore be 16 percent of that figure or $37 billion USD With an estimated Big Data market of about $88 billion USD in 2021 the Big Data-as-a-service market at 35 percent of that or about $30 billion USD13

There are multiple ways to address the Big Data market with as- a-service offerings These can be roughly categorized by level of abstraction from infrastructure to analytics software CSPs must base their decisions on their customersrsquo demands their own skill base and the relative technology strengths and weaknesses of their partner ecosystem

bull Hosted data model Hardware software data infrastructure and business intelligence are dedicated to single customer on the service providerrsquos premise

bull SaaS Business Intelligence SaaS BI (also known as on-demand BI or cloud BI) involves delivery of BI applications to end users from a hosted location while the data infrastructure remains on the customer premises This model is scalable and makes start up easier

bull Cloud BI broker Service providers become a cloud business intelligence broker and uses cloud-based tools and data from different sources that include the customer data CSPs subscriber data and social media sites that best serve the business customer purposes

Real-life exampleKansys provides event-monitoring solutions for CSPs The company needed to streamline and speed up the processing of massive amounts of service-provider data and also increase the speed of reporting and ad-hoc querying HP Vertica delivered a solution that gives Kansys dramatically faster performance as well as massive scalability11

11 Read about Kansys12 Worldwide and Regional Public IT Cloud

Services 2012ndash2016 Forecast IDC August 201213 Big Data drives rapid changes in infrastructure and

$232 billion in IT spending through 2016 Gartner October 2012

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Big Data as a service deployment

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 9 Machine to machineBillions of connected devices start to be deployed in the world with ebook readers smart meters and connected cars tracking devices on containers health monitoring wearable home appliances building infrastructure monitoring bridge or dam surveillance environment sensors and so on Sensor technology is becoming smaller and smaller more and more sensitive and accelerometers are now inserted on chipsets Communication protocols especially wireless have also developed in many directions to enable very diverse scenarios from low bandwidth low power to high bandwidth more energy-consuming technologies

According to the independent wireless analyst firm Berg Insight the number of cellular network connections (wireless WAN) worldwide used for M2M communication was 477 million in 200814 The company forecasts that the number of M2M connections will grow to 187 million by 2014 This represents an opportunity of more than $50 billion USD for CSPs alone in 2015 according to Harbor15 All those devices need to be connected to ldquotherdquo network authenticated provisioned and monitored Data needs to be collected analyzed stored secured dispatched presented or leveraged by some sort of applications or end users

The opportunity is huge for new solutions new services that combine connectivity data and service management and ecosystem management As a CSP you are uniquely positioned to serve this market and deliver federated trusted and flexible environments for device and application vendors to collaborate and deliver these future solutions

HP M2M Solution offering covers a broad spectrum of service provider responsibility in this value chain connectivity and communication data and service management and ecosystem management Communication includes device management SIM management and self-care portal device HLR for authentication security and location Unstructured Supplementary Service Data (USSD) and SMS gateway Data and service management addresses data collection aggregation correlation repository provisioning and control as well as monitoring quality of service and usage tracking without forgetting authorization authentication and security As traffic grows Big Data analytics becomes critical and HP is well positioned with solutions leveraging HP Vertica real-time analytics

14 ldquoThe global wireless M2M marketmdash4th editionrdquo Berg Insight April 2012

15 ldquoCreative M2M partnerships drive new value for wireless carriersrdquo white paper Harbor Research Inc March 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Machine to machine componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Whatrsquos your next move Consider these seven questionsOur customers and partners are rapidly embracing HAVEn for a wide variety of industry-specific uses tailoring the platform to solve their specific Big Data challenges

The best way to get started on the road to addressing your unique data needs is to ask yourself these questions

1 Are you able to process store and index all critical data across your organization structured unstructured and semi-structured

2 Do you have insights into customer behavior sentiment churn and brand loyalty And how easily and quickly can you gain these insights and act on them

3 Do all components of your data management infrastructure work together Or are data and applications siloed with little or no centralized access

4 Are you adequately addressing your business mandates for compliance monitoring and data retention

5 Do you have the Big Data expertise in-house to efficiently handle explosive data growth and the increasing need to deliver meaningful analytics or insights to the business

6 Can you draw connected actionable intelligence from a combination of traditional transactional new ldquonetwork-of-thingsrdquo machine data and unstructured data such as social media and disparate knowledge worker documents and records

7 Can you enable new business model as you are starting to realize that the information you have on your subscribers is an untapped asset Can you choose the potential offered by new revenues stream as the biggest opportunity

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

The HAVEn ecosystemA rich ecosystem extends the HAVEn platform The HAVEn ecosystem combines the platform with a wealth of HP resources partners and integrators across the globe There are three key differentiators that make the HAVEn ecosystem one of the industry leaders in Big Data

1 Vision and breadth Working closely with our global IT partner ecosystem HP delivers the hardware software services infrastructure and business-transformation consulting required for you to achieve a successful Big Data strategy HP is the largest IT company in the world with the most extensive partner network and is the only vendor with leading Big Data software namelymdashHP Autonomy HP Vertica and HP ArcSight

2 Openness HAVEn makes connected intelligence a realitymdashwhile preserving customer choice in technologies and protecting existing investments

3 Not just technology but business transformation We have decades of experience helping businesses transform CSPs IT and network operation HAVEn extends that record of success and industry leadership to 100 percent of your meaningful data And our global scale enables us to deliver innovations based on knowledge gained across industries worldwide

4 HP CMS Service offers a broader portfolio of solution services that can help you to navigate your transformation journey HP CMS services portfolio includes CMS telco Big Data workshop Connecting CSPsrsquo business strategies with Big Data implementation roadmap

Predefined telco-specific analytic use case Deploying large set of predefined ready-to-use analytic use cases that can be monetized quickly

CMS architecture services CSP high-skilled architects can help you to easy and with low risk integrated new analytic solution with existing ones with networks with CRM and any other Network systems

HP CMS Big Data and analytic services for CSPs Providing high-skilled consultants who help the CSPs implement analytic use cases to help optimize traditional business and discover new revenue streams

Across the globe enterprise customers rely on HP CMS Services to design deploy operate and support the IT and network systems that run their businesses HP CMS Services capabilities cover consulting and integration outsourcing and support services With more than 3000 services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Use case 9 Machine to machineBillions of connected devices start to be deployed in the world with ebook readers smart meters and connected cars tracking devices on containers health monitoring wearable home appliances building infrastructure monitoring bridge or dam surveillance environment sensors and so on Sensor technology is becoming smaller and smaller more and more sensitive and accelerometers are now inserted on chipsets Communication protocols especially wireless have also developed in many directions to enable very diverse scenarios from low bandwidth low power to high bandwidth more energy-consuming technologies

According to the independent wireless analyst firm Berg Insight the number of cellular network connections (wireless WAN) worldwide used for M2M communication was 477 million in 200814 The company forecasts that the number of M2M connections will grow to 187 million by 2014 This represents an opportunity of more than $50 billion USD for CSPs alone in 2015 according to Harbor15 All those devices need to be connected to ldquotherdquo network authenticated provisioned and monitored Data needs to be collected analyzed stored secured dispatched presented or leveraged by some sort of applications or end users

The opportunity is huge for new solutions new services that combine connectivity data and service management and ecosystem management As a CSP you are uniquely positioned to serve this market and deliver federated trusted and flexible environments for device and application vendors to collaborate and deliver these future solutions

HP M2M Solution offering covers a broad spectrum of service provider responsibility in this value chain connectivity and communication data and service management and ecosystem management Communication includes device management SIM management and self-care portal device HLR for authentication security and location Unstructured Supplementary Service Data (USSD) and SMS gateway Data and service management addresses data collection aggregation correlation repository provisioning and control as well as monitoring quality of service and usage tracking without forgetting authorization authentication and security As traffic grows Big Data analytics becomes critical and HP is well positioned with solutions leveraging HP Vertica real-time analytics

14 ldquoThe global wireless M2M marketmdash4th editionrdquo Berg Insight April 2012

15 ldquoCreative M2M partnerships drive new value for wireless carriersrdquo white paper Harbor Research Inc March 2011

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Machine to machine componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Whatrsquos your next move Consider these seven questionsOur customers and partners are rapidly embracing HAVEn for a wide variety of industry-specific uses tailoring the platform to solve their specific Big Data challenges

The best way to get started on the road to addressing your unique data needs is to ask yourself these questions

1 Are you able to process store and index all critical data across your organization structured unstructured and semi-structured

2 Do you have insights into customer behavior sentiment churn and brand loyalty And how easily and quickly can you gain these insights and act on them

3 Do all components of your data management infrastructure work together Or are data and applications siloed with little or no centralized access

4 Are you adequately addressing your business mandates for compliance monitoring and data retention

5 Do you have the Big Data expertise in-house to efficiently handle explosive data growth and the increasing need to deliver meaningful analytics or insights to the business

6 Can you draw connected actionable intelligence from a combination of traditional transactional new ldquonetwork-of-thingsrdquo machine data and unstructured data such as social media and disparate knowledge worker documents and records

7 Can you enable new business model as you are starting to realize that the information you have on your subscribers is an untapped asset Can you choose the potential offered by new revenues stream as the biggest opportunity

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

The HAVEn ecosystemA rich ecosystem extends the HAVEn platform The HAVEn ecosystem combines the platform with a wealth of HP resources partners and integrators across the globe There are three key differentiators that make the HAVEn ecosystem one of the industry leaders in Big Data

1 Vision and breadth Working closely with our global IT partner ecosystem HP delivers the hardware software services infrastructure and business-transformation consulting required for you to achieve a successful Big Data strategy HP is the largest IT company in the world with the most extensive partner network and is the only vendor with leading Big Data software namelymdashHP Autonomy HP Vertica and HP ArcSight

2 Openness HAVEn makes connected intelligence a realitymdashwhile preserving customer choice in technologies and protecting existing investments

3 Not just technology but business transformation We have decades of experience helping businesses transform CSPs IT and network operation HAVEn extends that record of success and industry leadership to 100 percent of your meaningful data And our global scale enables us to deliver innovations based on knowledge gained across industries worldwide

4 HP CMS Service offers a broader portfolio of solution services that can help you to navigate your transformation journey HP CMS services portfolio includes CMS telco Big Data workshop Connecting CSPsrsquo business strategies with Big Data implementation roadmap

Predefined telco-specific analytic use case Deploying large set of predefined ready-to-use analytic use cases that can be monetized quickly

CMS architecture services CSP high-skilled architects can help you to easy and with low risk integrated new analytic solution with existing ones with networks with CRM and any other Network systems

HP CMS Big Data and analytic services for CSPs Providing high-skilled consultants who help the CSPs implement analytic use cases to help optimize traditional business and discover new revenue streams

Across the globe enterprise customers rely on HP CMS Services to design deploy operate and support the IT and network systems that run their businesses HP CMS Services capabilities cover consulting and integration outsourcing and support services With more than 3000 services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Machine to machine componentsClick on each icon to read more

Data access

Business intelligence

Presentation and visualization

Data sources

Data collections

Data management and structuring

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Whatrsquos your next move Consider these seven questionsOur customers and partners are rapidly embracing HAVEn for a wide variety of industry-specific uses tailoring the platform to solve their specific Big Data challenges

The best way to get started on the road to addressing your unique data needs is to ask yourself these questions

1 Are you able to process store and index all critical data across your organization structured unstructured and semi-structured

2 Do you have insights into customer behavior sentiment churn and brand loyalty And how easily and quickly can you gain these insights and act on them

3 Do all components of your data management infrastructure work together Or are data and applications siloed with little or no centralized access

4 Are you adequately addressing your business mandates for compliance monitoring and data retention

5 Do you have the Big Data expertise in-house to efficiently handle explosive data growth and the increasing need to deliver meaningful analytics or insights to the business

6 Can you draw connected actionable intelligence from a combination of traditional transactional new ldquonetwork-of-thingsrdquo machine data and unstructured data such as social media and disparate knowledge worker documents and records

7 Can you enable new business model as you are starting to realize that the information you have on your subscribers is an untapped asset Can you choose the potential offered by new revenues stream as the biggest opportunity

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

The HAVEn ecosystemA rich ecosystem extends the HAVEn platform The HAVEn ecosystem combines the platform with a wealth of HP resources partners and integrators across the globe There are three key differentiators that make the HAVEn ecosystem one of the industry leaders in Big Data

1 Vision and breadth Working closely with our global IT partner ecosystem HP delivers the hardware software services infrastructure and business-transformation consulting required for you to achieve a successful Big Data strategy HP is the largest IT company in the world with the most extensive partner network and is the only vendor with leading Big Data software namelymdashHP Autonomy HP Vertica and HP ArcSight

2 Openness HAVEn makes connected intelligence a realitymdashwhile preserving customer choice in technologies and protecting existing investments

3 Not just technology but business transformation We have decades of experience helping businesses transform CSPs IT and network operation HAVEn extends that record of success and industry leadership to 100 percent of your meaningful data And our global scale enables us to deliver innovations based on knowledge gained across industries worldwide

4 HP CMS Service offers a broader portfolio of solution services that can help you to navigate your transformation journey HP CMS services portfolio includes CMS telco Big Data workshop Connecting CSPsrsquo business strategies with Big Data implementation roadmap

Predefined telco-specific analytic use case Deploying large set of predefined ready-to-use analytic use cases that can be monetized quickly

CMS architecture services CSP high-skilled architects can help you to easy and with low risk integrated new analytic solution with existing ones with networks with CRM and any other Network systems

HP CMS Big Data and analytic services for CSPs Providing high-skilled consultants who help the CSPs implement analytic use cases to help optimize traditional business and discover new revenue streams

Across the globe enterprise customers rely on HP CMS Services to design deploy operate and support the IT and network systems that run their businesses HP CMS Services capabilities cover consulting and integration outsourcing and support services With more than 3000 services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

Whatrsquos your next move Consider these seven questionsOur customers and partners are rapidly embracing HAVEn for a wide variety of industry-specific uses tailoring the platform to solve their specific Big Data challenges

The best way to get started on the road to addressing your unique data needs is to ask yourself these questions

1 Are you able to process store and index all critical data across your organization structured unstructured and semi-structured

2 Do you have insights into customer behavior sentiment churn and brand loyalty And how easily and quickly can you gain these insights and act on them

3 Do all components of your data management infrastructure work together Or are data and applications siloed with little or no centralized access

4 Are you adequately addressing your business mandates for compliance monitoring and data retention

5 Do you have the Big Data expertise in-house to efficiently handle explosive data growth and the increasing need to deliver meaningful analytics or insights to the business

6 Can you draw connected actionable intelligence from a combination of traditional transactional new ldquonetwork-of-thingsrdquo machine data and unstructured data such as social media and disparate knowledge worker documents and records

7 Can you enable new business model as you are starting to realize that the information you have on your subscribers is an untapped asset Can you choose the potential offered by new revenues stream as the biggest opportunity

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

The HAVEn ecosystemA rich ecosystem extends the HAVEn platform The HAVEn ecosystem combines the platform with a wealth of HP resources partners and integrators across the globe There are three key differentiators that make the HAVEn ecosystem one of the industry leaders in Big Data

1 Vision and breadth Working closely with our global IT partner ecosystem HP delivers the hardware software services infrastructure and business-transformation consulting required for you to achieve a successful Big Data strategy HP is the largest IT company in the world with the most extensive partner network and is the only vendor with leading Big Data software namelymdashHP Autonomy HP Vertica and HP ArcSight

2 Openness HAVEn makes connected intelligence a realitymdashwhile preserving customer choice in technologies and protecting existing investments

3 Not just technology but business transformation We have decades of experience helping businesses transform CSPs IT and network operation HAVEn extends that record of success and industry leadership to 100 percent of your meaningful data And our global scale enables us to deliver innovations based on knowledge gained across industries worldwide

4 HP CMS Service offers a broader portfolio of solution services that can help you to navigate your transformation journey HP CMS services portfolio includes CMS telco Big Data workshop Connecting CSPsrsquo business strategies with Big Data implementation roadmap

Predefined telco-specific analytic use case Deploying large set of predefined ready-to-use analytic use cases that can be monetized quickly

CMS architecture services CSP high-skilled architects can help you to easy and with low risk integrated new analytic solution with existing ones with networks with CRM and any other Network systems

HP CMS Big Data and analytic services for CSPs Providing high-skilled consultants who help the CSPs implement analytic use cases to help optimize traditional business and discover new revenue streams

Across the globe enterprise customers rely on HP CMS Services to design deploy operate and support the IT and network systems that run their businesses HP CMS Services capabilities cover consulting and integration outsourcing and support services With more than 3000 services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

The HAVEn ecosystemA rich ecosystem extends the HAVEn platform The HAVEn ecosystem combines the platform with a wealth of HP resources partners and integrators across the globe There are three key differentiators that make the HAVEn ecosystem one of the industry leaders in Big Data

1 Vision and breadth Working closely with our global IT partner ecosystem HP delivers the hardware software services infrastructure and business-transformation consulting required for you to achieve a successful Big Data strategy HP is the largest IT company in the world with the most extensive partner network and is the only vendor with leading Big Data software namelymdashHP Autonomy HP Vertica and HP ArcSight

2 Openness HAVEn makes connected intelligence a realitymdashwhile preserving customer choice in technologies and protecting existing investments

3 Not just technology but business transformation We have decades of experience helping businesses transform CSPs IT and network operation HAVEn extends that record of success and industry leadership to 100 percent of your meaningful data And our global scale enables us to deliver innovations based on knowledge gained across industries worldwide

4 HP CMS Service offers a broader portfolio of solution services that can help you to navigate your transformation journey HP CMS services portfolio includes CMS telco Big Data workshop Connecting CSPsrsquo business strategies with Big Data implementation roadmap

Predefined telco-specific analytic use case Deploying large set of predefined ready-to-use analytic use cases that can be monetized quickly

CMS architecture services CSP high-skilled architects can help you to easy and with low risk integrated new analytic solution with existing ones with networks with CRM and any other Network systems

HP CMS Big Data and analytic services for CSPs Providing high-skilled consultants who help the CSPs implement analytic use cases to help optimize traditional business and discover new revenue streams

Across the globe enterprise customers rely on HP CMS Services to design deploy operate and support the IT and network systems that run their businesses HP CMS Services capabilities cover consulting and integration outsourcing and support services With more than 3000 services

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated

Business white paper | Transforming CSPsrsquo Big Data to knowledge and revenue

professionals operating in 170 countries acknowledged technology leadership and a heritage of innovation in services HP Services can point to an extensive track record of helping customers improve their ability to support their changing business needs

HP Software ServicesGet the most from your software investment We know that your support challenges may vary according to the size and business-critical needs of your organization

HP provides technical software support services that address all aspects of your software lifecycle This gives you the flexibility of choosing the appropriate support level to meet your specific IT and business needs Use HP cost-effective software support to free up IT resources so you can focus on other business priorities and innovation

HP Software Support Services gives you

bull One stop for all your software and hardware services saving you time with one call 24x7365 days a year

bull Support for VMware Microsoftreg Red Hat and SUSE Linux as well as HP Insight Software

bull Fast answers giving you technical expertise and remote tools to access fast answersreactive problem resolution and proactive problem prevention

bull Global Reach Consistent Service Experience giving global technical expertise locally

For more information go to hpcomservicessoftwaresupport

Learn more athpcomhaven and hpcomgotelcobigdata

Rate this documentShare with colleagues

copy Copyright 2013 Hewlett-Packard Development Company LP The information contained herein is subject to change without notice The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services Nothing herein should be construed as constituting an additional warranty HP shall not be liable for technical or editorial errors or omissions contained herein

Microsoft and Windows are US registered trademarks of Microsoft Corporation Googletrade is a trademark of Google Inc

4AA4-4914ENW September 2013 Rev 1

Sign up for updates hpcomgogetupdated