Fast data access over cloud technologies€¦ · Fast data access over cloud technologies Red Hat Summit 2015 Boston, June, 24 th. ... Oracle%ODI12c Extractor:%Oracle% GoldenGate11

Fast data access over cloud technologiesRed Hat Summit 2015 Boston, June, 24th

Copyright © 2015 Accenture All rights reserved.

This document and the information contained herein are proprietary to Accenture. This document, either in whole or in part, may not be reproduced in any form or by any means without Accenture’s prior written permission. This document provides no warranties with respect to the contents described within. Any third-‐party names, trademarks and copyrights contained in this document are the property of their respective owners.

Disclaimer

2


▪ Digital Business Goals

▪ Customer context and major needs

▪ Proposed solution

▪ Architecture framework

▪ Technology stack

▪ Achievements

▪ Project timeline

▪ Roadmap and possible extensions

▪ Collaboration value

Agenda

3


Connected generation of customers raise the competition bar for service satisfaction and excellence

▪ Smartphone penetration constantly grows & customers become more diverse

▪ Mobile Apps users are Digital Natives as well as Digital Immigrants: 45% of those over 50 years old use mobile internet

▪ Users spend over 30 hours per month any time of the day with their Smart handy devices

▪ Even in the Italian context self-‐caring mobile app usage for a telco player is above tenths of thousands sessions per hour, night and day

A new paradigm for the Telco Italian market

4


Innovative, digital business enabling platforms must be supported by elastic infrastructure providing highest versatility and SLAs

Needs for an architecture with: ▪ 100% availability / zero downtime ▪ High performance (throughput, response time) ▪ Flexibility ▪ Stop investment on legacy systems ▪ Decouple (as much as possible)

channels from legacy operational systems ▪ Horizontal, seamless scalability ▪ Commodity off the shelf infrastructure /

Private Cloud ▪ Streamlined Service Operations (platform,

customer data flows, end user view)

Customer context and major needs

5

Systems of Record(CRM, Billing, OrderManagement, …)

Systems of Differentiation

Systems of Innovation(Mobile apps, web portals, …)

Fast moving 0-‐3 years Web, mobile, external APIs, M2M, cognitive technologies etc.

Services layer 3-‐10 years Web, mobile, APIs, M2M, cognitive technologies etc.

Stable core 10+ years Package SW, core custom systems, data warehouses, functional batch etc.

High

Low

Digital Business


Accenture and Redhat helped a major worldwide mobile and wireline telco operator to design and realize a new decoupling platform, leading on premise cloud technologies

Proposed Solution

6

...decoupling information retrieval from customer transactions

▪ New architecture to serve large amounts of diverse business data at high speed▪ An enterprise platform, leveraging private cloud technologies

• Always up & running • Low cost horizontal scalability

▪ The best way to serve the growing demand from channels in a heavy legacy environment…

To

Back EndSystems

Information Retrieval

Transactions Back EndSystems

Transactions

New elasticplatformInformation Retrieval

From

Copyright © 2015 Accenture All rights reserved. 7

Architecture frameworkOpen standards based, high redundant streamlined platform, seamless linkable into the BSS ecosystem of the client with zero impacts on surrounding systems

On prem

ise, scalable

off the sh

elf clou

d infrastructure

Co

re platfo

rm

Elastic scaling application layer

Fast Data Access Memory Grid

Real time business Data Processing

Business Services

Data Extraction Layer

Staging Layer

Fast Data Access Layer

Operational Data Layer

Channel Service Layer

Business Services Business Services

Data Source 1 Data Source 1 Data Source 1

PaaS deployment Centralized logging and monitoringIntegrated configuration management Hot release management

Read optimized common business data model Hybrid model

Continuous near real-‐time data replication Reconciliation Data models unification


s

Service Gateway Components

Data Access Manager

Other External Service Proxy

Leading technologies integrated together in order to match requirements on all layersTechnology stack

8

Staging Layer Staging Data Base (DB)

Channel Service Layer

Operational Layer

Data Extraction Layer

Extract Transform Load (ETL )

Data Replicas

Backend Legacy SourceSystems Layer

Data Source 2

Data Source 2

This is the access point for Client Applications. It exposes interfaces for both internal and external clients.

DAM exposes reusable, fast and reliable data access methods. Proxy module directly interfaces network systems in order to retrieve rela time values.Application logs are collected centrally on a scalable MongoDb cluster

Fast Data Access Layer In Memory Data Grid IMDG is a data structure that resides entirely in RAM. Distributed among

multiple servers. IMDG is suited to retrieve data with velocity and high volumes.

RDBMS database used to prepare data to be loaded on IMDG.Can also be used for certain data sets or in yellow / red mode

ETL extracts data (via Extractor) from Data Sources replica to create de-‐normalized view of the information accessible by the channels providing Common Data Model

Primary BSS platforms where the data needed for front-‐end channels reside.

IAAS / PAAS

PaaS: Read Hat Open Shift 2.1

Transformation: Oracle ODI 12c

Extractor: Oracle Golden Gate 11

MongoDB 2.6 IaaS: VMWare VSphere 5.0

IM Data Grid Pivotal Gemfire 7.02

Description Technologies

Oracle RDBMS RAC 11g


Platform core capabilities

9

Multi-Mode Red/Yellow

/Green

Each service for each client channel may run in different ways, depending on various working conditions of internal components or integrated data sources. Client channels get a clear message in every condition.

Caching

ThrottlingMonitor and limit client channels throughput, requests/transactions to/from external systems, use queues to decouple and manage throughput gap between clients requests and external source systems.

CommonData Model

Data collection and processing stages realize a clean, flat, unified business data model, de-coupled from involved source systems and flexible in order to accommodate client channel needs

Each service accessing data from external systems may use advanced caching capabilities. Caching kicks in to overcome external data source unavailability, pace and lower server-side workload and increase services performance.

Provides a centralized log collection platform, realized on a non relational databases in order to better scale versus high transaction volumes. Log can be easily accessed by both command line and web consoles for troubleshooting and analytics.

Software updates can be executed without any service unavailability, thanks to both PaaS native capabilities (i.e. continuous deployment and rolling reboots of application components) and specific architecture design (i.e. data redundancies, yellow mode for temporary data access, etc.)

A unified web console allows administrators to monitor all services operational statuses, alarms, notifications, etc. It allows also for channel configuration, throttling management, green/yellow/red mode switching

The platform is able to guarantee defined SLAs thanks to its capability of providing a dynamical scalability of the computation engine due to the number of interrogations. Guaranteed SLAs can be different for each channel.

Log Management

Configuration Management

SLA Management

HotDeployment


▪ Service Access • > 30Million user sessions per month

• Deployed for up to 2.000 interactions per seconds, hundreds of concurrent client sessions

▪ Platform Core Engine • 24x7x365: Always up & running

• More than 100 hundred standard x86 virtual machines involved

• > 1TByte of In memory data grid total RAM space

▪ Legacy Systems integration • More than 1.1Bln record retrieved from source systems

• 30GB/hour of update throughput

Some achievements

10


Project timeline

11

Month1 Month9 Month3 Month4 Month5 Month6 Month7 Month8 Month9

1. Preliminaryanalysis and feasibility •Requirement collections

•Technology scouting•High level solution definition

• Software selection, vendor PoC, …

Month10 Month11 Month12 Month13 Month14

2. Project first phaseDetailed design Build

TestPerformance Test Deploy

Completefunctional PoC Platform ready in

production environment

GO Live serving 1st mobile App

Second group of client apps served

Third group of client apps served


▪ Enable new functionalities, leveraging the common data model • Real time analytics

• Push services/notification

▪ Complete «multi-‐tenancy» capability of the whole platform in order to host different application context on the same scalable platform

▪ Leverage the current platform Service Exposure Interface in order to start an holistic API strategy program

• Per user AAA functionalities; improved security

• Service composition/orchestration

• Paradigm shift in the whole software lifecycle for new business services

Further extensions

12


▪ Result based approach: availability and performance KPI driven

▪ “End to End” accountability, shaping to delivery and platform operation.

▪ Deep knowledge of existing BSS systems and related processes

▪ Aggressive timeline introduction (6 month from “ok to go” to the first production GO LIVE)

▪ Leveraging of main Open Source project and cloud related initiatives

Customer, Accenture and RedHat collaboration value

13


Giovanni De [email protected]

#GionDeMa

Thank you very much for your time!

14

mailto:[email protected]


▪ PaaS Deployment schema on OpenShift Enterprise

▪ MongoDB logging schema

Backup Slides

15


PaaS Deployment schema on OpenShift Enterprise

16

Virtual Machine …

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

……

Virtual Machine 2

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

2.n…

Red Hat Open Shift general concepts and entities

Application …

Module X

Module Y

Application B

Module X

Module Y

Application A

Module X

Module Y

Virtual Machine 1

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

1.n…

Services

Logical/softwareelements

Physicaldeployment

Virtual Machine …

G e ar

1. 1G e ar

1. 2G e ar

1. 3G e ar

……

Virtual Machine …

G e ar

1. 1G e ar

1. 2G e ar

1. 3G e ar

2. n…

Virtual Machine …

G e ar

…G e ar

…G e ar

… G e ar

……

App1 for Client2…

Module X

Module Y

App2 for Client1

Network Proxy

getSimInfo()

getCustomerInfo()

getXYZ(..)

App1 for Client1

Proxy

Data Access Module

Virtual Machine …

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

……

Virtual Machine 2

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

2.n…

Virtual Machine 1

Ge ar

1.1

Ge ar

1.2

Ge ar

1.3

Ge ar

1.n…

Our platform PaaS deployment

Datacenter A

Datacenter B


MongoDB shard labelling to reduce single point of failuresCentralized log schema

17

vm001

Gear AAA

Gear AAB

Gear AAC

Gear AAD

vm003

Gear AAE

Gear AAF

Gear BBA

Gear AAG

vm005

Gear CCA

Gear CCB

Gear CCC

Gear CCD

vm007

Gear CCE

Gear CCF

Gear CCG

Gear CCH

vm009

Gear CCI

Gear CCL

Gear CCM

Gear CCN

vm002

Gear AAH

Gear AAI

Gear AAL

Gear AAM

vm004

Gear AAN

Gear AAO

Gear BBB

Gear AAP

vm006

Gear CCO

Gear CCP

Gear CCQ

Gear CCR

vm008

Gear CCS

Gear CCT

Gear CCU

Gear CCV

vm010

Gear CCZ

Gear CCW

Gear CCJ

Gear CCK

Channel1 Chanel2 Channel3

LogNode1 shard000 {“vm001”} {“vm003”}

LogNode3 shard001{“vm005”} {“vm007”}

LogNode5 shard002{“vm009”}

LogNode2 shard003{“vm002”} {“vm004”}

LogNode4 Shard004{“vm006”} {“vm008”} {“vm010”}

Channel1_Log Channel2_Log Channel3_Log Channel3_Log Channel2_Log Channel3_LogChannel1_Log

Documents

Fast data access over cloud technologies€¦ · Fast data access over cloud technologies Red Hat Summit 2015 Boston, June, 24 th. ... Oracle%ODI12c Extractor:%Oracle% GoldenGate11