Kiwibank: From Startup to Enterprise in 7 years

Kiwibank: Startup to Enterprise in 7 years

Wayne Knowles Senior Infrastructure Architect

Make change work for youIBM Insight Forum 09®

Ki ib k t St t2

Kiwibank at Startup

Established by Labour-Alliance Coalition GovernmentLow Startup BudgetLow Startup Budget Leveraged NZ Post for Infrastructure

Branch Network using existing Postshop’s and franchiseBranch Network using existing Postshop s and franchise networkLeverages ISP ServicesRe-use existing Vendor Relationships


Ki ib k t St t 20023

Kiwibank at Startup - 2002

IBM p-Series for “Core” banking database

IBM x-Series machines running Microsoft ApplicationsFile & Printer servicesE h M ilExchange Mail serverCitrixSeveral SQL Server databases

Entry Level IBM SANFil & P i t l SQL S D t b (< 1TB)File & Print plus SQL Sever Databases (< 1TB)


R d T S4

Road To Success

2002 2003-2008 2009Launch

• Project team

2003-2008

• Strong growth

2009

• Success!j

• No customers

g g

• Acquisitions • Many customers

• Guess requirements

Not sure what we

• Brand developed

• Building market knowledge

• Many products

• Know what customers expect• Not sure what we

would become!knowledge expect


Ki ib k G th5

Kiwibank Growth


U d t d Th A li ti6

Understand The Application

Understand how CPU, memory and storage is impacted by growth -

Linear: resource_usage = n x customersExponential: resource_usage = n x customers2

Silicon is faster than Disk (Ferrite):Keep “working set” in memory for 24 hoursExpect high I/O and performance impact after a rebootEnsure backups don’t displace working set from memory


U d t d Th A li ti7

Understand The Application

Eliminate the high cost operations:Disk I/O ReadsDisk I/O ReadsProcess startups (forks)

Best way to find bottlenecks: perform full load testCommercial tools available (Tivoli Loadrunner etc)Commercial tools available (Tivoli, Loadrunner etc)Use Standard OS Tools (truss, sar and perfmon)

Especially when sharing performance data with external vendorsEspecially when sharing performance data with external vendors

Optimize for the common casep


U d t d St G th8

Understand Storage Growth

Data can be classified into 2 categories:StructuredStructuredUnstructured

Most organizations recognize that unstructured dataMost organizations recognize that unstructured data growth is out of control.

Multiple copies of the structured data can also result in a large storage requirement.


St G th9

Storage Growth

UAT

UAT

PROD DR UAT

UAT

Stage MIS ODS

Consider the storage requirements of multiple test genvironments!

A hi i d D D h l thi thArchiving and De-Dup can help manage this growth


S t C it M t10

Smart Capacity Management

“Radar Screen” forcastingOne should understand how the application growsOne should understand how the application grows

Track the upper limitTrack the upper limitIe. xx Months before next impactWhen highlighting issues to management use correctWhen highlighting issues to management use correct terminology

Understand Vendor Lead-TimesPut simply: This is ITIL Capacity Planning


B ildi M bl I f t t11

Building Manageable Infrastructure

Engineer upfront for cater for future upgradesCPU or Memory upgrades are easyCPU or Memory upgrades are easyCapacity on Demand

Consider future Forklift upgradesPut the Customer first and minimize downtime. Smart choice of technology to minimize downtimeKiwibank has performed 2 major forklift upgrades with minimum downtime (< 4 hours)


Bi B U d12

Big Bang Upgrades

Triggered by Batch Window timeTriggered by Batch Window timeMigrated to new machine using mksysb migration (AIX)Implemented in p-Series LPARImplemented in p Series LPARResources allocated to allow future AIX upgradesOutsourced the solution – implemented in 8 weeksOutsourced the solution implemented in 8 weeksImplemented with less than 4 hours outage


R li bilit13

Reliability

Measure and improve reliabilityFull analysis of all problemsu a a ys s o a p ob e sConcentrate on repeat problems first

More Fingers = More ProblemsReduce amount of touching on the system to a minimumg yAutomated monitoring and reporting to eliminate daily checks

Think Smart: Automate!


Wi k d P bl14

Wicked Problems

"Wicked problem" is a phrase used in social planning to describe a problem that is difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Moreover because of complex interdependencies theMoreover, because of complex interdependencies, the effort to solve one aspect of a wicked problem may reveal or create other problems.p

Same can be said for most large IT systems…….g y


F t P f I f t t15

Future Proof Infrastructure

Thin ProvisioningVirtualization (VMWare, LPAR etc.)Virtualization (VMWare, LPAR etc.)De-dupRedundancyRedundancy In place scale-out and scale upAim for zero outage upgrades & changesAim for zero outage upgrades & changes


B ildi M bl A li ti16

Building Manageable Applications

Use established industry standards, where possibleSOA Architecture & XML MessagesSOA Architecture & XML MessagesUse Message Queuing Implement solutions that are Horizontal & VerticallyImplement solutions that are Horizontal & Vertically scalableBreak into small manageable componentsg pPackaging and Deployment are important


B i C ti it17

Business Continuity

Consider sharing with pre-production environmentDR hardware can used as a pre-production environment a d a e ca used as a p e p oduc o e o efor testing and production support.Visualize with VMWare / LPAR

Implement replication as high up the stack as possibleIe. Oracle Data-guard, SQL Server Replication

Can be implemented (or tested) in stages with minimum disruption to operations


BCP / DR Cl i A h18

PRODUCTION

BCP / DR – Classic Approach

MiddlewareInternet Banking

Core BankingDatabase

EFTPOS & ATM

Database

DR

MiddlewareInternet BankingBanking


EFTPOS & ATMATM


BCP / DR Cl i A h19

PRODUCTION

BCP / DR – Classic Approach



EFTPOS & ATM

Database

DR



EFTPOS & ATMATM


BCP / DR Fi G i d20

BCP / DR Fine Grained

Kiwibank uses a “Fine Grained” DR strategyAny system can be separately DR’edy y p ySeparate DR procedure for each component

Kiwibank has future plans to use active-active where iblpossible.


BCP / DR Fi G i d21

PRODUCTION

BCP / DR – Fine Grained



EFTPOS & ATM

Database

DR



EFTPOS & ATMATM


BCP / DR Fi G i d22

PRODUCTION




EFTPOS & ATM

Database

DR



EFTPOS & ATMATM


BCP / DR Fi G i d23

PRODUCTION




EFTPOS & ATM

Database

DR



EFTPOS & ATMATM


Ki ib k C t & F t24

Kiwibank Current & Future

Has completed several upgrades due to customer growth

Agile and able to react to change

Significant investment in IBM x-Series and p-Series hardware

Currently “Enterprising up” its infrastructure


S25

Summary

Understand your applications

Think Smarter, not Harder

Maximize the use of your assets

Adopt new technology and use it to grow your business

Minimize outages and Customer Impact


26

Th kThank you

Questions?Questions?

IBM Insight Forum 09®

Make change work for you