14
7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 1/14 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth How big data analytics impose huge challenges for storage professionals and the keys for preparing for the future David Vellante, David Floyer Analysis from The Wikibon Project May 2012 A Wikibon Reprint

A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

Embed Size (px)

Citation preview

Page 1: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 1/14

A Special Report on Infrastructure Futures:Keeping Pace in the Era of Big Data Growth

How big data analytics impose huge challenges for storageprofessionals and the keys for preparing for the future

David Vellante, David Floyer

Analysis from The Wikibon Project May 2012

A Wikibon Reprint

Page 2: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 2/14

Wikibon.org 1 of 13

View the live research note on Wikibon. 

The cumulative effect of decades of IT infrastructure investment around a diverseset of technologies and processes has stifled innovation at organizations around theglobe. Layer upon layer of complexity to accommodate a staggering array of applications has created hardened processes that make changes to systems difficultand cumbersome.

The result has been an escalation of labor costs over the years to support thiscomplexity. Ironically, computers are supposed to automate manual tasks, but thestatistics show some alarming data that flies in the face of this industry promise. Inparticular, the percent of spending for both internal and outsourced IT staff hasexploded over the past 15 years. According to Wikibon estimates, of the $250Bspent on server-and storage-related hardware and staffing costs last year, nearly60% was spent on labor. IDC figures provide further evidence of this trend. Theresearch firm’s forecasts are even more aggressive than Wikibon’s, with estimatesthat suggest labor costs will approach 70% by 2013 (see Figure 1 below).

The situation is untenable for most IT organizations and is compounded by theexplosion of data. Marketers often cite Gartner’s three V’s of Big Data —volume,velocity, and variety — that refer respectively to data growth, the speed at whichorganizations are ingesting data, and the diversity in data texture (e.g. structured,unstructured, video, etc). There is a fourth V that is often overlooked: Value. 

Wik iT re n d : By 2 0 1 5 , t h e m a jo r i t y o f I T o rg a n i za t i o n s w i l l co me to t h e  

rea l i zat ion t ha t b ig da t a ana ly t i cs i s t ipp ing t he sca les and m ak ing  i n f o rm a t i o n a sou rce o f com p e t i t i ve va lu e t h a t ca n b e mo n e t i ze d a n d n o t  

 j u st a l iab i l i t y t h at n eed s t o b e m an ag ed . Th ose o r g an izat ion s w h ich canno t cap i ta l i ze on da ta as an oppor t un i t y , r i sk los ing mar ke tshare .

From an infrastructure standpoint, Wikibon sees five keys to achieving this vision:

▪  Simplifying IT infrastructure through tighter integration across the hardwarestack;

▪  Creating end-to-end virtualization beyond servers into networks, storage, andapplications;

▪  Exploiting flash and managing a changing hardware stack by intelligentlymatching data and media characteristics;

▪  Containing data growth by making storage optimization a fundamental capability

of the system;▪  Developing a service orientation by automating business and IT processes

through infrastructure that can support applications across the portfolio,versus within a silo, and provide infrastructure-as-a-service that is

 “application aware.” 

This research note is the latest in a series of efforts to aggregate the experiences of users within the Wikibon community and put forth a vision for the future of infrastructure management.

Page 3: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 3/14

Wikibon.org 2 of 13

The I T Labor Prob lem  The trend toward IT consumerization, led by Web giants servicing millions of users,often with a single or very few applications, has ushered in a new sense of urgencyfor IT organizations. C-level and business line executives have far betterexperiences with Web apps from Google, Facebook, and Zynga than with their

internal IT systems as these services have become the poster children of simplicity,rapid change, speed, and a great user experience.

In an effort to simplify IT and reduce costs, traditional IT organizations haveaggressively adopted server virtualization and built private clouds. Yet relative tothe Web leaders, most IT organizations are still far behind the Internet innovators.The reasons are quite obvious as large Web properties had the luxury of startingwith a clean sheet of paper and have installed highly homogeneous infrastructurebuilt for scale.

Both vendor and user communities are fond of citing statistics that 70% of ITspending is allocated to “Running the Business”, while only 30% goes towardgrowth and innovation. Why is this? The answer can be found by observing IT labor

costs over time.

Data derived from researcher IDC (see Figure 1) shows that in 1996, around $30Bwas spent on IT infrastructure labor costs, which at the time represented onlyabout 30% of total infrastructure costs. By next year, the data says that more than$170B will be spent on managing infrastructure (i.e. labor), which will account fornearly 70% of the total infrastructure costs (including capex and opex). This is awhopping 6X increase in labor costs, while overall spending has only increased 2.5Xin those 15+ years.

Figure 1 – IT Labor Cost Over TimeData Source: IDC 2012

Page 4: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 4/14

Wikibon.org 3 of 13

What does this data tell us? It says we live in a labor-intensive IT economy andsomething has to change. The reality is IT investments primarily go toward laborand this labor-intensity is slowing down innovation. This trend is a primary reasonthat IT is not keeping pace with business today — it simply doesn’t have theeconomic model to respond quickly at scale. In order for customers to go in newdirections and break this gridlock, vendors must address the REAL cost of computing, people.

The answer is one part technology, one part people, and one part process.Virtualization/cloud is the dominant technology trend, and we live in a world whereIT infrastructure and applications, and the security that protects data sources, areviewed as virtual, not physical entities. The other three dominant technologythemes reported by Wikibon community practitioners are:

1.  A move toward pre-engineered and integrated systems (aka convergedinfrastructure) that eliminate or at least reduce mundane tasks such as patchmanagement;

2.  Much more aggressive adoption of virtualization beyond servers;

3.  A flash-oriented storage hierarchy that exploits automated operations and areduction in the manual movement of data — i.e. “smarter systems” that areboth automated and application aware — meaning infrastructure can supportapplications across the portfolio and adjust based on quality of servicerequirements and policy;

4.  Products that are inherently efficient and make data reduction features likecompression and de-duplication fundamental capabilities, not optional add-ons, along with new media such as flash and the ability to automatemanagement of the storage infrastructure.

From a people standpoint, organizations are updating skills and training people inemerging disciplines including data science, devops (the intersection of applicationdevelopment and infrastructure operations), and other emerging fields that willenable the monetization of data and deliver hyper increases in productivity.

The goal is that the combination of improved technologies and people skills will leadto new processes that begin to reshape decades of complexity and deliver a muchmore streamlined set of services that are cloud-like and services-oriented.

The hard reality is that this is a difficult task for most organizations, and anintelligent mix of internal innovation with external sourcing will be required to meetthese objectives and close the gap with the Web giants and emerging cloud service

providers.

New M odel s o f I n f r as t r uc tu r e Managem en t  IT infrastructure management is changing to keep pace as new models challengeexisting management practices. Traditional approaches use purpose-builtconfigurations that meet specific application performance, resilience, and space

Page 5: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 5/14

Wikibon.org 4 of 13

requirements. These are proving wasteful, as infrastructure is often over-provisioned and underutilized.

The transformative model is to build flexible, self-administered services fromindustry-standard components that can be shared and deployed on an as-neededbasis, with usage levels adjusted up or down according to business need. These IT

services building blocks can come as services from public cloud and SaaS providers,as services provided by the IT department (private clouds), or increasingly ashybrids between private and public infrastructure.

Efforts by most IT organizations to self-assemble this infrastructure have led to arepeat of current problems, namely that the specification and maintenance of allthe parts requires significant staff overhead to build and service the infrastructure.Increasingly, vendors are providing a complete stack of components, includingcompute, storage, networking, operating system, and infrastructure managementsoftware.

Creating and maintaining such a stack is not a trivial task. It will not be sufficientfor vendors or systems integrators to create a marketing or sales bundle of component parts and then hand over the maintenance to the IT department; thesavings from such a model are minimal over traditional approaches. The stack mustbe completely integrated, tested, and maintained by the supplier as a single SKU,or as a well-documented solution with codified best practices that can be applied forvirtually any application. The resultant stack has to be simple enough that a singleIT group can completely manage the system and resolve virtually any issue on itsown.

Equally important, the cost of the stack must be reasonable and must scale outefficiently. Service providers are effectively using open-source software and focusedspecialist skills to decrease the cost of their services. Internal IT will not be able tocompete with services providers if their software costs are out of line.

The risk to this integrated approach according to members of the Wikibonpractitioner community is lock-in. Buyers are concerned that sellers will, over time,gain pricing power and return to the days of mainframe-like economics. Thisconcern has merit. Sellers of converged systems today are providing largeincentives to buyers in the form of aggressive pricing and white glove service in aneffort to maintain account control and essentially lock customers into their specificoffering. The best advice is as follows:

▪  Consider converged infrastructure in situations where cloud-like services provideclear strategic advantage, and the value offsets the risk of lock-in down the

road.

▪  Design processes so that data doesn’t become siloed. In other words, make sureyour data can be migrated easily to other infrastructure.

▪  Don’t sole source. Many providers of integrated infrastructure have realized theymust provide choice of various components such as hypervisor, network, andserver. Keep your options open with a dual-sourcing strategy.

Page 6: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 6/14

Wikibon.org 5 of 13

Wik iT re n d : Desp i t e t h e r i s k o f l o ck - i n , b y 2 0 1 7 , m o re t h a n 6 0 % in f ra s t ru c tu re w i l l b e p u rch a se d a s so me t yp e o f i n t e g ra te d syste m , e it h e r  

as a sing le SKU o r a p re - t es ted re fe rence a rch i tec tu r e . 

The goal of installing integrated or converged infrastructure is to deliver a worldwithout stovepipes, where hardware and software can support applications acrossthe portfolio. The tradeoff of this strategy is it lessens the benefits of tailor-madeinfrastructure that exactly meets the needs of an application. For the fewapplications that are critical to revenue generation, this will continue to be a viablemodel. However, Wikibon users indicate that 90% or more of the applications donot need a purpose-built approach, and Wikibon has used financial models todetermine that a converged infrastructure environment will cut the operationalcosts by more than 50%.

Figure 2 – Traditional Stove-piped Infrastructure ModelSource: Wikibon 2012

The key to exploiting this model is tackling the 90% long tail of applications byaggregating common technology building blocks into a converged infrastructure.There are two major objectives in taking this approach:

1.  Drive down opera t ion a l cos ts by using an integrated stack of hardware,operating systems, and middleware;

2.  Accelerate the deployment of applications.

Page 7: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 7/14

Wikibon.org 6 of 13

Figure 3 – Infrastructure 2.0 Services Model

Source: Wikibon 2012

Vi r tua l i za t ion : Mov ing Beyond Serve rsVolume servers that came from the consumer space only had the capability of running one application per server. The result was servers that had very lowutilization rates, usually well below 10%. Specialized servers that can run multipleapplications can achieve higher utilization rates but at much higher system and

software costs.Hypervisors, such as VMware, Microsoft’s Hyper V, Xen and hypervisors from IBMand Oracle, have changed the equation. The hypervisors virtualize the systemresources and allow them to be shared among multiple operating systems. Eachoperating system thinks that it has control of a complete hardware system, but thehypervisor is sharing those resources among them.

The result of this innovation is that volume servers can be driven to much higherutilization levels, thee-to-four times that of stand-alone systems. This makes low-cost volume servers that are derived directly from volume consumer products suchas PCs much more attractive as a foundation for processing and much cheaper thanspecialized servers and mainframes. There will still be a place for very high-performance specialized servers for some applications such as certain performance-critical databases, but the volume will be much lower.

The impact of server virtualization on storage is profound. The I/O path to a serverprovides service to many different operating systems and applications. The result isthat the access patterns as seen by the storage devices are much less predictableand more random. The impact of higher server utilization (and of multi-coreprocessors) is that IO volumes (IOPS, IOs per second) will be much higher.

Page 8: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 8/14

Wikibon.org 7 of 13

Increasingly, few processor cycles will be available for housekeeping activities suchas backup.

Server virtualization is changing the way that storage is allocated, monitored, andmanaged. Instead of defining LUNs and RAID levels, virtual systems are definingvirtual disks and expect array information to reflect these virtual machines and

virtual disks and the applications they are running. Storage virtualization enginesare enabling the pooling of multiple heterogeneous arrays, providing bothinvestment protection and flexibility for IT organizations with diverse asset bases.As well, virtualizing the storage layer dramatically simplifies storage provisioningand management, much in the same way that server virtualization attacked theproblem of underutilized assets.

Conclus ion s for Stor age: Storage arrays will have to serve much higher volumesof random read and write IOs with applications using multiple protocols. Inaddition, storage arrays will need to work across heterogeneous assets andvirtualized systems and speak the language of virtualized administrators. Newerstorage controllers (often implemented as virtual machines) are evolving that will

completely hide the complexities of traditional storage (e.g., the LUNS and RAIDstructures) and be replaced with automated storage that is a virtual machine (VM)focused on providing the metrics that will enable virtual machine operators (e.g.,VMware administrators) to monitor the performance, resource utilization, andservice level agreement (SLA) at a business application level.

Storage networks will have to adapt to providing shared a transport for thedifferent protocols. Adaptors and switches will increasingly use lossless Ethernet asthe transport mechanism, with different protocols running underneath.

Backup processes will need to be re-architected and linked to the application versusa one-size-fits-all approach. Application consistent snaps and continuous backupprocesses are some of the technologies that will become increasingly importantover time.

Wik iT re n d :   V i r t u a l i za t i o n i s m o v in g b e yo n d j u s t se rve rs   a n d w i ll i m p a ct t h e  

e n t i r e i n f ra s t ru c tu re s t a ck , f r o m s to ra g e, b a cku p , n e tw o rks , i n f ra st ru c tu re  m anagement , and secur i t y . Overa l l , th e s t ron g t rend t ow ards a converged  

i n f ra s t ru c tu re , w h e re s to ra g e f u n ct i o n p la ce me n t i s mo r e d yn a mic , b ein g  s ta g ed o p t im a l ly i n a r ra ys , in v i r t u a l ma ch in es o r i n se rve rs w i l l  n e ce ss i t a t e a n d e n d - t o -e n d a n d m o re i n t e l l i g en t m a n a g em e n t p a ra d ig m.  

Flash Sto r age : I m p l i cat ion s to th e Stack  

Consumers are happy to pay premiums for flash memory over the price of diskbecause of the convenience of flash. For example, the early iPods had disk drivesbut were replaced by flash because the device required very little battery powerand had no moving parts. The results were much smaller iPods that would work fordays without recharging and would work after being dropped. This led to hugeconsumer volume shipments and flash storage costs dropped dramatically.

In the data center, systems and operating system architectures have had tocontend with the volatility of processors and high-speed RAM storage. If power was

Page 9: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 9/14

Wikibon.org 8 of 13

lost to the system, all data in flight was lost. The solutions were either to protectthe processors and RAM with complicated and expensive battery backup systems orto write the data out to disk storage, which is non-volatile. The difference betweenthe speed of disk drives (measured in milliseconds, 10-3) and processor speed(measured in nanoseconds, 10-9) is huge and is a major constraint on systemspeed. All systems wait for I/O at the same speed. This is especially true fordatabase systems.

Flash storage is much faster than disk drives (microseconds, 10-6) and is persistent– when the power is removed the data is not lost. It can provide an additionalmemory level between disk drives and RAM. The impact of flash memory is alsobeing seen in the iPad effect. The iPad is always on, and the response time forapplications compared with traditional PC systems in amazing. Applications arebeing rewritten to take advantage of this capability, and operating systems arebeing changed to take advantage of this additional layer. iPads and similar devicesare forecast to have a major impact on portable PCs, and the technology transferwill have a major impact within the data center, both at the infrastructure level andin the design of all software.

I O Cent r ic Process ing: B ig Data Goes Real - t im eWikibon has written extensively about the potential of flash to disrupt industries and designing systems and infrastructure in the Big Data IO Centric era. The modeldeveloped by Wikibon is shown in Figure 4.

Page 10: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 10/14

Wikibon.org 9 of 13

Figure 4 – Real-time Big Data Processing with IO Centric StorageSource: Wikibon 2012 

The key to this capability is the ability to directly address the flash storage from theprocessor with lockable atomic writes, as explained in a previous Wikibon discussionon designing systems and infrastructure in the Big Data IO Centric era. Thistechnology has brought down the cost of IO intensive systems by two orders ormagnitude, 100 times, whereas the cost of hard disk-only solutions has remainedconstant. This trend will continue.

This technology removes the constraints of disk storage and allows the real-timeparallel ingest of transactional, operational and social media data streams, andsufficient IO at low-enough cost that allows parallel processing of Big Datatransactional systems at the same time performing Big Data indexing and metadataprocessing to drive Big Data Analytics.

Wik iTrend : Flash w i l l enab le changes in sys tem and app l i cat ion d es ign tha t  

a re p ro fou nd . Transac t iona l sys tems w i l l evo lve , as f lash a rch i tec tu r es w i l l  r e m o ve l o ck in g co n s t ra in t s a t t h e h ig h e s t p e r f o rm a n ce t i e r . B ig Data  

a n a ly t i c s w i l l b e i n t e g ra te d w i t h o p e ra t i o n a l syste m s a n d Big Da ta st r e am s  

w i l l becom e d i rec t inpu ts t o app l i ca t ions peop le , dev ices and m ach ines.Me ta d ata e x t r a ct i o n , i n d ex d a ta a n d o th e r su m m a ry d a ta w i l l b ecom e  

d i rec t inpu ts to opera t ion a l Big Data s t r eams and enab le m ore va lue to be  d e r i ve d a t l o w e r co st s f ro m a rch i val a n d b a cku p syste m s .

Page 11: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 11/14

Wikibon.org 10 of 13

Conclus ion s for Stor age: Flash will become a ubiquitous technology that will beused in processors as an additional memory level, in storage arrays as read/write

 “Flash cache”, and as a high-speed disk device. Systems management software willfocus high I/O “hot-spots” and low latency I/O on flash technology and allow high-density disk drives to store the less active data.

Overall within the data center, flash storage will pull storage closer to theprocessor. Because of the heat density constraints mentioned above, it is mucheasier to put low power flash memory rather than disk drives very close to theprocessor.

The result of more storage being closer to the processor will be for some storagefunctionality to move away from storage arrays and filers and closer to theprocessor, a trend that is made easier by multi-core processors that have cycles tospare. The challenge for storage management will be to provide the ability to sharea much more distributed storage resource between processors. Future storagemanagement will have to contend with sharing storage that is within servers as wellas traditional SANs and filers outside servers.

St orag e Ef f ic iency Techno log iesStorage efficiency is the ability to reduce the amount of physical data on the diskdrives required to store the logical copies of the data as seen by the file systems.Many of the technologies have become or are becoming mainstream capabilities.Key technologies include:

▪  Storage virtualization:

Storage virtualization allows volumes to be logically broken intosmaller pieces and mapped onto physical storage. This allows muchgreater efficiency in storing data, which previously had to be storedcontiguously. This technology also allows dynamic migration of datawithin arrays that can also be used for dynamic tiering systems.Sophisticated tiering systems, which allow small chunks of data (sub-LUN) to be migrated to the best place in the storage hierarchy, havebecome a standard feature in most arrays.

▪  Thin provisioning:

Thin provisioning is the ability to provision storage dynamically from apool of storage that is shared between volumes. This capability hasbeen extended to include techniques for detecting zeros (blanks) in filesystems and using no physical space to store them. This again has

become a standard feature expected in storage arrays.▪  Snapshot technologies:

Space-efficient snapshot technologies can be used to store just thechanged blocks and therefore reduce the space required for copies.This provides the foundation of a new way of backing up systemsusing periodic space-efficient snapshots and replicating these copiesremotely.

Page 12: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 12/14

Wikibon.org 11 of 13

▪  Data de-duplication:

Data de-duplication was initially introduced for backup systems, wheremany copies of the same or nearly the same data were being storedfor recovery purposes. This technology is now extending to inlineproduction data, and is set to become a standard feature on storage

controllers.

▪  Data compression:

Originally data compression was an offline process used to reduce thedata held. Data compression is used in almost all tape systems, is nowbeing extended to online production disk storage systems, and is setto become a standard feature in many storage controllers. Thestandard compression algorithms used are based on LZ (Lempel andZiv), and give a compression ratio between 2:1 and 3:1. Compressionis not effective on files that have compression built-in (e.g., JPEGimage files, most audio visual files). The trend is toward real timecompression where performance is not compromised.

Wik iTrend : Sto r age ef f i c iency techno log ies w i l l have a s ign i f i can t im pac t  o n t h e a mo u n t o f s t o ra g e sa ve d. Ho w e ver , t h e y w i l l n o t a f f e ct t h e n u m b e r  

o f I / Os an d t h e b a n d w id t h re q u i re d t o t ra n s fer I / Os . Sto ra g e ef f i c ie n cy  t e ch n iq u es w i l l b e ap p l i ed t o t h e m o s t a p p ro p r i a t e p ar t o f t h e  

i n f ra s t ru c tu re a n d b e co m e in c re as in g l y e m b e d d ed i n t o sys te ms a n d  

s to rage des ign . 

M i lestones f o r Nex t Gene ra t i on I n f r as t r uc tu r e Exp lo i t a t i on  

Some key milestones are required to exploit new infrastructure directions in generaland storage infrastructure in particular:

1.  Sel l th e v is ion to senior business managers.

2.  Create a Nex t Gene rat i on I n f ras t ruc tu re Team , including cloudinfrastructure.

3.  Set aggr essive ta rg e ts for Infrastructure implementation and costsavings, in line with external IT service offerings.

4.  Select a st ack for each set of application suites:

▪  Choose a s ing le vendo r I n f ras t ruc tu re s tack from a large vendorthat can supply and maintain the hardware and software as a single

stack. The advantage of this approach is the cost of maintenancewithin the IT department can be dramatically reduced if the software istreated as a single SKU and updated as such, and the hardwarefirmware is treated the same way. The disadvantage is lack of choicefor components of the stack, and a higher degree of lock-in.

▪  Limit lock-in with a sourcing strategy. Choose an Ecosystem

I n f ras t ruc tu re Stack of software and hardware components that canbe intermixed. The advantage of this approach is greater choice and

Page 13: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 13/14

Wikibon.org 12 of 13

less lock-in, at the expense of significantly increased costs of internalIT maintenance.

5.  Reorgan ize and f la t t en I T suppor t by stack(s), and move away from anorganization supporting stovepipes. Give application development andsupport groups the responsibility to determine the service levels required,

and the Next Generation Infrastructure team the responsibility to provide theinfrastructure services to meet the SLA. Included in this initiative should be amove to DevOps, where application development and infrastructureoperation teams are cross-trained with the goal of achieving hyperproductivity.

6.  Crea te a sel f -se rv ice I T env i r onm ent with a service catalogue andintegrate charge-back or show-back controls.

From a strategic point of view, it will be important for IT to compete with externalIT infrastructure suppliers where internal data proximity or privacy requirementsdictate the use of private clouds, and use complementary external cloud services

where internal clouds are not economic.

Overa l l Sto rage Di rec t ions and Conc lus ion s  Storage infrastructure will change significantly with the implementation of a newgeneration of infrastructure across the portfolio. There will be a small percentage of application suites that will require a siloed stack and large scale-up monolithicarrays, but the long tail (90% of applications suites) will require standard storageservices that are inherently efficient and automated. These storage services will bemore distributed within the stack with increasing amounts of flash devices anddistributed within private and public cloud services. Storage software functionalitywill become more elastic and will reside or migrate to the part of the stack that

make most practical sense, either in the array or in the server or in a combinationof the two.

The I/O connections between storage and servers will become virtualized, with acombination of virtualized network adapters and other virtual I/O mechanisms. Thisapproach will save space, drastically reduce cabling, and allow dynamicreconfiguration of resources. The transport fabrics will be lossless Ethernet withsome use of InfiniBand or other high speed interconnects for inter-processorcommunication. Storage will become protocol agnostic. Where possible, storage willfollow a scale-out model, with meta-data management a key component.

The storage infrastructure will allow dynamic transport of data across the network

when required, for instance to support business continuity, and with somebalancing of workloads. However, data volumes and bandwidth are growing atapproximately the same rate, and large-scale movement of data between sites willnot be a viable strategy. Instead, applications (especially business intelligence andanalytics applications) will often be moved to where the data is (the Hadoop model)rather than pushing data to the code. This will be especially true of Big Dataenvironments, where vast amounts of semi-structured data will be available withinthe private and public clouds.

Page 14: A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

7/31/2019 A Special Report on Infrastructure Futures: Keeping Pace in the Era of Big Data Growth

http://slidepdf.com/reader/full/a-special-report-on-infrastructure-futures-keeping-pace-in-the-era-of-big 14/14

Wikibon.org 13 of 13

The criteria for selecting storage vendors will change in the future. Storage vendorswill have significant opportunities for innovation within the stack. They will have totake a systems approach to storage and be able to move the storage softwarefunctionality to the optimal place within the stack in an automated and intelligentmanner. Distributed storage management function will be a critical component of this strategy, together will seamless integration into backup, recovery and businesscontinuance. Storage vendors will need to forge close links with the stack providers,so that there is a single support system (e.g., remote support), a single updatemechanism for maintenance, and a single stack management system.

Act i on I t em : Nex t gene ra t i on s to rage i n f ras t ruc tu re i s coming t o a t h eate rnea r you . The bo t t om l i ne i s i n o rde r t o sca le and “ compe te ” w i t h c l oud

serv ice p rov ide rs , in t e rna l I T o rgan iza t ions m ust spend less t im e on labor -i n tens i ve i n f ras t ruc tu re m anagemen t and m o re e f f o r t on au tom a t ion , and

prov id ing e f f i c ien t s to r age se rv ices at sca le. The pa th to t h is v is ion w i l l goth r ough i n teg ra t i on i n t he f o rm o f conve rged i n f rast ru c tu re across t he

s tack w i t h i n t e l li gen t m anagem en t o f new t ypes o f st o rage (e .g. f l ash ) andthe i n t eg ra t i on o f B ig Da ta ana ly t i cs w i t h ope rat i ona l system s to ex t rac tnew va lue f rom in fo rm a t i on sou rces.