Embed Size (px)
DESCRIPTION
This session discusses the process to move legacy applications "into the cloud". It is intended for a diverse audience including developers, architects, and managers. We will discuss techniques, methodologies, and thought processes used to analyze, design, and execute a migration strategy and implementation plan -- from planning through rollout and operational. An important aspect of this is the necessity for technical staff to effectively communicate to mid-level management how these design decisions and strategies translate into cost, complexity and schedule. Commonly used migration strategies, cloud technologies, architecture options, and low level technologies will be discussed. The case will be made that investing in strategic refactoring and decomposition during the migration will reap the benefits of a modern, decoupled and simplified system. The end game being alignment and adoption of current best practices around PaaS, Saas, SOA, event-driven architectures, and message-oriented middleware, at scale in the cloud, to provide quantifiable business value. This talk will focus more on the big picture, at times delving into technical architectures and discussion of certain technologies and service providers. Use of Containers (Docker) is evangelized for decoupling and decomposing legacy systems.
Citation preview
inBloom, Inc.
server to cloudconverting a legacy platform to an open source
paas
todd fritz
April 3. 2014
inBloom, Inc. 2
agenda
• whoami• goals• change• genesis• decompose• refactor• architecture• ajug• questions• references• appendix
APRIL 2014
inBloom, Inc. 3
whoami
bio: http://www.linkedin/com/in/tfritz• architect at inbloom
opinions contained within this presentation may not represent my employer, but
I think they should • evangelist of layered, distributed, message-oriented-
middleware• current focus is middleware through caching, nosql data store• exposed to different companies, projects, people and
technologies• novice bass player• recent father of a five month old• scuba diver. next adventure:
https://www.bikiniatoll.com/divetour.html
APRIL 2014
No sleep for you!
inBloom, Inc. 4
thanks
credit for those who helped (or listened)• altisource
todd nist
• inbloom verlin henderson paul lawler vincent mayers ben morgan bill siggelkow
• red hat ray ploski
APRIL 2014
inBloom, Inc.
inBloom, Inc.
email: [email protected]
twitter: @inBloomDev
dev.inbloom.org
APRIL 2014 5
inBloom, Inc. 6
goals
APRIL 2014
inBloom, Inc. 7
objectives for next 45 minutes
• to delay you from happy hour and inspire curiosity• discuss the benefits of change• shine a light on a path forward• cover the if, why and how to modernize• define basic cloud migration criteria• explain techniques to decompose legacy apps• virtualization and containerization• evangelize containers as an architecture enabler• discuss migration strategy and architectures• this talk does not (yet) include code examples
(coming soon to atlanta java users group)
APRIL 2014
inBloom, Inc. 8
change
APRIL 2014
inBloom, Inc. 9
what? change? why?
APRIL 2014
an unknown manager in the wild, ready to spray a can of dilbert.
http://d1r5i20o8cadcu.cloudfront.net/designs/images/76856/original/programmer_creattica_full.jpg
inBloom, Inc. 10
may we live in interesting times
innovation is impossible without change
• “Software is eating the world.” – Marc Andreesson, 2011, Wall Street Journal “We are in the middle of a dramatic and broad technological and
economic shift in which software companies are poised to take over large swathes of the economy. More and more major businesses and industries are being run on software and delivered as online services—from movies to agriculture to national defense.”
• “Every company is a software company -- or at least aspiring to be one. That reality will shake up industries, lead to huge successes and failures and potentially make or break brands.” – Larry Dignan
• “…this software revolution in every company will be similar to how enterprise resource planning changed the game for businesses and their processes.” – Forrester analyst John McCarthy
APRIL 2014
http://blogs.flexerasoftware.com/ecm/2014/01/if-every-company-is-a-software-company-whats-the-key-to-business-model-success-.html
inBloom, Inc. 11
innovation economics
• a growing economic doctrine that adapts conventional economics theory so that knowledge, technology, entrepreneurship, and innovation are at the center of the model rather than independent forces that are largely unaffected by
policy good fit for companies that manage software
• two fundamental tenets:1. economic policy should drive productivity through
innovation2. market reliance on resources and price signals alone
may not be as effective to create both productivity and economic growth
• differs from conventional economic doctrines
• companies benefit from innovation in obvious waysAPRIL 2014
inBloom, Inc. 12
a path forward
how does a company that has not traditionally been in the software space become a successful software company? also applies to a software company that is reinventing itself, and startups.
1. protect intellectual property
2. re-think software monetization models based on how customers want to pay subscription models
3. automate the entire software, device and entitlement lifecycle software installation, provisioning, activation subscription management, software updates and upgrades software entitlements (features)
4. purpose-built software licensing and entitlement management
APRIL 2014
http://blogs.flexerasoftware.com/ecm/2014/01/if-every-company-is-a-software-company-whats-the-key-to-business-model-success-.html
inBloom, Inc. 13
status quo gets left behind
• today will not be tomorrow• cloud provides competitive advantage for some use cases
amazon is market leader ongoing competition between cloud providers; amazon and google encourages use standards and adoption of new tech and patterns
• technology advances disrupt computing infrastructure and software to expose opportunity
• quantum computing, e.g. d-wave qubit ok, some controversy, but it is fast significant advance in compute power will disrupt software design and how we scale, e.g. compute grids it is real, maturing, and is not going away outpaces moore’s law as it scales http://www.gizmag.com/d-wave-quantum-computer-supercomputer-ranking/27476/
APRIL 2014
inBloom, Inc. 14
cloud adoption – 36% increase – benefits
APRIL 2014
http://www.secure-24.com/the-advantages-of-infrastructure-outsourcing/
inBloom, Inc. 15
good candidates for cloud
1. strong business sponsorship at tip of spear2. well understood with few dependencies3. uses common standards and
implementations4. can be decomposed or decoupled5. already modularized or service oriented6. already virtualized7. favorable cost-benefit analysis8. security model translates to cloud9. opportunities to improve by modernization
APRIL 2014
http://www.websitepulse.com/blog/the-great-cloud-migration-are-your-apps-ready
inBloom, Inc. 16
unfavorable candidates – who has one?
1. lacks strong business sponsorship 2. security equation difficult to understand3. complex architecture and external dependencies4. technology or deployment lock in5. latency sensitive (real time apps)6. not standards based or uses home-grown
libraries7. etl heavy (not parallelized) or long running jobs8. government regulations, contractual
requirements, or certifications (pci)9. unfavorable economics (cost-benefit/risk
analysis)10.transaction lifecycles not well understoodAPRIL 2014
http://www.websitepulse.com/blog/the-great-cloud-migration-are-your-apps-ready
inBloom, Inc. 17
the go forward decision
APRIL 2014
inBloom, Inc. 18
genesis
APRIL 2014
inBloom, Inc. 19
hello world! i am a legacy system
APRIL 2014
inBloom, Inc. 20
what to do with it?
break it apart; analyze
employ an iterative methodology leverage existing knowledge and literature some analysis before decompose and modularize (scale cube) separation of concerns proof of concepts are your friend
deployment environments private servers cloud (private, public) hybrid
platform architectures legacy paas xpaas
APRIL 2014
inBloom, Inc. 21
terms
• paas “Platform as a service (PaaS)…provides a computing
platform and a solution stack as a service. Along with software as a service (SaaS) and infrastructure as a service (IaaS), it is a service model of cloud computing… the consumer creates the software using tools…from the provider. The consumer also controls software deployment and configuration... The provider provides the networks, servers, storage, and other services that are required to host the consumer's application” -wikipedia
• xpaas standardization of enterprise paas create platforms from a catalogue of paas/saas/iaas
APRIL 2014
inBloom, Inc. 22
paas by segment
gartner’s 2012 paas market share chart (from red hat’s xpaas whitepaper).
APRIL 2014
inBloom, Inc. 23
xpaas topology
APRIL 2014
https://img.en25.com/Web/RedHat/JB_xPaaS_Tech_Overview_11454037_v3_0913cd_web.pdf
• enables specialization, layering, separated concerns, decoupling
inBloom, Inc. 24
decompose
APRIL 2014
inBloom, Inc. 25
i am the fragments of a legacy system
APRIL 2014
hear me roar…
inBloom, Inc. 26
what have I gotten myself into?
APRIL 2014
http://www.secure-24.com/the-advantages-of-infrastructure-outsourcing/
“legacy spaghetti”the code sucks…
i can’t even build it…
inBloom, Inc. 27
getting started
• involve devops from the beginning• if your company does not have devops then call
pressureManager()• make decisions from quantitative assessments• agile systems analysis and integration modeling• agile modeling best practices• use the afk scale cube
http://akfpartners.com/techblog/2008/05/08/splitting-applications-or-services-for-scale/
• read “The Art of Scalability” by abbott and fisher http://theartofscalability.com/
• perform functional decomposition and service identification• the cloud is not “all or nothing”
phased migrations can realize immediate value new technologies can coexist with legacy
• iterative design ahead• use containers to enable decoupling, architectural flexibility, confine legacy
APRIL 2014
inBloom, Inc. 28
legacy analysis and design
• decompose into layers; functions and separate concerns• design decoupled components and services• isolate technologies within components (insulate lock-in)• conceptualize legacy components and services as
“legos”• future state runs alongside current state to provide value• take heed of transaction lifecycle, batch jobs,
data retention and use cases• be mindful of customer impact, cost and schedule
constraints• consider security at each layer and service
APRIL 2014
inBloom, Inc. 29
success – value add criteria
APRIL 2014
“For every complex question there is a simple and wrong solution.”- Albert Einstein
avoid “Khan’s paradigm”: a top-down plan led by an overlord of super-humans will fail.
my Java kung fu can crush any project.
tactical • time, cost, qualitystrategic • sustainability, relevance,
effect
inBloom, Inc. 30
agile modeling best practices
APRIL 2014
http://www.agilemodeling.com/essays/bestPractices.htm
inBloom, Inc. 31
refactor
APRIL 2014
inBloom, Inc. 32
assembling the new system
APRIL 2014
inBloom, Inc. 33
we can rebuild it
• cynical optics rather than one bowl of spaghetti; several small, independent plates investment in analysis, design ahead, separating concerns has
tangible benefits use containers!
APRIL 2014
before after
inBloom, Inc. 34
putting it together
• favor continuous deployment• prefer reusable, modularized components• decoupled services; soa done right
(microservice)• message oriented and event driven• parallelize development across business
function• select the right tool for each component;
polyglot• include automated unit and integration
tests• leverage containerization instead of
virtualization when possible (see next slide)
APRIL 2014
inBloom, Inc. 35
why containers?
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 36
revisting paas requirements
• “Virtualization vs. Containers to support PaaS” by Dua, Raja, Kakadia http://www.slideshare.net/rajdeep/conference-presentationv3 Basis of next three slides
• paas focuses on developer productivity and abstracts out underlying infrastructure
• 3 key paas requirements for the infrastructure1. network, compute and storage programmatically
managed and provisioned2. h/a infrastructure (e.g. nodes) efficiently utilized3. ability to bind applications/services to external
network (dns, routers)
APRIL 2014
inBloom, Inc. 37
paas requirements (cont.)
1. network, compute and storage programmatically managed and provisioned
2. h/a infrastructure (e.g. nodes) efficiently utilized3. ability to bind applications/services to external network (dns,
routers)
vms good for #1 & #2 apps can reside within:
• vms• containers• vms with containers
containers better for #3 as resources better utilized and light weight
APRIL 2014
inBloom, Inc. 38
paas requirements (cont.)
• containers have weaknesses (for now) standardization strong security os independence robust monitoring
• “Containers have inherent advantage over VMs for PaaS use case”.- Dua, Raja, Kakadia
APRIL 2014
inBloom, Inc. 39
container platform contenders
• warden (cloud foundry) https://github.com/cloudfoundry/warden
• docker https://www.docker.io/
• google lmctfy (let me contain that for you) https://github.com/google/lmctfy december, 2013
• openvz
APRIL 2014
inBloom, Inc. 40APRIL 2014
container of the day
inBloom, Inc. 41
about docker
docker is a micro container framework for paas• https://www.docker.io/learn_more/• open-source• easier scalability• lightweight, portable, insulated containers • reusable from dev (“local cloud”) through
production • can run at scale on vms, bare metal, cloud;
virtually anywhere• encapsulate any payload (application)• run consistently on and between virtually any
serverAPRIL 2014
inBloom, Inc. 42
more about docker
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 43
brief overview about how it works
• docker builds on lxc which offers system-level virtualization and has existed since linux 2.6.32 (December, 2009) – but use 3.8+
• docker has three parts docker daemon runs as root to manage containers docker containers spawn from images, which are tiny and can be versioned docker repository allows images to be exchanged and versioned like code
(public or private) • each container has its own ip address• link exposed ports and variables across containers through configuration using
abassador containers to avoid hard coding (svendowideit): consumer redis-ambassador redis
• port and pipework to expose containers outside host• can share volumes, multi home, integrate containers into host network, and
much more (ajug)• continuous integration can generate versioned docker images, web hooks, repo
notifications• supervisor management tool to manage processes within container (
http://supervisord.org/introduction.html)
APRIL 2014
inBloom, Inc. 44
even more about docker
• common use cases: automate application packaging and deployment lightweight paas environments automate testing, continuous integration, and
deployment deploy and scale web apps, databases, backend
services• growing adoption since dec 2013
• red hat fast-tracks docker apps for enterprise linux http://www.infoworld.com/t/application-virtualization
/red-hat-fast-tracks-docker-apps-enterprise-linux-238122
• production ready deployment planned for december, 2014. Support services planned for early 2015APRIL 2014
inBloom, Inc. 45
containers available for reuse
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
a docker container image may already exist
inBloom, Inc. 46
docker image registry – versioning!
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
artifactory for virtualization images?
inBloom, Inc. 47
New in Docker 0.9
• execution driver api customize execution environment around container;
enables use of other isolation tools• built in execution container - libcontainer
alongside lxc, boosts stability, insulates docker from different versions of lxc
APRIL 2014
inBloom, Inc. 48
architecture
APRIL 2014
inBloom, Inc. 49
containers enable architecture
• containers enable architecture and design• design, build, or migrate, each layer or module into a container• containers encapsulate technology, isolate lock-in, and are easy to
scale• enables use of best tool; go polyglot• easy to upgrade what’s in a container
replace node.js with vert.x technology portability
• understand workload and transaction use cases (data stores, social)
• service oriented (done right, not soap)• learn message oriented middleware (mom)
enterprise integration patterns (eip) apache camel queues (amqp)
• prefer data streams to batch jobs• leverage power of compute grids and distributed caching• pay the piper and dedicate effort to data/domain architectureAPRIL 2014
inBloom, Inc. 50
ajug
APRIL 2014
inBloom, Inc. 51
coming soon
• v2 of this presentation with code examples in-depth architecture ideas development concepts
• to be presented to atlanta java users group (ajug)
APRIL 2014
inBloom, Inc. 52
questions
APRIL 2014
inBloom, Inc. 53
references
APRIL 2014
inBloom, Inc. 54
reference material
• https://www.docker.io/• http://deis.io/overview/• http://www.ambysoft.com/essays/brokenTriangle.html• http://martinfowler.com/articles/microservices.html#MicroservicesAndSoa• http://www.susannemadsen.co.uk/1/post/2013/12/the-iron-triangle-and-the-tripleconstraintsare-outdated.html• http://www.slideshare.net/pet-computacao/software-evolution-from-legacy-systems-service-oriented-architect
ure-to-cloud-computing
• http://www.cloudcomputingpatterns.org/Message-oriented_Middleware• https://img.en25.com/Web/RedHat/JB_xPaaS_Tech_Overview_11454037_v3_0913cd_web.pdf• http://www.infoq.com/articles/sustainable-architectural-design-decisions• http://www.sciencedirect.com/science/article/pii/S0268401201000135• http://martinfowler.com/bliki/TolerantReader.html• http://www.cakesolutions.net/teamblogs/2014/01/28/deploying-scala-akka-based-applications-to-amazon-clou
d/
• http://highscalability.com/blog/2013/5/8/typesafe-interview-scala-akka-is-an-iaas-for-your-process-ar.html• http://www.informationweek.com/software/moving-legacy-apps-to-the-cloud/d/d-id/1108964• http://www.forbes.com/sites/louiscolumbus/2012/11/08/cloud-computing-and-enterprise-software-forecast-upd
ate-2012/
• http://blogs-images.forbes.com/louiscolumbus/files/2013/01/Bain-Go-To-Market-System.jpg• http://www.infoq.com/articles/SaaSificationOfLegacyApps• http://www.agilemodeling.com/essays/bestPractices.htm• http://osintegrators.com/node/145• http://appsembler.com/blog/paas-bakeoff-comparing-stackato-openshift-dotcloud-and-heroku-for-django-hosti
ng-and-deployment/APRIL 2014
inBloom, Inc. 55
More Reference Material
• Redmonk on DB technology: http://www.youtube.com/watch?v=HClEcT8n5Lc&app=desktop
• http://gigaom.com/2014/03/25/heres-the-google-vs-amazon-pricing-break-down/• Decomposing applications for scalability and deployability
http://vimeo.com/49392435
• http://venturebeat.com/2008/10/13/the-cloud-isnt-for-everyone/• http://blogs.flexerasoftware.com/ecm/2014/01/if-every-company-is-a-software-compa
ny-whats-the-key-to-business-model-success-.html
• https://blog.codecentric.de/en/2014/01/lightweight-virtual-machines-made-simple-docker-run-100-virtual-maschines/
• https://blog.codecentric.de/en/2014/01/docker-networking-made-simple-3-ways-connect-lxc-containers/
• http://blog.docker.io/2014/03/docker-0-9-introducing-execution-drivers-and-libcontainer/
APRIL 2014
inBloom, Inc. 56
appendix(slides that did not fit within time constraint)
APRIL 2014
inBloom, Inc. 57
paas -> xpaas
an xpaas is composed of multiple, specialized paas systems
“…xPaaS services augment core container functionality with integration, business process management (BPM) and mobile capabilities.”
integration paas (ipaas) -- simplifies connections, messages, route definitions, and data transformations
bpm paas (bpmpaas) -- process modeling, process engine, simplify definition and evolution of business processes
mobile paas (mpaas) -- push notifications, data synchronizations and back-end integration
…
APRIL 2014
inBloom, Inc. 58
agile legacy systems analysis and integration modeling
APRIL 2014
Methodology: http://www.agilemodeling.com/essays/agileLegacyIntegrationModeling.htm
inBloom, Inc. 59
the scale cube
APRIL 2014
inBloom, Inc. 60
Analysis Considerations
• Refine understanding of current and future state solutions Client-Server (beware of stateful thick clients) N-Tier SaaS ready components?
• Features, use cases, request lifecycle• Messaging models, usage patterns, volume and velocity – capacity
planning• Messaging? Network configuration? • All aspects of security• Identify components and concerns, layers• Impact of latency • Caching, and Master Data Management (MDM)• Identify problems and do not ignore technical debt…• Product Management != Project Management (or scrum master)
The unknown may lead to failure, delays or increased cost.
APRIL 2014
inBloom, Inc. 61
Security First
FEBRUARY 2014
• Foundational• Assess all concerns and services
• Network, OS, Disk (data at rest)• Applications • Encryption• Data
• Protecting Personally Identifiable Information (PII)• Securing logged information• Securing in-memory objects and caches
• User Authentication and Authorization• Manage users, groups, roles and permissions, SSO• Separate abstractions for Authentication from Authorization (CAS, etc)• OAUTH, SAML2
• Learn about SENDS • Science-Enhanced Networked Domains and Secure Social Spaces• Security needs to be more than a technology solution• http://www.theatlantic.com/technology/archive/2011/03/cyber-security-cant-ignore-human-behavior/7282
6/
• Be aware of industry bias toward technical solutions with security.• Can’t control stupid human behaviors such as taping a password to keyboard.
• APRIL
inBloom, Inc. 62
What?
Perhaps you are wondering…• Why Docker?
The next several slides provide background• (Borrowed from Docker’s site.)
Keep in mind:• Docker containers encapsulate concerns• Avoids Holy Wars about specific architectures to
implement• Great for the dozens of Spring projects..
Run each within its own Container
APRIL 2014
inBloom, Inc. 63
how a docker works
• uses linux kernel for containment (jailing) chroot: changes root directory of process w/ child cgroups: control groups
• groups processes (to unit of thread)• pseudo filesystem• numerous subsystems functions implemented
– CPU sets, etc
• linux containers: lxc file system isolation network and process isolation resource allocation
APRIL 2014
inBloom, Inc. 64
Converting the Legacy App into Docker
• Containers enable extreme choice – use what you want.
• Container model facilitates separates concerns Applications (various technologies) Back End systems including “Big Data” Messaging Caching (e.g. Hazelcast!) Microservices. Use Data as glue.
APRIL 2014
inBloom, Inc. 65APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 66
Docker Ecosystem
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 67
Why Developers should Care about Docker
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 68
DevOps & Docker
APRIL 2014
http://www.slideshare.net/dotCloud/docker-intro-november
inBloom, Inc. 69
More about Docker
• Deis is a framework that caps Docker and Heroku http://deis.io/deis-0-5-1-docker-containers-all-the-way-down/ “Deis…is an open source PaaS that makes it easy to deploy and scale Docker
containers and Chef nodes used to host applications, databases, middleware and other services. Deis leverages Chef, Docker, Heroku Buildpacks to provide a private PaaS that is lightweight and flexible.”
Supported Languages• Java, Scala, PHP, Ruby, Python, Node.js, Clojure, Play, Perl, Dart, Go.• Deis can deploy anything using Heroku Buildpacks or Dockerfiles.
Supported Providers• Any system including every public cloud, private cloud or bare metal.• Automatic provisioning for EC2, Rackspace, Digital Ocean
• Integration testing with Maven and Docker http://giallone.blogspot.co.uk/2014/03/integration-testing-with-maven-
and.html
• Industry timeline from 1995 to Docker http://5pi.de/docker-intro/#/step-1
• Decker
APRIL 2014
inBloom, Inc. 70
Message Oriented Middleware
APRIL 2014
