Upload
jaxlondonconference
View
933
Download
7
Tags:
Embed Size (px)
DESCRIPTION
Presented at JAX London 2013 Per-tenant resource management can help ensure that collocated tenants peacefully share computational resources based on individual quotas. This session begins with a comparison of deployment models (shared: hardware, OS, middleware, everything) to motivate the multitenant approach. The main topic is an exploration of experimental data isolation and resource management primitives in IBM’s JDK that combine to help make multitenant applications smaller and more predictable.
Citation preview
Steve Poole IBM
JVM Support for Multitenant Applications
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE. IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’S
CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE.
IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
- CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
Important Disclaimers
!Works at IBM’s Hursley Laboratory in the UK
Involved in IBM Java VM development since before Java was 1
Currently leading IBM’s OpenJDK technical engagement
Steve Poole
JVM Support for Multi-Tenant Applications
What this talk is about
!!!!This experimental technology is being developed to help address important pressures on Java !Google ‘IBM Java 8 beta’ for more information !
■ By the end of this session, you should be able to: !
– Understand what multitenancy is and what it’s good for !
– Describe the challenges of multitenant Java deployments !
– Understand ideas for new JDK features to convert existing applications into multitenant deployments
!
Agenda
1. What Multi-tenancy is all about aka how to simplify to save time and money
2. The challenges of building a multi-tenant application in Java 3. How to keep your application running in a multi-tenant world -
dealing with bad behaviour 4. Risk vs. Reward: How dense can we go? 5. Wrap-up: Summary, and next steps
Part 1What is Multitenancy?
!(simplify to save time and money)
Who owns one of these?
Who owns one of these?
Who owns one of these?
Who owns one of these?
At a basic level Multitenancy is a drive to
reduce unnecessary complexity and duplication
this is sophisticated but expensive
this is simpler and cheaper
Who owns one of these?
At a basic level Multitenancy is a drive to
reduce unnecessary complexity and duplication
this is sophisticated but expensive
this is simpler and cheaper
Multitenancy helps with
finding the right balance
Simplifying the software stack by removing all extraneous pieces makes better use of hardware and the people who run it. !
Simple == Cheaper == Predictable == Robust
Its not just a hardware story
“Every piece of knowledge must have a single, unambiguous, authoritative representation within a system” Pragmatic Programmer (Hunt & Thomas)
(or: copy-and-paste encourages problems)
Dont Repeat YourselfSimple
We want to avoid the clone army
•Are your choices only
•fully dedicated machine or shared hardware?
•Multitenancy isn’t just a checkbox.
•There are various levels of ‘tenancy’ defined today.
Are you already using Multitenancy
SaaS Tenancy Spectrum
SaaS Tenancy Spectrum
Your basic laptop, desktop or server machine.
yours, all yours..
SaaS Tenancy Spectrum
A simple shared machine LPAR’d or VM’d into multiple complete
stacks.
SaaS Tenancy Spectrum
Reduced overhead by running multiple app servers on one O/S Keep everything else
separate though. “just in case”
SaaS Tenancy Spectrum
Multiple applications
running on one app server - sharing db server and
middleware!
SaaS Tenancy Spectrum
sharing one app!
SaaS Tenancy Spectrum
Sharing everything including database tables.
SaaS Tenancy Spectrum
A Multitenant JVM covers theses levels of sharing
SaaS Tenancy Spectrum
Merging these bubbles saves money, reduces complexity and can be a real business differentiator.
But it requires engineering effort
SaaS Tenancy Spectrum
Remember, from the end user point of view, they see it like this
■ Customer viewpoint !
– Cost: provider offers the service more cheaply – Time to Value: up and running fast, typically
upgraded often & quickly – Quality of Service: focus on SLA needed not your
ability to run infrastructure – Bypass IT Backlog: streamlined deployment
(handled by provider) !
■ Provider viewpoint – Cost: Minimal moving parts / duplication – Agility: Upgrades, backups, on-boarding
Efficencies
– Fewer Parts == Better Density == $$$ for the service provider
– Fewer Parts == Better Density == £££ for the service provider
Part 2Getting there..
!
Increasing density of the software stack is impacting us all. !Cost of hosting is a business differentiator already !Its only going to increase in importance !If you’re in the cloud you already know !!Moving from ‘free’ to ‘not free’ hosting is an intensive exercise. Your software choices are starting to be decided by CPU and memory usage. With your own hardware you could let it run slow. Can’t do that in the cloud! !So how do move forward with Java and Multitenancy?
Climbing MT tenant
Challenge #1 Isolation
■The fear factor is loosing isolation ■unexpected side effects from other parts
■This is a reasonable concern ■Same number of eggs (apps), fewer baskets ■You want really good baskets arranged carefully
■But it’s not a new problem ■We can reduce the change of failure (more later) ■We can reduce the impact with good choreography
■You need to think about choreography today ■Today we use clusters to provide fail-over and arrange clusters to avoid same machine/rack/site failures
■Apply the same ideas to Multitenancy
http://circa71.wordpress.com/2010/07/
http://bit.ly/e7G1jb
Challenge #2 Cost of Entry
J Easy == No app changes
J Hypervisor sharing only
J Port Collisions
J File System Collisions
J Security Challenges
merge
merge
J Data Isolation between apps
J Control over resource hogs
J JVM can help!!
simplest (hence most popular) Still have neighbours but they are far away
Saves footprint (GB) But now we have collisions Ops guys can help with O/S JVMs can use -Xshareclasses
More footprint savings But now we have real isolation concerns (closer neighbours) We need control over resource hogs
Java Heap consumes 100’s of MB of memory –Heap objects cannot be shared between JVMs –GC has a helper thread-per-core by default
Just-in-Time Compiler consumes 10’s of MB of memory –Generated code is private and big –Generated code is expensive to produce
• Steals time from application • Multiple compilation threads by default
No choreography between JVM instances –Compilation or GC activity can happen at identical (and bad) times
Challenge #2.5 Building isolation
We need to fix the following !L Data Isolation between applications !
L Control over resource hogs
Without forcing people to change their applications!
Challenge #2.5 Building isolation
Data Isolation Challenges
■ Applications embed deployment information like url patterns in code
■ Wait! What happens if we try to deploy two copies of this servlet to a single server?
■ Static variables are bad (for sharing) ■ Most libraries are full of static variables
Wait! What happens ifeach tenant needs a different default locale?
•Usual solutions: – Wrap the whole thing in a ClassLoader (i.e. rewrite your code) – Get ride of the static variable (i.e. rewrite your code) – Use BCI to rewrite the code automatically (erodes robustness)
!!We need a silver bullet!
Wait! What happens ifeach tenant needs a different default locale?
■ Concept: Add a single argument (–Xmt for multi-tenant) to your Java command-line to opt into sharing a runtime with others.
■ Result: Your application behaves exactly as it if had a dedicated JVM, but in reality it runs side-by-side with other applications.
■ Benefits: Smaller, faster, and eventually smarter –Less duplication: (1 GC, 1 JIT), Heap object sharing –Fast Startup: JVM is already running and warm when starting apps
Multitenant JDK
■ Required: No code Changes!
Part 3Demo (of a demo)
Launch your application
■ Opt-in to multitenancy by adding –Xmt
Register with javad daemon
■ JVM will locate/start daemon automatically
locate
Create a new tenant
■ New tenant created inside the javad daemon
locateTenant1
Create a 2nd tenant
■ New tenant created inside the javad daemon
locateTenant1
Create a 2nd tenant
■ New tenant created inside the javad daemon
locateTenant1
Tenant2
One copy of common codelives in the javad process.
Most runtime structures
are shared.
■ What if … the JVM knew about tenants and provided each one with a different view of static variables?
■ Meet the @TenantScope annotation.
■ @TenantScope Semantics: Static variable values are stored per-tenant ■ Each tenant has their own LocaleSettings.defaultLocale ■ Now many tenants can share a single LocaleSettings class
Tenant1
Tenant2
… LocaleSettings.setDefaultLocale( LocaleSettings.UK );
…
… LocaleSettings.setDefaultLocale( LocaleSettings.US );
…
Providing data isolation
Did I say ‘no code changes?’
Application Changes
merge
■ @TenantScope markup gets added automatically as classes are loaded ■ Tenants see dedicated middleware – but behind the curtains classes (and
JIT’ed code) are actually shared
If it’s invisible - why have @TenantScope?
■ Allows middleware to opt out ■ Opportunities for even more density. !
■ Basic operations on Tenants available to the middleware –Data Isolation –Resource Management (more in this in a minute) !
■ Ability for the middleware to differentiate between Tenants –Which one is causing the problem? !
■ Querying the state of Tenants –How much free memory do you have?
Part 4Hey, aren’t we done yet?
Dealing with bad behaviour
images from http://www.rra.memberlodge.org/Neighbourhood-Watch-Reporting
http://bit.ly/ficwkl
■ The closer your neighbours the better your controls must be
■ Multitenant JDK provides controls on –CPU time –Heap size –Thread count –File IO: read b/w, write b/w –Socket IO: read b/w, write b/w
!
Shared environments need resource ctl
■ Simple command-line switches for new resources –-Xlimit:cpu=10-30 // 10% minimum CPU, 30% max –-Xlimit:cpu=30 // 30% max CPU –-Xlimit:netIO=20M // Max bandwidth of 20 Mbps
■ Existing options get mapped for free –-Xms8m –Xmx64m // Initial 8M heap, 64M max
■ Plus some JMX beans to see how much of each resource you are using – i.e. understand how your code uses resources by wrapping in a tenant
Resource Control Ergonomics
■ Throttling at Java layer for portability ■ Or, leveraging OS WLM directly for efficiency (Linux & AIX)
– Note: many WLMs tend to like processes, not groups of threads
Network: XXXKB/S
Hardware resources
CPU: XXX GHZ DISK: XXXKB/S Memory: XXX GB
OS Level Resources Management
Thread
SocketHandler
OS Workload Manager (WLM)
OS resources
JVM Resource Management
Resource Throttle Layer
JSR 284 API
GC (Heap Mgmt)
Tenant Tenant Tenant Tenant Tenant
OS
JVM
Resource native API
Memory CPU Thread File I/O Socket I/O CPU File I/O Socket I/O
Building on JSR 284
JSR-284 Resource Consumption Mgmt API
Round OS as controller JVM as controller
1 1362s 1267s
2 1167s 1239s
3 1452s 1390s
4 1094s 1122s
5 1139s 1123s
6 1244s 1134s
Average 1243s 1212s
Benchmark setting • Duration comparison: Linux AMD64, run a CPU-intensive app
with 10 threads with 100% CPU quota, each thread doing the same Fibonacci calculation, benchmark the duration
• Accuracy comparison: Linux AMD64, run two CPU-intensive apps each doing the same Fibonacci calculation, but with different CPU quota: 60% vs 30%, benchmark the accuracy
Result: JVM control achieves comparable performance, but less accuracy.
Accuracy
cpu throttling in os controller
cpu%
0.00
20.00
40.00
60.00
80.00
time
00:0000:0100:0200:0300:0400:0500:0600:0700:0800:0900:1000:1100:1200:1300:1400:1500:1600:1700:1800:1900:2000:2100:2200:2300:2400:2500:2600:2700:2800:2900:3000:3100:3200:3300:3400:3500:3600:3700:3800:3900:4000:4100:4200:4300:4400:4500:4600:4700:4800:4900:5000:5100:5200:5300:5400:5500:5600:5700:5800:5901:0001:0101:0201:0301:0401:0501:0601:0701:0801:0901:1001:1101:1201:1301:1401:1501:1601:1701:1801:1901:2001:2101:2201:2301:2401:2501:2601:2701:2801:2901:3001:3101:3201:3301:3401:3501:3601:3701:3801:3901:4001:4101:4201:4301:4401:4501:4601:4701:4801:4901:5001:5101:5201:5301:5401:5501:5601:5701:5801:5902:0002:0102:0202:0302:0402:0502:0602:0702:0802:0902:1002:1102:1202:1302:1402:1502:1602:1702:1802:1902:2002:2102:2202:2302:2402:2502:2602:2702:2802:2902:3002:3102:3202:3302:3402:3502:3602:3702:3802:3902:4002:4102:4202:4302:4402:4502:4602:4702:4802:4902:5002:5102:5202:5302:5402:5502:5602:5702:5802:5903:0003:0103:0203:0303:0403:0503:0603:0703:0803:0903:1003:1103:1203:1303:1403:1503:1603:1703:1803:1903:2003:2103:2203:2303:2403:2503:2603:2703:2803:2903:3003:3103:3203:3303:3403:3503:3603:3703:3803:3903:4003:4103:4203:4303:4403:4503:4603:4703:4803:4903:5003:5103:5203:5303:5403:5503:5603:5703:5803:5904:0004:0104:0204:0304:0404:0504:0604:0704:0804:0904:1004:1104:1204:1304:1404:1504:1604:1704:1804:1904:2004:2104:2204:2304:2404:2504:2604:2704:2804:2904:3004:3104:3204:3304:3404:3504:3604:3704:3804:3904:4004:4104:4204:4304:4404:4504:4604:4704:4804:4904:5004:5104:5204:5304:5404:5504:5604:5704:5804:5905:0005:0105:0205:0305:0405:0505:0605:0705:0805:0905:1005:1105:1205:1305:1405:1505:1605:1705:1805:1905:2005:2105:2205:2305:2405:2505:2605:2705:2805:2905:3005:3105:3205:3305:3405:3505:3605:3705:3805:3905:4005:4105:4205:4305:4405:4505:4605:4705:4805:4905:5005:5105:5205:5305:5405:5505:5605:5705:5805:5906:0006:0106:0206:0306:0406:0506:0606:0706:0806:0906:1006:1106:1206:1306:1406:1506:1606:1706:1806:1906:2006:2106:2206:2306:2406:2506:2606:2706:2806:2906:3006:3106:3206:3306:3406:3506:3606:3706:3806:3906:4006:4106:4206:4306:4406:4506:4606:4706:4806:4906:5006:5106:5206:5306:5406:5506:5606:5706:5806:5907:0007:0107:0207:0307:0407:0507:0607:0707:0807:0907:1007:1107:1207:1307:1407:1507:1607:1707:1807:1907:2007:2107:2207:2307:2407:2507:2607:2707:2807:2907:3007:3107:3207:3307:3407:3507:3607:3707:3807:3907:4007:4107:4207:4307:4407:4507:4607:4707:4807:4907:5007:5107:5207:5307:5407:5507:5607:5707:5807:5908:0008:0108:0208:0308:0408:0508:0608:0708:0808:0908:1008:1108:1208:1308:1408:1508:1608:1708:1808:1908:2008:2108:2208:2308:2408:2508:2608:2708:2808:2908:3008:3108:3208:3308:3408:3508:3608:3708:3808:3908:4008:4108:4208:4308:4408:4508:4608:4708:4808:4908:5008:5108:5208:5308:5408:5508:5608:5708:5808:5909:0009:0109:0209:0309:0409:0509:0609:0709:0809:0909:1009:1109:1209:1309:1409:1509:1609:1709:1809:1909:2009:2109:2209:2309:2409:2509:2609:2709:2809:2909:3009:3109:3209:3309:3409:3509:3609:3709:3809:3909:4009:4109:4209:4309:4409:4509:4609:4709:4809:4909:5009:5109:5209:5309:5409:5509:5609:5709:58
cpu throttling in jvm controller
cpu%
0.00
22.50
45.00
67.50
90.00
time
00:0000:0100:0200:0300:0400:0500:0600:0700:0800:0900:1000:1100:1200:1300:1400:1500:1600:1700:1800:1900:2000:2100:2200:2300:2400:2500:2600:2700:2800:2900:3000:3100:3200:3300:3400:3500:3600:3700:3800:3900:4000:4100:4200:4300:4400:4500:4600:4700:4800:4900:5000:5100:5200:5300:5400:5500:5600:5700:5800:5901:0001:0101:0201:0301:0401:0501:0601:0701:0801:0901:1001:1101:1201:1301:1401:1501:1601:1701:1801:1901:2001:2101:2201:2301:2401:2501:2601:2701:2801:2901:3001:3101:3201:3301:3401:3501:3601:3701:3801:3901:4001:4101:4201:4301:4401:4501:4601:4701:4801:4901:5001:5101:5201:5301:5401:5501:5601:5701:5801:5902:0002:0102:0202:0302:0402:0502:0602:0702:0802:0902:1002:1102:1202:1302:1402:1502:1602:1702:1802:1902:2002:2102:2202:2302:2402:2502:2602:2702:2802:2902:3002:3102:3202:3302:3402:3502:3602:3702:3802:3902:4002:4102:4202:4302:4402:4502:4602:4702:4802:4902:5002:5102:5202:5302:5402:5502:5602:5702:5802:5903:0003:0103:0203:0303:0403:0503:0603:0703:0803:0903:1003:1103:1203:1303:1403:1503:1603:1703:1803:1903:2003:2103:2203:2303:2403:2503:2603:2703:2803:2903:3003:3103:3203:3303:3403:3503:3603:3703:3803:3903:4003:4103:4203:4303:4403:4503:4603:4703:4803:4903:5003:5103:5203:5303:5403:5503:5603:5703:5803:5904:0004:0104:0204:0304:0404:0504:0604:0704:0804:0904:1004:1104:1204:1304:1404:1504:1604:1704:1804:1904:2004:2104:2204:2304:2404:2504:2604:2704:2804:2904:3004:3104:3204:3304:3404:3504:3604:3704:3804:3904:4004:4104:4204:4304:4404:4504:4604:4704:4804:4904:5004:5104:5204:5304:5404:5504:5604:5704:5804:5905:0005:0105:0205:0305:0405:0505:0605:0705:0805:0905:1005:1105:1205:1305:1405:1505:1605:1705:1805:1905:2005:2105:2205:2305:2405:2505:2605:2705:2805:2905:3005:3105:3205:3305:3405:3505:3605:3705:3805:3905:4005:4105:4205:4305:4405:4505:4605:4705:4805:4905:5005:5105:5205:5305:5405:5505:5605:5705:5805:5906:0006:0106:0206:0306:0406:0506:0606:0706:0806:0906:1006:1106:1206:1306:1406:1506:1606:1706:1806:1906:2006:2106:2206:2306:2406:2506:2606:2706:2806:2906:3006:3106:3206:3306:3406:3506:3606:3706:3806:3906:4006:4106:4206:4306:4406:4506:4606:4706:4806:4906:5006:5106:5206:5306:5406:5506:5606:5706:5806:5907:0007:0107:0207:0307:0407:0507:0607:0707:0807:0907:1007:1107:1207:1307:1407:1507:1607:1707:1807:1907:2007:2107:2207:2307:2407:2507:2607:2707:2807:2907:3007:3107:3207:3307:3407:3507:3607:3707:3807:3907:4007:4107:4207:4307:4407:4507:4607:4707:4807:4907:5007:5107:5207:5307:5407:5507:5607:5707:5807:5908:0008:0108:0208:0308:0408:0508:0608:0708:0808:0908:1008:1108:1208:1308:1408:1508:1608:1708:1808:1908:2008:2108:2208:2308:2408:2508:2608:2708:2808:2908:3008:3108:3208:3308:3408:3508:3608:3708:3808:3908:4008:4108:4208:4308:4408:4508:4608:4708:4808:4908:5008:5108:5208:5308:5408:5508:5608:5708:5808:5909:0009:0109:0209:0309:0409:0509:0609:0709:0809:0909:1009:1109:1209:1309:1409:1509:1609:1709:1809:1909:2009:2109:2209:2309:2409:2509:2609:2709:2809:2909:3009:3109:3209:3309:3409:3509:3609:3709:3809:3909:4009:4109:4209:4309:4409:4509:4609:4709:4809:4909:5009:5109:5209:5309:5409:5509:5609:5709:58
60% throttling30% throttling
Duration
CPU Throttling
os throttling jvm throttling
Part 5Performance so far
■ Environment: Measure standard benchmarks in a 1 GB + 1 core VirtualBox guest – Advantage: Easy to control, highly reproducible
■ Methodology: Add applications until the system swaps, then it’s ‘full’ – More applications is better – Per tenant cost is amount of RAM / # tenants
Current Performance Data
0
5
10
15
20
instances
Standard JVM Hand tuned Standard JVM MT support
Standard JVM
0
15
30
45
60
instances
Standard JVM Hand tuned Standard JVM MT support
Best hand-tuned JVM config
0
75
150
225
300
instances
Standard JVM Hand tuned Standard JVM MT support
Simple out of the box -Xmt
4MBtenant
Part 6Wrap up
■ Focus to date has been ‘zero application changes’ – We can do even better with tenant-aware middleware
■ API’s used to provide isolation & throttling are available to stack products – JSR-284 (Resource Management) – JSR-121 (Isolates) – @TenantScope fields
■ Java language and frameworks (EclipseLink) are evolving to have first-class multitenant support
■ Stay tuned for progress: watch the IBM Java 8 beta program
This is still ‘experimental’
This is still ‘experimental’
Simplifying the software stack by removing all extraneous pieces makes better use of hardware (and people who run it). !Multitenancy can make us more efficient:
–Trades isolation for footprint and agility –JVM support makes multitenancy safer and easier –Measuring resource usage and load patterns is critical –Multitenant JDK primitives give us room for future growth
Now that you’ve completed this session, you are able to:
– Understand what multitenancy is and what it’s good for • Per-tenant costs measured in single-digit MB are possible
– Describe challenges of multitenant Java deployments • Hard for VM guys, should be easy for you • Choreography of load / deployment is up to you
– Understand new JDK features to convert existing applications into multitenant deployments • Are we on the right track? Could you use this in your business?
Conclusion
Thank you - any questions?