Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
© 2010 IBM Corporation
Multitenant JVM
Ryan Sciampacone – Senior Software Developer
August 1st 2012
JVM Languages Summit2012
© 2010 IBM Corporation2
Important Disclaimers
§ THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONALPURPOSES ONLY.
§ WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THEINFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUTWARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
§ ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN ACONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE,SOFTWARE OR INFRASTRUCTURE DIFFERENCES.
§ ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.
§ IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBM’SCURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM,WITHOUT NOTICE.
§ IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGESARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANYOTHER DOCUMENTATION.
§ NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECTOF:
§ - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES ORITS OR THEIR SUPPLIERS AND/OR LICENSORS
© 2010 IBM Corporation3
What’s this talk all about anyways?
Look at the challenges and technical details involved in fitting amultitenancy story onto the Java runtime. This will include goals, basicbuilding blocks, and how existing problems are changed or magnified byhaving multiple tenants.
© 2010 IBM Corporation4
Dude, what the …?!
This is JVMLS!
© 2010 IBM Corporation5
Multitenancy == Simplification
Multitenancy refers to a principle in software architecture where a single instance ofthe software runs on a server, serving multiple client organizations (tenants).
Multitenancy is contrasted with a multi-instance architecture where separate softwareinstances (or hardware systems) are set up for different client organizations.
With a multitenant architecture, a software application is designed to virtually partition itsdata and configuration, and each client organization works with a customized virtualapplication instance.
Thanks to
© 2010 IBM Corporation6
Boiling Multitenancy Down
§ Working Definition– A single instance of a software application that serves multiple customers. Each
customer is a tenant.– Generally tenants can customize some parts of the application (look & feel, security,
business rules) but not the code.– Infrastructure usually opaqueà opportunity for provider
Why? Cost Savings: As compared to single-tenant deployment model
© 2010 IBM Corporation7
Multitenancy : Trading Isolation for Density
Hardware
OS Images
Middleware
Application
Hardware
OS Images
Middleware
Application
Hardware
OS Images
Middleware
Application
OS Images
Middleware
Application
Hardware
OS Image
Middleware
Application
Middleware
Application
Hardware
OS Image
Middleware (e.g. WAS)
Application Application
Share-nothing(maximum isolation)
Shared hardware Shared OS Shared Process
Hardware
OS Image
Middleware (e.g. WAS)
Application
Tenant Tenant
Share-everything(maximum sharing)
Isolation
Density
Isolate API
‘Mission critical’apps
‘free’ apps
1+ GB / tenant 100’s MB / tenant1+ GB / tenant 10’s MB / tenant 10’s KB / tenant
© 2010 IBM Corporation8
Challenge: Lowering the Cost of Entry
§ We need to fix the following– Data Isolation between applications– Control over resource hogs
§ Without forcing people to change their applications! ApplicationChanges
© 2010 IBM Corporation9
Multitenant JDK: Easy isolation and control
§ Concept: Add a single argument (–Xmt for multi-tenant) to your Javacommand-line to opt into sharing a runtime with others.
§ Result: Your application behaves exactly as it if had a dedicated JVM, but inreality it runs side-by-side with other applications.
§ Benefits: Smaller, faster, and eventually smarter– Less duplication (1 GC, 1 JIT), Heap object sharing– JVM is already running and warm when starting apps
© 2010 IBM Corporation10
Multitenant JDK: Launch your application
§ Opt-in to multitenancy by adding –Xmt
© 2010 IBM Corporation11
Multitenant JDK: Register with javad daemon
§ JVM will locate/start daemon automatically
javad
locate
© 2010 IBM Corporation12
Multitenant JDK: Create a new tenant
§ New tenant created inside the javad daemon
javad
Tenant1
© 2010 IBM Corporation13
Multitenant JDK: Create a second tenant
§ New tenant created inside the javad daemon
javad
Tenant1
Tenant2
One copy of common codelives in the javad process.
Most runtime structuresare shared.
© 2010 IBM Corporation14
Middleware == Master Tenant
§ Shared data structures are part of the “global economy”– Growing a shared Hashtable
§ The middleware effectively acts as a tenant– Looser restrictions (?)– Still tries not to interfere with everyone else
§ Consistency in ownership– An object is owned by a Tenant– Private resources may percolate to the global level
Tenant1
Tenant2
Master Tenant
© 2010 IBM Corporation15
You need an Identifier: TenantContext
§ Basic operations on Tenants available to the middleware– Data Isolation– Resource Management
§ Ability for the middleware to differentiate between Tenants– Which one is causing the problem?
§ Querying the state of Tenants– How much free memory do you have?
Data IsolationData IsolationData Isolation
Create & QueryCreate & QueryCreate & Query
Resource ManagementResource ManagementResource Management
© 2010 IBM Corporation16
Data Isolation Challenges
§ Static variables are bad (for sharing)
§ Most libraries are full of static variables
Wait! What happens ifeach tenant needs adifferent default locale?
© 2010 IBM Corporation17
Isolating Statics Through Class Loaders
§ Isolation through the usual class loader tricks– Every time you load a tenant, stick it in a new class loader– Statics are now no longer shared!
ClassLoader
ClassAClass
AClassA
Tenant1
ClassLoader
ClassAClass
AClassA
Tenant2
ClassLoader
ClassAClass
AClassX
Master Tenant
© 2010 IBM Corporation18
Isolating Statics Through Class Loaders
§ Isolation through the usual class loader tricks– Every time you load a tenant, stick it in a new class loader– Statics are now no longer shared!
ClassLoader
ClassAClass
AClassA
Tenant1
ClassLoader
ClassAClass
AClassA
Tenant2
ClassLoader
ClassAClass
AClassX
Master Tenant
§ Plenty of duplication!– Class data– JITted code– Duplicated static variables, most of which are likely identical
© 2010 IBM Corporation19
Isolating Statics Through Class Loaders
§ Isolation through the usual class loader tricks– Every time you load a tenant, stick it in a new class loader– Statics are now no longer shared!
ClassLoader
ClassAClass
AClassA
Tenant1
ClassLoader
ClassAClass
AClassA
Tenant2
ClassLoader
ClassAClass
AClassX
Master Tenant
§ Per-Tenant “scoping” of master tenant data!
© 2010 IBM Corporation20
Isolating Statics through Bytecode Re-writing
§ Let’s just focus on the statics– Rewrite the bytecodes!
§ Use an data structure for each static
§ Use property access style to get at statics
§ Still error prone: Class load rules, Reflection, JNI, Other bytecode rewriters
§ Ok, so you need to start marking things as sharable (default) vs. isolated (new)
…
<Tenant1 value>
Tenant1 Contextstatic index
© 2010 IBM Corporation21
Solving the Data Isolation Challenge
§ What if … the JVM knew about tenants and provided each one with a different view ofstatic variables?
§ Meet the @TenantScope annotation.
§ @TenantScope Semantics: Static variable values are stored per-tenant
§ Each tenant has their own LocaleSettings.defaultLocale
§ Now many tenants can share a single LocaleSettings class
…LocaleSettings.setDefaultLocale(LocaleSettings.UK );
…
…LocaleSettings.setDefaultLocale(LocaleSettings.UK );
…
Tenant1
Tenant2
…LocaleSettings.setDefaultLocale(LocaleSettings.USA );
…
…LocaleSettings.setDefaultLocale(LocaleSettings.USA );
…
© 2010 IBM Corporation22
Multitenant JDK: Sharing without Changing
§ @TenantScope markup gets added automatically as classes are loaded
§ Tenants see dedicated middleware – but behind the curtains all classes are actuallyshared
merge
Hardware
OS Image
Middleware (e.g. WAS)
Application Application
Hardware
OS Image
Middleware (e.g. WAS)
Application
Tenant Tenant
Isolate API
ApplicationChanges
© 2010 IBM Corporation23
Per-Tenant Heap Consumption
§ IBM JDK’s have new region-based GC technology which maps nicely to tenants(more @ http://ibm.co/JtWfXr)
§ Technique:– Each tenant is initially given enough GC regions to satisfy its minimum reservation– Code running in tenant scope allocates objects in a region it owns– New regions can be requested up to tenant maximum reservation
§ Details:– Finalization needs to run in the proper tenant context– We must be able to map from an objectà tenant easily– GC read/write barriers provide an opportunity to control inter-tenant references
221111
heap (divided into regions)
Tenant1 Tenant2
© 2010 IBM Corporation24
Hey! We’re done!
Right?
Image source: www.wallpaperslibrary.com
© 2010 IBM Corporation25
Neighbourhood Watch: Dealing with bad behaviour
images from http://www.rra.memberlodge.org/Neighbourhood-Watch-Reportinghttp://mcsholding.com/DetailsPage.aspx?Page_Id=42
http://bit.ly/ficwkl
© 2010 IBM Corporation26
Image source: http://bit.ly/ND07gz
© 2010 IBM Corporation27
Java is Memory: That might be a problem…
§ How do we measure each tenants memory consumption?
§ Shared memory across Tenants– Jar files (R/O class data)
§ Levels of contribution to global data– Shared data structures (add/remove/grow)
§ Implicit vs. Explicit creation of data– String constants– Growth of data structures
§ Querying of System Information– Where does java.lang.management fit in? System.freeMemory() ?
§ Configuration– Quota’s governing maximums, levels of contribution– Simple question: How do you handle a tenant deployment with –Xmx / -Xms options?
§ Associated CPU (and other non memory) costs?– Garbage Collection– Finalization
© 2010 IBM Corporation28
Dealing with Bad Behavior
§ Scenario: Application needs to be taken down quickly– Bugs – Misbehavior, runaway consumption of resources
§ Operating Systems / Hypervisors great at this– kill -9, ctrl-c, task manager and kill– Shutdown guest O/S
§ JVM not so much– Thread.stop() is a non-starter– Implement TM semantics on all data structures?
§ Worse Case Scenario: 1 Tenant crashes the JVMà All Tenants are torn down
§ Without co-operation, tear down of a single Tenant is a challenge
§ Throttling : Consider slowing down / restricting Tenants with bad behavior
§ The closer your neighbours are the better your controls must be
© 2010 IBM Corporation29
JSR-284 Resource Consumption Mgmt API
§ Throttling at Java layer for portability
§ Or, leveraging OS WLM directly for efficiency (Linux & AIX)
Network: XXXKB/S
Hardwareresources
CPU: XXX GHZ DISK: XXXKB/SMemory: XXX GB
OS Level Resources Management
Thread
SocketHandler
OS Workload Manager (WLM)
OS resources
JVM Resource Management
Resource Throttle Layer
JSR 284 API
GC(Heap Mgmt)
Tenant Tenant Tenant Tenant Tenant
OS
JVM
Resource native API
Memory CPU Thread File I/O Socket I/O CPU File I/O Socket I/O
© 2010 IBM Corporation30
JVM vs. Operating System CPU Throttling
Benchmark Setting
§ Accuracy Comparison: Run two CPU-intensive apps each doing the same Fibonaccicalculation, but with different CPU quota: 60% vs 30%, benchmark the accuracy
§ Note: Duration in both OS and JVM controlled environments were roughly the same
Result: JVM control achieves comparable performance, but less accuracy.
© 2010 IBM Corporation31
Shared State Outside the JVM
§ Native libraries contain state that may not be shareable across tenants
§ Could rewrite the libraries to support tenants– Exactly what we are trying to avoid– May not even be possible (don’t have the source)
§ Remember java –mt –jar one.jar ?
Tenant1
Tenant2
JVM
SharedLibrary
SharedState!
© 2010 IBM Corporation32
Shared State Outside the JVM
§ Use separate processes to manage different state
§ Each process now holds the context– Cheap relative to JVM cost– Limits on OS capabilities (1000’s of processes)
§ Interesting possibilities– Base class library support– Cross platform support
§ Challenges– Latency
Tenant1
Tenant2
JVM
ProxyLibrary
SharedLibrary
SharedLibrary
© 2010 IBM Corporation33
Locking Interactions Between Tenants
§ Collisions between tenants on “shared” objects– Java level: Global structures which require synchronized contribution– VM level: String constant table
§ Approaches to locking– Is spinning appropriate anymore?– Tenant Local Locking vs. Global Locking
§ Atomic field access appropriate for 1 tenant but not for many tenants– Code was only expected to be run by 1 or 2 threads– Should be a shared resource?
§ Need to potentially consider different implementations for existing data structures– Starting using concurrent equivalents of known data structures (e.g., Hash tables)
§ Potential for deadlock between different tenants – Debug story is required
© 2010 IBM Corporation34
Debugging a Tenant
§ Connecting the debugger to a single tenant– Is this something we want to allow?
§ How do you handle stepping through code using shared data?– Master tenant is using a single data structure for all tenant values?– Have to show some opaque value perhaps when stepping?
§ What if you “breakpoint” in the middle of holding a shared monitor?– Need to be prepared for all tenants to “stop”
§ It may well be that if you are supporting debugging in this environment, you must trust things– The debugger connecting is trusted– So don’t allow debugging on shared / super high density deployed systems
§ How do you support other things like Profiling / Monitoring tools?– Profile and you likely need to be on a trusted system– But still need support for safe tooling for “production” systems
© 2010 IBM Corporation35
Keeping Things Separated: Post Mortem Problem Determination
§ Scenarios– Your VM crashes and you are left with a core file / system dump– You get a heap dump for object liveness / connectivity
§ How do you debug this?– Single tenant view point required, dealing with 1000’s of other tenant data
§ Becomes a runtime / middleware issue– Dump ONLY that data associated with your tenant– What about global shared state?– Core file (etc) rewriters?– Proxy process to help with custom native data?
§ Challenge: Core files are still an issue– “Scrubbing” of details from heap– What about native artifacts? Threads / shared libraries?
§ Tenant space identity within the software stack (finding / separating tenants quickly)– Debugging middleware issues related to tenants– Tenant is causing DB issues – how do you identify the DB transactions to a particular tenant?
© 2010 IBM Corporation36
Conclusions
§ Think deeper and have the checks and support in the runtime
§ Want to build strong walls– Today lots of applications share common app services– We’re talking about introducing language features for compile time walls– Java EE exploring options
§ May cause us to ask more of the OS and HV– Not necessarily tighter dependencies as much as better APIs for collaboration
§ There are big wins to be found!
§ Barely scratching the surface here