The Eclipse Virgo Experience
at CME Group
Jan Fetyko03/29/2012
© 2012 CME Group. All rights reserved 2
CME Group
CME Group, formerly known as Chicago Mercantile Exchange Holdings Inc., was founded in 1898 and operates the CME, CBOT, NYMEX, and COMEX regulatory exchanges worldwide.
It is the world’s leading derivatives marketplace with ~2500 full time employees and a market capitalization of over $15B.
http://www.cmegroup.com/
© 2012 CME Group. All rights reserved 3
• About our application
• Development strategy
• Getting it running in Virgo
• Web Layer
• Impact on our team
Topics
© 2012 CME Group. All rights reserved 4
• Surveillance tool – monitors the state of exchange and the market
• Rewrite of an existing (legacy J2EE) app
• Many services
• Web application(s) that use the services
• Uses: Spring, JDBC, LDAP, EHCache, H2, c3p0, Apache Tiles, snaps
About our application
Development strategy
© 2012 CME Group. All rights reserved
Split all functionality into services / bundles
• Defined interfaces for services
• Skeletons only
6
© 2012 CME Group. All rights reserved
Implemented the interfaces
• NO OSGi
• Not running in Virgo
• Unit tests with mocks
7
Getting it running in Virgo
© 2012 CME Group. All rights reserved
Running in Virgo
Most bundles deployed and started without problems
Exposed and used implementations
3rd party libs a big pain
9
© 2012 CME Group. All rights reserved
Breaking in Virgo Exposed and used implementations
10
Unable to satisfy dependencies of bundle 'impl-use' at version '1.0.0': Cannot resolve: impl-use
Resolver report:
An Import-Package could not be resolved. Caused by missing constraint in bundle <impl-use_1.0.0>constraint: <Import-Package: com…some.internal; version="0.0.0">
Solution
• Anything internal needs to stay internal• Even a factory should be a “service”: Interface + Impl• Should prevent others to see what is inside the bundle
© 2012 CME Group. All rights reserved 11
• Not OSGi-fyied• Works great as long as no replication is needed• Replication using RMI probably works fine• Replication setup fails if using jgroups – it is loaded using
Class.forName(…)cacheEventListenerFactory.setClass("net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory");
• Ehcache-jgroups classes are not in Import-Package, it will throw ClassNotFoundException
• Also, ehcache-jgroupsreplication.jar is not OSGi-fyied
Solution
• Use bnd tool to modify ehcache jar(s)• Create a bnd tool accepted settings file• Push the new ehcache jar into common place (maven repo, git)
Breaking in Virgo 3rd party libraries : EHCache
© 2012 CME Group. All rights reserved 12
Bundle-SymbolicName: ehcache-core-2.3.1Bundle-Version: 2.3.1Bundle-Name: ehcache-core-2.3.1Bundle-ManifestVersion: 2Implementation-Version: 2.3.1Implementation-Title: EH Cache CoreExport-Package: *;version=2.3.1Import-Package: org.jgroups.util,*
Breaking in Virgo 3rd party libraries : EHCache
ehcache bnd tool properties
© 2012 CME Group. All rights reserved 13
Bundle-SymbolicName: net.sf.ehcache.jgroupsreplicationBundle-Version: 1.4Bundle-Name: net.sf.ehcache.jgroupsreplicationImplementation-Version: 1.4Implementation-Title: EH Cache jGroups ReplicationExport-Package: *;version=1.4
Breaking in Virgo 3rd party libraries : EHCache
bnd tool properties for ehcache-jgroupsreplication
© 2012 CME Group. All rights reserved
What is happening in caching service
CachingService
On Node 2
XYZService
On Node 1
ObjectsObjects
Serialize
010011001010110010011001010110
Deserialize
ClassNotFoundException in ehcache bundle
CachingService
On Node 1
© 2012 CME Group. All rights reserved 15
• Replication is possible only for objects known to ehcache• Deserialization of keys and values fails with ClassNotFoundException,
because your classes are not in the Import-Package
Solution
• Store only objects that are known to ehcache – serialize and deserialize them manually
Breaking in Virgo 3rd party libraries : EHCache
© 2012 CME Group. All rights reserved 16
Solution - continued
Public interface CachingService { CacheElement get(CacheKey key)}
public interface CacheElement { Object getValue(Class<?> classLoadingClass);}
Breaking in Virgo 3rd party libraries : EHCache
© 2012 CME Group. All rights reserved 17
• It is easier than bnd tool• Faster results• You will learn a lot about MANIFEST.MF which is essential to know OSGi
anyway• Most changed settings
• Import-Package• Export-Package• Version(s)
Breaking in Virgo 3rd party libraries : MANIFEST.MF
After hacking the MANIFEST.MF
• Use bnd tool to have repeatable results
Web Layer
© 2012 CME Group. All rights reserved 19
Web Layer
host
Virgo snaps
© 2012 CME Group. All rights reserved 20
Web Layer
host
Apache Tiles & Spring MVC
Shared Tiles Definitions
Snap Tiles definitionsextend hostdefinitions
Fragments: content
© 2012 CME Group. All rights reserved 21
Web Layer Apache Tiles
Host
TileTile
Tile fragmentTile fragment
© 2012 CME Group. All rights reserved 22
• Design for modularity• Understand class loading• Understand Manifest• If possible use 3rd party libs that are OSGi ready• Avoid serialization across bundles
Running in Virgo Mini Lessons learned
• For Virgo 2.1, we had to dig through the snaps source and request a tag for a version that worked in Virgo 2.1 (currently on Virgo 3.0.1)
• Snaps + Tiles not the best combination• Provides good flexibility• Snaps provides a single point of entry (filters, security, etc.)
Web Layer
ToolingDevelopmentDeploymentetc.
© 2012 CME Group. All rights reserved
Tools
• STS (SpringSource Tool Suite)
• Apache maven + bundlor plugin (SpringSource)
• maven archetypes to create services and web bundles
• Flyway for database versioning
• Firefox + firebug
• No Virgo tooling or Libra
24
© 2012 CME Group. All rights reserved
Developer Workflow
• Maven + profiles for easy deployment
• Custom plans in virgo while working on a bundle
<plan name="com.cme.security.plan” ….>
<artifact type="bundle" name="com.cme.web.security" version=”…" />
</plan>
25
© 2012 CME Group. All rights reserved
Deployment
• QA and up: Package and deploy everything including Virgo every time
• Everything is scripted including DB schema update (flyway)
• QA time is reduced because changes are only made to a well defined set of bundles
• Risk of running the wrong code in production is reduced to zero by not including the bundle in the deployment package
– We have different deployment targets for 2 different internal customers. Their code cannot impact or risk being deployed together in production.
26
© 2012 CME Group. All rights reserved
Team
• OSGi is not the easiest to understand for monolithic app developers
• Virgo is just like any other container to most users
• Once basic concepts are established, there are no issues
• There are few experts who can help everybody else
• 99% of code is not different from a non-OSGi code
• We started with 2 people using Virgo, now there are 16 at CME and 6 in Brazil
• Today we have 37 service bundles and 23 web bundles
27
Conclusions of the Eclipse Virgo experience
© 2012 CME Group. All rights reserved
Virgo experience conclusions
• Did not hit any Virgo bugs (yet), stable, didn’t experience any crashes
• Problems only come from our code
• Virgo 3.x release improved memory consumption comparing to 2.1
• The learning curve is steep
• Design for modularity upfront is important
• Cannot go back to monolithic app
29
© 2012 CME Group. All rights reserved
Give Feedback on the Sessions
1 Sign In: www.eclipsecon.org
2 Select Session Evaluate
3 Vote
© 2012 CME Group. All rights reserved 32
Futures trading is not suitable for all investors, and involves the risk of loss. Futures are a leveraged investment, and because only a percentage of a contract’s value is required to trade, it is possible to lose more than the amount of money deposited for a futures position. Therefore, traders should only use funds that they can afford to lose without affecting their lifestyles. And only a portion of those funds should be devoted to any one trade because they cannot expect to profit on every trade.
The Globe Logo, CME®, Chicago Mercantile Exchange®, and Globex® are trademarks of Chicago Mercantile Exchange Inc. CBOT® and the Chicago Board of Trade® are trademarks of the Board of Trade of the City of Chicago. NYMEX, New York Mercantile Exchange, and ClearPort are trademarks of New York Mercantile Exchange, Inc. COMEX is a trademark of Commodity Exchange, Inc. CME Group is a trademark of CME Group Inc. All other trademarks are the property of their respective owners.
The information within this presentation has been compiled by CME Group for general purposes only. CME Group assumes no responsibility for any errors or omissions. Although every attempt has been made to ensure the accuracy of the information within this presentation, CME Group assumes no responsibility for any errors or omissions. Additionally, all examples in this presentation are hypothetical situations, used for explanation purposes only, and should not be considered investment advice or the results of actual market experience.
All matters pertaining to rules and specifications herein are made subject to and are superseded by official CME, CBOT, NYMEX and CME Group rules. Current rules should be consulted in all cases concerning contract specifications.