OSGi Enablement for Apache Tuscany

Raymond Feng

rfeng@apache.org

Overview

Motivations

• Provide modularity for Tuscany to formalize/enforce the SPI contracts and minimize the package dependencies across modules

• Enable Tuscany to work with OSGi environment such as JEE application servers, Eclipse RCP or Spring DM

• Provide versioning and isolation so that Tuscany extensions can depend on different versions of the same library

• Provide lifecycle management for extensions: install/uninstall/start/stop a module

Objectives

• Tuscany runtime will be able to run in both non-OSGi and OSGi environment– No OSGi APIs should be used for non-OSGi

specific modules

• Build a consistent OSGi story to facilitate developing, building, launching, running and testing Tuscany– Makes developers’ life a bit easier to work

with OSGi

What problems do we try to solve?

• Turning Tuscany modules into OSGi bundles

• Turning 3rd party dependencies into OSGi bundles

• Developing, building, launching, running and testing with OSGi

• Producing distributions that are compatible with the OSGi bundle structure in an efficient way (in one-two minutes)

Turning Tuscany modules into OSGi bundles

Creating bundles for Tuscany modules

• Modularize Tuscany after the maven modules– One OSGi bundle per maven module– Be consistent for build, packaging and runtime– Much easier to aggregating than decomposing

• Developers are responsible for authoring and maintaining MANIFEST.MF (instead of MF generation tools)– Tooling friendly– Developers pay more attentions to the dependencies

Things to consider to create an OSGi bundle for a module

• Add META-INF/MANIFEST.MF (don’t stop here …)

• Clean up module dependencies and package visibilities

• Prepare for code changes– Service Discovery– ClassLoading

Rules of thumb• Identify the absolutely necessary SPI packages that need to be

exported – Don't try to export everything – For model modules, try to only export the package that contains the

interfaces – Try not to export any package from the xxx-runtime moudles – Any classes that can be accessed via the ExtensionPointRegistry using

interfaces should not be exported • Only import other packages when it's needed

– For example, we used to have a lot of modules pull in assembly for just a constant for the SCA namespace

– Avoid DynamicImport-Package=* – Avoid Require-Bundle

• Do not export "private" packages as a workaround or hack which can fail the "clean SPI". Refactor things into SPIs if necessary

Granularity Discussion

• 180+ bundles cumbersome• Multiple bundles required to enable one

capability• Much debate about right level of granularity

(flexibility vs. usability) • Some opinions

– Fine-grained bundles suitable for developer view– Features/Profiles used to “group” bundles to provide a

user view• Inspired by Eclipse Features

Dealing with 3rd party jars

Handling 3rd party jars

• We have to live with the reality:– Many 3rd party jars are not OSGi bundles. No repo is

available to get all converted bundles• We need to convert them (build time, run time)

– Some of the 3rd party bundles are broken (w/ bogus MF)

• We need to fix them (replacing the MF)

– More and more 3rd party jars are upgraded to be OSGi bundles

• We should be able to incrementally consume 3rd party jars as OSGi bundles

Converting 3rd party jars to OSGi bundles

• A folder bundle is created to represent a 3rd party jar

• The folder structure looks like:– jaxb-impl-2.1.12

• META-INF– MANIFEST.MF

• jaxb-impl-2.1.12.jar

• It is non-invasive and flexible– The original jar is not changed (avoid legal, signature issues)– Multiple jars can be easily aggregated (control the bundle granularity)– The MANIFEST.MF can be easily customized

third-party.jar

MANIFESTBundle-Classpath: third-party.jarDynamic-Import-Package: *Export-Package: ......

third-party.jar

Install

OSGi Framework

third-party-osgi.jar

third-party.jar

MANIFESTBundle-Classpath: third-party.jarImport-Package: ...Export-Package: ......

Sample META-INF/MANIFEST.MFManifest-Version: 1.0Bundle-ManifestVersion: 2Bundle-SymbolicName: com.sun.xml.bind.jaxb-implBundle-Name: com.sun.xml.bind.jaxb-implBundle-Version: 2.1.12DynamicImport-Package: javax.transaction;version="1.1",javax.transacti on.xa;version="1.1",*Bundle-ClassPath: jaxb-impl-2.1.12.jarExport-Package: com.sun.xml.bind.v2.schemagen;version=2.1.12,com.sun.x ml.bind.v2.util;version=2.1.12,com.sun.xml.txw2.annotation;version=2. 1.12,com.sun.xml.bind.v2.model.runtime;version=2.1.12,com.sun.xml.bin d.v2.runtime;version=2.1.12,com.sun.xml.txw2.output;version=2.1.12,co m.sun.xml.bind.v2;version=2.1.12,com.sun.istack;version=2.1.12,com.su n.xml.bind.v2.runtime.property;version=2.1.12,com.sun.xml.bind.v2.run time.unmarshaller;version=2.1.12,com.sun.xml.bind.v2.model.nav;versio n=2.1.12,com.sun.xml.bind.marshaller;version=2.1.12,com.sun.xml.bind. v2.runtime.reflect.opt;version=2.1.12,com.sun.xml.bind.v2.runtime.out put;version=2.1.12,com.sun.xml.txw2;version=2.1.12,com.sun.xml.bind.v 2.schemagen.xmlschema;version=2.1.12,com.sun.xml.bind.v2.runtime.refl ect;version=2.1.12,com.sun.xml.bind.v2.model.impl;version=2.1.12,com. sun.xml.bind.unmarshaller;version=2.1.12,com.sun.xml.bind.util;versio n=2.1.12,com.sun.xml.bind;version=2.1.12,com.sun.istack.localization; version=2.1.12,META-INF.services;partial=true;mandatory:=partial,com. sun.xml.bind.v2.model.annotation;version=2.1.12,com.sun.xml.bind.api. impl;version=2.1.12,com.sun.xml.bind.api;version=2.1.12,com.sun.xml.b ind.v2.model.core;version=2.1.12,com.sun.xml.bind.v2.bytecode;version =2.1.12,com.sun.xml.bind.v2.schemagen.episode;version=2.1.12,com.sun. xml.bind.annotation;version=2.1.12

Developing Tuscany with OSGi

Checking out source code and building with maven

• Check out the Tuscany source code from SVN– svn co https://svn.apache.org/repos/asf/tuscany/java/sca/

• Run maven build– mvn clean install -fn

• Generate Eclipse projects– mvn -Peclipse

Setting up Eclipse PDE

• Download Eclipse (w/ plugin development)– http://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/ga

lileo/SR1/eclipse-jee-galileo-SR1-win32.zip

• Configure PDE target platform– File Open File

• <tuscany>\sca\distribution\all\target\features\tuscany-pde35.target

– Click on “Set as target platform”

• Load Tuscany modules into Eclipse– File Import … General Import Existing

Projects …

What’s happening during the Tuscany build?

• Validating OSGi constraints to report bundle resolution issues during compilation

• Generating PDE project so that Tuscany modules can be loaded in Eclipse to leverage the nice OSGi tooling

• Generating OSGi bundles for 3rd party jars• Generating PDE target platform definitions so that 3rd

party bundles can be used in Eclipse as plugin dependencies

• Generating Equinox configuration so that Tuscany distribution can be directly loaded as an OSGi framework

Maven/OSGi Tools provided by Tuscany

• Maven-bundle-plugin– Generates bundles for 3rd party jars– Generates OSGi enabled PDE projects– Version 1.0.4 released at 10/09/2009

• Maven-eclipse-compiler– Expose OSGi aware Eclipse JDT compiler– Expose Eclipse OSGi bundle validation– Version 1.0.1 released on 04/07/2009

• Maven-osgi-junit-plugin– JUnit tests in a OSGi enabled environment– Version 1.0 released on 10/28/2009

• https://svn.apache.org/repos/asf/tuscany/maven-plugins/trunk/

Development Time

• Eclipse PDE provides nice OSGi tooling– Graphical editor for MANIFEST.MF (w/

additional tools to analyze/calculate dependencies)

– Plugin Dependencies Classpath Container– Compilation errors/warning for OSGi

constraint violations (w/ Quick Fix suggestions)

– OSGi test environment to validate and launch OSGi framework (w/ console)

Runtime

• Provide a SCA node launcher to start an OSGi runtime and run SCA application with the OSGi-enabled Tuscany runtime

• We need to use Tuscany's service discovery mechanism to discover and instantiate the factories

• Change how JDK factories such as XMLInputFactory, XPathFactory, and DocumentBuilderFactory, are discovered and instantiated

• XMLInputFactory.newInstance() used the JSE service provider pattern which doesn't work with OSGi

• Adjust the code that assumes there is a flat thread context classloader that can access all the classes in the Tuscany runtime

• With OSGi, there is a network of classloaders (one classloader per module) and the class visibility is constrained by the OSGi headers such as Import-Package and Export-Package

• The classloader for the current class is a good starting point for Class.forName() with OSGi

Launching OSGi framework within Eclipse PDE

Launching OSGi from Tuscany distribution

• Equinox:– java -jar osgi-3.5.0-v20090520.jar -clean -console -configuration ..\features\configuration

• http://cwiki.apache.org/confluence/display/TUSCANYxDOCx2x/Running+Tuscany+SCA+2.x+with+Equinox+and+Felix

Testing Tuscany with OSGi

A few options

• Tuscany maven-osgi-junit plugin– Generate two bundles for the main and test classes respectively.

The test bundle is a fragment of the main bundle. • For example, the plugin generates the following bundles for sample-

calculator-osgi.jar: * sample-calculator-osgi-osgi.jar * sample-calculator-osgi-osgi-tests.jar

– Find the EquinoxHost class from tuscany-node-launcher-equinox and create an instance

– Start the equinox runtime– Set up the classloader for surefire so that it uses OSGi

classloading for the test cases with the test bundle. – Delegate to surefire to run the unit tests

• Apache Felix junit4osgi – http://felix.apache.org/site/apache-felix-ipojo-junit4osgi.html

Trouble-shooting OSGi related issues

Common traps

• System packages need to be explicitly imported, such as javax.xml.namespace

• Be careful of TCCL (ThreadContext Classloader)– No well-defined TCCL in OSGi

– Equinox uses ContextFinder to find the closest class on the calling stack that is loaded by an OSGi bundle

– JDK and some 3rd party libraries uses service discover pattern relying on TCCL being able to load/see everything

Some tips

• Most common exceptions:– ClassNotFoundException– NoClassDefFoundError

• Bundle is not resolved:– Missing Import-Package (including system packages

from the JDK)– Split package from Export-Package– Different versions of the same package are exported– Chain of bundle resolution (Exported packages from

unresolved bundle cannot be seen.)

Useful Links

• http://cwiki.apache.org/autoexport-2.2.9/TUSCANYxDOCx2x/osgi-enablement-for-tuscany-runtime.html

• Infoq.com articles:– http://www.infoq.com/articles/modular-java-what-is-it– http://www.infoq.com/articles/modular-java-static-modularity– http://www.infoq.com/articles/modular-java-dynamic-modularity

• http://www.dynamicjava.org/