Making everything better with OSGi - a happy case study in building a really nice runtime - H Cummins

©2014 IBM Corporation

Making everything better with OSGi

a happy case study in building a really nice runtime

Holly Cummins [email protected]

@holly_cummins


Let’s talk about WebSphere.



Awesome capabilities…







… for production.




… for production.

For development, historically not so well-suited.


History


WebSphere Application Server (the full profile) has been around since 1998

History



History

forever



Big codebase

History

forever



Big codebase

Big customer base

History

forever



Big codebase

Big customer base

Big workloads

History

forever



Big codebase

Big customer base

Big workloads

… Big inhibitors to massive change

History

forever



Big codebase

Big customer base

Big workloads


History

This is not a complaint.

This is a problem we arehappy to have.

forever



Big codebase

Big customer base

Big workloads


History

This is not a complaint.

This is a problem we arehappy to have.

But it is still a problem.

forever



Code that has been around forever…



No matter how good you are, or how smart you are




If your code lives long enough, and is used enough, it ends up looking like…





?!





dragons

?!





dragons

?!

Especially code that has roots

going back to late ‘90s…


By Steve Collis from Melbourne, Australia [CC-BY-2.0 (http://

creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

How do you manage dragons?




Clever WebSphere architects





Clever WebSphere architects OSGi



OSGi and WAS: The first pass…

OSGi was included in WAS v6.1, in 2006

Went from lots of arbitrary jars to a few bundles

Achieved some modularity enforced by OSGi

We did not use or expose OSGi services

Compatibility constraints: WAS is the bottom of the stack

Assumptions about resource initialization and availability

Entrenched dependencies between some core elements


Did that fix everything?


Not quite.

Did that fix everything?

By Steve Collis from Melbourne, Australia [CC-

BY-2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons


What if… If we could start over, what would we want?



Developer-friendly



Developer-friendly

Simple



Developer-friendly

Simple

Dynamic



Developer-friendly

Simple

Dynamic

Light-weight



Developer-friendly

Simple

Dynamic

Light-weight

Composable / Flexible



Developer-friendly

Simple

Dynamic

Light-weight


Extensible



Developer-friendly

Simple

Dynamic

Light-weight


Extensible

selectable content

clear API/SPI runtime/app

isolation

human usable configuration


runtimeonly

some combination oftechnologies

!app-centered

everything

What if…

grokable config

provisioning


What if…

grokable config

provisioning

runtimeonly


!app-centered

everything

How do we do this?


What if… runtime

only


!app-centeredeverything

And yet allow this?

no restarts


What if…

And for crying out loud, can we prevent THIS?!

runtime3rd party

bundleB

systembundle

A


What if…


runtime3rd party

bundleB

systembundle

A

Application


What if…


runtime3rd party

bundleB

systembundle

A

Application


What if…


runtime3rd party

bundleB

systembundle

A

Application

X


The egg problem

“Create a light-weight profile of WebSphere that starts in under 2 seconds… Don’t break any eggs”

— Ian Robinson (cleverest WebSphere architect)


The egg problem

“Create a light-weight profile of WebSphere that starts in under 2 seconds… Don’t break any eggs”

— Ian Robinson (cleverest WebSphere architect)

By miya (miya's own file) [CC-BY-3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons


OSGi maturity model


OSGi maturity modelLevel 1: Ad Hoc



Level 2: Modules



Level 2: Modules

Level 3: Modularity



Level 2: Modules

Level 3: Modularity

Level 4: Loose coupling



Level 2: Modules

Level 3: Modularity


Level 5: Devolution



Level 2: Modules

Level 3: Modularity


Level 5: Devolution

Level 6: Dynamism



Level 2: Modules

Level 3: Modularity


Level 5: Devolution

Level 6: Dynamism

Level 7: Peter Kriens



Level 2: Modules

Level 3: Modularity


Level 5: Devolution

Level 6: Dynamism


Websphere Full Profile



Level 2: Modules

Level 3: Modularity


Level 5: Devolution

Level 6: Dynamism


Websphere Liberty Profile


Dealing with our legacy

We needed a new kernel

Used the new base to re-group…

Lots of code still common with full profile

Wrap/Shim: New face on old code

Patch: tweak and replace bits where necessary


Building a kernel from scratch

OSGi-based for all the reasons

First-class use of OSGi services

Must react to configuration changes

Runtime composition on-the-fly


Bundle manifests


Bundle manifests? What manifests?We use Eclipse .classpath to

Determine project build order

Generate ant classpath

Bnd takes care of the rest

Declare services using annotations

Bnd makes overrides really easy

Useful when integrating legacy code


Bundle manifests? What manifests?We use Eclipse .classpath to

Determine project build order

Generate ant classpath

Bnd takes care of the rest

Declare services using annotations

Bnd makes overrides really easy

Useful when integrating legacy code

New developers need to have almost no awareness we’re


ConfigurationSettled on XML for configuration format

Ubiquitous

Expressive

BUT, for simplicity:

single file

usable defaults

<server description=“simple”> <featureManager> <feature>jsp-2.2</feature> </featureManager> ! <httpEndpoint id=“defaultHttpEndpoint” httpPort=“9080” httpsPort=“9443” /> </server>


ConfigurationComposable system requires composable configuration:

Individual components own their config

No centralized repository

No externally defined global config model





No externally defined global config modelConfiguration Admin and Metatype

- job done





No externally defined global config modelConfiguration Admin and Metatype

- job mostly done


Metatype Equinox impl + extensions

Uniform validation of user input

Define configuration and constraints in one place, it gets used everywhere else.

We favor metatype.xml for this reason

Custom namespace for additional types and validators

ibm:type — duration, location, password

pid/reference








pid/reference

human readable: 1h30m converted to unit of

choice








pid/reference


choiceused by developer tools to help prompt for the right kind of

path: file vs. url








pid/reference


choiceused by developer tools to help prompt for the right kind of

path: file vs. url

type=“String” ibm:type=“password”

The value is a “SerializedProtectedString”,

which is not a String.


Using OSGi Services…

?

BlueprintDeclarative Services

yes, there are others. We focused on these two.


Using OSGi Services… Who in their right mind wants to manage OSGi services themselves??

?




Using OSGi Services… Who in their right mind wants to manage OSGi services themselves??

Exactly. NOBODY.?




Declarative ServicesWe chose DS for two main reasons:

Timing: Blueprint and Aries were just getting started

Integration with Configuration Admin and Metatype!

Config injected as one unit

activate/modified/updated methods

Service instance creation based on metatype-declared factory pid

DS target filters can be set via configuration


DS is AWESOME!

DS is a central part of the Liberty runtime

CA + M + DS = “magic”We do insane things with config-derived target filters

Our runtime would not be what it is without DS in the middle of it


But …


Remember this?We need to prevent

THIS

runtime3rd party

bundleB system

bundleA

Application

X


Package privacy


Package privacy

OSGi lets you keep your internals private.


Package privacy


Your externals are public.


Package privacy



PublicPrivate


Package privacy



PublicPrivate


Granularity

bundle C

bundle Abundle B


Granularitybundle D

bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K

bundle Jbundle O bundle P

bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B


Granularitybundle D

bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

Wait a minute, it wasn’t supposed to be that

public …


Granularitybundle D

bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

Wait a minute, it wasn’t supposed to be that

public …


Granularitybundle D

bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B


Granularitybundle D

bund

le F

bund

le E

bundle G

bundle M

bund

le N

bund

le I

bundle K

bund

le Jbu

ndle O

bund

le P

bund

le N

bund

le L

bundle M

bund

le Q

bundle C

bund

le A

bund

le B


Granularitybundle D

bund

le F

bund

le E

bundle G

bundle M

bund

le N

bund

le I

bundle K

bund

le Jbu

ndle O

bund

le P

bund

le N

bund

le L

bundle M

bund

le Q

bundle C

bund

le A

bund

le BWhat bundles do I

actually need?


Granularitybundle D

bund

le F

bund

le E

bundle G

bundle M

bund

le N

bund

le I

bundle K

bund

le Jbu

ndle O

bund

le P

bund

le N

bund

le L

bundle M

bund

le Q

bundle C

bund

le A

bund


actually need?

Could I just have an archive, please?


Granularitybundle D

bund

le F

bund

le E

bundle G

bundle M

bund

le N

bund

le I

bundle K

bund

le Jbu

ndle O

bund

le P

bund

le N

bund

le L

bundle M

bund

le Q

bundle C

bund

le A

bund


actually need?

Could I just have an archive, please?


This is not a new requirement



Lots of attempts to solve this problem




Eclipse features




Eclipse features

Equinox composite bundles




Eclipse features


Apache Aries Enterprise Bundle Archives (.ebas)




Eclipse features


Apache Aries Enterprise Bundle Archives (.ebas)

Liberty features (mark 1)


OSGi Subsystems are AWESOME



A new(ish) OSGi standard




Liberty uses the Apache Aries implementation





Granularity





Granularity

Package isolation





Granularity

Package isolation

Packaging





Granularity

Package isolation

Packaging





Granularity

Package isolation

Packaging





Granularity

Package isolation

Packaging


When better boxes is not enoughbundle D

bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B



bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

bundle M



bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

bundle M



bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

bundle M Extension



bundle F

bundle E

bundle G

bundle H

bundle M

bundle N

bundle I

bundle K


bundle N

bundle L bundle M

bundle Q

bundle C

bundle Abundle B

Resolver hook

bundle M Extension


Sophisticated package visibility

Features explicitly declare API and SPI packages (IBM-* metadata in the feature manifest)

Isolation between API/SPI, apps/extensions/runtime is enforced in a few ways:

Subsystems (the Aries impl) for OSGi Applications (API)

Resolver hooks and/or Eclipse Regions for isolation between runtime, extensions (SPI), and containers (API).


Provisioning


Provisioning

Dynamically install features (subsystems) based on configuration


Provisioning


Adding or removing features …


Provisioning



installs or uninstalls bundles…


Provisioning




which adds or removes configurations…


Provisioning





which triggers the creation or removal of services!


Provisioning





which triggers the creation or removal of services!

No restart needed. !

#really


Did we get rid of all the dragons?



By Abraham (Own work) CC-BY-3.0 via Wikimedia Commons



The OSGi dragon




The OSGi dragon



What version should this be?


What version should this be?This is ALWAYS a hard problem in any non-trivial evolving system



OSGi just makes the problem more obvious




Semantic versioning is …





Beautiful





Beautiful

Insufficient





Beautiful

Insufficient

Our developers never touch versions





Beautiful

Insufficient

Our developers never touch versions

That’s what a release team is for :)


How do I interact with X?Let DS do it. Really.

DS is excellent at managing non-trivial service dependencies

It is very unlikely that you will be able to do better— just let DS do it. That means:

Don’t register services inside a component

Don’t manage references inside a component


Why is my test failing intermittently ?


Why is my test failing intermittently ?

Because something hasn’t happened yet.


Service dynamics can hurt!



Service dynamics are a huge hurdle for “new” developers




DI and IoC can turn even experienced brains inside out if they aren’t prepared. Thankfully, they do seem to recover.





DS helps a LOT





DS helps a LOT

But …


Service dynamics can still hurt.



Being really lazy is hard work!




Easy to make assumptions





Service availability






Event ordering






Event ordering

Especially in our tests


Is the server started?



The server



The server


Is the server started?What does this boundary even mean

in an extensible system?

The server




The server




The server




The server




Care about the services

you care about


What’s the schema for the configuration?



Devolved configuration is great. But ..




Tolerating unrecognised elements is great when your config is correct.





It’s not so good when your config is wrong.






Aggregated documentation is good for users.







Aggregated documentation is (sort of) needed for tooling







Aggregated documentation is (sort of) needed for tooling

We generate an xsd at build-time so tooling can validate user config


Is it worth it?


Is it worth it?OH YES.



Our server is GREAT.




Starts in 3s …




Starts in 3s …

… which we never have to do anyway…




Starts in 3s …


… because it’s fully dynamic




Starts in 3s …


… because it’s fully dynamic

Concise, shareable configuration


Did it all turn out the way we expected?

A funny thing happened while we were developing for developers …


OSGi was designed for …


OSGi was designed for …

Small embedded system


… but turned out to be perfect for …


… but turned out to be perfect for …


Liberty was designed for …


Liberty was designed for …

Developer-scale


… but turned out to be perfect for…



BIG scale


BlueMix server farm


BIG scale


BlueMix server farm


BIG scale

Lots of Liberty instances


Thank you!

Questions?

@holly_cummins www.wasdev.net

http://www.wasdev.net


Thank you!

Questions?

Oh, and I have a book!

@holly_cummins www.wasdev.net

http://www.wasdev.net


Backup


</cleanSlate>Of course, we didn’t really get a clean

slate.Application compatibility had to be

preserved. !


Configuration Admin We rolled our own (sorry)

Parse and merge user configuration and bundle-provided defaults

Resolve variables

Provide configuration to consumers as required by the spec (mostly)







pid/reference

unique, final, variable, etc.


http://www-01.ibm.com/support/knowledgecenter/was_beta_liberty/com.ibm.websphere.wlp.nd.multiplatform.doc/ae/rwlp_extensions_osgi_metatype.html

human readable: 1h30m converted to unit of used by developer tools to

help prompt for the right kind of path: file vs. url







pid/reference




type=“String”ibm:type=“password”

The value is a “SerializedProtectedString”,

which is not a String.







pid/reference




This is some crazy stuff.

ibm:type=“pid” ibm:reference=“specific.service.pid”

Allows nested configuration elements

to define service relationships


IsolationWe mean this in a good way.

Liberty runtime serves two masters:

Typical Application Server paradigm(apps strictly separated from runtime) — API

Platform extender paradigm(the “app” is the runtime) — SPI

Persistent problem:how to allow apps or extensions to use their own versions of libraries that don't conflict with the runtime!?


Private-package declarations pull in redundant classes

Split packages never end well

Technology

Making everything better with OSGi - a happy case study in building a really nice runtime - H Cummins