Cloud & OSGi - The Dawn of Composite Clouds (Now with demo videos)

Transforming the Way the World Runs Applications

Copyright © 2011 Paremus Ltd.May not be reproduced by any means without express permission. All rights reserved.

Service Fabric Overview March 2011

Cloud & OSGi - The Dawn of Composite Clouds

[email protected] Nicholson

mailto:[email protected]




Mythology, marketing...

:-)

:-(



Cloud Computingaddresses this

?

The Problem is...

Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010



✗ Deployment of static software images

✔ Dynamic Assembly composites applications

Modular Systems are Maintainable Systems

Anne Thomas Manes (Gartner) – SOA Symposium: Berlin, October 2010



IaaS

SaaS

Introducing the Paremus Service Fabric

(1..m) ‘Systems’ may run upon a single Service Fabric

(1..n) compute resource may contribute to a Service Fabric

Fabric ‘EclipseCon’ = ∑ all ‘EclipseCon’ fibres

An OSGi™ technology based cloud runtime



Off Fabric node

Bootstrap Atlas ManagerSource Repository for Examples

POSH CLIApplication UI

Service Fabric Nodes

Service Fabric InfrastructureCached Examples

Deployed Systems

Demo EnvironmentA ‘blue’ Atlas Agent



Managing characteristics of population

rather than each individual node has a

dramatic effect on decreasing runtime

complexity -

http://adaptevolve.blogspot.com/

2008/01/complexity-part-ii-it-all-

depends-on.html

Atlas & Resource Target StateA blue !"bre#

Service Fabric $ EclipseCon

Atlas agents:host resource property: colour

nonered

blue

fibres: \ infra; quota=[10..10]; filter=(&(Group=Red)(os.name=Darwin), \ default; quota=[100..1000]; filter=(Group=Blue) \

http://adaptevolve.blogspot.com/2008/01/complexity-part-ii-it-all-depends-on.html








Demo: Bootstrapping a Service Fabric

A video of the demo is available on the next page



System Description Running System

A Model Driven Runtime



System

=System

ElementsScaling Behaviour

(Replication Handlers)

+Resource Contract

= (cost_center=engineering)

= !(os.name=Windows)

= (os.name=linux) & (CPU.speed > 3 Ghz)

= fn(z)

= fm(y)

= fl(x)

A System

A Reference to...an OSGi assembly ora WAR, or EAR or a package name or evena named VM image.

service wires (remote) Service AService BService DService EService F

OSGi Service Registry

Service Propertiesfor E

ExportFabric SystemSystem Element

Topology Manager

Remote Service AdminEssence RMI

Remote Service AdminAVRO

Discovery ProviderSLP

Discovery ProviderDDS

E

E

E

Service E EndpointEssence RMIE

Service Eover Essence RMI

E

!

"

#

$

%

&

'

Export Registration

OSGi - Remote Service / Remote Service Admin

+



= 1

= 3

= 1

Target State Runtime State

Deploy

Starting a System



M!del

Target State

"Structure#SLA$

Runtime

Entity

Monitor

Provision Delta

Planned Deltase.g. Configuration

changes

Unplanned Deltase.g. Resource

failures

= 1

= 3

= 1


The Service Fabric responds by creating the corresponding System

Starting a System



Runtime State

= 1

= 5

= 1

Target State

To change a runtime System, change its model in the required manner.

Re-Configure

Updating a System



= 1

= 5

= 1

M!del

Target State

"Structure#SLA$

Runtime

Entity

Monitor

Provision Delta

Planned Deltase.g. Configuration

changes

Unplanned Deltase.g. Resource

failures


The Service Fabric responds by modifying the running System

Updating a System



Demo: Application Agility


❶



JEE

EAR

Each ‘fibre’ dynamically assembles and configures any local infrastructure services in response to the runtime dependencies of each hosted System Element

A self-configuring PaaS Modular & runtime dependencies

WAB

WAR

triggers policy based runtime assembly

triggers policybased runtime assembly

❷

❶

❷

message:%3C163824867-1246262625-cardhu_decombobulator_blackberry.rim.net-423254383-@bxe1080.bisx.produk.on.blackberry%3E

message:%3C163824867-1246262625-cardhu_decombobulator_blackberry.rim.net-423254383-@bxe1080.bisx.produk.on.blackberry%3E



Robust Systems...

Reddit Went Down: Blame Amazon, the Cloud or Both?

“I am a data storage/data center specialist. It pains me to see the "cloud" illusion come crashing down on a great product like Reddit.”......

“It's no secret among most IT folks that the cloud really isn't cheaper than rolling your own infrastructure for reasons exactly like this.”

http://www.readwriteweb.com/cloud/2011/03/ipad-for-business-round-up-web.php

Financial Services Organization: “How do you stop people rebooting machines....?”





•A ‘Robust’ platform is designed to survive cascading failure.

•A ‘Robust’ platform is adaptive & self-repairing: continually attempting to ‘settle’ into the desired state.

•No central or static points of command & control

Adaptive & Recovery Oriented

What do we mean by Robust?



Whac-A-Mole :)



Whac-A-Mole the Paremus Way




A

B

C

Management & Monitoring

ProvisionerSystem Managers

RegistryService Advertisements

RepositoryOSGi bundles

System DescriptionsNimble Policies

WAR EAR

General artifacts

Principles:• No ‘special’ management nodes• Source so*ware repository may fail• Population changes over time (nodes come and go)• Systems change over time (upgrades / roll-backs)• Attempting to cache everything / everywhere / is doomed to

failure

Approaches:• All infra services: Dynamic Group formation / re-formation• All infra services : Dynamic leadership election• On-fabric Repository Cache: Eventual Consistency across

self-selecting set of participating members

Robust PaaS Services



CDS Behaviour - Form Group & Elect Leader

CDS Group Lead! active members



A System is Loaded

= 1= 5= 1

!



Artefacts are automatically cached in response

Business Group Repository

Business Division Repository

!



Artefacts propagated around Group

!

"

#

$

Data Flow



Lead Fails



New leader elected - Group closes around failure

!

"#



New members join team

!

"

#



No Silver Bullets!Consistency, Availability, Partition-ability (CAP) trade-offs still apply. Use most appropriate data service for each business application.

• Unstructured data processing - Hadoop

• Key / Value - Voldemort

• Column - Cassandra

• Graph Database - Neo4j

• Relational - Derby, MySQL

• Distribution to nodes - BitTorrent

What about Application Data?

http://blog.nahurst.com/visual-guide-to-nosql-systems





Late Break Demos....



Introducing Zeus

+A video of the demo is

available on the next page



Lab49 is a strategy, design and technology consulting firm. Lab49 deliver strategic consulting and build advanced solutions for the world’s leading investment banks, hedge funds and exchanges.

“Lab49 deliver world-class solutions to some of the most sophisticated firms in financial services today. Our innovative approach stems from adopting user-centric practices enabling clients to respond quickly to today’s challenges, as well as realize new business opportunities in the modern world of trading.

SDP Foreign Exchange (FX) functionality;

‣Request for Quote (RFQ)

‣Request for Stream (RFS)

‣ Execution blotter

‣Research/News

Lab49 SDP leverages;

‣ Paremus OSGi Service Fabric

‣ Akka Open Source Agent-Actor Framework

‣ Lab49 Design and Engineering

‣HTML5/JavaScript

Copyright © 2011 Lab49 UK Ltd.May not be reproduced by any means without express permission. All rights reserved.



Demo: Single Dealer Platform

Copyright © 2011 Lab49 UK Ltd.May not be reproduced by any means without express permission. All rights reserved.

For access to demo videoplease contact

[email protected]





Industry Standards...

• OSGi framework - Equinox, Knopflerfish or Felix

• Paremus OSGi command shell (Posh): OSGi RFC147

• Paremus Nimble Resolver: OSGi OBR resolver (RFC112)

• Paremus implementation of OSGi Remote Services Administration

• OSGi ConfigAdmin

• Paremus implementation of OSGi Web Archive Bundle (RFC66)

• RTI implementation of OMG DDS: service discovery, configuration, management & monitoring



Industry Standards...

• Cloud / OSGi workshop organised by Peter Kriens - EclipseCon 2011

• Ongoing EEG conference calls throughout 2010 / 2011 chaired by David Bosschaert

• RFP-133 Cloud Computing - Current Status

• To get involved: contact David Bosschaert or Peter Kriens



http://dspace.mit.edu/bitstream/handle/1721.1/60085/BBFRFC66.pdf?sequence=1

Modularity is a Must Have not a Maybe!

Patterns of Modular Architecture (Kirk Knoernschild) - http://www.kirkk.com/modularity/chapters/

Diversity and Complexity - Scott E. Page



http://www.kirkk.com/modularity/chapters/

http://www.kirkk.com/modularity/chapters/



Distributed OSGi - Dr Robert DunneTuesday, 10:40, 20 minutes | Ballroom BC

Any Questions?

& drop by the Paremus stand for a chat!

OSGi DevCon BoFTuesday, 7:30 | Napa

Technology

Cloud & OSGi - The Dawn of Composite Clouds (Now with demo videos)