25 Years Interface Management25 Years Interface Management
28.11.2016
Tarmo Ploom, Integrtion Architecture
Key Messages
� A good child (interface) has many names – SOA, API, Micro-
service …
� Technologies come and disappear
� Interface descriptions remain� Interface descriptions remain
�Management of interfaces is key in sustainable IT-Landscapes
28.11.2016Tarmo Ploom, Integrtion Architecture 2
Agenda
� SOA @ Credit Suisse
� Interface Taxonomy
� SOA and middleware footprint by Credit Suisse
� Interface Management history by Credit Suisse
�Classification of Interface Management Repositories
� Interface Management System – IFMS
�Quality Control (QC) Process
� Key Performance Indicators
�Q/A
28.11.2016Tarmo Ploom, Integrtion Architecture 3
� Key objectives – Reducing complexity and costs– Increase agility and flexibility
� Service-oriented Architecture (SOA) – Separation of concerns� Service interface (contract)
– Consumer-interface-provider
Service-oriented Architecture – SOA at Credit Suisse
Consumer
Co
rpo
rate
Se
rvic
es
Legal & Compliance
Operations
Distribution
Processing
Finance
Risk
3rd
Se
rvic
e P
rovid
ers
Operations
– Consumer-interface-provider coupling
– Inside vs. outside data� Service implementation
� Integration Building Blocks� Service Repository (IFMS)
� Service interface descriptions, versioned
� Implementation-independent� Governance (e.g. Golden Source)� Service Bus
Channels
Service Bus
Service Repository
Work
flow
s
Provider
Consumer
Interface Implementatio
n
28.11.2016Tarmo Ploom, Integrtion Architecture 4
Interface taxonomy
�CS language:
− Event: fire and forget style asynchronous communication
− Service: request and response style synchronous communication
− Bulk: Bulk data transfer (>100 MB)
− Interface: abstraction of communication (not grouping)
28.11.2016Tarmo Ploom, Integrtion Architecture 5
SOA footprint by Credit Suisse
� Synchronous services
− Managed, ca 5100
− Unmanaged, ca 3000
� Bulk Services
− Managed, ca 3200
− Unmanaged ca 25 000
� Event services
− Managed, ca 50
− Unmanaged, ca 40 000
28.11.2016Tarmo Ploom, Integrtion Architecture 6
Middleware footprint (measurements or minimum estimates)
AVG calls per
day
in % AVG message
KB
AVG load per
day MB
in %
Bulk 228 571 0.1% 15000 3 348 214 77.0%
B2B FT proxy 8 667 0.0% 3938 33 333 0.9%
CORBA 30 000 000 10.7% 6 175 781 4.0%
28.11.2016Tarmo Ploom, Integrtion Architecture
CORBA 30 000 000 10.7% 6 175 781 4.0%
WS 1 400 000 0.1% 20 2 930 0.1%
MQ 250 000 000 89.1% 3 789 063 18.0%
Total 280 387 238 100.0% 4 349 321 100.0%
7
Implementation history
� 1991 – 1999, Client/Server Banking System:
− Two tier approach
− Paper based interface management
� First interface repository (1997 – 2001):
− For service (CORBA) interfaces only
− PHP/MySQL based
� Second interface repository (2002 – 2006):
− For service (CORBA) interfaces only
− MOF 1.3 based, JAP platform
− Paper based SOA governance processes
28.11.2016Tarmo Ploom, Integrtion Architecture 8
Implementation, problems from the past
�How to manage SOA landscape consisting of thousands of
interfaces?
�Who are active consumers of interfaces?
�What interfaces are deployed and used in production?
�Decommissioning of interfaces?
�How to make SOA governance less bureaucratic?�How to make SOA governance less bureaucratic?
� Integration of interface repositories with other repositories.
� Integration of interface development with interface repository.
�How to bridge gap between interface design and implementation?
28.11.2016Tarmo Ploom, Integrtion Architecture 9
Framework for analysing interface repositories
SOA IDE
(Integrated Development
Environment)
Model
Driven SOA
Third generation
interface
repository
Fourth generation
interface
repository
28.11.2016Tarmo Ploom, Integrtion Architecture
Management of interface metadata
(classical interface repository)
SOA Governance
(engineering, decommissioning, etc.)
First generation
interface
repository
Second
generation
interface
repository
10
First generation interface repository
SOA IDE
(Integrated Development
Environment)
Model
Driven SOA
Third generation
interface
repository
Fourth generation
interface
repository
28.11.2016Tarmo Ploom, Integrtion Architecture
Management of interface metadata
(classical interface repository)
SOA Governance
(engineering, decommissioning, etc.)
First generation
interface
repository
Second
generation
interface
repository
11
First generation interface repository
� Passive management of interface metadata
� Interface catalog
− paper based
− Excel
− Confluence
− Etc.
28/11/2016Tarmo Ploom, Integrtion Architecture 12
First Generation Interface Repository, Architecture
� Two main components:
− Interface Dictionary
− Reporting
28.11.2016Tarmo Ploom, Integrtion Architecture 13
First Generation Interface Repository, Problems
� Problems
− What if there are more than 1000 or more interfaces?
− Who are active consumers of an interface?
− What interfaces are deployed and used in production?
− SOA standards enforcement?
− SOA long term direction?
28.11.2016Tarmo Ploom, Integrtion Architecture 14
Second generation interface repository
SOA IDE
(Integrated Development
Environment)
Model
Driven SOA
Third generation
interface
repository
Fourth generation
interface
repository
28.11.2016Tarmo Ploom, Integrtion Architecture
Management of interface metadata
(classical interface repository)
SOA Governance
(engineering, decommissioning, etc.)
First generation
interface
repository
Second
generation
interface
repository
15
Second Generation Interface Repository
� Elements:
− Management of interface metadata
− Active semi-automated SOA governance processes
� Interface portfolio management
� SOA governance processes:
− Interface engineering
− Interface decommissioning
− Interface migration
− Interface usage
28/11/2016Tarmo Ploom, Integrtion Architecture 16
Second Generation Interface Repository, Common Architecture
28.11.2016Tarmo Ploom, Integrtion Architecture 17
Second Generation Interface Repository, Problems
� Problems:
− SOA bureaucracy
− Resistance to SOA bureaucracy
− Integration with other repositories
− Gap between design and implementation of services
28.11.2016Tarmo Ploom, Integrtion Architecture 18
Third generation interface repository
SOA IDE
(Integrated Development
Environment)
Model
Driven SOA
Third generation
interface
repository
Fourth generation
interface
repository
28.11.2016Tarmo Ploom, Integrtion Architecture
Management of interface metadata
(classical interface repository)
SOA Governance
(engineering, decommissioning, etc.)
First generation
interface
repository
Second
generation
interface
repository
19
Third Generation Interface Repository
� Elements:
− Management of interface metadata
− Active semi-automated SOA governance processes and
− Integrated development environment
� Integration: � Integration:
− Graphical design environment
− Design repository
− Application portfolio
− Log manager
� Infotype reuse:
− Interface infotype dictionary
28/11/2016Tarmo Ploom, Integrtion Architecture 20
Third Generation Interface Repository,Common Architecture
28.11.2016Tarmo Ploom, Integrtion Architecture 21
Third Generation Interface Repository, Problems
� Problems:
− Integration
− Gap between interface design and implementation
28.11.2016Tarmo Ploom, Integrtion Architecture 22
Fourth generation interface repository
SOA IDE
(Integrated Development
Environment)
Model
Driven SOA
Third generation
interface
repository
Fourth generation
interface
repository
28.11.2016Tarmo Ploom, Integrtion Architecture
Management of interface metadata
(classical interface repository)
SOA Governance
(engineering, decommissioning, etc.)
First generation
interface
repository
Second
generation
interface
repository
23
Fourth Generation Interface Repository,Model Driven SOA Repository
� Elements:
− Management of interface metadata
− Active semi-automated SOA governance processes
− Integrated development environment
− Generation of interface stubs, skeletons and code
� Linking design to implementation:
− Software configuration management
− IDL/WSDL/XSD/PL1/Java generator
28/11/2016Tarmo Ploom, Integrtion Architecture 24
Fourth Generation Interface Repository, Common Architecture
28.11.2016Tarmo Ploom, Integrtion Architecture 25
Service Description - IFMS Example 1/2
General
DescriptionDescription
Name
Brief/Full Description
Usage Notes
Changes
NotesBusiness Objects
Focus Business Object
Attachments
Classification
InterfaceType
Scope
InterfaceGroup
Example createFxTrade
� Synchronous service for building channel user and machine interfaces
� Service requires offer
− streamed quote offers
− request for quote
� Service Data Model
− Order master data (generic)
− Product data
InterfaceGroup
Direction
ActionType/Sub-Category
Compliance
28.11.2016Tarmo Ploom, Integrtion Architecture 26
Service Description − IFMS Example 2/2
Criticality
Contract
Non-Functionals
Conditions &
Exceptions
Signature
Criticality
C3, I1
OrderMasterData
id : UUID
extRef : String (0..n)
state : LifeCycleStateCd
FXOrder
master : OrderMasterData
cpty : PartyId
traded : ::Amount
rate : ::FXTradedRate
value : ::Date
28.11.2016Tarmo Ploom, Integrtion Architecture 27
Service Governance – Quality Assurance ProcessCentrally defined process and federated execution (service
landscape)
� QC1– Integration Architecture
– Is the service understandable?
� Is the name well defined?
� Is the description self-
explanatory?
– Is the service necessary?
– Is the service positioned correctly?
� Domain
QC1
QC2
QC3
Accept
Rework
Accept
(Registration,
De
sig
nC
od
in
g
Te
stin
gA
na
lysi
s
� Domain
� Public vs. internal vs. partner
� QC2 – Cross-functional review team
– Is the service and its data types
well-designed and following the
principles?
– Is the service using the correct
types?
� QC3 – Tool & Process Support
– Are all QC Obligations fulfilled?
– Is it implemented as specified?
QC3(Registration,
industry
standards)Go-Live
“Normal” relation to
development phases
Governance Process Entry Points
QC1: – New service
QC2: – Extension of public service
QC3: – Extension of internal service
– Additional implementation (transport)
28.11.2016Tarmo Ploom, Integrtion Architecture 28
Service Design Principles – First Principles 1/2Service description defined according to principles, the assessment
criteria
� Design from a business perspective
– Service name captures business
intention
� Defined naming conventions
– Standardized verbs
– Objects refer to business objects
� Defined semantics
– Read returns objects with
attributes
<CRUD Verb><Qualified Object>
createFxOrderOffercreateFxOrder
<action><Qualified Object>On<Event>
createConfirmationOnNewFxOrder
createSettlementOrderOnNewFxOrder
attributes
– Search returns objects with their
identifying attributes
– Business constants defined for code
values (enumerations, code tables)
– Reuse
� Data types available in the domain
– Recurring structures and variants
thereof
� Information types (from landscape)
– Service-fulfillment exception standard
� System exceptions
� Business exceptions
� Business attentions (degraded
Enum OrderState
{active, canceled, matured}
Type FxOrder is ComplexType
traded : Amount
rate : ExchangeRate
state : OderState
…
Exception : OfferExpired
Attention : CreditApprovalPending
28.11.2016Tarmo Ploom, Integrtion Architecture 29
Service Design Principles – First Principles 2/2Service description defined according to principles, the assessment
criteria
� Design preferences
– Generic vs. use-case specific
– Aggregated vs. elementary
– Asynchronous vs. synchronous
integration
– Coarse grained – “right-grained”
� Wide and high, but limited depth
– Wide: many attributes
– High: generalized business
Union Level {withMarkup Boolean,
withLinks Boolean}
Type FxOrder is ComplexType
identifier : OrderId
...
details : Level
party : PartyId
markup : FxMarkups (0..1)
– High: generalized business
objects
– Depth: few related business
object
with attributes
� Grouping supporting level of detail
selection
– Industry-standards vs. enterprise-
specific
� FpML, ISO 20022, etc.
� Design non-functional aspects
– Risk-adjusted criticality
� Confidentiality, integrity, availability
– Operational independence
– Size, latency, throughput
CT::CreditTransferSEPAasEBA
{pacs.008,pacs.002,camt.056,
camt.029,pacs.004}
Interbank credit transfer service using SEPA (Single Euro Payment Area) with the central counterparty EBA Clearing (Euro Banking Association).
ISO 20022 based protocol: messages mean simply “payment”, “error”, “request-recall”, “refuse-recall” and “accepted-recall”.
links : FxLinks (0..1)
...
28.11.2016Tarmo Ploom, Integrtion Architecture 30
Service DB/IFMS Adoption: 2001 - 2015
C2WS migration launched - Q4
2012
DWH import - Q4
2014
28.11.2016Tarmo Ploom, Integrtion Architecture
2012
IFMS 3.0 - Q3 2011
31
IFMS Adoption Trend: Q2 2013 – Q2 2015
Ratio modeled to registered increased from 24% to 57% from Q2/2013 to Q2/2015
28.11.2016Tarmo Ploom, Integrtion Architecture 32
IFMS Adoption Trend by Quarter: Interface Type
Increased the # of active synchronous interfaces by 370
or +8% in 12 months.
Increased the # of active bulk interfaces by > 2300 or
+330% in 12 months.
28.11.2016Tarmo Ploom, Integrtion Architecture
+330% in 12 months.
Increased the # of active message interfaces by 1
or +7% in 12 months.
33
IFMS Interface Structure in Q2 2015
28.11.2016Tarmo Ploom, Integrtion Architecture 34
Generator Stats: Facts & Figures Q1/14 – Q1/15
� > 600 Users out of 53 SubDepts
� > 12’000 Generations Accomplished
− 8’293 CLI’s
− 7’244 JSB’s
− 7’389 Web Service Wrappers
− 138 XSD’s
− 117 CORBA Wrappers− 117 CORBA Wrappers
� Total > 5.9 Mio Generated Artifacts
� The average generation time per artifact was reduced dramatically in order the handle the elevated load, caused by the increased user community and the intense C2WS activities.
28.11.2016Tarmo Ploom, Integrtion Architecture 35
QC Stats: Facts & Figures 2009 - 2015
28.11.2016Tarmo Ploom, Integrtion Architecture 36
IFMS Adoption: Provided & Consumed Interfaces
� 703 out of 5600 applications are
currently using IFMS*.
� In total 323 Applications provide
8’003 services and 624 applications
consume 13’910 services.
� The IFMS adoption is with >95% of
provisions and consumptions more provisions and consumptions more
or less limited to the IT PWS
department.
� A providing application provides in
median 7 services and a consuming
applications consumes in median 5
services.
� The top application (DWH Staging)
provides and consumes >1’000
services.
* Report date: May 27 2015
28.11.2016Tarmo Ploom, Integrtion Architecture 37
Applications using IFMS by region
AmericasEMEA
Switzerland
28.11.2016Tarmo Ploom, Integrtion Architecture
Americas
Asia-Pacific
Overall Applications using IFMSApplication Ownership:
Source: IFMS and CATI, Report date: 27.05.2015
38
Questions?
28.11.2016Tarmo Ploom, Integrtion Architecture 39