Manage complexity

IT to Manage Complexity

The Challenge of ComplexityModeling and AbstractionsMetasafe as a SolutionConclusion

www.Metasafe-Repository.com 2013_01_10

The World Is Complex

Nature is a complex eco-system.We must understand and respect itto be able to use and to preserve it.

Our society is a complexsocio-economic-technical system.

A complex infrastructure isrequired to support it.

IT-systems are an indispensablepart of this infrastructure.

IT

IT – the Victim of its own Success

IT-Systems became complex• Dynamic business requirements• Complex compliance rules• Hundreds of applications – legacy, new…• Conflict between “Open” and “Secure”• Complex user interfaces• Numerous flavors of architectures• Several DBMSs in parallel• Hundreds of databases with lots of columns• Diverse system-software and hardware

The consequence of this complexity• IT – a business driver or an obstacle for business development• Information gap between top management and operations level widens• Application backlog grows, cost of application acquisition and maintenance grows• Users resort to Shadow IT

Complicated vs Complex

Complicated Too Simple Complex

ManyManyNot Visible

ManyMissingIncomplete

ManyManyOrganized

EntitiesRelationships

Structure

(the world is complex, discarding information is not an acceptable solution)

Abstraction of Reality usinga Conceptual Data Model

Reality – describedby a Picture

OrderOrderNrDate

CustomerNameGender

PaymentDateAmount

pays

places

refersTo

1

0..n

0..n0..n

1

1..n RAmount

The conceptual data modeldescribes the data ->

The data describe relevant factsabout the reality.

Abstraction – to reduce Complexity

Reality – described by TextCustomer places 0..n orders and pays 0..n payments.Orders and payments belong to exactly one customer.A payment may refer to 1..n orders (split by RAmount).An order can be paid in several installments (payments).An order may be placed without simultaneous paymentPayments without orders are not accepted.

“Models” an Abstraction of Reality

Abstraction

- Entities (for real and virtual things, e.g. Customer, Order)

- Relationships (between entities, e.g. places, pays)

- Attributes (properties of entities or relationships, e.g. Gender, Amount…)

a Relationship

InstancesEntity-Type

Relationship-Type

Attribute-Typean Attribute

Storage Model Conceptual Model

Entities

Entity-RelationshipDiagram*)

(* Entity-Relationship Diagrams “ER” are the standard visualization technique for semantic models)

Complexity-Reduction – The Idea

DATA(0)

Model(1)

metaModel(2)

metametaModel(3)

1012

106

102

100

describesData

Describethe world

describesModels

describesmeaModels

Name Size Purpose

enti

ty

relation

attrib

ute

entity

relation

attribute

Reduction of complexity and size by abstraction with multiple levels of models

Information Models

Conceptual Model

(ER) Entity-Relationship(RL) Relational (Tables)(oo) Object(HI) Hierarchical(Do) Document(NS) NOSQL………..

Storage Models*)

NS

Do

HI

DatabaseServices

OO-Programming

RL

(ER) Entity-RelationshipER

ER

Storage Model aka “logical” model or “internal” model – describes the implementation concepts

(the de facto Standard for visualization)

Models and ViewsConceptual Model Submodels User Access

Sales

Procurement

Customers

Submodels of the conceptual model describe individual views and access rightsSubmodels (aka external models) make large models transparent and manageable

Representations and Storage

Models (languages, e.g. UML)

- Entity-Types (for classes of things..)

- Relationship-Types (e.g.has, belongs…)

-Attribute-Types (e.g. size, color…)

Instances (data, e.g. Text, XLS..)- Entities (for things..)

- Attributes (size, color…)

- Relationships (has, belongs…)

Graphical Represention

places

Instances

Customer

John

Order

N3245

Model

Customer

Name

Order

OrderNr

places

Textual Representation

Storage in a Repository

Instances

Models

Data about the reality are complex and voluminous-> we store them in databases-> we describe them by models

Models about the reality are (less, but still) complex and voluminous-> we store them in a repository

Part 2: The Metasafe-Repository

Metasafe-Repository

Instances

Models

Model based Tool Set Integrated• Eclipse and Web-based model- and instance-data editor• erSQL-query language with graphical query builder• Java-API for model and data access

Models AND Instance Data in the Database• Conceptual model and submodels (derived views)• Model-based access rights for models and instances• Instance data with versioning

Integration in Environment• Import / Export facility XLS, XML, Graphics• Transactions for Multi-Entity and Multi-User access• Large models (nK) and Instance Data (GB)• Persistence with Standard RDBMS

Tool-Set

ClientApplication

MetasafeCore

Database

RDMS

MetasafeDataServer

XML,…XLS,…

Metasafe-Architecture

Conceptual Data Model(Entity-Relationship-Model)

Persistence(a relational DBMS)

Instances (Versioning)

(Entity-Relationship-Model)

Dic

tio

nar

yEl

em

en

ts+

Do

cExternal ModelsM

etas

afe

AP

I

Physical Architecture

ClientApplication

MetasafeCore

Database

RDMS

ClientApplication

MetasafeCore

Database

RDMS

MetasafeDataServer

WEB -Client

Web-Application

Browser

Web-Server

MetasafeDataServer

MetasafeCore

RMI

Web

Co

vers

the

full

ran

ge

Full Life-Cycle Support1. Design – conceptual models:

Design types and rights with the design dictionaryDesign instance data structure with the master modelDesign views with submodels (external model)

3. Implement – the model is blueprint:

Make complex information transparentInstances with versions, variants and access rights

4. Use – API + Toolset + erSQL query language

Powerful API to process models and instancesToolset for modeling, data edit and exchangeModel based query language with graphical query builder

2. Test the Data Model – to assure the applicability:

Load instance data into the repositoryCheck data against requirement documents (eg. with erSQL)Improve and maintain the model during the lifetime of the data

Metasafe Tool Set

API Core API - Framework for Tools and Applications

Model Manager - manage models in the database

Data Browser - Edit, View, Visualize Instances

erSQL Query – Query-Builder, Executor

Exchange – Im/Export with XML, XLS, DBMS, Files

(1) Model Editor Toolset

Dictionary of Types

Design the model

Document the Model

Generated Graphics

metaModeler: design, document …

MetasafeDatabase

….and store the model in the database

(2) Data Editor Toolset

Browse Instance DataEdit Attributes

Navigate with Graphics

metaEditor• no programming• edit/view your data• automatic - model driven• set of windows• configurable• based on Eclipse• extensible• easy to use• protected with access rights• desktop-version• web 2.0 version

(3) erSQL and the Query Builderre

sult

s@

you

rfi

nge

rtip

s

Execute

(B) select attributes

(C) generate query

(D) generate graphics

(A) define navigation

(B)

(C) (D)

(A)

Display ResultsExport Results – XLS, XML

erSQL vs SQL

“select Customer.country,Customer.Name,OrderDetail.quantityOrdered, OderDetail.price,Product.productCode, Product.productLine;

start with type = ˈCustomerˈ, name = ˈ*ˈwhere Customer.country = ˈUSˈ;

follow ˈplaceˈ to type ˈOrderˈ as Order;follow ˈhasˈ to type ˈOrderDetailˈ as OrderDetail;follow ˈrefersToˈ to type ˈProductˈ as Product;”

place

has

refersTo

CustomercustomerNamecountry

OrderorderNumber

OrderDetailquantityOrderedprice

ProductproductCodeproductLine

Request: “Report the orders placed by US-customers”

The Model The erSQL-Query:

SELECT Customer.country, Customer.Name,OrderDetail.quantityOrdered, OrderDetail.price,Product.productCode, Product.productLine

FROM Customer INNER JOIN OrderON Customer.customerNumber= Order.customerNumber

INNER JOIN OrderDetailON Order.orderNumber= OrdertDetail.orderNumber

INNER JOIN ProductON OrderDetail.productCode

= Product.productCodeWHERE Customer.country = ˈUSˈ

The SQL-Query:

Created by the userin a few minutes

using the query builder

Requires a specialisttakes a few days(to reach him)

Application Szenarios

Enterprise Architecture Management

Data and Meta-Data Management

Repository for Software-Tools

Platform for Configuration Management

DBMS for User friendly Applications

Where Management of Complexity Matters

Enterprise Architect

Report/Export XLSXML

BusinessServices

ApplicationsProjects

Report

EDITEdit – Browse – View

Import – ConsolidateFrom multiple sources

MetaData

DashBoard

Models +Metadata

AccessRights

Tool Developer

ProprietaryService Layer

RelationalDatabase

Tool-A

Files

Tool-B

Silo-Architecture

…..

Etc.

XML_Exchange

P2P-Copies

Model drivenRepository

Tool-A

Repository Services and Tools

Tool-B

Integrated Repository

View BViewA

………..

View-A View-B

… time consuming… expensive… complicated tools

… better tools faster… less time and cost… extensible, efficient

MOVE!

ConclusionComplexity

is the challenge in modern societycreates misunderstandings and costcreates the decision dilemma

Transparency BY IT and IN IT(meta) IT is required to manage ITsells and provides better service to customersreduces cost – avoids redundancy & waste

Metasafe - a Dedicated Platformto make complex information transparent,to make information useful and easily and fast accessiblein an efficient and user friendly manner

Visit us @ www.metasafe-repository.comQuestions @ info/at/metasafe-repository.com2013-01-10