Vincenzo Del Vecchio Banca d’Italia Statistics Collection and Processing Department [email protected] 2012 ESSnet Workshop – 4 December

Vincenzo Del VecchioVincenzo Del Vecchio

Banca d’ItaliaBanca d’Italia

Statistics Collection and Processing DepartmentStatistics Collection and Processing Department

[email protected]@bancaditalia.it

2012 ESSnet Workshop – 4 December 2012

The Data Warehouse of Banca d’Italia

Guiding Principles and Architecture of an Guiding Principles and Architecture of an Integrated Statistical WarehouseIntegrated Statistical Warehouse

2 4 Dec 2012 2012 ESSnet Workshop

Agenda

1. Guiding principles

2. Architecture of the statistical data warehouse of the Bank of Italy


Reporting units (Banks & OFI (>4.000),

enterpr.& families (>15.000), individuals (>150.000))

Other Institutions (IMF, OECD, ECB, BIS Eurostat, ISTAT, …)

Market Providers(Bloomberg, IBCA,

Enterprise Register, .…)

Internal sources(payment system,

accounting system …)

Institutionalstatistics

BI users(research, supervision,markets, >2.500 users)

Public data(> 750.000 inquiries/year)

Return flows (to > 5.000 reporting agents)

Other Flows(to other Institutions )

1.1 – Integrated Approach

Economic analysis

Supervision

Central banking

Payment system

C.C.R.

Stat.Domain

s

(> 1 billion observations / year )

DATADATA

DEFINITIONSDEFINITIONS

DATADATA


DATADATA


DATADATA



Information are shared by many organizational functions

and accessible to users who have rights;

Data are collected and processed minimizing redundancies

The integrated use of data from different sources and the

reuse of data for many purposes are fostered through:

Organizational Measures (statistical committee, specialized

units for warehouse administration, …)

Metodological and Technical Measures (reference information

model, common data dictionary …)

Harmonization of Concepts, Code Lists, Data Contents

(concepts and data administration)

1.1 – Integrated Approach


Model of meta-models L4

L3Model of data definitions

(“meta-model”: how to make definitions)Object group:

property

L1Data obser.Green cars:

40%

Real world

L0

(Information Model: Matrix)

Cars by colour: percentage

L2Data definitions“dictionary”

1.2 – Information completness

Stats Definer

Stats Producer

Stats User

6 4 Dec 2012

1.3 - Active Definitions (model driven sw)

DATADATA

Software Services

Administrator

Information Model


User

System AutomationTime to marketAccurate and up-to-date

7 4 Dec 2012

User

1.4 - User oriented model & languages

Administrator :Subject Matter Expert

Information Model


Based on Mathematics & Statistics

Independent of the IT implementation

and the IT people


Statistical DomainMonetary & financialBalance of Payments…Data typeQuantitative / qualitativePeriodical / notMultidimensionalTime seriesRegistersQuestionnaires…

DefinitionExtraction,

transformation,transmission

Collection and storage

Compilation Dissemination Use

1.5 - Unique model and approach (integration of methods and techniques)

DATADATA

Software Services

Administrator

Information Model



Two different histories:

of the real world (e.g. when something is true or

false)

of the information system (e.g. when something is

known or unknown)

History of all the I.S. contents:

definitions

data observations

1.6 – Historical representation


Agenda

1. Guiding principles

2. Architecture of the statistical data warehouse of the Bank of Italy


• A unique information model - the Matrix model –

(designed and maintained by the Bank of Italy) – descibing concepts, data structures and algorithms for validation and calculation;

• A unique data dictionary – a data base structured according to the Matrix model – storing the users’ definitions;

• A logically unique warehouse storing the data observations;

• A common software platform – Infostat - made of reusable services driven by the users’ definitions stored in the data dictionary.

2.1 - The current reference architecture


SOFTWARE SERVICES

Define

Send

Receive

Calculate & Check

Release

Monitor

Inquiry

Import/export

… W3C, WS-IBPELGSBPM

PROCESSES

Receive

CheckRemarks

Release

Calculate

Calculate

Microdata

Macrodata

Indicators

Send

Send

2.2 - User application architecture

DATA WH.

Data Definitions

Data Observations

Calc. AlgorithmsMATRIX(SDMX, XBRL, CSV …)

Application Application User Applications

DATADATA



Warehouse Administration

Common software services

DATADATA


DATADATA


DATADATA


DATADATA


DATADATA


DATADATA


DATADATA


DATADATA


Statistical community / Information system “A”

Information segm

ents

(surveys, production lines …)

Statistical community / Information system “B”


IMF, OECD, ECB, …

D1

D2

D3D4

D5

A1

A3

A2

D10

D12D13

D15

D17

D16A13

A12

A1

4

Banks & OFI’s reports

D51

D52

A53

A52

A51

Economic research models

D54

D53

A54

C.C.R.

D21

D22

D23

D24A22

A21

D60D61

Statistical bulletin

A60A61

Statistical products

D70A71

A70A72D71

D72

D41

A42

A41

D42

Supervision models

• page 14• page 14

2.2 – Graph of Data and Algorithms


Supporting more Warehouses

F.I.U.ESCB(RIAD)

Economic analysis

Supervision

Central banking

Payment system

Shared

Data

Specific

Data

Other ItalianInstit.

…

Statistical Communities

B.I.Institut.

FunctionsDATADATA


DATADATA


DATADATA


DATADATA



8:34-8:36

Processmonitor

Metadataadministration

Informationprovider

Informationconsumer

User interface A2A

Dataanalyst

Regulardata production

Inquiry, search,analysis tools

Reportgeneration

Dataservices

User interface A2A

Metadataadministrator

Metadataimport/export

Data entryMessages upload,remarks download

Formatconversion

Checks

Collection &

validation

Inquiry, search Data services

Dissemination

Data definition Analysis & reporting

Collaboration

Notifications,Alerts

Workflowengine

Warehouse

Data

Dictionary

Documents

Event

s

Operationsadministrator

Monitor

INFOSTAT architecture

Calculationengine


THE ’60s & ’70s first IT solutionsfirst “active” software (metadata driven)

THE ’80s launch of the integrated approachthe Matrix schema and the first integrated

solutions

THE ’90s integration of many silos applicationsevolution of the Matrix Modelsupport to GESMES-CB standardization

THE 2000s Statistical Dictionarysupport to SDMX and XBRL standardization

THE 2010s INFOSTAT: a service oriented platform2009: data collection and data quality services2012-13: full set of servicesFrom 2009 on: migration of old surveys and data bases

Brief history of the IT support for Statistics


Thank you !

Vincenzo