Information Infrastructure: Foundations for ABS Transformation

Stuart Girvan, Australian Bureau of Statistics

MSIS Paris, April 2013

Outline

• ABS 2017 Transformation Vision and Information Infrastructure

• Metadata Registry and Repository (MRR) & Statistical Workflow Management system (SWM)

• Achievements so far

• What’s next?

• The journey

VISION FOR ABS 2017 TRANSFORMATION & THE ROLE OF INFORMATION INFRASTRUCTURE

Enablers Business Change Benefits Strategic Goals

Metadata driven

processes

Re-use metadata

Managed, consolidated & consistent

business processes

Re-use processes

Re-engineered, Improved Business Processes

Enterprise Architecture

Information Governance, Architecture & Management

Survive and ThriveProcess

Assembly and

Automation

Managed, consolidated & consistent

metadata

Migrate corporate/foundation metadata to MRR

Reduce time and cost of

business operations

Grow business via new

statistical products/services

Statistical Workflow Management System (SWM)

Metadata Registry and Repository (MRR)

EDW

Existing/new applications integrate with MRR/EDW

(Better) Tools and applications integrated with MRR

Existing/new applications integrate with SWM/SOA

Processes captured as Services, Workflow – SOA

Managed, consolidated & consistent applications

& components

Business

Information

Application

Re-use metadata

Re-use application components

Component/service assembly

Enterprise

Business Transformation

Re-use of common business processes

Process assembly, configuration and automation

The role of MRR and SWM

• MRR and SWM provide foundations for processes and metadata to be:– Managed– Consistent– Discoverable– Governed– Executable

– Ultimately, re-usable, easily assembled, and in the case of processes, automatable

MRR AND SWM

What is MRR?

• The Metadata Registry and Repository consists of two parts:– Repository – the centralised store for standards

based metadata

– Registry – the catalogue that tells you what’s in the repository

MRR – Registering & Retrieving Metadata

Registry

Repository

Register key elements of metadata package

E.g. Register metadata

Complete metadata package resides in

Repository

MRR – Registering & Retrieving Metadata

Repository

Search for metadata here

E.g. Metadata search

Returns list of relevant metadata held in the

repository

Registry

?

Pointer to Metadata in Repository

Registry

MRR – Registering & Retrieving Metadata

RepositoryE.g. Retrieve metadata from herebased on search results

MRR & standardsRepository

This will be the metadata content in the most appropriate standard (DDI, SDMX, etc)

E.g.DDI,SDMX,And ?

Registry

Model of what we keep in a registration

(ATMM- GSIM)

Mapping between the registration and the standard

What is SWM?

• Statistical Workflow Management System • ActiveVos (BPMS)

+ governance (e.g. Alignment of new processes to business architecture, Duplication of existing processes?)+ processes for building workflow and orchestration+ role and responsibilities

WHAT WE HAVE DONE SO FAR AND WHAT’S NEXT

What have we learnt?

• MRR works

• DDI/SDMX successfully used– E-forms generation– Table production for analysis– Metadata queries for end user data sets

What have we learnt?

• SWM good for – doing things quickly – joining things up – coordinating and re-using processes– handling long, asynchronous events

• Exposes and documents business process

What’s next?

Input Data Warehouse

and Tools

Collection/Data

Definition Tools

Instrument Definition

Tools

Run Survey/ Data

Collection Tools

Processing,Editing, Analysis

Data Warehouseand Tools

Output Data Warehouse and Tools

Instrument Generation

Tools

DisseminateTools

Corporate Metadata Tools

Classification Management

Glossary Management

Standard Question, Data Element (variable) Definition

Topics Management

Corporate Metadata Registry and Repository

Corporate Business Process Tool and Repository SWMMRR

REACHING THE VISION

Vision for Re-use: Existing Applications/Tools decomposed into re-usable bits

User Interface

Business Rules and Processes

Data and Metadata

SWM

MRR

Common (largely), stable, reusable, automatable workflow

Common Statistical Metadata

Any given application consists of

Common Statistical Data EDW

Common, stable, reusable, modular services

Services

Statistical Data

Information about Statistical Data

Workflow & process orchestration

Code that takes metadata and data, and does something

with it (e.g. imputation)

Vision for Re-use for New Applications/Tools

User Interface

SWM

MRR

Common (largely), stable, reusable, automatable workflow

Common Statistical Metadata

Common Statistical DataEDW

Common, stable, reusable, modular services

Services

Application Specific

Code

New Application

Business Rules and Processes

Data and Metadata

User InterfaceUser InterfaceUser InterfaceUser Interface

SWM

SWM MRR EDW Services

User InterfaceUser Interface

Information Infrastructure Backbone

Vision for re-use and automation (& assembly)

InformationInfrastructure

Backbone

User Interface

Business Rules and Processes

Data and Metadata

• Code most new applications • Different tools don’t play well

together

• Some re-use of metadata and process via integrated tools

• Some coding, some re-use of code

• Tools begin to integrate

The journey, and wins on the way

User Interface

User Interface

Business Rules and Processes

Data and Metadata

Business Rules and Processes

Data and Metadata

SWM

MRR

EDW

Services

• Configure and assemble processes from components

• Business process greatly automated

• Code small number of things, re-using a lot others

Enablers Business Change Benefits Strategic Goals

Metadata driven

processes

Re-use metadata

Managed, consolidated & consistent

business processes

Re-use processes

Re-engineered, Improved Business Processes

Enterprise Architecture

Information Governance, Architecture & Management

Survive and ThriveProcess

Assembly and

Automation

Managed, consolidated & consistent

metadata

Migrate corporate/foundation metadata to MRR

Reduce time and cost of

business operations

Grow business via new

statistical products/services

Statistical Workflow Management System (SWM)

Metadata Registry and Repository (MRR)

EDW

Existing/new applications integrate with MRR/EDW

(Better) Tools and applications integrated with MRR

Existing/new applications integrate with SWM/SOA

Processes captured as Services, Workflow – SOA

Managed, consolidated & consistent applications

& components

QUESTIONS?

Contributors: Chris Conran, Simon Wall, Gillian Nicoll