24
NSW Government Analytics and Data Warehousing Solutions Standard September 2015

[Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

  • Upload
    buikhue

  • View
    224

  • Download
    1

Embed Size (px)

Citation preview

Page 1: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

NSW Government

Analytics and Data Warehousing

Solutions Standard

v1.0

September 2015

Page 2: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

CONTENTS

1. CONTEXT 3

1.1. Background 3

1.2. Purpose 3

1.3. Scope and application 3

1.4. Policy context 3

1.5. The ICT Services Catalogue 3

2. KEY PRINCIPLES 4

3. REQUIREMENTS 5

3.1. Analytics and data warehousing 5

3.2. Service level and complexity 5

3.3. Requirements tables 6

3.3.1 Silver (standard) – Use Cases / Scenarios 7

3.3.2 Gold (complex) – Use Cases / Scenarios 8

3.4. Elements of this standard 9

DOCUMENT CONTROL 14

APPENDIX A – DEFINITIONS 15

APPENDIX B – ABBREVIATIONS 16

APPENDIX C – REFERENCES 17

APPENDIX D – STANDARDS 18

Developing technical standards 18

Management and implementation 18

Page 3: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

1. CONTEXT

1.1. BackgroundThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard contains minimum technical and functional requirements that agencies should consider when procuring ICT services for analytics and data warehousing solutions.

By defining the necessary and common elements across agencies the standard provides an opportunity to leverage the buying power of Government as a whole, improve procurement efficiency, and increase interoperability.

1.2. PurposeThe purpose of this standard is to assist NSW Government agencies to develop, procure and implement analytics and data warehousing service solutions and tools, as well as take full advantage of their benefits. This standard also helps agencies procure in a strategic manner that reflects the NSW Government’s priorities as outlined in the NSW Government ICT Strategy.

This standard details the issues that need to be considered so each agency can identify the available options that best suit their business requirements, helping them achieve value for money through cost savings and improved flexibility of service offerings.

1.3. Scope and applicationThis standard applies to all NSW Government departments, statutory bodies and shared service providers. It does not apply to state owned corporations, but is recommended for their adoption.

For the purposes of this standard, analytics means data analysis services (including business intelligence (BI)), and data warehousing means services for storage of data extracted from transaction systems, operational data stores and external sources.

This standard sets out service definitions as minimum requirements that vendors must meet, for inclusion in the NSW ICT Services Catalogue. Agencies should consider any specific operational or regulatory factors that impact their requirements, and specific requirements they have in addition to those detailed in this standard.

1.4. Policy contextThe NSW Government ICT Strategy and Digital+ 2015 Final Update set out the Government’s plan to: build capability across the NSW public sector to deliver better, more customer-focused services that are available anywhere, anytime; and to derive increased value from the Government’s annual investment in ICT.

Information sharing, open data and reuse of technology are priority initiatives of the ICT Strategy, to maximise the return on government investments, support better policy development and service delivery. The NSW Government ICT Investment Policy and Guidelines establishes these requirements for all new ICT projects, particular to make better use of the functionality in existing systems.

The NSW Government Enterprise Architecture (NSW GEA) provides direction and practical guidance to accelerate the development of agency EA capability and enabling a common, intra and inter agency approach to the design of digital government. It encompasses all aspects of enterprise architecture activity at the business, information, application and technology

Page 4: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

infrastructure layers. The NSW GEA is mapping the landscape of Whole of Government systems available across the sector, highlighting opportunities for reuse and where APIs can add value.

NSW Government, along with many governments in other jurisdictions, has moved towards opening up previously protected databases and applications, so that data and functionality can be accessed across agency boundaries or reused in new systems. Within NSW this has been reflected in the development of the NSW Government Open Data Policy, which provides clear direction for agencies to make their data available to the public in machine readable forms, including through the availability of APIs.

Developing whole of NSW Government ICT technical standards is a key initiative of the NSW Government ICT Strategy, driven by the ICT Procurement and Technical Standards Working Group. These standards leverage principles defined in the NSW Government ICT Strategy and the NSW Government Cloud Policy, and they support the NSW ICT Services Catalogue.

The standards set out service definitions as minimum requirements that vendors must meet to be able to offer their services through the NSW Services Catalogue. This helps achieve consistency across service offerings, emphasising a move to as a service sourcing strategies in line with the NSW Government ICT Strategy, and it signals government procurement priorities to industry.

This standard should be applied along with existing NSW Government policies and guidance, including the NSW Digital Information Security Policy. More information on the process for the development of standards that populate the ICT Services Catalogue is at Appendix D – Standards.

1.5. The ICT Services CatalogueThis catalogue provides suppliers with a showcase for their products and services, and an opportunity to outline how their offerings meet or exceed standard government requirements. The standards, together with supplier service offerings, help to reduce red tape and duplication of effort by allowing suppliers to submit service details only once against the standards. The offerings are then available to all potential buyers, simplifying procurement processes for government agencies.

Implementing this category management approach embeds common approaches, technologies and systems to maintain currency, improve interoperability, and provide better value ICT investment across NSW Government.

2. KEY PRINCIPLESThis standard is based on the following principles:

End-to-end digital: Analytics and data warehousing solutions should enable end-to-end digital business processes and management.

Data quality: Data should possess characteristics indicating data quality, including in relation to: accessibility, the institutional environment, relevance, timeliness, accuracy, coherence and interpretability (see the NSW Government Standard for Data Quality Reporting for more details).

Single source of truth: Where applicable, agencies should establish a dataset as the single source of truth, to help eliminate duplication, inconsistency, and to support data currency. This will also be the authoritative version of the dataset, and its custodian will also be accountable for data information and quality (see the NSW Government Data and Information Custodianship Policy for more details).

Page 5: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

Timeliness where practical: Analytics and BI should be current and timely, to enhance its value for real-time monitoring, assessing progress, feedback and future predictions – particularly for service delivery.

The data warehouse as an ‘enterprise memory’: The data warehouse should be a repository for agency knowledge, and an ongoing resource for the agency. The data can be used for ongoing analytics and developing BI, informing future operations, increasing productivity and efficiency, and enhancing service delivery.

Facilitating as a service: Analytics and data warehousing solutions should facilitate the agency transition to as a service.

Interoperability: Analytics and data warehousing solutions should meet applicable recognised open standards.

Managing data as an asset: Datasets should be viewed as a strategic asset for agencies, and analytics and data warehousing solutions should help ensure agencies achieve maximum value and strategic business intelligence from their data.

Data sharing: Analytics and data warehousing solutions should support data and information sharing across NSW Government to enhance service delivery, and comply with principles set out in the NSW Open Data Policy.

Data accessibility: Where appropriate, and in line with the NSW Open Data Policy, data should meet accessibility requirements.

3. REQUIREMENTS

3.1. Analytics and data warehousingWhen assessing analytics and data warehousing solutions, additional elements to be considered include BI and service management. For the purpose of this standard data analytics includes BI.

3.2. Service level and complexityAnalytics and data warehousing can be provided in a range of ways. For example, the supplier of the service may manage some of the service or environment during the course of the contract, or the supplier of the service may manage the entire service for course of the contract. Analytics and data warehousing services may be provided as one or two separate solutions.

The following requirements are set out in use case tables, separated into two service levels – Silver and Gold, reflecting the complexity of the analytics and data warehousing solution required:

Silver: A Silver standard analytics or data warehousing service is the provision of a ‘Platform as a Service’, where the required tools are deployed into a cloud environment and made available to the buyer with minimal configuration. Further specific configuration is required by the buyer to allow successful delivery of the solution – for example, customisation of data models and configuration of ETL.

Gold: A Gold standard analytics or data warehousing service is the provision of a fully functional ‘Software as a Service’ where the solution is delivered as a fully configured and maintained service to meet the business needs of the buyer. Such a solution would be deployed wholly by the vendor to acquire and process data, with subsequent presentation to the user in a manner that can be consumed by the buyer using reporting or OLAP tools with little additional configuration effort.

Page 6: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

3.3. Requirements tablesThe following tables set out the recommended business and technical requirements for NSW Government. They provide a consistent approach for all NSW Government agencies regardless of their size. Explanations for each element of the following use cases are provided at section 3.4.

Page 7: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

3.3.1 Silver (standard) – Use Cases / Scenarios

‘Use cases’ that are anticipated in agencies are set out in the table below. The corresponding requirement sections of this standard are ticked in the columns.

Use Case / ScenarioSILVER

Data

pro

filin

g

Data

qua

lity

Data

qua

lity

dash

boar

ds

Data

qua

lity

asse

ssm

ent

Auto

mat

ed d

ata

clea

nsin

g

Retr

ospe

ctive

ana

lytic

s

Real

-tim

e an

alyti

cs

Pred

ictiv

e an

alyti

cs

Requ

irem

ents

ana

lysis

and

des

ign

Data

mod

ellin

g

Data

line

age

Data

hist

ory

/ mai

nten

ance

of

hist

ory

Data

acq

uisiti

on

ETL

– op

erati

onal

met

adat

a

ETL

– tr

ansf

orm

ation

cap

abili

ties

ETL

– ex

cepti

on m

anag

emen

t and

re

porti

ng

ETL

– re

conc

iliati

on re

porti

ng

ETL

– sc

hedu

ling

capa

biliti

es

ETL

– pe

rfor

man

ce

Resil

ienc

e

Data

base

supp

ort

Secu

rity

Busin

ess g

loss

ary

Perf

orm

ance

Analytics Data warehousing

Use Case / ScenarioSILVER

Sem

antic

Lay

er

OLA

P

Pred

efine

d re

port

s

Self-

serv

ice

adm

inist

ratio

n

Full-

serv

ice

adm

inist

ratio

n

Clou

d co

mpl

iant

hos

ting

faci

lity

NSW

Gov

ernm

ent D

ata

Cent

re

Ons

hore

/offs

hore

m

anag

emen

t

Serv

ice

leve

l m

anag

emen

t

Mul

ti-se

rvic

e br

oker

pr

ovisi

on

Data

sove

reig

nty

Back

up a

nd

reco

vera

bilit

y

Arch

ive

and

rete

ntion

Audi

t

Analytics Data warehousing

7

Page 8: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

3.3.2 Gold (complex) – Use Cases / Scenarios

‘Use cases’ that are anticipated in agencies are set out in the table below. The corresponding requirement sections of this standard are ticked in the columns.

Use Case / ScenarioGOLD

Data

pro

filin

g

Data

qua

lity

Data

qua

lity

dash

boar

ds

Data

qua

lity

asse

ssm

ent

Auto

mat

ed d

ata

clea

nsin

g

Retr

ospe

ctive

ana

lytic

s

Real

-tim

e an

alyti

cs

Pred

ictiv

e an

alyti

cs

Requ

irem

ents

ana

lysis

and

des

ign

Data

mod

ellin

g

Data

line

age

Data

hist

ory

/ mai

nten

ance

of

hist

ory

Data

acq

uisiti

on

ETL

– op

erati

onal

met

adat

a

ETL

– tr

ansf

orm

ation

cap

abili

ties

ETL

– ex

cepti

on m

anag

emen

t

ETL

– re

conc

iliati

on re

porti

ng a

nd

repo

rting

ETL

– sc

hedu

ling

capa

biliti

es

ETL

– pe

rfor

man

ce

Resil

ienc

e

Data

base

supp

ort

Secu

rity

Busin

ess g

loss

ary

Perf

orm

ance

Analytics Data warehousing

Use Case / ScenarioGOLD

Sem

antic

laye

r

OLA

P

Pred

efine

d re

port

s

Self-

serv

ice

adm

inist

ratio

n

Full-

serv

ice

adm

inist

ratio

n

Clou

d co

mpl

iant

ho

sting

faci

lity

NSW

Gov

ernm

ent

Data

Cen

tre

Ons

hore

/offs

hore

m

anag

emen

t

Serv

ice

leve

l m

anag

emen

t

Mul

ti-se

rvic

e br

oker

pr

ovisi

on

Data

sove

reig

nty

Back

up a

nd

reco

vera

bilit

y

Arch

ive

and

rete

ntion

Audi

t

Analytics Data warehousing

8

Page 9: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

3.4. Elements of this standardData profiling

The capability to profile the candidate data for inclusion within the data warehouse to understand quality in terms of consistency (e.g. Street vs. St) and completeness (e.g. % completed compared to real world).

Data quality

Provision of measure(s) to determine the quality of data being analysed.

Data quality dashboards

The ability to deploy continuous data quality monitoring of source data.

Data quality assessment

The ability to define rules to check valid values and relationships between data to ensure a complete and accurate picture is available, or understand steps needed to remediate quality issues.

Automated data cleansing

The ability to correct known quality issues automatically, including for example:

Standardisation: Correcting AU, AUS, Australia or ensuring phone numbers are a standard format;

Matching: Matching multiple records that represent the same real-world ‘thing’, e.g. John C = Mr. John Citizen; and/or

Merging: Creating a single ‘golden’ record.

Retrospective analytics

The ability to deliver analytics retrospectively based on available data sets within scope.

Real-time analytics

The ability to deliver analytics in real-time based on available data sets within scope.

Predictive analytics

The ability to deliver predictive analytics based on available data sets within scope.

Requirements analysis and design

Provision of specialist data warehousing business analysts to understand and document information requirements.

Data modelling

Provision of specialist data warehousing data modellers who are experienced with industry standard data warehousing techniques that capture business concepts, business change over time and provide flexibility for future reporting or extension. Dependent upon the application, vendors should have demonstrated experience in COTS data models, data vault, dimensional modelling or third normal form data models.

Data modelling is core to the success of data warehouse projects as it sets the foundation for future analysis and reporting capability, therefore it is desirable that the modeller has experience within the relevant sector.

9

Page 10: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

Data lineage

All tools within the data warehouse should provide full traceability from reported information back to the specific data extracts from source systems to enable full audit and reconciliation of reported information.

Data history / maintenance of history

Where required by the business, the data warehouse should provide the capability to retain the history of slowly changing attributes (e.g. marital status) over time to ensure historic reporting can be delivered.

Data acquisition

Desirable data acquisition techniques should include extract files, direct data base connection to a range of major databases, and database change data capture. More modern tools should also accept ‘stream’ data to a web service, e.g. social media feed content.

The data acquisition techniques are often separately licenced by vendors and should be selected based on source system impact, data volume, budget and data timeliness requirements.

ETL – operational metadata

The Extract, Transform, Load (ETL) tool should capture operational metadata including information on processing times, record counts and exception counts for the constant monitoring of the solution.

ETL – transformation capabilities

ETL should provide appropriate data transformation capabilities for the data being processed. Typical capabilities will include calculations, text manipulation, joins and lookups. Where possible, transformation rules should be ‘metadata-driven’, allowing update of the rules without extensive code changes.

ETL – exception management and reporting

ETL should provide the ability to inspect data that has been rejected or that breaks specific rules and provide options for remediating that data and including within the data warehouse.

ETL – reconciliation reporting

The solution should provide for end-to-end reconciliation of information for each load. This will enable the organisation to have confidence in the accuracy of information presented by the solution and identify issues.

ETL – scheduling capabilities

It should be possible to schedule processing activities to occur in required batch windows or in response to external events.

ETL – performance

Performance should be sufficient to run the required processing within available time windows and support the data timeliness requirements of the solution. Dependent upon specific requirements, this may be supported by capabilities to allow:

Scalability to multiple processing nodes; and

Parallel processing of the dataset (e.g. dividing the data up and using multiple processors to accelerate the end-to-end run).

Resilience

Solutions should meet criteria for:

Backup frequency (e.g. what is the acceptable information loss);

Availability;

10

Page 11: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

Recoverability (e.g. in the event of failure); and

Restartability (e.g. can processing be restarted from where it was if there is a failure part-way through processing).

Database support

Where procured as Platform as a Service, appropriate relational or in-memory database technology should be available to the buyer. The use of specific database technology is less critical should the solution be delivered as a fully managed service.

Security

Security should be available throughout the analytics solution including security of the semantic layer at a ‘per-object’ level, report security and data-level security (e.g. ability to view only clients related to your work group).

Data warehouses are by their nature attractive targets due to the large volume of valuable data. Dependent upon the classification of data stored, all layers of the data warehouse should include:

Authorisation and authentication;

Access audit;

Data-driven security; and

Data masking or redacting of sensitive information.

Business glossary

A glossary tool that can be used by users to understand the definition of a term and/or the method used to calculate specific measures.

Performance

BI tools should be evaluated on their performance when reporting upon expected data volumes. Specific performance based aspects are:

Time taken for report or query execution;

Ability to pre-execute and cache common reports; and

Ability to make use of pre-aggregated figures in the database or a BI cache.

Semantic layer

BI tools should include a semantic layer that isolates the user from the underlying database. Such a layer will allow the inclusion of plain-English names, and where appropriate allow the buyer to embed specific business rules, calculations, prompts and filters for reuse by report users.

OLAP

Online Analytical Processing (OLAP) should be available in advanced solutions to allow rapid what-if, ad-hoc queries, filtering and drill-down capabilities.

Predefined reports

The following aspects of the reporting environment should be considered by the buyer of a BI solution to ensure that it meets the buyer’s needs:

Report functionality (e.g. tables, graphs, control breaks, in-report calculations, report user prompts, ad-hoc report capability);

Report distribution (e.g. mobile devices, periodic emails, web access);

Scheduling (e.g. automatic execution and distribution);

Reporting environment (e.g. provision of a WYSIWYG editor for creation/update of reports);

Drill down and hyperlinking capability; and11

Page 12: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

User sand-pit for creation of private reports and subsequent sharing/publication.

Self-service administration

The ability to automatically provision and de-provision for all agency resources within the system, together with other appropriate administration and management tasks that can be delegated from the service provider that do not impinge on the solution being provided to other customers.

Full-service administration

All provisioning, de-provisioning, together with all other administration and management tasks required to operate the environment, are provided as part of the service offering. The only exception will be service management of the provider which remains the sole responsibility of the initiating agency.

Cloud compliant hosting facility

All relevant cloud services for the solution may be provisioned from a compliant hosting facility. Compliant hosting is defined as having the following attributes and/or capabilities:

The location of the hosting facility must be identified either by name and/or location (city and country) in any response.

The hosting location cannot be changed without first informing the agency concerned.

The service provider undertakes, maintains and provides access to SSAE 16 Service Organization Control (SOC) Type II reports (or equivalent) for the services and facilities in scope for the engagement.

The hosting facility must comply with minimum Tier 3, as defined by the Uptime Institute, ANSI TIA-942, or an equivalent industry standard.

The hosting facility must be certified against ISO 27001; compliance with the following international standards is desirable:

o ISO 9001

o ISO 27002

o ISO 20000-1:2011

o ISO 14001

Other desirable certifications may include, but are not limited to:

o PCI-DSS v3.0 or later

o Australian Signals Directorate

o ASIO-T4

o Uptime Institute

o CSA

Also consider contractual obligations relating to the service provider allowing security assessments and treatment of outcomes as agreed with the client.

If the hosting facilities changes to a location that is deemed unacceptable either to NSW Government or to the agency and/or loses attributes and/or capabilities identified above, the agency may need to consider termination of services.

NSW Government Data Centre

All relevant services for the solution may be provisioned from one or both NSW Government Data Centre (GovDC). Depending on the service offering and agency requirements, it may be possible to ‘burst’ some elements of services to other location(s) subject to agreement with the commissioning agency.

12

Page 13: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

Burst data centres must be deemed ‘compliant’. If the ‘burst’ data centre facilities change to a location that is deemed unacceptable either to NSW Government or to the agency, the agency may need to re-examine the ‘burst’ service or the full service.

Onshore/offshore management

All solution providers must be able to articulate where their services will be provided from, including any remote support services. For example, with a ‘follow the sun’ support model, the locations of each of their support sites around the globe need to be identified. Any changes to these need to be communicated to the customer agency promptly and if this causes issues, the agency has the right to cancel the service with appropriate notification.

Service level management

Agencies will retain ultimate responsibility for service level management in any solutions engagement, which would ordinarily be covered by a service level agreement (SLA). Agencies, service-brokers and solution providers need to agree all SLA reporting and other related activities as part of any transition-in process.

Multi-service broker provision

Any solution provider must work within the confines of a multi-service provider environment where either the agency or nominated provider will perform broker service provision. This will be defined as one provider being made accountable for the provision of all associated services, whether these are provided by the provider itself, or other third-party providers.

Data sovereignty

Ownership of the business information stored within the data warehouse or analytics solution must remain with the buyer and not be transferred to the service provider.

Back up and recoverability

Information held within the data warehouse solution should be backed up in accordance with the criticality of the data to the client agency. Business requirements should be considered that specify the frequency of backup, any requirement to store back-ups off-site, and the time to recovery of the service in the event of outage.

Archive and retention

The data warehouse solution should provide the capability to retain data for the duration required by the buyer for business or regulatory needs. Following expiration of this retention period, appropriate archive facilities should be available to maintain performance.

Audit

Each tier of the data warehouse or analytics solution should provide audit of information access and configuration change appropriate to the classification of the information stored – for example this may not be required for public domain information, but particularly stringent for personally identifiable client records.

Examples of audit capability may include audit of BI report executions, database logins and changes to the ETL configuration.

13

Page 14: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

DOCUMENT CONTROL

Document historyStatus: Published

Version: 1.1

Approved by: PTS WG

Approved on:

Issued by: Department of Finance, Services and Innovation

Contact: Derek Paterson - ICT Services, Services & Digital Innovation, Department of Finance, Services & Innovation

Email: [email protected]

Telephone: (02) 9372 7445

Review This standard will be reviewed as required.

14

Page 15: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

APPENDIX A – DEFINITIONS

These definitions are based on Gartner (2015).

Analytics

The process of analysing information from a particular domain, such as website analytics. It could involve applying the breadth of BI capabilities to a specific content area (for example, sales, service, supply chain etc.).

Increasingly, analytics is used to describe statistical and mathematical data analysis that clusters, segments, scores and predicts what scenarios are most likely to happen.

Data warehousing services

Offerings for warehousing data extracted from transaction systems, operational data stores and/or external sources. The warehouse combines data in an aggregate, summary form suitable for enterprise-wide data analysis and reporting for predefined business needs.

The five components of a data warehouse are: production data sources, data extraction and conversion, the data warehouse database management system, data warehouse administration and BI tools.

Business intelligence services

Offerings to design, develop and deploy enterprise processes and to integrate, support and manage the related technology applications and platforms. These include business and infrastructure applications for BI platforms, analytics needs and data warehousing infrastructure.

15

Page 16: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

APPENDIX B – ABBREVIATIONS

AIIA Australian Information Industry Association

ASCII American Standard Code for Information Interchange

ASIO Australian Secret Intelligence Organisation

BI Business Intelligence

COTS Commercial Off The Shelf

CSA Canadian Standards Association

ETL Extract, Transform, Load

GovDC Government Data Centre

ICT Information & Communications Technology

ISO/TC International Organization for Standardization / Technical Committee

IT Information Technology

LDAP Lightweight Directory Access Protocol

OLAP Online Analytical Processing

PCI-DSS Payment Card Industry – Data Security Standard

PTS Procurement & Technical Standards

SLA Service Level Agreement

16

Page 17: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

APPENDIX C – REFERENCES Agencies should have regard to the following statutes, NSW Government policies and standards:

AS/NZS ISO 31000 Risk management – Principles and guidelines Copyright Act 1968 Electronic Transactions Act 2000 Government Information (Public Access) Act 2009 Health Records and Information Privacy Act 2002 NSW Government Data and Information Custodianship Policy NSW Government Digital Information Security Policy NSW Government Open Data Policy NSW Government Cloud Policy NSW Government Standard for Data Quality Reporting NSW Government ICT Strategy NSW Government ICT Technical Standards – Mobility Standard NSW Government Information Classification and Labelling Guidelines NSW Procurement: Small and Medium Enterprises Policy Framework Privacy and Personal Information Protection Act 1998 Public Finance and Audit Act 1983 Public Interest Disclosures Act 1994 State Records Act 1998 TPP 09-05 - Internal Audit and Risk Management Policy for the NSW Public Sector

17

Page 18: [Policy Name] - procurepoint.nsw.gov.au€¦  · Web viewThis is a technical standard developed through the NSW ICT Procurement and Technical Standards Working Group. The standard

Analytics and Data Warehousing Solutions Standard

APPENDIX D – STANDARDS

Developing technical standardsDevelopment of a standard begins with identifying the need for a new standard, which is followed by the development of the standard in consultation with the industry and experts groups, including the Australian Information Industry Association (AIIA).

The following diagram outlines the process.

The ICT Procurement and Technical Standards Working Group (PTS Working Group) is chaired by the Department of Finance, Services & Innovation, and includes senior representation from across NSW Government.

Agencies engage with the PTS Working Group concerning services for inclusion in the ICT Services Catalogue. This drives the development of technical standards, where none exist. The PTS Working Group has the leading role in reviewing and endorsing the technical standards developed in response to agencies’ requirements.

The PTS Working Group is supported by two sub-groups responsible for the areas of Telecommunications and Services and Solutions. The sub-groups are responsible for initial development and review of standards relating to their areas of responsibility.

Management and implementationThere is scope to modify standards through the NSW Government ICT governance arrangements as necessary. Standards are designed to add value, augment and be complementary to, other guidance, and they are continually improved and updated.

This standard does not affect or override the responsibilities of an agency or any employee regarding the management and disposal of information, data, and assets. Standards in ICT procurement must also address business requirements for service delivery.

NSW Procurement facilitates the implementation of the standards by applying them to the goods and services made available through the ICT Services Catalogue.

18

Need for new or amended standard

identified

Standard developed (Industry/agencies

consulted)

Standard approved and released by PTS

Working Group

Market engagement for services which meet the standard

Services added to Catalogue

Business requirements change