Introducing Pentaho reporting solutions for CERN’s ... · Pentaho reporting solutions 16...

Preview:

Citation preview

Introducing Pentaho reporting solutions for CERN’s administrative dataGabriele Thiede - CERN

06.03.2018 1

? !

What is CERN?

• CERN = "Conseil Européen pour la Recherche Nucléaire"

• Founded in 1954

• Situated in Switzerland, Geneva

• 12 founding Member States: Belgium, Denmark, France, the Federal Republic of Germany, Greece, Italy, the Netherlands, Norway, Sweden, Switzerland, the United Kingdom, and Yugoslavia

• Mandate:Establish a world-class fundamental physics research organization in Europe

CERN Member States – 64 years later

The Activities of CERN

• Probing the fundamental structure of the universe

• Study the basic constituents of matter - the fundamental particles -and the forces acting between them

• Constructing and running particle accelerators and detectors

How CERN is financed

• Budget: ca. 1 Mrd. CHF per year

• Calculation of contributions based on GDP (gross domestic product) of each Member State.

• Additional contributions from Associate Member States, Associate Member States in the pre-stage to Membership etc.

People at CERN

Total: 21.840

• Members of the personnel (17.540)o Employed members of the personnel (3.440)

o Associated members of the personnel (14.100)

• Contractor’s staff + Temporary Labour (4.300) 16%

64%

20%

Employed members of the personnel

Associated members of the personnel

Administrative data and reporting

Administrative Domains at CERN

9

The Classics

Human resources Finance Purchasing Logistics

Extras

Training Access Transportation Accomodations

Social security

Health insurance Pension Fund School Fees Unemployment benefits

Host states

Customs formalities Removals Licence plates

Work and residence

permits

06.03.2018

Infrastructure – current situation

3/7/2018 10

Infrastructure – coming soon

11

• Analyser

• Report Designer

• C-Tools

Pentaho

PDI

06.03.2018

Time Frame

12

• Start: 2016

• End: 2018

= 2,5 years for• Replacement of data warehouse solution

(responsible service: Infrastructure section)

• In parallel: Migration of existing reporting(responsible service: Reporting section)

06.03.2018

Migration of existing Reporting

13

• Ca. 380 existing Business Objects reports

• Covering all administrative domains

• From small & simple to huge & complex reports with over 80 tables and 20 graphs

• Self service reports, scheduled reports

• For internal and external clients

06.03.2018

Resources for migration

14

• 3 permanent staff (avg. 1,87 FTE/year)• Migration in addition to regular work

• Temporary help, short term contracts (5 persons = avg. 2,2 FTE/year)• Dedicated to migration, only

06.03.2018

Project Organisation

15

Inventory Migration Plan

• List of all used reports

• Main characteristics

• Frequency of usage

• Data sources

• Client(s)

• Priorities• Complexity

• Availability of DWH tables

• Preferred Pentaho solution

06.03.2018

Pentaho reporting solutions

16

• Pentaho Analyser• Predefined reports (scheduled + on demand)

• Analyser dashboards

• Ad-hoc analysis on cubes

• Report Designer (scheduled + on demand)

• CTools dashboards

06.03.2018

Requirements for report content

Provision of data in DWH

Implementation of new report

Organisation of work with clients

17

Analysis of needs

Inventory of BO reports

Discussion and identification of solutions

Investi-gation of solutions

TestImplementa

tionUseRelease

06.03.2018

Interim Balance

Where are we now?

19

• 4 temporary resources trained, 1 in training

• 17 dashboards developped

• Ca. 280 reports migrated

• Backlog: reports for training and recruitment

• Many new technical competencies acquired

06.03.2018

Our experience with Pentaho (1/3)

20

Pentaho Analyser for specialised users

Comfortable tool to establish ad-hoc data analysis

Not self explanatory, training necessaryOccasional users get lost and refer to reporting service

Consequences: Desired effect (more autonomous users) did not materialize

06.03.2018

Our experience with Pentaho (2/3)

21

Access right management of Analyser

Separate access rights to report folders and to cubes

Access right to corresponding cube necessary to executeAnalyser reportNo possibility to disable layout panel and list of available fields

Consequences: For users without access to cube ony Report Designer reports possible

06.03.2018

Our experience with Pentaho (3/3)

22

Pentaho‘s strenghts and weaknesses

High acceptance of end users:- Attractive reports and dashboards with Pentaho; Analyser

Developpers see room for improvement:- Report Designer cumbersome- Pentaho components patch work, missing integration- Poor scheduling and search functions and graphs

Consequences: no use of Interactive reports; inhouse development of scheduler and search tool; Excel workaround for graphs

06.03.2018

1st Lesson learned

23

Parallel work on infrastructure and reporting

User need drivenReporting serves as validation

Time consuming, delaysFixes of infrastructure impact reporting (rewrite of queries)

Recommendation: Infrastructure („data lake“ like) before reporting

06.03.2018

2nd Lesson learned

24

Temporary short term resources for reporting

Working on Pentaho is attractive, many interested candidates

Business knowledge (important but complex!) is lackingTraining adds workload to permanent staffKnowledge transfer at departure

Recommendation: avoid short term resources

06.03.2018

3rd Lesson learned

25

Version control and documentation

• Preserve and transfer of knowledge

• Coordinate team work

• Use of tools like GIT and Sourcetree, JIRA

06.03.2018

Conclusion

26

Migration aslearningprocess

Migration as chance

• Technical knowledge aboutPentaho

• Reporting techniques

• Project management

• Massive tidy up

• Synergy effects

• Standardisation and harmonisation

• Improvement of data and reporting quality

06.03.2018

Thank you for your attention

Recommended