ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ExtendingBIRTwithgeospatialdata
visualizationcapabilitiesbyintegratingthe
SOLAPLayersmappingcomponent
BachelorThesisDepartmentofComputerScience
UniversityofAppliedScienceRapperswil
SpringTerm2011Author: ChristophSüessAdvisor: Prof.StefanKellerProjectPartner: Spatialytics,QuébecCity,CanadaExternalCo‐Examiner:ClaudeEisenhutInternalCo‐Examiner: Prof.HansjörgHuser
BachelorthesisSS2011 Page2/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
1 Abstract
Department ComputerscienceNameofstudent ChristophSüess
Academicyear Springsemester2011Titleofthebachelorthesis Extending BIRT with Geospatial Data Visualization
capabilities by integrating the SOLAPLayers mappingcomponent
Examiner Prof.StefanKellerTopic Reportingtools/OSBI/GeoBIProjectpartner SpatialyticsInstitution InstitutfürSoftwareHSRSummaryToday, Business Intelligence (BI) is an important part for everymodern company. BIallowsanalyzingalargeamountofdata.Theoverviewovertheinformationsimplifiestotakedecisions,whichinfluencethestrategiesofthecompany.ThedifferentpartsofBIaresupportedbyalotofproprietaryandopensourceapplications(OSBI).Spatialytics from Québec, Canada is a small start‐up company, which established in2009.ThemaingoalofSpatialyticsistoprovidetheuseofgeo‐spatialdatafordifferentOSBI‐tools.Spatialyticshasthreesoftware‐toolswhichcoveringdifferentaspectsoftheBI‐process. One of these solutions is SOLAPLayers. SOLAPLayers is a reporting tool,whichdisplaysdatafromdifferentdatasourcesonaninteractive,webbaseddashboard.Themainfeature is thepossibilitytoretrievegeo‐spatialdataanddisplay itonamapcomponent.This bachelor thesis forced to restructure the existing SOLAPLayers 2.0 version, toprovideamore flexible, extendableanddynamical softwarecomponent,whichcanbeintegrated into other considerable reporting tools. The new version allows providingnew data‐source drivers and output formats in an easy way to the framework. In asecond step, a driver for relational databases has been added to the application. ThisdriverwasnecessarytoextendtherangeofpotentialusersofSOLAPLayers,sincenotevery company owns a datawarehouse. The resulting software is not a final version.Moredata sourcesandother featureswill beadded toSOLAPLayersbeforeprovidingthesoftwaretothepublic.The second goal of this projectwas to create amap component for the popular opensource reporting toolBIRT.BIRT is onof themainprojects of theEclipse foundation.The founder andmost active collaborator is the companyActuate. Based on differentfacts,thedecisiontointegrateSOLAPLayersintoBIRTwasdone.Finally,SOLAPLayersisusedasdatasourceandreportitemofthenewBIRT‐plugin.Thisdocumentcontainstheanalysisandtheprojectdocumentationofbothpartsofthisbachelorthesis.Additionalanexcursiononthetopic“stateoftheartofreportingtools”can be found in the appendix of this document. The whole thesis was produced byChristophSüessatSpatialyticsinQuébecCAduringathreemonthlonginternshipandsupervisedbyProf.StefanKellerattheHochschulefürTechnikatRapperswilCH.
BachelorthesisSS2011 Page3/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
2 ManagementSummary
In den 1990er Jahren wurde ein Verfahren mit dem Namen Business Intelligencepopulär.HinterdemBegriffBusiness Intelligence ‐kurzBI ‐ verbirgt sichderProzesszursystematischenAnalysevonDaten,welchemeistinelektronischerFormvorhandensind. Dieses Werkzeug wird von Unternehmen genutzt um sich eine Übersicht überdiverse Zahlen zu verschaffen und die Erkenntnisse in die Strategien einfliessen zulassen. Somit können die Unternehmensziele besser verfolgt und kontrolliertwerden.Das BI‐Verfahren kann grob in drei Phasen unterteilt werden. In der ersten Phasewerden die Daten zusammengetragen. Diese Daten werden normalerweise in einemsogenanntenData‐Warehouseabgelegt. InderzweitenPhasewirdeinZusammenhangzwischendenInformationengebildet,sodassausdenDatenSchlüssegezogenwerdenkönnen. In der letzten Phase, werden die Fakten dem Unternehmen präsentiert unddafürgesorgt,dassdieErkenntnissegenutztwerdenkönnen.DaessichmeistensumgrosseDatenbeständehandelt,istesnichtverwunderlich,dassdieserProzessdurchdiverse IT‐Systemeunterstütztwird. Sowurden imVerlaufederJahre Software‐Komponenten für alle drei Phasen des BI‐Verfahrens entwickelt.Inzwischen gibt es mehrere Unternehmen, die etablierte Applikationen für den BI‐Prozess vermarkten. Die wohl namhaftesten Produkte sind diejenigen des weltweittätigenUnternehmensSAP.
In den letzten Jahren wurde das Vertrauen inOpen Source Softwarelösungen durch effizienteund erfolgreiche Produkte gestärkt. VieleAnwender erkannten die Stärken von OpenSource und legten das weiterverbreiteteMisstrauengegenüberderfreienSoftwarenieder.DieserTrendsetzesichauchinderBusinessweltfort. Neben den proprietären BI‐Tools entstandfreie Software, welche mit OSBI (Open SourceBusiness Intelligence) betitelt wurde. MancheProdukte unterstützen den gesamten BI‐Prozessund andere konzentrieren sich auf bestimmteTeile, wie zum Beispiel dem Zusammentragen,der Zurverfügungstellung oder dem AuswertenderInformationen.
2009 entstand inQuébecCity inKanada das Start‐UpUnternehmen Spatialytics.Spatialytics ist nachwie vor einsehr frischesUnternehmen,dassnoch diverse Entscheidungentreffenmuss.DieFirmagingauseinigenProjektenausderGeoinformatik,welcheanderUniversitéLavalFuss fassten,hervor. Die treibende Kraft hinter dem Projektwar Prof. Thierry Badard. Spatialyticskannte den BI‐Markt und interessierte sich dafür, den Aspekt der räumlichenInformation der Daten in einem Data‐Warehouse für spezifischere Auswertungen zu
Figure1‐LogovonpopulärenOSBI
Figure2‐LogodesUnternehmesSpatialytics
BachelorthesisSS2011 Page4/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
nutzen. Ein populäres Zitat aus der Geowelt besagt, dass achtzig Prozent allergespeichertenBusinessdateneinenräumlichenBezughaben.BiszumheutigenTagwirddieserAspektjedochvonvielenBI‐undOSBI‐Systemenzuweniggenutzt.DerräumlicheGesichtspunktistaberwiederZeitlicheinderWirtschaftsehrwichtig.SpatialyticsverfolgtimMomentdreiProjekte.MitGeoKettlebietetdasUnternehmeneinETL‐Tool (Extract, Transform, Load) an, welches neben den herkömmlichenDatenformaten auch mit diversen geometrischen Datenformaten umgehen kann.Mondrian isteinOLAP‐Server,einSystemdasperformanteundkomplexeZugriffeaufein Data‐Warehouse ermöglicht. Spatialytics erweiterte dieses quelloffene Projektmitdem räumlichen Aspekt, so dass die resultierenden Daten auch geometrischeInformationenenthaltenkönnen.DarausentstanddasProduktGeoMondrian.Das dritte wichtige und im Zusammenhang mit der Bachelorarbeit interessantesteProjekt von Spatialytics, widmet sich dem Auswerten der Daten. Mit SOLAPLayerskönnen Daten aus verschiedenenDatenquellen abgefragtwerdenundin einem interaktiven Reportdargestellt werden. Natürlich liegtauchhierwiederdasHauptmerkmalauf der Präsentation dergeografischen Daten. Das Herz vonSOLAPLayers bildet die interaktiveKarte, welche komplexegeografische Zusammenhängeabbilden kann. In Zukunft willSpatialytics diese geometrischenFeatures vonSOLAPLayers auch fürnamhafte Open Source ReportingTools anbieten. Dazu gehörenProdukte wie Pentaho Reporting,JasperReportsundBIRT.Diese Bachelorarbeit forcierte die Integration einer interaktiven Karte in Berichte,welche mit Hilfe von BIRT erstellt werden können. Es war nicht gegeben, abernaheliegenddasmanhierfürdieexistierendeSOLAPLayers‐Lösungnutzte.DasZielwareinPrototypzuerstellen,welchervorallem fürDemosverwendetundgegebenenfallsspäterfürdenErnstfallerweitertwerdenkann.Die erste Version von SOLAPLayers war noch nicht bereit um in eine bestehendeSoftware eingegliedert zu werden. Darum begann man an einer neuen Version zuarbeiten. Neben dem Hinzufügen von neuen Features, stand vor allem dasUmstrukturieren von SOLAPLayers im Mittelpunkt. Die neue Version sollte flexiblersein, so dass die Applikation einerseits besser erweitert und andererseits einfach inDrittapplikationen eingegliedertwerden kann. Die Umstrukturierung ging zu GunstendieserBachelorarbeit.DasResultaterlaubtnunneueDatenquellenundAusgabeformateflexiblehinzuzufügen.DerAufwandbeschränktsichdabeiaufdasEntwickelnvonneuenTreibern, die Integration ist mit sehr geringem Aufwand verbunden. Neben dieserUmstrukturierung wurde SOLAPLayers durch einen neuen Datenquellen‐Treiberergänzt. Ab sofort können auch relationale Datenbanken mit SOLAPLayers benutztwerden. Zuvor wurden nur SOLAP‐Quellen unterstützt. Die Unterstützung vonrelationalenDatenbankenwarausmehrerenGründenüberfällig.Entscheidenwar,dass
Figure3‐SOLAPLayersDemoDashboard
BachelorthesisSS2011 Page5/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
nachwievornichtalleUnternehmenmit fürdieAnalyseoptimiertenData‐Warehousearbeiten. Öfters sind die Daten in einfacheren relationalen Datenbanken vorhanden.Auch wenn die Verwendung solcher Datenquellen, auf Grund von Performance‐Einbussen, nicht empfehlenswert ist, sollte ein Reporting Tool deren Anbindungunterstützen.NachdemFertigstellendieserZwischenversionvonSOLAPLayerswurdedieEntwicklunganSOLAPLayersausserhalbderBachelorarbeitweitergeführt.
Die Bachelorarbeit widmetesichnunderBIRTIntegrationvon SOLAPLayers. DieMöglichkeit der Integrationwurde zuerst analysiert unddiverse Szenarienbesprochen. DieSchlussfolgerung zeigte, dasses Sinn macht SOLAPLayersals Datenquelle sowie zurPräsentation der Daten zuverwenden. Die Integrationstelltesichalsschwierigeralsgeplant heraus. Trotzdemkonnte das geplante Szenarioumgesetztwerden.Neben der resultierendenSoftware wurde ein Bericht
zumThema„StateoftheArtofReportingTools“erstellt,welcherdieaktuelleSituationvonToolszurErstellungvonBerichtenaufzeigt.DieUmständemachtendasProjektnichteinfach.DadasProjekt ineinemsehr jungenUnternehmen, welches sich schnell entwickelt, durchgeführt wurde, konnten vieleAbhängigkeiten nicht vermieden werden. Des Weiteren existierte für SOLAPLayerskeine Dokumentation und auch das sehr etablierte Tool BIRT ist leider sehr schlechtdokumentiert,wasdieErweiterungdesReportingToolsbetrifft.DieKommunikationansich war jedoch sehr gut, obwohl eine begrenzte Sprachbarriere vorhanden war.Schwierigkeiten brachten aber auch die verschiedenen Interessengruppen mit sich.Einerseits sollte meine Arbeit bei Spatialytics das Unternehmen weiterbringen undandererseits war das Bachelorprojekt gewissen Richtlinien unterstellt. DieseKoordinationerfordertevielDurchhaltewillenundAufwand.
Figure4‐BIRTReportmitSOLAPLayers‐Karte
BachelorthesisSS2011 Page6/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
3 Scope
Extending BIRT with Geospatial Data Visualization capabilities byintegratingtheSOLAPLayersmappingcomponentBachelorarbeitvonChristophSüessimAuslandAbteilungInformatik,Frühjahrssemester2011
AusgangslageReporting‐Programme(Berichterstellungsprogramme)automatisierendieAnalyseundAufbereitung von Informationen. Damit lassen sich Berichte (Reports) in Form vonTabellen und Businessgrafiken (Charts) erstellen und in den gängigen Formaten (z.B.HTML, PDF, XLS und CSV) ausgeben. Die Berichte können nach der einmaligenKonfigurationimmerwiedererzeugtwerden.“Business Intelligence Reporting Tools” (BIRT http://www.eclipse.org/birt/) ist einsolchesReporting‐Programm.Esbasiert aufEclipseund ist JavaOpenSource (EclipsePublicLicense).UmeinenReportzuerstellen,musszunächsteineVerbindungzueinerDatenquelle,z.B.zu einem Data Warehouse, hergestellt werden. Dann kann der Report mit Hilfe des"BIRTReportDesigners"interaktivdefiniertwerden.FürflexibleAnfragengibtesaucheine Scriptingsprache. Das Resultat ist eine XML‐Report‐Design‐Datei. Der eigentlicheReportwirdmitder"BIRTReportEngine"(Laufzeitumgebung)erzeugt.DieseSoftwarelässtsichauchgutinWebapplikationen(Java/J2EE)integrieren.WirdeinerInformationeinRaumbezugzugeordnet(z.B.eineAdresseoderKoordinate),sosprichtmanvonGeodaten.Geodatensindkomplex,aufwändiginderErfassungundRechenintensiv in der räumlichen Auswertung (z.B. Umkreissuche). Sie werdentypischerweise in bzw. Geo‐Datenbanken (‐Erweiterungen) gespeichert und mittelsGeo‐Informationssystemen(GIS)verwaltet.Für die räumlich erweiterte Prozessierung von Informationen aus Data Warehouses(sog. „SpatialOnlineAnalytical Processing“, SpatialOLAP, bzw. SOLAP)undv.a. derenDarstellung in einer Onlinekarte, gibt es die JavaScript‐Bibliothek „SOLAPLayers“.SOLAPLayers basiert auf den Bibliotheken GeoExt/OpenLayers (vgl.http://www.spatialytics.org/?page_id=10). ImBereich SOLAP hat sich die Univ. Laval,QuébéceinenNamengemacht.
AufgabenstellungZurzeitwerdenvonBIRT‐ReportsnurBusinessdiagrammeunterstützt.Wasfehlt,istdieAusgabealsOnlinekarte.DieAufgabederArbeitbestehtdaherausfolgendenTeilen:torestructuretheSOLAPLayersapplicationontheserverside,toprovideamoreflexibleapplicationwhichallowstoaddnewdatasourcesandoutputformatsinaneasyway
BachelorthesisSS2011 Page7/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
to analyse how itwill be possible to add amap component in the toolbox offered byBIRTandwhich features this componentwill have toprovide (access to some spatialdatasources,thematicmappingcapabilities,etc.)to analyse how itwill be possible to add amap component in the toolbox offered byBIRTandwhichfeaturesthiscomponentwillhavetoprovidetodevelopthecodeinordertoeffectivelycarryoutthisintegration.Towriteanexcurseonthetopic:„ThestateoftheartofOSBI‐tools“
BeteiligteDiplomand: ChristophSüess([email protected])Projektpartner: Prof.ThierryBadardundMitarbeiterdes
CentreforResearchinGeomatics(CRG),DépartementdesSciencesGéomatiques,UniversitéLaval,Québec,Qc,G1V0A6,Canada.
BetreuungHSR,VerantwortlicherDozent: Prof.StefanKeller,IFS‐HSR([email protected])
Gegenleser: Prof.HansjörgHuserExperte: ClaudeEisenhut,EisenhutInformatik
Burgdorf([email protected])
Projektabwicklung
TermineBeginnderArbeit: 28.03.2011Abgabetermin: 17.06.2011(12.00Uhr).Weitere Termine: Siehe https://www.hsr.ch/Termine‐Diplom‐Bachelor‐und.5142.0.html(intern)
ArbeitsaufwandFürdieerfolgreichabgeschlosseneArbeitwerden12ECTSangerechnet.DiesentsprichteinerArbeitsleistungvon360Stunden.
HinweisefürdieGliederungundAbwicklungdesProjektesGliedernSieIhreArbeitin4bis5Teilschritte.SchliessenSiejedenTeilschrittmiteinemMeilenstein ab. Definieren Sie für jedenMeilenstein,welche Resultate dann vorliegenmüssen!FolgendeTeilschrittebzw.MeilensteinesollteninderPlanungvorgesehenwerden:
• Schritt1:Projektauftraginkl.Projektplan(mitMeilensteinen),• Meilenstein1:ReviewdesProjektauftragesabgeschlossen.Projektauftragvon
AuftraggeberundDozentgenehmigt Termin:ca.zweiWochennachBeginnderArbeit
• LetzterMeilenstein:Systemtestsabgeschlossen Termin:ca.eineWochevorAbgabe
BachelorthesisSS2011 Page8/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• DieSoftwareistineinemagilen,d.h.iterativenundinkrementellenProzesszuentwickeln:ManplanemöglichstfrüheinenerstenlauffähigenPrototypenmitdenwichtigstenundkritischstenKernfunktionen.IndenfolgendenPhasenkanndannschrittweiseausgebautundgetestetwerden.
• FallsinderArbeitneueoderunbekannteTechnologieneingesetztwerden,solltemanparallelzumErarbeitendesProjektauftragesmitdemTechnologiestudiumbeginnen.
• EssindUnit‐Testseinzusetzen.SoftwareundDokumentewerdenaufeinemRepositoryverwaltet(z.B.SVN,git).Ev.isteinBuildServerhilfreich(z.B.CruiseControl).
• ManhaltesichimÜbrigenandieVorgabenausdemModulSE‐Projekt.• Projektadministration:• EsisteinProjekttagebuchzuführenausdemersichtlichwird,welcheArbeiten
durchgeführtwurden(inkl.ungefähremZeitaufwand).DieseAngabensolltenggf.eineindividuelleBeurteilungermöglichen.
• DieArbeitensindlaufendzudokumentieren.ManlegedieProjektdokumentationmitderaktuellenPlanungunddenBeschreibungenderArbeitsresultateelektronischineinemProjektordnerab.DieserProjektordnersolltejederzeiteinsehbarsein(z.Bsvn‐ServeroderFile‐Share).
Fortschrittsbesprechung:• RegelmässigfindetzueinemfixenZeitpunkteineFortschrittsbesprechungstatt
(ArbeitenimAusland:Wochenbericht).• TeilnehmersindDozentundStudenten,beiBedarfauchVertreterder
Auftraggeber• Termingem.Absprache(ArbeitenimAuslandz.B.Montags).• Fallsnotwendig,könnenweitereBesprechungen/Diskussioneneinberufen
werden.• SieerstellenzujederBesprechungeinKurzprotokoll,welchesSiespätestens2‐3
TagenachderSitzungperE‐MailandenBetreuersenden(ArbeitenimAusland:EintragimWikio.ä.).
InhaltderDokumentation• DiefertigeArbeitmussfolgendeInhaltehaben:
1. Abstract,ManagementSummary,Aufgabenstellung2. TechnischerBericht3. DokumentedesProjektdokumentation4. Anhänge(Literaturverzeichnis,CD‐Inhalt)
• Mind.dieDokumentederPunkte1,2sowieInstallationundCodemüsseninEnglischsein.
• DieAbgabeistsozugliedern,dassdieobigenInhalteklarerkenntlichundauffindbarsind.
• Zitatesindzukennzeichnen,dieQuelleistanzugeben.• VerwendeteDokumenteundLiteratursindineinemLiteraturverzeichnis
aufzuführen.• Projekttagebuch,DokumentationdesProjektverlaufes,Planungetc.• WeitereDokumente(z.B.KurzbeschreibungfürBroschüre,Poster)gemäss
www.hsr.chundgemässAbsprachemitdemBetreuer.
BachelorthesisSS2011 Page9/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
FormderDokumentation• Bericht(StrukturgemässBeschreibung)gebunden(2Exemplare)undinOrdner
(1Exemplar„kopierfähig“inlosen,gelochtenBlättern).• AlleDokumenteundQuellendererstelltenSoftwareaufCD;CD'ssauber
angeschrieben(3Ex.).
BewertungsschemaAls Bewertungsschema gilt das in www.hsr.ch erwähnte, d.h. die folgenden Aspektewerdenbewertet:
• Projektorganisation(Gewicht1/6)• Bericht(Gewicht1/6):Inhalt,Gliederung,Sprache• Inhalt(Gewicht½):1.Vorstudie,AnforderungsanalyseundDomainanalyse;2.
Entwurf(Systemarchitektur,BeschreibungdesEntwurfs,EntwurfBenutzerschnittstelle);3.RealisierungundTest
• MündlichePrüfung(Gewicht1/6)Es gelten ansonstendie üblichenRegelungen zumAblauf und zurBewertungderBA‐ArbeitdesStudiengangsInformatikderHSR.Rapperswil,27.Oktober2010,S.Keller
BachelorthesisSS2011 Page10/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
4 Tableofcontents
1 Abstract ...........................................................................................................................................................22 ManagementSummary.............................................................................................................................33 Scope.................................................................................................................................................................64 Tableofcontents ......................................................................................................................................105 Overview......................................................................................................................................................126 SOLAPLayers2.0Extended..................................................................................................................136.1 Overview .............................................................................................................................................136.2 Technicalreport...............................................................................................................................136.2.1 Introduction................................................................................................................................13UsageofSOLAPLayers ....................................................................................................................13SOLAPLayersversions ....................................................................................................................13Guidelines.............................................................................................................................................14Courseofaction .................................................................................................................................14
6.2.2 Stateoftheartofreportingtools ......................................................................................146.2.3 Resultsandevaluation...........................................................................................................15Achievementofobjectives ............................................................................................................15FutureImprovements .....................................................................................................................15
6.3 Projectdocumentation..................................................................................................................156.3.1 Requirementsspecification .................................................................................................156.3.2 Analysis‐ResultsettoOLAP‐cubetransformation ..................................................15WhyusingSQLfordataanalyzing? ...........................................................................................15WhyparsingSQL‐resultstoOLAP‐cubes?..............................................................................15WhatisanOLAP‐cube?...................................................................................................................16Howdoesthetransformationwork?........................................................................................16
6.3.3 Architecture................................................................................................................................17ArchitectureofSOLAPLayers1.0 ...............................................................................................17ArchitectureofSOLAPLayers2.0Extended ..........................................................................18Activitydiagram ................................................................................................................................20Packagediagram/layeroverview............................................................................................21Classdiagram......................................................................................................................................22Technologies .......................................................................................................................................23Clientapplication ..............................................................................................................................24
6.3.4 Implementation ........................................................................................................................25Systemtest ...........................................................................................................................................25Codequality.........................................................................................................................................29
6.3.5 Futureimprovements ............................................................................................................30Chores ....................................................................................................................................................30Extensions ............................................................................................................................................30
6.3.6 Developersguide......................................................................................................................31Installation ...........................................................................................................................................31Addanewconnectiontype...........................................................................................................32Addanewoutputformat...............................................................................................................33Registernewoutputformat .........................................................................................................34Addanewdatasource....................................................................................................................34UsingMDX‐queries...........................................................................................................................35UsingSQL‐queries.............................................................................................................................36
BachelorthesisSS2011 Page11/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Creatingadashboard.......................................................................................................................407 Geoextensions2Birt .................................................................................................................................417.1 Overview .............................................................................................................................................417.1.1 Technicalreport .......................................................................................................................41Introduction.........................................................................................................................................41StateoftheArt....................................................................................................................................41
7.1.2 Projectdocumentation ..........................................................................................................41Requirementsspecification ..........................................................................................................41Analysis .................................................................................................................................................42Architecture.........................................................................................................................................47Implementation .................................................................................................................................48Systemtest ...........................................................................................................................................48Codequality.........................................................................................................................................51FutureImprovements .....................................................................................................................51UserGuide ............................................................................................................................................52Developersguide...............................................................................................................................59
8 Projectmanagement...............................................................................................................................609 Projectreview............................................................................................................................................609.1 Acknowledgment.............................................................................................................................609.2 Fieldreport.........................................................................................................................................609.3 Lessonslearned................................................................................................................................61
10 Furtherdocumentation.......................................................................................................................6111 Appendices ...............................................................................................................................................62APPENDIXA‐ContentoftheCD ..........................................................................................................62APPENDIXB‐Glossary.............................................................................................................................62APPENDIXC‐References........................................................................................................................64APPENDIXD‐Tableoffigures ..............................................................................................................65APPENDIXE–Projectmanagement...................................................................................................67APPENDIXF‐Licenseagreement........................................................................................................81APPENDIXG‐Agreementoftheauthor............................................................................................82APPENDIXH‐Instructions:CreatingadashboardusingSOLAPLayers.............................82APPENDIXI–CustomizedSCRUM‐processforbachelorthesis.............................................84APPENDIXJ–Interimreport .................................................................................................................86APPENDIXK‐Resultpresentations ...................................................................................................88APPENDIXL–Stateoftheartofreportingtools ..........................................................................89
BachelorthesisSS2011 Page12/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
5 Overview
Thisdocument is structured in twodifferentmainparts.The firstpartdocuments theSOLAPLayers improvement,whichwas done at the beginning of the bachelor project.ThesecondpartdescribestheintegrationofSOLAPLayers2.0ExtendedintotheBIRT‐plugin. Both parts contain a technical report and project documentation. Informationabout the project progress and approach can be found at the end of this document.Additionaldocumentsareattachedtothisdocumentintheappendices.Note: SOLAPLayers 2.0 Extended is not an official name of a SOLAPLayers version.SOLAPLayers2.0ExtendedreferstothestateofSOLAPLayers,whichwasreachedattheendofthebachelorthesis‐project.
BachelorthesisSS2011 Page13/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
6 SOLAPLayers2.0Extended
6.1 Overview
Main use of this chapter is to document SOLAPLayers 2.0 Extended. For betterUnderstanding some parts of the SOLAPLayers base ideas will complete thisdocumentation.Thedocumentisstructuredbytwomaintopics,thetechnicalreportandtheprojectdocumentation.
6.2 Technicalreport
6.2.1 Introduction
UsageofSOLAPLayersSOLAPLayers is a product developed by the University of Laval. Spatialytics extendsSOLAPLayerssince2009.SOLAPLayers is a Java‐reporting‐framework, which is used to build interactivedashboardswithmappingcapabilities.SOLAPLayersretrievesdata fromdifferentdatasourcesanddisplaystheinformationondashboards.ThecorefunctionofSOLAPLayersistocreateinteractivemaps.On theonhand,SOLAPLayerscanbeusedasaweb‐basedreporting tool.Dashboardscan be defined using simple HTML‐tags. On the other hand, SOLAPLayers can beintegratedinexistingreportingtoolstoextendthemwithmappingcapabilities.
SOLAPLayersversions
SOLAPLayers1.0
Figure5‐SOLAPLayerslogo
The initial version of SOLAPLayers provided the possibility of querying one singleGeoMondrian‐data sources. The whole project was structured in four classes. Theconnectionstringwasconfiguredintheweb.xml‐file.GeoJSONandOLAPJsonexistedasdatatransport format.Thewholesourcecodewashighcoupledanddidnotprovidealotofreusability.SOLAPLayers1.0usedDojoandOpenLayersfortherepresentationofthemapontheclientside.TheSOLAPLayers1.0sourcecodeisavailablefordownloadat http://sourceforge.net/projects/spatialytics/. SOLAPlayers 1.0 is more a proof ofconceptthananactualversion.
SOLAPLayers2.0SOLAPLayers 2.0 is a revision of the concept of SOLAPLayers 1.0. The client and theserverpartwerereengineered.OntheclientsidetheDojo‐andOpenLayers‐frameworkswere replaced with ExtJS and GeoExt. SOLAPLayers 2.0 supports many morecomponentsfordatarepresentation(crosstab,barchart,barchart,etc.)
BachelorthesisSS2011 Page14/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
SOLAPLayers2.0ExtendedSOLAPLayers2.0ExtendedisnoofficialreleaseofSOLAPLayers,moreoveritreferencesto the state,whichwas reachedafter thebachelorproject. SOLAPLayers2.0Extendedhastwomaingoals:
• Buildaflexibleapplication,whichsimplifiestoaddnewdatasourcesandoutputbuilders.Forreachingthisgoal,thewholesourcecodewillbereviewedandrefactored.
• AddaSQL‐datasourcetoSOLAPLayersforqueryingrelationaldatabases.
GeoBIExt1.0GeoBiExt1.0willbetherenamedproductofthefinalSOLAPLayers2.0release.GeoBiExt1.0willbepresentedattheFOSS4G2011inDenver,Colorado.
GuidelinesSOLAPLayers was an existing project based on some technology‐ and architecture‐desicions and principles they were defined before the bachelor project started. Thisbachelorthesisdidnotevaluatethisdecisionsandcontinuedonthegivenbase.Ontheonehand,thiswasnotadesireofSpatialytics,ontheotherhand,theevaluationwouldbetootimeconsumingforthisproject.Thefollowingkey‐factsaffectedthecourseoftheproject:
• SOLAPLayersretrievesdatafromanydatasourceandsendsittotheclientintheOLAPJson‐format.OLAPJsonisaundocumentedformatinventedbySpatialytics.OLAPJsonrepresentsaOLAPCubeinaJSON‐format.
• Theclient‐sideofSOLAPLayerswasnevertouchedbythestudentduringthisproject.Theclient‐sideparsestheOLAPJson‐fileandrepresentsthedatausinghtml,cssandjavascript.AslongastheOLAPJson‐fileisvalid,thepossiblefailuresontheGUIareproblemsoftheclient‐side.Asmalloverviewoftheclient‐sidecanbefoundinthesub‐chapter“clientapplication”.
• TheOLAPCubetransformation(OLAP4JtoOLAPJson)wasalreadyapartofthe1.0versionDuringtheprojectthesourcecodewassplitedupindifferentpartstomaketheapplicationmoreflexible.Someimprovementsonthesourcecodeandthealgorithmweremadeaswell.Theevaluationofthistransformationwasnotapartofthisthesisandwasneverreviewedindetail.
• Allthekeytechnologydecisionsweredoneinadvanced:o Javaasthemainprogramminglanguageo OLAP4J‐Librarytohandlethreedimensionalmdx‐result‐cubeso The2D‐spatial‐java‐libraryJTSTopologySuite,whichisusedtointerpret
geometry‐formatso JSONasbasedatatransportformato AllthetechnologiesusedbytheclientsideofSOLAPLayers(Javascript,
HTML,CSS,OpenLayersandtheEXTJS‐framework)
CourseofactionThisprojectwasdevelopedinacustomizedSCRUM‐process.Forfurtherinformationsee“APPENDIXI–CustomizedSCRUM‐Processforbachelorthesis”.
6.2.2 StateoftheartofreportingtoolsFor information about the state of the art of reporting tools, consider the document“Stateoftheartofreportingtools”whichisattachedtothisdocument.ForthestateoftheartofSOLAPLAyersseethechapter“SOLAPLayers2.0“.
BachelorthesisSS2011 Page15/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
6.2.3 Resultsandevaluation
AchievementofobjectivesThegoal of thisproject couldbe reached completely. Spatialyticswas satisfiedby thesolution. The version was used for extending SOLAPLayers right after finishingSOLAPLayers2.0Extended.
FutureImprovementsSOLAPLayers2.0ExtendedisnotamajorreleaseofSOLAPLayers.It isnotyetdefinedwhich featuresGeoBIExt1.0willprovide. Somerecommendationsareavailableat thechapter“FutureImprovements”oftheprojectdocumentation.
6.3 Projectdocumentation
6.3.1 RequirementsspecificationThe requirement specificationwas done by Spatialytics in an agileway. The SCRUM‐Taskboardcanbefoundintheappendix(see“APPENDIXE–projectmanagement”).
6.3.2 Analysis‐ResultsettoOLAP‐cubetransformationAbigpartoftheanalysisisalreadydonebySpatialytics.Theusedtechnologiesaregivenand the applicationwill be developed based on the version 2.0 of SOLAPLayers. ThefollowingchaptersdocumenttheanalysisofthesolutionsofSOLAPLayers2.0Extended.The following text shows theway inwhichSOLAPLayers2.0Extended transforms theresultsetofaSQL‐queryintoanOLAP‐cuberepresentation.
WhyusingSQLfordataanalyzing?Why accessing simple relational databases with SQL‐queries instead of using MDX?OLAP‐data sources are much faster for the analysis of a large amount of data, thenaccessing databases using SQL. Anyway, many enterprises still work with simplerelationaldatabases,evenforanalyzingissues.ThatiswhySOLAPLayershastosupportthiskindofdatasourcesaswell.
WhyparsingSQL‐resultstoOLAP‐cubes?The data format is based on a data cube on all logical layers of SOLAPLayers. Thecommondataformat,whichisprocessedbytheoutputbuilders, isbasedonanOLAP‐cube.TheOLAPJson‐format,whichtransportstheresultfromSOLAPLayerstothecallingapplication,representsacubeaswell.Thiswasgivenbytheprojectguidelines.
BachelorthesisSS2011 Page16/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
WhatisanOLAP‐cube?
Figure6‐VisualizationofanOLAP‐cube
OLAP‐cubes result fromMDX‐queries and can be imagine as three dimensional resultsetsrespectivelyasdatacubes(morethanthreedimensionsarepossible).Inthesamplecube above “Product”, “Time” and “Location” are dimensions and “174” is a specificmeasure. OLAP‐cubes are often represented as cross‐tables in reports or dashboards.Furtherinformationcanbefoundonhttp://en.wikipedia.org/wiki/OLAP_cube.
Howdoesthetransformationwork?SQLisnotabletoreceiveanOLAP‐Cube.Thatisthereasonwhyitisnecessarytoparsethe two‐dimensional result set toanOLAP‐cube representation.Thereare somestepsnecessarytoreachagoodresult:
1. RequirementsfortheSQL‐querya. Allthemeasuresanddimensionshavetobeapartoftheselect‐
statementb. Itisnecessarytogroupthedatabythedimensions
Basically,everyrowoftheresultsetwillcontaintwoimportantpiecesofinformation.Thefirstpartoftheinformationcontainsthecoordinates.Thecoordinatesdescribethepositionofthemeasure/s.Thesecondpartoftheinformationisthemeasure,respectivelyabunchofmeasures.Thefollowingexampleshowsthedatacolumn,whichdescribesthevalueinthecubeabove.Product,LocationandTimearethedimensions,whicharepresentinthegroupby‐clauseofthesql‐query.“174”isthemeasure.Withthisinformationitispossibletolocateallgivenmeasuresinthecube.Ifaspecificcoordinateisnotavailableintheresultset,itmeans,thatthevalueofthismeasureiszero.Product Location Time ValueCellphones Euro 2000 174… … … …
c. Anaggregate‐functionhastobespecifiedforthemeasuresBecausewealwaysusethegroup‐by‐clauseitisnecessarytodefineaggregate‐functionsforallthemeasures.
BachelorthesisSS2011 Page17/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Forexample:Themeasureinthecubeabovecouldbetheaveragepriceofacellphoneatacontinentinayear.Inthequerywecalculatetheaveragepriceofallthecountriesofonecontinentusingtheavg‐function.Product Location Time Country ValueCellphones Euro 2000 Spain 171Cellphones Euro 2000 Germany 177 174
2. AdditionalmetadataToextracttheinformationcorrectlyadditionalmetadataisnecessary.Thetypehastobedefinedforeachrowofaresultset:
a. SpatialdimensionWhichrowrepresentsthenameofthespatialdimension?
b. GeometrydataWhichrowrepresentsthegeospatialdata(usuallyapolygonoramulti‐polygon)ofthespatialdimension?
c. GeometrypointrepresentationWhichrowcontainsapointrepresentationofthespatialdimensionsinordertodisplaydataadditionallyasasimplemarkeronamap?
d. DimensionsWhichrowscontaintheotherdimensions?
e. MeasuresWhichrowsrepresentthemeasures?
Forguidelinesandfurtherinformationhavealookatthechapter“UsingSQL‐Queries“inthedevelopersguide.
LimitationsThecube‐transformationisstilllimited.Metadataarenotapartoftheresultingcubeatthemoment.Datacanberelatedtojustonespecificspatialdimension.
6.3.3 Architecture
ArchitectureofSOLAPLayers1.0SOLAPLayers 2.0 has a very basic architecture. The whole process of retrieving andtransforming data is done in one shot. SOLAPLayers does not provide any hooks toextend the software. It is an in four class structured procedural application,which isdeveloped only for the reason of sending mdx‐queries and receiving OLAPJson.SOLAPLayerswasdevelopedtoaccessoverawebservlet.
BachelorthesisSS2011 Page18/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ArchitectureofSOLAPLayers2.0Extended
Figure7‐OverviewSOLAPLayers2.0Extended
The new architecture of SOLAPLayers uses the advantages of objectorientedprogramming.Thedifferentstepsoftheworkflowweresplittedintoflexibleparts.ThisenhancestheextendabilityofSOLAPLayers.Inuseofconfigurationsandparameterstheprozess can be customized easely without touching the core source code. The newarchitecturebringsthefollowingadvantages:
• Thesourcecodeislogicallystructuredindifferentclassesandpackages.Thismakesthesourceclearandunderstandableingeneral.
• Specificdatasourcescanbeconfiguredusinganxml‐file.Nofurtherchangesinthesourcecodearenecessary.
• Newconnectiondrivers(e.g.flatfiles,webservices,etc.)canbeaddedeasily.Interfacesandabstractclasseshelptodevelopandintegratethenewconnectiontype.Nochangesatthecorearenecessary.Definingthepathtothenewdriverinthedatasourceconfigisallwhathastobedone.SOLAPLayersusesreflectiontoloadandconstructthedrivers.
• Scaffoldingsfornewoutputformatsexist.Afterimplementingannewoutputbuilderjustverylittlechangesonthecoresourcecodehastobedone.Infutureversionsthisshouldnotbenecessaryatall(seechapter“Futureimprovements”).
• SOLAPLayers2.0ExtendedprovidesaccesspointsforwebapplicationsaswellasforsystemstheyusingSOLAPLayersasalibrary.
BachelorthesisSS2011 Page19/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Figure8‐OverviewvisionSOLAPLayers2.0Extended
BachelorthesisSS2011 Page20/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Activitydiagram
Figure9‐Activitydiagram
The activity diagram shows the workflow of SOLAPLayers. The following steps wereperformed:
1. TheSeverretrievesthequeryandparameters.Thesourceofthisinformationcanbeaninputofanuserorthedataofasystem,whichusesSOLAPlayersasapartoftheapplication.Inthesamestep,SOLAPLayersreadsthedatasourceconfiguration,whichspecifiytheconnectionforthedatasource(e.g.db‐url,password,user,etc.).Withthequery,theadditionalparamtersandtheopenconnectionthedatafromthedatasourcecanbereceived.
2. Acomondataformatisusefulltobuildoutputsforeachdatasource.Thatiswhythereceiveddatastructuregetstransformedintoacomonformat.ThiscomonformatismainlyaOLAP‐cube‐representationplussomeadditionalyinformation(e.g.theexecutedquery).
3. Thecomondatastructureallowstocreatedifferentoutputformatsforeverykindofdatasource.Theoutputisnotyetsendedtotheclient,thismakesitpossibletoreuseanoutputbuilderforanotheroutputbuilder.Inthiswayreallycomplexoutputbuilderscanbedeveloped.
4. Inthelaststep,theoutputissendedbacktotheclientorthecallingapplication.Dependingonthecallingsystem,someinformationhavetobeadded(e.g.HTTP‐headerinformations).
AlltheactivitysfromaboveareflexiblepartsofSOLAPLayers.Fornewdatasourcesoroutputformatsthisactionshavetobeimplementedandcontributedtotheframework.
BachelorthesisSS2011 Page21/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Packagediagram/layeroverview
Figure10‐Packagediagram
• Theservletpackagecontainsalltheclasses,whicharecalledbythedashboardorthethird‐partysoftware.
• Theconnectionpackagecontainsallcomponents,whichareresponsibletocreateanduseaconnectiontodifferentdatasources.Thedriverpackage,whichisapartoftheconnectionpackage,containsspecificdriversfordifferentdatasources,whicharestructuredindifferentclasses.Thishelpstoholdtheoverviewoverthedifferentdrivers.
• Thedatapackagecontainsclasses,whichworkasdatatransportobjects.Usuallytheseclassesareplaindataclasseswithoutalotoflogic.Theyworkasdatainterfacebetweenthedatasourceandtheoutputbuilder.
• Theoutputpackagecontainsutilityclasses.Theyconstructtheoutputbuilderandhelptocreateoutputs.Concreteoutputbuildersforaspecificformatarelocatedinthebuilderpackageinsideoftheoutputpackage
• TheexceptionpackageholdssomeSOLAPLayersspecificexceptions.Itisusedbyallofthepackages.Theseexceptionsallowamorespecificexceptionhandling.
BachelorthesisSS2011 Page22/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Classdiagram
Figure11‐Classdiagram
TheclassdiagramshowsthemostimportantclassesintheSOLAPLayers‐project.Someclassesarenotdisplayedforbetterunderstandingandoverview.Formoreinformationhavealookatthejavadoc.The light blue colored classes are components of the framework. They regulate theworkflow and call the defined hook classes. The dark blue classes are flexiblecomponents,whichareattached to thehooksof the framework.Theredclassesshownot yet developed classes, they are just displayed to show the possibilities ofSOLAPLayers.
SOLAPLayersProcessorThemain class of SOLAPLayers. The SOLAPLayersProcessor starts the data retrievingandexecutesthedatatransformations.
SOLAPLayersServletSOLAPLayersServlet is the entry point, if SOLAPLayers is used as a webapplication.SOLAPLayersreadstheparametersoutoftheHTTP‐requestandforwardsthemtotheSOLAPLayersProcessor.
ConnectionA Connection gives access to a concrete data source. A ConnectionConfiguration isnecessary for theconstruction.At themoment, twodifferentconcretconnection‐typesare implemented (SOLAPConnection and DBConnection). More Connections will bedevelopedforafurtherversionofSOLAPLayers.
BachelorthesisSS2011 Page23/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ConnectionConfigurationTheConnectionConfigurationcanbeseenasatextualdescriptionofaConnection.TheConnectionConfiguration provides access to the source parameters defined in thesources.xml.AConnectionConfigurationisnecessarytocreateaConnection.
OutputBuilderThisinterfacecontainsthemainoperationforeveryOutputBuilder.Itallowstobuildaoutputoutofrawdataandprovidesaccesstotheresultingouput.
HttpOutputBuilderThisOutputBuilder‐interfaceaddsamethodtosendaHTTP‐responsebacktotheclient.At themoment all of the concrete OutputBuilders implement this interface. Concreteimplementations are OLAPJsonOutputBuilder, SOLAPJsonOutputBuilder,GEOJsonOutputBuilderandtheCellSetToOLAPJsonOutputBuilder.
TechnologiesThe technologies were already given, since this project was an enhancement of anexisting software. This chapter illuminate the libraries and technologies, which wereusedadditionallytotheJavacorelibraries.ThewholeprojectisdevelopedonJavaSE6.
JSON(JavaScriptObjectNotation)The client‐server communication uses the HTTP‐protocol. The server response isformatted as JSON. JSON is a really compact data transport format with a smalloverhead.SinceJSONisaJavaScript‐technologytheinterpretationontheclientsidecanbedoneverysmooth.BecauseofthelargeamountofdataisJSONfarmorecapableforSOLAPLayersthenothertechnologieslikeXML.Furtherinformation:http://www.json.org/
GeoJSONGeoJSONisastandardizedJSON‐formatwiththepurposetotransportgeo‐spatialdata.SOLAPLayerssupportsthegenerationofGeoJSON.Furtherinformation:http://geojson.org/
MapfishMapfish is aopen‐source framework tobuild rich‐mappingapplications. SOLAPLayersuses just a part ofMapfish.Mapfishs allows to createGeoJSON representations out ofJava objects in an easy way. SOLAPLayers uses this library to create the GeoJSONoutputs.Furtherinformation:http://mapfish.org/
Log4JThepopularLog4Jisusedforloggingpurposes.Furtherinformation:http://logging.apache.org/log4j/
MondrianMondrianisanOLAP‐server,whichallowstoanalyselargequantitiesofdatainrealtime.SOLAPLayersusesMondriantodefinetheXML‐SchematomaptherelationaldataintoaOLAP‐cuberepresentation.Furtherinformation:http://mondrian.pentaho.com/
BachelorthesisSS2011 Page24/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
OLAP4JWhat JDBC is forrelationaldatabases isOLAP4J formultidimensional‐queries.OLAP4JsupporttocreateMDX‐queriesandretrievetheresultingdataasOLAP‐Cube.Furtherinformation:http://www.olap4j.org/
POSTGisPOSTGis isaPostegreSQL‐extension,whichallows to storegeo‐spatialdataefficiently.ThePOSTGisdriverisusedtoquerythedatabaseoutoftheJavaapplication.Furtherinformation:http://postgis.refractions.net/
PostgreSQLAllthetestdataisstoredinPostgreSQL‐databases.IngeneraleverydatabasewithJDBCsupportcanbeused,aslongitisabletostoregeo‐spatialdata.Furtherinformation:www.postgresql.org
JTSTopologySuiteTheopen‐sourceJTS‐libraryisusedforrepresentationofgeo‐spatialinformationasJavaobjects.Theresultsfromthedatabasecanbeparseddirectlyintogeometric‐objectsandcanbeusedforfurtherpurposes.Furtherinformation:http://www.vividsolutions.com/products.asp?catg=spaapp&code=jts
OthertechnologiesAll other librarieswhich can be found in the project directory are used by the abovemetionedtechnologies(dependencies)andwillnotbedescripedinthisdocumentation.
ClientapplicationAsmentionedintheguidelineschapter,theclient‐sideofSOLAPLayerswasnotapartofthis project. The client‐side will be described shortly for a better understanding ofSOLAPLayers.SOLAPLayersclientsideisaHTML5.0,JavaScriptandCSSbasedapplication.Itdisplaysmapsandotherdashboardelements,liketables,inthebrowser.Forthepresentationofthe map and the dashboard elements the EXT(http://www.sencha.com/products/extjs/) and GeoEXT (http://www.geoext.org/)librarywere used. The use of this library allows to create easyely really flexible andcomplex dashboards. The transformation of received OLAPJson‐data is done bySOLAPLayers,respectivelyisnotapartoftheusedlibraries.SOLAPLayers 1.0 used Dojo (www.dojotoolkit.org/) as JavaScript‐framework. In thenewSOLAPLayers2.0version,Dojoisnotapartoftheapplicationanymore.
BachelorthesisSS2011 Page25/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
6.3.4 Implementation
Systemtest
GoalThis test plan assures the correct behaviour of the SOLAPLayer 2.0 Extended serverpart.Ademodashboardisusedforthetest,tosendandreceiveinformation.Theobjectof test is the creation of output‐formats using a mdx‐ or sql‐query and specificparameters.ThistestdoesnotverifytheclientpartofSOLAPLayers,sincetherewasnoworkdoneduringthedevelopmentperiodofthebachelorproject.
RequirementsandtestdataThe test data can be found in the Eclipse project in the directory“SpatialyticsMapReportingTool/testqueries/”. The files containing the queries and thequeriesthemselvesarenumbered.Informationabouttheusedparametersperqueryisavailableaswellinthequery‐files.The “Query and parameters”‐column gives information about the used query andparameters. Technical information about the data sources can be found in the file“SpatialyticsMapReportingTool/webapp/WEB‐INF/sources.xml”. Preconditions are notavailable, since the application is stateless and every test conatins the whole A‐to‐Bscenario.
ExpectationsTheresultofeveryqueryisthepresentationonthedashboard(includesamap,atableandachart).Foronegivenmeasureandnodimensions,thepolygonsonthemapwillbeindifferentcolors,whichhighlight theproportionalrelationsof themeasures. Ifmorethanonemeasureoratleastonedimensionexists,thepolygonswillappearinthesamecolor,butincludingabar‐chart,whichshowsthedifferentmeasures.Thisfactswillnotbementionedasexpectationsineverytestcase.
BachelorthesisSS2011 Page26/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
TestplanThistestplanwasaccomplishedon28Marchand1May2011.ThesoftwarewastestedonaLinuxUbuntu20.10onthebrowserGoogleChrome11.
MDX‐queriesFor all the following tests just the parameters “Output Format”, “Source Id” and “Query” have to be set, since the other information isincludedintheMDX‐query.Forallteststhefollowingparametersarepredefined:
• OutputFormat: olapjson Mentionedifohtersource• SourceId: 1
Thequeriesaredefinedinthefile“1‐mdx‐requests.txt”.Number Testcase Queryandparameters Result1.1 MDX‐Query(seequery) Query:1A Pass1.2 MDX‐Query(seequery) Query:1B Pass1.3 MDX‐Query(seequery) Query:1C Pass1.4 MDX‐Query(seequery) Query:1D Pass1.5 MDX‐Query(seequery) Query:1E Pass1.6 MDX‐Query(seequery) Query:1F Pass1.7 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1A Pass1.8 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1B Pass1.9 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1C Pass1.10 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1D Pass1.11 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1E Pass1.12 MDX‐Query(seequery).Usingdeprecatedoutput‐builder. Query:1F Pass
BachelorthesisSS2011 Page27/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
SQL‐queriesIf a parameter is not listed in the “Query and parameters”‐column, it has the default value or stays empty. For all tests the followingparametersarepredefined:
• OutputFormat: olapjson Theonlyformatwhichcanbedisplayeddynamicallyonthemap• SourceId: 3 Mentionedifohtersource• GeometryData: the_geom Mentionedifothervalue• GeometryPointRepresentation: the_geom_point Mentionedifothervalue
Thequeriesaredefinedinthefile“3‐sql‐request‐db_test2.txt”.Number Testcase Queryandparameters AdditionalExpectations Result2.1 Simplequerywithonemeasureand
nodimensions.SpatialDimension:supnameMeasures:nopersonsQuery:3A
Pass
2.2 Simplequerywithtwomeasuresandnodimensions.
SpatialDimension:zip_codeMeasures:nopersons,totalsavingsQuery:3B
Pass
2.3 Simplequerywithonemeasureandonedimension.
SpatialDimension:supnameDimensions:sexeMeasures:nopersonsQuery:3C
Pass
2.4 Querywithtwomeasuresandonedimension.
SpatialDimension:supnameDimensions:sexeMeasures:nopersons,totalsavingsQuery:3D
Pass
2.5 Querywithtwomeasuresandtwodimensions
SpatialDimension:zip_codeDimensions:sexe,civilstatusMeasures:nopersons,totalsavingsQuery:3E
Pass
BachelorthesisSS2011 Page28/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
2.6 Querywiththreemeasuresandthreedimensions
SpatialDimension:supnameDimensions:sexe,civilstatus,agegroupMeasures:nopersons,totalsavings,avgsavingsQuery:3F
Pass
2.7 Querywithonemeasureandtwopotentialspatialdimensions(oneofthemwillworkasanormaldimension).
SpatialDimension:supnameGeometryData:the_district_geomGeometryPointRepresentation:the_district_geom_pointDimensions:zip_codeMeasures:noperonsQuery:3G
“the_district_geom”willbedisplayedonthemapandthe“supname”astherowvalue.“zip_code”willbedisplayedlikeasimpledimension.
Pass
2.8 Querywithonemeasureandtwopotentialspatialdimensions(oneofthemwillworkasanormaldimension).
SpatialDimension:zip_codeGeometryData:the_zip_code_geomGeometryPointRepresentation:the_zip_code_geom_pointDimensions:supnameMeasures:nopersonsQuery:3G
“the_zip_code_geom”willbedisplayedonthemapandthe“zip_code”astherowvalue.“supname”willbedisplayedlikeasimpledimension.
Pass
2.9 Simplequerywithonemeasureandnodimensions.
SourceId:2SpatialDimension:supnameMeasures:noincidentsQuery:2A
Worksfineforanothersourceaswell.
Pass
2.10 Simplequerywithonemeasureandonedimension.
SourceId:2SpatialDimension:supnameDimensions:dayofweekMeasures:noincidentsQuery:2C
Worksfineforanothersourceaswell.
Pass
BachelorthesisSS2011 Page29/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
CodequalityThe project was reviewed automatically by the Eclipse plugins Metrics(http://metrics.sourceforge.net/)andPMD(http://pmd.sourceforge.net/).
Codeanddependenciesanalysis
Figure12‐MetricsreportofSOLAPLayers2.0Extended
The red highlighted problems occur because of the deprecated classesCellSetToOLAPJsonBuilderandGeoBIFeature.TheclassCellSetToOLAPJsonBuilderisaclassfromSOLAPLayers1.0andremainsintheprojectjustfortestinganddevelopingreasons.AllthefunctionalityisgivenbythenewclassOLAPJson.NorefactoringfortheclassCellSetToOLAPJsonBuilderwasdone,thatiswhytheMcCabeCyclomaticComplexityistohigh.TheclassGeoBIFeatureisdeprecatedandwillberemovedinafurtherversion.Alltheinformationwhich this data class contains, is providedby the class CubeDTO aswell.Thisrefactoringismentionedinthechapter„FutherImprovements“.
BachelorthesisSS2011 Page30/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
6.3.5 FutureimprovementsSOLAPLayers2.0Extendedisnotafinalversion.Duringthedevelopmentprocesssomeideas ,to improve theapplication, cameup.Chores canbedonewithout analyzing thesolutiondeeply.Theextensionideashavetobeanalyzedbeforetheimplementation.
Chores• Addloggingtotheworkflow,toallowtomonitortheworkflowandsimplifythe
debugging.• Differentiatethevaluesnullandzerowhenretrievingdatafromadatasource
andcreatingtheoutput.Itisimportanttoknowifthevaluedoesnotexistsorifthevalueisequalszero.ThishastobemadeintheOLAPJson‐specification.
• Accomplishtheexceptionhandlingbypassingtheexception,respectivelyarelatedmessage,totheclient.AddasendException()‐methodtotheabstract„OutputBuilder“‐class.Thismethodwillbecalledwhenanexceptionarisesduringtheprocess.Soeach„OutputBuilder“cansendanappropriateexception.
• Addingallexceptiontextsintoaresource‐file.Thisallowsbetteradministrationofthetextsandmakesiseasytodisplayexception‐messagesindifferentlanguages.
Extensions
ExtendthedataabstractionTherearetwomainrequirementstothedataabstraction:
1. Theexchange‐format„RawData“doescontainredundancy.Alltheinformationtheoutputbuildersneed,arecontainedintheCubeDTO(arepresentationofanOLAP‐cube).Thelistof„MfFeature“isunessential.Ontheonehand,TheFeature‐listsimplyfiesthecreation‐processoftheoutputbuilder,ontheotherhandisitunnecessaryfortheconnectiontobuildthislist.Thislisthastoberemovedforacleanimplementation.
2. TheCubeDTOisnotcomplete.Itcontainsalltheessentialdatareadbytheconnection.ThemetadataandfilterAxisarenotyetapartoftheCubeDTOandshouldbeaddinanotheriteration(attention:Thesechangesaffectthe“OutputBuilder”‐and“Connection”‐classes.Theaffectedclasseshavetobeextendedaswell).
AddfeaturestoaddanOutputBuilderwithoutmodifyingcoreAt themoment every newOutputBuilder has to be registered bymodifying the core‐source code. This is a very poor implentation. This circumstances canbe removedbyaddingaconfiguration‐fileorusingreflection.
AddConnectionmanagerSOLAPLayersisstatelessatthemoment.Aspecificconnectionisopenedandclosedforeachrequest.Thisneedsalotofressources.Aconnectionmanager,whichoverviewstheused connection, could remove this circumstances and cache open connections forfurtherusebeforeclosingthem.
AddCacheforgeospatial‐dataReading geospatial‐data and relations to geospatial‐data need a lot of CPU power,respectively time. Different requests refer often to the same geometries. It would beuseful tocachethisdataontheserver‐orclient‐side insteadofrecalculating themforeachrequest.
BachelorthesisSS2011 Page31/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
DefineXML‐schemaforsources.xmlForbettervalidationandlessexceptionsitisrecommendedtocreateaXML‐schemaforthe sources.xml. It allows to recognize problems in advanced, before running theprogramm.
Addquery‐builderThere are different guidelines for a SOLAPLayers SQL‐query and the additionalparameters (See “Using SQL‐Queries” and “Analysis ‐ Result set to OLAP‐cubetransformation”).Aquery‐builder could simplify the constructionof a valid sql‐query.With an intelligentquery‐builder it is possible to validate thequery in relation to theparametersandprovidelistboxestosetthequery‐parameters.
StandardizeOLAPJsonOLAPJson is an undocumented data format, which contains an OLAP‐cuberepresentation. For better understanding and collaboration with other application, itwouldbeusefultodocumentanddescribetheOLAPJsonformat.InafurtherstepaXMLschemacouldbedefined.
6.3.6 DevelopersguideSOLAPLayers is a framework which can be used by developers. It is not yet enoughunderstandableandmaturetoprovide it topeoplewithnotechnicalbackground. It isrecommended to read this instructions for using the SOLAPLayers‐Frameworksuccessful.The instructionshelp tobuildnewconnection‐andoutput‐typesand teachhowtoworkwiththeSOLAPLayersserverpart.
InstallationThe installationwas tested on a LinuxUbuntu20.10 system.The computer has to beconnectedtotheInternettoruntheapplicationwiththetestdatabases.
ToolsFortosetupandrunSOLAPLayers2.0Extendedsomesoftwarehastobeinstalledontheworkingmachine.Installationanduserguidesfortheseapplicationscanbefoundontheirwebsites:
• EclipseHelios.ForfurtherusewiththeBIRT‐Pluginthe“EclipseIDEforJavaandReportDevelopers”isrecommended(http://www.eclipse.org/downloads/packages/eclipse‐ide‐java‐and‐report‐developers/heliossr2).
• Tomcat6.Theprogramistestedontheversion6,newertomcatversionsshouldworkaswell(http://tomcat.apache.org/tomcat‐6.0‐doc/index.html).
• Additionally:Subclipse(http://subclipse.tigris.org/)orotherSVN‐PluginforEclipse.IfthesourcecodewillbeimportedusingSVNanappropriateEclipse‐pluginisuseful.
Stepbystepmanual• StartEclipseandcreateoropenaworkspace• Getthesourcecode
o ImportSOLAPLayers2.0Extendedprojectcodeusingtheimport‐functionalityofEclipse“ExistingProjectsintoWorkspace”.Thezip‐fileislocatedontheprojectCD(“SOLAPLayers2.0Extended/solaplayers_2.0_extended.zip”).
• TorunSOLAPLayers2.0Extendedproperly,someadjustmentshavetobedone:
BachelorthesisSS2011 Page32/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
o Openthefile“build.properties”andcustomizethe“tomcat_webapps_path”.Thepathhastopointonthewebapps‐directoryofyourtomcatinstallation.
o Adjustthepathtothesources.xml‐fileinthefile“/webapp/WEB‐INF/web.xml”.Pointonthesources.xml‐file,whichislocatedintheworkspace“webapp/WEB‐INF/sources.xml”orcreateyourownsources.xmlatanydirectory(useanabsolutepath).
o Notice:Checkthesources‐configurationinthesources.xmlfile.ForusingthesampleOLAP‐source,the“catalog”‐parameteroftheconnectionstring‐elementhastobecustomized,sothatitpointsonthe“SOLAPLayers‐2.0/WEB‐INF/schemas/Canpop.xml“‐fileonthetomcat‐server.
o Copythepostgresql.jarintothe“lib”‐directoryofthetomcatinstallation.ThelibrarycanbefoundontheprojectCDwidespread(“SOLAPLayers2.0Extended/Utils/postgresql.jar”).
• Itisrecommendedtostoptomcatbeforedeployingtheapplication.• Tobuildtheprojectdraganddropthebuild.xml‐fileintothe“Ant”‐viewand
doubleclickon“deploy‐war”.Executefirst“clean”,ifSOLAPLayers2.0Extendedisalreadyinstalledontheserver.
• Starttomcat• AccessSOLAPLayersusingabrowser:http://localhost:8080/SOLAPLayers‐2.0/
(notallbrowsersdisplaySOLAPLayers2.0Extendedproperly.SOLAPLayersrunsrightonUbuntuusingGoogleChrome).
• LoginintoSOLAPLayers(Username:demo,password:Spatialytics)• Clickon“TestyourMDXqueriesandobservetheJSONoutputfromtheserver.”• SendacorrectSQLandMDX‐querytochecktheSOLAPLayersinstallation.
o Notice:Ifyouusethesamplequerieslocatedinthe“testqueries”‐directorycheckforwrongdisplayedspecialcharacters.Itcanhappenthattheoperatingsystemdisplaysthemwrong.
AddanewconnectiontypeEverythingthatprovidesdata,whichcanbetransformedinanOLAPCube,isapotentialdatasource.Beforestartingwithdevelopingitisusefultoanalyzethetransforming.As soon the idea is ready, a new class, which has to implement the “Connection”‐interface,canbecreated.Threepublicmethodshavetobecoded,toprovideaworkingconnection‐implementation:
Connect()‐CreatingaconnectiontothedatasourceDependingonthekindofdatasourceaconnectionmaybeinitializedbeforeitispossibletoreaddata.Ifnoconnectionisrequired,thismethodhastobeimplementedempty,orcanbeusedtoreaddataoutoftheconfiguration.
Execute(Parametersparameters)‐RetrievingandtransformingdataThis is the most important and complex part of the connection implementation.Depending on the complexity of this procedure, it is recommended to use additionalclasses tomakethesourcecodemoreunderstandable(E.g.Parser‐orquery‐wrapper‐class).The first thing is to use the already established connection to retrieve the data. Thequeryor/andtheparameterswhichleadtothedataarecontainedintherequest‐object,whichispassedtothismethod.
BachelorthesisSS2011 Page33/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Afterreadingthedata,theinformationhastobeparsedintoa“RawData”‐object.Atthemomenta“RawData”‐objectcontainsa“CubeDTO”,whichrepresentsanOLAP‐cubeandaListof“MfFeatures”,whichrepresentsthegeospatial‐datainasimpleway.ThecontentofRawDatamaychangewitheveryversionofSOLAPLayers.
Close()–CleanupandcloseconnectionThe last step of integrating a new connection type is to clean up and close theconnection. As mentioned above does not every connection type need a permanentconnection,sinceitispossibletoaccessthedatadirectly.In thenormal case, the connection to thedata sourcehas tobe closed in thismethod(E.g.database).
AddanewoutputformatIncomprehensiontotheconnectiontypethedeveloperhastodosomechangesinthecoreapplication(thismaybe improvedbyanextversion,see“AddfeaturestoaddanOutputBuilder withoutmodifying core“). The following steps show how to develop anewoutputformat.After creating a new class, which extends the abstract class “OutputBuilder” thedeveloperhastoimplementthreemethods:
Build(RawDatadata)–BuildtheoutputThismethodreceivesthetransformeddatafromaconnectionandhastoretransformitinto a specific output format. Themethod does not return the converted data to thecalleranddoesnotyetsendanyresponsetotheclient,soitcanalsobeusedbyotheroutputbuilders(e.g.Chartoutputbuilderdeliverscharttoapdfoutputbuilder).
getOutput()–AccessthegeneratedoutputThis method allows accessing the generated output. The method returns an object,whichcanbeparsedintheexceptedtype.
sendHttpResponse(HttpServletResponseresponse)–SendtheresponsetotheclientThismethodsets theHeaderof thehttp‐responseandmaydosome lastmodification,whicharespecificforthereceiver.Finally,itsendsthehttp‐response.Thisistheendoftheworkflow.
BachelorthesisSS2011 Page34/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
RegisternewoutputformatThe framework chooses the output‐format by a parameter of the HTTP‐request. Theconstructionofalloutput‐buildersisdoneinthe„OutputBuilderFactory“‐class.EditthegetInstance(String outputFormat)‐method to provide the new output‐format to theframework.
Figure13‐OuputBuilderFactoryclass,whichwillbereplacedinafutureversion
AddanewdatasourceBeforeaddinganewdatasource twothingshave tobeensured.Firstly,adatasourcehave to exist and the connection information has to be available. Secondly, anappropriateconnectiontypehastobeavailable.Now the developer can simply add a new source at the end of the sources‐files (orbetweentwosources).Everynewsourceneedsaidandhastoencloseaclass‐tag,whichcontainsthefullpackagepathtotheconnection,respectivelythedriver,class.Allotherrequiredparametersdependonthedriver.
BachelorthesisSS2011 Page35/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Figure14‐Sampleofsources.xmlwithdifferentdatasources
DBConnectionThe following parameters have to be set additionaly for a connection using the„DBConnection“‐driver:connectionstring Pathtothejdbc‐ressourceuser Nameofthedatabaseuserpasswort Passwortofthedatabaseuserdriver Driver package depending on the type of database (Example:
Postgres,Oracle,DB2,MySql,etc.)
SOLAPConnection“The following parameters have to be set additionaly for a connection using the„SOLAPConnection“‐driver:connectionstring PathtotheOLAP‐ressourceincludingnameandpasswordofDB‐
userdriver Driver package depending on the type of database (Example:
Postgres,Oracle,DB2,MySql,etc.)
UsingMDX‐queriesDriver:org.spatialytics.solaplayers.connection.driver.solap.SOLAPConnectionUsingMDX‐Queries is the simplest way to display BI‐ and geospatial data. Just threeparametershavetobesendtotheserver,allotherparameterswillbeignored:
• OutputFormat:FordisplayingthedataonaSOLAPLayers‐dashboardenter“olapjson”asoutputformat.Youcanuseanyotherexistingoutputformat,ifyounotwanttodisplaythedataonaSOLAPLayersdashboard.Analternativeto“olapjson”isthe“oldolapjson”‐output‐format.Itcreatestheexactsameoutputas“olapjson”,butusesadifferentwayofdatatransformationontheserverside,whichisdeprecated.Thispossibilityshouldbeusedjustfortestinganddevelopingissues.
• SourceId:Entertheidofanexistingsource.ThesourceidhastorefertoavalidSOLAPConnection‐source.Thelocationofthesourcefileisdefinedinthe“WEB‐INF/web.xml”‐filelocatedintheprojectdirectory.Fordefininganewsource,readchapter“Addanewdatasource”.
BachelorthesisSS2011 Page36/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• Query:Enteramdx‐queryforretrievingdatafromthechoosendatasource.Atthemomenttheonlylimitationisthatthegeographicdatahastobeselectedonrowsasasinglefield.
Example
Figure16‐ResultofsampleMDXquerydisplayedascrosstable
UsingSQL‐queriesDriver:org.spatialytics.solaplayers.connection.driver.db.DBConnectionAclassicSQL‐querycontainsnotasmuch informationasamdx‐query.SinceweuseaOLAPJson‐formatasthedatatransferformatbetweentheserver‐andtheclientside,wehavetoprovidesomeadditionalinformationtoconverttheresultsetofaSQL‐queryto
Figure15‐SampleMDXqueryforSOLAPLayers
BachelorthesisSS2011 Page37/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
an OLAPCube.We have to define the request in two steps for creating a sql‐request:Creatingaqueryanddefiningtheparameters.
HowtobuildavalidqueryTheusedsql‐querycanbeascomplexaswished(usingjoins,where‐conditions,unions,etc.).Theonlyconditionis,thatthefollowingconsiderationshavetobedone:
1. Whichgeographicinformationshouldberepresentedonthemap?Selectalabel(SpatialDimension),apolygon‐geometry(GeometryData)andapointrepresentationofthegeometry(GeometryPointRepresentation).GroupthequerybySpatialDimensionandbyeachindependentGeometry.Example:Countryname,geometryandpointrepresentationgeometryofcountry
2. Whichvalues(Measures)shouldbedisplayedinthecrosstable?Atleastonemeasurehastobelistedintheselectpart.Themeasurecolumnhastocontainnumericvalues.Useacorrespondingaggregationfunctiontogroupthemeasures.Example:GDP,Population,Averagesalary,etc.
3. Fromwhichperspectiveyouwanttolookatthemeasures(Dimension)?Everymeasurehastobelistedintheselectandthegroupby‐clause.Example:Sex,age,nationality,profession,etc.
4. Doyouwanttoreadallthedata?Usewhere‐conditionstorestraintheretrievingdata.Usuallythewhereclauseisusedondimensions,butitispossibletousethemaswellonmeasuresandthegeometry‐data,ifitisuseful.Example:Receivejustinformationaboutfemales,adults,Canadians,teachers,etc.
RequirementsoftheadditionalparametersThe following list shows the required and optional parameters, which are necessarybesidethequery.Thecarelessuseoftheparameterscanproducewrongreproductionoftheinformation:
• OutputFormat:FordisplayingthedataonaSOLAPLayers‐dashboardenter“olapjson”asoutputformat.Youcanuseanyotherexistingoutputformat,ifyoudonotwanttodisplaythedataonaSOLAPLayersdashboard.Donotusethe“oldolapjson”‐format.ThisisadeprecatedformatwhichcanonlybeusedtogetherwithaSOLAPdatasource.Possibledataformatsinthecurrentversionare“geojson”,“olapjson”,“oldolapjson”and“solapjson”.
• SourceId:Entertheidofanexistingsource.ThesourceidhastorefertoavalidDB‐source.Thelocationofthesourcefileisdefinedinthe“WEB‐INF/web.xml”‐filelocatedintheprojectdirectory.Fordefininganewsource,readchapter“Addanewdatasource”.
• SpatialDimension:Thisrequiredparameterhastomatchasingleselectedcolumninthesql‐query.Thereshouldbealogicalrelationbetweenthe“SpatialDimension”andthe“GeometryData”,sincethe“SpatialDimension”isnothingelsethananaliasforthegeometry.
• GeometryData:Thisrequiredparameterhastomatchasingleselectedcolumninthesql‐query.Itrepresentsapolygonoramulti‐polygononthemap.TheSRIDofthecolumnhastobe4326,sincetheSOLAPLayerclientpartdoesnotsupportanyotherSRIDatthemoment(PostgreSQL/PostGIS:theST_Transform(geometry,srid)‐methodcanprovidetransformationsfromothergeo‐datarepresentations).
• GeometryPointRepresentation:Thisisapointrepresentationofthe“GeometryData”,whichisusedtopositiondiagramsonthemap.Itisrequired
BachelorthesisSS2011 Page38/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
andthenamehastomatchacolumnofthesql‐query.Thesameconventionsasforthe“GeometryData”arenecessary(PostgreSQL/PostGIS:theST_PointOnSurface(geometry)‐methodcanbuildapointrepresentationoutofthe“GeometryData”,ifthedatabasetabledoesnotprovideapointrepresentation‐column).
• Dimensions:Thisparameterisrequiredifthereisadimension(besidethespatialdimension)inthesql‐query.Theparameterhastomatchthedimension/sinthesql‐query.Eachdimensionshastobelisted(separatedbycomma).Theorderofthedimensionsaffectsthepresentation.Thefirstdimensionnameisonthetopofthetableheadings.Donotsendanyparameters,ifnodimensionisprovidedbythesql‐query.
Figure17‐SamplecrosstableinSOLAPLayersdashboardwithdimensionscivilstatusandsex
Figure18‐SamplecrosstableinSOLAPLayersdashboardwithdimensionssexandcivilstatus
• Measures:Atleastonemeasurehastobeset.Ifthesql‐queryprovidesmorethanonemeasure,onecanbechosenortheycanbelist(separatedbyacomma).Thefirstmentionednamewillappearasthefirstcolumninthetablerepresentation.
Figure19‐SamplecrosstableinSOLAPLayersdashboardwiththemeasuresnumberofpersons,totalsavings
BachelorthesisSS2011 Page39/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Figure20‐SamplecrosstableinSOLAPLayersdashboardwiththemeasurestotalsavings,numberofpersons
Example
Figure21‐SampleSQLqueryforSOLAPLayers
Figure22‐ResultofsampleSQLdisplayedastable
BachelorthesisSS2011 Page40/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
CreatingadashboardAnexamplehowtocreateadashboardcanbefindinthepresentation“Dashboardandreporting tools: IntegrationofBI tools in thegeospatialdomain“byThierryBadardofSpatialytics.Thementionednotescanbefoundintheappendix.
Figure23‐SOLAPLayerssampledashboard
BachelorthesisSS2011 Page41/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
7 Geoextensions2Birt
7.1 Overview
Thispart of thedocumentationexplains the integrationof SOLAPLayers2.0ExtendedintoaBIRT‐plugin.ThepluginhasthenameGeoextensions2Birt.Pleasehavea lookatthedocumentationaboveforSOLAPLayersrelatedcontent.Besidethetechnicalcontent,auser‐anddeveloper‐guidecanbefoundattheendofthischapter.
7.1.1 Technicalreport
IntroductionOne of the main goal of Spatialytics is to integrate their products into well‐knownproducts.OneofacoupleofwidespreadreportingtoolsisBIRT.BIRTisaproductoftheAmerican company Actuate and a part of the Eclipse Foundation. For further steps aprototypeofamapreportitemforBIRThastobedeveloped.Decisionsandresultsaredocumentedinthefollowingchapters.
StateoftheArtBIRTitselfdoesnotcontainanymappingfeatures.TheonlysolutiontodisplaymapsinaBIRTreportistouseoneofthemap‐libraries,whichActuateprovides.
MapIt(GoogleMaps)andFlashMapsShowcaseMapItandFlashMapsShowcasearenotarealBIRTplugins.ThebothextensionsareBIRTreport‐libraries.Thelibrarycontainsacoupleofpredefinedtext‐items,whichcanbedragged into a report. The items contain JavaScript‐code,whichdisplays aGoogleMaportheFlashrepresentation.Thedifferentcode‐samplesallowcustomizingthemapandbindingdata.AGoogleMapAPI‐keyisnecessaryforMapIt.
Advantages• itisreallyflexibleandeasytocustomizeMapItandFlashMapsShowcase,since
theycontainjustpre‐scriptedJavaScript‐andsomeFlash‐files,.
Disadvantages• Cannotbeinstalledthateasyasaplugin• Doesnotallowtoconfigurethemapusingextendededitorfunctionality• Displayedasabigconstructoftableandtextfieldsinthereportdesigner.The
reportdesignerdoesnotrepresentthemapinanyway.• Hardtohandlefornon‐developers
ConclusionItiseasywaytoworkwithamapinBirttouseoneoftheseextensions.Theusabilityisverybadandthemap ishardtocustomize fornon‐developers.All inall thesepluginsaremoreaworkaroundtodisplaymaps,thanarealextensionforBIRT.
7.1.2 Projectdocumentation
RequirementsspecificationTherequirementspecificationisdonebySpatialyticsinanagileway.TheSCRUM‐Taskboardcanbefoundintheappendix(see“APPENDIXE–projectmanagement”).
BachelorthesisSS2011 Page42/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
AnalysisThere are different possible solutions to receive the data from a data source. Thesedifferent solutions were analysed and rated to find the best possible one for thisparticularplugin.Thedecisionrelayedondifferentaspects:
• UsabilityThegoalistomakeiteasyfortheusertodefineanewdatasource.Lessuserdefinedinformationaspossibleshouldberequired.Weight:1*:Sincethisversionofthepluginisaprototypeandwillbeusedfordemonstrationpurpose,featureswillbemoreimportantthanagoodusability.
• NumberofconnectionsReportingandperformancehaveastrongrelation.Forabigamountofdata,thereport‐generationcanusehourstofinish.Thatiswhyaslesspossibleconnectionsaspossibleshouldbecreatedtoreceivethedata.Inthebestcaseoneconnectionperdatasourceisnecessary.Weight:2*:Multipleconnectionshaveaninfluenceontheusability,theperformanceandthecomplexity.
• DatatransformationForperformancereasonsthereceiveddatashouldbedisplayedwithouttotransformthedataintoanotherrepresentation.ThiscancostalotofCPUandtime,ifthedataamountislargeWeight:1*:Datatransformationscostperformance,butarenecessaryforalmosteverydatasource,tofitthecommondatainterface(ODA).
• ReusabilityBIRTisjustoneplatformSpatialyticsdealswith.Inthefuturepluginsforotherproductsshouldbedeployedaswell.Asmuchaspossibleoftheimplementedcodeshouldbereusableforfurtherprojects.Weight:3*:Spatialyticswillprovidethiskindofpluginsforfurtherreportingtools.Itsavesalotoftimeandeffort,iftheworkcanbereusedforotherprojects.
• EffortandtimeTimeismoney.Forthisreasonsthepluginshouldbeimplementedinatimesavingway.Weight:2*:Thegoalistocreateaprototypeasfastaspossible,thatiswhy“effortandtime”isanimportantaspect.
• ExtensibilityThegoalistocreateaplugin,whichcanbeextendedtofitallpossibleupcomingrequirements.Ifitisnotpossibletoextendtheplugin,alotofadditionalworkhastobedone.Weight:3*:Aprototypeisademo‐object.Inthefuturealotofrequirementswillcomeup.Itshouldbepossibletoimplementthenewfeaturesintotheprototype.
Some of the following solution ideas use SOLAPLayers as a component of the BIRTplugin. SOLAPLayers provides the possibility to query different data sources and anhtml‐basedinteractivemapcomponent.Twoimportantguidelineshavetobeconsidered:
• Alltheguidelines,whichweregivenforSOLAPLayers2.0Extended,arevalidfortheBIRT‐pluginaswell.Havealookatthe“Guidelines”‐chapterinthe“Technicalreport”‐chapteroftheSOLAPLayers2.0Extendeddocumentation.
BachelorthesisSS2011 Page43/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• TheBIRTdatasourceconnectionsarebasedontheODAinterface.AllthedataBIRTreceivesisinatwo‐dimensionalformat.Three‐ormultidimensionalformatsarenotsupported.
Variant1‐SOLAPLayerswithJSON‐outputandtwoconnections
Figure24–Illustration:SOLAPLayerswithJSON‐outputandtwoconnections
DescriptionTheBIRTpluginworkswithtwodifferentconnections.Thefirstconnectionreadsdatafromthegivensourcetodisplayitinatableoranotherreportitem.Thiscouldbedonein an easywaywith theBIRT core features, no extensionwould be necessary. For todisplaythemapitemasecondconnectionisnecessarytoreceivethemap‐information.
Dis‐/AdvantagesAdvantages Disadvantages+ProfitfromSOLAPLayersfeatures
‐ Twoconnections(anddatasourcedefinitions)necessary
‐ Mapdatasourceisnotcompatiblewithcorereportitems
‐ HackforODAdatasourcenecessary(resultisnotatwo‐dimensionalresultset)
Report Items
Data Sources
Report
Data Source
SOLAP
Layers
Server
BIRT
Map Item
BachelorthesisSS2011 Page44/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Variant2‐SOLAPLayerswithImage‐outputandtwoconnections
Figure25–Illustration:SOLAPLayerswithImage‐outputandtwoconnections
DescriptionAs invariantone, thedata for thenormaldatasourcecanbereceivedusing theBIRTcore functionality. For todisplay themap, SOLAPLayers to receivedata and create animage‐map(ForavariantwithoutSOLAPLayersseevariant4).
Dis‐/AdvantagesAdvantages Disadvantages+ProfitfromSOLAPLayersfeatures
‐ Twoconnections(anddatasourcedefinitions)necessary
‐ Imageisnotinteractive‐ HackforODAdatasource
necessary(resultisnotatwo‐dimensionalresultset)
Report Items
Data Sources
Report
Data
Source
SOLAP
Layers
Server
BIRT
Map Item
BachelorthesisSS2011 Page45/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Variant3‐SOLAPLayersandoneconnection
Figure26–Illustration:SOLAPLayersandoneconnection
DescriptionThis solution just creates a connection to SOLAPLayers. No more connections arenecessary. The received data contains the data as two‐dimensional result set and anOLAPJson for to interpret themap representation. To display the received data fromSOLAPLayersthedatahastobetransformedintoatwo‐dimensionaldataset.
Dis‐/AdvantagesAdvantages Disadvantages+ProfitfromSOLAPLayersfeatures+Justoneconnection+Highflexibility
‐ HackforODAdatasourcenecessary(resultisnotatwo‐dimensionalresultset)
Report Items
Data Sources
Report
Data
Source
SOLAP
Layers
Server
BIRT
Map Item
BachelorthesisSS2011 Page46/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Variant4–OneconnectionwithoutSOLAPLayers
Figure27–OneconnectionwithoutSOLAPLayers
DescriptionThis variant doesnot use SOLAPLayers, instead thedata sources are connectedusingtheBIRTdrivers.Theproblemwillbetostorethedataforthemaprepresentationintoatwo‐dimensionalresultset.Mapscandisplaycomplexrelations,whicharehardtostorein a two‐dimensional data set. The big advantage would be, that the ODA‐definitioncouldbeusedregularly.
Dis‐/AdvantagesAdvantages Disadvantages+Justoneconnection+ProperuseoftheODA‐interface+AllBIRTdriverscanbeused
‐ Badreusability
Report Items
Data Sources
Report Data Source
BIRT
Map Item
BachelorthesisSS2011 Page47/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ConclusionThefirstnumbershowsthenumberofpoints,thesecondthefinalimportanceincludingtheweightoftheaspect. Variant1 Variant2 Variant3 Variant4Usability(1*) 1/1 1/1 2/2 3/3Numberofconnections(2*) 1/2 1/2 2/4 2/4Datatransformation(1*) 2/2 2/2 1/1 3/3Reusability(3*) 2/6 2/6 3/9 1/3Effortandtime(2*) 3/6 2/4 3/6 1/2Extensibility(3*) 3/9 3/9 2/6 1/3Total 26 24 28 20Thevariant3willbe implemented.Theideasofthevariant1and4willbedismissed.The variant 3 allows to implement an image‐output asmentioned in variant 2 in thefuture.
ArchitectureThe architecture of the BIRT‐plugin is quite simple. The project used differentextension‐pointstointegratethepluginintheEclipseenvironment.
Figure28‐ExtensionpointsusedbytheBIRT‐plugin
SomeoftheextensionspointsaredirectlyprovidedbyEclipse.Thesearetheextensionpoints, which realize the ODA‐interface and a common property page. All extensionpoints, which refer directly to BIRT specific features, are provided by BIRT. Thefollowinghookswereusedtointegratethemap‐plugin:
• …birt.report.designer.ui.reportitemUIo ThisimplementationextendsthereportitemintheBIRTreportdesigner.
TheconnectedclassimplementstheIReportItemImageProvider‐interface,todisplaythemapinthedesignerasanSWT‐Image.
• …birt.report.model.reportItemModelo Thishookdefinesthemodelbehindtherepresentationofthemapitem.
Themodeldefinesthereportitemspecificproperties.• …birt.report.engine.reportitemPresentation
o Theclass,whichislinkedtothisextensionpoint,isresponsibleforthepresentationofthemapiteminarenderedreport.AtthemomentjustHTMLisgenerated,thiscouldbeextendedinfutureversions.
• …birt.report.designer.ui.elementAdapterso Thisextensionpointconnectstothemapitemasanextendedreportitem
• .…datatools.connectivity.oda.dataSource
BachelorthesisSS2011 Page48/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
o This,fromtheEclipsecoreprovidedextensionpointisresponsibleforthedatasourcedriver.ThelinkeddriverclassimplementstheIDriver‐interface.
• …datatools.connectivity.oda.design.ui.dataSourceo Theimplementationofthisextensionpointdefinesthewizardsforthe
definitionofaspecificdatasourceanddataset.• …ui.propertyPages
o Theimplementationofthisextensionpointprovidesadefaultdatasourcepropertypage.
Implementation
Systemtest
GoalThistestplanassuresthecorrectbehaviouroftheSOLAPLayers‐BIRT‐plugin.TheBIRT‐pluginworkswithSOLAPLayers2.0Extended,whichisdescribedinthefirstChapterofthis document. The use of SOLAPLayers in the Eclipse‐ and BIRT‐enviroment will betested. Some presentation‐issues are based on the SOLAPLayersmap, whichwas nottouched during the bachelor thesis. Please notice the known problems (see chapter“Futureimprovements–unsolvedproblems”).ThistestplandoesnottestanyBIRTcorefeatures,justtheinteratctionofthepluginwithBIRT.
RequirementsandtestdataThesametestdataasinthetestofSOLAPLayers2.0Extendedwasusedforthistest.Forfurtherinformationlookatthe“Systemtest”ofSOLAPLayers2.0Extended.
BachelorthesisSS2011 Page49/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
TestplanThistestplanwasaccomplishedon12May2011.ThesoftwarewastestedonaLinuxUbuntu20.10withEclipseIDEforJavaandReportDesignersintheversionHeliosRelease2.
DefaultAtoBScenarioThefollowingtestplancontainsadefaultAtoBscenario.Ateachstepthepreconditionistheerror‐freecompletitionofthepreviousstep.Fortheinitialpositionofthetestanew“ReportProject”andinsideofthisprojectanewreportcalled“test.rptdesign”hastobecreated.Usethe“ReportDesign”perspectiveinEclipsetogothroughthetestplan.Number Testcase Expectation Result1.1 Rightclickonthe“DataSources”iconinthe“Data
Explorer”view.Choose“NewDataSource”inthecontextmenu
“SOLAPLayersDataSource”isdisplayedasoneofthedatasourcesinthelist.
Pass
1.2 Selectthe“SOLAPLLayersDataSource,enterthename“SOLAPLayersDataSource”andclickonthe“Next”button.
Thepropertywindowisdisplayed. Pass
1.3 Enterthe3Asql‐stringandtherelatedparameters(see“SOLAPLayers2.0Extendedsystemtest”forfurtherinformation”).Clickonthe“Finish”button.
“SOLAPLayersDataSource”isdisplayedonthe“DataSources”‐listintthe“DataExplorer”.
Pass
1.4 Rightclickonthe“DataSets”iconinthe“DataExplorer”.Choose“NewDataSet”inthecontextmenu.
Theentered“DataSource”isdisplayedinthe“SOLAPLayersDataSource”list.
Pass
1.5 Choosethejustentereddatasource,enterthename“SOLAPLayersDataSet”andpressthe“Next”‐button.
The“GeneratedColumns”viewisdisplayedandthethreecolumns“OLAPJSON”,“supname”and“nopersons”arelisted.
Pass
1.6 Clickonthe“Finish”‐button. The“OutputColumns”viewisdisplayed.Thecolumnsarelisted.
Pass
1.7 Clickonthe“PreviewResults”link. Thethreecolumnsaredisplayed.ThefirstcolumncontainsjusttheOLAPJSONinthefirstcell.Allothercellsofthefirstcolumnareempty.Theothercolumnscontaintherightdata.Totalelevenrowsaredisplayed.
Pass
BachelorthesisSS2011 Page50/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
1.8 Clickonthe“OK”button Thewindowclosesandanewdatasetcalled“SOLAPLayersDataSet”appearsinthe“DataSets”list.
Pass
1.9 Dragthe“SOLAPLayersDataSet”intothereport.Deletethecolumn“OLAPJSON”.
Atableisdisplayedinthereportdesigner. Pass
1.10 Changetothe“Palette”viewanddragamap‐itemintothereport.Resizethemaptothesizeofthetable.
Themapappearsintherightsizeonthereport. Pass
1.11 Selectthemapitemandchangethetitleinthe“PropertyEditor–Map‐Properties”viewto“SOLAPLayersMap”andthezoomto“1”.Changetothe“Binding”tabandchoosethe“SOLAPLayersDataSet”fromthedrop‐down.Savethereport.Clickonthe“ViewReport”itemanchoose“ViewReportasHTML”.
Thereportisdisplayedinanewwindow.Thetableshowsalltheelevenrowsandthemapisdisplayedintherightsize.Theareasonthemaparehighlightedindifferentgreens.Theyshowthehighnumbersinlightcolors.
Pass
AlternativescenariosNumber Testcase Expectation Result2.1 SamescenariousingaMDX‐query Thegeneratedcolumn‐namesarecomposedoutofthe
differentmeasures.Pass
2.2 SamescenariousingaSQL‐querywithdimensions Thegeneratedcolumn‐namesarecomposedoutofthedifferentmeasures.
Pass
BachelorthesisSS2011 Page51/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
CodequalityThe project was reviewed automatically by the Eclipse plugins Metrics(http://metrics.sourceforge.net/)andPMD(http://pmd.sourceforge.net/).
Figure29‐MetricsreportofthefinalGeoextensions2Birt‐plugin
The numbers calculated byMetrics are all in a suitable range. Some code smells aregiven through the implemented interfaces. Some of the used interfaces have hugeamountofmethods,whichhavetobeintegrated.
FutureImprovements
Unsolvedproblems
PresentationjustasbasicHTMLForunknownreasonsthemapisdisplayedjustinthebasicHTMLview(inBIRTEclipseplugin).Theproblemishardtoevaluate,becausethereisnowspecificdocumentationofthe way html gets interpreted in the other HTML‐based views (preview and webviewer). The problem relays probably on the compatibility of the JavaScript‐librarieswiththeviewers,orisevokedbythepagination‐algorithm.
LinktoprojectinternalJavaScriptresourcesDifferentJavaScript‐librariesarenecessarytodisplaythemapcomponentintheHTML‐view.Thegoalistoaccesstheselibrariesintheprojectdirectoryusingrelativepaths.Itcould not be figured out, how the BIRT‐browser addresses JavaScript‐links. For thisreason,allJavaScripthavetobeaccessiblethroughanURL‐address.Atthemomentthelibrariesarelocatedin“public_html”,whichshouldbelocatedinthewepapps‐directoryofthetomcatserver(oronanothertypeofserver).
MultiplemapsforonereportSOLAPLayersclientsidedoesnotsupportmultipleviewsinthecurrentversion.Thatiswhy the BIRT‐plugin is not able to display more then one map as well. As soonSOLAPLayerhasthisfeature,theBIRT‐pluginhastoadapttoSOLAPLayers.
BachelorthesisSS2011 Page52/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Chores
DifferentviewsforMDX‐andSQL‐queryCreate a different view for entering MDX and SQL‐queries. This would improve theusability,sinceMDXdoesnotneedthesameamountofparameters.
Separatedatasource‐fromdataset‐parametersAtthemoment,allparametersforaccessingadatasourceareenteredinthedatasourcepropertywindow.Inthefuturejusttheconnectioninformationappearsandthequeryaswellastheparameterswillbeenteredinthepropertiesofthedataset.
ErrorHandlingSOLAPLayers has a complete error handling implemented. The BIRT‐plugin shouldadaptthisandgenerateusefulerrormessage,whichcanbedisplayedinthefront‐end.Thisisnecessaryespeciallyforvalidationandconnectionreasons.
ReuseSOLAPLayerslibrariesSOLAPLayers is integrated in theBIRT‐plugin as a library. This library contains otherjars. TheBIRT‐pluginneeds to access these libraries. There is nopossibility to accessthesesub‐libraries inEclipseat themoment.Thisversionof theplugincontainssomelibraries twice. They increase the file‐size and could lead to problems with thecompatibility(whenusingtwodifferentlibraryversions).
LocalizationAtthemomentjustanEnglishbundleexistsandthelocalizationisignored.Thisshouldbefixedtoprovidethepluginindifferentlanguages.
AlignthemapinthecenterThe map is displayed without to align the important data in the middle of the maprepresentation.Theuserhastofindthehighlightedpolygons,lineorpointsbythemself.Thiscouldbesolvedprogrammatically.
Extensions
SupportotheroutputformatsSOLAPLayers 2.0 Extended supports no other output than the JSON‐formats. As soonSOLAPLayers support other formats other report‐formats can be provided as well.Especiallytheoutputofamapasanimageisimportant,thiswouldallowtocreatepdf,Excel,PowerPointandotherusefulreport‐formats.
UserGuide
InstallationThe installationwas tested on a LinuxUbuntu20.10 system.The computer has to beconnectedtotheInternettousethetestdatabases.
• Copythe“Geoextensions2Birt/datasource_1.0.0.jar”intothe“plugins”directoryofyourEclipseinstallation.
• Copythe“Geoextensions2Birt/Utils/public_html”‐directoryintothewebappsdirectoryofyourtomcatandruntomcat.The“public_html”‐directoryhastobeaccessibleusingtheURLhttp://127.0.0.1/public_html/(Forfurtherinformationsee“Unsolvedproblems‐LinktoprojectinternalJavaScriptresources)
• Copythesources.xmltotherootdirectory(“/”)ofthecomputer.
BachelorthesisSS2011 Page53/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• StartEclipseusingtheconsoleandtheargument“‐clean”tomakesurethatthepluginwillbeloaded.
Tutorial–Addingamapitemtoareport1. AfterinstallingtheBIRTplugin,openBIRTandchangetothe“ReportDesign”
perspective.Use“File–New–Project…”tocreateanew“ReportProject”.Enteranameandgothroughtheprojectwizard.
Figure30‐Printscreen:CreateaReportProject
2. Executearight‐clickonthenewprojectinthe“Navigator”‐viewandclickon“New–Report”.Enteranameandfinishthewizard.
3. Thenextstepistodefinethedatasource.Openthenewreportandright‐clickonthe“DataSources”‐iconinthe“DataExplorer”.Choose“NewDataSource”.AWindowpopsup.
BachelorthesisSS2011 Page54/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
4. Chosethe“SOLAPLayersDataSource”andclickonthe“Next”‐button.
Figure31‐Printscreen:Selectadatasourcetype
5. EntertheSQLorMDXqueryandthecorrectparameters.Forfurtherinformationabouttheparametersandtheformatofthequeryhavealookatthe“SOLAPLayers2.0Extended–Developersguide”–chapter.Clickon“OK”.Youcanseenowthenewdatasourceinthe“DataExplorer”.
Figure32‐Printscreen:Filloutthedatasourceparameterform
BachelorthesisSS2011 Page55/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
6. Performaright‐clickonthe“DataSets”‐icon,whichisdisplayedaswellinthe“DataExplorer”.Choose“NewDataSet”.Apop‐upwindowappears.
7. ChoosetheSOLAPLayersdatasourceandenteranameforthedataset.Clickon“Next”tocontinuethewizard.
Figure33‐Printscreen:Choosethedatasource
BachelorthesisSS2011 Page56/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
8. Forthetwo‐dimensionalrepresentationofMDX‐resultsandSQL‐results,withdimensionsasparameters,thecolumnsarenotobviousinthequery.Thatiswhyyoucanseeonthispageofthewizardthegeneratedcolumn‐names.Clickonthe“Finish”button.
Figure34‐Printscreen:Generatedcolumnnames
Figure35‐Printscreen:Previewcolumnnames
9. Changeintheappearingwindowtothepage“PreviewResults”tohaveanoverviewofthedata.Ignorethefirstcoloumn,itwillbehideinafurtherversion
BachelorthesisSS2011 Page57/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ofGeoextensions2Birt.Clickon“OK”tocompletethedatasetcreation.
Figure36‐Printscreen:Previewresultset
10. Youcandraganddropthedatasetonthereport.Pleasedeletethefirstcolumn“OLAPJSON”.
11. Changetothe“Palette”viewtodraganddropthemapreportitemonthereport.
Figure37‐Printscreen:Draganddropmapreportitem
BachelorthesisSS2011 Page58/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
12. Youcannowresizethemapitemandenterthenameandthezoomlevelusingthe“PropertyEditor–Map”‐view.
Figure38‐Printscreen:Definepropertiesformapreportitem
13. Toviewthefinalreportclickonthe“ViewReport”‐iconandchoose“ViewReportasHTML”.Noticethatthepreviewandotherformatsarenotworkinginthisversion.
Figure39‐Printscreen:ReportasHTML‐output
BachelorthesisSS2011 Page59/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Developersguide
ExportPlugin• Right‐clickontheproject• Clickon“Export…”• Choose“Plug‐inDevelopment–Deployableplug‐insandfragments”• Defineadirectoryandclickon“Finish”
PluginInstallationThe installationwas tested on a LinuxUbuntu20.10 system.The computer has to beconnectedtotheInternettousethetestdatabases.
• ImportGeoextensions2birt‐projectcodeusingtheimport‐functionalityofEclipse“ExistingProjectsintoWorkspace”.Thezip‐fileislocatedontheprojectCD(“Geoextensions2Birt/geoextensions2birt.zip”).
• Copythe“Geoextensions2Birt/Utils/public_html”‐directoryintothewebappsdirectoryofyourtomcatandruntomcat.The“public_html”‐directoryhastobeaccessibleusingtheURLhttp://127.0.0.1/public_html/(Forfurtherinformationsee“Unsolvedproblems‐LinktoprojectinternalJavaScriptresources)
• Specifythepathtothesource.xml(SOURCES_XML)inthe“util/messages.properties”‐file.
• Runprojectas“EclipseApplication”.AnewWorkspaceopensandthepluginwillbeloaded.
BachelorthesisSS2011 Page60/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
8 Projectmanagement
The process of the two subprojects is documented in the appendix. (“APPENDIX E –Project management”) For more information about the chosen project managementmethod,havealookat“APPENDIXI–CustomizedSCRUM‐processforbachelorthesis”.“
9 Projectreview
9.1 Acknowledgment
Iwould like to thank to Prof. StefanKeller for the big support before and during thebachelor thesis. This project could not proceed so unproblematic without the goodassistanceofhim.AbigthanksgoestoSpatialytics,namelyDr.ThierryBadard,LucVaillancourtandJeanMathieu,fortheirefforttointegratemeintotherecompany,thegoodteamworkandtheassistance.IwishyouandSpatialyticsjustthebestforthefuturechallenges.
9.2 Fieldreport
ImAnhangdiesesDokumentesbefindetsichderZwischenberichtderinderHalbzeitdesProjektserfasstwurde(siehe„APPENDIXJ–Interimreport“).HiernochmalseinkurzerRückblicküberdasgesamtProjekt:Die Bachelorarbeit, die ich bei dem Unternehmen Spatialytics in Québec, Canadaerarbeitet habe, brachte vieleHerausforderungenmit sich.MitmeinemVorhaben, dieBA im Ausland zu schreiben, bin ich diese Challenge bewusst eingegangen. Auch imNachhineinbinichsehrglücklich,dassichmichdieserAufgabegestellthabe.IchkonntewährenddiesenzweieinhalbMonatenvonvielenverschiedenenAspektenprofitieren.Die Arbeit bei Spatialytics war sehr angenehm. Das eingespielte Team hat mich sehrfreundlichempfangenundmichschnell indieFirmaeingegliedert. Spatialytics isteinsehrmotiviertesStart‐UpUnternehmen,dasmitvielElanundMotivationdieAufgabenangeht. Der Fakt das Spatialytics erst in denKinderschuhen steht, beeinflusstemeineArbeitstark.EswarleidernichtmöglicheintotaleigenständigesProjektdurchzuführen,wasimSinneeinerBachelorarbeitsehrzuschätzenwäre.ZuvieletechnischeaberauchorganisatorischeAbhängigkeitenwarendurchdieUmständevorhanden.Eines dieser Problemewar, dass sich die Applikation SOLAPLayers zu Beginn der BAnicht auf einemQualitätsniveauwar,welchesdieHSR füreineBAakzeptierenwürde.Dieser Fakt brachtemich bereits früh in einen gewissenZwiespalt, da ichmitmeinerArbeiteinerseitsSpatialyticsundandererseitsdie fürdieBewertungverantwortlichenPersonenzufriedenstellenwollte.DieszuvollbringenerfordertevielMehraufwandundeineguteKoordination.DaderzeitlicheRahmenesnichtzugelassenhat,dassichmichzu fest in die bereits vorhandenen Umstände einmischte, musste ich mirEinschränkungenmachen. So akzeptierte ich gewisse Entscheidungen, ohne den SinnundZweckzuhinterfragen.
BachelorthesisSS2011 Page61/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
DieorganisatorischeHerausforderunghattezurFolge,dassdertechnischeAspektetwaszukurzkam.LeiderkonnteichmichmitmeinerBAnichtwiegewünschtprofilierenundeine technische Arbeit mit dem gewünschten Anspruch entwickeln. Trotzdem, daserzielteResultatistinmeinenAugentechnischsauberumgesetztundverständlichundausführlich dokumentiert. Das Hauptziel war die Zufriedenstellung meiner Betreuer.SpatialyticshatmehrmalsihreDankbarkeitausgesprochenundmichsomitbestätigt.Ichfreuemich,dasssievonmeinerArbeitprofitierenkonnten.Die Zeit in Québecwar definitiv eine prägende Zeit. Nicht nur durch die Arbeit, auchdurchdasLebennebenderBAkonnte ichvielprofitieren.Sokonnte ichzumBeispielmeine Englisch‐Kenntnisse verbessern und meinen kulturellen Horizont erweitern.Nebenbei hatte ich eine super Zeit, mit vielen netten Begegnungen und einem superUmfeld. Die etwas schwierigeren Umstände waren eine super Vorbereitung auf daskommendeBerufsleben,woichauchaufähnlicheSituationentreffenwerde.HerzlichenDankanalle,diemichvorundwährendmeinerZeit inQuébecunterstützthaben!
9.3 Lessonslearned
Die errungenen Erkenntnisse aus dieser Arbeit sind in den wöchentlichen Sprint‐Berichten dokumentiert. Diese können im Anhang E oder auf der Projekt CD imDokument “project plan.xls” eingesehen werden. Weitere Informationen zum VerlaufderArbeit könnenauchdemSchlussbericht („field report“) unddemZwischenbericht(„interimreport“)entnommenwerden.
10 Furtherdocumentation
Additionaly to this document a JavaDoc documentation for each project exists. TheJavaDoc documents all classes briefly and important public methods more detailed.Overwritten methods are documented in the implemented interface or the extendedsuperclass. Simple getter and setter methods as well as private‐methods are notdocumented.
BachelorthesisSS2011 Page62/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
11 Appendices
APPENDIXA‐ContentoftheCD
• BachelorThesiso Containsthefinalprojectdocumentation,theprojectplan,all
presentations,whichwereheldduringtheinternshipandtheJavaDoc‐documentationforbothsub‐projects.Theprojectpostercanbefoundinthisdirectorytoo.
• Geoextensions2Birto ContainsthesourcecodeoftheBIRT‐plugin,thedeployedpluginand
someutilities,whicharenecessarytoinstallanduseGeoextensions2Birt.• SOLAPLayers2.0Extended
o ContainsthesourcecodeofSOLAPLayers2.0ExtendedasEclipse‐projectandasdeployedjavalibrary,andutilities,whicharenecessarytorunSOLAPLayers2.0Extended.
• Utilitieso Containsfiles,whichareusefulforbothsub‐projects,besideallthe
documentation.
APPENDIXB‐Glossary
This is a glossary about used terms and definitions. To understand the topic it isrequiredtoengagewiththetopicsofBIsandspatialinformationandsystems.Term ExplanationBI‐Tools BusinessIntelligence‐Systemsaretools,whichallowsanalysing
businessdataandcreatingreports.SAPisthemostcommoncommercialtool.Therearealotothercompetingcommercial‐andopensourcesoftwareproductslikeOracleBI,PentahoBISuite,Baanetc.
BIRT BIRTisanEclipse‐basedreportingtool.EasytointegrateintootherJava/JavaEEapplicationtocreatecompellingreports.Seehttp://www.eclipse.org/birt/phoenix/.
BIRTROM BIRTReportObjectModelisusedtocreateandsavereports.TheBIRTObjectModelisdescribedbyanxml‐file.Seehttp://www.eclipse.org/birt/phoenix/ref/rom/index.html.
DataMining DataMiningdescribestheprocessoffindingpatternindatasets.AsimilarnameforDataMingis„KnowledgediscoveryinDatabases“.
Data‐Warehouse Containsdatafromdifferentsources.TheprovideddatagetpushedintothewarehousethroughETL.Thecreateddata‐warehouseisusedfordataanalysisandtomakebusinessdecisions.Inadatawarehousedatagetsneverdeletedandalotofinformationisredundant.Theredundancyincreasestheperformance.
DBS DataBaseSystemssavedataefficiently,consistentanddurableandprovideittouserandsystems.
EclipseRCP EclipseRichClientPlatformisaplatformforcreatinganddeployingdesktoprichclientsoftware.EclipseisbasedonanOSGi‐framework
BachelorthesisSS2011 Page63/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
calledEquinox.Seehttp://wiki.eclipse.org/index.php/Rich_Client_Platform
ETL ETListheabbreviationforExtract,Transform,andLoad:• Extractdataoutofdifferentdatasources.• Transformdatatofitinthetargetdatabase.• Loadthetransformeddataintothedatabase.
GEF TheGraphicalEditingFrameworkisahelpfulframeworktoextendtheEclipseWorkbench.ItaffordstoextendEclipsewithSWT‐basedtreeandDraw2d‐basedgraphicaleditors.Seehttp://www.eclipse.org/gef/.
GeoKettle AKettleextensionofSpatialyticstoextendtheETL‐productforspatialdata.Seehttp://www.spatialytics.org/projects/geokettle/.
GeoMondrian GeoMondrianisaprojectofSpatialyticstoextendthestandardMondrianproduct.ThisSOLAP‐ServerprovidesintegrationofspatialobjectsintotheOLAPdatacubestructure.SOLAPdoesnotneedanadditionaldatabaseforthegeographicdata.Seehttp://www.spatialytics.org/projects/geomondrian/.
GIS GeographicInformationSystemscapture,store,manage,analyzeandpresentdata,whichislinkedtolocation/s.Thecomplexnatureofthespatialdatarequiressuchinformationsystems.
JasperBISuite AnOpensourceOSBItool.FullydevelopedinJava.Since2007completesJasperETLtheOSBI‐solution.Seehttp://www.jaspersoft.com.
Jedox TheworldwideleadingProviderofOSBI‐systems.JedoxislocatedinFreiburg,Germany.Awell‐knownproductisthePalo‐suite.Seehttp://www.jedox.com/de/home/uebersicht.html.
Kettle KettleIsthePentahoDataIntegrationtool(ETL).Seehttp://kettle.pentaho.com/.
MDX MultidimensionalExpressionsisaquerylanguageforOLAPdatabases.MDXallowsqueryingandmodifyingmultidimensionaldatastoredinOLAPcubes.
Mondrian Astatisticaldata‐visualizationsystemofPentaho.Itallowsthevisualizationofalmostanykindofdata.Itstrengthistoworkwithlargedata.MondrianisanOLAP‐Server.Seehttp://mondrian.pentaho.com/.
ODA ODAistheshortcutforOpenDataAccess.ODAdescribesastandardforadatarepresentationasatwo‐dimensionalresultset.ThisstandardismainlyusedbyBIRTandotherEclipseprojects.
OLAP OLAPistheabbreviationforOnlineAnalyticalProcessing.OLAPisamethodtosendcomplex,multidimensionalanalyticallyqueriestothedatabase(MDX).Theanswerofthequeryistypicallysuppliedasamatrix.
OLAPcube OLABcubeisamethodtopresentlogicaldata.Thedataarearrangedthree‐ormulti‐dimensional.Thedatacanbechosenthroughoneormoreaxes.
OLAP4j OLAP4JisaJDBCacronymforaccessingdifferentOLAPserver.ItallowschangingtheusedOLAPserverrapidlyandwithoutchanging
BachelorthesisSS2011 Page64/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
anysourcecode.Itisanopensourceproduct.Seehttp://www.olap4j.org/.
OLTP Online‐Transaction‐Processinglabelsaparadigmfordatabasesystems.Thesystemsusesdatabasetransactionsandresponsesimmediatelytouserrequests.
OSBI OpenSourceBusinessIntelligenceproductsaretheopponentstothemorecommoncommercialproductslikeSAP,OracleBI,Baanetc.Well‐knownOSBIstoolsarebyTalend,SpagoBI,Palo,JasperBISuiteandPentahoBISuite.
PaloOLAPServer PaloisamultidimensionalrealtimeOLAP‐server(MOLPA).Itisusedtomanageeconomicallyandstatisticallydatasets.Itcanbeintegratedthroughdifferentsoftwareenvironments(Java,PGP;C,.NETetc.).TheJedoxcompanydevelopsPaloOLAPServer.Seehttp://www.jedox.com/de/produkte/palo‐suite/palo‐olap‐server.html.
PentahoBISuite Ajava‐basedOSBItoolofthePentahoCompany.PentahohasETL,reporting,OLAP/analysisanddata‐Miningfunctionality.Developedsince2004.Seehttp://www.pentaho.com.
Snowflake AschemausedforOLAPanddatawarehouses.Itnormalizesthedimensions‐tableofthestartschemaandprovidesinthiswayfasterrepeatedrequestsonlargedimensiontablesandsavingmemorycapacity.
SOLAP SOLAP(SpatialOn‐LineAnalyticalProcessing)extendsOLAP.ItallowstoexploreradditionalspatialinformationprovidedbyGeographicInformationSystems(GIS).
SOLAPLayers SOLAPLayersisalightweightwebcartographiccomponent,whichenablesnavigationinSOLAPdatacubes.Itaimstobeintegratedintoexistingdashboardframeworksinordertoproduceinteractivegeo‐analyticaldashboards.Seehttp://www.spatialytics.org/projects/solaplayers/.
SpagoBI InJavawrittenOSBI.Coversthefullrangeofanalytic‐tools.DevelopedbytheOW2Consortium.Seewww.spagobi.ow2.org.
Star AschemausedforOLAPanddatawarehouses.Canbedescribedasadenormalizeddatabase,whichholdsduplicateddatatoprovideafastaccessfortheproveofanalysingandprocessingoflargeamountofdata.AstarhasalwaysafactanddifferentdimensiontablesThefacttablecontainsthedataandthedimensionslimitandextendthespecificviewonthedata.
APPENDIXC‐References
Literature
Source• BorisGloger„SCRUM–Produktezuverlässigundschnellentwickeln“,©2008
CarlHanserVerlagMünchenWien,ISBN978‐3‐446‐41495‐2
BachelorthesisSS2011 Page65/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• JasonWeathersby,TomBondur,IanaChatalbasheva,DonFrench“IntegratingandExtendingBIRT”,©2008AddisonWesley,ISBN978‐0‐321‐58030‐6
InternetType Source DateImage http://www.infobarrel.com/A_Quick_Scrum_Tutorial 02.03.2011Image http://training.inet.com/OLAP/Images/kube0006.gif 01.05.2011Image PresentationofThierryBadard“Dashboardandreporting
tools:IntegrationofBItoolsinthegeospatialdomain”06.05.2011
Image PresentationofThierryBadard“Spatialytics”,2010 06.05.2011
APPENDIXD‐Tableoffigures
Figure1‐LogovonpopulärenOSBI...........................................................................................................3Figure2‐LogodesUnternehmesSpatialytics.......................................................................................3Figure3‐SOLAPLayersDemoDashboard ..............................................................................................4Figure4‐BIRTReportmitSOLAPLayers‐Karte ...................................................................................5Figure5‐SOLAPLayerslogo.......................................................................................................................13Figure6‐VisualizationofanOLAP‐cube..............................................................................................16Figure7‐OverviewSOLAPLayers2.0Extended ...............................................................................18Figure8‐OverviewvisionSOLAPLayers2.0Extended..................................................................19Figure9‐Activitydiagram..........................................................................................................................20Figure10‐Packagediagram ......................................................................................................................21Figure11‐Classdiagram.............................................................................................................................22Figure12‐MetricsreportofSOLAPLayers2.0Extended .............................................................29Figure13‐OuputBuilderFactoryclass,whichwillbereplacedinafutureversion ..........34Figure14‐Sampleofsources.xmlwithdifferentdatasources...................................................35Figure16‐ResultofsampleMDXquerydisplayedascrosstable.............................................36Figure15‐SampleMDXqueryforSOLAPLayers ..............................................................................36Figure17‐SamplecrosstableinSOLAPLayersdashboardwithdimensionscivilstatusandsex..................................................................................................................................................................38Figure18‐SamplecrosstableinSOLAPLayersdashboardwithdimensionssexandcivilstatus .....................................................................................................................................................................38Figure19‐SamplecrosstableinSOLAPLayersdashboardwiththemeasuresnumberofpersons,totalsavings.....................................................................................................................................38Figure 20 ‐ Sample cross table in SOLAPLayers dashboard with the measures totalsavings,numberofpersons.........................................................................................................................39Figure21‐SampleSQLqueryforSOLAPLayers................................................................................39Figure22‐ResultofsampleSQLdisplayedastable ........................................................................39Figure23‐SOLAPLayerssampledashboard.......................................................................................40Figure24–Illustration:SOLAPLayerswithJSON‐outputandtwoconnections..................43Figure25–Illustration:SOLAPLayerswithImage‐outputandtwoconnections ...............44Figure26–Illustration:SOLAPLayersandoneconnection..........................................................45Figure27–OneconnectionwithoutSOLAPLayers ..........................................................................46Figure28‐ExtensionpointsusedbytheBIRT‐plugin....................................................................47Figure29‐MetricsreportofthefinalGeoextensions2Birt‐plugin............................................51Figure30‐Printscreen:CreateaReportProject..............................................................................53Figure31‐Printscreen:Selectadatasourcetype...........................................................................54Figure32‐Printscreen:Filloutthedatasourceparameterform ............................................54
BachelorthesisSS2011 Page66/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Figure33‐Printscreen:Choosethedatasource ..............................................................................55Figure34‐Printscreen:Generatedcolumnnames .........................................................................56Figure35‐Printscreen:Previewcolumnnames..............................................................................56Figure36‐Printscreen:Previewresultset.........................................................................................57Figure37‐Printscreen:Draganddropmapreportitem.............................................................57Figure38‐Printscreen:Definepropertiesformapreportitem ...............................................58Figure39‐Printscreen:ReportasHTML‐output.............................................................................58Figure40‐IntroductionofcreatingaSOLAPLayersdashboard–Step1...............................82Figure41‐IntroductionofcreatingaSOLAPLayersdashboard–Step2..............................83Figure42‐IntroductionofcreatingaSOLAPLayersdashboard–Step3..............................83Figure43‐SCRUMProcess .........................................................................................................................84
BachelorthesisSS2011 Page67/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
APPENDIXE–Projectmanagement
Thefollowingillustrationsshowtheprojectplanandtheprojectprogress.DetailedinformationcanbefoundontheCDinthedocument“projectplan.xls”.
Scheduling
BachelorthesisSS2011 Page68/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
TaskboardTask Priority Story
PointsTask Description Category State
1 30 4 FlexiblearchitectureforSOLAPLayersdatasources
Analyse and develop an architecture forSOLAPLayers,which allows a flexible chooseofthedatasource.ThedatasourcecanbeanOLAP‐oraSOLAP‐server,asimpleDB,flatfilelike XML, Excel, etc. The client sendsdepending on the data source a query andadditional information to get the desireddata. At the moment just SOLAP‐data isprovided.
ExtendingSOLAPLayer
Done
2 15 3 FlexiblearchitectureforSOLAPLayersoutputformats
Analyse and develop architecture forSOLAPLayers,whichallowsa flexibleoutput‐format. At the moment just a couple ofdifferent JSON‐formats are provided. At thefuture,formatslikeimages,KLM,etc.shouldbeprovided.
ExtendingSOLAPLayer
Done
3 15 5 PossibilityofDBasaSOLAPLayerdatasource
IncludeastrategyintoSOLAPLayerstoaccessdatafromadatabasethroughanSQL‐query.
ExtendingSOLAPLayer
Done
5 20 4 GetconfidentwithBirt,BIandspatialinformation
For the futurework it is necessary to knowhowtoworkwithBirt.Aswellitisamusttoknow the different technologies used bySpatialytics.
Knowledgeaccumulation
Done
6 20 2 CreateasampleextensionforBirt Create a small sample extension for Birt, inorder to get confident with the Birtenvironment and figure out the possibilitiesofextendingBirt.
Knowledgeaccumulation
Done
BachelorthesisSS2011 Page69/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
7 20 1 AnalyzeexistingmapsolutionsforBirt
Havealookatthetwoexistingmapsolutionsfor Birt and get the dis‐/advantages. Thishelpstoimprovethequalityoftheextension,whichhastobedeployed.
Knowledgeaccumulation
Done
8 30 4 Extendthe"SQL‐resultsettoOLAPJson"‐process
This task should add functionality to thetransform‐process to allow more complexSQL‐queries. The goal is to create a crosstable between a location and one or moredimension.Differentmeasuresarepossible.
ExtendingSOLAPLayer
Done
9 10 2 DocumentthenewSOLAPLayerarchitecture
Create a documentation of the newSOLAPLayers server architecture. Theprincipal topic is how to extend the existingsolution.
Documentation
Done
10 10 2 DocumenthowtouseSOLAPLayersforadashboard
Createasimpledocumentationforusingthedashboard. This may be a part of thearchitecturedocument.
Documentation
Done
11 25 3 AnalyzeBIRTandfindasolutionforintegratingamapcomponent
Find a way to integrate a map into BIRT.Search different solutions and figure outwhichistheonetogo.
ExtendingBIRT
Done
12 10 3 DocumentanalysisofBirt‐pluginforfuturetraceability
Show on which facts the final decision isbased.
ExtendingBIRT
Done
13 30 3 Createdatasourcepluginwithdummydata
Create as fast as possible a solution toconnect to an individual data source and toretrievedata inthecorrectway.ToreachasfastaspossibleanA toB scenario, thedatawill be static for themoment. The receiveddata has to be in the right format, to getdisplayedbyareportitem.
ExtendingBIRT
Done
BachelorthesisSS2011 Page70/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
14 30 3 Createamapreportitempluginwithdisplaysstaticcontent
Extend the report item palette with amap‐component,whichshowsamapwithdefault,staticdata.
ExtendingBIRT
Done
15 25 3 RetrievedynamicdatausingadefaultSQL‐queryandSOLAPLayersanddisplayitwithexistingreportitems
The retrievedOLAPJsonhas tobeconvertedintoa result set representation.All the corereport items handle data in a two‐dimensional‐format
ExtendingBIRT
Done
16 20 2 RetrievedynamicdatausingadefaultSQL‐queryandSOLAPLayersanddisplayitwiththenewmapitem
For representing themap theOLAPJson hastobeforwardedtothereportitem.
ExtendingBIRT
Done
17 15 3 CreateGUIandthelogicbehindforusingcustomizableSQL‐queries
ReplacethehardcodedSQL‐queryandallowthe user to define his own query with theaccordingparametersusingaGUI.
ExtendingBIRT
Done
18 15 3 CreateGUIforspecifythemapreportitemwithcustomizableparameters
The displayed SOLAPLayers‐map should becustomizableusingaGUI.
ExtendingBIRT
Done
19 10 6 Writeadocumentationaboutthestateoftheartofreportingtools
Write a documentation about the state oftheartofreportingtools
Documentation
Done
Excursus:StateofArtofReportingTools Page71/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
TimetableThe following report shows the time overview on the 18.05.11. Somemore workingtimewillbeneededforcorrections.Fordetailedinformationabouttheusedtimehavealook at http://sueess.mite.yo.lk (the user‐ and password‐data were send to thesupervisor)oratthefile“BachelorThesis/time‐entries.xls”ontheprojectCD.
Excursus:StateofArtofReportingTools Page72/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint0(28.02.‐06.03.11)
Excursus:StateofArtofReportingTools Page73/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint1(07.03.‐13.03.11)
Excursus:StateofArtofReportingTools Page74/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint2(14.03.‐20.03.11)
Excursus:StateofArtofReportingTools Page75/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint3(21.03.‐27.03.11)
Excursus:StateofArtofReportingTools Page76/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint4(28.03.‐03.04.11)
Excursus:StateofArtofReportingTools Page77/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint5(04.04.‐10.04.11)
Excursus:StateofArtofReportingTools Page78/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint6(25.04.‐01.05.11)
Excursus:StateofArtofReportingTools Page79/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint7(02.05.‐08.05.11)
Excursus:StateofArtofReportingTools Page80/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Sprint8(09.05.‐15.05.11)
Excursus:StateofArtofReportingTools Page81/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
APPENDIXF‐Licenseagreement
Agreement[This is atranslationof theoriginalGermanagreement,which is related to the Swisslaw.]
1.MatteroftheagreementThisagreementdefinestherightsfortheuseandthefurtherdevelopmentoftheresultsofthebachelorthesis„ExtendingBIRTwithGeospatialDataVisualizationcapabilitiesbyintegrating theSOLAPLayersmappingcomponent“byChristophSüess.The thesiswasdevelopedincooperationwiththecompanySpatialytics.
2.IntellectualpropertyrightsThestudentkeepsallhisintellectualpropertyrights.
3.UsageTheresultingsourcecodeistheexclusivepropertyofSpatialytics.ThestudentandHSRhaveno rights to use and extend the resulting source code.All documents beside thesourcecodecanbeusedandextendedbySpatialytics,byHSRandbythestudent.Thestudentdeclarestohavereadthisagreementandagreestotheseregulations.
Vereinbarung
1.GegenstandderVereinbarungMit dieser Vereinbarung werden die Rechte über die Verwendung und dieWeiterentwicklungderErgebnissederBachelorarbeit„ExtendingBIRTwithGeospatialData Visualization capabilities by integrating the SOLAPLayers mapping component“von Christoph Süess unter der Betreuung von Prof. Stefan Keller geregelt. DieBachelorarbeitwurdeinZusammenarbeitmitdemUnternehmenSpatialyticserarbeitet.
2.UrheberrechtDieUrheberrechtestehendemStudentzu.
3.VerwendungSämtliche Rechte des resultierenden Quellcodes stehen exklusiv Spatialytics zu. DerStudentunddieHSRhabenkeinRecht,denresultierendenSourceCodezuverwenden.DieresultierendenDokumente–abgesehenvomSourceCode–könnenvonSpatialytics,derHSRunddemStudentenverwendetunderweitertwerden.DerStudenterklärthiermitvondieserVereinbarungKenntnisgenommenzuhabenundstimmtdenBedingungenzu.QuébecCity,29.04.11
.........................................................
ChristophSüess,StudentHSR
Excursus:StateofArtofReportingTools Page82/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
QuébecCity,29.04.11
.........................................................
Dr.ThierryBadard,SpatialyticsRapperswil,
........................................................
Prof.StefanKeller,SupervisorHSRRapperswil, ......................................................... Prof. Hansjörg Huser, Head of department
ComputerScienceHSR
APPENDIXG‐Agreementoftheauthor
Icherklärenhiermit,• dassichdievorliegendeArbeitselberundohnefremdeHilfedurchgeführthabe,
ausserderjenigen,welcheexplizit inderAufgabenstellungerwähnt istodermitdemBetreuerschriftlichvereinbartwurde,
• dass ich sämtliche verwendeten Quellen erwähnt und gemäss gängigenwissenschaftlichenZitierregelnkorrektangegebenhabe.
Ort,Datum: Name,Unterschrift: ChristophSüess
APPENDIXH‐Instructions:CreatingadashboardusingSOLAPLayers
Figure40‐IntroductionofcreatingaSOLAPLayersdashboard–Step1
Excursus:StateofArtofReportingTools Page83/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Figure41‐IntroductionofcreatingaSOLAPLayersdashboard–Step2
Figure42‐IntroductionofcreatingaSOLAPLayersdashboard–Step3
Excursus:StateofArtofReportingTools Page84/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
APPENDIXI–CustomizedSCRUM‐processforbachelorthesis
Figure43‐SCRUMProcess
IntroductionSCRUMasaprojectmanagementinstrumentisusedfortheproject„ExtendingtheBIRTReporting Toolwith Geospatial Data Visualization”. The goal is toworkwith an agileprojectmanagementinstrumentthatfostersafastandhighqualitywayofdevelopment.InthecaseofthisbachelorthesisSCRUMhastobecustomizedfordifferentreasons:
• Theprojecthasjustonemainparticipant• Itisnotpossibletorealizetheprojectisolatedfromtherestoftheteam,
becauseofexistingtechnologicallydependencies• Themainparticipantassumesdifferentroles• Theprojectenvironmentdoesnotworkwithaspecificsoftwaredevelopment
processThisdocumentdescribesthecustomizedSCRUM‐processforthisspecificproject.
BasicprinciplesIncomparisontonon‐agiledevelopmentprocesses,SCRUMdoesnotignorethefact,thatitisnotpossibletocompletelyplanaprojectinadvanced.SCRUMknowsthereality:Thevision,thegoalsandtheprojectprocessareproductsofthewholeprocess.SCRUMbaseontheseimportantguidingprinciples:
• Attheendofeachsprintexistsafulltested,runningversionoftheapplication
Excursus:StateofArtofReportingTools Page85/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
• Nodocumentationforthesakeofjustdoingit.• "Workingmorehours"doesnotnecessarilymean"producingmoreoutput"
RolesSCRUMdefinesthefollowingroles:
• ProductOwnero DefinestheVision.TheScrummasterhelpstofindtherightdirection.The
ProductOwnerdefinesthegoalduringthesprint.Heplansandreviewsthedoneworkformalattheendofeverysprint.
• Teamo CreatestogethertheProductBacklog,alistofbybusinessvalue
prioritizedfeatures.Theteamdefinestheneededtimeforeachfunctionality.Duringasprint,theteamiscompletelyorganizedbythemself.Ifaproblemoccurs,theteamtalkstotheScrumMaster.
• Scrummastero Helpstheteamifitisstuck.Hewritesnocodebyhimself.Hehasthe
connectiontothemanagement.• Management
o TalksprimarytotheScrummasterconcerningfinancialandothermatters.• Customer• User
Becauseofthesizeoftheprojectteamtherolescannotbeassignedanddefinedinthesame way as usually: The team (Christoph Süess) will adopt the duties andresponsibilities of the product owner and the Scrummaster in co‐operation with themanagement(ThierryBadard).
SprintAsprintwillhavethedurationofoneweek,startsatMondayandendsusuallyatFriday(weekendasreserve).Thesprintendswithareporttotheprojectadvisor(Prof.Keller)on Monday. A sprint contains an amount of items out of the task board, which getsexpandedcontinually.Asprintshouldnotbechangedduringtheweek.Insomecasesitisnotpossibletoretainon the items (organizational reasons,dependencies tootherprojects). In this case theteamchangesthesprintandnotesthereason.
ProjectdocumentationBesidestheknowledge,designandanalysisdocuments,everysprintisrepresentedbyadocumentcontainingthefollowinginformation:
• Sprinttaskboardo Thechosenitemsfromtheprojecttaskboard
• Stateofprogresso Theactualstateofprogress
• Retrospectiveo Whatdidtheteamlearnduringthesprints?Improvementsforfuture
sprints?• Sprintreview
o Whatdidtheteamachieve?• ImmediateBacklog
Excursus:StateofArtofReportingTools Page86/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
o Wherearetheproblems?Whydoesdedevelopmentnotgoasfastasplaned.
• Logbooko Dateandreasonfortheadjustmentsofthesprint
APPENDIXJ–Interimreport
Seitdem23.Februar2011befindeichmichinQuébecCity,einersympathischenStadtan der Ostküste Kanadas. Ich habe hier eineWohnung gefunden undmir ein kleines,aber feines soziales Umfeld geschaffen. Meine Entscheidung, die BA im Ausland zuschreiben, bereue ich trotz allem Aufwand und einigen Hindernissen (sieheHerausforderungenundProbleme) inkeinemFall.Bereitsschaue ichmitWehmutansEndemeinesAufenthalts.IchschreibehiermeineBachelor‐ArbeitmitdemTitel„ExtendingBIRTwithGeospatialDataVisualizationcapabilitiesbyintegratingtheSOLAPLayersmappingcomponent“imdrei Mann Unternehmen Spatialytics. Der Kopf der Firma bilden Luc Vaillancourt(Management) undThierryBadard (Professor an derUniversität Laval, Entwicklung).Jean Mathieu ergänzt das Team. Alle drei haben Geologie studiert und sind aufverschiedeneWegeindieInformatikgekommen.AlswaschechterInformatikerergänzeichdasTeamsuperundkannmitmeinemWissendemUnternehmenweiterhelfen.
1.Woche–Umfeld,ToolsundTechnologienkennenlernenDieersteWocheimneuenUnternehmen:Esgabvielzulernen.VorallemWissenrundum OSBI, Reporting Tools und Birt, aber auch das klassische kennenlernen desUnternehmens hielten mich während den Tagen auf Trab. Nebenbei plante ich meinVorgehenundversuchtedieerstenZielezuformulieren.
2.–4.Woche–SOLAPLayers–OSBImeetsGeo‐DashboardIn den drei Wochen kümmerte ich mich voll und ganz um SOLAPLayers. MeineAusgangslage war ein kleines Projekt. Das Tool war funktionstüchtig, aber nur inwenigen Hinsichten objektorientiert und für den Quellcode hätten Sie mirwahrscheinlichmeineSE2‐Noteentzogen.WasistSOLAPLayers?WelchesistdieVisionhinterSOLAPLayers?„SOLAPLayersistprimäreinFrameworkumgeografischeDatenvonSOLAP‐DatenbankenineineminteraktivenDashboarddarzustellen.SekundäradressiertSOLAPaberauchandereDatenquellenund präsentiert auch nicht geografische Daten. Somit ist SOLAPLayers ein Tool um kompletteDashboard‐Anwendungen zu erstellen. SOLAPLayers ist eine Webapplikation, die serverseitig inJava umgesetzt ist und im Clientbereich mit JavaScript (EXT JS‐Framework), HTML und CSSarbeitet.SOLAPLayersbefindetsichnochindenKinderschuhenundstehtuntereineropen‐sourceLizenz(MehrInfos:www.solaplayers.org)“Mein Aufgabenbereich befand sich ausschliesslich im Server‐Bereich. GrundsätzlichhabeichzweigrosseÄnderungen,sprichErweiterungenforciert:
1. UmgestaltenderSoftwareArchitektur,sodassneuenDatenquellenundOutput‐FormateeinfachundimobjektorientiertenSinnehinzugefügtwerdenkönnen.DazugehörteaucheinRefactoringdesQuellcodes(sodassderQuellcodeSE2würdigist).
Excursus:StateofArtofReportingTools Page87/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
2. HinzufügeneinesTreibersfürrelationaleDatenbanken.DieserSOLAPLayers‐TreibererlaubtesanhandeinesSQL‐QueriesundweiterenParameterneinenMDX‐QueryzusimulierenunddieResultatewiegewohntanzuzeigen.DieseErweiterungwarnötig,danachwievorrelationaleDatenbankendasZepterhalten(auchwennsieimBereichderAnalysesehrvieleNachteilemitsichbringen).
DaichanfänglichvondermirzurVerfügunggestelltenBasisetwasgeschocktwar,habeichsehroft inEigenregiegehandelt.Thierrywardannaber jeweilssehrzufriedenmitdemResultat.
5.–6.WocheNunkamdasReportingToolBIRT insSpiel,welchesmir seit zweiWochendasLebenschwer macht. BIRT ist ein Eclipse‐Core‐Project und wird vor allem von demUnternehmen Actuate entwickelt. BIRT ist zur Anwendung und zum Integrieren inandere Java‐Applikationen bestimmt ein brauchbares Werkzeug. Wer BIRT jedocherweiternwill, fühlt sichöfterswieeinNomadeaufderSuchenachWasser irgendwotiefinderSahara.NachvielenüberwindetenHürdenhabe ich es geschafft ein simples „A toB“‐Scenarioumzusetzen.AbHeuteistesnunalsoMöglich,SOLAPLayersalsBIRT‐Plug‐Inzunutzen,auchwenndieBedienungnochweitvonBenutzerfreundlichentferntist.ThierrysiehtdieProblememitBIRTundäussertsichsehrpositivzumeinemaktuellenStand.
WeiteresVorgehenDienächstenzweiWochensucheichkeinWasser,sonderngeniesseichdieneueWeltinvollenZügen.NachdenzweiWochenBreak,wende ichmicherneutdemBIRT‐Plug‐Inzu.DaicheinesehrfixeDeadlinehabe,steht indenletztendreiWochenvorallemdieDokumentation an erster Stelle. Ziel ist es Spatialytics ein gut dokumentiertes,funktionsfähiges BIRT‐Plug‐In zu hinterlassen, mit dem Spatialytics, Sie und auch ichsehrzufriedensind.
HerausforderungenundProblemeDie Themen des Semesterprojekts und meiner Studienarbeit habe ich jeweils selberdefiniert und zusammen mit meinem Team sehr gute Resultate erzielt. DieHerausforderung lag jeweils ganz klar im technischen Bereich, sprich Performance,Concurrencyusw.warenProblemediewirforcierten.MeineBAbringtganzandereProblememitsich.DieseHindernissesindnichtwenigerlehrreich, aber leider verhindern sie teilweise, dass ich mich den interessanten,technischenProblemewidmenkann:
1. SpatialyticsisteinStart‐Up,diestabilenSäulenfehlennoch,sprichallesistimAufbau.Auchwenndassehrspannendist,istesgrundsätzlicheinschlechtesUmfeldfüreineBA.EsbestehenvieleAbhängigkeiten,diesichimmerwiederändern.
2. AuchwennSpatialyticsinzwischeneingrossesIT‐Knowhowhat,istderQuellcodenichtsoprofessionell,wieeranderHSRerwartetwird.Daraufaufzubauenistschwierig.DarumerfordertmeineBAvielZeitumsolchenCodein
Excursus:StateofArtofReportingTools Page88/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Ordnungzubringen(Refactorings).GewisseVorgabenakzeptiereichaberauszeitlichenGründen,ohnedentechnischenSinnzuhinterfragen.SchliesslichwillichmitmeinerBAnebenSpatialyticsauchSie,alsmeineBetreuerundBewerter,zufriedenstellen.
3. InzwischenklapptdasDokumentierenundDiskutiereninEnglischziemlichgut,trotzdemdieFremdsprachekostetZeitundEnergie.
4. AlleineaneinemspezifischenProjektzuarbeitenbringtgenausovieleVor‐wieNachteile.EinerseitsentfällteinGrossteilanKommunikation,andererseitsfehltaucheinePersonumtechnischeundorganisatorischeProblemezudiskutierenundsicheineZweitmeinungeinzuholen.
5. DaseinzigeProblem,dasmichwirklichaufTrabhältundauchmalaufmeinegrundsätzlichhoheMotivationschlägtheisstBIRT.DasErweiternvonBIRTistsehrschlechtdokumentiert,auchwennBIRTgrundsätzlicheinenormaleRCPist,gibtesvieleBIRT‐spezifischeDinge,dienichtdokumentiertsind.DiezweiexistierendenoffiziellenTutorialssindnuraufWindowslauffähig,daSieeinBasis‐KonzeptvonSWTignorieren.SelbstdaseineexistierendeBuchistnichtsmehralseinAbbilddesohnehinschonschlechtenJavaDocs.WerinderCommunityundderDokumentationkeineAntwortenfindet,hatnureineMöglichkeit:DirektdenQuellcodeanalysieren.BeiBIRTistmandortbeiderQuelledesGrauensangekommen.UnglaublichwieschlechtderCodeist,einesuperVorlagefüreinSE2‐Projekt.Eskannsehrfrustrierendsein,wennmanstundenlanganstellevonproduktiverArbeitsichdemformulierenvonGoogle‐SuchstringsunddemdurchforstenvonForenwidmenmuss.
FazitObwohl die Probleme viel Zeit in Anspruch nahmen, bin ich nachwie vor überzeugt,dassichmichaufeinemsehrgutenWegbefinde.DasFeedbackvonThierrywarbisherimmersuperundvonderBetreuer‐SeitewurdenbisanhinnochkeineBedenkenoderähnlichesgeäussert.Auchwennesetwas schade ist,dass ichaufGrundderProblemenichtgleichvielAufwandindasbewältigenvontechnischenHürdensteckenkann(wiebeimSemesterprojektundderSA)istdieseArtvonBAsichereinesuperVorbereitungfürdieArbeitnachdemStudium,wodieBedingungenauchunterschiedlichsind.Ich habe in den letzten sechsWochen sehr viel gelernt und freuemichdieBA in denletzten dreiWochen fertigzustellen. Ich hoffe, dass ich auch Siemitmeinem Resultatbegeisternkann.
APPENDIXK‐Resultpresentations
Duringtheprojectmultiplepresentationwerehold.ThenotescanbefoundontheCD:• BIRT‐Open‐sourceBusinessIntelligenceandReportingTool• SOLAPLayers‐Changesandimprovementsofthenewversion• MapextensionforBIRT• SOLAPLayerstoBIRT–Finalresultspresentation
Excursus:StateofArtofReportingTools Page89/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
APPENDIXL–Stateoftheartofreportingtools
1 Introduction.......................................................................................................................................902 Purposeofreportingtools...........................................................................................................903 Functionalityofabasicreporttools........................................................................................914 Dashboard...........................................................................................................................................915 Opensourcereportingtools .......................................................................................................92Pentaho..................................................................................................................................................92About .................................................................................................................................................92Reportingtools..............................................................................................................................92
JasperSoft..............................................................................................................................................92About .................................................................................................................................................92Reportingtools..............................................................................................................................93
Actuate ...................................................................................................................................................93About .................................................................................................................................................93ReportingTools.............................................................................................................................93
ComparisonMatrix...........................................................................................................................94DesignParadigm‐Pixelpositioning ................................................................................. 103Reportcompilation/Reportformat................................................................................. 103Eclipseplug‐inavailable......................................................................................................... 103DataSourcesTypes .................................................................................................................. 104GraphicalQueryDesigner...................................................................................................... 104
Alooktothefuture........................................................................................................................ 1046 Conclusion ....................................................................................................................................... 1057 Sources.............................................................................................................................................. 1058 Tableoffigures .............................................................................................................................. 106
Excursus:StateofArtofReportingTools Page90/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
1 Introduction
Figure44‐Ilustrationofthedecisionmakingprocess
In the nineties the term Business Intelligence came up. Business Intelligence is abusiness‐processfordecision‐making.BIhelpstonavigatetherunningbusinessintotherightdirection.Tomakecorrectdecision, thedecision‐makerneedsanoverviewofallinformation, which could influence his decision. BI is a process, which manages thecollationandthetransformationofrawdataintomeaningfulinformation.Nowaday,BIissupportedbyalotofpowerfulapplication,whichcanhandlethehugeamountandthecomplexrelationsoftheinformation.This excursion will inform the reader about a special part of BI, the reporting part.Reporting tools are applications they access data and represent the information in ahumanreadableformat.Thegoalistocreatereports,whichhelpsthedecision‐makertooverviewandfindtheimportantinformation.Thisarticledescribesthepurposeandtheevolution of reporting tools aswell as it shows some concrete open source software‐toolsandthereadvantagesanddisadvantages.
2 PurposeofreportingtoolsNotallcompaniesuseprofessionalreportingtools.Especiallysomesmallerfirmsworkwithoutreportingtools.MicrosoftExcelisnotarealreportingtool,butitisusedbyalotofcompaniesforcalculationandreportingreasons.ThemajorityofbigcompaniesusesBI‐suitestocoverthewholebusiness‐process.TheleadingcompanyinbusinesssoftwareisSAP.SAPhastoolsfordifferentpartsoftheBI‐process. One of there reporting tools is SAP Business Explorer. Other well‐knownproprietaryreportingtoolsareOracleReports,BissantzDeltaMaster,Cognos8BI,SAPCrystalReportsandCubewareCockpitV6pro.JusttheOracleandtheSQP‐toolsarepurereporting tools, the other products are complete BI‐suites, which include a reportingcomponent.
Excursus:StateofArtofReportingTools Page91/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
With the evolution of the proprietarysoftware, some open source toolsentered the market as well. Theproprietary providers of BI‐tools arestill leading, but the open source partatmarketisgrowingfast.Alotofopensourceproject haveproven, that opensourcedoesnot just stand fora socialthought, moreover it has a lot ofadvantages for the customers and forthe companies, which contribute thesourcecode.
3 FunctionalityofabasicreporttoolsAllcommonreportingtoolsfunctioninthesamebasicallyway:
Thefirststepistodefineoneormoredatasources.Thiscanbearelationaldatabase,anOLAP‐Server,flatfiles,webservicesoranyothersupporteddatasource.
Inasecondsteptheusercreatesareport‐template.Alotofreportingtoolsprovideadesigner,whichallowscreatingareportinaneasyway(WYSIWYG).Thereportcreatordefineswhichdata,fromwhichdatasourceisdisplayedinwhichway.Hecanchoosereportitemsliketables,crosstables,chartsandmanyotherrepresentations.Thisreportisoftenrepresentedbyanxml‐fileonthefilesystem.
Theresultingreportisnowinterpretedandfilledwiththeinformationusingthegivendatasources.Commonoutputsarepdf‐,xls‐,html‐ordoc‐files.
Formoreadvanceduse,thereportscanbedeployedonservers.Thisallowsprovidingthereportstodifferentpeoplesorevenasawebservice.
4 DashboardReports represent a static view on data.Aftertheenginecreatedthereports,theyare saved in a common format like pdf,xls, ppt, doc, etc. These reports can becreated new at any time, to get actualinformation.Dashboardsgoastepfurther.Dashboardsallow navigating through data, gettingmore details and seeing related content.Dashboard are often implemented aswebsites, this allows to set access rightsfordifferentusers(e.g.differentviewsfordirector, department heads and teamleaders). Some reports provide somedashboardfunctionalitythat iswhyinsomecases it isnotpossibletoclearlyseparatedashboardsfromreports.Thefollowingtableshowstheadvantagesofreportsanddashboardsingeneral:
Figure45‐CrystalReportsdashboard
Figure46‐Drilldownreport
Excursus:StateofArtofReportingTools Page92/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Report Dashboard• EasytoprintandsendbyE‐Mail• Canbeusedofflineaftercreation• Easytoarchive
• Allowstonavigatethroughtheinformation(drilldown/rollup)
• Bettervisualisationofrelationsbetweendifferentdata
• Betterunderstandingandoverviewthroughinteractivepresentation
5 Opensourcereportingtools
Figure47‐Well‐knownOSBI‐tools
The most famous open source reporting tool providers are JasperSoft, Pentaho andActuate, all of themuse Java. For all thisproducts exists a commercial version. In thefollowingchapters thecompaniesandtherereportingtoolsareexplained. Ina furtherchapterthethreeproductsarecomparedandsomeoftheimportantfactsaredescribedmoredetailed.
Pentaho
AboutPentahoisacompanyfromOrlando,Floridawitharound200employees.Thecompanyoffers a complete BI‐suite. Important and well‐known tools of Pentaho are Kettle,MondrianandWeka.Withmore thansixmilliondownloadsof thereproducts,aroundthousandenterprisecustomersandacommunityof20’000peopleisPentahotheheadoftheopensourceBI‐movement.
ReportingtoolsThePentahoReportingCommunityEditionprovidesthreedifferenttools.
ThePentahoReportDesignerisusedtocreatereports.Itisastandaloneapplication,whichallowsdragginganddroppingreportitemsintoareport‐document.Propertiesallowcustomizingtheseitems.OpenFormula/Excel‐formulaexpressionsmakeitpossibletoformatdatadynamically.PentahoReportDesigneriseasytousefordevelopersaswellasforbusinesspeople.
ThePentahoReportingEngineisapartofthePentahoReportDesigner.Thisenginecreatesthereportswithinthedesigner.
ThePentahoReportingSDKisusedtointegratePentahoinserverorofflineapplications.ItallowsusingtheReportingEngine‐featuresasajavalibrary.
AdditionallyareportservercalledPentahoBIServerexists.
JasperSoft
AboutLikePentaho,JasperSoftprovidesacompleteBi‐suiteaswell.JasperSoftislocatedinSanFranciscoandhasaround100employees.Fromthe twelvemilliondownloadsaroundtwelve thousand people use the commercial licence. One of the popular products ofJasperSoftistheOLAPserverMondrian.
Excursus:StateofArtofReportingTools Page93/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Reportingtools
Figure48‐IReportfromJasperSoft
IReport is the name of the JasperSoft report designer. With IReport reports for theJasperReports engine are created. JasperServer allows uploading reports andprovidingthemaswebsites,webservicesandscheduleddistributions.
Actuate
AboutThe company Actuate has his headquarter in San Francisco, California. Actuate is themaincontributorof theEclipsecoreprojectBIRT.ActuatedoesnotprovideanyotherBI‐products.Around600peopleworkforActuate.
ReportingToolsBIRTisanEclipsebasedreportingtool.ItcanbeintegratedintoJ2EEapplicationsorcanbe used as a simple standalone‐reporting tool. BIRT allows specifying almost everydetail of a report. BIRT is consisted of two components. The Eclipse integratedBIRTREPORTDesignerallowstocreatexml‐basedreportsusingacomfortableWYSIWYG‐editor.TheBIRTReportEngineprocessesthexml‐basedreportsandcreatesreportsindifferentoutputformats.TheBIRTIServerisaproprietaryproduct,whichdeliverstheinformationtoanunlimitednumberofpersons.
Excursus:StateofArtofReportingTools Page94/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
ComparisonMatrixThe following table was published by Innovent Solutions (http://www.innoventsolutions.com/birt‐jasper‐pentaho‐comparison‐matrix.html)andupdatedon03.05.2011bytheauthorofthisdocument.Themarkedrowsaredescribedmoreindetailattheendofthedocument. BIRT2.6.2 JasperSoft 4.0.2
CommunityPentaho Reporting CommunityEdition3.8
Website www.birt‐exchange.comwww.eclipse.org/birt
www.jasperforge.com reporting.pentaho.com
License EclipsePublicLicense GNULesserGeneralPublicLicense GNULesserGeneralPublicLicenseREPORTDESIGNER BIRTReportDesigner2.6.2 JasperIReport4.0.2 PentahoReportDesigner3.8stableDesignerPlatforms Windows,Linux,MacOSX Windows,Linux,MacOSX Windows,Linux,MacOSXEclipsePlug‐inAvailable Y N
JasperAssistant is availablecommercial, non‐free, third‐partyplug‐in
NNote that the Pentaho DesignStudio, an Eclipse plug‐in, is not areportdesigner.
NetBeansPlug‐inAvailable N Y N
StandaloneJavaClientAvailable
Y Y Y
DesignParadigm WebPageDesignparadigm:frames,tables,lists
banded reports, pixel positioning,table
bandedreports,pixelpositioning
ReportCompilation Notrequired Required Notrequired
Excursus:StateofArtofReportingTools Page95/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
ReportFormat XML(.RPTDESIGNfilesarepureXML)
.JRXML report design files arecompiled into .JASPER report files,whichareJavaByteCode.Youthendeploythe.JASPERfile.
XML(.PRPT report file is a ZIP thatcontains the XML file plus otherresources)
ReportDesignerComponentsCommonReportDesignerComponents:‐ReportEditorPalette‐DataExplorerProperty‐EditorOutlineview‐PreviewExpression‐BuilderReport‐ProblemsChartBuilder‐ScriptEditor
Y Y Y
Sub‐reports Y Y YSide‐by‐sidereportcomponents
Y Y Yviasub‐reports
Tables Y Y NCross‐tabs Y Y “Experimental”inPentaho3.5SortingHorizontalPanning Y Alwaysscrolldown,evenifcrosstab
expandsside‐to‐sideN
Newspaper/multi‐columnlayout
N Y N
Excursus:StateofArtofReportingTools Page96/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
Hyperlinkswithinareport Y Y Y
Actionablecharts Ydrill‐down,hyperlinks,mouse‐overs
Partial–hyperlinks N
CascadingStyleSheets(CSScontrolledformat)
Y Y N
ConditionalFormatting Y Partial Y
DataSourcesMultipledatasourcesandqueriesperreport
Yes,plussupportforjoiningthem Only via sub‐reports (one datasourcepersub‐report)
Only via sub‐reports (one datasourcepersub‐report)orincharts
SupportforjoiningmultipledatasourcesintheDesigner
Y N N
Reportcanfurtherre‐sort,filter,orgroupdatareturnedfromaquery.
Y Partial – Jasper can furthermanipulate data sets beforebuilding cross‐tabs, but not forregulartables.
N
‐DatabaseJDBC ‐DatabaseJDBC ‐DatabaseJDBC‐XMLFile ‐XMLFile ‐XMLfiles‐WebServices ‐CSVFile ‐Table‐Flatfiles:CSV,SSV,PSV,TSV ‐MicrosoftExcel ‐OLAPMDX‐Customdatasource ‐JavaBeans ‐PentahoMetadata
DataSourcesTypes
‐OLAPMDX ‐HibernateHSQL ‐PentahoDataIntegration
Excursus:StateofArtofReportingTools Page97/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
‐Scripted Data Source: POJO, EJB,Hibernate,XMLStream
‐SpringHibernateEJBQL ‐OLAPMDX
‐XMLAServer ‐Scripted Data Source:POJO,EJB,Hibernate
‐MondrianOLAP ‐TextFile ‐NetBeansJDBC ‐POJOs ‐Custom
‐RemoteXMLfiles QueryDesigner Y Y YGraphicalQueryDesigner Prototypeonly Y(SQLLeonardo) Y(SQL‐Leonardo)
‐JavaScript ‐JavaScript ‐JavaScript‐JavaEventHandlers ‐Groovy ‐BeanScriptFramework(BSF) ‐Java ‐Bean‐ScriptHost(BSH)
Scripting
‐SingleValueQueryOutputFormatsPaginatedHTML Y Y YUnpaginatedHTML Y N NPDF Y Y YExcel(XLS) Y Y YXML Y Y YPlainText Y Y YRTF Y Y YPowerpoint(PPTorPPTX) Y Y N
Excursus:StateofArtofReportingTools Page98/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
CSV Y Y YPostscript Y Y YOpenOfficereporttypes(document&spreadsheet)
N Y N
MicrosoftOffice2007reporttypes(DOCX,XLSX)
N Y N
Flash(SWF) N Yviewed within Jasper Server’s flashreportviewer
N
CustomFormats Y Y YGeometricshapes&lines Y Y Y
Barcodes Y Y YCharts ChartWizard Y Partialtwo‐stepwizard,restisabig
dialogboxN
ChartInteractivity Ymouse‐over, tool tips, hyperlinks,etc.
Yhyperlinksonly
N
Chartthemes Y Y NPrecisecontroloverformatofallcontrolelements
Y N N
Excursus:StateofArtofReportingTools Page99/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
Chartstypesthatallhave:2D,3D,Pie,Multi‐pie,Bar,StackedBar,BarXY,Line,LineXY,Area,AreaXY,StackedArea,BarLine,Bubble,Scatter,Plot,Multi‐Axis
Y Y Y
StudyCharts Y N NRingChart N N YTubechart Y N NConechart Y N NPyramid Y N NTimeSeries Y Y NMeter/Gauge Y Y NWaterfall N N YStep N N YStepArea N N YDifference Y N YRadar/Spider Y Y YThermometer N Y NCandlestick/StockChart(High/Low)
Y Y N
Gantt Y Y NSurveyScale N N YBarSparkline N N YLineSparkline N N YPieSparkline N N Y
Excursus:StateofArtofReportingTools Page100/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
ReportParameterizationStaticParametersselectparametervaluesfromahard‐codedlistofvalues
Y Y Y
DynamicParametersusersselectparametersfromalistofvaluesthatcamefromadatabase
Y YCascading input controls are reportindependent
Y
Cascadingparameterslinkeddatadrivenpromptse.g.‐Country|‐State|‐City
Y Y Y
Calendardate‐pickerforparametersoftypedate.
Y Y Y
Canspecifydefaultvalues Y Y Y
Drop‐downlistboxes Y Y YRadiobuttons Y Y YCheckboxes Y Y YComboboxes Y Y YAggregates/SummarizeData
‐Average ‐Average ‐AverageCommonAggregations‐Count ‐Count ‐Count
Excursus:StateofArtofReportingTools Page101/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
‐DistinctCount ‐DistinctCount ‐CountbyPage‐First ‐Sum ‐GroupCount‐Is‐Bottom‐N ‐First ‐Sum‐Is‐Botton‐N‐Percent ‐Lowest(Minimum) ‐Minimum‐Is‐Top‐N ‐Highest(Maximum) ‐Maximum‐Is‐Top‐N‐Percent ‐StandardDeviation ‐SumQuotient‐Last ‐Variance ‐SumQuotientPercent‐Max ‐System ‐Calculation‐Median ‐CountforPage‐Min ‐SumforPage‐Mode ‐Sum(Running)‐MovingAve ‐Count(Running)‐Percentile ‐GroupCount(Running)‐Percent‐Rank ‐CountDistinct(Running)‐Percent‐Sum ‐Average(Running)‐Quartile ‐Minimum(Running)‐Rank ‐Maximum(Running)‐RunningCount ‐PercentofTotal(Running)‐RunningSum ‐StandardDeviation ‐Sum ‐Variance
‐WeightedAverage UserDefinedFunctions/Expressions
YJava,JavaScript
YJava,JavaScript,orGroovy
YOpenFormula(Excel‐like),Java
UserDefineAggregates Y Y Y
Excursus:StateofArtofReportingTools Page102/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
BIRT2.6.2 JasperSoft 4.0.2Community
Pentaho Reporting CommunityEdition3.8
Designerprovides“codehooks”(akacallbacks,eventhandlers.…)foreachreportelement.
Y Partial – only the report itself hascallbacks – not at the per‐elementlevel.
NotinDesigner.
SupportforReuseWithinDesignerTemplates(customreportstartingpoints)
Y Y Y
User‐definedLibraries(reusablereportpieces)
Y N N
Stylescolors,fonts,borders,margin…
Y Y Y
CSS Y Y N
Excursus:StateofArtofReportingTools Page103/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
DesignParadigm‐Pixelpositioning
Thereare twoways toposition the report items, like tables, charts, etc. intoa report.Pentaho and JasperSoft allow positioning their elements using pixels. In thisway thepositioning can be done really exactly. This may have some advantages. The bigdisadvantages of this variant are that the reports are shown the sameway on all thedifferent screens and that all elements have to be positioned, since there is noautomatism.Thebothreportingtoolsshowareportinthesamewayonawide‐screenastheyshowitonanormalscreen.Thisisnotverycomfortable.BIRTusesanotherway topositionhiselements.Thereport canbesplit intodifferentcellsusingagridlayout.Theeffectivesizeofreportitemswillbecalculateddynamicallyforeachscreen.
Reportcompilation/ReportformatPentahoandBIRTareworkingthesameway.ThedesignercreatesanXML‐file,whichisusedbythereportenginetoretrievethedataandcreatetheexpectedoutput.UsingaXML based file has some important advantages. XML is a human‐readable format. Adesigner is not necessary to create reports, as long the syntax is documented well.Another advantage is the collaboration with source control tools. The differencebetweentwoversionsofareportiseasytofigureout.JasperSoftcompilesthereportsinthedesignerandcreatesanunreadabledataformat.This may speed up the interpretation‐process, but all the mentioned advantages ofreadableformatsarelost.
Eclipseplug‐inavailableWhy dowe care about anEclipse plug‐in? Does itmatter if I create myreport in Eclipse or in astandalone application?Yes,itdoes.Bigcompaniesuse the reporting toolsrarely for to create somereports on a standalonecomputer. They want toprovidereportstopersonsin different executivepositions. For this matterJ2EE applications werewritten. Since Eclipse isthe most widespread IDE,it makes sense to be abletoworkonthereportand
theapplicationinthesameenvironment.
Figure49‐BIRT‐pluginforEclipse
Excursus:StateofArtofReportingTools Page104/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
DataSourcesTypesA lot of data sources have something important in common.There results of this data sources ends in a two‐dimensionalresultset.All thereport itemsareabletodisplayatableoracelloutofthetable.MDXqueriesforexampledonotdeliveratwo‐dimensional result set, the result is three‐ or evenmultidimensional. BIRT for example supports OLAP just in anon‐nativeway.Itallowstocreatedatacubesoutoftables,butit is not possible to send amdx query to aOLAP‐server anddisplaytheresultdirectlyasacrosstable.
GraphicalQueryDesignerAcommonproblemwith reporting tool is there complexity. Creating complex reportsusingdifferentdatasourcesandreportitemsinaneasywayisobviouslyanantithesis.Thisresults inthe fact thatreportsoftenhavetobecreatedbydevelopersandnotbybusiness‐people, which are familiar with the thematic. Another fact is, that thecommunication between to different involved parties often raises problems. Featureslikeagraphicalquerydesignertrytoreducethesedependencies.
AlooktothefutureThe business‐world grows fast. New trends, like social medias, change themethodologies and strategies of companies. New technologies allow the company tocapturenewandmoreinformation,whichcouldinfluencetheirstrategy.Reportingtoolshavetoadapttothisalternations.
Ontheonehand,itisthetechnicalaspect,whichhastobeimproved.Thegiganticamountofinformationmakesit difficult to create reports and dashboards in anappropriate time. Inour fast‐changingworld isno timetowaitforhourstogetanoverviewoverthefacts.That
iswhyknowndistributorsofreportingtoolsworkonasolution, to make the process faster then ever before.
Formillionsofdatasets,itisnormaltowaitforsomehours.Usingthenewsystemstheprocesscanbedoneinreal‐time.Oneofthesesolutionswaspresentedattheendof2010bySAP.SAPHANA(SAPHigh‐PerformanceAnalyticAppliance)ishighperformancesystem.HANAusesatechnologycalled In‐Memory Computing. The two main character of In‐Memory are, that thedatabasesarestoredinthemainmemoryandwithanewdatabaseconcepttheamountoftoucheddataforarequestcouldbereduced.Thiscombinationofsoft‐andhardwareinnovation improves theusage of reporting toolsimmense. Hilti, aconstruction‐companyfrom Liechtenstein, is oneof the first well‐knowncompanies, which use SAPHANAforthereBI‐process.
Figure50‐Visualizationofadatacube
Figure51‐LogoSAP
Figure52‐SpatialyticsconnectsBIwithgeo‐spatialcomponents
Excursus:StateofArtofReportingTools Page105/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Besidesoftheuseofnewtechnologiestospeed‐uptheprocess,thetypeofinterestinginformationchanges.Agoodexampleforthegrowingoftheamountofdatashowsthebonusprogramstheycameupintheearlynineties.Withnewinformationofcustomers,themarketing‐strategycanbeoptimizedforeachcustomerindividual.Aninterestingaspectofinformation,whichwinsmoreandmoreoftheattention,isthegeo‐spatialaspect.Withtheevolutionofsmartphones,locationbasedappscameupfast.A new trend is the location based real‐life games, likefoursquare. It is for sure, that this trend will reach thebusiness‐worldaswell and thegeo‐spatial informationwillinfluentthestrategiesanddecisionsofthecompanies.The modern reporting tools contain no or just simplecomponentstodisplaygeo‐spatialinformation.Butapinona map is not the best and only way to show geo‐spatialinformation.Therearemuchmoreopportunities todisplaycomplexformsandchartsdirectlyonmapsandrelatethemto other data representations. Spatialytics is a company,whichworkson thisparts and tries tobring this feature toopensourcereportingtoolsAthirdaspect,whichwillbeimprovedinthefuture,isthewayofthepresentationofthedata. Already now, some contributors are developing apps for displaying reports onmobiledevices.Especiallytablet‐pcsallowtodiscussdecisionseverywhere,atanytimeinaconfortableway.
6 ConclusionTheimportanceofreportingtoolsishugeandit isstillgrowing.Theinfluenceofopensourceproductsisbiggertheneverbefore.Allthepresentedopensourceproductshavesimilarfeatures.Therealdifferentcanbefoundinthedetails.Inthefuturenewfeatureswillamplifytheusageforbusiness‐companiesandthecontributorsofreportingtoolcanbenefitfromthesedevelopments.
7 SourcesType Source DateImage http://a3.mzstatic.com/us/r1000/008/Purple/ea/c2/40/mzi.n
agcebvp.175x175‐75.jpg15.05.2011
Data http://de.sap.info/hana_newdb_westmere_inmemory/44054 15.05.2011Data http://en.wikipedia.org/wiki/Report 15.05.2011Data http://fico‐forum.de/Painter.php Data http://help.sap.com/saphelp_ish471/helpdata/DE/84/b0b739
7a672963e10000000a114084/content.htm13.05.2011
Data http://it‐republik.de/jaxenter/artikel/Pentaho‐BIRT‐und‐JasperReports‐im‐Vergleich‐1737.html
14.05.2011
Data http://jasperforge.org/plugins/espnews/?group_id=102 13.05.2011Data http://jasperforge.org/plugins/espnews/?group_id=112 14.05.2011Data http://jasperforge.org/plugins/espnews/?group_id=83 14.05.2011Image http://multivu.prnewswire.com/mnr/sap/37408/images/374
08‐hi‐sales_report.png13.05.2011
Image http://plugins.netbeans.org/data/images/1197052333893_ireport.jpg
14.04.2011
Figure53‐BIRTMobileViewerIPadapp
Excursus:StateofArtofReportingTools Page106/106
ChristophSüess Version:1.0 Lastchange:16.06.1119:48
Data http://www.cmscritic.com/pentaho‐bi‐suite‐3‐8‐puts‐more‐power‐in‐the‐hands‐of‐end‐users/
14.05.2011
Data http://www.computerwelt.at/detailArticle.asp?a=133566&n=2 14.05.2011Data http://www.computerwoche.de/software/bi‐
ecm/1934051/index2.html13.05.2011
Data http://www.computerwoche.de/software/bi‐ecm/2368387/ 13.05.2011Image http://www.conbusinessintelligence.com/wp‐
content/uploads/2009/11/database_and_business_intelligence.JPG
15.05.2011
Image http://www.fernstudium‐fernschulen.de/assets/images/sap.jpg
15.05.2011
Data http://www.heise.de/developer/meldung/Sapphire‐Now‐SAP‐forciert‐In‐Memory‐Computing‐1003014.html
13.05.2011
Data http://www.innoventsolutions.com/open‐source‐reporting‐comparison.html
14.05.2011
Data http://www.innoventsolutions.com/open‐source‐reporting‐comparison.html
13.05.2011
Image http://www.panorama‐novaview.de/images/cube_medium_2.jpg
15.05.2011
Image http://www.pentaho.com/images/rich_chart_editor.jpg 15.05.2011Data http://www.refocusingtechnology.com/2009/03/03/4‐adv‐
dashboard/15.05.2011
Data http://www.sap.com/germany/plattform/in‐memory‐computing/index.epx
15.05.2011
Data http://www.sap.com/germany/solutions/sap‐crystal‐solutions/index.epx
13.05.2011
Image http://www.stonetemple.com/images/UNICA_Drill_Down.jpg 14.05.2011Data http://www.tecchannel.de/server/sql/1750691/business_intel
ligence_teil_8_bi_software_tools/index9.html13.05.2011
Data http://www.youtube.com/watch?v=RecSOmN34Sc 13.05.2011Data https://foursquare.com
15.05.2011
Image /Data
OSBI market analysis of Luc Vaillancourt of Spatialytics, 2011,QuébecCity,Canada
15.05.2011
8 TableoffiguresFigure44‐Ilustrationofthedecisionmakingprocess...................................................................90Figure45‐CrystalReportsdashboard ..................................................................................................91Figure46‐Drilldownreport .....................................................................................................................91Figure47‐Well‐knownOSBI‐tools .........................................................................................................92Figure48‐IReportfromJasperSoft ........................................................................................................93Figure49‐BIRT‐pluginforEclipse ...................................................................................................... 103Figure50‐Visualizationofadatacube .............................................................................................. 104Figure51‐LogoSAP ................................................................................................................................... 104Figure52‐SpatialyticsconnectsBIwithgeo‐spatialcomponents......................................... 104Figure53‐BIRTMobileViewerIPadapp.......................................................................................... 105