Upload
mm47252
View
218
Download
0
Embed Size (px)
Citation preview
September911,2013Anaheim,California
WebIntelligenceReportValidationTipsandTechniques
MichaelWelter
2LearningPoints
First,becomefamiliarwithwhatisright Carefullyobservehowyourreportdivergencesfromright Categorizethedivergence Systematicallyfixtheproblem
3 Timesavedinvalidatingandfixingreports ConsumerconfidenceinreliableBIreports Properbusinessdecisionsbasedoncorrectanalysis Reducedwearandtearonreportdeveloper(s)
ReturnonInvestment
4 WebIntelligenceReportValidationTipsandTechniques HaveyoueverhadunexpectedresultswhendevelopingacomplexWeb
Intelligencereport?ThissessionisforWebIntelligencereportdesignersandITstaffwhosupportthem,lookingforsometimesavingpointersandfrustrationalleviatingtoolsfortroubleshootingandvalidatingreports.Seehowtoidentifyproblemsinthesourcedata,intheuniverse,andinthereportitself.Learntechniquesforverifyingaccuracy,andexplorethemysteriesofcalculationcontextsinWebIntelligence.Seethesemethodsinactionwithliveexamples.
BreakoutAbstract
51. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
Agenda
6Afancy,beautifulreportthatswrongisuseless(orworse)
WhyAreWeTalkingAboutValidation?
1. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
BusinessObjectsGivesYouthePowerofPerspective
Turnstoreddataintodecisions Decisionmakingisamaze
Wehaveagroundlevelperspective
Needanaerialperspective Businessdecisionsrequireaccess
toinformation Flexible Dependable Accurate
7
8 Queryingadatabaseisnotatrivialmatter SQLcanbecomplex Databasedesignsareoftencomplicated
BusinessObjectsWebIntelligenceisabiglever Draganddroptobuildcomplexqueries Navigatesthelabyrinthforyou
Knowstherulesofconnectingdatabasetables Knowswhentoseparatestatementsforyou
Youdonthavetounderstandwhathappensbehindthescenes Consistentlycorrectresultsamazinglyoften!
ABigJobThatLooksEasy
9 Stillplentyofroomforerror! Onedaytobuild;Twoweeksto
debug Errorscanbeelusive Oftentakesmuchlongertofindand
fixthanoriginaldevelopment Generallyfrustrating
Weneedawaytofindandfixtheseerrors Asystematicapproachtodetecting
problems Amapforwhentofixandwhento
gethelp Asetoftoolsforwhenitstimeto
divein
DetectandFixErrors
10
YouhavebasictointermediateWebIntelligenceskills YouareusingWebIntelligencetoreportfromdatathat
Isbelievedtobecorrect Hassomebenchmarks(perhapsapreviousreport)
You(oracolleague)knowthedataandthebusinessitrepresents Thedatabaseyouarereportingfromiseither
Thesameastheoriginalwork,or Adatawarehouse/martbasedontheoriginaldatabase(andisvalid)
YouhaveITsupport(orskills) Youhaveextract/transform/loadsupport(ifwarehouse) Youhaveuniversesupport(orskills) YouhavesomeideahowtoreadSQL
Assumptions
11
Whattargetareyoutryingtohit?
WhatsRight?
1. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
12
Definebenchmarksfirst Youllsavetimeifyouknowthe
answer Ifyoudontknowwhatcorrectis,
youwontknowwhenyouhitormissit
Thenanalyzeyourreportsdeviationfromthebenchmark
Thefollowingisastrategyfordefiningbenchmarks
StartWithWhatisCorrect
13
Collectbenchmarkdatapointsfrombusinessexperts Perhapsthepersonwhowasresponsibleforthepreviousversionofa
reportyourereplacing Someonewhoknowsthesubjectandthedata Someonewhoknowshowitwasdone
Collectdatapointsatthelowestgrain Mostspecific Example:Sales_Qty,Sales_AmtofProduct1059on8/1/2007atStore21by
Employee99 andatvariouslevelsofaggregation
Increasinglymoresummarized Acrossalldimensionsinvolvedinthereport Example:Sales_Qty,Sales_AmtofallProductson8/1/2007atStore21by
allemployees
CollectDataPointsFromBusinessExpert(s)
14
Whatelementswereused? Whatcalculationswereused? Howwastimerelated? Anyoutsidedatasources?
Isanydatasourcerequiredoutsidethedatabasefromwhichwerereporting?
Ifso,howwasthatdatarelatedtotherest? GettheSQL
Ifpossible,getacopyoftheSQLused Softcopyisbest
DiscoverHowTheseDataPointsWereProduced
15
Interview Theresnosubstitutefortalking Dothisinpersonifpossible
Observation Example1:reportscurrentlyproducedfromAccess
Notewhatdatabase,queries,macros(ifany)theyused GetacopyoftheSQLused Notetheprocessingflow:Whatstepsandinwhatorder? Noteanytemporarytablesused
Example2:reportscurrentlycomingfromERPsystem Notetheprocessingflow:Whatutilities/commands,inwhatorder? AskITtoexplainhowthatappgetsitsreport AskITforcopyofSQLtheappgeneratesifpossible
DiscoveryMethods
16
VerifyyouorITisabletoduplicateresultsofbenchmarkSQL Atalllevelsofaggregationprovidedinbenchmarkdatapoints Insamesystemasoriginalreportcamefrom IndatawarehousewithanalogousSQL
VerifyrequiredSQLispresentinuniverse Askuniversedesignerforhelphere Fragmentssimilartobenchmarkexamples,or Analogousfragmentsbuilttoworkagainstdatawarehouse
Duplicatebenchmarknumberswithsimplereports MakebarebonesWebIntelligencereport CheckSQLagainstbenchmarkSQL GetIT/designerhelpasnecessary
VerifyResults
17
Howdoesyourreportdeviatefromthebenchmarks?
WhatsWrong?
1. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
18
Canadianbanktrainingtechnique Plumblineexample Nowwecanstarttoseewhats
wrong Wehavesomeknowngooddata
points Themoreyouknowthesourcedata,
themoreintuitiveyoullbeatspottingdeviation
NowYouCanStarttoSeeWhatsWrong
19
DatabaseproblemsgotoIT Universeproblemsgotoyouruniversedesigner Asermon/wordofcautionregardingyourteam
Whenaproblemisinanothersareaofexpertise Itsoutofyourhands(relief) Whydidnttheycatchthatproblem(frustration) Bytheway,its,um,outofyourhands(morefrustration)
Youneedthewholeteam Dataproblemsarearealityinyoungdatawarehouses Avoidblame
Resentmentanddefensivenessdegradeteamworkquickly Youworkhalfspeedwithaswordinonehand
YourBusinessObjectsTeam
20
Thecureisofteneasyonceyouknowwhatswrong
Thetools Thefollowingstrategyhelpsfocus
yourefforts Thefollowingtechniquesgiveyou
leverage Theapproach
Grouppossibleissuesbysymptom Workthroughthepossibilities
TheTools
21
Reportorblockfilter Theresafilterinplacethatisntobvious Demo Toolowconditiononly
Queryconditionisntright Youmayhavetoomanyortoofewwherestatements Youmaybefilteringonthewrongelements
Wronguniversemeasure(unitsinsteadofdollars?) Measureisdefinedincorrectlyorimprecisely
Ex:bookedordersvs.revisedbookings Semanticproblemneedstobeaddressedinthebusinessandinthe
universe
NumberValuesConsistentlyTooHighorTooLow
22
Showfilterpane
Viewstructure
CheckFilters
23
Selectfiltersonmapstab
CheckFilters
24
Clickthroughreportelementstofindfilters...
Aha...foundone!
CheckFilters
25
Lookforunintendedconditions
CheckQueryConditions
26
Shouldhave$insteadofunits
WrongUniverseMeasure?
27
Checkforwhatcorrectis...
ObjectDefinedIncorrectly
28
ChecktheSQLyourobjectisgenerating
ObjectDefinedIncorrectly
29
Wrongaggregationtypeonauniversemeasure Unusual,butpossible Useobjectinbarebonesreport Checkthemathonaggregation
Useobjectintableblockfooter(noagg function) Comparetoobjectusedinfooterw/aggregatefunction
Designeraddressesinuniverse
NumberValuesConsistentlyTooHighorTooLow
30
Valueseemstoolow
Especiallythetotal
WrongAggregationType
31
IDthemiscreantobject
Makeasimpledocwithit
WrongAggregationType
32
Insertasumonyoursimpletable
WrongAggregationType
33
...butwithnoaggregatefunction
WrongAggregationType
34
Thatlooksmorelikeanaveragethanasum...
WrongAggregationType
35
Ah,thebareobjectis projectingasanaverage
Askuniversedesignertofixthemeasuresaggregationfunction
WrongAggregationType
36
Symptom:somevaluesaretoohigh,otherstoolow
Universeisconnectedtotestdata Youmayknowbydate,orsomeothertelltaledata Askuniversedesignertocheckthis
Wrongdatekeyinfacttablejoinedtodatetable Manydatawarehouseshaveadatelookup Facttablemayhavemanydates designerpickedone Example:Salesbydateshoulduseorderdate,notshipdate Whenviewbydate,somemonthsaretoohigh,otherstoolow ReviewSQLtoseewhatdateisbeingused
NumberValuesAreInconsistentlyWrong
37
Resultsaretoolowsomequarters,toohighothers
WrongDateKey
38
ExamineSQLstatement:itappearstouseashipdate
Askdesignertofixthedate
WrongDateKey
39
Symptom Somevaluestoohigh,othersmaybecorrect Whentoohigh,itsbyanevenmultipleofthecorrectvalue
ThissuggestsaSQL(fan/chasm)trapintheuniverse IDtheoffendingobject(s)bysystematicallypullingobjectsoutofthequery
untilthevaluesarecorrect Sharefindingswithyouruniversedesignersotheycanfix
SomeValuesTooHighbyEvenMultiples
40
Somevaluestoohigh Thosethataretoohigharesobyevenmultiples
($190,174*2=$380,348)
SomeValuesTooHighbyEvenMultiples
41
ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroomTown?
SomeValuesTooHighbyEvenMultiples
42
ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroomTown?
No.IsitShowroom?
SomeValuesTooHighbyEvenMultiples
43
ToIDaSQLtrap,startpullingobjectsoutofthequery IsitShowroom?
Nope.HowaboutSaleQty?
SomeValuesTooHighbyEvenMultiples
44
ToIDaSQLtrap,startpullingobjectsoutofthequery BINGO!
NowaddShowroomobjectsbackin
SomeValuesTooHighbyEvenMultiples
45
ToIDaSQLtrap,startpullingobjectsoutofthequery Revenuestilllooksgood
Nowgotellyouruniversedesigner: SalesRevenueistoohighforsomecustomers
Byevenmultiples WhenyouaddSalesQtytothequery
Yoususpectafanorchasmtraptheyllbeimpressed(orannoyed)
SomeValuesTooHighbyEvenMultiples
46
Example TY/LYsalesbylineiswrongforlastyear(buttotalisOK) WhenyoucompareTY/LYsalesbystate,everythingisfine
HistorybyCertainDimensionisWrong
47
Totalsareright,buttwolinesarewrongforlastyear
HistorybyCertainDimensionisWrong
48
Maybeslowlychangingdimensionindatawarehouse DiscusswithbusinessandIT MaybesomeproductshavechangedLinessincelastyear Businessmustdecidetoviewbycurrentorhistoricalalignment Solutionisindatawarehouse(ifextract/transform/loadpersonis
abletochangebehavior)
HistorybyCertainDimensionisWrong
49
Example Salesbyproductlacksnosaleproducts
MissingDimensionValuesWhereNothingHappened
50
Innerjoinusedinuniverse,butforgoodreason Weneedtoseeallcustomers,salesornosales Youcancheat:Demo
Forcereportsideouterjoin
MissingDimensionValuesWhereNothingHappened
51
Addquery:fulllistofdimensionvalues(listitemsonly)
MissingDimensionValuesWhereNothingHappened
52
Addtable:fullproductlist
MissingDimensionValuesWhereNothingHappened
53
Verifydimension(s)merged
MissingDimensionValuesWhereNothingHappened
54
Makemergedtable Usedimensionfrom2nd queryorthemergeddimension
MissingDimensionValuesWhereNothingHappened
55
1. Wrongcalculation(checkarguments)2. Objectsusedbyvariablerenderingwrongresults
Duplicatereportusingreferencedobjects Workthroughthetoolow/highsection
3. Calculationcontextisincorrect Example:calculatedcolumnvaluenotcomputedright Requiresspecificoutputcontext Demoofcalculationcontexts
CalculatedVariablesAreWrong
56
Well,THATsnotright
CalculatedVariablesAreWrong
57
Clickcell noticeitcontainsavariable
CalculatedVariablesAreWrong
58
Editvariable(HmmformulalooksOK)
Butletscheckhelpanyway
CalculatedVariablesAreWrong
59
Help>FormatDate>IndextoDate&NumberFormats
ArgumentshouldbeMM/dd/yyyy(notmm/dd/yyyy)
CalculatedVariablesAreWrong
60
Fixthevariablesformula
CalculatedVariablesAreWrong
61
Thatsbetter!
CalculatedVariablesAreWrong
62
Businessanalystsaysmargingoalcalculationistoolow
MargingoalcalculationlogiclooksOK
ObjectsUsedbyCalculationareWrong
63
Whatabouttheunderlyingobject? Margingoalsvariablesformulais=[Marginw]*1.1 Isthemarginwmeasuregivingustherightvalues? Letstestit:
ObjectsUsedbyCalculationareWrong
64
Ah:marginwisnotgivingcorrectresults:
Calculationiswrongbecauseunderlyingobjectiswrong Followpreviousstepstodiagnoseproblemwiththisobject
ObjectsUsedbyCalculationareWrong
65
Storesshouldbeclassifiedbasedonyearsopen Young,averageorold,relativetotheaverageforallstores Butallofourssayaverage somethingisamiss...
WrongCalculationContext
66
Storeclasscolumnisempty
WrongCalculationContext
67
Analerterpopulatesthecolumn
WrongCalculationContext
68
Thealerterslogicdependsonvariables
WrongCalculationContext
69
whichdependonothervariables
WrongCalculationContext
70
Letsseewhatsgoingonwith.Avg YearsOpen Duplicatethereport,andplaceanaverageonYearsOpen
Theaveragestorehasbeenopen14.6yearsatthistime
WrongCalculationContext
71
Letslookatour.AvgYearsOpenobjectinthetable:
.AvgYearsOpeniscalculatingtheaverageofeachrow
WrongCalculationContext
72
Whatwewantisforeachrowtohaveavalueof14.6 Theaverageofallrowsintheblock Justlikeourformulasittinginthefooteroftheblock
Bydefault,BusinessObjectsmeasures: Considerallthedimensionstheysee(intheirblockorsection) Aggregatetothelevelappropriatetowheretheyareplaced
Wewantour.AvgYearsOpenmeasureto: Consideralldimensionsforeachrow(default),but Aggregateasifitwereitplacedintheblocksfooter
WrongCalculationContext
73
ThisisCalculationContext: InputContext=thelistofdimensionsthecalculationconsiders OutputContext=theleveltowhichthecalculationaggregates Syntax:=AggregateFx([measure] input context) output context Inourcase,weonlywanttochangetheoutputcontext:
WrongCalculationContext
74
Now.AvgYearsOpenbehaveslikeafooterformula Butitdoessoforeachrow
WrongCalculationContext
75
Andnowthatourcontextforaverageiscorrect Ouralertercompareseachstoretotheaverageofallstores
WrongCalculationContext
76
Conclusion
1. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
77
Becomewellfamiliarwithwhatrightlookslike Carefullyobservehowyourreportdivergencesfromright Categorizethedivergence Worktheproblemsystematically
KeyLearnings
78
Indeterminingwhatisright",collectnotonlyexistingreportsandSQL,butalsoknownaccuratedatapointsatvariouslevelsofgranularity
Takethetroubletounderstandhowthebusinessworks VerifyresultsofsuppliedSQL Beoneofthoseplayersthatraisesyourteammates'game
BestPractices
79
Q&A
1. Whyarewetalkingaboutvalidation?2. WhatsRight?3. WhatsWrong?4. Conclusion5. Q&A
Thank you for participating.
Please provide feedback on this session by completing a short survey via the event
mobile application.
SESSION CODE: 1209
Learn more year-round at www.asug.com