View
273
Download
11
Category
Preview:
Citation preview
IBMDB2forz/OSDevelopment
MichaelDewert– dewert@de.ibm.com
DB211forz/OS:Migration,ExperiencesandTransition
Disclaimer:
Informationregardingpotentialfutureproductsisintendedtooutlineourgeneralproductdirectionanditshouldnotbereliedoninmakingapurchasingdecision.TheInformationmentionedregardingpotentialfutureproductsisnotacommitment,promise,orlegalobligationtodeliveranymaterial,codeorfunctionality.Informationaboutpotentialfutureproductsmaynotbeincorporatedintoanycontract.Thedevelopment,release,andtimingofanyfuturefeaturesorfunctionalitydescribedforourproductsremainsatoursolediscretion.
PerformanceDisclaimer:
Thisdocumentcontainsperformanceinformationbasedonmeasurementsdoneinacontrolledenvironment.Theactualthroughputorperformancethatanyuserwillexperiencewillvarydependinguponconsiderationssuchastheamountofmultiprogrammingintheuser’sjobstream,theI/Oconfiguration,thestorageconfiguration,andtheworkloadprocessed.Therefore,noassurancecanbegiventhatanindividualuserwillachievethroughputorperformanceimprovementsequivalenttothenumbersstatedhere.
2
Objectives• Sharelessonslearned,surprises,pitfalls
• Providehintsandtips
• Addresssomemyths
• Provideadditionalplanninginformation
• Provideusageguidelinesandpositioningonnewenhancements
• Helpcustomersmigrateasfastaspossible,butsafely
3
Agenda
• Introduction
• MigrationConsiderations
• APPLCOMPAT
• PlanManagement
• Utilities
• Availability
• DataSharing
• PerformanceandScalability
• DB2andREALSTORAGEmanagement
• EXTENDEDRBA/LRSN
• Summary
4
INTRODUCTION
5
DB211MajorThemes• Out-of-the-boxCPUSavings
– Improvingefficiency,reducingcosts,noapplicationchanges
– Upto10%forcomplexOLTP
– Upto10%forupdateintensivebatch
– Upto40%forqueries
– AdditionalperformanceimprovementsthroughuseofnewDB211features
• EnhancedResiliencyandContinuousAvailability– Improvedautonomicswhichreducescostsandimprovesavailability
– Makingmoreonlinechangeswithoutaffectingapplications
– OnlineREORGimprovements,lessdisruption
– DROPCOLUMN,onlinechangeofpartitionlimitkeys
– Extendedlogrecordaddressingcapacity- 1yottabyte(or1Bpetabytes)
– BIND/REBIND,DDL,OnlineREORGtobreakintopersistentthreads
• Enhancedbusinessanalytics– ExpandedSQL,XML,andanalyticscapabilities
– TemporalandSQLPLenhancements
– Hadoopintegration,NoSQLandJSONsupport
– Transparentarchiving
• Simpler,fasterDB2versionupgrades– Improvedproductquality/reliability– throughiterativeapproachon3monthlycycle(1:Nrallies)toFVT,SVT,Performancetesting,and
stabilizationphaseaheadofstartofESP
– ApplicationchangesdivorcedfromDB2systemupgrade(APPLCOMPAT)
– Accesspathstabilityimprovements
6
Announced Oct. 1, 2013
GA Oct. 25 2013
MIGRATIONCONSIDERATIONS
7
SoftwareRequirements• z/OSV1.13BaseServices(5694-A01)atminimum• DFSMSV1R13– theDB2Catalog isSMSmanaged• LanguageEnvironmentBaseServices• z/OSVersion1Release13SecurityServer(RACF)• IRLMVersion2Release3(ShippedwithDB211forz/OS)• IBMInfoSphere DataReplication(IIDR)10.2.1
– Noticethatthisisapre-req forentryintoV11ifrunningCDCand/orQ-Rep
• IBMDB2AnalyticsAcceleratorV4• IBMDB2ToolsCompatibility• DB2LUW/Connect/DataServerDriver/JDBC/ODBC
– Anyin-supportIBMInformationManagementsclients/drivers– V10.5FP2advisedfornewfeatures(seamlessupgrade9.7fp6,10.1fp2,10.5fp4)– SeetheProgramDirectoryformoredetails(exDB2forz/OSclienttoDB2LUWserver)– SeethefollowingslidesonDB2Connectinthissection
8
PrerequisiteSummary&Planning...• z/Architecture(z10andsubsequent64-bitz/ArchitectureProcessors)
• DataSharing:CFLEVEL12,with13or14recommended– GBPEnhancementsrequireCFLEVEL17orhigher
• 2-Gigabyteframesizeforbufferpools– RequireszEnterprise EC12processorandAPAROA40967forz/OSVersion1Release13
• Pageable 1-Megabytelargepages– RequireazEnterprise EC12processorwithFlashExpresscapabilityplusRSMEnablement
Offering(FMIDJDB778H)
• z/OSUnicodeServicesandappropriateconversiondefinitionsarerequired
• Checkthefollowingdocumentationformoreinformationaboutsoftwareandhardwarepre-requisites:
– TheDB211ProgramDirectory:http://publibfp.boulder.ibm.com/epubs/pdf/i1089450.pdf
– TheDB211AnnouncementLetter:http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=AN&subtype=CA&htmlfid=897/ENUS213-376&appname=USN
9
PrerequisiteSummary&Planning...• PID5615-DB2
– FMIDsHDBBB10,HIYBB10,HIZBB10,HIR2230,HDREB10• Check/correctincompatibilities
– RunacurrentV10DSNTIJPBpre-migrationjob• DeliveredinPM94057,currentlevelPI37038• SameasV11DSNTIJPM• Checkmaintenanceforcurrencyfixcat DB2MIGV11/K
– ReleaseIncompatibilitiesdocumentedin:• InstallationGuide• ApplicationProgrammingandSQLGuide
• Checkprogramminglanguagerequirements– Precompiler DSNHPC7includedinthebaseforolderCOBOLandPL/I
• RestrictsSQLtoDB2forz/OSV7level• SeetheProgramDirectory
• MigrationonlyallowedfromDB210forz/OSNFM– WithFallbackSPEPM31841withassociatedcoreqs &prerequisites– TheDSNHDECPparametermodulemustberecompiled– SeeInformationalAPARsII14660/II14745(Migration),II14730/II14732(Client&DDF)
• EstablishperformancebenchmarksatDB210NFM– SeethelatersectiononPerformanceandScalability
10
PrerequisiteSummary&Planning...• Inz/OSparametermemberIEASYSxx,configureaminimumof:
– 1TBofcontiguoussharedprivateperDB2– HVSHARE• z/OSdefaultis510TB
– 6GBofcontiguoussharedextendedprivateperDB2– HVCOMMON• SameasDB210• Default64GB
• Nosignificantstorageusagedifferences– On-demandBufferPoolallocationfromDB210wasremoved
• Thisprobablydoesnotrepresentastoragerequirement• VPSIZEisallocatedwhenBufferPoolisallocated
– Earlytestingindicates0– 5%additionalmemorydependentonqueryworkload• Consider10%-15%cushionforpotentialgrowthwithlesswell-behavedworkloads
– Extra37Kperopenindexpartasusedbyautonomicpseudodeletecleanup• SMSmanagedcataloganddirectory
– ThiswasarequirementforDB210• DataClassattributesof
– ExtendedFormat– ExtendedAddressability
– NewareasoftheCat/DirmovedunderSMSmanagementduringmigrationto11– AdditionalmigratedareasREORGed afterV10ENFMwouldalsohavebeenconverted– SDSNSAMPmemberDSNTIJSS isprovidedasasampleforconfiguration
11
DB211forz/OSRelatedNoChargeProduct• IBMDB2AccessorySuiteforz/OSV3.3(12/11/15)
– 5697-Q04- http://www-01.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_sm/4/872/ENUS5697-Q04/index.html&lang=en&request_locale=en
– IBMDB2AccessoriesSuiteforDB211feature• Spatialdata,UnicodeandInternationalizationsupportfor
– IBMTextSearchforDB2forz/OS–DataStudio4.1(4.1.2isnewest)
• EnhancedsupportforDB2asJSONDocumentstore• InstallableInformationCenterFramework
–Nolonger includedbutstillcurrentlysupported• SeeProgramDirectoryGI10-8957• DB2Adaptorforz/OSConnect
– IBMDB2AccessoriesSuiteforDB210feature• IncludeswhatisintheV11AccessoriesSuite• JSONSupportviaAPARII14727forDB210
12
PRE-MIGRATIONPLANNING
13
Pre-migrationSummary&Planning• Developconversionandcoexistencegoals
– HowdidyourV10testplanswork?– Reuseandimproveuponyourexperiences
• Migrationoccursinthreefamiliarphases– ConversionMode(CM)– EnableNewFunctionMode(ENFM)– NewFunctionMode(NFM)
• UsetheproactivePMRprocess– ThisistonotifyIBMDB2Supportofyourupcomingmigrationactivity
• ItistypicallyroutedtothesupportDutyManagerandbroadcasttothedutyteam– Ifproblemsoccurduringthemigration,openanewPriority1Severity1PMR
• REBIND/REGENERATEwhileinCM– UseREBINDPlanManagement(Package/BindStability)
• ConsiderFREEing OriginalPackagestoestablishanewDB210backup– Consider:
• REBIND...EXPLAIN(YES)...APREUSE(WARNorERROR)• REBIND...EXPLAIN(YES)...APCOMPARE(WARNorERROR)
14
Pre-migrationplanning
• RunDSNTIJPMorDSNTIJPB,toidentifythem– DSNTIJPMshipswithDB211andshouldberunonDB210toidentifypre-migrationcatalogclean-uprequirements• DSNTIJPMmayprovideDDLorutilitystatementsfortheclean-up
– DSNTIJPBisthesamejobandisshippedforDB210tomaximizepreparetime
15
Importantpreparation• OldplansandpackagesbeforeV9->REBIND• DonotrelyonAUTOBINDofinvalidplansandpackagesonentrytoV11CM.
– DealwiththeissuebeforeleavingV10NFM.
• Views,MQTs,andTablefunctionswithPeriodSpecification->DROP– ThosecreatedinV10arenotsupported
• DSNUTILSstoredproceduredeprecated• Synonymsaredeprecatedà but,stillsupportedinfuturereleasesasconsideredasa
incompatiblechange– ShouldbelookingatusingAliases
• Simpletablespacesà needtotracktheiruse,keepmovingtowardUTS• ASUTIMEmaybeapplieddifferentlyfordynamicstatementsofnestedroutinesinNFM
– SQLCODE-905returnedislimitfornestedroutineisexceeded– Previouslyhadtoexceedthetoplevelpackagelimit
• ALTERLimitKey– NowaPendingAlterONLYifitisTABLECONTROLLED
• PREVENT_ALTERTB_LIMITKEY
• ClientAccountingInfoSpecialRegisterExpansion
16
Itemsdeprecatedinearlierversions– Noweliminated• Passwordprotectionforactivelogandarchivelogdatasets
• DSNHCLISTNEWFUNvaluesofV8andV9– UseV10orV11
• SomeDB2suppliedroutines– SYSPROC.DSNAEXP–>UsetheEXPLAINPrivilegeandissueEXPLAINdirectly
– AMI-basedDB2MQ(DB2MQ)functions–>usetheMQI-basedfunctionsinschema(seeAPARPK37290forguidance)• DB2MQ1C.*, DB2MQ2C.*
• DB2MQ1N.*, DB2MQ2N.*
• CHARSETapplicationprogrammingdefaultvalue(KATAKANA)– useCCSIDs
• BINDPACKAGEoptionsENABLEandDISABLE(REMOTE)REMOTE(location-name,...,<luname>,...)-- specificnamescannotbespecified
• SysplexQueryParallelism– Singlememberparallelismisstillsupported
• DSN1CHKR– TherearenolongeranylinksintheCatalogorDirectory
17
MIGRATIONOVERVIEW:DB210->DB211
18
MigrationOverviewDB210->DB211
19
DB2 11 Enabling New Function Mode (ENFM)
DB2 11 Catalog
Data Sharing Coexistence
DB2 11 Conversion Mode (CM)
DB2 11 New Function Mode
(NFM)
DSNTIJTC(CATMAINT UPDATE)
DSNTIJNF(CATENFM
COMPLETE)
DSNTIJEN(CATENFM
START)
DB2 10 Catalog
DB2 11 Libraries
DB2 10 Libraries
DB2 10 New Function Mode
(NFM) With SPE
1 – 2 months
1 week
Minutes
Use APPLCOMPAT(V10R1) here
PreparingyourcurrentDB210NFMforMigrationtoDB211CM• ApplytheFallbackSPEAPAR,PM31841andanyprerequisitefixes
– YourDB210systemMUSTbeattheproperservicelevel
– SeeInfoAPARsII14660
• Non-DataSharing– CurrentDB210mustbestartedwiththeSPEapplied,ormigrationtoDB211willterminate
• DataSharing– BeforemigratingamembertoDB211,allotherstartedDB210membersmusthavethefallbackSPEapplied
– ThefallbackSPEmustbeonallactiveDB210groupmembersforDB211tostart
20
Important – Apply SPE to ALL Data Sharing Members Before Starting Migration!
Otherrecommendations• RunOnlineREORGsagainstCatalogandDirectoryobjectspriortotheENFM/NFMmigration
– CheckthatREORGcanbreakin
– CheckdataconsistencyofCatalogandDirectory
– ImprovetheperformanceoftheENFMprocess
• CATMAINTandENFMwillnotexecuteifentriesfoundinSYSUTILX– DB2willnolongerblindlyre-initializeit
21
MigrationPreparations• RunhealthchecksonDB2CatalogandDirectorytoflushoutanylatentinconsistencyandcleanupbeforethemigration
• QueriesfrommigrationjobDSNTESQ- shouldalwaysreturnzerorows
• REPAIRDBDTESTorDIAGNOSE
• COPY,orDSN1COPYwithCHECKoption
• BasicRUNSTATSonallobjects
• ForobjectswithLOBcolumns:– CHECKLOB
– CHECKINDEXontheAUXindex
– CHECKDATAonthebasetablespaceusingSCOPEAUXONLYAUXERRORREPORT
• CHECKINDEXonallindexes
• CHECKDATAforDSNDB01.SYSUTILX,DSNDB06.SYSPKAGE,DSNDB06.SYSSTATS,DSNDB06.SYSSTR
22
APPLCOMPAT-APPLICATIONCOMPATIBILITY
23
APPLCOMPAT– ApplicationCompatibility• Requirements
– De-coupletheneedforapplication programchangestodealwithincompatibleSQLDMLandXMLchangesfromtheactualDB2systemmigrationtothenewDB2releasewhichintroducedtheincompatibleSQLDMLandXMLchanges
– ProvideamechanismtoidentifyapplicationprogramsaffectedbyincompatibleSQLDMLandXMLchanges
– Provideamechanismtointroducechangesatanindividualapplicationprogram(package)level• Enablesupportsothatapplicationprogramchangescanbephasedinovermuchlongertime
• EnablesupportformixedDB2releaseco-existenceindatasharing
• EnablesupportforuptotwobacklevelreleasesofDB2(N-2)
• Solution– APPLCOMPATwhichseparatesDB2systemmigrationtothenewDB2releasefromapplicationprogrammigrationtodealwithincompatibleSQLDMLandXMLintroducedbythenewrelease
24
The image part
APPLCOMPAT– ApplicationCompatibility...• APPLCOMPATZPARMprovidesdefaultforBIND/REBIND
– V10R1forDB210SQLDMLbehaviour
– V11R1forDB211SQLDMLbehaviour
– DefaultisV11R1fornewinstalls,V10R1formigration
• APPLCOMPAToptiononBIND/REBINDtooverrideZPARMdefault
• CURRENTAPPLICATIONCOMPATIBILITYspecialregisterandDSN_PROFILE_ATTRIBUTESforDDF
– FordynamicSQL
– DynamicJCCpackages(i.e.SYSNLxxx)needtobereboundtoinitiatespecialregister
• DoesnotaddressissueswithnewreservedwordsorotherincompatibilitiesthatcouldonlyberesolvedbyhavingmultiplelevelsoftheDB2parser
• BIF_COMPATIBILITYZPARMisindependentofAPPLCOMPAT
• NewSQLfunctionalityavailableinV11NFMcannotbeuseduntilpackageisboundwithAPPLCOMPATvalueofV11R1
– RememberifapackagewasboundinCMithas‘V10R1’isSYSPACKAGEandyoumustconsciouslyrebinditwith‘V11R1’tousenewfunctionality
25
The image part
APPLCOMPAT– ApplicationCompatibility...• MigrationautomaticallysetsV10R1priortoNFM…otherwise
– DSNT225I-DSNBINDERRORFORPACKAGElocation.collid.memberAPPLCOMPAT(V11R1)OPTIONISNOTSUPPORTED
– IFCID376– SummaryofV10functionusage• IT is recommended to use use IFCID 376 under V11 CM in Development/Test and
Production (include PI61920, PI21454, OA49474)
– WeexpectchangesnecessarytoavoidV10R1usagetohappenafterreachingNFM
• WorkaroundtodistinguishpackageswhichhavetoabsolutelyrunasV10R1untiltheyarecorrected
– AnnotatethepackageusingSQLCOMMENTONPACKAGEcolid.name.”version”IS‘V10R1’• Ifversionisapre-compilertimestampthenthedoublequotesarenecessary
– StoredintheREMARKScolumninSYSIBM.SYSPACKAGEtable• Canbequeriedandbeexploitedbyhousekeeping
26
APPLCOMPATvs.BIF_COMPATIBILITY• BIF_COMPATIBILITY=V9|V9_DECIMAL_VARCHARisstillhonoredinallmodesofV11
– The‘undocumented’timestampsupportisbackagainwithV11CMonlywithAPPLCOMPAT(V10R1)e.g.,• EURdateformatconcatenatedtotheTIME(andmicroseconds)
• AlsoDDF_COMPATABILITYZPARM– IDNTFY_V11_PRIOR_VER
• DB211CMmemberisidentifiedasDB210NFMmember
27
PLANMANAGEMENT
28
AccessPathStabilityImprovements(APSI)...• PlanManagementPolicies(PLANMGMT)
– DB29deliveredAccessPath/BINDStability(PLANMGMT)viaREBIND• PLANMGMTRebindkeywordorSystemParameter
–BASIC:Keeps2copiesofapackage(CurrentandPrevious)
–EXTENDED:Keeps3copies(Current,Previous,andOriginal)• UsetheProfiletablestosetupautomaticPlanManagementPolices
– SystemParameteroptionsarethesameasV9withdefaultchangeinCM• OFF(wasV9default)• BASIC• EXTENDED(V10default)
– CopiesarekeptbydefaultinDB210
29
APSI/PlanManagementPolicies...• PLANMGMT
– DB210evolvesthiscapability• MoreformalCatalogimplementationforPLANMGMT(V9delivery)
–SYSPACKCOPYispopulatedaspartoftheENFMprocess» COPYIDcolumn
» 1:IndicatesaPreviouscopy» 2:IndicatesanOriginalcopy
• OldcopiesthatarefrombeforeV6willstillbemarkedinvalid
– REBINDchanges•APRETAINDUPforPLANMGMT(BASICorEXTENDED)
–APRETAINDUPkeyworddefaultstoYES–NO:AnewaccesspathidenticaltoaCurrentaccesspathdoesnotgetcopiedtoaPreviousorOriginal
30
APSI/PlanManagementPolicies• PLANMGMTSCOPE
– SystemParameterandProfileAttribute
– Fornow,theonlyoption(anddefault)isSTATIC(V9behavior)
• PLANMGMTextendedtoNativeSQLProcedures
• EXPLAINPACKAGEcommand
– ToretrievePLAN_TABLErecordsforexistingpackage
•Originalbind/rebindmayhavebeenEXPLAIN(NO)
•MusthavebeenboundonV9orV10towork,needExplainDataBlock
•PopulateexplaintableswithoutBINDorREBIND.
31
>>-EXPLAIN----PACKAGE----------->
>>-----COLLECTION--collection-name--PACKAGE--package-name--------->
>----+--------------------------+----+-------------------+-------->| | | |+---VERSION-version-name---+ +---COPY--copy-id---+
AccessPathStabilityImprovements...• APCOMPARE/APREUSE
– OptionstostabilizeaccesspathsacrossBINDorREBIND
– ThereplacedpackagemustbefromDB29orlater
– APCOMPARE• StructuralcomparisonoftheaccesspathacrosstheBIND/REBIND
• Whendifferencesarediscovered(forexistingstatements),theoutcomedependsontheoptionused
– WARN:PackageiscreatedwithanRC=04
– ERROR:NopackageiscreatedwithanRC=08
– APREUSE• Hintsallstatementsforthenewpackageusingtheaccesspathfromthepackagetobereplaced
• Whendifferencesarediscovered(forexistingstatements):– WARN:PackageiscreatedwithanRC=04,somestatementshavenewaccesspath
– ERROR:NopackageiscreatedwithanRC=08,BIND/REBINDfails
32
AccessPathStabilityImprovements• APCOMPARE/APREUSE
– Comparisonsaredoneatthestatementlevel
– FeedbackisplacedinthePLAN_TABLEifEXPLAINYES orONLY specified
• HINT_USEDcontains‘APREUSE’whenhintsucceeds
• REMARKSusedforreuseorcomparisonfailureinformation
• Theseareplacedonthe‘new’PLAN_TABLErows
• Ifan‘old’PLAN_TABLErowisunmatchedinthe‘new’setofrows– AnewrowiscreatedtoshowtheREMARKS
– Thesearenot“sticky”;mustspecifyforeachBIND/REBIND
– AUTOBINDdefaultstoNO
– HINTcanfailduetocolumnorderofjoins,merging,correlation/decorrelation
33
AccessPathStabilityImprovements• APCOMPARE
– DSNT285Ilists
• #ofstatementssuccessfullycompared
• #ofstatementswithunsuccessfulcomparison
• #ofstatementsthatcouldnotbecompared• APREUSE(NO/NONE,WARN,ERROR)
– InvokesAPCOMPARE– AppliesahintfromtheEDBforthenewpackage– CangetthesameerrorsaswhenusingOPTHINTs
• +395/ReasonCode• ReasonCode50addedforcomparisonfailure
• MigrationandAPCOMPARE/APREUSE– ConsiderusingREBIND...EXPLAIN(YES)...APREUSE(ERROR)
– ThenanalyzetheREBINDfailuresoruseAPREUSE(WARN)
34
APREUSE• DB211providesAPREUSE(WARN)
– Attemptstohintthesameaccesspathformatchedstatements– Whentheaccesspathcannotbemaintained,anewoneiscalculatedforthosespecificmatchingstatements
– Effectivelyoperatesatthestatementlevel– ThesameconsiderationsfromDB210carryforward
• UsestheExplainDataBlock,asdoesAPREUSE(ERROR)andAPCOMPARE(WARN/ERROR)• SomePLAN_TABLEcolumnsarenothint-able(MATCHCOLS)
– APREUSE(WARN)willallowtheBIND/REBINDtocontinue
• DB210APREUSE(ERROR)...EXPLAIN(ONLY)mayrepresentinvalidplan– InDB211,thiswillberolledbacked
35
APPLCOMPAT-ApplicationCompatibilityandPlanManagementSummary• DifferentiateAPPLCOMPAT-ApplicationCompatibilityandPlanManagement• UnderstandAPPLCOMPATwhichseparatesDB2systemmigrationfromapplicationprogrammigration
– MonitorandprepareforincompatiblesusingIFCIDTRACINGandsettherightBINDoption» UseIFCID376underV11CMinDevelopment/TestandProduction.
• APPLCOMPATisastickyoption.– WillneedtouseitexplicitlyonBINDREPLACEand REBINDofexistingpackage
togettoV11R1.• UseAccessPlanStabilityImprovementsasoptionstostabilizeaccesspathsacrossBIND
orREBIND– ConsiderDB2V11withAPCOMPARE/APREUS– ThereplacedpackagemustbefromDB29orlater– ConsiderbindingpackagelikeSPUFI,TEP2,etc withAPPLCOMPAT(V11R1)onentrytoV11
NFM.
• DonotrelyonAUTOBINDofinvalidplansandpackagesonentrytoV11CM.– DealwiththeissuebeforeleavingV10NFMusingPLANMANAGEMENT.
36
UTILITIES
37
REORGEnhancements
• SWITCHphaseETreductionof91%measuredwhenreorging 20parts
– ReductionattributedtointernalLOG/SWITCHphaseprocessingchanges
• NewSWITCHTIMEoptionavoidstheneedformultiplejobstocontrolthestartofthedrain
– PriortoDB211hadtocontrolSWITCHphasewithMAXRODEFERandaseparatejobtoissue–ALTERUTILITY…MAXRO(n)
– NewSWITCHTIMEparameterusedtodetermineearliestpointatwhichdrainprocessingwillbeattempted• CanalsosetNEWMAXRO-integerforsecondstowaitafterSWITCHTIME
38
REORGEnhancements…• Fasterdrainacquisitionforpart-levelREORGinCM
• Preventnewclaimsonalltargetpartitionswhilewaitingfordrains– IfREORGPART1-10,priortobeginningdrainonpart1(outof10),willsetflagtopreventclaimsonanyofthe10partsinvolvedinREORG.
– Willthendrainpart1,part2andsoontillall10partsdrained
• UseofDRAIN_ALLPARTSYESoption(notdefault)hasthepotentialtosignificantlyreducethe‘outage’
– AvoiddeadlocksbetweendrainsandclaimsacrossNPIsandpartitionswhenreorganizingsubsetofpartitions
– TemporarilydrainentiretablespaceuntilwegetdrainonNPI,thenreleasealltablespacepartsexceptthoseinvolvedinREORG
– MorelikelytobesuccessfulingettingsuccessfulDRAINtomaketheSWITCH
– BigreductionsseenintheelapsedtimetocompleteDRAINandSWITCH
– REORGsshouldrunwithfewerproblemsusingthisfeature
• REORGmessageoutput- DSNU1138Iprovidesdrainbegin/endinformation
39
REORGEnhancements…• Part-levelinlineimagecopywhenreorganizingasubsetofpartitions
– Tapesupportadded,butnosupportyetforSTACKYES
– Changesrequiredtoexistingjobs• Unlesspartition-levelinlineimagecopiesusingTEMPLATEwith&PAor&PART
• REORGSORTDATANORECLUSTERYES|NO– RECLUSTERNOwillbypasssort(andspeedupconversiontoextendedformat)
– RECLUSTERNOisenforcedforSHRLEVELCHANGEwithSORTDATANO• SpecifySORTDATAtogetreclustering (unloadedthroughclusteringindex)
– DSNU2904IDATARECORDSWILLBEUNLOADEDVIAunload-method• CLUSTERINGINDEX
• TABLESCAN
• TABLESPACESCAN
40
REORGEnhancements…• Automatedbuildingofthemappingtablewithnew10byteLRBA/LRSN
– V11CMbehavior• AnexistingmappingtableinV10(6-byteLRBA/LRSN)orV11(10-byteLRBA/LRSN)formatisreused
• Ifthemappingtabledoesnotexist,amappingtableisautomaticallytemporarilycreated
– V11NFMbehavior• AnexistingmappingtableinV11formatisreused
• IfamappingtableexistsbutisinV10format,aV11formatmappingtableisautomaticallycreatedinthesamedatabaseastheoriginalmappingtable
• Ifmappingtabledoesnotexist,mappingtableisautomaticallycreatedindatabasespecifiedbyZPARMREORG_MAPPING_DATABASE,orindeclareddatabaseorinDSNDB04
– AutomaticallycreatedmappingtablesaredroppedattheendofREORGortheendofthelastREORGiftherearemultipleREORGsinthejobstep
– Recommendations• PredefineandkeepmappingtablestoavoidSQLDDLcontentionontheCatalog
• UseasinglespecificdatabaseasspecifiedinZPARMforallmappingtables
• ModifyschemaofexistingmappingtablestoV11formataspartofmigrationprocesstoNFMi.e.,ALTERTABLETBMAPALTERCOLUMNLRSNSETDATATYPECHAR(10)NOTNULL;
41
REORGEnhancements…• PARALLEL(maximumnumberofsubtasks)optiontocontrolthenumberofsubtasksusuallybasedoneitherpartitionsorparallelindexbuild
– OverridesZPARMPARAMDEG_UTIL
• REORGLISToptionLISTPARTSn(NFM)– limit#ofpartitionstobeprocessedinasingleREORG
• Ifinputisapart-levelLISTDEF
– OverridesZPARMREORG_LIST_PROCESSING– DefaultvaluecorrespondstosettingofZPARMREORG_LIST_PROCESSING
• PARALLEL=Allpartsreorganizedinoneexecution- SameasLISTPARTS[unlimitedor#ofpartitions]
–DefaultZPARMsetting• SERIAL=Partsreorganizedoneatatime- SameasLISTPARTS1
– TakestheplaceofPARALLELYES/NOonREORGLIST
• Beawareofchangeddefaultse.g.,NOPADYESforREORGDISCARD• LOGRANGESNOoptionshouldonlybeusedwhenSYSLGRNXisknowntobelogicallycorruptedand/orhastobereset
42
REORGEnhancements…• REORGREBALANCE
– NowsupportsSHRLEVEL(CHANGE)– bigstepforwardfor24*7
– Cannowdealwithpartitionsthatwereempty(ordidnotcontainenoughdataforacompressiondictionarytobebuiltduringtheUNLOADphase)beforetheREORG• Willnowbuildasinglecompressiondictionarythatwillgetappliedtoalltargetpartitions
• ThereisnolongeraneedforsubsequentREORGtogaincompression
– GoodchanceofbreakinginonpersistentthreadsrunningRELEASE(DEALLOCATE)packages
• PartitionpruningforUTSPBGtablespaces– OptiontophysicallyremoveandcontractthenumberofUTSPBGpartitions
– OnlyperformedwhenZPARMREORG_DROP_PBG_PARTS=ENABLE
– Disabledbydefault
– ThereisnosupportforPITrecoverytoapointintimepriortoSWITCHphaseforaprunedtablespace
43
ReservedSpaceforUpdate
• Indirectreferencesarecausedbyupdatestoatablewhentherownolongerfitsontheoriginalpage…sothereisapointertowhereitnowresides
• Impactcausedbyindirectreferences(overflowrecords)– Additionalgetpages,potentiallyadditionalI/Os totheoverflowpages
– DegradedclusteringratioàMorefrequentREORGTablespace
– InDB210synclogwritesforGBPdependentpagesetswithoverflowrows
44
SG24-8222 – V11 Performance Topics
• DB211:– INSERTwillreservespaceforupdaters
– NewDDLkeywordforCREATETABLESPACEonPCTFREEclausetospecify%offreespacetoleaveineachdatapagebyINSERT,LOADorREORG
• FORUPDATEn|AUTO
– n isaspecifiedpercentage0to99
– AUTOusesRTSvaluestocalculatethe%foryou
– NewZParm:PCTFREE_UPD• SystemdefaultforFORUPDATEvaluewhenitisnotspecified
• Ifnotspecified,samebehaviorasDB210
REORGofLOBdata• SupportREORGofLOBdataeventhoughauxindexisunavailable
– InV10ifLOBtablespaceisREORPandindexisRBDPLOBscan’tbeREORG’d andindexcan’tberebuilt
• REORGperformanceimprovementforLOBtablespaceofupto70%elapsedand40%CPUreduction
– http://www-01.ibm.com/support/docview.wss?uid=swg1PI17945
• REORGSHRLEVELNONEforLOBschangedtoRC=8from11CMonwards– NotsupportedinV10NFM,butreturnsRC=0withMSGDSNU126I
45
CompressionDictionaries• AvoiddecompressionfailuresforIFCID306readerswhennewcompressiondictionarybuiltbyREORG/LOAD
– Oldcompressiondictionarystoredonlog• NewSYSCOPYrecordpointstooldcompressiondictionaryforCDCtables
– IndicatedinSYSIBM.SYSCOPYcolumnICTYPE=J• IFI306readautomaticallyretrievesoldcompressiondictionaryifnecessary• Avoidneedforreplicationtargetrefreshwhendictionarychanges
46
RUNSTATSandRTSEnhancements• InlineStatisticsareestimatesandshouldnotbecomparedwithstandaloneRUNSTATS• NowpossibletoavoidDSNU602ISTATISTICSARENOTCOLLECTEDFORNONPARTITIONED
INDEXonREORGPARToperation*– REORGsortsallofthenon-partitionedindexkeysbecausetheamountofdatathatwasbeing
reorganizedrelativetothesizeofobjectsexceededinternalthresholds– WhenSORTNPSIoptiononREORGjoborREORG_PART_SORT_NPSIZPARMsettoYESorAUTO:AUTO
specifiesthatifsortingallkeysofthenon-partitionedsecondaryindexesimprovestheelapsedtimeandCPUperformance,allkeysaresorted
• NewRESETACCESSPATHoption– Resetmissingand/orconflictingaccesspathstatisticsintheCatalog– DoesnotaffectspacestatisticsintheCatalogorRTS
• AvoidDSNU1363ITHESTATSPROFILEFORTABLEtable-name NOTFOUND– Willusefixeddefaults
• NosupportforUSEPROFILEwithinlinestatisticsinREORGandLOAD• CanadditionallyExternalizeRTSin-memoryblocksviathefollowingcommand
– –ACCESSDATABASE(DB)SP(TS)MODE(STATS)command– ConsiderexecutingjustbeforerunninghousekeepingwhichwilluseRTStomakedecisions.
47
StatisticsandzIIP Enhancements…• MorezIIP offloadforRUNSTATSdistributionstatistics
– Columngroupdistributionstatisticscollection– RUNSTATSutilityrunsunderanenclaveSRBandisthereforezIIP eligible– Upto80%zIIP-eligible
• EnhanceinlinestatisticsforRUNSTATSavoidance– Inlinehistogramstatistics– InlineDSTATS
• AddedCOLGROUPtoinlinestats• Uptoadditional30%offloadtozIIP (comparedtoV10)
• zIIP offloadforLOADREPLACEPARTclearingofNPIs– 100%forLOADREPLACEDUMMYinput
48
RECOVERenhancements• FastLogApply(FLA)nowimplementedforRECOVERINDEX
– PreviouslyDB2wouldwaituntilalogrecordwastobeappliedbeforereadingtheassociatedindexpageintothelocalbufferpoolwhereitwouldthenbecached
– NowDB2willuselistprefetchtoreadalltheindexpagesthatareneededtoapplylogrecordsfor,beforeapplyinganylogrecord
– Potentialforsignificantsavingsinelapsedtime
– Shouldnowreconsiderdecision:runRECOVERINDEXinparallelwithRECOVERTABLESPACE[PART]vs.waitforRECOVERTABLESPACE[PARTs]tocompleteandthenrunREBUILDINDEX
– EnhancementtakenbacktoV9andV10viaAPARPI07694
• Optimizationtopoint-in-timeRECOVERlistofobjects– RecoverobjectsonlywhennecessarywhenperformingPITrecoverywhenTOLOGPOINTorTORBAarespecified
– Itdoesnotapplytologonlyrecoveries,RECOVERBACKOUT,andrecoverstocurrent
– DIAGNOSETYPE(607)isrequiredtoactivatethisbehavior
49
RECOVEREnhancements• RecovertoPITfortablespacewithmaterializedpendingalters
– NowsupportedformostALTERs• ForPBRtablespace:SEGSIZE,DSSIZE,BPID,MEMBERCLUSTER
– MustissueaRECOVERthensubsequentREORG• ObjectinhardREORPafterrecover,dataisrecoveredbutnotDDL
• WhenyoudroppendingALTERsinV11AREORstatusremoved
50
time
1> PendingALTER issued
Alter is deferred
3> RECOVER to PIT start
2> REORGmaterialize changes
4> REORG SHRLEVEL
REFERENCE
Backup&RecoveryEnhancements• Fastercatalog/directoryrecovery
– DB211EnablesSYSLGRNXrecordingfortheseobjects(inCM)• DSNDB01.SCT02• DSNDB01.SPT01• DSNDB01.SYSSPUXA• DSNDB01.SYSSPUXB• Indexesonthesetablespaces
– RECOVERutilitywillnottakeadvantageoftheSYSLGRNXrecordsuntilNFM• NewVCATnametranslationonRESTORESYSTEMforsystemcloning
– DB211enhancedtheRESTORESYSTEMutilitytoacceptoldandnewVCATaliases,andtoswitchtothenewVCATaliasesduringlogapplyprocessing• SupportlogapplywhenRESTORESYSTEMusedforcloningpurposes
• ImprovedrecoverabilitywithCOPY-REORGconcurrency– AllowCOPYSHRLEVELCHANGEtorunatthesametimeasREORGSHRLEVELCHANGE,untilREORGisabletodraintheclaimers
51
Partition-levelinlineimagecopy• Fasterpartition-levelrecoveryfrominlineimagecopy
• Createpartition-levelinlineimagecopiesifusingTEMPLATEwith&PAor&PART– Separateoutputdatasetcreatedforeachpartition
• PreV11usedsingleoutputdatasetforALLpartitions
– NonewoptionorkeywordneededonREORG
• RECOVERofsinglepartitionofa20partitiontablespace– ETreducedby28%
– CPUreducedby49%
52
EnhancementsintheMaintenanceStream• REORGLOGphaseperformanceimprovement
– APARPI32491
• SupportLOADSHRLEVELCHANGEPARALLELforPBGpagesets inV11– APARPI44674
• LOADutilityIGNORE(WHEN)option– AvoiddiscardprocessingforrecordsthatdonotsatisfyWHENclause
• APARsPI44415&PI44465
• AvoidrepeatedtapemountstoretrievecompressiondictionaryforIFI306reads– APARPI54885
• Reducedindexpagep-lockingindatasharingforREORGINDEX,REORGTABLESPACEpartlevel withNPSIs&RUNSTATSINDEX
– Example:RUNSTATSETcutby70%,CPUby25%,indexleafpagep-lockscutfrom2.6mtozero• APARPI46320
53
UtilitiesSummary• EnsurethatRUNSTATSaspartofREORGwasatleastcollectedunderV9
– IfnottheDataRepeatFactorwillbeoutlinedDRF=-1!Queryoptimisation!• ManyREORGEnhancements,e.g.
– NewSWITCHTIMEoption– Part-levelinlineimagecopywhenreorganizingasubsetofpartitions– REORGSORTDATANORECLUSTERYES|NO– Automatedbuildingofthemappingtablewithnew10byteLRBA/LRSN– AbilityofusingDRAIN_ALLPARTSYESoption– REORGREBALANCEsupportsSHRLEVEL(CHANGE)– SupportREORGofLOBdataeventhoughauxindexisunavailable– REORGofLOBbigperformanceimprovement
• ConsiderRUNSTATSandRTSEnhancements• LookforStatisticsandzIIP offload• FastLogApply(FLA)nowimplementedforRECOVERINDEX• PendingaltersandRecovertoPITenhancements• Fastercatalog/directoryrecoveryandFasterpartition-levelrecoveryfrominlineimagecopy
54
AVAILABILITY
55
BIND/REBIND/DDL/OnlineREORGbreakingintopersistentthreadrunningpackagesboundwithRELEASE(DEALLOCATE)
• PersistentthreadswithRELEASE(DEALLOCATE)whichwerepreviousblocking– e.g.,IMSPseudoWFI,CICSProtectedENTRYthreads,etc
• TypesofREORGswhichinvalidatedpackageswerepreviouslyblocked– REORGREBALANCE
– MaterializingREORG
• The'break-in'behaviorisONbydefault(ZPARMPKGREL_COMMIT=YES)
• Break-inisperformedona“bestefforts”basis
• Break-inmechanismcanhandleidlingthreadsatatransactionboundary(i.e.,wherecommitorabortisthelastthingperformed)
56
BIND/REBIND/DDL/OnlineREORGbreakingintopersistentthreadrunningpackagesboundwithRELEASE(DEALLOCATE)…
• Severalfactorscomeintoplayforasuccessfulbreak-in– PersistentthreadmustCOMMIT
– ThetimingoftheCOMMITandthefrequencyoftheCOMMITsarebothkey
– IncreasingtheZPARMforIRLMresourcetimeout(IRLMRWT)helpstokeeptheBIND/REBIND/DDL/OnlineREORGoperationwaitingtoincreasethechancesofasuccessfulbreak-in
• Thebreak-inmechanismdoesnotapplywhen– RunningpackagesboundKEEPDYNAMIC(YES),or
– OPENcursorsdefinedWITHHOLDatthetimeofCOMMIT,or
– IftheCOMMIThappensinsideastoredprocedure
• RELEASE(COMMIT)wouldalsonotbreak-infortheaboveconditions
57
BIND/REBIND/DDL/OnlineREORGbreakin- Howdoesitwork1. BIND/REBIND/DDL/OnlineREORGisinitiatedandwaitsonapackagelock
– Willtimeoutafter3xIRLMtimeoutlimit(IRLMRWT)
2. At1/2oftheIRLMtimeoutlimit,DB2willgetnotifiedbyIRLMthatsomeoneisstuckonapackagelock– IfDB2hasanS-holder,DB2willpostasystemtasktotakefurtheraction
3. DB2systemtaskisawakenedandcheckstoseeifa‘recycle’oflocallyattachedthreadshasbeendoneinthelast10seconds– Ifnot,thebreak-inoperationwillproceed– DB2istryingtoavoidabatteringofthesystemviaBIND/REBIND/DDL/OnlineREORG
4. SendbroadcasttoallDB2memberstoperformarecycleoflocallyattachedthreads5. Iftaskproceeds,itwillloopthroughalllocallyattachedthreads(notDIST!)andseeif
theywerelastcommitted/abortedin>1/2oftheIRLMtimeoutlimit– Ifso,theBIND/REBIND/DDL/OnlineREORGislikelywaitingonthem
6. ThenexttestistoseeifDB2candoanythingaboutit?– Eachthreadmustbeatatransactionboundary(i.e.,commitorabortisthelastthing)– Ifso,DB2canprocessthethread
58
BIND/REBIND/DDL/OnlineREORGbreakin- Howdoesitwork…7. DB2willfencetheAPIforthethread,grabtheagentstructureanddrivea‘dummy
COMMIT’– ThecommitistransactionallyOKsinceweareatatransactionboundary– DB2willbethecoordinatorasthisissingle-phasecommitandgetout– OntheCOMMIT,RDSseesthatthereisawaiterforapackagelockheldbythisagentandwill
switchtoRELEASE(COMMIT)forthiscommitcycle– ThelockisfreedandDB2isonestepclosertotheBIND/REBIND/DDL/OnlineREORGbreaking
in
8. Repeatforallqualifyingthreads9. BIND/REBIND/DDL/OnlineREORGshouldbreak-inprovidedtherearenoblockersthat
hadtobeexcludede.g.,longrunningreadonlyapplicationprocesswithoutacommit10. Iftheapplicationstartsusingthethreadduringtherecycleprocessing,itwillbeblocked
attheAPIlevel– DB2willspinthethreadinatimedwaitloopuntiltherecycleisdone– DB2willwaitamillisecondapproximatelybetweenpolls– DB2hasalsotakencaretofenceend-of-task(cancelapplicationTCB),end-of-memory(force
downthehomeASIDduringrecycle),associate,dissociate,etc
59
BINDBreak-in– Simplecustomertest
60
Time-----------à
Thread: T1 Treadreuse
BIND: BIND
1 transaction
One CICS ENTRY Thread
Break in
BIND waits approximately 30 sec before it breaks into an idle thread
30 sec is half the transaction time out interval
BINDbreakin– additionalcustomertesting
61
Action Threads Result
BIND BatchNoCommit Nobreak-in
BIND BatchfrequentCommit Break-in
BIND 50*CICS ENTRY Break-in
DDL:
CreateIndex CICS ENTRY Break-in
DropIndex CICS ENTRY Break-in
AlterTableAddColumn CICS ENTRY Break-in
Alterindex(NO)cluster CICS ENTRY Break-in
AlterTablespacetoUTS CICS ENTRY Break-in
AlterPartition CICS ENTRY Break-in
ALTERLIMITKEYenhancement• Behaviorisdifferentdependingonhowthetablepartitioningiscontrolled• Withtable-controlledtablepartitioning,thisisapendingalteration
– Droppingofthesealterscanoccuratanytime
• Withindex-controlledtablepartitioning– IfalterisdoneviaALTERINDEXALTERPARTITION
• Partitiongoesinto‘hard’reorgpending(REORP)!• Tablespaceremainsindex-controlled• Altercannotbewithdrawn!
– IfthealterisdonebyALTERTABLEALTERPARTITION• Ifthepartitionisnotempty
–Partitiongoesinto‘hard’reorgpending(REORP)!–Tablespaceisconvertedtotable-controlledpartitioning!–Altercannotbewithdrawn
• Ifthepartitionisempty
–Alterisexecutedimmediately–Tablespaceisconvertedtotable-controlledpartitioning
62
ALTERLIMITKEYenhancement…• TwonewZPARMsintroduced
– PREVENT_ALTERTB_LIMITKEY• ALTERINDEXALTERPARTITIONisstillpossible– donotuseitbecauseofREORP
• ALTERTABLEALTERPARTITIONleadstoSQLCODE-876
– PREVENT_NEW_IXCTRL_PART• Cannolongercreatenewindex-controlledpartitionedtablespaces
• MaterializingREORGcannowbreak-intoapersistentthreadrunningRELEASE(DEALLOCATE)package
• REORGREBALANCE– NotpossibleforpartitionswithpendingALTERLIMITKEYchanges
– Willworkfortheotherpartitions
– Willworkforpartitionswhichare‘hard’reorgpending(REORP)
63
DROPCOLUMN• Workswell
– CanconverttoUTSandconcurrentlyDROPCOLUMN
– MaterializingREORGcanberunatthepartitionlevel– ifallpartitionsarecovered
– Allpackagestouchingthetablewillbeinvalidated
• Restrictions– CannotuseDROPCOLUMNinclassictablespacetype(SQLCODE-650)
– Cannotdropacolumncontainedinanindexorview(SQLCODE-478)
– CannotaddadroppedcolumnbeforethematerializingREORG(SQLCODE-20385)
– Cannotcreateaviewwithadroppedcolumn(SQLCODE-20385)
– CannotdropthesamecolumnasecondtimebeforethematerializingREORG(SQLCODE-205)
– CannotunloadfromanimagecopytakenbeforethematerializingREORG(DSNU1227I)
– CannotrecovertoaPITbeforethematerializingREORG(DSNU556I)
64
AvailabilitySummary• ConsiderusingthenewDB2V11Bind,Rebind,DDLandonlineREORGbreakintopersistedthreadwithRELEASE(DEALLOCATE)mechanism
– Understandthebreakinbehavioranditsdesign
• BeawareofimplicationswithTablepartitioningcomparedtoIndexcontrolledpartitioningandALTERLIMITKEY
• DROPColumnfeatureandtherestrictions
65
PERFORMANCEANDSCALABILITY
66
PerformanceEnhancements- noREBINDneeded(CM)• DDFperformance improvements
– Reduced SRBscheduling onTCP/IPreceive using new CommServer capabilitiesimproved autocommit OLTPperformance
• xProcs above the bar
• zIIPenablementforallSRB-modeDB2systemagentsexceptP-Locknegotiation
• Avoid cross-memoryoverhead for writing logrecords
• Datadecompression performance improvement
• INSERTperformance– Latch contention reduction
– CPUreduction for Insertcolumn processing and logrecord creation
– Datasharing LRSNspin avoidance
– Pagefix/free avoidance inGBPwrite
67
PerformanceEnhancements- noREBINDneeded(CM)...
• Sort performance improvements
• DPSIperformance improvements for merge
• Performanceimprovements with largenumber of partitions
• XMLperformance improvements
• Optimize RELEASE(DEALLOCATE)execution sothat it is consistently betterperforming than RELEASE(COMMIT)
• IFI306filtering capabilities to improve QReplication capture performance
• Utilitiesperformance improvements
• Automatic index pseudodelete clean-up
• ODBC/JDBCType2performance improvements
• Javastored procedures
– Multithreaded JVMs,64-bitJVM– more efficient
68
PerformanceEnhancements– noREBINDneeded(CM)...
• ACCESSDATABASEcommand performance
• DGTTperformanceimprovement– AvoidincrementalbindsforreducedCPUoverhead
• P-procs for LIKEpredicates against Unicodetables
• Improved performance for ROLLBACKTOSAVEPOINT
• zEC12exploitation
• Datasharing performance improvements
69
PerformanceEnhancementsrequiringREBIND(CMwithorwithoutAPREUSE)
• MostIn-memorytechniques
• Noncorrelatedsubquerywithmismatchedlength
• Selectlistdo-once
• Columnprocessingimprovements
• RIDoverflowtoworkfile handledforDataManagersetfunctions
• Performanceimprovements for common operators
• DECFLOATdata typeperformance improvements
70
PerformanceEnhancementsrequiringREBIND(CMwithoutAPREUSE)
• Querytransformationimprovements– lessexpertiserequiredtowriteperformantSQL
• Enhancedduplicateremoval
• DPSIandpagerangeperformanceimprovements
• OptimizerCPUandI/Ocostbalancingimprovements
71
PerformanceEnhancements– DBAorapplicationeffortrequired(NFM)
• Suppress null-indexes
• NewPCTFREEFORUPDATEattributetoreduceindirectreferences
• DGTTperformanceimprovements
• Globalvariables
• Optimizerexternalizationofmissing/conflictingstatistics
• Extendedoptimization- selectivityoverrides(filterfactorhints)
• Opendatasetlimitraisedto200K
72
OptionalEnhancementsneedNFM+DBAeffort
• DSNTIJCB– Optional– ConvertBSDSforextended10-byteRBAs– - STOPDB2MODE(QUIESCE)
• DSNTIJCV– Optional– ConvertCatalog andDirectorytableandindexspacestoextended10-byteRBAformat
– ReorgsallCatalog andDirectorytablespacesSHRLEVELCHANGE
– Canbesplituptorunreorgsinparallel
73
ExampleofCustomerPerformanceTesting• DB210NFMbaseline
• DB211CMbeforeREBIND
• DB211CMafterREBIND
• DB211NFM(noneedforfurtherREBIND)
• DB211NFMafterREORG(tomigrateobjecttoextendedLRSN)
• DB211NFMExtendedLRSN
74
ExampleofCustomerPerformanceTesting...• MakesurethattheCPUnumbersarenormalizedacrossthoseintervalsi.e.,useCPUmillisecondspercommit
• EasytocombinestatisticsandaccountingbystackingthevariouscomponentsofCPUresourceconsumption:
– MSTRTCB/(commits+rollbacks)
– MSTRSRB/(commits+rollbacks)
– MSTRIIPSRB/(commits+rollbacks)
– DBM1TCB/(commits+rollbacks)
– DBM1SRB/(commits+rollbacks)
– DBM1IIPSRB/(commits+rollbacks)
– IRLMTCB/(commits+rollbacks)
– IRLMSRB/(commits+rollbacks)
– AverageClass2CPCPU*occurrences/(commits+rollbacks)
– AverageClass2SECPU*occurrences/(commits+rollbacks)
75
CICSTestTransactionProfile
Avg.DML• 3 Insert
• 7 Select
• 5 Open
•103Fetch
Avg.Bufferpool•65 Getpages•13 SyncRead
CPUconsumption•Class1: 3.2msec
•Class2: 2.3msecWithCP-Speed802MIPS
76
Fetch Intensive
• Transaction types– E-Banklogon
– Balancecheck
– FinancialStatementhistory
– Accountsearch
CICStransactionCPUtime
77
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
10NFM 11CM 11RBND 11NFM 11Reo 11EL
CPUmsec
DB2ASzIIPperTRAN
DB2ASGCPperTRAN
CL2CPUperTRAN
5% Activity moved to zIIP
In CM
DB2systemaddressspaceCPUperCICStransaction
78
0,00
0,10
0,20
0,30
0,40
0,50
0,60
0,70
10NFM 11CM 11RBND 11NFM 11Reo 11EL
CPUTimemSec
IRLMSRB
IRLMTCB
DBM1zIIP
DBM1SRB
DBM1TCB
MSTRzIIP
MSTRSRB
MSTRTCB
Activity moved to zIIP
in CM
TestBatchjobprofile
DML• Commits:4528
• Delete: 37613• Rows:482836
• Update: 45099
• Select: 119773
• Insert: 525884
• Fetch: 548947
79
Bufferpool• Getpage: 4.8M
• SyncRead: 133K
• DynPref: 57K
CPU• Class1: 69Sec
• Class2: 67SecWithCP-Speed802MIPS
Elapsed:• Class1: 07:02min
Insert and Delete Intensive
BatchtestCPUtime
80
0,00
20,00
40,00
60,00
80,00
100,00
120,00
10NFM 11CM 11RBND 11NFM 11Reo 11EL
CPUSec
DB2ASzIIPperBATCH
DB2ASGCPperBATCH
CL2CPUperBATCH
LRSN spin loop eliminated
in V11 E LNo credit to V11
For V11 Reo improvement
DB2systemaddressspaceCPUforBatch
81
0,00
5,00
10,00
15,00
20,00
25,00
10NFM 11CM 11RBND 11NFM 11Reo 11EL
CPUsec
IRLMSRB
IRLMTCB
DBM1zIIP
DBM1SRB
DBM1TCB
MSTRzIIP
MSTRSRB
MSTRTCB
Activity moved to zIIP
In CM
LRSN spin loop eliminated in 11 EL
reduces MSTR SRB
AutomaticPseudoDeletedIndexEntryClean-up• Recaponimpactofpseudodeletedindexentries
– Indexsizegrowswithincreasingnumberofpseudo-deletedindexentries• Moregetpagesandlockrequestsrequired• IncreasedCPUcostandpossiblylongerelapsedtimesforaccessviaindexsearch
– ApplicationsmayencounterdeadlocksandtimeoutsduringINSERT/UPDATE/DELETE• Collisionswithcommittedpseudo-deletedindexentries• RIDreusebyINSERTfollowingDELETE=>deadlock
• PriortoV11,howaretheycleanedup– DB2removespseudo-deletedentriesduringmainlineoperations
• Insert/deleteoperationsremovepseudo-deletedentriesfromindexpages• SQLrunningwithisolationlevelRRremovespseudo-deletedentries
– Pagesthatonlycontainpseudo-deletedindexentriesarecalledpseudo-empty• DB2attemptstocleanuppseudo-emptyindexpagesaspartofDELETEprocessing
– REORGINDEXremovespseudo-emptyindexpagesandpseudo-deletedentriesthatwerenotcleanedupbythemainlineprocessing
82
AutomaticPseudoDeletedIndexEntryClean-up…• AutonomicsolutionprovidedinCMandturnedonautomaticallyforallindexes24*7
– Automaticclean-upofpseudo-deletedindexentriesinindexleafpages– Automaticclean-upofpseudo-emptyindexpages– DesignedtohaveminimalornodisruptiontoconcurrentDB2work– Clean-upisdoneundersystemtasks,whichrunasenclaveSRBsandarezIIP eligible– Parentthread(oneperDB2member)loopsthroughRTStofindcandidateindexes– Childclean-upthreadsonlycleanupanindexifitalreadyisopenedforINSERT,
UPDATEorDELETEontheDB2member• AvoidcreatingGBPdependencyonindexes
• Potentialdisruptioncanbeminimizedbymanagingdownthenumberofclean-upthreadsorspecifyingtimewhenindexesaresubjecttoclean-up
– Cancontrolthenumberofconcurrentclean-upthreadsordisablethefunctionusingZPARMINDEX_CLEANUP_THREADS
• 0=Disable,1-128,10isdefault
– EntriesinnewCatalogtableSYSIBM.SYSINDEXCLEANUP• Definewhen/whichobjectsaretobeconsideredinagenericway
83
AutomaticPseudoDeletedIndexEntryClean-up• Upto39%DB2CPUreductionpertransactioninDB211comparedtoDB210
• Upto93%reductioninPseudodeletedentriesinDB211
• ConsistentperformanceandlessneedofREORGinDB211
84
0
500000
1000000
1500000
2000000
2500000
0
0,0005
0,001
0,0015
0,002
0,0025
0,003
0,0035
Day1 Day2 Day3 Day4 Day5#o
fpseud
o_de
leteden
tries
CPUtime(sec)
WASPortalWorkload5DaysPerformance
V10TotalCPUtime
V11TotalCPUtime
V10sumofREORGPSEUDODELETES
V11sumofREORGPSEUDODELETES
PerformanceEnhancements• Qreplication logfiltering
– ReduceIFIlogreadcostbyqualifyingtheobjectswithDBID/PSID
– Additionalbenefitifobjectsarecompressed
– MovefilteringfromQreplication capturetasktoDB2engine
– Potentialforverysignificantreductioninthenumberoflogrecordsreplicated
– RequiresIBMInfosphere DataReplicationQReplicationorChangeDataCapture10.2.1
• ArchiveTransparency
– Veryusefulnewfeature
– NeedtocarefullyexaminetheadditionalcostofORDERBYsortwhenaccessingthearchive
– Whenapplicationonlyfetchesalimitednumberofrowsfromtheresultsetthenthecostcanincreasesignificantlywhenalsoaccessingthearchive
– Willtypicallybeusedbycustomersselectivelyonacasebycasebasis
85
zIIPExpansion• NotexpectedtoassistasmuchasV10,butcanprovideincreaseuse• Upto100%ofprocessingforasynchronousenclaveservicerequestblocksthatexecuteintheDB2ssnmMSTR,ssnmDBM1andssnmDIST addressspaces.
– Prefetch (V10)– Deferredwrite(V10)– Logwriteandread– GBPwriteandcastout– Indexpseudodelete/XMLcleanup– DPSIparallelaccess(PARAMDEG_DIPSI)
• ExpandedRUNSTATSandLOADredirect– EnclaveSRBtasksforLOAD,REORG,andREBUILDINDEXInlineStats– EnclaveSRBtasksforRUNSTATSColumnGroupDistributionStats– EnclaveSRBtasksforLOADREPLACEindexmanagement
• IIPHONORPRIORITY=NO– Disablesallsystemagent(includingprefetch)redirect
86
P-LockNegotiation NOT zIIP Eligible
SynchronousReceive- Allremoteapplications…
• PreDB211DDFusesasynchronousreceive,whichrequiresexpensiveSRBswitchwhendatabecomesavailable
– ClientconnectionterminationsoDDFcancancelthread• DB211DDFwillexploitsynchronousreceive
– AvailableinCMwithoutapplicationchangesorREBINDsrequired– z/OSCommunicationsServerprovidesnewTerminationNotification– DDFwillbenotifiedofanysocketfailure,evenwhileinsyncreceivemode– DDFcanabortthreadsrunningsynchronously– PreliminarymeasurementsusingthedistributedIRWW*suite
• 6%to8%ITR**improvement• 4%to6%CPUimprovement
• Onlyaffectsnon-storedprocedureworkloads
87
*IBM Relational Warehouse Workload**Internal Throughput Rate
CancelthreadandInterruptstatement…• PreDB211DDFCANCELTHREADmaynotworkinsomecases
– Historically"soft",orreactiveinnature
– Canceldetectionpointmaynotbeencounteredinatimelymanner
• DB211NewFORCEkeywordonCANCELTHREADcommand– Purges thread ofaremoteconnectionintheDB2
– FORCE optionacceptedonlyafterCANCEL THREADissuedwithoutthe FORCE option
– AvailableinCM
• PurgerunningDDFSRBusingnewz/OSpurgeSRBfunction– Requiresnewz/OSCALLRTMTYPE=SRBTERMservice
• Availableinz/OS1.13orlaterviaAPAROA39392
88
WithSyncReceiveandIDTHTOIN• (DB211)- NowwhenidlethreadtimeoutishitDDFmustissueTCPIP.DROPcommandtokillthesocketassociatedwiththethread
– Customersawtimeoutsandhadtomanuallycancelthreadstoletbatchbreakin..– SodidyouseeIDTHTOINpopinthelog,whataboutinthepreviousrelease?
• 00D3003BinthelogforthreadsthathitIDTHTOIN
• IfthreadsareremaininginthesystemlongerthanonDB210,andtheidlethreadsarenotbeingcanceled(causingtimeoutsorcontentionwithotherprocesses),thenMVS.VARY.TCPIP.DROPOPERCMDSmissing
– GetDSNL512I-111RC=77E800DC(EACCES/JRSAFNotAuthorized)
• ProcessisdescribedinsettingupDDFandUNIXsystemservicessectionoftheinstallationguide
– http://www-01.ibm.com/support/knowledgecenter/SSEPEK_11.0.0/com.ibm.db2z11.doc.inst/src/tpc/db2z_enableddf4uss.dita?lang=en
• PI06325– messageDSNL512Iisenhancedtoshowsocket=EZBNMIF4_DROPCONtoalertyouthatservicefailed
– http://www-01.ibm.com/support/docview.wss?uid=swg1PI06325
89
BufferPoolenhancements…• BPReclassification
– DB29/DB210• PagesremainclassifiedasprefetchevenafterrandomI/Oagainstpage
– DB211• Prefetchedpagescanbereclassifiedasrandomifrandomgetpageagainstpage
• PotentiallylesssyncI/Osformixedaccess(prefetch+random)
– Mayneedtore-evaluateVPSEQTsettingforcertainbufferpools
– VPSIZEMINandVPSIZEMAXattributesforAUTOSIZE(YES)bufferpools• Setahardupperandlowerlimitforautonomicsizing
90
BufferPoolenhancements• OnDemandStorageAllocation
– DB29• EntirebufferpoolallocatedusingVPSIZEforvirtualandrealstorage
– DB210• Bufferpoolallocatedondemand
– Ifaquerytouchesonlyafewdatapages,onlyasmallamountofvirtualandrealbufferpoolstorageisallocated
–CouldcauseperformanceproblemswhentryingtoquicklyaddmorepagesforadditionalGETPAGEs
– DB211• SimilartoDB29andprior
• EntireVPSIZEofbufferpoolvirtualstorageallocatedwhenbufferpoolbecomesactivebutonlyenoughrealstorageforcurrentGETPAGEs
• OvercomesDB210performanceissuewithoutusingrealstorageifnotnecessary
91
BPSimulation(NFMmode)• AbilitytosimulatechangingVPSIZEandVPSEQT
– AssumesLRUmechanisminplace
• Storagecostforasimulatedbufferpoolislessthan2%for4KpagessimilartoHiperpoolexceptnotallocatingactualbuffers
– Simulationisdonewithaseparatebuffersearchhashtablewithbuffercontrolblocks
– For4KpagesizeBP,thestorageoverheadis2%ofSPSIZE*4K(e.g.20MBfor1GBsizebufferpool)
– CPUoverheadisbetween1-3%whenrunninglab.Workloads
– ShowsupinDSNB431I-432I,andOMPE(IFCID002)
92
SelectiveDecompression• Compression/Decompressionisthesinglemostexpensiveinstructioninz/OS
• DB2usesrowlevelcompression,henceeachrowwascompletelydecompressedwhenitwasaccessed
• V11usessoftwareinstructionstoaugmenttheHWdecompression– AvailableinCMbutpartialrowdecompressionneedsarebind(RRFformat)
93
Otherperformancerecommendations• MakesureHVCOMMONinIEASYSxx canaccommodatelogoutputbuffer
• Configureadditional1MBLFAREA(z/OSparameterinIEASYSxx)formaximumbenefit
• LRSNspinavoidancerequiresbothBSDSandobjectsconversioninNFM
• MonitorlogI/Operformanceduetologrecordsizeincrease– 3%to40%increaseinlogrecordsizeobservedfollowingBSDSconversion
• EssentialtomakesureenoughzIIP capacityavailablebeforeV11CMmigration– zIIP ‘HelpFunction”IIPHONORPRIORITYshouldbesettoYESincasethereisashortageofzIIP capacity
– ContinuetomonitorzIIP capacitythereafter
• MRU(MostRecentlyUsed)usedforpagesbroughtinbyutilities
• NewFRAMESIZEparameterindependentfromPGFIXparameter– IfyouPGFIXafterV11FRAMESIZE1MisneededtouseLFAREA
94
PerformanceSummary• Opportunityforimprovedperformanceforlegacyapplicationprograms
• REBINDofstaticSQLpackagesisveryimportant
• GoodvalidationofpotentialfromESPcustomersandIBMinternalworkloads
• YourmileagewillvarybasedonyourSQLapplicationworkloadascertainfeaturesonlyapplytocertainworkloads
• ImpressiveCPUsavingsobservedforsomeworkloads
• HighlyoptimizedstaticSQLand/orsimpleSQLmaynotseemuchbenefit
• MorebenefitformorecomplexSQLi.e.,notreadasinglerowbyprimarykey
• Donotsell(orbuy)thesavingsbeforeyouhaveseenthemforyourworkload
• BestthingforCPUperformanceonentrytoV11CMistorebindfrequentlyusedpackages(re-enableSPROCs,newxPROCs,avoidpuffing).
9595
REALSTORAGEMANAGEMENT
96
Agenda• LargeMemoryValue
• RealandVirtualConcept
• Runningsafely
• RealStorageManagementControls
• ReportingandMonitoring
• Bufferpool ConsiderationsandLFAREA
• Summary
97
Largerealmemoryvalue• SubstantiallatencyreductionforOLTPworkloads
– Significantresponsetime reductions
– Increasedtransaction rates
• In-memorydatabasesfordramaticgainsinresponsetimesbyavoidingIOwaits
• Batchwindow reductions
– Improvesinglethread performance
– Canprocessdatamoreefficientlyhelpingorganisations keeppacewithinfluxof data
– Reducingtimetoinsightforanalytic workloads
– Reducestimeittakestogetbusinessinsightfromraw data
• CPUperformance improvements
– ReducesCPUresourceconsumptionper transaction
– z/OSlabmeasuredcostofI/Oat20us-70usofCPU
– Improvesresponsetimesandshrinksbatch windows
– Reducetheneedforapplicationand/orsystemredesigntomeetservice goals
98
DB2forz/OSandLargeReal Memory
99
“Memoryischeaporone time charge, CPUsare expensive”“ForeveryI/Othatyousave,youavoidthesoftwarechargefor
theCPUthatittooktootherwisedothat I/O”
77
CPUCostSavingbyReducingDB2SyncI/Os• Banking(60Maccount)workloadwith2waydatasharing:
• 11%responseand6%CPUreductionfrom52GBGBPto398GBforbothmemberswithsameLBPsize(60GB)
• 40%responseand11%CPUreductionfrom30GBLBPto236GBLBP forbothmemberswithsamereasonableGBPsize(60GB)
100
Summary• SyncI/OreductiontiescloselywithCPUreduction
– Thiswasdoneinz/OS2.1,zEC12,DB211
• Forthisparticularworkload,thehigherbenefitoninvestingonlocalbufferpools– NotalltheobjectsbecomeGBP-dependent
– Largetablespaces definedwithMemberCluster
– Mostofaccesspatternisfairlyrandom
• GBPhastobebigenoughtosupportenoughdirectoryentries
• ThebestperformerwaswithbothlargeLBPandlargeGBP
101
IBMBrokerageWorkload– EnlargingLocalBuffer Pool
102
l 10->24GB14%CPUreduction,3%Throughputimprovement(26->13sync I/O)l 24 ->70GB 3%CPU reduction,15% Throughputimprovement (13->8sync I/O)
• Globaldynamicstatementcache– EDMSTMTCupto4GwithDB211,default110MB
– ReductionofCPUtimebyavoidingfullprepare
• Localstatementcache– MAXKEEPDupto200KstatementswithDB211,default
5000
– ReductionofCPUtimebyavoidingshortprepare
• In-memorydatacacheforsparseindex– MXDTCACHupto512MBperthread,default20MB
– ReductionofCPUandelapsedtimewithpotentiallybetteraccesspathselectionwithDB211
• RIDPool(MAXRBLK)– LowerCPUtimeifRIDlistsdon’tspillintoworkfiles
• DB2localandgroupbufferpools– ReductionofelapsedtimeandCPUtimebyavoiding
I/Os
– PGSTEAL(NONE)inDB210=Inmemorydatabase
– CPUreductionfromPGFIX=YESandlargepageframes
• ThreadreusewithIMSorCICSapplications
– ReductionofCPUtimebyavoidingthreadallocationanddeallocation
• ThreadreuseandRELEASE(DEALLOCATE)
– ReductionofCPUtimebyavoidingpackageallocationandparentlocks
– DDFHighperformanceDBATssupportwithDB210
– Abilitytobreak-inpersistentthreadwithDB211
DB2 Areas that Benefit from Larger Memory
103
REALSTORAGEMANAGEMENTANDCONTROLS
104
8
DB2pushedouttoAUXstorage• AnypagingofDB2isbad
– PagingaddstodirectCPUcostandhurtsresponsetime
– FlashExpressmemoryhelpsmitigatetheproblem,butdoesnotsolveit
• DB2dumpswilltakelongerifz/OShastoreadpagesinfromAUXstorageorFlashExpressmemory,impactingtheavailabilityofDB2system
– Heldlocksmagnifiestheproblem,causingsympathysicknessonotherLPARsinthesysplex
• DFSORTmayconsumeallavailablerealmemoryandpushDB2framesout– RiskofworkloadspikesoccurringwhileDFSORTisrunning
• Donotoversizeyourlocalbufferpools– Ifyouregularlyseepaging,reducethesizeofyourbufferpools(andotherDB2memorypools)
– UsingPGFIX(YES)andlargesizeframessavesCPUtime,butbecarefultomakesureyouhaveenoughrealmemorysothatthereisnopaging
105
Whataffectsthestorage?• 64-bitprivate
– Mostlybufferpools
– Fixed-RIDpool
– EDMpool
• 64-bitsharedthreadandsystem– UsedtoexecuteSQL
– Variable-dynamicstatementcaches,CT,PT,SKCT,SKPT
– *Comparetonumberofsystemanduseragents/threads,parallelism
• 64-bitstack– JustasinV9,usedbythreadtoexecuteSQL
– *same
• 64-bitcommon– Distributedagents,packageaccounting,rollup
• AllareaffectedbyREALSTORAGE_MANAGEMENT
106
MonitoringREAL/AUXstorageusage– Mappingforreference
IFCIDFIELD OMPEFIELD OMPEPDBCOLUMNNAME MEMU2Description
QW0225RL QW0225RL REAL_STORAGE_FRAME DBM1REALinusefor31and64-bitpriv (MB)
QW0225AX QW0225AX AUX_STORAGE_SLOT DBM1AUXinusefor31and64-bitpriv (MB)
QW0225HVPagesInReal SW225VPR A2GB_REAL_FRAME DBM1REALinusefor64-bitpriv (MB)
QW0225HVAuxSlots SW225VAS A2GB_AUX_SLOT DBM1AUXinusefor64-bitpriv (MB)
QW0225PriStg_Real SW225PSR A2GB_REAL_FRAME_TS DBM1REALinusefor64-bitpriv w/oBP(MB)
QW0225PriStg_Aux SW225PSA A2GB_AUX_SLOT_TS DBM1AUXinusefor64-bitpriv w/oBP(MB)
QW0225RL QW0225RL DIST_REAL_FRAME DISTREALinusefor31and64-bitpriv (MB)
QW0225AX QW0225AX DIST_AUX_SLOT DISTAUXinusefor31and64-bitpriv (MB)
QW0225HVPagesInReal SW225VPR A2GB_DIST_REAL_FRM DISTREALinusefor64-bitpriv(MB)
QW0225HVAuxSlots SW225VAS A2GB_DIST_AUX_SLOT DISTAUXinusefor64-bitpriv(MB)
QW0225ShrStg_Real SW225SSR A2GB_SHR_REALF_TS REALinusefor64-bitshared(MB)
QW0225ShrStg_Aux SW225SSA A2GB_SHR_AUXS_TS AUXinusefor64-bitshared(MB)
QW0225ShrStkStg_Real SW225KSR A2GB_SHR_REALF_STK REALinusefor64-bitsharedstack(MB)
QW0225ShrStkStg_Aux SW225KSA A2GB_SHR_AUXS_STK AUXinusefor64-bitsharedstack(MB)
QW0225ComStg_Real SW225CSR A2GB_COMMON_REALF REALinusefor64-bitcommon(MB)
QW0225ComStg_Aux SW225CSA A2GB_COMMON_AUXS AUXinusefor64-bitcommon(MB)
QW0225_REALAVAIL S225RLAV QW0225_REALAVAIL REALAVAIL(MB)(S)
107
Note:AllREAL/AUXstoragefieldsinIFCID225andOMPEperformancedatabaseareexpressedin4KBframesorslots– theyshouldbeconvertedtoMB(conversionisalreadydoneinMEMU2)
RealStorageManagementandControls…• MakesureMAXSPACEissetproperlyanddefensively
– RepresentsthetotalamountofstorageforcaptureddumpsfortheentireLPAR– MAXSPACEvalueshouldnotbesetsohighthatpagingcanoccurcausingmassiveissuesto
theLPAR– IfmultipleDB2sonsameLPARcanwildcardtothesamedump,thenMAXSPACEneedsto
besetappropriately
• MAXSPACE=16Gisagoodstarttocopewithmorethan90%ofallcases– MAXSPACEat16GmaybetoobigforsmallLPARs/DB2subsystems.
• LookatsizeofexistingdumpsandcutbackonMAXSPACE.
• MAXSPACErequirementshouldbe– (DBM1– Bufferpools)+Sharedmemory+DIST+MSTR+IRLM+COMMON+ECSA
• z/OSparameterAUXMGMT=ON– NonewdumpsareallowedwhenAUXstorageutilizationreaches50%– CurrentdumpdatacapturestopswhenAUXstorageutilizationreaches68%– Oncethelimitisexceeded,newdumpswillnotbeprocesseduntiltheAUXstorage
utilizationdropsbelow35%
108
RealStorageManagementandControls…• DB2pagespagedoutandadumphappens?:
• Whatismyexposure?– IncreasedMASTERCPUtime,z/OStriestostealframestomeettheexcessivedemandcausedbythedump
– Elongateddumptimes
• Auxiliarystoragenumber>0– Intheoryeachslotcouldhavetobepagedin
• z/OScanhavethepageinbothplaces,REALandAUX
• Iftotalsizeacrossallbufferpools ismorethan800MBthenthebufferpools arenotdumped
– Noprefetch onAUXstorage,soallI/Os aresynchronous
– Worstcaseisthenumberofpages*page-inI/Otime• Forexample2GBof4Kpages*3ms=524288*0.003=26mins
– Guinnessworldrecordforadump37mins
– FullSysplex hangresulted
109
RealStorageManagementandControls…• SYSPLEXsympathysickness:
• Slowdownandnoapparentreasonwhy?
• DB2takingthedumpmayhaveTCBsnon-dispatchable
• P-Locknegotiationaffected
• Locksnotreleasedinatimelymanner
• ExcessivedumptimecausedbypagingontheLPARmaycausemassivesympathysicknessslowdowns
• HowcanamemberslowdownwhenthereisplentyofCPU/StorageontheLPAR– MaybetheownerofaP-lockisbeingdumpedorispagingalot
110
TheDUMPeffect– notransactionsbeingprocessed
111
0
1000
2000
3000
4000
5000
6000
7000
NUM_ENDED NUM_ARRIVED
One Flash Express Card
CPUnanos
Flash -microseconds
Disk -milleseconds
Access Time
Ifyouhavenochoicebuttopage– useFlashExpress
• IfthereisnotenoughspareREALmemoryontheLPARtoavoidpaging;considerprovisioningFlashExpress
– FlashExpressisaPCIe IOadapterwithNANDFlashSSDs
– Usedtodeliveranewtierofmemory- storageclassmemory
– SizedtoaccommodateallLPARpaging• Eachcardpairprovides1.4TBusablestorage
• Maximum4cardpairs(4X1.4=5.6TB)
• FLASHExpresshelps1MBpageable frames• LargepageswillbepagedouttoFLASH
• Retain‘largepage’attributewhenpagedbackin
• VeryimportantifyouhaveHyperswap andz/OSCRITICALPAGING=YES
– HVSHAREandotherstorageareasprotectedfrompaging
• AlsousefulifMAXSPACEcannotbebackedwithREAL
112
RealStorageManagementandControls…• ControluseofstoragebyDFSORT
– SetEXPMAXdowntolimitmaximumDFSORTusage
– SetEXPOLD=0topreventDFSORTfromtaking"old"framesfromotherworkloads
– SetEXPRES=%orn{reserveenoughforMAXSPACE}
113
0
5000
10000
15000
20000
25000
30000
2013-07-29-00.02.00.000161
2013-07-29-04.08.00.000157
2013-07-29-08.14.00.000209
2013-07-29-12.20.00.018877
2013-07-29-16.26.00.000163
2013-07-29-20.32.00.000178
2013-07-30-00.38.00.000166
2013-07-30-04.44.00.000904
2013-07-30-08.50.00.016955
2013-07-30-12.56.00.000175
2013-07-30-17.02.00.000175
2013-07-30-21.08.00.000310
2013-07-31-01.14.00.059911
2013-07-31-05.20.00.019601
2013-07-31-09.26.00.000123
2013-07-31-13.32.00.000499
2013-07-31-17.38.00.000158
2013-07-31-21.44.00.000342
2013-08-01-01.50.00.370531
2013-08-01-05.56.00.093954
2013-08-01-10.02.00.130884
2013-08-01-14.08.00.000155
2013-08-01-18.14.00.000274
2013-08-01-22.20.00.015079
2013-08-02-02.26.00.000102
2013-08-02-06.32.00.000152
2013-08-02-10.38.00.004328
2013-08-02-14.44.00.003906
2013-08-02-18.50.00.036134
2013-08-02-22.56.00.000143
2013-08-03-03.02.00.000214
2013-08-03-07.08.00.000181
2013-08-03-11.14.00.000159
2013-08-03-15.20.00.014000
2013-08-03-19.26.00.000364
2013-08-03-23.32.00.000201
2013-08-04-03.38.00.000168
2013-08-04-07.44.00.000197
2013-08-04-11.50.00.033578
2013-08-04-15.56.00.000174
2013-08-04-20.02.00.000151
(MB)CBP4- REALstorageavailable
RealStorageSummary• UseREAL_STORAGE_MANAGEMENT=AUTO
• KeeptheLPARwellprovisionedwithREALstorage
– Haveenoughfor4Kandusewhatisleftfor1M!
• AvoidPagingtoAUXorFlashExpress
• WatchoutforMAXSPACEandlargedumpsthatmaycausesystemtopage
• Use‘commoncurrency’forreportingandmonitoringREALandAUXusage
• ItisimpossibletoknowhowmuchrealstorageDB2isaactuallyusinguntiltheLPARstartstopage(z/OSRSMinstrumentation).
• CPUperformanceadvantageusingREALSTORAGE_MANAGEMENT=NO/AUTO
– REALSTORAGE_MANAGEMENT=NOonlyifrealmemoryisverygenerouslyallocatedontheLPAR
– REALSTORAGE_MANAGEMENT=AUTOisdesignforeffectivethreadreuseforhighvolumetransactionstomitigateCPUoverheadofdiscard
114
EXTENDEDLRBA/LRSN
115
ExtendedLRBA/LRSN• WhatyouneedtoknowforDB211CMandDB210NFM?
– 6byteformat- LRBA/LRSNbeforeDB211è x‘LLLLLLLLLLLL’
– 10byteextendedformat– LRBA/LRSNhasaddressingcapacityof1yottabyte(2**80)• 10byteextendedformat- LRSNwithDB211 è x‘00LLLLLLLLLLLL000000’
• 10byteextendedformat- LRBAwithDB211 è x‘00000000RRRRRRRRRRRR’
– WheredowefindLRBA/LRSN?• DB2Catalog è SYSCOPY,SYSxxxxPART,…..
• DB2Directory è SYSUTILX,SYSLGRNX,….
• BSDS è pointer,Active&ArchiveLogvalues..
• DB2Logs è active&archivelogs
• DB2Pagesets è Catalog&Directoryandalluser-pagesets
116
ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211CMandDB210NFM?…
– DB211CM• DB2internalcodingdealswith10byteextendedformatLRBA/LRSNvaluesonly
• LRSNinUtilityoutputisshownin10byteextendedformatwithprecision‘000000’ except– QUIESCEutility,whichexternalizesLRSNin10byteextendedformatwithprecision‘nnnnnn’
• RECOVERutilityhandles10byteextendedformatLRBA/LRSNinput
• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPART issetto‘B‘fornewdefinedorobjects,whicharereorged orloadedwithreplace-option(possiblevalues‚B,blank‚U,E)
– DB211CM/DB210NFMcoexistenceindatasharing• Fulltolerationof10byteextendedformatLRBA/LRSNvalueasinputtotheRECOVERUtility
• Sanitychecksincludedfor‘wronglyused6byteformatLRBA/LRSN’
117
ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?
– MigrationtoDB211NFM(viaDSNTIJEN)• Catalog&DirectoryTable‘LRBA/LRSNColumns’arealteredto10byteextendedformat
• SYSIBM.SYSLGRNXentriesarenowstoredas10byteextendedformatLRBA/LRSNvalues
• SYSIBM.SYSCOPY– ConversionofallLRBA/LRSNvaluesisdoneforexistingdatato10byteextendedformatwith
leadingbyte‘00’andprecision,‘000000’forLRSNandrightjustifiedrightwithleading‘00000000’forLRBAvalues
– Newdataisstoredin10byteextendedformatwithprecision‘nnnnnn’
• LRBA/LRSNforallUtilitiesusenow10byteextendedformat
• LRBA/LRSNvaluesarestillwrittentoDB2logsin6byteformat
• LRBA/LRSNvaluesarestillwrittentoDB2pagesetsin6byteformat
118
ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…
– BSDSconvertedto10byteextendedformatLRBA/LRSNinNFMonly(DSNJCNVT)• ThereisnowaybackforBSDS!
• NowLRBA/LRSNvaluesarewrittentoDB2logsofthesubjectDB2membernowin10byteextendedformatwithprecision‘nnnnnn’
• LRBA/LRSNvaluesarestillwrittentoDB2pagesetsin6byteformat– Conversion(10to6or6to10byte)hastobedone
– LRSNSpincanstillhappen
– DSN1LOGPandREPORTRECOVERoutputwillshow10byteextendedformatLRBA/LRSNalthoughneverexternalizedtopagesets(differentoutput,forDSN1PRNTofpagesets)
• Canbedone,wheneveryouwanttodoitafterentrytoV11NFM,regardlessofpagesetformats
119
ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…
– ReorgCatalogandDirectorypagesetsto‘extendedformat’(inNFMonly!)• Canbedonewheneveryouwantto,regardlessofBSDSanduserpagesetformats
• NowLRBA/LRSNvaluesarewrittentoconvertedpagesetsin10byteextendedformat– LRSNwithprecision‘nnnnnn’,ifupdateisdoneonaDB2memberwith10byteextendedformat
BSDS
– LRSNwithprecision‘000000’,ifupdateindoneinamemberwith6byteformatBSDS
• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPARTisupdatedto‘E’
• LRSNSpincouldstillhappenforDB2memberwith6byteformatBSDS
• Canbeconvertedbackto6byteformat(alloratpartlevel)
120
ExtendedLRBA/LRSN…• WhatyouneedtoknowforDB211NFM?…
– ReorgUserpagesets to‘extendedformat’(inNFMonly!)• Canbedonewheneveryouwantto,regardlessofBSDS,Catalog&Directorypageset
formats
• NowLRBA/LRSNvaluesarewrittentoconvertedpagesets in10byteextendedformat– LRSNwithprecision‘nnnnnn‘,ifupdateisdoneinamemberwith10byteextendedformatBSDS
– LRSNwithprecision‘000000‘,ifupdateisdoneinamemberwith6byteformatBSDS
• Column‘RBA_FORMAT’inSYSIBM.SYSxxxPART issetto‘E‘
• LRSNSpincouldstillhappenforaDB2memberwith6byteformatBSDS
• Canbeconvertedbackto6byte(alloratpartlevel)
• IsdonebyREORG,LOAD..REPLACEorREBUILDwith‘RBALRSN_CONVERSIONEXTENDED’orifZPARMOBJECT_CONVERTED=EXTENDED
• ‘RECOVER...TOCOPY...’usinga6byteCopycanresetformatbackto‘basic’
121
ExtendedLRBA/LRSN…• Enhancementstoimproveusabilitycharacteristicsbasedon6byte/10byte
formatLRBA/LRSNhandling– PreventDSNJCNVTfromconvertingDB210NFMBSDStoextendedformat– Support10byteextendedformatinputtoRECOVERinDB210– PerformsanitycheckstoguardagainstinvalidLRSNvaluesi.e.,6byteLRSNvalues
withleadingbyteofzeros,topreventPITrecoveriesusingbadRBA/LRSNfromfailing(RC=8inUTILINITphaseinstead)
– SanitycheckalsoperformedinDB210(coexistence)– Supportfor‘NOBASIC’valueforOBJECT_CONVERSIONZPARMtopreventconverting
backpagesets inextendedformat,andto‘EXTENDED’asdefaultif‘NOBASIC’issetandcatalogcolumnis<>‘E’
– AddLRSNvaluestoarchiveloginformationinREPORTRECOVERYutilityoutput– Technicalwhitepaperproducedexplainsabout‘6/10byteLRBA/LRSNhandling’– SeveralenhancementstoDB211books
122
ExtendedLRBA/LRSN…• Recommendedbestpracticemigrationstrategy
1. Runpre-migrationjobsandstepstoclean-up2. MigrationtoDB211CM3. MigrationtoDB211NFM4. ConvertALL BSDSofdatasharinggroupwithin‘n’weekends5. ReorgALL Directory&CatalogPagesets to‘extendedLRBA/LRSNformat’6. SetOBJECT_CREATEandUTILITY_CONVERSIONZPARMstoEXTENDED
- Newobjectswillbecreatedin10byteextendedformat- REORG,LOADREPLACEandREBUILDwillconvertuserobjectstoextendedformatwithout
needtochangeutilitycontrolstatements
7. ReorgallobjectstoextendedLRBA/LRSNformatbyexecutingnormalreorgjobsorsomeadditionaljobs• PerformregularcheckforongoingprogressbyselectingrowswhereRBA_FORMAT=‘E’in
SYSIBM.SYSxxxxPART
8. Ifalldone,setOBJECT_CONVERSIONZPARMtoNOBASIC
123
ExtendedRBA/LRSN– Conversionstrategies
124
DB211CMDB210NFM DB211NFM ConvertBSDS
DataSharing– EndLRSNproblem
ConvertDB2Objects
DB211CMDB210NFM DB211NFM ConvertDB2Objects
NonDataSharing– EndlogRBAproblem
ConvertBSDS
DB211CMDB210NFM DB211NFM ConvertBSDS
DataSharing– EndlogRBAproblemonlyononeormoremembers
• ConvertingtheBSDSimplicitlyconvertstherecoverylogtoextendedformat.
ConvertingtheBSDSupfrontwillprovideincrementallybenefitfromLRSNspinavoidanceasobjectsaregraduallyconvertedtoextendedformatbyREORGutilities
LeaveconvertingoftheBSDSuntiltheveryendwillavoidadditionalloggingacceleratingtowardstheendoflogRBA
ExtendedLRSNSummary• DistinguishbetweenendofLogRecordSequenceNumber(LRSN)andendofLogRelativeByteAddress(RBA)
• WithDB2V11internalcodingdealswith10byteextendedformatLRBA/LRSNvaluesonly
• MonitortheboundariesofLRBAandLRSNvaluesandtakeactionifneeded.– DifferentchoiceswithRBAlimits,butonlyoneforendofLRSN
• NoticethatallDB2objectsofagivenneedtoREORGEDtogetconvertedto10ByteLRSN
• FollowbestpracticeshaveasolidmigrationstrategyandmigrationplanforRBAandLRSNconversionto10Byte.
• CPUreductionfromavoidingLRSNspinisatmostcustomerinstallationnotthatsignificant.Thereshouldbeabalance,e.g.
– IfnotenforcedbyapproachingLRSNand/orRBAwhygoingthroughprocess of converting DB2 Cat/Dir and application objects to extended format.
125
OVERALLSUMMARY
126
OverallSummary• Sharelessonslearned,surprises,pitfalls
• Providehintsandtips
• Addresssomemyths
• Provideadditionalplanninginformation
• Provideusageguidelinesandpositioningonnewenhancements
• Helpcustomersmigrateasfastaspossible,butsafely
127
DB211Resources• IBMInformationCenter/KnowledgeCenter• DB211TechnicalOverviewRedbook(SG24-8180)• DB211forz/OSPerformanceTopics(SG24-8222)• DB211links:https://www.ibm.com/software/data/db2/zos/family/db211/
– LinkstoDB211AnnouncementLetter,webcastsandcustomercasestudies– IBMDB211extendedLRSNandRBAformatsupportenhancesavailabilityandaddresses
businessrisk– Whitepaper:“DB211forz/OS:UnmatchedEfficiencyfor
BigDataandAnalytics”– Whitepaper:“HowDB211forz/OSCanHelpReduce
TotalCostofOwnership”
• DB211MigrationPlanningWorkshop– http://ibm.co/IIJxw8
• FreeeBookavailablefordownload– http://ibm.co/160vQgM
• “DB211forSAPMissionCriticalSolutions”– http://scn.sap.com/docs/DOC-50807
128
129
Recommended