Harnessing Power of Parallelism in an Oracle Data …Days...Harnessing Power of Parallelism in an Oracle Data ... § Financial Services, Manufacturing, Health/NutriGon ... • Parallel

•ManagedServices•CloudServices•Consul3ngServices•Licensing

HarnessingPowerofParallelisminanOracleDataWarehouseUTOUGTrainingDays2016KaseyParkerSr.EnterpriseArchitectKasey.Parker@centroid.com

ManagedServices

CloudServices

Consul3ngServices

Licensing


Leader in Technology, Consulting and Managed Services since 1997

•  Part of Oracle’s Top 25 Strategic Partner Program •  Focused on Consulting, Managed Services, Cloud Services and Resell •  Capabilities From Oracle Applications to Technology to Infrastructure •  Oracle Engineered Systems & Tech Partner of the Year Winner: 2014 & 2015

Specializations Oracle Database & Core Technologies Oracle Engineered Systems Oracle Server & Storage Solutions Oracle Performance Tuning Oracle Data Warehousing Oracle Business Intelligence Oracle Applications

Top 25 Strategic Partner

CentroidOverview


AboutKaseyParker

§  ResideinPlano,TXwithwifeand5kids–recentlymovedfromUtah§  OracleArchitect/DBA

§  Over15YearsofOracleExperience§  OracleCerGfiedExadataSpecialist§  OracleCerGfiedProfessional-DBA§  PerformanceTuningspecialist§  DataWarehousingspecialist§  Last5yearsfocusedonOracleEngineeredSystems

§  Manyindustries§  FinancialServices,Manufacturing,Health/NutriGon,City/CountyGovernment,Retail

§  ServeonBoardoftheUtahOracleUsersGroup(UTOUG)§  Academic

§  BrighamYoungUniversityAlumnus§  ManagementInformaGonSystems

QUICKFACTS


Agenda

§  OracleParallelOverview&Architecture§  Implemen3ngParallelism§  ControllingParallelism§ MonitoringParallelism§  Summary


ParallelArchitectureSimpleModel


ParallelArchitectureSimpleModel

QueryCoordinator

PXServers


ParallelArchitectureProducer/ConsumerModel


ParallelArchitectureProducer/ConsumerModel

QueryCoordinator

PXProducers

PXConsumers


ParallelArchitectureDataRedistribuOon§  Dataisredistributedbetweenproducers/consumers§  SeveralParallelDistribu3onMethods

§  E.g.Hash,Broadcast,Range§  Methoddeterminedbasedonobjectsta3s3cs

§  Adap3veParallelDistribu3on§  HybridHashdistribu3onmethod§  Methodcanchangeatrun-3me§  ViaAdap3veExecu3onPlans(Sta3s3csCollector)

§  ParallelPar33on-wiseJoins§  Minimizedatadistribu3onforbeTerperformance


Implementing Parallelism Data Warehouse or Mixed Workload Database


ImplemenOngParallelismParallelismandParOOoning§  BesttoPar33onTablesUsedWithParallelism

§  Par33on-wiseJoins+Parallel=GreatPerformance

§  Par33onLargeFactandDimensionTables§  Facts

§ RangePar33onbyLoadFrequency§ HashSub-par33onbyCommonJoinKey

§  Dimensions§ HashPar33onbyJoinKey


ImplemenOngParallelism§  ParallelismEnabledbyDefaultfor

§  ParallelQuery§  ParallelDDL

§  ParallelDMLMustbeExplicitlyEnabled§  AtSessionLevel:

§ ALTER SESSION ENABLE PARALLEL DML;§  AtStatementLevel(Newin12c)

§  ENABLE_PARALLEL_DMLHint:§  /*+enable_parallel_dmlparallel(x)*/

INSERT /*+ parallel(8) enable_parallel_dml */ INTO t1 SELECT * FROM t2;


ImplemenOngParallelismTwoMethodstoManageDegreeofParallelism(DOP)§  AutoDOP§  ManualDOP§  ControlledByPARALLEL_DEGREE_POLICY

§ MANUAL(Default)§  LIMITED§  AUTO§  ADAPTIVE


ImplemenOngParallelismPARALLEL_DEGREE_POLICY•  MANUAL

§  StatementsparallelizedbyhintorgivingobjectaDOP§  Note:IfhintspecifiesnoDOP–AUTODOPCalcwillbeused

•  LIMITED–  AutoDOPCalcula3onOnlyUsedwithDefaultDOP–  ParallelStatementQueuingandIn-memoryParallelDisabled

•  AUTO–  AllAutoParallelFeaturesEnabled–  AutoDOPCalcula3onRegardlessofHintsorObjectDOP

•  ADAPTIVE(Newin12c)–  SimilartoAUTO,butquerymay“Adapt”basedonstatsfeedback


ImplemenOngParallelismAutoDOP•  OracledecideswhethertoParallelizeandtheDOP•  ControlledBy

•  ResourceRequirements(Cost)•  PARALLEL_MIN_TIME_THRESHOLD

•  Default:AUTO(10seconds)•  Statementsexpectedtorunlongerarecandidatesforparallel

•  ShouldCollectIOCalibra3onStatsBeforeUsing•  DBMS_RESOURCE_MANAGER.CALIBRATE_IO•  Recommendedfor12c,Requiredfor11gR2

•  AlsoEnables•  ParallelIn-Memory•  ParallelStatementQueuing


ImplemenOngParallelismAutoDOPDecisionPath(withoutDBRM)

SlideimagecourtesyofOracle


ImplemenOngParallelismAutoDOP:ParallelIn-Memory•  Tradi3onally,ParallelQueryUsesDirectReads

•  DatareadintoPGA,bypassingBufferCache•  ParallelIn-memoryUsesBufferCacheforParallelQuery•  ChallengesandLimita3onsin11gR2•  SignificantlyEnhancedin12c

•  Automa3cBigTableCaching(ABTC)•  SeparatePartofBufferCacheforParallelIn-Memory•  ControlledBy:DB_BIG_TABLE_CACHE_PERCENT_TARGET

•  %ofSGAreservedforParallelIn-Memory

•  UsesObjectBasedAlgorithm(opposedtostandardLRU)

•  HighlyRecommendedIfUsing12cIn-MemoryOp3onwithRAC•  EnablesQCtobeIMCUInstanceLoca3onAware


ImplemenOngParallelismAutoDOP:12cParallelIn-MemoryandRAC•  ABTCSplitsObjectsAcrossRACNodes

•  SharedNothingArchitecture•  EachFragmentMappedtoSpecificNode

•  RangeofBlocksorIndividualPar33ons•  AllAccessforFragmentIsolatedtoItsNode

•  PXServersonNodeAccessDataandReturntoQC•  NoDataMovedViaCacheFusion

•  WorksSameWayforIn-MemoryIMCUsonRAC


ImplemenOngParallelismAutoDOP:12cParallelIn-MemoryandRAC

QueriesdirectedtoPXserversonnodecontainingthepar33on

SlidematerialcourtesyofOracle


ImplemenOngParallelismAutoDOP:ParallelStatementQueuing•  LimitsConcurrentParallelProcesses

–  Queuesun3lenoughslavesavailable•  Avoidsoverwhelmingserverwithparallelprocesses•  Deliversamoreconsistentperformanceprofile•  PARALLEL_SERVER_TARGETcontrolswhenqueuingstarts

–  Default:8*CPU_COUNT*PARALLEL_THREADS_PER_CPU•  StatementsqueuedinFIFOmethodbydefault

–  Butcanbepriori3zedusingDBRM•  BypassedBy

–  NO_STATEMENT_QUEUINGHint–  DBRM:PARALLEL_STATEMENT_CRITICAL=BYPASS_QUEUE


ImplemenOngParallelismAutoDOP:ParallelStatementQueuing


ImplemenOngParallelismAutoDOP:ControllingtheDOPPARALLEL_DEGREE_LEVEL•  AllowsControllingtheAutoCalculatedDOP

•  UseifAutoDOPisTooAggressiveorConserva3ve•  SettoPercentageofCalculatedDOPDesired

•  Default=100•  10=Reduceto10%ofCalculatedDOP•  200=DoubleCalculatedDOP


ImplemenOngParallelismAutoDOP:ControllingtheDOPPARALLEL_DEGREE_LIMIT•  LimitsMaxDOPforAUTODOPCalcula3ons•  Default=CPU

•  UsesDefaultDOPforDatabase•  PARALLEL_THREADS_PER_CPU*SUM(CPU_COUNT)

•  CPU_COUNTforAllRACNodes•  ACTUALDOP=MIN(IDEALDOP,PARALLEL_DEGREE_LIMIT)•  UseDBRMforMoreFine-grainedControl


ImplemenOngParallelismManualDOP•  YouManageParallelExecu3on•  FixedDOPorDefaultDOP

•  DefaultmeantforSingle-userWorkloads•  StarvesSystemResourcesinMul3-userEnvironments•  PARALLEL_THREADS_PER_CPU*SUM(CPU_COUNT)•  SetBy:

•  ALTER TABLE [TableName]PARALLEL;•  SELECT /*+ parallel(default) */…

•  FixedDOPisPreferred•  AllowsYoutoChooseSpecificDOP•  SetBy:

•  ALTER TABLE [TableName]PARALLEL 16;•  SELECT /*+ parallel(8) */…


ImplemenOngParallelismManualDOP:FixedDOP•  Parallelismisforlarge,resource-intensiveopera3ons•  Designedtobreakalargetaskintosmallertasks•  RulesofThumb

•  BeDeliberatewithDOPbySeqngonTable•  UseaPowerof2forDOP•  TableDOPGuidelines

•  >200MB=DOPbetween4and16•  Exadata:>100MB

•  >5GB=DOP32•  Exadata:DOP16-32


ImplemenOngParallelismManualDOP:FixedDOP

•  IndexDOPGuidelines•  OracleRecommendsUsingSameasTableDOP•  Exadata:GenerallyDon’tUse•  AdjustDOPAxerParallelIndexDDL

•  ResultsWillVary-Test•  TestwhatDOPoffersbestperformancefortheresources•  Exadata:GenerallyneedlessDOP,butsmallertablesbenefit


ImplemenOngParallelismManualDOPvsAutoDOP•  AutoDOPWorthEvalua3ngWith12c

•  Avoidin11g•  AvoidwithExadata(unlessIn-MemoryOp3on)•  UseifUsingIn-MemoryOp3ononRAC•  TweakasNecessaryUsingPARALLEL_DEGREE_LEVEL•  BestOp3onforMixedwithSignificantOLTPWorkload

•  ManualDOPS3llProvidesMostControlledResults•  BeDeliberateinSeqngDOPonObjects•  PossibletoS3llUseParallelStatementQueuing

•  _parallel_statement_queuing=TRUE


Controlling Parallelism i.e Harnessing its Power


ControllingParallelismKeyInitParameters•  PARALLEL_ADAPTIVE_MULTI_USER

•  ThroTlesDOPBasedonConcurrentWorkload•  Default:TRUE•  Recommend:FALSE

•  PARALLEL_THREADS_PER_CPU•  NumberofparallelthreadsaCPUcanhandle•  Default:2•  Considerseqngto1onsystemswithhyper-threading

•  1isrecommendedforExadata


ControllingParallelismKeyInitParametersconOnued…•  PARALLEL_MAX_SERVERS

•  Max#ofinstanceparallelserverprocesses•  Default:CPU_COUNT*PARALLEL_THREADS_PER_CPU*20

•  IfusingSGA_TARGETorMEMORY_TARGET•  Mayneedtolowerifhighconcurrency

•  PARALLEL_MIN_SERVERS•  Min#ofinstanceparallelserverprocesses(default=0)•  Helpscontroloverheadofcrea3nganddestroyingPXProcesses•  Recommendseqngtohighdailyaverageofparallelslaves


ControllingParallelismDBRM:ManagingConcurrentParallelProcessing•  EnablesUsingParallelisminMixedWorkloadEnvironment•  Control#ofParallelServers

•  StatementLevel•  DOPLimit

•  ConsumerGroupLevel•  Max%ofPARALLEL_SERVERS_TARGET

•  Priori3zeParallelStatementsAcrossConsumerGroups•  SeparateQueuesforStatementQueuing


ControllingParallelismDBRM:Overview

Element DescripOon

ResourceConsumerGroup Containerforagroupofsessionswithsameresourcerequirements,towhichDBRMallocatesresources

ResourcePlan Containerfordirec3vesthatspecifyhowresourcesareallocatedtoconsumergroups

ResourcePlanDirec3ve Associatesaconsumergroupwitharesourceplanandhowresourcesareallocated


ControllingParallelismDBRM:PlanDirecOveParallelControls•  LimitDOPforAnyParallelOpera3oninaConsumerGroup

•  PARALLEL_DEGREE_LIMIT_P1direc3veaTribute•  Limit#ofParallelServersUsedinaConsumerGroup

•  PARALLEL_SERVER_LIMITdirec3veaTribute•  SettoMax%ofPARALLEL_SERVER_TARGET•  AffectsWhenParallelStatementsareQueued

•  ParallelStatementQueuingControls•  AllowParallelStatementstoBypassParallelQueue

•  PARALLEL_STMT_CRITICALdirec3veaTribute•  SettoBYPASS_QUEUE


ControllingParallelismDBRM:PlanDirecOveParallelControlsConOnued…

•  Priori3zeParallelStatementSelec3onForConsumerGroups•  SeparateParallelQueuesforEachConsumerGroup•  MGMT_Pndirec3veaTribute

•  HigherValueIncreasesPriorityofStatementSelec3onFromQueue•  SameDirec3veToSpecificCPU(andIOonExadata)%

•  LimitTimeaStatementWaitsinParallelStatementQueue•  PARALLEL_QUEUE_TIMEOUTdirec3veaTribute•  SettoMaxTimeinSecondsaStatementWillWait•  TimeoutRaisesaORA-07454Error


ControllingParallelismDBRM:ParallelPlanDirecOveAXributesExample

BEGIN SYS.DBMS_RESOURCE_MANAGER.create_pending_area(); SYS.DBMS_RESOURCE_MANAGER.create_plan_directive( plan => 'DW_REPORT_CRIT_PLAN', group_or_subplan => 'DSS_CRITICAL_GROUP', mgmt_p1 => 90, parallel_degree_limit_p1 => 32, parallel_queue_timeout => 300, parallel_stmt_critical => bypass_queue); SYS.DBMS_RESOURCE_MANAGER.create_plan_directive( plan => 'DW_REPORT_CRIT_PLAN', group_or_subplan => 'OTHER_GROUPS', mgmt_p1 => 10, parallel_degree_limit_p1 => 12, parallel_queue_timeout => 300, parallel_server_limit => 50); SYS.DBMS_RESOURCE_MANAGER.submit_pending_area();END;


ControllingParallelismDBRM:ParallelPlanDirecOveAXributes–OEM


Monitoring Parallelism


Parallelism“Gotchas”•  ParallelDowngradeorSerializa3on•  ParallelDML–ForgeqngtoEnable•  Par3allyParallelizedSQL•  ParallelIndexCrea3on

–  Usebutdon’tforgettoremoveordropDOP

•  ParallelBulkLoadOpera3ons–  UsewithDirectPathLoading–  Findouthowtousewith3rdPartyETLtoolto:

•  EnableParallelDML•  DirectPath:i.e.CTAS,Insert/*+APPEND*/,BulkLoadAPI,etc


MonitoringParallelism

•  SQLMonitorReport–  AccessedviaEMorDBMS_SQL_MONITOR

•  GV$PX PROCESS–  OneRecordperParallelWorker

•  GV$SQL_MONITOR–  AlsoShowsQueuedParallelStatements

•  ParallelHealth-CheckandDiagnos3csReport–  pxhcdr.sqlscript(MOSNote:1460440.1)


MonitoringParallelismSQLMonitor:ParallelExecuOons


MonitoringParallelismSQLMonitor:QueuedParallelExecuOons


MonitoringParallelismSQLMonitor:ParallelExecuOonExample


MonitoringParallelismSQLMonitor:ParallelExecuOonExample


MonitoringParallelismSQLMonitor:ParallelExecuOonPlanNote


MonitoringParallelismSQLMonitor:ParallelDOPandDowngrade


MonitoringParallelismSQLMonitor:ParallelDOPandDowngrade

DowngradeReasons:•  350=duetoadap3veDOP•  351=duetoresourcemanagermaxDOP•  352=duetoinsufficientnumberofprocesses•  353=becauseslavesfailedtojoin


MonitoringParallelismSQLMonitor:PXServerWorkDistribuOon


MonitoringParallelismSQLMonitor:NotEnablingParallelDML


MonitoringParallelismSQLMonitor:ParOallyParallelizedSQL


Summary•  ParallelismOverview

–  SplitssingleDBOpera3onintoMul3pleConcurrentProcesses–  BeTeru3lizesavailablesystemresources–  Usefulforlarge,resource-intensiveopera3ons–  SuitedforDataWarehouseandMixedWorkloadDBs

•  WorksBestwithPar33oning•  AutoDOPvsManualDOP•  ControlwithInitParameters,DBRMandQueuing•  MonitorwithSQLMonitor


Ques3ons?


References•  ParallelExecu3onFundamentalsWhitePaper

–  hTp://www.oracle.com/technetwork/database/bi-datawarehousing/twp-parallel-execu3on-fundamentals-133639.pdf

•  ParallelExecu3onandWorkloadManagementWhitePaper–  hTp://www.oracle.com/technetwork/database/bi-datawarehousing/

twp-bidw-parallel-execu3on-130766.pdf

•  MonitoringParallelExecu3onUsingRSM–  hTps://blogs.oracle.com/datawarehousing/entry/

monitoring_parallel_execu3on_using_real

•  MOSNotes:–  UsingParallelExecu3on(DocID203238.1)–  Setup,Monitor,AndTuneParallelismInTheDB(DocID1549214.1)–  ParallelExecu3onHealth-CheckandDiagnos3cs(DocID1460440.1)–  DoYouNeedIOCalibrateStatsforAutoDOPin12C?(Doc1963310.1)

Documents

Harnessing Power of Parallelism in an Oracle Data …Days...Harnessing Power of Parallelism in an Oracle Data ... § Financial Services, Manufacturing, Health/NutriGon ... • Parallel