©2017Percona1
PeterZaitsev
Innodb PerformanceOptimizationMostimportantpractices
CEOPerconaTechnicalWebinarsDecember20th,2017
©2017Percona2
AboutthisPresentation
InnodbArchitectureandPerformanceOptimization
3hIn-depthtutorialdeliveredatPerconaLiveConferences
Thisisthemostimportanthighlights+somenewmaterial
©2017Percona3
PerformanceOptimizations
WhataretheGoalsof“PerformanceOptimization”?
©2017Percona4
GoalsofPerformanceOptimization
• PerformanceMakeQueriesRunFaster
• AndSaveCostsImproveEfficiency
• Run“Larger”applicationsonthesamesystem
AchieveScalability
©2017Percona5
ThisPresentationFocus
GettingmostoftheSingleNode
NottouchingArchitectureOptimizations
©2017Percona6
OptimizationAreas
Hardware/Infrastructure
OperatingSystems
MySQLVersion
MySQLSettings
Queries
Schema
©2017Percona7
HardwareandInfrastructure
©2017Percona8
Hardware/InfrastructureOptimizations
CPU
Memory
Storage
Network
©2017Percona9
MostSuggestionsapplytoCloudandNon-CloudEnvironments
SpecificRecommendationsforAWSCloud
GeneralandAmazonCloud
©2017Percona10
CPUforInnodb
FasterCPUsareBetter
MostlySingleThreadedhandlingofasingleQuery
Multi-CoreforHighlyConcurrentWorkload
MySQLnowscalesto64+Cores
©2017Percona11
MySQL/Innodb MultiCoreScalabilityhttps://www.mysql.com/why-mysql/benchmarks/
©2017Percona12
HowmuchConcurrencydoyouHave?StatusVariablethreads_running isagreatindicator
©2017Percona13
OKbutwhataboutCPU?Numberof“runnable”processesinVMSTATgoodproxy
©2017Percona14
CPUonAWS
•GeneralPurposeM5InstanceType
• FasterCPUbutnotasmuchmemoryC5InstanceType
•MoreMemorybutlessCPUR4InstanceType
• EvenMoreMemoryperCPUX1/X1eInstanceType
• BurstableCPUforlightworkloadsT2InstanceType
©2017Percona15
Memory
Memoryismostlyusedforcaching
Trytofit“WorkingSet”inmemory
CachingImprovesReadsandWritesPerformance
©2017Percona16
MemoryandStorageCapacityPlanning
• HitratioisuselessThinkaboutMisses(StorageIO)
• DifferentPerformanceProfileThinkaboutReadsandWritesseparately
• DonotpushmoreIOthandevicecanhandleThinkThroughput
• HowmanyIOquerycandoandstillbefast?ThinkLatency
©2017Percona17
Writes
DelayingWritesCan’tgoonforever
BehindManyWritesthereis
aRead
©2017Percona18
FlushingImpactsPerformanceFasterPerformanceatFirst,UntilFlushingneedstohappen
©2017Percona19
Innodb ChangeBuffer– LongWarmup
©2017Percona20
Storage
EvenFastestStorageisalotslowerthanMemory
StorageIOhastobedonein16Kpages(bydefault)
Memoryaccessescanread/writefewbytes
©2017Percona21
TypeofStorage
UseSSD
WheneverDirectlyAttachedornot
LocalNVMeandIntelOptanearefastest
©2017Percona22
Storage
IOlatencyiscritical
Whatisoptimalconcurrency(queuedepth)
ReadsvsWrites
WritesandFlushes
WriteCache,butDurableWriteCache
©2017Percona23
ReadandWriteLatency
©2017Percona24
UnderstandingIOConcurrency
©2017Percona25
AWSStorageChociesEBSio1(Provisioned
IOPS)
• Upto500MB/secand32.000IOPSpervolume
• ScaleIOcapacityindependentlyfromthesize
• Expensive
EBSgp2
• Burstableforsmallvolumes(lessthen1TB)
• Performancescaleswithsize– 3IOPSperGB
LocalNVMe(i3instancetype)
• NonRedundant• PerformanceandSpaceDependsonInstanceSize
• Upto3.3MIOPSand16GB/sec
©2017Percona26
PXCBestPracticesonAWSExtensiveBenchmarksdonehttp://bit.ly/2zeRpSZ
©2017Percona27
Network
“FrontEnd”Network
•TotheMySQLClient(Application)
“BackEnd”
•ForNetworkAttachedStorage
©2017Percona28
NetworkPerformance
• TheMostCriticalLatency• CanbebottleneckforLargeDataTransferBandwith
• NetworkperformanceisnotconstantVariance
©2017Percona29
NetworkinAWS
SingleAvailabilityZone(AZ)
• LowestLatency(Fastest)
SingleRegion,DifferentAZ
• MediumLatency
MultipleRegions
• HighestLatency(Slowest)
©2017Percona30
HardwareRelatedInnodb Settings• Innodb_buffer_pool_size
• HowmuchmemorytoallocateforCaching• Innodb_buffer_pool_instances
• Increaseupto16ifyouhavelargenumberofCPUcores• Innodb_flush_method=O_DIRECT
• BypassOScache.GoodchoiceformostIOsubsystems• Innodb_flush_log_at_trx_commit
• 1– durable.0– notdurable.2– inbetween• sync_binlog
• 1– binlog durabilitybuthighperformancecost• innodb_io_capacity
• TellInnodb howmuchIOstoragesystemcanhandle
©2017Percona31
OperatingSystem
©2017Percona32
Linux
ModernVersion
CloudOptimizedifRunningintheCloud
RHELcompatibleandDebian/Ubuntuaremostcommon
©2017Percona33
OperatingSystemSettings• NumberofOpenFilesforMySQLProcess
• Ifusinglargenumberofconnectionsorlargenumberoftables• Filesystem
• XFSorEXT4• Mount–onoatime
• DiskScheduler• DeadlineorNoop• cat/sys/block/sda/queue/scheduler• NVMe StorageDoesnothaveScheduler
©2017Percona34
OperatingSystemSettings• NUMA
• OnlyonNUMA(Multi-SocketHardware)• MySQL5.7Optioninnodb_numa_interleave=1• PerconaServer5.5+numa_interleave=1
• CPUGovernor• SavingPowercancostyouperformance• cat/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor• cpufreq-set-r-gperformance
©2017Percona35
Swapping• ShouldyouEnableSwap?
• WouldyoupreferMySQLtobekilled(crash)duetolackofMemory?• WouldyoupreferMySQLtoslowdownwithlackofMemory?• WouldyouwantunusedstufftobeswappedoutfromMemory?
• Ifusingswap• Setvm.swappiness=1• echo'vm.swappiness =1'>>/etc/sysctl.conf
©2017Percona36
SwaponSSDisbadbutnotthatBadhttp://bit.ly/2BJBTAJ
©2017Percona37
MySQLVersions
©2017Percona38
MySQLVersionBestPractices
UseModernVersion(MySQL5.7)
ConsiderPerconaServer
©2017Percona39
BewareSingleThreadPerformanceRegressionshttp://bit.ly/2oMvu2a
©2017Percona40
MySQLConfiguration
©2017Percona41
SettingstoThinkAbout• HowManyConnectionsdoyouneed?
• max_connections=N• HowManyTablesdoyouhave?
• table_open_cache=X• table_definition_cache=Y
• WhatCharacterSetdoyouuse?• character-set-server=utf8• collation-server=utf8_general_ci
• HowLongtoKeepBinaryLog• log_bin• expire_log_days=N
©2017Percona42
ConnectionandTableStatus
©2017Percona43
SettingsforInnodb• IsAdaptingIndexHelpingyouorHurtingyou?
• innodb_adaptive_hash_index=1/0• innodb_adaptive_hash_index_parts=N
• HowmuchspacetoallowforChangeBuffer• innodb_change_buffer_max_size=5
• HowmuchofBufferPooltoReload?• innodb_buffer_pool_dump_pct=80
• DoubleWriteCanbedisabledonZFS• innodb_doublewrite
• DisableInnodb FlushNeighborsifrunningonSSD• Innodb_flush_neighbors=0
©2017Percona44
ChangeBufferandAdaptiveHashIndex
©2017Percona45
SettingsforInnodb• HowMuchIOtoGiveInnodb
• Innodb_io_capacity• Innodb_io_capacity_max
• Howquicklytogiveupwaitingforlockandreturnerror?• innodb_lock_wait_timeout
• LogBufferForcachingWrites• innodb_log_buffer_size=32M
• TradeoffofWritePerformanceandRecoverTime• Innodb_log_file_size=1G
• ProtectionfromPurgeLag• innodb_max_purge_lag=10000000• innodb_max_purge_lag_delay=10000
©2017Percona46
LogfileandLogBuffer
©2017Percona47
SettingsforInnodb• Innodb BufferPoolScanResistance
• innodb_old_blocks_pct=70• Innodb_old_blocks_time=5000
• HowmanyfilesInnodb cankeepopen• innodb_open_files=N
• SmallerPagesizesbetterforsomeworkloads• innodb_page_size=4096
• Onlocktimeoutrollbacktransactionorstatement?• innodb_rollback_on_timeout
• SpeedupInnodb IndexBuildsifyouhavememory• innodb_sort_buffer_size
©2017Percona48
SettingsforInnodb• LongerSpinscanreduceContextSwitching
• innodb_spin_wait_delay• HowInnodb IndexStatisticsworks
• innodb_stats_persistent• innodb_stats_persistent_sample_pages• innodb_stats_transient_sample_pages
• ExcessiveContentionProtection• innodb_thread_concurrency• OruseThreadPool
• Innodb RandomReadAheadcanbehelpful• innodb_random_read_ahead
• Innodb IOConfiguration• Ensureinnodb_use_native_aio ison• innodb_read_io_threads• innodb_write_io_threads
©2017Percona49
IsRead-AheadHelpingorHurting
©2017Percona50
QueriesandSchema
©2017Percona51
DataisClusteredbyPrimaryKey
UsePrimaryKeys
UseShortPrimaryKeys
UseSequentialPrimaryKeys(ieauto_increment)
PrimaryKeyLookupsarefaster(especiallyranges)
©2017Percona52
SecondaryKeysRefertorowbyPK
SecondaryKeyLookupsareslower
Longprimarykeys=bloatedsecondarykeys
Primarykeyessentiallyappendedtoprimarykeys
KEY(A)meanKEY(A,ID)forQueries
©2017Percona53
MultiVersionConcurrencyControl(MVCC)Readersdonotblockwriters
Writersdonotblockreaders
Readersread“stale”data
LockingReadsareavailablewhencurrentstateisneeded
OldversionsofrowsarekeptinseparateUndoSpace
Oldindexentriesarekeptintheindexes
PurgeThreadsareconstantlycleaningoldgarbage
©2017Percona54
OptimizingforMVCC
AvoidLongrunningReadTransactionsconcurrentwithHeavyWrites
AvoidTransactionswhichmodifylargeamountofrows
AvoidveryHotRowsastheycangeneratelongUndoChains
©2017Percona55
Repeatable-ReadvsRead-Committed
REPEATABLE-READ
• DefaultIsolationMode• Viewthedataatthepointoffirstselectintransaction
• CanadditionallydelayPurgeProcess
READ-COMMITTED
• Viewdataatthepointofstartofthestatement
• Canhelptoreduceunpurgedbloat
• Canincrease“transactionallocationoverhead”athighconcurrency
©2017Percona56
UnderstandingTopQueries
©2017Percona57
Understandingwhatmakesthemslow
©2017Percona58
WherealltheGraphsComeFrom?
PerconaMonitoringandManagement(PMM)
http://bit.ly/GetPMM
http://pmmdemo.percona.com
©2017Percona59
SAVE THE DATE!
CALL FOR PAPERS IS NOW OPEN!www.perconalive.com
April 23-25, 2018Santa Clara Convention Center
DATABASE PERFORMANCEMATTERS
DatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMatters