Improve speed and performance of informix 11.xx part 2

Click to add text

© 2011 IBM Corporation September 12, 2011

Improve the Speed and Performance – Session 2

Samar Desai ([email protected]) 14th September 2011 3:00 pm – 4:00 pm

Information Management – IDS 11

© 2011 IBM Corporation 2

Talk Outline

Diagnostics – Performance History

– SQLTrace

– onstat’s

Storage Enhancements – Storage Provisioning

– Storage Optimization

Miscellaneous – Non-Blocking Checkpoints

– Other Performance

Benefits



Optimizing Data Sorting

Examine ways to make sorts run faster

How can the DBA find out information about

Sorting?

SORTING



Number of times the database server has

been asked to sort data Number of times the database server has

has sort data solely in memory

Number of times the database server has

has sort data using disk and memory

Information About Sorting

dbaccess sysmaster -

select * from sysprofile

where name matches "*sort*";

name value

totalsorts 2842

memsorts 2401

disksorts 441

maxsortspace 1234

The size in base pages of the larges sort



Sort Space and Type by User

Indicates a specific user/application a DBA can

examine for efficiency

dbaccess sysmaster -

select total_sorts, dsksorts,

max_sortdskspace from syssesprof;

sid total_sorts dsksorts max_sortdiskspace

11 0 0 0

13 23 0 0

14 3819 0 0



Avoiding Sorts

– Improve indexes on tables

– Sorts consume less resources then reading the

index when the number of index keys to read

compared to the index size is large

Narrow columns require less work to sort

Reduce the data to sort, only select the rows you

need

– SELECT FIRST N

– SELECT FIRST N SKIP M

Ways of Optimizing Sorts



OLTP or Small Sort Task

Tune DS_NONPDQ_QUERY_MEM

– The amount of memory given to sorts who have a

PDQ of 0

– Minimum Value 128KB

– Maximum Value 25% of DS_TOTAL MEMORY

Modify the parameter while online – onmode -wf DS_NONPDQ_QUERY_MEM=500

Example – A query with a sort would previously be given 128KB of memory to

sort data.

– If the sort exceeded 128KB of data to sort, it would go to disk

– Now the sort only goes to disk if the data sorted exceeds

DS_NONPDQ_QUERY_MEM



Dynamic Runtime Index Compaction

Attempts to merge two partially used index pages and

free one index page

Currently an index page must be more than 80% free

before Informix will attempt to look for merges

Provide an adjustable system wide default

Provide a way at an index fragment level to specific the

compaction level

A way to monitor the system and per index compaction

levels



Run Time Index Compaction Examples

Set the global runtime index compaction

BTSCANNER num=1,threshold=5000,rangesize=-1,alice=6,compression=high

onmode –C compression high

EXECUTE FUNCTION ADMIN("set index compression", "1048967", "high");

SELECT TASK("set index compression", partn, "low")

FROM dbs1:systables t, dbs1:sysfragments f

WHERE f.tabid = t.tabid AND f.fragtype = 'I' AND indexname ='idx1';

• Use the SQL Admin API to set a specific index fragment to

high mode compression

• Individual index compaction levels will be lost when server restarts

• Use a startup task to save between server restarts

• Change the compaction mode to low for all fragments of

index "idx1" in database "dbs1



What is a Index Page Merge?

Moves index information to adjacent page, unlinks from index and

removes pages from the index

When are index pages merged?

– Is index page 1 below the merge threshold?

– Read Index page #2

– Is index page 1 + index page 2 below the split threshold?

Page Header

Slot Table & Timestamp

Index page 1 is 5 % Full

Page Header

Slot Table & Timestamp

Index page 2 is 5 % Full

Index Page is free’d

YES

YES



Automatic Index Merge

Will the current index page

attempt an automatic index

merging??

Only the btree scanner

thread(s) merge index pages

Operation Before 11.50.xC2

11.50.xC2 and Later

Index Page which is 15% used No Merged if below threshold

Empty Index Page with deleted items

Merged if below 10%

Merged if below threshold

Empty Index Page w/o deleted items

No Merged if below threshold



ALICE Functionality

Autonomic Linear Index ClEaning

Reduces I/O done by the btree cleaner

Example of enabling ALICE

And performance response? A batch purge process that started

around 7pm, and finished at 3pm the next day. Since enabling

alice, that application is now finishing before 7am. So some

applications are seeing an 8 hour improvement. A second batch

report dropped from 28 hours to 12 hours.

To say the application team for this particular box is happy is a bit

of an understatement.

BTSCANNER num=2,threshold=5000,alice=6



Performance Diagnostics



Tool for Collecting Data for Specific Problems

New ifxcollect tool to collect diagnostic data if necessary for troubleshooting a specific problem

– An assertion failure

– A Performance Problem

Located in the $INFORMIXDIR/bin directory

Output files located in the $INFORMIXDIR/isa/data directory

Examples

– To collect information for a general assertion failure

ifxcollect –c af –s general

– To collect information for a performance problem related to CPU utilization

ifxcollect –c performance –s cpu

– To include FTP information, specify the additional information

-f -e user_name@company_name.org -p 9999.999.999

-f -m machine -l /tmp -u user_name -w password



Performance History



Performance History

Many new improvements

Two new categories of

statistics

– Memory

– Virtual Processor

Ability to compare

– Multiple statistics at one time

– Multiple servers at one time



Performance History – Multiple Statistics



Performance History - Multiple Servers



Viewing CPU Efficiency onstat –g glo

OS Scheduling problems

Give the DBA a clear picture of what is happening

MT global info:

sessions threads vps lngspins

1 26 7 51

sched calls thread switches yield 0 yield n yield forever

total: 1646766 1393829 23070 701894 143416

per sec: 0 0 0 0 0

Virtual processor summary:

class vps usercpu syscpu total

cpu 1 26.15 1.11 27.26

aio 2 1.44 3.40 4.84

lio 1 0.07 0.82 0.89

pio 1 0.02 0.59 0.61

total 7 27.68 5.97 33.65

Individual virtual processors:

vp pid class usercpu syscpu total Thread Eff

1 23322 cpu 26.15 1.11 27.26 185.03 14%

2 23326 adm 0.00 0.05 0.05 0.00 0%

3 23327 lio 0.07 0.82 0.89 34.74 2%

4 23352 pio 0.02 0.59 0.61 7.20 8%

5 23353 aio 1.12 2.30 3.42 78.70 4%



Viewing All Threads onstat –g ath

Remove/Reduce the number of thread status of

“sleeping forever”

Give the DBA a clear picture of what is happening

Threads:

tid tcb rstcb prty status vp-class name

2 10bbf36a8 0 2 sleeping forever 3lio lio vp 0

3 10bc12218 0 2 sleeping forever 4pio pio vp 0

4 10bc31218 0 2 sleeping forever 5aio aio vp 0

5 10bc50218 0 2 sleeping forever 6msc msc vp 0

6 10bc7f218 0 2 sleeping forever 7aio aio vp 1

7 10bc9e540 10b231028 4 sleeping secs: 1 1cpu main_loop()

8 10bc12548 0 2 running 1cpu tlitcppoll

9 10bc317f0 0 3 sleeping forever 1cpu tlitcplst

10 10bc50438 10b231780 2 sleeping forever 1cpu flush_sub(0)

11 10bc7f740 0 2 sleeping forever 8aio aio vp 2

12 10bc7fa00 0 2 sleeping forever 9aio aio vp 3

13 10bd56218 0 2 sleeping forever 10aio aio vp 4

14 10bd75218 0 2 sleeping forever 11aio aio vp 5

15 10bd94548 10b231ed8 3 sleeping forever 1cpu aslogflush

16 10bc7fd00 10b232630 1 sleeping secs: 26 1cpu btscanner 0

32 10c738ad8 10b233c38 4 sleeping secs: 1 1cpu onmode_mon

50 10c0db710 10b232d88 2 cond wait netnorm 1cpu sqlexec



Improved view of All Threads onstat –g ath

Threads: tid tcb rstcb prty status vp-class name

2 10bbf36a8 0 2 IO Idle 3lio lio vp 0

3 10bc12218 0 2 IO Idle 4pio pio vp 0

4 10bc31218 0 2 running 5aio aio vp 0

5 10bc50218 0 2 IO Idle 6msc msc vp 0

6 10bc7f218 0 2 running 7aio aio vp 1

7 10bc9e540 10b231028 4 sleeping secs: 1 1cpu main_loop()

8 10bc12548 0 2 running 1cpu tlitcppoll

9 10bc317f0 0 3 sleeping forever 1cpu tlitcplst

10 10bc50438 10b231780 2 IO Wait 1cpu flush_sub(0)

11 10bc7f740 0 2 IO Idle 8aio aio vp 2

12 10bc7fa00 0 2 IO Idle 9aio aio vp 3

13 10bd56218 0 2 IO Idle 10aio aio vp 4

14 10bd75218 0 2 IO Idle 11aio aio vp 5

15 10bd94548 10b231ed8 3 sleeping forever 1cpu aslogflush

16 10bc7fd00 10b232630 1 sleeping secs: 34 1cpu btscanner 0

32 10c738ad8 10b233c38 4 sleeping secs: 1 1cpu onmode_mon

50 10c0db710 10b232d88 2 IO Wait 1cpu sqlexec



Virtual CPU Time by User onstat –g cpu

Last time the thread executed on a VP

Display how much Virtual CPU time each thread has consumed

The number of times it ran on a VP

Thread CPU Info:

tid name vp Last Run CPU Time #scheds status

2 lio vp 0 3lio* 02/12 13:28:03 129.2453 7448 IO Idle

3 pio vp 0 4pio* 02/12 13:28:03 14.6759 428 IO Idle

4 aio vp 0 5aio* 02/12 13:28:03 35.5862 6800 IO Idle

5 msc vp 0 6msc* 02/11 23:57:05 8.1349 568 IO Idle

16 aio vp 4 11aio* 02/12 13:28:03 16.3634 1198 IO Idle

17 aio vp 5 12aio* 02/12 13:28:03 15.8946 119 IO Idle

18 aslogflush 1cpu 02/12 13:39:48 0.5008 94383 sleeping secs: 1

19 btscanner_0 1cpu 02/12 13:38:55 0.3053 3514 sleeping secs: 58

20 onmode_mon 1cpu* 02/12 13:39:48 0.6606 94385 sleeping secs: 1

41 dbScheduler 1cpu* 02/12 13:35:26 0.9059 555 sleeping secs: 38

42 dbWorker1 1cpu 02/12 13:25:25 8.4765 5206 sleeping forever

43 dbWorker2 1cpu 02/12 13:25:25 3.2069 3299 sleeping forever

46 bf_priosweep() 1cpu 02/12 13:38:04 0.3248 314 cond wait bp_cond

236 sqlexec 1cpu 02/11 16:10:43 0.0549 185 cond wait sm_read



Monitoring I/O Resources by File onstat -g iof

AIO global files:

gfd pathname totalops dskread dskwrite io/s

3 /dev/tsinfo1 1506 1393 113 0.0

4 /dev/tsinfo2 776 3 773 0.0

5 /dev/tsinfo0 22205 7994 14211 0.4

6 /dev/tsinfo3 55092 54102 990 1.1

dskread/dskwrite

Number of read/write system calls

io/s

Average I/O operations per second



AIO global files:

gfd pathname bytes read page reads bytes write page writes io/s

3 rootdbs 16384 8 456542208 222921 81.3

op type count avg. time

seeks 0 N/A

reads 0 N/A

writes 0 N/A

kaio_reads 8 0.0089

kaio_writes 872 0.0123

4 data_00 11231232 5484 0 0 182.9

op type count avg. time

seeks 0 N/A

reads 0 N/A

writes 0 N/A

kaio_reads 1371 0.0055

kaio_writes 0 N/A

Monitor I/O by service level

I/O broken down by individual operation

Monitoring I/O Resources by File onstat -g iof



Operating System Information onstat –g osi

Provides a platform independent way of collecting operating

system resources and parameter

Available in both sysmaster and onstat

onstat –g osi works when the system is not online

Machine Configuration....

OS Name Linux

OS Release 2.6.9-34.ELsmp

OS Node Name idas

OS Version #1 SMP

OS Machine x86_64

Number of processors 4

Number of online processors 4

System memory page size 4096 bytes

System memory 7970 MB

System free memory 1536 MB

Number of open files per process 1024

shmmax 33554432

shmmin 1

shmids 4096

shmNumSegs 2097152

semmap << UnSupported >>

semids 128

semnum 32000

semundo << UnSupported >>

semNumPerID 250

semops 32

semUndoPerProc << UnSupported >>

semUndoSize 20

semMaxValue 32767



History of Resource Usage

Average, maximum, minimum resource usage

Weekly resource usage

– 5 years tracked by week

– Recycled after 5 years



SQL Cache Information

SELECT name,

TRUNC(decode(hits,-1,0,0,0,100*hits/(hits+misses)),2) ||'%' AS HIT_RATE,

format_units(mem_used,'b') as used, format_units(mem_free,'b') as free,

total_entries, inuse_entries

FROM syssqlcacheprof

Stored Procedure Cache 99.84% 464 KB 16.3 KB 69 0

Data Distribution Cache 83.06% 202 KB 5.86 KB 32 0

Extended Type by Name 66.34% 73.4 KB 2.65 KB 46 9

Extended Type by ID 81.05% 73.4 KB 2.65 KB 21 0

Cast Cache 78.87% 58.8 KB 17.2 KB 54 1

Procedure Name Cache 0.00% 80.8 KB 7.23 KB 0 0

OPclass Instance Cache 0.00% 18.9 KB 1.13 KB 0 0

Routine Resolution Cache 0.00% 7.77 KB 4.23 KB 0 0

User Define Aggregregate 90.40% 4.77 KB 3.23 KB 75 0

Security Policy Info 98.56% 0.000 B 0.000 B 17 1

User LBAC credential 0.00% 0.000 B 0.000 B 3 0

Data Dictionary Cache 72.52% 1.73 MB 13.5 KB 2 281

Statement Cache 0.00% 336 KB 4.02 KB 0 313

External Directive Cache 0.00% 5.84 KB 2.16 KB 0 5



SQLTRACE



What is SQLTRACE?

SQLTRACE is a new feature in IDS 11 that allows you to collect SQL statement history information, including statistics and resource usage at the statement level.

SQLTRACE level=low,ntraces=1000,size=2,mode=global Each trace information includes

– Optimizer statistics (like SET Explain and EXPLAIN_STAT) • Iterator database and table names are available in MED mode

– Buffer and read/write statistics (like in onstat –p) – SQL info like onstat –g sql (database name, session id, database

isolation level, statement host variables) • Host variables are only available in HIGH mode

– Procedure stacks are available in HIGH mode



Query Drill Down onstat –g his

Database: sysmaster

Statement text:

select count(*) from systables,syscolumns where systables.tabid > ? and

systables.nrows < ?

SELECT using tables [ systables syscolumns ]

Iterator/Explain

================

ID Left Right Est Cost Est Rows Num Rows Partnum Type

3 0 0 17 42 146 1048579 Index Scan

4 0 0 5249 2366 2366 1048580 Seq Scan

2 3 4 5266 99372 345436 0 Nested Join

1 2 0 1 1 1 0 Group

Host Variables

==============

1 integer 100

2 float 1000.0000000000000000



Query Drill Down -- Continued

Statement information:

Sess_id User_id Stmt Type Finish Time Run Time

5 2053 SELECT 01:08:48 0.4247

Statement Statistics:

Page Buffer Read Buffer Page Buffer Write

Read Read % Cache IDX Read Write Write % Cache

65 75638 99.91 0 0 0 0.00

Lock Lock LK Wait Log Num Disk Memory

Requests Waits Time (S) Space Sorts Sorts Sorts

345729 0 0.0000 0.000 B 0 0 0

Total Total Avg Max Avg I/O Wait Avg Rows

Executions Time (S) Time (S) Time (S) IO Wait Time (S) Per Sec

1 0.8493 0.8493 0.4247 0.000203 0.0026 2.3548

Estimated Estimated Actual SQL ISAM Isolation SQL

Cost Rows Rows Error Error Level Memory

5266 1 1 0 0 CR 25504



Query Drill-down (cont’d)










Seq Scan is on the customer

table – rows processed is

16384.

Index Scan is on the orders

table – rows processed is 23.

What is this telling you ?



Query Drill-down: Add External Directive

Until you can run update statistics, you decide

to use external directives to improve query

performance.

** Check that the directive has been added to stores_demo: sysdirectives .

Adding and applying directives here is same executing following in stores_demo:

save external directives {+FULL(orders), INDEX(customer 100_1)} active for select c.city, c.state,

o.ship_date from customer c, orders o where c.customer_num =o.customer_num and c.state = ?;



Storage Provisioning



What is Storage Provisioning

To proactively or reactively add storage to eliminate out of space errors

– Monitoring spaces and automatically grow a container when its free space falls below a specific amount.

– Stalling an SQL which is about to fail because of insufficient space until space is allocated to the depleted container

The ability to tell Informix about disk space that can be used to solve storage issues in the future

– Raw Devices

– Cooked Files

– Directories



Benefits of Storage Provisioning

"Out-of-space" errors are virtually eliminated.

Manual expansion and creation of storage spaces without having to worry about where the space will come from

Automatic expansion of dbspaces, temporary dbspaces, sbspaces, temporary sbspaces, and blobspaces.

Feature is fully incorporated into OAT.



Storage Provisioning: The Power of 2

Two available modes: – Manual

– Automatic

Two available space expansion methods: – Chunk extension

– Chunk creation

Two available interfaces: – sysadmin task()/admin() functions (SQL interface)

– OAT (Graphical interface)



Storage Pool Facts

What is the Storage Pool – How the DBA tell’s Informix about space it can use to

solve future space issues

– A file, device, or directory in the pool is called an entry.

There is one storage pool per IDS instance.

You can add, modify, delete and purge storage

pool entries.

EXECUTE FUNCTION task("storagepool add", “/work/dbspaces/dbs1",

“0", “1GB", “100MB", “1");



OAT’s View of the Storagepool

Summary of space left

in the storage pool

Automatic policies



Extendable Chunks

The ability to expand an existing chunk

Default upon creation is non-expanding chunks

Example of enabling the extendable property of chunk 13

A Chunk can be extended automatically or manually

Example of manually extending chunk 27 by 2 GB

Extending chunks do NOT consume space from the

storagepool

EXECUTE FUNCTION task(“modify chunk extendable on”, “13”)

EXECUTE FUNCTION task(“modify chunk extend”, “27”, “2GB”);



OAT’s View of the Chunk Pod

Fragmentation map of

selected chunk Chunk Actions

• Extend a Chunk

• Add a new chunk

• Modify chunk settings

• Drop a chunk



Expanding a Storage Container

Keep the addition of space to a storage

container simple – The creator of a storage container specifies how

a space should grow

– Manual allocations of space, Just say do it

Use the predefined container provisioning

policies to allocated new space to a

container 1.Determines if any chunk in the storage container

is expandable

2.If no chunk can successfully expand, then add a

new chunk



Expanding a Space in OAT



Creating or Dropping a Space with the Storagepool

You can create a new storage container utilizing

the space from the storage pool

Example of create a 100MB dbspace called

orders_dbs

You can drop an existing storage container and

return the space to the storage pool

Example of dropping a dbspace called dbs1

EXECUTE FUNCTION ADMIN ('create dbspace FROM STORAGEPOOL', 'orders_dbs', '100M')

EXECUTE FUNCTION ADMIN ('drop dbspace to storagepool', 'dbs1');



Storage Optimization



Storage Optimization Overview

Defragment Extents – Combine extents reduce the

Data Compression – Reduces the amount of storage taken

by a single row

Table Compaction – Reduce the number of pages utilized by

a table

Automate the optimization of table

storage – Applies policies to optimize tables

49



The number of extents a

table/partition can have has increased

Defragment Extents

– Moves extents to be adjacent

– Merges the extents into a single

extent

Example

Customer Extent 3 2

Optimizing Table Extents - Defragment

Customer Extent 1

Customer Extent 2

Customer Extent 5

Customer Extent 4

Orders Extent 1

Items Extent 1

Orders Extent 2

Products Extent 1

Items Extent 2

dbspace1

Customer Extent 1

3

4

Customer Extent 1 MERGE

MERGE

2

3 EXECUTE FUNCTION ADMIN (‘DEFRAGMENT', ‘db1:customer')

Number of extents for

the customer table 5 4 3



Optimizing Tables and Indexes



Defragment Table Extents OnLine



Data Compression

Reduce the space occupied by the row

Compressing a table can be done online

Compress either a table or fragment

Custom dictionary built for each fragment to

ensure highest levels of compression

Tables with compressed rows are ALWAYS

variable length rows

• Many Benefits

Smaller Archives

More data in the buffer pool

Fewer long/forwarded rows

Few I/O for same amount of data

read/written

execute function task(“compress table”, “tab1”,”db”)



Performance Impact of Compression

IO-bound workloads – Compression may improve performance by reducing IOs

(both data page and logical log)

– More data fits on a page, so more in buffer pool

– Log records are smaller, so less logging

For CPU-bound workloads – Additional CPU used to compress and expand rows

– Should not be a large impact



REPACK Command

Moves all rows in a table/fragment

to the beginning, leaving all the

free space at the end of the table

Online operation, users can be

modifying the table

Tim

Frank

Chris

Jamie

John

Steve

Travis

Roy

Lenny

Customer

execute function task(“table repack”, “customer”, ”db”)



SHRINK Command

Frees the space at end of table so

other table can utilize this space – Entire extents are free

– The last extent in a table can be

partially freed

– Will not shrink a table smaller than the

first extent size • New command to modify first extent size

• “ALTER TABLE MODIFY EXTENT SIZE”

Online operation

Tim

Frank

Chris

Jamie

John

Steve

Travis

Roy

Lenny

Customer

execute function task(“table shrink”, “customer”, ”db”)



Optimizing Tables

As a DBA I need to … – Reduce the number of extents a table contains

– Move all rows to the beginning of a table

– Return unused space at the end of a table to the system

– Shrink a partial used extent at the end of a table

All this while accessing and modifying the table!!!

AND

While you are watching your favorite TV show



Automatically Optimize Data Storage

58



Miscellaneous



Checkpoints



What are checkpoints?

A checkpoint is a point in time

where modified data is flushed

from the bufferpool to disk to

create a physical consistency

point where the database

server can restart from in the

event of an unexpected outage.

– Frequent checkpoints reduce

fast recovery time

– Too many checkpoints can

consume excessive system

resources



Benefits of Nonblocking Checkpoints

Eliminate transaction blocking during

bufferpool flush

Relax LRU flushing

More frequent checkpoints enable shorter

fast recovery times



Automatic Checkpoint Tuning

When checkpoint blocking has occurred, the

system will adjust to trigger a checkpoint sooner

to avoid transaction blocking – Adjusts for worst case scenario

Automatic LRU tuning – When page replacement can’t find a free, low priority

page, LRU flushing will adjust to be 1% more aggressive



Tuning non-blocking Checkpoints

To avoid transaction blocking When transaction blocking takes place, its taking longer to

flush the bufferpool than there are critical resources to contain the transaction activity during checkpoint processing

– Increase physical log space • Changing the physical log size and location is now an online

feature – Increase logical log space – Improve I/O so buffers can be flushed faster – Make LRU flushing more aggressive



RTO_SERVER_RESTART

Recovery Time Objective – Configures the server to adjust checkpoint frequency to

maintain a target fast recovery time

– Automatically adjusts to fluctuations in work load

– Mutually exclusive with CKPTINTVL



Tuning for RTO_SERVER_RESTART

Goal is to fit all data needed by fast recovery into

the bufferpool – For small configurations, physical log is 110% of the

bufferpool

– For large configurations, physical log must contain

update working set



Other Performance Enhancements

Automated DB Scheduler tasks added to help with Performance – Timeout users that have been idle for too long” – Automatically allocate CPU VPs to match

hardware/licensing when IDS starts – Alerts for tables that have outstanding in-place – Ability to configure the automatic compressing, shrinking,

repacking, and defragmenting of tables and extents Large Page Support on Linux

– Previously, only AIX and Solaris systems were supported – The use of large pages can provide performance benefits

in large memory configurations



Software

Improve speed and performance of informix 11.xx part 2