Migrations, Health Checks, and Support Experiences - Postgres from the Service Provider Perspective

© 2013 EDB All rights reserved. 1

Migrations, Health Checks and Support Experiences - Postgres from the Service Provider PerspectiveMarc Linster | April 2014

© 2013 EDB All rights reserved. 2

• EnterpriseDB Overview (quick)

• Migrations

• Postgres Support Experiences

• Health Checks

Why are we doing this?− Share some of our metrics with the community− Learn how to resolve customer problems faster− Drive our product strategy to make customer problems go away

Agenda

© 2014 EnterpriseDB Corporation. All rights reserved. 3

Postgres Plus Advanced Server Postgres Plus

Cloud Database

High AvailabilityPerformanceManagement

REMOTEDBA 24x7

SUPPORTPROFESSIONAL

SERVICES

TRAINING

EDB Serves All Your Postgres Needs

PostgreSQL

Security


POSTGRESinnovation

ENTERPRISE reliability

24/7 support

Services & training

Enterprise-class features & tools

Indemnification

Product road-map

Control

Thousands of developers

Fast development

cycles

Low cost

No vendor lock-in

Advanced features

Enabling commercial adoption of Postgres


Migrations


• Majority of interest for migrations from Oracle, followed by MS SQL, DB2 (few) and MySQL (rare)

• Obstacles− Missing language equivalents (e.g., autonomous_transaction,

merge)− Scalability (vertical and RAC-like)− Need additional constructs in compatibility layer (EDB

proprietary 10th generation)

• Greatly facilitated by Object Relational Frameworks, such as Hibernate – great indicator for easy migration to Postgres

• Experience shows that 70%+ of migrations can be executed quickly and result in comparable performance

Migrations


Postgres Support Experiences


• Adoption of Postgres is growing fast− More and more Postgres users are not not ‘dyed in the wool’ Postgres

users – they come from Oracle, MS SQL, DB2, etc.

• Postgres is moving into more complex and more mission-critical applications− Question patterns are changing from simple to complex− Expertise level is changing – when the customer calls, it is with a

(more) difficult problem or urgent issue

• How do we make Postgres users even more successful?− Targeted white papers− Targeted training for support team− Pro-active training for users as part of our customer enablement

process− Balanced product strategy of new features, new capabilities and

usability

Why Analyze Support Tickets


Ticket Volumes & Multi Year Growth

• Q1 2014: ~ 2500 total customers• Analysis in this presentation is based on a representative

sample of tickets raised over a 12 months period


Observations

• Postgres Plus Advanced Server: Strong emphasis on proprietary or commercially supported OSS

• PostgreSQL: Significantly less emphasis on proprietary or commercially supported OSS – even though the customer has commercially supported Postgres Distribution

Tickets and Platforms


When do customers call?


What Do Customers Call About

POC Dev Deployment Maintenance0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

What are customers calling about?

Tuning

Product Awareness

How to

Enhancements

Corruption

Bug

# Tickets PhaseCategory POC Dev Deployment Maintenance Grand TotalBug 8% 22% 9% 7% 10%Corruption 0% 0% 0% 6% 2%Enhancements 3% 4% 1% 0% 1%How to 41% 33% 41% 29% 35%Product Awareness 46% 39% 44% 39% 41%Tuning 3% 2% 5% 19% 10%Grand Total 100% 100% 100% 100% 100%


Where Do Users Need Help# Tickets PhaseCategory POC Dev Deployment Maintenance Grand TotalConnectors 0% 2% 1% 0% 3%

Bug 0% 0% 0% 0% 1%How to 0% 1% 0% 0% 1%Product Awareness 0% 1% 0% 0% 1%

Database 10% 9% 17% 37% 73%Bug 1% 2% 1% 2% 6%Corruption 0% 0% 0% 2% 2%Enhancements 0% 0% 0% 0% 1%

How to 4% 3% 6% 10% 23%Product Awareness 4% 3% 8% 15% 31%Tuning 0% 0% 1% 7% 9%

Replication 1% 4% 4% 2% 12%Bug 0% 1% 0% 0% 2%How to 0% 0% 3% 1% 4%Product Awareness 1% 2% 1% 0% 4%Tuning 0% 0% 0% 1% 1%

Utilities 1% 3% 6% 3% 12%Bug 0% 1% 1% 0% 2%Enhancements 0% 0% 0% 0% 0%How to 1% 1% 3% 1% 6%Product Awareness 0% 1% 3% 1% 5%

Grand Total 12% 17% 28% 42% 100%


• Deployment/Database/Product Awareness & How To − How to set up connection pooling?− How to set up Hot Standby and Streaming Replica?− How to add table spaces?− How to partition?− How to upgrade from major version?

• Maintenance/Database/Product Awareness & How To− How to find bloat in tables & indexes?− How to enable auditing selectively without performance impact?− What are the effects of changes to the values of postgresql.conf

file parameters wal_writer_delay, bgwriter_delay?− What is the correct configuration of hba_conf?

Select Top questions (not in rank order)


Lessons Learned from the AHC


• Systematic top-to-bottom review of the customer’s Postgres production installation

• Identifies issues and improvement opportunities in:− Security− Performance− Backups− Replication− Maintenance− Upgrades

• The following analysis is based on a set of 20 AHCs executed in 2013

Architectural Health Check


Red bars: experts expected top parameters, including

vacuum_cost_limit, log_line_prefix, log_min_duration_statement,

log_checkpoints


Kernel Parameters

vm.d

irty_

back

grou

nd_r

atio

vm.d

irty_

ratio

kern

el.s

hmm

ax

swap

pine

ss

kern

el.s

hmal

l

vm.o

verc

omm

it_m

emor

y

kern

el.m

m.tr

ansp

aren

t_hu

gepa

ge.e

nabl

ed

kern

el.m

m.tr

ansp

aren

t_hu

gepa

ge.d

efra

g0

2

4

6

8

10

12

14

16


Postgres Platform Distribution

PostgreSQL; 17

Post-gres Plus Ad-

vanced Server;

3

Postgres Plus Cloud Database; 1


Backup and Standby

hot47%

warm24%

None18%

cold12%

Standby ApproachPhysical Only36%

Logical Only29%

Physical and Log-

ical14%

FS Snap-shot7%

None14%

Backup Approach43% of customers could not execute PITR (prior to AHC) – some had NO backup in place

30% of customers would have been slow or unable to recover from failure


Maintenance Tasks

Unused Indexes39%

Bloated Indexes30%

Bloated Tables13%

Missing Indexes

9%

Missing PK9%

Bloat management continues to be a problem


• Windows was less prevalent (10%) than indicated in support tickets – may be due to a focus on Windows for development versus production

• Most DBAs do not appear to be familiar enough with the storage system/SAN

• OS System Adjustments (dirty_background_ratio & dirty_ratio) are also a major adjustment (70% & 45%)

• Customers would benefit from targeted education about Postgres-specific parameters and storage system/SAN guidance

Observations from the AHCs


• The analysis helped us understand where we need to collect more data during our support interactions to better pinpoint problem trends and drive knowledge base development

• This will drive our product development and feature prioritization so that we can make customer problems go away

Summary


• Many EDB Team Members contributed to this analysis− Thom Brown− Deepanshu Sharma− Robert Haas− Kevin Grittner− Gabrielle Roth

• Contact− Marc Linster

[email protected]+1 (617) 306 6059

Thank you

Internet

Migrations, Health Checks, and Support Experiences - Postgres from the Service Provider Perspective