My two cents about Mysql backup

My two cents about Mysql backup

Aleksandrs Asafovs

MySQL 5 Database Administrator

Oracle Database 11g Administrator Certified Master

Oracle9i Database Administrator Certified ProfessionalOracle Database 10g Administrator Certified ProfessionalOracle Database 11g Administrator Certified ProfessionalOracle Database 12c Administrator Certified Professional

Oracle9i Database Administrator Certified Professional - Special Accreditation: Managing Oracle9i on LinuxOracle RAC 11g Release 2 and Grid Infrastructure AdministrationOracle Database 10g Real Application Clusters Administrator Certified Expert

Oracle Application Server 10g Administrator Certified AssociateOracle Weblogic Server 11g: System Administration I

Certified Ethical Hacker(CEH)

About me

My name is Aleksandrs AsafovsWorking with Oracle for over 15 years

MySQL 5 years

Certificate experts 200+

Competence

5

8 GB RAM

35 GB database

SSD disk

Environment

Oracle VM 5.1.8Oracle Linux 7.2MySql 5.7.14

6

Environment

7

SQL Dump

mysqldump -u root -p --databases stream1 > backup-test-file.sql

#procedure or functionmysqldump -u root -p --databases stream1 --routines > backup-test-file.sql

#Users + point in time recoverymysqldump -u root -p --databases mysql > backup-test-file.sql

mysqldump -u root -p --databases stream1 --single-transaction --routines --master-data=2> backup-test-mysql.sql

8

SQL Dump

Full DB Backup

Restore with bin_logs

Restore without binlogs

Sql Dump

20m 1h36m 1h04m

File size 28G**The size before we use any compression,After pigz compression 3.0G

9

File system snapshotLogical Volume Manager(LVM)

#!/bin/sh#make snapshot script , execute manually

#script freeze DB, flush buffers , crate snapshot , then unfreeze DB

echo "Flushing tables"/usr/bin/mysql --login-path=/backup/user_l -e "FLUSH TABLES WITH READ LOCK;"

echo "Syncing"/bin/sync

echo "Show master status"/usr/bin/mysql --login-path=/backup/user_l -e "SHOW MASTER STATUS;"

echo "Creating snapshot"/sbin/lvcreate -L1G -s -n dbsnapshot /dev/ol/home

echo "Unlocking tables"/usr/bin/mysql --login-path=/backup/user_l -e "UNLOCK TABLES;"

echo "Mounting filesystem "mount /dev/ol/dbsnapshot /snapshot/ -onouuid,ro

10

File system snapshotLogical Volume Manager(LVM)

Demo_1

dsmc ba image -snapshotproviderimage=none /dev/ol/dbsnapshotdsmc rest image /dev/ol/dbsnapshot

11

Percona XtraBackup

Install Percona XtraBackup

https://www.percona.com/software/mysql-database/percona-xtrabackup

Create user for backup

mysql> CREATE USER 'bkpuser'@'localhost' IDENTIFIED BY 's3cret'; mysql> GRANT RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'bkpuser'@'localhost';

DEMO_2

https://www.percona.com/software/mysql-database/percona-xtrabackup

12

Percona XtraBackup

innobackupex --login-path=/backup/user_l \ --no-timestamp \ --stream=xbstream \ --history=$(date +%d-%m-%Y-%H-%M) /mysql/backups/full > \ /mysql/backups/full/full-backup_$(date +%d-%m-%Y-%H-%M).xbstream 2> \ /mysql/backups/full/full-backup-progress_$(date +%d-%m-%Y-%H-%M).log &

Full Backup

13

Percona XtraBackup

innobackupex --login-path=/backup/user_l \ --no-timestamp \ --use-memory=1G \ --stream=xbstream \ --parallel=4 \ --incremental \ --incremental-history-name=$(date +%d-%m-%Y-%H-%M) \ --history=$(date +%d-%m-%Y-%H-%M) /mysql/backups/full > \/mysql/backups/full/incremental-$(date +%H)-mysqlbackup$(date +%d-%m-%Y-%H-%M).xbstream 2> \ /mysql/backups/full/incbackup_progress$(date +%d-%m-%Y-%H-%M).log &

xbstream -x < incremental.xbstream

Incremental Backup

--incremental-history-name: Searches the PERCONA_SCHEMA.XtraBackup_history table for the mostrecent backup record containing t

the defined name value, and then starts the backup using its innodb_to_lsn value.

14

Percona XtraBackup

prepare: full backup

innobackupex --login-path=/backup/user_l --apply-log --redo-only /mysql/backups/full --use-memory=1G

prepare: inc 1 day

innobackupex --apply-log --redo-only /mysql/backups/full --incremental-dir=/mysql/backups/mon/

prepare: inc 2 day

innobackupex --apply-log --redo-only /mysql/backups/full --incremental-dir=/mysql/backups/tue/

prepare: apply log

innobackupex --apply-log /mysql/backups/full

15

Percona XtraBackup

Restore Database

innobackupex --copy-back /mysql/backups/full

Change mysql folder permissions

chown -R mysql:mysql /var/lib/mysql

16

Percona XtraBackup

Partial Backup

innobackupex --login-path=/backup/user_l \ --include='^stream[.]vod_log ' /mysql/backups/one_table

Prepare

innobackupex --apply-log --export /mysql/backups/one_table

17

Percona XtraBackupCompress adn Encrypt

innobackupex --defaults-file=/etc/mysql/backup-my.cnf \ --no-timestamp \

--use-memory=1G \ --stream=xbstream \

--parallel=4 \ --encrypt=AES256 \

--encrypt-key-file=/etc/mysql/.mykeyfile \ --encrypt-threads=4 \

--compress \ --compress-threads=4 \

--history=$(date +%d-%m-%Y) ./ > \ mysqlbackup$(date +%d-%m-%Y).qp.xbc.xbs 2> \

backup-progress.log &

• Send it to another server

$ innobackupex --stream=tar ./ | ssh user@desthost "cat - > /data/backups/backup.tar"

18

No single backup option

version 5.7

• Filesystem cold backup

• Datadir• innodb_data_home_dir• innodb_data_file_path• innodb_undo_directory

• innodb_temp_data_file_path• Log-bin• My.cnf

• http://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html

19

Full DB Backup



Sql Dump 20m 1h36m 1h04mPercona XtraBackup

5m 5m

Percona XtraBackup

20

MySQL Enterprise $BackupDownload from

https://edelivery.oracle.com

Create user

CREATE USER 'mysqlbackup'@'localhost' IDENTIFIED BY 'new-password'; GRANT RELOAD ON *.* TO 'mysqlbackup'@'localhost';

GRANT CREATE, INSERT, DROP, UPDATE ON mysql.backup_progress TO 'mysqlbackup'@'localhost'; GRANT CREATE, INSERT, SELECT, DROP, UPDATE ON mysql.backup_history TO 'mysqlbackup'@'localhost';

GRANT REPLICATION CLIENT ON *.* TO 'mysqlbackup'@'localhost'; GRANT SUPER ON *.* TO 'mysqlbackup'@'localhost';

GRANT PROCESS ON *.* TO 'mysqlbackup'@'localhost';

mysqlbackup is copying InnoDBtables, reads and writes to both InnoDB and MyISAM tables can continue. During the copying of MyISAM tables, reads (but not writes)

to those tables are permitted.

https://edelivery.oracle.com/

https://edelivery.oracle.com/

21

MySQL Enterprise BackupFULL DB BACKUP(37G)

./mysqlbackup --login-path=/backup/user_l --host=127.0.0.1 \--backup-image=/mysql/ent_back/$(hostname)_mysql.mbi \

--backup-dir=/mysql/ent_files backup-to-image

Incremental backup

./mysqlbackup --login-path=/backup/user_l --host=127.0.0.1 --incremental --incremental-base=history:last_backup --backup-dir=/mysql/ent_inc2

--backup-image=/mysql/ent_back/$(hostname)_incremental_$(date +%d-%m-%Y-%H-%M).mbi backup-to-image

22

MySQL Enterprise BackupFULL DB RESTORE

./mysqlbackup --datadir=/mysql/mysql/ --backup-image=/mysql/ent_back/localhost.localdomain_mysql.mbi --backup-dir=/mysql/restore-tmp copy-back-and-apply-log

Incremental backup

./mysqlbackup --datadir=/mysql/mysql/ --backup-image=/mysql/ent_back/localhost.localdomain_incremental_06-11-2016-23-33.mbi

--backup-dir=/mysql/restore-tmp --incremental copy-back-and-apply-log

23

MySQL Enterprise BackupInclude table1 and table2

--include-tables=^test\.table[12]$

Exclude table1 and table2

--exclude-tables=^mydb\.t[12]$

* selective restore is only supported for backups created using transportable tablespaces (TTS)

24

MySQL Enterprise BackupSymantec NetBackup

IBM Tivoli Storage Manager Oracle Secure Backup

Tivoli Storage Manager

./mysqlbackup --port=3307 --protocol=tcp --user=backup_user --password=foobar \--backup-image=sbt:my-first-backup --sbt-lib-path=/usr/lib/libobk.so \

--sbt-environment="TDPO_OPTFILE=/path/to/my/tdpo.opt" --backup-dir=/path/to/my/dir backup-to-ima

https://blogs.oracle.com/svetasmirnova/entry/story_of_success_mysql_enterprise?cm_mc_uid=30726269129114773169864&cm_mc_sid_50200000=1478294216

25

MySQL Enterprise Backupmysqlbackup --backup-image=/backups/image.enc --encrypt \

--key=23D987F3A047B475C900127148F9E0394857983645192874A2B3049570C12A34 \ --backup-dir=/var/tmp/backup backup-to-image

To use a key file for the same task:

mysqlbackup --backup-image=/backups/image.enc \--encrypt --key-file=/meb/key --backup-dir=/var/tmp/backup backup-to-image

To decrypt a backup when extracting it:

mysqlbackup --backup-image=/backups/image.enc --decrypt --key-file=/meb/key --backup-dir=/backups/extract-dir extract

26

Point in time Recovery

Mysqldump ---master-data=2

LVM_snapshot

27


MySQL Enterprise Backup

Percona XtraBackup

28


mysqlbinlog mysql-bin.000001 --start-position=11872 \ --stop-datetime="2016-11-6 16:00:00"

29

TIME

https://www.mysql.com/products/enterprise/backup.html

30

Full DB Backup



Sql Dump 20m 1h36m 1h04mPercona XtraBackup

5m 5m

MySQL Enterprise Backup

5m 4m

TIME

31

Conclusion--Incremental

Xtrabackup

This procedure only affects XtraDB or InnoDB-based tables. Other tables with a different storage engine, e.g. MyISAM,

will be copied entirely each time an incremental backup is performed.

MysqlEnterprise

For non-InnoDB files, the entire file is included in an incremental backup if that file has changed since the previous backup, which means the savings for

backup resources are less significant when comparing with the case with InnoDB tables.

Thank You!

Aleksandrs AsafovsMob. +371 28353668

[email protected]

Q & A

Education

My two cents about Mysql backup