Embed Size (px)
Citation preview
OpenStack Project FreezerIntroduction
Lê Quang Long
Trương Thanh Hà
2
NỘI DUNG
Agenda
1. What is Freezer?
2. Architecture
3. Experiences
4. Demo
3
“Freezer is a Backup and Restore as a Service, automate the databackup and restore process”
• Open source since 07/2014
• Supported in HP Helion OpenStack 1.1
• Supported and integarated in HP Helion OpenStack 2.0
4
Multiple platform
• Linux Based OS (Ubuntu, CentOS,…)
• Windows
• Mac-OS
• BSD based OS
Multiple type of storage:
• OpenStack Object Storage (Swift, Ceph,…)
• SSH Server
• Local directory (NFS, iSCSI, GlusterFS,…)
Features
5
• Low memory footprint even with big backup
• Incremental backup capabilities
• Old backup tidying
• Restore from a specific date
• Network bandwidth usage limitation
• Jobs:
o Collection of backup / restore actions
o Useful to backup multiple files, directory, application in one go
• Sessions:
o Sync your backup on multiple servers
o Userful for cluster backups
Oustanding Features
6
NỘI DUNG
Agenda
1. What is Freezer?
2. Architecture
3. Experiences
4. Demo
7
1. Client side components:
• Freezer-agent: executes the backup and restore actions
• Freezer-scheduler: runs the freezer-agent and talks to the freezer-api
2. Server side components:
• Freezer Web UI: orchestrate backup and restore from OpenStack Horizon
• Freezer API: REST API to access and create freezer resources
• ElasticSearch: Retrieve and storage metric, metadata session, job status,…
Freezer Components
8
Architecture
Source: https://github.com/openstack/freezer
9
Architecture
Source: https://docs.hpcloud.com/pdf/HPE_Helion_OpenStack_2.0_and_2.1_Complete_Documentation.pdf
10
Applications• MySQL• SQL Server• MongoDB
Incremental backup• Using tar to perform incremental backup
Compression• Gzip/Zip/Zlib (low-compression)• Bzip2 (high-compression)• XZ (lossless data compression)
Consistency• LVM snapshot (Linux OS)• VSS snapshot (Windows OS)
Encryption• OpenSSL (AES-256)
Backup Application Data, FileSystem
11
Swift
Freezer-Scheduler
Freezer Agent
Remote FS
LV/var/lib/mysql streamMySQL DB
LV-snap/var/snapshot-
backup
Lock Create LV Snapshot and mount FS
Archive and compress data, upload backup
Backup Application Data, FileSystemWorkflow
SSH
HTTP
metadata
metadata
Release Lock
12
Backup Application Data, FileSystemUsing Stream to implement
Raw Data
BackupBackup Storage
(SSH or Swift)
SFTP or
HTTP
Tar and
zip
Tar chunk
Queue
Backup Stream
Tar chunk
Tar chunk
Tar chunk
Tar chunk
Tar chunk
Tar chunk
Write
backup
Tar chunk
Tar chunk
Job Metadata Format{"encryption": false, "compression": "gzip", "engine_name": "tar"}
13
Backup Application Data, FileSystem
Backup Metadata FormatGNU tar-1.27.1-2
147870684188110196001478705988250796000645122356966.NU14.04_cloud_3.qcow2Ncirros-0.3.2-x86_64-disk.img
14
Consistency• Nova: Snapshot VM • Cinder: Snapshot Volume
Behaviour• Snapshot VM, Volume• Create new Volume from Snapshot• Snapshot new Volume to Image• Upload Image to Swift
Advantage• Live backup • Consistent recovery
Disadvantage• Waste resource (CPU, RAM, Storage)• Waste time• Full Backup only
Backup Nova VMs, Cinder Volumes
15
Behaviour
• Backup Volume using Cinder-backup API
Advantage:
• Live backup (from Liberty)
• Incremental backup (backend dependently)
• Consistent recovery
Disadvantage:
• Storage backend must support cinder-backup
Backup Volume using Cinder Backup
16
Swift
Freezer-Scheduler
Freezer Agent
Remote FS
Restore Directory(/usr/restore_dir)
Restore
Restore Application Data, FileSystemWorkflow
metadata
metadata
18
Swift
Freezer-Scheduler
Freezer Agent
Remote FS
Glance Image
Cinder Volume
Restore Volume
Restore Image
Restore Nova, CinderWorkflow
metadata
metadata
Nova VM
Restore VM
19
NỘI DUNG
Agenda
1. What is Freezer?
2. Architecture
3. Experiences
4. Demo
• Resize segments size upload Swift (Byte)#freezer/openstack/osclients.py
def download_image(self, image):
…
return utils.ReSizeStream(stream, image.size, 1000000)
• Resize Swift chunk size download (Byte)#/freezer/openstack/restore.py
def _create_image(self, path, restore_from_timestamp):
…
stream = swift.get_object(self.container, "%s/%s" % (path, backup),
resp_chunk_size=10000000)
15
Experiences
• Glance API v2 is unable to delete RBD Images#/freezer/openstack/osclients.py
16
#/freezer/openstack/backup.py
Experiences
22
NỘI DUNG
Agenda
1. What is Freezer?
2. Architecture
3. Experiences
4. Demo
23
DEMO
24
Demo 1: Backup and Restore Nova VM
Demo 2: Backup and Restore Cinder Volume
Demo 3: Backup and Restore File System
Demo 4: Backup and Restore DB (MySQL)
DEMO
25
Reference
• https://www.youtube.com/watch?v=IeVJgwHiurI
• https://github.com/openstack/freezer
• https://docs.hpcloud.com/pdf/HPE_Helion_OpenStack_2.0_and_2.1_Compl
ete_Documentation.pdf
• https://docs.hpcloud.com/hos-4.x/helion/bura/bura_overview.html
• https://docs.hpcloud.com/hos-4.x/helion/bura/freezer_scheduler.html
26
Thank You !!
