Backing up Zarafa Collaboration Platformwith Veeam SureBackup

Joep Piscaer, VMware vExpert, VCDX #101

j.piscaer@virtuallifestyle.nl

@jpiscaer

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Introduction Joep Piscaer

● Consulting Architect at OGD ict-diensten● VMware VCDX5 #101, vExpert 2009, 2011, 2012● Know Veeam since 2007 and in love with them ever since

(best. VMworld. parties. ever.)

Past Projects Past implementations of Veeam B&R

● Commonly see a VMware virtualization layer with Windows VMs on top● But my own company uses Linux-based applications internally

Notable projects include● Bi-directional DR for 200-250 VMs with 2 infrastructures● 150+ VM backup and replication within a single large datacenter● Application consistent backups of Zarafa Collaboration Platform without

bringing database down (or any other downtime)● Numerous smaller projects for DR or backup at customer sites

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Consistency states

Crash consistency“A crash consistent state does not have any quiescing and is equivalent to a VM being powered off without being properly shutdown.”

Filesystem consistency“A file system consistent state is better because the operating system is in a proper state to be backed up.”

Application consistency“Finally, the best state is application consistent where applications have also been properly prepared for backup.”

Why is application consistency important?

A consistent application (or transactional) state is important, because:

It will prevent data loss or data corruption

It will improve RTO significantly for large data sets

How to force consistency?

Quiescing the VM, filesystem and/or application:Pause VM and write all the transactional datain memory to disk

Forces the state of the disk image to be consistent, preventing data corruption or lengthy crash recovery checks on filesystems and databases

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Zarafa components

Zarafa application components

● Core applications, WebAccess / WebApp, Z-Admin, postfix, Z-Push, spamassassin, clamav, amavis, etc

MySQL database for metadataand a file based content system

Authentication directory

DNS

Backing up Zarafa – What’s the big deal?

The application components or dependenciesare not the actual issueWe can rely on file system journal to recover data from these application components

But the MySQL database isIt needs to be application-consistent instead of crash or filesystem consistent

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Hot Backup of MySQL on a Linux VM

Two options to produce a transaction consistent backup of MySQL databases:

● Option 1 stops the database andapplication; all caches are lostand need to be rebuilt; too biga performance hit and impactsservice availability

● Options 2 produces a full copyof the database, which is notstorage-friendly, and takes aconsiderable amount of time.Restore requires additional steps.

Hot Backup of MySQL on a Linux VM

Both options are not viable in our environment.

We can’t stop the database or application for each backup:

● For service availability reasons

● Because the built-up caches for database and applications would be lost, severely impacting application performance.

We cannot create a dump of the database:

● It would take too long with a big impact on storage (IOps and capacity)

● Restoring data from the dump requires additional steps

Hot Backup of MySQL on a Linux VM

We came up with a 3rd option:

Option 3. Lock database tables and flush to disk.No downtime or service availability impactNo need to rebuild cache or create an entire copy of the database.

Flushes all open tables in the database to disk while preventing any changes to databases

Hot Backup of MySQL on a Linux VM

One problem though:“Script can’t quit the mysql shell else you will lose the lock”

This does not play nice with pre-freeze andpost-thaw scripts; since the pre-freeze scriptquits and loses the lock…

Jelmer Jaarsma to the rescue!My colleague founda work-around

pre-freeze-script.sh

Create two temporary lock files and launch background script. Pre-freeze script will run until database is locked.

Background script will flush tables to disk and write-lock database. Finally, it removes a temporary lock file.This signals the pre-freeze script that the database is ready for snapshotting; the script terminates without error

With the pre-freeze script terminated, Veeam will now create the snapshot (with the database locked)

post-thaw-script.sh

During the snapshot creation, the background script is still running, keeping the database locked.

post-thaw-script.sh

After the snapshot has been successfully created, the post-thaw script removes the 2nd temporary lock file. This will shut down the background script, unlocking the database and continuing normal database operation

No downtime; just a couple of seconds databasequery queue

Veeam backup job settings

There’s no specific settingsinside the job

Enable VMware Tools quiescence

No ‘post job activity’ in thebackup jobs settings

No application-awareimage processing (VSS)

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Backup and recovery verification

Transactionally consistent backups need to be verifiedperiodically to meet RPO and other SLA goals

Testing a restore manually is a painfully long and hard process; it should include tests on all layers(VM, filesystem, application)

The only real verification is to bring the VM back online in a safe and isolated environment

Can be used to restore granular application items, too

Integrate Zarafa into SureBackup

Add a new role to SureBackup using ZarafaServer.xmlhttp://forums.veeam.com/viewtopic.php?f=24&t=15103&p=69768

Create a Virtual Lab

Create appropriate application groups(including any dependencies like LDAP, DNS)and select both ‘mail server’ and ‘zarafa server’ boxes

Restoring Zarafa

It’s Linux; please prepare your VM’s NIC settings accordinglystatic MAC address or retain IP after MAC changes

Full VM recovery does not need any manual steps

● the database is in a consistent state (no crash recovery)and ready to go (no need to restore from a dump file)

Restoring Zarafa

Single Object Restore / Single Mail Restore http://forums.veeam.com/viewtopic.php?f=24&t=15103&p=69768

● Add server to Managed Servers for File CopyAdvanced - Preferred TCP Connection Role - "Run Server on this side“

● Use U-AIR to boot up VM

● Extract an User:/usr/bin/zarafa-backup -a -v -u max -o /var/backup/

● Use Veeam File Copy to restore files to production

● Restore User/usr/share/zarafa-backup/full-restore.sh max

Agenda Introduction What is backup application consistency

and why is it so important? Evaluating Zarafa ZCP components

and relevance to application consistency What options to backup Zarafa ZCP are available

within Veeam Backup & Replication? Integrating application consistency into SureBackup Demo

Live Demo

Q & A

Joep PiscaerVMware vExpert, VCDX #101

j.piscaer@virtuallifestyle.nl

@jpiscaer

Add Jelmer Jaarsma to LinkedIn:http://nl.linkedin.com/pub/jelmer-jaarsma/8/456/717