NOAA WEBShop 20041 A low-cost standby system for an OAR-wide budgeting application Eugene F. Burger...

NOAA WEBShop 2004 1

A low-cost standby system for an OAR-wide budgeting application

Eugene F. Burger (NOAA/PMEL/JISAO)

NOAA WebShop

July 27-29 Philadelphia PA

NOAA WEBShop 2004 2

Overview The systems in question Existing contingency planning The type of standby system we chose Alternatives Architecture Implementation Recovery scenario

NOAA WEBShop 2004 3

The challenge Continuation of Operations (COOP) planning is

a requirement with all operational systems. Backups Full standby system

NOAA WEBShop 2004 4

FDMS is

OEORTAPO

Financial Data Management System• OAR Commitment tracking system• Supports NOAA business practices• Operational since January 2001• Envisioned as the “core” element of the NOAA Research Science Management Information System

NOAA WEBShop 2004 5

FDMS is FDMS built on Access Source data bases

MS SQLServer Oracle 8i (soon 9i)

Server through Citrix Secure Gateway and Citrix Metaframe

Daily downloads of CAMS data to FDMS

NOAA WEBShop 2004 6

FDMS Server architectureFDMS

Inte

rnet

FDMS Domain

Databaseserver

FDMSApplication

server CSGServer

Webserver

OAR HQ

FSL

NSSL

OAML

SEC

OAR offices

Firew

all

NOAA WEBShop 2004 7

Existing precautions Redundancy in server storage devices

RAID 5 disk array’s with spare drive’s Database backups Backups of backups Offsite backups

NOAA WEBShop 2004 8

Objective Investigate options

To provide a low cost contingency (failover) system Addresses potential disaster scenario:

operations in Seattle are disastrously disrupted (earthquake) backup tape recovery difficult network infrastructure disrupted

FDMS recovery period of 7 to 14 days is tolerable Although application not critical to NOAA’s mission Downtime will cause administrative disruption

NOAA WEBShop 2004 9

OptionsFull FDMS standby system

Complete replication of the FDMS infrastructure Cost

Backups What about catastrophic disaster?

Backups and replication

NOAA WEBShop 2004 10

Our choice Remote storage device with complete

application and data file replication Lower cost Lower maintenance overheard More infrastructure flexibility

When we change hardware or software infrastructure

NOAA WEBShop 2004 11

Recovery scenario Purchase new hardware (server) infrastructure

CSG Application Web Database

Install and configure applications FDMS DB Servers Active directory Citrix

NOAA WEBShop 2004 12

Recovery scenario Two options

Serve data from NAS box Copy data files to Server

NOAA WEBShop 2004 13

Replication challenges Replicate everything needed to rebuild system

Multiple server machine state Active directory Relational databases (Oracle 9i & SQL Server)

Database user information encapsulated in backups MS Access data files FDMS application data FDMS tools FDMS Documentation Citrix application Citrix configuration information

System restore documentation

NOAA WEBShop 2004 14

Standby infrastructure Hardware

NAS Box Software

Mirroring software Extensive and detailed documentation

Installation information Data linking Server configuration

NOAA WEBShop 2004 15

Replication infrastructure

FDMS - PMEL, Seattle

FDMS Server 1

FDMS Server 2

FDMS Server 3

RepliWebR1 controller

(software)

OAR HQ - Silver Spring

NAS devise

Repliwebsatellite(client)

Internet

NOAA WEBShop 2004 16

Hardware considerations Network Attached Storage device

IOMega 320Gb NAS Cheaper than a full server

$2k vs. $12k

Server could be obsolete by the time it is needed

Chose windows OS to sync with the FDMS application platform

NOAA WEBShop 2004 17

Mirroring software requirements Mirror over the internet Transport encryption

Including authentication Differential mirroring Continuous file replication Multiple channels per file

NOAA WEBShop 2004 18

Mirroring software Applications considered

Peer software PeerSync

Availl Availl Replication

Falconstor software Hardware & software solution

Repliweb RDS & R1

NOAA WEBShop 2004 19

Testing Replication over LAN

To familiarize ourselves with software Test software compatibility

Ensure data are encrypted Time to first complete snapshot FDMS failover test

Serve data from NAS box

NOAA WEBShop 2004 20

Results Mostly positive Replication software works well

Replicates very large files in reasonable time 2 hours for 12Gb file

Hardware acceptable Some issues, but may be related to the

IOMega NAS device. Very little impact should FDMS change

hardware/software architecture

NOAA WEBShop 2004 21

Tradeoffs Unable to mirror open files

Copying backups of the databases Do replication when there are no users

online Works with “OpenFile Manager” (third party

software) No immediate system to revert to

not really a requirement

NOAA WEBShop 2004 22

Tradeoffs Mirroring relies completely on network

(Internet) No FDMS usage during internet outage –

acceptable risk

NOAA WEBShop 2004 23

Conclusion Viable, cost effective solution as a

standby file system to ensure COOP Will be implemented as a operational

component of FDMS

NOAA WEBShop 2004 24

Contact information

Eugene F. Burger

OAR/PMEL – Seattle

eugene.burger@noaa.gov

(206) 526-4586