CXtec Migration from Large SMP to Oracle RAC Cluster on Linux
Dean Bettinger
Storage Technology
Business Development Manager
CXtec Migration from Large SMP to Oracle RAC Cluster on Linux
o World-wide VAR for Network, Voice, and Data Center equipment – new and certified pre-owned.
o equal2new®: 10-step certification process, 99.51% reliability rating and lifetime warranty
o CABLExpress® cables, including fiber jumpers. Also new and equal2new® Fibre Channel switches & HBAs.
o $135M annual sales, 6 consecutive years on VAR 500 (#231), ISO 9001:2000 certified.
o 300+ employees.o Great Place To Work – last 4 years in top 25.
CXtec Company Overview
o ERP o Oracle Applications eBusiness Suite 11.5.9o Sun E4500, 12 x 400MHz CPU, 12G RAMo Oracle Database 9i, named user licenseo Running out of CPU cycles and RAM
o Web Siteo Sun 450, 400MHz CPU, 4G RAM for our web
siteo Oracle Database 9i, per CPU license,
unlimited userso Day old copy of production data replicated
from ERP
Where We Were - 2004
o Business desire to bring live transaction data to our web site, especially to our Extranet for customers
o Business / IT desire to leverage Oracle for everything
o IT developer desire to simplify our development environmento Avoid complexity of live synchronization
between inside & outside databaseso Cease daily replication of day-old data to
Extraneto Utilize web services from Extranet to access
up to the moment live data
Challenges
o Migrate internal Oracle database licensing from named user to per CPU, unlimited userso Can now serve 5,000+ Extranet users
directly. Avoided $800 per user -- $4Mo Can now leverage Oracle database for all
application development: ERP, Extranet, Intranet for all employees, not just ERP users
o Eliminate Oracle database from our DMZo Corporate web site and Extranet will now use
web services to get data from the one true source
Database Licensing Migration
o Oracle charges $40K per CPU licenseo Sun CPUs were 900MHz max and we figured we
needed 12 CPUs -- $480Ko New Sun enterprise-class server -- $350Ko Sun SPARC and Sun Solaris – out the window!
o CXtec had been a long-time committed customer of Sun, and we loved its reliability
o Solaris was favored operating system due to features and stability, but Sun had abandoned Solaris x86.
Ouch!
o In Intel we trust!o Scale CPU speeds faster than CXtec growso 3GHz Xeon much faster than 900MHz SPARCo Desire to move to commodity 2 socket
serverso Goodbye Sun, Dell here we come!
o Decision point – How do we get to 12GHz?o Single Quad socket server?
o Simple, but hardware more expensiveo Two Dual socket servers in a RAC cluster?
o Complex, but commodity servers, resiliency
The Way Forward
o Raw device mapping?o Not visible to SysAdmins, complexity borne by
DBAo Oracle ASM not yet available
o Cluster file system?o Oracle Cluster File System – OCFS1
o Not ready for prime timeo Sistina GFS (now a Red Hat product)
o Single node lock managero PolyServe Matrix Server (now an HP product)
o Distributed lock manager, BCF reference
Mitigating RAC Complexity
o We settled on PolyServe’s Database Utility for RACo Common mount points on all servers
o SysAdmin friendlyo Simple for DBAo Shared Oracle HOME
o Distributed Lock Managero No single point of failure
o Context Sensitive Symbolic Linkso Oracle Disk Manager (ODM) integration
PolyServe Matrix Server
Database Server9iRAC
Dababase Server9iRAC
Oracle Apps9iAS
Web Apps9iAS Java Edition
DiscovererAS 10g Enterprise Edition
Brocade 2800Fibre Channel Switch
Brocade 2800 Fibre Channel Switch
Xiotech Magnitude 3D
Cisco 3508 GbECluster Interconnect
Core Network Stack – Nortel 5510 switches
Corporate LAN – Nortel 450 switches
`
Middle Tier
Servers
EndUsers` `
Oracle RAC
Cluster
1st GenerationRAC Cluster
2nd GenerationRAC Cluster
Database Server10gRAC Dababase Server
10gRAC
Oracle AppsAS 10g
Web AppsAS 10g Java Edition
DiscovererAS 10g Enterprise Edition
Qlogic 5200Fibre Channel Switch
Qlogic 5200Fibre Channel Switch
Corporate LAN – Cisco 6509 Core switch
`
Middle Tier
Servers
EndUsers` `
Oracle RAC
Cluster
Compellent Storage Center
Cisco SFS 300110G InfiniBand
Switch
Cisco SFS 300110G InfiniBand
Switch
ClusterInterconnect
o Improved Performanceo Batch job take minutes instead of hourso Dmart loading window dramatically reduced
o Improved End-User Productivityo Most application response now sub-second
o Improved IT Productivityo Single database for all applications. Simple!
o Live Extranet information for Customerso Enhanced Reliability
o Run on 2 nodes, 3rd standby node
Results
o Startup Costs - $137Ko $20K for 3 Dell 2 socket 3GHz Xeon servers,
8GBo $24K for PolyServe Matrix Filesystem, DBUo $93K Oracle License Migration to RAC
o Annual Maintenance Commitmento +$16K / year
Economic Analysis
o Cost Avoidance - $670Ko $350K for a new Sun Enterprise-Class SMP
servero $320K in Oracle CPU licensing for slow Sun
CPUs
o Annual Maintenance Cost Avoidanceo $112K on Sun Server and Oracle Licenses
o Bottom Lineo $1,017,075 Cumulative 5 Year Net Benefito ROI 473%, Payback Period – 2 Months
Economic Analysis
o Cisco / TopSpin SFS 3001 InfiniBand switcheso 12 ports of 10G Full-Duplex InfiniBando 2 port of 2G Fibre Channel
o For dual pathing to two Fibre Channel switches
o Cisco HCA (Host Channel Adapter) cards for serverso 2 ports of 10G Full-Duplex InfiniBando For dual pathing to two InfiniBand switches
o 10x bit rate of Gigabit Etherneto 1/10th latency of Gigabit Etherneto 1/10th CPU utilization compared to Gigabit
Ethernet
InfiniBand Cluster Interconnect
o Storage Virtualization on steroidso Boot from SANo Snapshots and Viewso Enterprise Managero Thin Provisioningo Automated Tiered Storageo Fast Tracko Thin Importo Windows File Space Recovery
Compellent Storage Center
o ???
Questions & Answers