Upload
veriton-limited
View
1.880
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Exalogic is an impressive piece of hardware offering immense performance. However the smallest configuration is 96 cores, 768 GB memory and a 40TB SAN... way bigger than many smaller customers could imagine using (even including test environments).This session takes a look at how you could use modern server technology, such as blades, to build a smaller version of Exalogic, and yet still benefit from some of the cost savings from sophisticated automation. This will include a case study of a mid-sized installation where these techniques have been used. Delivered on 5 December 2011 at UKOUG 2011 by Simon Haslam.
Citation preview
1 (1.0h)
Build Your OwnBuild Your OwnBuild Your OwnBuild Your OwnMiddleware MachineMiddleware MachineMiddleware MachineMiddleware Machine
Simon Haslam
Veriton Limited
2 (1.0h)
Simon Haslam / VeritonSpecialised consultant & Oracle Partner, established for 15 years
Demanding web & call-centre applications
Architecture & development strategy; health-checks; disaster recovery; tuning
Oracle Fusion Middleware(JavaEE, SSO, OAM, OID, clustering)ADF Applications (esp. strategy & admin)
Database & related technologies(Solaris/Linux, load balancers, firewalls, …)
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Non-functional Factors5. Case Study6. Summary
4 (1.0h)
Background to this presentation
• Julian Dyke & I came up with ‘RAC-in-a-BOX’ in 2006
– 2 * HP DL365 (8 cores each)
– 1 * MSA1000 SAN (HA controllers)
– 2 * Cisco 3750 switches
• Standard OFA style installation of RAC, config of ASM etc
• Ready to plug into network & power
�
5 (1.0h)
From the ‘Old’ Days to Today
• Was physical hosts with DAS
• Storage arrays for consolidating storage
• Now: most mid-size sites
– Production: physical hosts for production databases
– Dev & Test: virtualised environments very common
• (leaving discussions about public cloud / hybrid models)
�
6 (1.0h)
From now to future
Appliance
PublicCloud
PrivateCloud
DedicatedHardware
for Application
On Premise
Remote
VirtualisedPhysical
Or heavyconsolidation
Appliance
Compute platformStorage platform
�
7 (1.0h)
Trends
• Green IT– Carbon costing– Power costs– SCR
• Convergence– FCoE– iSCSI maturing
• Technology– Speed of commodity hardware– Virtualisation– Networks VLANs: now traffic management, LACP etc.– 10GbE more common– Not the bandwidth/latency of InfiniBand but may be good enough
for most, certainly mid-size, sites
�
8 (1.0h)
Exalogic is BIG!• Even ¼ rack Exalogic is:
– 8 compute nodes @ 12 = 96 cores– WebLogic Suite list price = £1.4M
• Even if ½ processor & ½ NUP = £870k
– Add: WLS Management Pack, Cloud Management Pack..?– List price of hardware $475k (~ £320k)
• Many of my customers only have two or three big web/call centre apps:– modern processors => production can run on 2 compute nodes (for HA)– with virtualisation 1 compute nodes (12 cores) can run multiple test envs– many still have Forms & Reports – expansion of Java EE apps likely over
next 5 years but not from Day 1– Web app to database traffic is modest
• Nothing wrong with Exalogic… it is just very BIG
�
Oracle OpenWorld 2011: announcement 1/8 rack Exalogic(4 x 12 core 96GB compute nodes, same NAS and IB)
9 (1.0h)
My ideal ♥ Middleware Machine!• Hardware
– Compute: fast processors & ‘plenty’ of memory
– Stateless compute nodes
– Networks for public, private, management
– Power management
– Totally managed remotely
– (note: I’ve not mentioned storage)
• Software– WebLogic
– Web Tier
– Load Balancing
– Management
– Layered products (old and new)
�
10 (1.0h)
Confession: I am a blade convert!
�
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Non-functional Factors5. Case Study6. Summary
12 (1.0h)
Ingredients: 1. Compute�
13 (1.0h)
Ingredients: 2. Power
• Single or 3 phase
• Note: health and safety implications
�
14 (1.0h)
Ingredients: 3. Storage�
15 (1.0h)
Ingredients: 4. Network
• Core switches
• Load balancers
�
16 (1.0h)
Bake & Leave to Cool �
17 (1.0h)
Traditional Rackmount vs Blade?
• Power consumptionhttp://www.veriton.co.uk/roller/fmw/entry/...hp_blade_vs_rack_servers
• Costs
• Note some chassis-wide costs:
– Converged networking
– Administration modules
• Vendor lock-in
�
18 (1.0h)
Size of errorbars unknown!
Changesfrom adding fans/power supplies
�
19 (1.0h)
Exalogic1/8th rack 4n (inc storage)
2.4-4.3kW*
* Exalogic “typical” to “max” (from Oracle datasheet 14-NOV-11) – rounded to 2 sig. fig.
Exalogic1/4 rack 8n (inc storage)5.3-7.2kW*
Exalogic1/2 rack 16n: 8-11kW 1 rack 32n: 13-18kW
Note: Exalogichas 96GB RAMper node cf 24GB,SSD boot disksand slightly faster procs(3.0GHz cf 2.8GHz)
�
20 (1.0h)
Compute Node Storage
• What would you put on a server if you had shared storage?– Operating system files?
– Swap space
– Crash dumps
– Application log files
– Transaction logs?!
• Do you need SSD for this?
• As soon as you add storage to a server you make it stateful
�
21 (1.0h)
Hardware Summary
• Very fast processors
• No local storage; boot from SAN
• Shared 10GbE networking (max 20GbE per blade)
• Centralised hardware management
• Centralised power management
�
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Case Study 5. Non-functional Factors6. Summary
23 (1.0h)
Provisioning
• Stateless blades (hardware)
– No disk (or SDcard!)
– No MAC addresses
– No WWNs
– No Lights-Out Management addresses
= ~virtualisation of the hardware
�
24 (1.0h)
Hardware/OS Configuration
• Pre-assign IP addresses
– Management
– Public Prod
– Test
– Private
– Cluster
�
25 (1.0h)
Software Installation - today
• Centralised Deployment
– Scripted (e.g. WLST & response files)
– Enterprise Manager – Deployment Pack (still some work to be done)
�
26 (1.0h)
Virtualisation - notes
• VMs – build from kickstart/jumpstart etc
• There’s a lot to learn from AWS
• Oracle Virtual Assembly Builder is a very interesting product for the future
– VM template libraries
• OVAB probably the future if you’re on OVM
• OVAB apparently underpinning Java Service of Oracle Public Cloud
�
27 (1.0h)
Middleware Installation• Shared middleware homes
– Pros and Cons
– Great for support ☺
• E.g. Exalogic EDG
�
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Case Study5. Non-functional Factors6. Summary
29 (1.0h)
Case Study
• Mid-sized UK customer
• ~800 agents in 4 call centres + web site
• Previously 10g iAS on 6/8 2U rack servers (production, primary site)
• Migration to WebLogic Suite 11g (plus OID 11g & Reports 11g)
• Moving from Red Hat to Oracle Linux 5.6*
• Virtualised
• Centralised Oracle homes and management
�
30 (1.0h)
How did it go?
• Power was a challenge– you need to size for full capacity
• Cooling– fortunately able to free up space, but could have been
tricky due to heat density
• Boot from SAN– Not really a problem – much more common these days– I suspect (I) will get blamed for anything bad
• Networking– Forced through a rethink of some of the VLANs and
subnets– We did have some problems with LACP/trunking
�
31 (1.0h)
Now?
• A project appeared from blue, sneaked in and went live on the blade system before Oracle project out of testing!
• Chassis will probably be full within a year
�
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Case Study5. Non-functional Factors6. Summary
33 (1.0h)
Remember…Licences
and support
Hardware
�
34 (1.0h)
Politics
• Multiple teams are often responsible for support:
– Application Development
– Oracle Administration
– System (e.g. Unix) Administration
– Storage Management
– Data Centre Management
�
35 (1.0h)
Real-World Example in 2011
• Oracle team had bought lots of SOA Suite licences (sizing method a bit unknown – partly historical, partly reseller)
• Production hardware still to be purchased• Licences only efficiently “fitted” on certain combinations of hardware
at primary and DR site• System Administration team different to Oracle/project team• Cost of Oracle licences > 15x cost of hardware• Cost centre for hardware/OS/VMware seemed to be different from cost
centre for Oracle => smallest amount of hardware bought*
� Oracle perpetual licences plus Support/Updates unused
• This is not unusual behaviour!
* I do have some sympathy here – the licences should have been right-sized in the first place
�
36 (1.0h)
Real-World Example: What if they had chosen Exalogic?
• 2 Exalogic ¼ racks (production and DR, would have probably had to run dev/test on DR)
• Presumably Oracle licences would be bought at same time so match
�One decision to make
�1 Purchase Order
�Choice cannot be ‘unbalanced’
�
37 (1.0h)
Exalogic vs Build-Your-Own
©2010 Markus Eisele
�
38 (1.0h)
Exalogic
• Comes assembled & cabled up from Oracle• All decisions have been taken for you – you can only
choose the size• Oracle love you & you’ll probably get a better deal on your
licences• Oracle remote diagnostics & single point of support• You’ll get all the best Middleware features & super
performance, especially if you have Exadata too• OVAB etc means it may be more “plug & go”• Oracle “Red Stack Administrator” is King/Queen!• But:
– May have to buy more hardware than you need up-front– You’ll need at least 2 & possibly 3 – all are same (top) spec– Your admins may not be used to Sun hardware, LOM etc– (Vendor lock-in)
�
39 (1.0h)
Build Your Own (Blade)
• Flexibility to run other stuff on same infrastructure• Very neat cabling• Can buy the fastest processors and migrate slower ones to
test envs• Great automation options• Hardware will be consistent with other non-Oracle stuff• Fits in with virtualised compute layer, storage layer,
network layer => very clean architecture• But:
– You’ll probably need to hire a consultant to set up to get this degree of automation (costly)
– You may have internal bickering over storage, network, boot fromSAN, spec of servers etc
– Oracle hold back best features for Exalogic– With blades you will still have vendor lock-in
�
Agenda
1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Case Study5. Non-functional Factors6. Summary
41 (1.0h)
• Exalogic is bigger than many organisations need• Reasons to buy Exalogic may be as much political
as technical• Modern alternative ways to build a “middleware
machine” include stateless compute nodes booting directly from centralised storage
• If building your own - careful configuration required with scripted management – don’t reinvent the wheel. Standardise!
42 (1.0h)
Thanks for listening!
Twitter: @simon_haslam
Oracle Fusion Middleware Admin blog:
http://simonhaslam.co.uk