Site Studio Performance Tuning
Brian “Bex” HuffChief Software Architect
2
Agenda
General web-site tuning tips• The complex basics
High-performance architecture• Tune your system, network, and servers
Tuning Site Studio pages• Oracle UCM specific tips
3
Philosophy of Performance Tuning
“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil”
-- Donald Knuth
Need a holistic view of your systems and processes• Use data-flow analysis and diagrams
4
Why Are Network Applications Slow?
Not enough CPU• Needed for text processing, number crunching, cryptography
Not enough memory• Needed to cache temporary data for faster processing
Not enough bandwidth• Needed to support large data transfers, large user bases
Too much latency• Multiple round-trips to servers makes any single request slow
Most web applications are slow because of bandwidth/latency • extra CPU and memory help, but usually are not enough
5
Bandwidth and Latency in Real Life
48 inches wide
Bandwidth:• 2,000,000 barrels per day!
800 miles long
Latency:• 15.8 days!
6
Bandwidth and Latency On The Web
Bandwidth:• Number Of Simultaneous Requests You Can Support
Latency:• Time Added To Each Request Due To Network and Distance
A web page with 50 images means 51 HTTP requests!
High bandwidth and high latency? Pages “feel” slow
80-90% of response time spent in downloading web resources• Images, CSS files, JS files, AJAX requests• Source: http://developer.yahoo.com/performance/rules.html
7
Analyze The Flow Of Data
How does the data get to the client?
How many applications are involved?
Where is the bottleneck?• Latency: ping, traceroute, or tracert• Bandwidth: mrtg, wget, or ask your ISP/IT department• CPU and Memory: top or taskmanager
On the web, latency is usually a bigger problem than bandwidth• Distance and network health between web client and web server• Assume New York to San Francisco latency is ~100 ms• if your page has 100 images, that's 10 seconds for just latency!
In practice, browser caching and multiple downloads help
8
UCM Data Flow Diagram
How do the actual bytes flow from client, to server, and back?
client
Database
File System
Content Server & Web Server
LDAP / Active Directory
9
Reduce CPU Needs of Web Pages
General tip: simplify your web pages so they render faster
Use fewer DOM elements• tables within tables within tables
Put CSS at the top, and JS at the bottom• CSS is rendered immediately• JS must be 100% loaded before it executes
Avoid complex CSS• AlphaImageLoader filter• CSS expressions
10
Reduce Bandwidth Needs of Web Pages
General tip: use smaller web resources
Compress your HTML, CSS, and JS files• minimize their size in general• inline GZIP compression works well
Use small, web-optimized images• avoid scaling them in HTML
Use GET for AJAX
Reduce use and complexity of cookies• cookie state is sent with every request
11
Reduce Latency Of Web Pages
General tip: use fewer web resources• eliminate images when CSS works just as well• consolidate CSS and JS into fewer files• correct any 404 error for missing images
Use CSS SPRITES• Like Google does
12
Reduce Latency Of Web Pages (cont.)
Make fewer network requests• Avoid multiple DNS lookups on the same page• Minimize URL redirects
Serve static files from multiple servers• Browsers download 2 web resources at a time from one domain• So, make more domains: img1.company.com, img2.company.com
Use content delivery network• Amazon Cloudfront : cheap, but mainly North American • Akamai : expensive, but global
Make content “cacheable”
13
High-Performance Architecture
Web Page Cache / Reverse Proxy• Cache rendered web pages to take load off of Site Studio• make sure it understands cookies and GZIP compression
Contribution / consumption environment• Optimize contribution for read/write• Optimize consumption for read (mostly)
Use a dedicated server for each Oracle UCM instance• Do not share with other applications
Schedule resource-intensive behavior in the off-hours• Virus scanners and web spiders• Backups, archives, batchloads
14
High-Performance Architecture (cont.)
Create a cluster• in most cases with 10gr3, significantly improves performance• general best practice for failover, high reliability, and uptime
Minimize latency in local network• keep database, LDAP, and other resources on same subnet• keep filesystem speedy: attached storage, SAN, NAS
Monitor CPU and Memory usage• bandwidth, and latency are usually the culprits• CPU and memory can be a bottleneck for heavy consumption servers
If you use UCM 7.5 with Verity search, CPU can be an issue• if you use database search, database CPU may be an issue
15
Sample High-Performance Architecture
Contribution Cluster
Consumption Cluster
internet
Replication
Reverse Proxy Cluster
DatabaseCluster
SAN
16
Make Static Content Cacheable
Web caches will store static content (JS, CSS, images)• Modest performance gains by keeping them in-memory• Caching dynamic content is better, but harder…
Add Expires or MaxAge HTTP header to content• allows a cache to know how long to store content• for static content, set this in the web server• for dynamic content, set it in Site Studio
Properly configure ETag HTTP headers on clusters• If you're not using them, just remove them
17
Make Dynamic Content Cacheable
Dynamic HTML pages might be different per user• Multi-step forms and personalization are cache-killers
• Different users see different things at the same URL!
• Not all caches work when cookies are present• Need smart caches, and smart personalization
Drive customization with static CSS and JS files• Read cookies from JS, use AJAX to alter pages• Configure cache to ignore cookies, or• Configure web server to omit cookie header
Security checks can be slow• How far away is your user database?• Is it cached locally, or do we need to check ActiveDirectory or LDAP?
18
Site Studio Specifics
UCM performance rule-of-thumb• 20 read-only service calls per second per GHz CPU• 5 read/write service calls per second per GHz CPU
Problem: some Site Studio pages are multiple service calls!• Each calls at least the SS_GET_PAGE service• Many fragments and templates call GET_SEARCH_RESULTS• Many also call DOC_INFO
Not uncommon for one page to have 4 UCM service calls!• Reduces performance to 5 page views per second per GHz CPU
19
UCM Specifics
Minimize number of queries per page• Use static lists instead of dynamic lists when possible
Optimize search queries• Use <matches> or <contains> instead of <substring>• Note: <contains> only works in "zone indexed" metadata fields• Avoid time stamps in queries
Tune Java VM to have as much memory as you can give it• JAVA_MAX_HEAP_SIZE=1024
Optimize the Oracle Database• Install the OracleQueryOptimizer component• Cursor_Sharing=SIMILAR• optimizer_mode=first_rows
20
Finding Slow Pages: Passive
Configure web server logs to measure page creation time• Apache: use mod_log_config and the LogFormat directive• IIS: use W3C extended log file format, and the "Time Taken" field
Enable UCM System Audit Information flags• requestaudit• pagecreation• searchquery• systemdatabase
Parse log files to find slow pages and slow queries
21
Finding Slow Pages: Active
Run a web spider on your system• Caution: this can crash your server!• Use web logs or spider logs to find slow pages
Test slow pages once with auditing turned on• You may be surprised how many database calls there are!
Test slow pages with a load tester• How many users can do this simultaneously?
22
Finding Slow Pages: Active (cont.)
Bandwidth: • How fast can you download a static 1MB HTML file?• How many users can do this simultaneously?
Latency: • How fast can you download a 1k HTML file with a hundred 10k JS files?• How many users can do this simultaneously?
Keep excellent notes!• Only way to measure performance improvement
23
Making Pages Faster
Reduce number of service calls
Reduce number of database calls
Optimize IdocScript• Minimize eval and executeService calls• Instead use setValue, setResourceInclude• Eliminate all IdocScript errors: these are performance drags• Use cacheInclude for slow sections of pages
Optimize JSP/ASP/Java code• In general, same rules apply• Reduce complexity, reduce number of service calls
24
Making Pages Faster (cont.)
Consolidate JS and CSS files• Place them into a global fragment, or global region template• Standard best-practice for maintainable code
Set flags to cache pages• Built-in with the custom section property "ssMaxAge"• Passes the cache control header• Watch out for cookies!
Disable unnecessary JavaScript files• Requires custom component• On consumption environment, disable contributor JS files• Most sites don't need the JS files that contain navigation
25
Review
General web site performance tuning is critical
Data flow diagrams can highlight bottlenecks
Some architectures scale better than others• Size CPU and Memory appropriately• Cluster your servers• Include a reverse proxy• Keep network resources close, and fast!
Understand why specific Site Studio pages are slow• Reduce number of service requests per page• Focus on making “cacheable” content
26
Useful Tools and References
Yahoo YSlow plug-in to Firefox: a must have!• http://developer.yahoo.com/yslow/
JMeter load tester• http://jakarta.apache.org/jmeter/
Heritrix web crawler• http://crawler.archive.org/
Varnish HTTP Accelerator• http://varnish.projects.linpro.no/
Yahoo web site performance tuning best practices• http://developer.yahoo.com/performance/rules.html
Oracle UCM performance tuning guide:• http://download.oracle.com/docs/cd/E10316_01/cs/cs_doc_10/documentati
on/admin/performance_tuning_10en.pdf
27
Now Available…
28
My Company: http://bezzotech.com
My Blog: http://bexhuff.com
My Self: [email protected]
Questions?