Upload
hackday08
View
697
Download
0
Embed Size (px)
DESCRIPTION
Uploaded from SlideSearch via http://www.plone4universities.org/Members/cmetc/dh-slides-perf.ppt
Citation preview
1
Delivering High Performance Plone
Internet Development Group (ILRT)Dominic Hiles
2
Outline
• Measuring performance and requirements
• Profiling code• Native caching in Zope and ZEO• Using Zope’s RAMCache Manager• Caching externally e.g. Squid• Scaling it all up -> load balancing and
redundancy• What’s on our horizon?
3
Measuring performance
• Decide how much performance you actually need– Easier when migrating an existing system, just analyse the
web logs and/or use monitoring tools– Allow plenty of headroom
• Use benchmarking tools, e.g. Apache Bench to assess performance before and after changes– Test different content types in isolation e.g. web pages, file
objects, images, CSS– Consider authenticated vs. anonymous access– Use varying numbers of requests and request concurrency– Use a reasonable sample size (for example all Web pages
on the site)
4
Profiling code
• Identify the “slow” content types with benchmarking
• Profile the code using appropriate tools, e.g. ZopeProfiler, PTProfiler
• Analyse the slowest parts of the code in more detail and optimise, but don’t necessarily aim for perfection!
• Re-test when you’re done to confirm your diagnosis
5
Hassle-free caching• Zope provides caching “out-the-box”• The ZODB cache size controls the number of
objects loaded from the ZODB and held in a RAM-based cache– If you’ve got lots of memory, make this bigger (e.g. 5,000 –
20,000 objects)
• ZEO also provides a disk-based ZEO cache, which holds objects loaded from the ZEO server– You’re likely to want to make this bigger than the default,
e.g. 100MB
• Performance of both can be assessed using standard Zope tools
6
More complex caching
• Zope provides a RAMCacheManager– Stores the results of e.g. a Python Script or Zope Page
Template in RAM. Results should therefore be lightweight and/or not too numerous.
– Useful for:• objects that change rarely e.g. a “Not Found” (404) response• objects that change according to a limited number of
parameters e.g. month of year• other specialised objects that you may have written to take
advantage of it– In a ZEO setup, the cache exists (and may be different) on
each ZEO client• This can complicate the process of programmatically removing
objects from the cache when stale
7
HTTP Caching• Zope/Plone can set a series of headers on the content it
serves. These determine how long the content should be considered “fresh” for
• Upstream caches (either your own or external ones) and users’ browsers inspect these headers and hold the content for the specified time, checking back for updated content when necessary
• Cache strategies are complex to implement and depend on your specific configuration
• CacheFu (a Plone product) provides some common strategies and some good documentation. The Cacheability Engine is useful for testing and also provides some good documentation.
• Once you have the correct headers, just implement your own cache…
8
Implementing an HTTP Cache
• Caches serve content much more quickly than Plone can, as they don’t need to generate the content, just serve it– Say, 2 requests/second vs. 100s – 1000s requests/second
• A good caching strategy should see most of your content (~90% in our case) being served from this cache, minimising the load on your Zope servers
• Use whatever suits your environment e.g. Squid, Apache 2.2 + mod_cache
9
Scalability and redundancy• ZEO
– Client/server architecture– ZEO Clients do the “hard” processing work and can be
spread across different servers– This provides redundancy and scalability– Bottleneck may just move to ZEO server instead!
• Keeping a “warm” copy of the ZEO server as well, gives a fully redundant Zope system
• Multiple ZEO clients need something to distribute the traffic between them -> a load balancer e.g. Pound
• Ensure that this redundancy is carried through the rest of the architecture
10
Bristol CMS Architecture
ZEO server
ZEO server ZEO client
ZEO client
ZEO client
ZEO client
ZEO client
ZEO client
Pound load balancer
Squid cache and redirectors
Pound load balancer
Squid cache and redirectors
Apache and onward links to other parts of
the UoB Web
Failover solution
implemented along lines of
Linux HA project, using IP takeover
Apache and onward links to other parts of
the UoB Web
Manual failover
12
Summary
• Measure performance before you start• Profile code and fix any bottlenecks• Use the “free” Zope caching• Use Zope’s RAMCache Manager• Implement an HTTP cache• Use ZEO and load balancing to create a
redundant and scalable system
13
Other ideas we’re looking at• memcached – a generic cache that could be
shared across all the ZEO clients (even on different servers)– May use this in preference to Zope’s RAMCacheManager– Would allow cached SQL queries to be shared across all
servers
• Splitting out the Data.fs, mounting different sites and/or the portal_catalog separately
• Increasing RAM • Using multiple “throw away” servers for the
ZEO clients• Improving the specification of the ZEO server