Upload
tess98
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Managing Enterprise Directories: Operational Issues
Performance Monitoring
Brendan Bellina, University of Notre DameBase CAMP – Tempe, ArizonaFebruary 5-7, 2003
Copyright Brendan Bellina, 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
Base CAMP - February 5-7, 2003 Middleware: Directories 2
How to present in 33 minutes (or less!)
• No humor• No arbitrary anecdotes• No incoherent rambling• … but how will we know it is a middleware
presentation?
That is why there is a page footer.
One more thing…
Volunteer alarm clock
Base CAMP - February 5-7, 2003 Middleware: Directories 3
Presentation Overview
• Why Monitor Performance• What Performance Metrics To Monitor• How To Gather Performance Metrics• Tools for Performance Monitoring @ ND
– Orca– Orcallator– Look
Base CAMP - February 5-7, 2003 Middleware: Directories 4
Why Monitor Performance
• Availability
It’s up… isn’t it?– Directories often require 7x24 availability
• Responsiveness
It’s fast enough… maybe.– Directories often require extremely fast response
• Scalability
We can handle that… I think.– Structural changes, indices, increases in the # of entries or #
of attributes can all affect performance.
Base CAMP - February 5-7, 2003 Middleware: Directories 5
What Performance Metrics To Monitor
Response time
Connection requests
Bind requests
Bind errors
Search requests
Search errors
Average count & size of search results
Directory Cache Hits
Directory Cache Tries
Bind response time
Search response time
Current connections
Avg connection length
Current binds
Current searches
# Bytes transmitted
# Entries transmitted
Base CAMP - February 5-7, 2003 Middleware: Directories 6
How To Gather Performance Metrics
Directory internal query (iPlanet, OpenLDAP)base search on cn=monitorldapsearch -h directory.whatsamattau.edu -b"cn=monitor" -s base objectclass=*
objectclass=topobjectclass=extensibleObjectcn=monitorversion=Netscape-Directory/4.16 B01.300.2035threads=30currentconnections=27totalconnections=118499dtablesize=1024readwaiters=0
Base CAMP - February 5-7, 2003 Middleware: Directories 7
How To Gather Performance Metrics
base search on cn=monitor cont.opsinitiated=410383
opscompleted=410382
entriessent=173812
bytessent=59522106
currenttime=20030123212714Z
starttime=20030123115756Z
nbackends=1
dataversion=directory.whatsamattau.edu:389 020010212195544 0
ldapserverconfigdn=cn=ldap://:389,dc=directory,dc=whatsamattau,dc=edu
backendmonitordn=cn=monitor,cn=ldbm
Base CAMP - February 5-7, 2003 Middleware: Directories 8
How To Gather Performance Metricsbase search on cn=monitor, cn=ldbmldapsearch -h directory.whatsamattau.edu -b"cn=monitor,cn=ldbm" -s
base objectclass=*
objectclass=top
objectclass=extensibleObject
cn=monitor
database=ldbm
readonly=0
entrycachehits=505559
entrycachetries=540329
entrycachehitratio=93
currententrycachesize=10000
maxentrycachesize=10000
dbchehits=1011415
dbcachetries=1043689
dbcachehitratio=96
dbcachepagein=32268
dbcachepageout=851
dbcacheroevict=20160
dbcacherwevict=0
dbfilename-0=wuAttribute_1.db2
dbfilecachehit-0=26
dbfilecachemiss-0=4
dbfilepagein-0=4
dbfilepageout-0=4
Base CAMP - February 5-7, 2003 Middleware: Directories 9
How To Gather Performance Metrics
• LDAP log processing– Read the log files looking for events– Calculate statistics
• Directory Probing– Ping– Query to directory service– Bind to directory service– SNMP
Base CAMP - February 5-7, 2003 Middleware: Directories 10
How To Monitor Performance Metrics
• NMS (Network Management System)– HP OpenView– CA/UniCenter TNG– IBM NetView
• Operating System Tools– Win NT Event Viewer– W2K WMI (Windows Management Instrumentation)– W2K Network Monitor; W2K System Monitor– Sun Solaris SE Toolkit
Base CAMP - February 5-7, 2003 Middleware: Directories 11
Tools for Performance Monitoring @ ND
• Orca - http://www.orcaware.com• Orcallator (collector for Solaris statistics)• Look – LDAP operational Orca “k”ollector
Base CAMP - February 5-7, 2003 Middleware: Directories 12
Orca (http://www.orcaware.com)
About Orca:• Generates web friendly performance graphs based on collected
statistical data.• Free (will take donations)• Open Source• Perl• Used by Yahoo!/Geocities• Used at Notre Dame on 100+ servers• Hourly, Daily, Weekly, Monthly, Quarterly, Yearly, All• Did I say free?
Base CAMP - February 5-7, 2003 Middleware: Directories 13
ORCA @ Notre Dame
• CRON job runs every 15 minutes to regen graphs• Orcallator (SymbEL script included with ORCA)
– Written to monitor web servers and Solaris stats– Requires Sun Solaris SE Toolkit (free from http://
www.setoolkit.com)– Runs on each directory server to collect statistics – 5 minute
averages, 30 second peaks– Data is consolidated on one server for ORCA processing– Modifications by ND to read iPlanet Dir Server 4.16 log
Base CAMP - February 5-7, 2003 Middleware: Directories 14
ORCA @ Notre Dame
• ND Modifications- LDAP connection rate- LDAP Bind rate- LDAP Bind error rate- LDAP search rate- LDAP search error rate- LDAP search results returned rate- Picture is worth 1000 words:
- Go to <http://eds.nd.edu/perfmon>
Base CAMP - February 5-7, 2003 Middleware: Directories 15
Look – LDAP operational Orca "k"ollector
• LDAP supplement to Orca and/or Orcallator• Perl script• Produces Orca compatible output files• Processes iPlanet 4.16 log file• Queries directory cn=monitor• Queries directory cn=monitor, cn=ldbm• Beta 0.90 version currently available at
<http://www.nd.edu/~bbellina/downloads/Look>
Base CAMP - February 5-7, 2003 Middleware: Directories 16
Look – LDAP operational Orca "k"ollector
Connection Rate
Bind Rate
Bind Error Rate
Search Rate
Search Error Rate
Search Results Returned Rate
Connections
Operations Initiated
Operations Completed
Entries Sent
Bytes Sent
Entry Cache Hits
Entry Cache Tries
Cache Hit Ratio
Database Cache Hits
Database Cache Tries
Database Cache Hit Ratio
Peaks and AveragesView live results at <http://eds.nd.edu/perfmon/Look>
Base CAMP - February 5-7, 2003 Middleware: Directories 17
ORCA Look Graph Example #1
LDAP search rate before and after email routing was transferred from email-specific directory to EDS:
Base CAMP - February 5-7, 2003 Middleware: Directories 18
ORCA Look Graph Example #2:Troubleshooting
Daily abuse of open LDAP connections.What is different about the weekend?
Base CAMP - February 5-7, 2003 Middleware: Directories 19
ORCA Look Graph Example #3:Troubleshooting
How long has this been going on? Week 49.What happened in mid-December? Hmmm…
Base CAMP - February 5-7, 2003 Middleware: Directories 20
ORCA Look Graph Example #4:Troubleshooting
The culprit has been identified and shot.
Base CAMP - February 5-7, 2003 Middleware: Directories 21
ORCA Look Graph Example #5:Troubleshooting
The sky is blue, the birds are chirping, and all is right with the world.
Base CAMP - February 5-7, 2003 Middleware: Directories 22
LinksNotre Dame Enterprise Directory Service
http://www.nd.edu/~eds
Notre Dame EDS Performance Statistics - Orcallator
http://www.nd.edu/~eds/perfmon or http://eds.nd.edu/perfmon
Notre Dame EDS Performance Statistics – Look
http://www.nd.edu/~eds/perfmon/Look or http://eds.nd.edu/perfmon/Look
ORCA software
http://www.orcaware.com
Look software
http://www.nd.edu/~bbellina/downloads/Look
Brendan Bellina (mailto:[email protected])
<http://www3.nd.edu/~eds/cgi-bin/nd_ldap_search.pl?ldapfilter=uid=bbellina>
Base CAMP - February 5-7, 2003 Middleware: Directories 23
Copyright Statement
Copyright © Brendan Bellina, 2003. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.