84
Taking Control of Your Domino Domain © 2010 Technotics, Inc. Andy Pedisich Technotics

Taking Control of Your Domino Domain - AdminCamp · Taking Control of Your Domino Domain ... And a secondary person just in case the primary is unavailable ... Allows specific admins

Embed Size (px)

Citation preview

Taking Control of Your Domino Domain

© 2010 Technotics, Inc.

Andy PedisichTechnotics

In This Session ...

• Very few people have the opportunity to start up a brand new

Lotus Notes domain

� We usually start working with one that has already been

running for a while

� You don’t know how it’s been configured

� Or why!

1

• This session will take you through the steps to evaluate what

you’ve inherited

� It will help you to understand what the issues are and how you

can remediate them

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

2

• Picking policies that make sense

• Wrap-up

Start With the Access Control List of the Domino Directory

• More than two people with Manager Access to the directory is

too many

� There should be a primary person who has Manager Access

� And a secondary person just in case the primary is

unavailable

� Or if something horrible happens to the primary person like

they get a new position in Development

3

they get a new position in Development

• More than two managers simply is not necessary

� The ACL of the address book should remain quite static

� It should rarely change

� No one else really needs that kind of power

Pay Special Attention to Groups

• Sometimes people use groups that are intended for mailing lists

as groups in the ACL

� While that might be convenient, it often gives power to people

inadvertently

• Review all groups that are in the ACL

� If you have real doubts, start over and create new groups

4

Getting Rid of Former Administrators and Developers

• If you took over a domain and the old administrators are no longer

in charge, you need to extricate them

� They must be removed from ACLs

� This is not always easy, since the ACL also has roles

� Some roles control who can manage policies

• If an administrator has been retired you must re-sign all policy and

5

• If an administrator has been retired you must re-sign all policy and

policy settings documents, and agents

� Use a “functional ID” standard ID to do this, such as

� Notes Designer/MyCorp

• Then remove all the ACL entries for the admin

� Put him in the “denied” group if required

Where are the former administators?

• A great way to find all the places a former administrator might be

is to run a Find User in the Notes Administrator

� The AdminP task will do a search of the environment for you

6

AdminP Delivers the Answer

• AdminP will produce a document in the ADMIN4.NSF that contains

doclinks to all the places where the former administrator’s name

appears

� It’s a very helpful step in the process of getting the environment

under control

7

Eliminate Excess Designers

• The Domino directory design should rarely change

• When changes are needed, they should be carefully thought out

� Changes should be made only by a special functional ID

� Rarely, if ever, by administrators “on-the-fly”

• All modified design elements should be signed by this functional

ID

8

ID

� Such as Directory Manager/Mycorp or Design Manager/MyCorp

� Eliminate all other directory designers

Editors Are More Powerful Than You Might Think

• Editors are the sneakiest ones in the bunch

� They can modify any document

� Server docs, person docs, connection docs

� Regardless of the roles you assign them to

• They don’t follow roles when they cut out a document and put it

into a local copy of the directory

9

into a local copy of the directory

� Then they change the document and paste it right back in

� This technique can easily enable them to become the most

powerful person in the domain

� Full Access Administrator!

Keep only a few editors

• There will always be a need for certain users to have Editor

Access to the Domino directory

� Try to keep their numbers to a bare minimum

� Establish a very clear corporate policy

� Make it known to any editors that unauthorized changes to

the Domino directory are strictly forbidden

10

Functional Accounts Should Not Have Power

• You might find that a functional account has powerful privileges in

the ACL of the directory

� You’ll never be able to determine who has this ID

� Or what it is being used for

• Eliminate all functional accounts from the ACL of the Domino

directory

11

� In fact, make sure the ID is never used for any task

� Functional IDs should be used only for signing design

elements, agents, and policies

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

12

• Picking policies that make sense

• Wrap-up

HTTP Passwords Might Be Stored Incorrectly

• Back in Notes R4, Lotus used a method to store HTTP passwords

in the directory that was pretty weak

� If the same word was used by two different people, it was

stored the same way and looked the same in person documents

• For example, the encrypted password in the HTTP password

shown below is “lotusnotes”

It will look exactly the same for every person using the old R4

13

� It will look exactly the same for every person using the old R4

password storage method

Many Domains Never Changed

• Many administrators never upgraded to the more secure method

of password storage

� If your HTTP passwords look like this with all uppercase letters,

you are using the older, less secure methodology

� 553A2870901600E3385812D1A734A052

� If they look like this, you are in great shape and using the

newer, more secure methodology

14

newer, more secure methodology

� GffemZr/WQl24KbgRUj5

Two Steps to More Secure Password Storage

• First, you must update the password storage in existing person

documents

� Select all person documents

� Use menu choice “Upgrade to More Secure Internet Password”

� You will not be changing any user passwords

� You'll only be changing the way they are stored

15

You'll only be changing the way they are stored

Make Sure All Future Passwords Are Stored Securely

• Second, change the configuration so that all future passwords will

be stored using the newer method

� This feature is controlled in your domain’s directory profile

• Edit the directory profile in your domain’s directory

� Change it to “Use more secure Internet passwords”

� You will not be changing any user passwords

16

You will not be changing any user passwords

� You'll only be changing the way they are stored

How Are ID Files Created in Your Domain

• In a lot of domains, when administrators create a user they must

have direct file-level access to a certifier ID file

� And they must know the password to the ID file

• Once an admin has the certifier ID, they probably have it for the

rest of their lives

� They know the password

17

� There is a risk that they will use it maliciously

� Creating new users or to impersonate existing users

• That’s why you should investigate using the Certificate Authority,

native to Lotus Notes

Avoid All of Those Issues with Certificate Authority

• The Certificate Authority (CA) provides many advantages

� Grants an administrator the ability to create users one day

� And take it away the next

� Since they don’t have the ID and the password, they can

only do what you authorize

� Allows specific admins to create users for each organizational

18

� Allows specific admins to create users for each organizational

unit (OU)

� This lets you distribute user creation regionally or within

certain departments in your domain

� Allows you to use consultants (when needed) to create users

without fear of compromising the keys to your domain

• You can always use the certifier IDs if you want to

The Process of Setting Up Certificate Authority

• You can set up the CA in three easy steps

1. Migrate the certifier IDs to encrypted databases on a server

2. Specify who will be have the key roles for the certifier

� Certificate Authority Administrator (CAA)

� Registration Authority (RA) to create users and servers

3. Add “CA” to the SERVERTASKS= parameter and restart the

19

3. Add “CA” to the SERVERTASKS= parameter and restart the

server

� Or you could just do LOAD CA at the console

• There are plenty of details on the steps to do this in

Administrator Help

� Set up CA today and keep your domain more secure

� You’ll never regret it

Check the Implementation of Full Access Administrator

• Full Access Administrator is the highest level of administrative

access to the server

� Here are just some of the rights available:

� Manager access, with all access privileges enabled, to all

databases on the server, regardless of the ACL settings

� Access to all documents in all databases, regardless of

Reader names fields

20

Reader names fields

� The ability to create agents that run in unrestricted mode

with full administration rights

� Access to any unencrypted data on the server

Permission Granted in the Server Document

• This privilege is activated in the Notes Administrator client

• It is granted to users or groups in server documents

� You must check every server document to see who has this

supreme power

� This power must be monitored!

21

Domino Logs Full Access Administrator

• Whenever an administrator becomes a Full Access Administrator,

it is logged in the Domino log

� You should be notified when this happens

� We’ll talk about how to do this in a few moments

22

03/13/2009 06:52:39 PM Opened live remote console session for Andrew M Pedisich/Technotics03/13/2009 06:52:43 PM Andrew M Pedisich/Technotics was granted full administrator access.03/13/2009 06:52:51 PM Opened live remote console session for Andrew M Pedisich/Technotics03/13/2009 06:52:57 PM AMgr: Start executing agent 'replicaCheck' in 'blogs\BlogLog.nsf' by Executive '1'03/13/2009 06:52:57 PM AMgr: 'Andrew M Pedisich/Technotics' is the agent signer of agent 'replicaCheck'

Use Password Checking

• If someone’s ID and password has been compromised, how do

you keep the thief from using it?

� Best Practice is to use password checking

� It’s enabled in the Security section the server document

• Check to see if it’s enabled in your new domain

� If it’s not, work on an implementation plan

23

� If it’s not, work on an implementation plan

Complete Configuration in Person Documents or Policies

• After you turn it on in server documents, you must activate it for

your users

� You have a choice of turning it on:

� In person documents

� In policies

� Policies are easier to control and are considered by

24

� Policies are easier to control and are considered by

many experts to be the Best Practice

Password Checking in Person Documents

• Password checking is enabled on the administration tab

� Look for the Password Management section

• Trying it on a few people is a good way to get started

25

Other Decisions Controlling Passwords with Person Docs

• You’ll find a Set Password Fields Action in the Person View of the

Domino directory

� This will let you access dialogue boxes that will trigger AdminP

processes to control password management

26

Security Policies Can Manage Passwords

• Security policies also let you control the change interval and grace period

�But they also let you control how often a user can repeat using a password

� That’s password history

�And you can control when the

27

�And you can control when the user is prompted to change their password

� That’s the warning period

�You can even provide a custom warning message

• Plus there are internet password controls as well

A Good Solution That Takes Effort

• To be honest, password checking can be a real pain

� It is complex, but it is necessary

• When configured properly, Domino checks the password given by

the user and makes sure that it matches with a “password digest”

in the person document

� They must match or the user is denied access until they

change their Notes ID to the right password

28

change their Notes ID to the right password

� Hey, what’s wrong with that!

• This is the perfect solution if a user’s ID is compromised

� It will prevent the thief from using the ID even if the old

password is known

Public Key Checking Is Also Important

• Turn on the option to “Compare public keys”

� If an ID has been stolen, you can issue a new public key

� The person who stole the ID will be unable to connect to

the server because the public key doesn’t match the one in

the person document in the Domino directory

• Best Practice is to use the option for “all users”

29

• Check to see if your predecessor administrators have turned on

public key checking in the server document

Release 7 Added Public Key Mismatch Logging

• Before you turn on public key checking, you can turn on the

option to log public key mismatches

� This will let you fix cases where the public key in the ID file has

not properly synchronized with the one in the address book

� You’ll have to ask the user to copy their public key and

email it to you, so you can paste it into their person

document

30

document

� Ugh! It’s a painful process, but necessary

Review How Deny Access Groups Are Used

• Users that should no longer access Notes should be placed in the

Deny Access group that your server uses to keep people out

� It is not enough to remove names from the Notes environment

� The smallest access can compromise the environment

� Put the users in a Denied group and do not let them access

the server

31

How Is a Notes User Account Terminated?

• This is one of the biggest issues with many enterprises

� It’s not a Lotus Notes issue

� But it is a security issue and too many companies don’t do

this well enough

• There should be a spelled out process for when people are no

longer working for your company

32

� And part of that process should be notifying Notes

administrators when the event occurs

� Now that you have inherited the domain it is up to you to

make sure this piece is in place

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

33

• Picking policies that make sense

• Wrap-up

Create a Server Management System

• You’re going to be discovering issues and trying to remediate

them

� You need a drawing board and a safety net

� That means a test area and a backout system

• For a test area, install partitioned Domino servers on a laptop or

desktop

34

� I keep four partitioned servers on a VMWare virtual machine my

laptop and use them to test configurations I wouldn’t dare try in

production

� I use an old copy of XP Professional as my virtual maching

operating system

Your Backout System

• The Domino directory is the heart of your domain

� Make copies of it regularly, just in case important system

documents are changed or accidently deleted

� There is nothing quite as rewarding as just copying and

pasting a person document when it is removed by mistake

• Create a “Backout Address Book” from the PUBNAMES.NTF

template

35

template

� Before you change any server, server configuration, connection

or any other important document make a copy and paste it into

the backout address book

� If a change you make doesn’t work out well, you can return

it to its original state

Server Configurations Must Be Reviewed

• Now that it’s your domain you must review all server

configurations so that they:

� Perform optimally

� Are secure as possible

� Are easy to monitor

� Take advantage of Lotus Notes/Domino features

36

� Take advantage of Lotus Notes/Domino features

Are Your Servers Using Transaction Logging?

• Transaction logging can give you:

� The ability to do incremental backups rather than full backups

every night

� This requires third-party software that uses C-API

� Better server performance

� Faster server restarts

37

� Faster server restarts

� Point-in-time database recovery so that almost no data is lost in

the event of a disaster

• Whether you use them or not depends on:

� Your backup strategy

� How much data is on the server

� Whether you want to put up with transaction log issues

Setting Up Transaction Logs

• Enabled in Server Document

• Log path refers to location of log

• On Wintel system

� For best performance and best recovery,

log should be located on separate drive

� Drive should use striping (Raid 0)

38

Drive should use striping (Raid 0)

or Mirroring (Raid 1)

� Never on the system drive (C:\)

� Never on the data drive

• AIX and iSeries can keep tlogs on same drive as data with no

issues

Transaction Logging Basics

• Transaction logging records changes made to databases and

writes them to a log or “cache”

� The logged transactions are written to disk in a batch when

resources are available or at specified intervals

� If the server crashes before the data is written to the log, the

server “plays back” the log when it restarts

Result: No lost data and faster restart without fixups

39

� Result: No lost data and faster restart without fixups

• The caching increases “apparent performance” because heavy

database usage doesn’t cause a bottleneck at the server

Logging Styles

• Circular logging

� Reuses log space and overwrites old transactions

� Best used for fast restarts

• Archive logging

� Reuses old log space after they are

archived/backed up by the third-party

40

archived/backed up by the third-party

product

� Supports incremental backups

� Log file can be reused once it is inactive

• Linear logging

� Reuses the log files and overwrites old transactions for log size

greater than 4GB

Real Life and Transaction Logs

• Do all Notes shops use transaction logs?

� Not everyone does!

� Some feel it “adds complexity and instability”

� There is some truth to this, but usually once the bugs are

ironed out, it’s great!

• If you want incremental rather than full backups every night,

41

• If you want incremental rather than full backups every night,

archive style transaction logging is a requirement

� Along with third-party backup software

• If restarts after crashes take too long because of fixup, consider

transaction logs in circular style

Automatic Restarts

• All servers should be configured to restart automatically after a

crash

� Check the Basic tab and Automatic Server Recovery section

� This is so easy to do and can help ensure server uptime,

since manual intervention might not be needed

� I am always shocked to discover servers that are not

configured to use this great tool

42

configured to use this great tool

How Many Replicators Do Servers Use?

• By default, only a single replicator thread is used by Domino

servers

� This is definitely not enough for hub servers that initiate

replication

• How do you know if you need more?

� Check for the completion of the replication process

43

� Check the Replication section of the log

� Compare it against connections docs

� Make sure that the server is completing replication with all

servers it is supposed to

Also Check End-to-End Replication Window

• Check changes to databases, such as directory, to make sure that

total replication time from start to finish meets the requirements

� Complete replication throughout your environment should meet

a standard that ensures security changes will be distributed in

a timely way

� In most environments, this ranges from 1 hour to 3 hours

In extremely large environments, this can be 4 to 5 hours

44

� In extremely large environments, this can be 4 to 5 hours

Closing the Replication Gap

• By adding replicators

� You can ensure that hubs replicate with all the servers they are

supposed to

� And that all databases are replicated properly

� Reduce the time it takes for hubs to complete their cycles

• Use the “load replica” console command to temporarily add

45

• Use the “load replica” console command to temporarily add

replicator tasks

� But replicators added this way won’t be there after the next

server restart

• To permanently add replicators, use the REPLICATORS=

parameter and specify the number you need

Real Life and Replicators

• In a hub-spoke or hub-subhub-spoke design, additional replicators

are always added to the hubs

� Usually three to six or even more replicators are added,

depending on the strength of the platform and OS

� Generally two should be used on mail servers

• And always remember to restrict other resource hogging activities

on hub servers

46

on hub servers

� Disable full text indexing on hubs

� Update_No_Fulltext=1

� Don’t add additional view indexers

� Don’t use any tasks that aren’t absolutely necessary, such as

HTTP, Collect, Maps, and LDAP

System Resources Are not Limitless

• Provide enough resource space for Agents without letting them take over the server�Agent properties are controlled

in the Server document

� Daytime and nighttime defaults are a good starting point for mail servers

47

point for mail servers

How to Tell If Agent Parameters Need Adjustment

• Most common problem

� Agents that don’t complete within time limits

� Find these by performing a Log Analysis

� Look for:

Agent 'Agentname' in 'db.nsf' did not process all documents

successfully. Check the Agent Log for more information:

Agent did not complete within the time limit

48

Agent did not complete within the time limit

� You could even set up an event notification to tell you

when this occurs or use Domino Domain Montoring

• There are plenty of reasons why Agents can run long

� The Agent is having errors and is looping forever

� The Agent is poorly written and not efficient, or just really busy

Users Might Report That Agents Are Not Running

• You can correct the problem by:

� Increasing the number of max concurrent Agents

� Increasing the time limit

• If Agents are too demanding, you must not over-commit resources

or performance will suffer for users

• Solution:

49

• Solution:

Create a brand-new category of Domino servers in your

environment – The Agent Server

The Agent Server

• Agent servers only run scheduled Agents on databases that are

replicated into the environment

� They exclusively run Agents and are not generally accessed by

users

• Agent servers allow you to commit all resources to the running of

Agents

Run as many concurrent Agents, daytime and nighttime, as the

50

� Run as many concurrent Agents, daytime and nighttime, as the

system will allow

� Crank up the time limit

� These Agents can run for hours and not affect users!

Server Configuration Documents

• You don’t necessarily need a separate server configuration

document for each server

• But you do need a default server configuration document

� This applies to all servers

• Use it to apply NOTES.INI parameters domain wide, like

� SERVER_SESSION_TIMEOUT=30

51

� SERVER_SESSION_TIMEOUT=30

� This logs off idle users after 30 minutes

� Otherwise they are connected for a default of 4 hours

even when they are doing nothing

Help to Determine Why Servers Crash

• Another great tool is the centralized diagnostic collection

� Use the default server configuration document to collect this

information too

• Create a mail-in database from the Lotus Notes/Domino Fault

Report template – LNDFR.NTF

� All diagnostic info is sent automatically after a server crash

52

� It’s a lot of the information Lotus asks for when you make a

support call

Review Program Documents

• Lotus/IBM recommends:

� A monthly server restart

� A weekly fixup, updall, and compact of all databases while the

Domino server is online every week

� If you’re using transaction logging then you don’t need to

run fixup

53

• This is great advice:

� Make sure program documents are in place to ensure that

the weekly server tasks are done

� Review the monthly schedules to make sure your Domino

servers are restarted once per month

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

54

• Picking policies that make sense

• Wrap-up

The Two Things Needed

• Two things are required for statistics collection:

� The Collect task must be running on any server that is

designated to collect the statistics

� Not all servers should run the Collect task

� The EVENTS4.NSF database must have at least one Statistics

Collection document

55

� Statistics should be collected centrally on one or two

servers so that the data is easy to get to

� Stats should be collected every hour to be effective

� EVENTS4.NSF should have the same replica on all servers

in the domain

Want to Add Every EVENTS4.NSF to Your Desktop?

• Add this code to a button on your toolbar

� This is courtesy of Thomas Bahn

� www.assono.de/blog

� He’s speaking at this conference

_names := @Subset(@MailDbName; 1) : "names.nsf";

56

_servers := @PickList([Custom]; _names; "Servers"; "Select servers"; "Select servers to add database from"; 3);

_db := @Prompt([OkCancelEdit]; "Enter database"; "Enter the file name and path of the database to add."; "log.nsf");

@For( n := 1; n <= @Elements(_servers); n := n + 1; @Command([AddDatabase]; _servers[n] : _db) )

Add a Database to the Desktop

• This code will prompt you to pick the servers that have the

database you want on your desktop

� Then it will prompt for the name of the database

� And open it on all the servers you’ve selected

• Use it to make sure all the EVENTS4.NSF are the same replica in

your domain

57

A Required Design, but No Required Name

• There has to be a Statrep.nsf on every server

� It is used by the server to store monitoring data

� It must be designed using the Statrep5.ntf Monitoring

Results template

� Its default title is Monitoring Results

• But you don’t have to use one of those for your statistic collection

58

• But you don’t have to use one of those for your statistic collection

repository

� Create your own collection points and give the database a

unique name

City Collecting Server Monitoring Results DB

New York USNYAdmin1 USStatrep.nsf

Amsterdam EUNeHub01 EUStatrep.nsf

Three Important Things to Monitor

• Set an Event Generator in EVENTS4.NSF to let you know when

free disk space on data drives drops below a threshold such as

10%

� Set up an Event Handler so that you are notified by email if this

happens

59

Another Monitor That Everyone Should Use

• The ACL of Names.nsf should be monitored for changes in every

Notes domain

� Once properly set, the ACL of Names.nsf should rarely change!

� All kinds of bells and whistles should go off when it does

� Here’s how to set up the monitoring of the ACL

� Select New Database Event Generator

60

� Select New Database Event Generator

More on ACL Monitoring

• Select Names.nsf

� You can choose either a single

server, such as the administration

server for the address book, or

� All servers in the domain

• I like to pick all servers in the domain

61

� Admins won’t get away with

anything!

� But I do get a storm of messages

when an ACL change occurs

� Every server tells me about

the change

Keep Tabs on Full Access Administrator Enabling

• If you wanted to be notified every time someone turns on Full

Access Administrator, you could look for the following string: full

administrator access

� Set up one notification to log to Statrep

� And another notification to mail it to you so you always know

who is using this powerful privilege

62

Ensure Domino Domain Monitoring Is Correctly Configured

• DDM.NSF has the most value when it is the central repository for

all issues

� It will contain all of the issues that come from all of the servers

• This does not happen on its own

� There is no collection hierarchy set up by default

� Each domain has different monitoring requirements

63

Each domain has different monitoring requirements

Collection Hierarchy Is a Must

• Without a collection hierarchy, DDM probes run on

a server and report events to DDM.NSF that are on

that server

� Then they remain only on that server’s replica of DDM.NSF

� You have to check the DDM database on each server to

evaluate problems and discover potential issues

This is time consuming

64

� This is time consuming

� It reduces the time you could be spending solving

problems

� And you might miss important issues

Aggregate Data Centrally

• A DDM server collection hierarchy lets you aggregate the data

onto a key server or servers

� This must be configured in the EVENTS4.NSF

• The simplest hierarchy is to configure one server to collect from

all servers in the domain

65

More Complex Scenarios Are Possible

• Perhaps as you become more familiar with DDM, you’ll want to roll

up some data regionally

� So that regional administrators receive only information that is

pertinent to the server they maintain

66

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

67

• Picking policies that make sense

• Wrap-up

Cluster Replication Basics

• Cluster replication keeps the database on the primary server in

sync with the replica on the failover server

� Cluster replication is an event-driven process that occurs

automatically when a change is made to a database

� It’s vital that these replicas are synchronized

� But by default, servers in a cluster only have a single

cluster replicator thread between them

68

cluster replicator thread between them

Can the Single Cluster Replicator Keep Up?

• Occasionally there is too much data changing to be replicated

efficiently by a single cluster replicator

� If cluster replicators are too busy, replication is queued until

more resources are available and databases get out of sync

� Then a database on a failover server does not have all the

data it’s supposed to have

If users must failover to a replica on a different server, they think

69

• If users must failover to a replica on a different server, they think

their information is gone forever!

� All because replicas will not have the same content

� Users need their cluster insurance!

How Many Is Enough?

• Adding a cluster replicator will help fix this problem

� Use this parameter in the Notes.ini

� CLUSTER_REPLICATORS=#

� Add one dynamically from the console using this command

� Load clrepl

• The challenge is to have enough cluster

70

• The challenge is to have enough cluster

replicators without adding too many

� Adding too many clusters will have a negative effect on

server performance

• Here are some important statistics to watch so that you can make

a wise decision about how many to add!

Key Stats for Vital Information About Cluster Replication

Statistic What It Tells You Acceptable Values

Replica.Cluster.

SecondsOnQueue

Total seconds that last DB

replicated spent on work

queue

< 15 sec – light load

< 30 sec – heavy

Replica.Cluster.

SecondsOnQueue.Avg

Average seconds a DB spent

on work queue

Use for trending

Replica.Cluster. Maximum seconds a DB Use for trending

71

Replica.Cluster.

SecondsOnQueue.Max

Maximum seconds a DB

spent on work queue

Use for trending

Replica.Cluster.

WorkQueueDepth

Current number of databases

awaiting cluster replication

Usually zero

Replica.Cluster.

WorkQueueDepth.Avg

Average work queue depth

since the server started

Use for trending

Replica.Cluster.

WorkQueueDepth.Max

Maximum work queue depth

since the server started

Use for trending

What to Do About Stats Over the Limit

• Acceptable Replica.Cluster.SecondsOnQueue

� Queue is checked every 15 seconds, so under light load should

be less than 15

� Under heavy load, if the number is larger than 30, another

cluster replicator should be added

• If the above statistic is low and Replica.Cluster. WorkQueueDepth

is constantly higher than 10 …

72

is constantly higher than 10 …

� Perhaps your network bandwidth is too low

� Consider setting up a private LAN for cluster

replication traffic

The Documents Have More Information

• The cluster documents have much better information than the default cluster views

� But they still lack key stats, although they are in each doc

73

Stats That Have Meaning but Have Gone Missing

• The TechnoticsSTATREP10-9.NTF tracks the key statistics you

need to help track and adjust your clusters

� It also has a column for the Server Availability Index

74

My Column Additions to Statrep

Column

Title Formula Formatting

Min on Q Replica.Cluster.SecondsOnQueue/60 Fixed (One

Decimal Place)

Min/Q Av Replica.Cluster.SecondsOnQueue.Avg/60 Fixed (One

Decimal Place)

75

Decimal Place)

Min/Q Mx Replica.Cluster.SecondsOnQueue.Max/60 Fixed (One

Decimal Place)

WkrDpth Replica.Cluster.WorkQueueDepth General

WD Av Replica.Cluster.WorkQueueDepth.Avg General

WD Mx Replica.Cluster.WorkQueueDepth.Max General

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

76

• Picking policies that make sense

• Wrap-up

The Big Picture on Policies

• I could do a whole morning’s presentation about policies and still

not cover everything about them

• Policies can make an administrator’s life much easier

� But many have failed to implement them

� If you’re taking over, make the move to implement policies

• There are two types of policies

77

• There are two types of policies

� Explicit policies – that can be applied to people

� Organizational policies – that are applied based on certifier

structure

� And in Release 8.5, dynamic policies that can be assigned

to users and to groups dynamically

Two Policies You Should be Using

• Use a desktop policy setting to collect fault reports from all users

when they crash

� Put them into the Lotus Notes/Domino Fault Reporting database

� Don’t bother prompting user to send the diagnostic reports

� Do not prompt users for comments

� It only makes them angry and wastes their time

78

Use Registration Policies to Set Defaults

• Use a registration policy to set consistent defaults for:

� Registration and mail server

� Password quality

� Mail template

� Internet domain

� And many other settings that are tedious to set and hard to

79

� And many other settings that are tedious to set and hard to

remember

What We’ll Cover …

• Locking down the Domino directory

• Reviewing and repairing domain security

• Ensuring servers are properly managed

• Configuring statistics, event monitoring, and DDM

• Guaranteeing clusters are ready for failover

• Picking policies that make sense

80

• Picking policies that make sense

• Wrap-up

Resources

• Lotus Domino server maintenance tips

� www-01.ibm.com/support/docview.wss?uid=swg21248830

• The Lotus Security Handbook

� www.redbooks.ibm.com/abstracts/sg247017.html?Open

• Lotus Education on Demand: Domino Domain

Monitoring (DDM)

81

Monitoring (DDM)

� www-1.ibm.com/support/docview.wss?uid=swg27007060

• Which Domino Server databases have replica IDs related to

names.nsf?

� www-1.ibm.com/support/docview.wss?uid=swg21099635

7 Things to Remember from this Presentation

• Review the access control list of the Domino directory and lock it down

• Remove all rights of deposed administrators

• Set up a small test environment to try things out without taking risks in production

• Create a backout address book so that you can easily recover from misconfigurations

82

from misconfigurations

• Make sure HTTP passwords are stored using the more secure methodology

• Check your clusters to make sure synchronization doesn’t lag behind

• Set up monitoring of free disk space and notification when it falls below thresholds

Do you have questions about this presentation?

83

How to contact me:

Andy Pedisich

[email protected]

http://www.technotics.com

http://www.andypedisich.com