Practical solutions for connections administrators

Prac%cal Solu%ons for Connec%ons Administrators

Christoph Stoe,ner &

Sharon Bellamy

Who are we?

Sharon Bellamy Mum of three, Star Wars & Disney fan – Mandalorian costumer & admin for iCosplay an;-‐bullying campaign Administrator and Implementer Specializes in IBM Connec8ons, WebSphere and other collabora8on solu8ons Working with WAS based products since 2003 Lover of Linux Christoph StoeGner Bavarian, Dad of two, likes hiking Administrator and implementer Specialized in the infrastructure of IBM Connec8ons and IBM Domino Lover of Linux

Agenda

Installing and Configuring Business as Usual •  Autostart

IBM WebSphere IBM HTTP Server

•  Performance Tuning DataSources

•  SeBng J2EE Security Roles •  Configure JVM Log Files •  Set JVM Heap Sizes

•  Check External ID against LDAP •  AcNvate and DeacNvate Users •  Synchronize User External IDs •  Work with Policies •  Adding Policies to personal or

community libraries •  Database reorganizaNon

These scripts were wriGen for Connec8ons 4.5 – Most will work with V5. There are a few issues with the member scripts, we hope to have updated V5 scripts

available very soon.

Agenda (cont..)

Troubleshooting Documenting • ApplicaNon state • Database access

• JVM Heap Sizes • JVM Log SeBngs • Used Ports within WebSphere • WebSphere Variables

These scripts were wriGen for Connec8ons 4.5 – Most will work with V5. There are a few issues with the member scripts, we hope to have updated V5 scripts

available very soon.

Disclaimer & Cau8on

• With scripts •  Shell / BASH / ZSH / KSH / SH, Jython / JACL •  PowerShell / Batch / VB, SQL

• You can... •  Save a lot of Nme! •  Change tons of stuff in seconds!

• TIPS: •  Create Backups •  Install to a Test system (if you don’t have one – BUILD ONE) •  Document your changes

BACKUP before you make any changes

The use of all scripts shown in these slides or downloaded from our repositories are WITHOUT WARRANTY and at your own risk!

Using the scripts

• Using the Community Scripts • Download links at the end of this presenta8on •  Extract zip to DMGR/bin

• all files are in subfolder ibmcnx •  rename ibmcnx_sample.properNes to ibmcnx.properNes* • edit ibmcnx.properNes* • configure loading the db2 jdbc driver*

*(more details in resources secNon of the presentaNon and at h,p://scripNng101.org/ blog/documentaNon)

Installing & Configuring

Auto star8ng WebSphere

Number of ways of star8ng WebSphere/Connec8ons

Manually -‐ takes Nme and requires much typing or copy and pasNng to the command line.

AutomaNcally -‐ via a service can ofen be the easiest way to start the servers (assuming the monitoring policy is set correctly).

Using a script – run one command and start all the connecNons servers in one swoop.

Registering WAS services

Registering a Linux service Registering WebSphere as a Linux service is straight forward. Use the wasservice script -‐ which is held in the <WAS_HOME>/bin directory (default is /opt/IBM/WebSphere/AppServer/bin)

Running the wasservice commands with the appropriate arguments (see example above for syntax), will create a service for the Deployment manager and Nodeagent servers. Adding the –stopArgs will allow you to add the wasadmin user and password here so it is not held in a file or is required when the service runs. Adding the –stopservers argument to the nodeagent server will stop the WebSphere servers on the node when the nodeagent is stopped. If you now list the contents of /etc/init.d the services can be seen.

Registering WAS services

Registering a windows service

Once the wasservice command has been run successfully, the new services will be visible under

Registering a WebSphere windows services are very similar to adding a Linux service. From the <WAS Profile>\bin directory wasservice –add <service name> -‐serverName <name of server> -‐profilePath <path to server profile>

To enable the server to restart automaNcally and to encode the user name and password, addiNonal parameters are required.

-‐encodeParams –restart true -‐startType automa;c -‐stopArgs “-‐username wasadmin –password password ”

And to the nodeagent add the addiNonal -‐encodeParams –restart true -‐startType automa;c -‐stopArgs '-‐username wasadmin –password password -‐stopservers'

Se^ng the monitoring policy

Using the Web Console to set the monitoring policy Open each ApplicaNon Server Server Infrastructure – Java and Process Management – Monitoring Policy

Change Node restart state to “RUNNING”. Ensure that the configuraNon is saved before quiBng the ISC. When the server is restarted the monitoring policy will be set. NOTE: Ensure that the monitoring policy is set to STOPPED when applying fix packs and ifixes.

Se^ng the monitoring policy

Using a script to set the monitoring policy

Run the config monitoring policy script (cfgMonitoringPolicy.py) ./wsadmin.sh -‐lang jython -‐f cfgMonitoringPolicy.py A prompt is presented asking which state you would like to set stopped, running or previous.

Once complete the script will synchronise the changes to the nodeagents. Again note that prior to applying fix packs or ifixes the monitoring policy should be set to STOPPED.

Start / Stop with a script

Windows Create a service for the Dmgr, Nodeagent and ConnecNons servers. Create a windows batch file to stop and start the services.

***** Example of start servers batch file ****

@echo off echo Star;ng DeploymentManager NET START "IBMWAS80Service -‐ Dmgr" echo Star;ng NodeAgent NET START "IBMWAS80Service -‐ Node" echo Star;ng server Cubey NET START "IBMWAS80Service – cubey" echo Servers started now exi;ng pause exit

Use the NET STOP command for stopping the servers and NET START for starNng the servers.

When run, the start / stop batch file starts or stops the servers in the order specified in the file.

Start / Stop with a script

Linux If you prefer to manually start the ConnecNons servers under Linux by far the easiest way is to use the connctrl script (see download the scripts).

Copy the connctrl script to the /etc directory and edit the variables, path to App server profiles, server names and if a non root user is required

The script is then called from /etc

/etc/connctrl.sh Usage: ConnCtrl {start|stop} {0|3|4} {nru} 0=just 1 IC server, 3=3 IC servers & 4=4IC servers nru = if you are using a user other than root

i.e.. /etc/connctrl.sh stop 4 wasadmin

or /etc/connctrl start 0

The script will start and stop the servers in a similar way to the windows batch files. Each server in turn is stopped (or started).

DEMO

Auto start HTTP Server

Windows create a Service for IHS and Admin Service From <HTTP ROOT>\bin h,pd -‐k install -‐n <name of service> -‐f <path to config file> i.e.. h,pd -‐k install -‐n IBMHTTPAdmin -‐f C:\IBM\HTTPServer\conf\admin.conf Linux | AIX: Create Symbolic Link:

ln -‐s /opt/IBM/HTTPServer/bin/apachectl /etc/init.d/ ln -‐s /opt/IBM/HTTPServer/bin/adminctl /etc/init.d/

Add Service chkconfig –add apachectl chkconfig –add adminctl

Performance Tuning

• Performance Tuning IBM Connec8ons •  Increase min-‐ and max ConnecNons of Data Sources

Performance Tuning Guide IBM ConnecNons 4.0 Performance Tuning Guide Addendum

•  Review these seBngs periodically with “Tivoli Performance Viewer”

Performance Tuning Guide IBM ConnecNons 4.0 – h,p://www-‐10.lotus.com/ldd/lcwiki.nsf/dx/IBM_ConnecNons_4.0_Performance_Tuning_Guide

Performance Tuning Guide Addendum -‐ h,p://www-‐10.lotus.com/ldd/lcwiki.nsf/dx/IBM_ConnecNons_4.5_Performance_Tuning_Guide_Addendum

Configure Data Sources

15 + Data Sources to Change (at least 100 clicks)

Resources > JDBC >Data sources > <data source name> > AddiNonal Features > ConnecNonPools

Change the maximum and minimum connecNons for each data source, save each of the changes, sync the nodes and restart the servers. This process can take about 30 minutes depending on the speed of the machines you are working on.

Configure Data Sources

• Why not configure using a script: cfgDataSource.py About 30 seconds to change all needed parameters of all Data Sources

Set the data source properNes in the ibmcnx.properNes in the script directory – allowing edits and re-‐running of the script.

HTTP mod_deflate

•  See my blog, the white paper or scrip8ng101.org •  Set mod_deflate in your IBM HTTP server config

Edit the h,pd config and enable the deflate module.

Add the deflate informaNon.

Set the deflate compression level to the desired level.

Add the informaNon to the virtual hosts.

Restart the HTTP server.

Loading my profile with deflate on saves approx. 75% of the page load Nme.

DEMO

Set J2EE Security Roles

•  First found on Klaus Bild’s Blog: hGp://kbild.ch •  Extended version with Group support and input opNon. • No need to edit the script.

•  Two versions: •  Restricted

Each applicaNon ask for credenNals, no data visible for anonymous users.

• Unrestricted Default SeBngs, some content is visible to anonymous users.

NOTE -‐ ApplicaNons restart automaNcally, when you change J2EE roles.

Set J2EE Roles for special applica8ons As some applica8ons are configured different from environment to environment •  Save 8me and use following scripts:

• cfgJ2EERoleGlobalModerator.py • cfgJ2EERoleMetricsReader.py • cfgJ2EERoleMetricsReportRun.py • cfgJ2EERoleSocialMail.py

• Deac8vate or ac8vate the applica8on for users and groups

DEMO

J2EE Security Roles -‐ BACKUP

CR and Fix pack Installa8ons pre IBM Connec8ons 4.0 oien reset J2EE roles back to the default se^ngs. Mainly this isn’t an issue anymore, BUT before running an update or making significant changes to the security seBngs, it is wise to back up the applicaNon security roles. Script writes text files as backup to a local folder. Roles of all installed WebSphere applicaNons (including IBM Docs, Forms Experience Builder, Kudos etc.), not just the connecNons applicaNons are backed up. cfgJ2EERoleBackup.py

J2EE Security Roles -‐ RESTORE

Once you have a backup of the applicaNon security seBngs, it is possible to restore it if necessary. Advantages to using the script to restore. •  You can edit the backup files to add or change users. •  Backup of Dev or QA Systems can be restored in producNon (Admin Users and Groups must exist there).

To run a restore use the cfgJ2EERoleRestore.py script. The script will prompt for the directory where the security text files are stored. Once complete the roles are restored and visible in the ISC.

DEMO

Configure JVM Log Files

• Default Se^ng for JVM Log Files (SystemOut.log & SystemErr.log):

Size: 1 MB No historical Log Files Too small to troubleshoot errors

• Reconfigure: Size: 20 – 40 MB 5-‐10 historical Log Files

• Configure via the ISC (Wasadmin console) Many clicks, Nme consuming Especially for large environments TroubleshooNng > Logs and trace > <server name> > JVM Logs

Configure JVM Log Files

• Use a script to set the log size and history (cfgLogFiles.py)

•  The script prompts for

• RolloverType – Size (to just set the size) or Both for size and history. • Maximum log size in mb. • Maximum number of backup files.

The seBngs are saved to the WebSphere configuraNon and are picked up on the next server restart.

DEMO

Set JVM Heap Sizes

•  JVM (Java Virtual Machine) heap sizes should be set according to your environment.

• By default the maximum JVM heap is set to 2506 MB for each Connec8ons server.

The JVM heap size can be changed in the ISC. Each server must be changed individually. Server Types > WebSphere ApplicaNon Server > <server name> > Server Infrastructure > Java Process Management > Process DefiniNon > Java Virtual Machine

Many clicks can be Nme consuming depending on the size of the environment. Set the iniNal heap size, maximum heap size, save and sync once the servers are restarted and the new seBngs are picked up.

Set JVM Heap Sizes

•  To use the script to set the JVM heap, use the cfgJVMHeap.py script.

The script shows actual size of iniNalHeapSize and maximumHeapSize for all JVM. Prompts for iniNalHeapSize and maximumHeapSize for all JVM. “Return” leaves actual seBng intact with no changes.

Once the script is complete, the new seBngs are visible in the ISC and are picked up on the next server restart.

DEMO

Business as usual

Check External IDs against LDAP

•  Some8mes users can't logon to single Connec8ons Applica8ons

• ExtIDs are out of sync, you see several error messages. •  Script to check if LDAP UUIDs (GUID, SID) are equal to Connec8ons UUID

• Script reads PROF_GUID from peopleDB through JDBC and Check UUID in ApplicaNons.

Mail address of profile you want to check: sharon@cube-‐sof.co.uk Profiles: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 AcNviNes: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 Blogs: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 CommuniNes: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 Dogear: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 Files: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 Forums: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 News, Search, Homepage: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3 Wikis: 9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3

No need to run individual scripts per app to check the ID.

DEMO

Deac8vate and Ac8vate Users

• Aier rename or some8mes without real reason Users get out of sync. They can open Profiles, but no other ApplicaNon. Or they have problems with a single App.

• When you want to reac8vate a user you must provide UID and EMAIL (was deleted through Deac8vate)

execfile("profilesAdmin.py") ProfilesService.acNvateUserByUserId("9EE90251-‐33DE-‐4699-‐A6F1-‐BFDB381F7DD3", email="sharon@cube-‐sof.co.uk", loginId="sharon")

DeacNvaNng and ReacNvaNng through the ProfilesService can be a resoluNon for this issue. This forces the applicaNons to resynchronise with the profiles applicaNon.

Use the Deact&Act script

• Use the script -‐ cnxMemberDeactAndActByEmail.py •  Stores UID and Mail in memory and deacNvates and acNvates in one

command. •  It basically turns it off and on again – forcing a resync with the

ConnecNons applicaNons and resolving the users *out of sync* issues.

• Result = Happy Users J

Sync External IDs of all Users in all Applica8ons

•  SomeNmes LDAP IDs and UUID come out of sync • Switch LDAP System requires a synchronizaNon.

• Running the MemberService scripts can resolve this.

MemberService.syncAllMembersByExtId for each applicaNon. Can be Nme consuming, much copying and pasNng of commands and each one has to complete before the next one can be run.

Sync External IDs of all Users in all Applica8ons

• Use the script -‐ MemberSyncAllByEXID.py • The script resyncs ALL applicaNons for ALL users.

Running the script syncs all users against all applicaNons, rather than having to run each applicaNon sync all members script separately. ResulNng in happy users.

DEMO

Working with files policies

• Crea8ng or edi8ng policies is some8mes a pain •  Library size must be provided as Long (e.g. 2 GB = 2147483648L) •  Edit example:

• FilesPolicyService.edit("2d93497d-‐065a-‐4022ae25-‐a4b52598d11a", "My Policy", 2147483648L).

• Find UUID of Policy to edit, copy the size from calculator. • Use a script -‐ CnxFilesPolicies.py

DEMO

Add Policies to Libraries

• You need UUID from Policy and Community •  Community LisNng provides too much informaNon, hard to find UUID. •  Example assign Policy

•  FilesLibraryService.assignPolicy("f0d01111-‐9b21-‐4dd8-‐b8be-‐8825631cb84b", "2d93497d-‐065a-‐4022ae25-‐a4b52598d11a")

•  If you need to find the library informa8on it’s a pain: FilesLibraryService.browseCommunity("Ntle", "true", 1, 20)

Add Policies to Libraries

• Or you can use a script (LibraryPolicies.py) Allows for searching across Personal or Community libraries.

Select a community the policies will be listed and you may assign a new policy.

DEMO

Reparen8ng Communi8es

• Big thanks to Klaus Bild Script allows the moving of a main community to a sub (assuming it has no subs). Or a sub to a main community. Wild card searches are also available.

Data base maintenance

•  Lots of DB maintenance that can be run: • Op8miza8on – Check the Connec8ons performance guide for DB tuning and op8miza8on.

• ReOrg – Should be run regularly – scripts are available in the Connec8ons wizards folder (connec8ons.sql folder), can be set as a scheduled job.

• Run Stats – Can be set through automa8c maintenance. • Backup the DBs on a regular basis – either as part of your regular back up rou8ne or an offline backup regularly.

•  Lots of informa8on on the scrip8ng101.org site and on the DB2 info centres.

Troubleshoo8ng

Are all applica8ons running

•  To check If all IBM Connec8ons Applica8ons are running • Go to ISC – ApplicaNons – ApplicaNon Types – WebSphere Enterprise ApplicaNons.

•  Or use a script (checkAppStatus.py)

One bu,on press and the script checks all apps and lists them as stopped or started.

Can WAS connect to Data Sources

• Go to ISC – Resources – JDBC – Data sources • Check all Data Sources and click Test connecNon.

• Use a script (/check/DataSource.py) The script checks connecNon to the DB and tests the data Sources. Then lists successful, failures and databases not installed.

DEMO

Documenta8on

Connec8ons Documenta8on

•  Everyone knows you should document your install right?

• Never fear – there is a script to do most of the hard work for you.

Documenta8on scripts

• Use the scripts 1.   Show JVM Heap Sizes (ibmcnx/doc/JVMHeap.py) 2.   Show JVM Se^ngs (ibmcnx/doc/JVMSe^ngs.py) 3.   Show SystemOut/Err Log Sizes (ibmcnx/doc/LogFiles.py) 4.   Show all used ports (ibmcnx/doc/Ports.py) 5.   Show all used variables (ibmcnx/doc/Variables.py) 6.   Show all j2ee roles of inst. applica8ons (ibmcnx/doc/j2eeroles.py) 7.   Create a file with all documenta8on (ibmcnx/doc/Documenta8on.py) –

this isn’t implemented in the menu as yet – call with wsadmin.sh|bat -‐lang jython -‐f ibmcnx/doc/Documenta>on.py

DEMO

Documenta8on Demo

Resources

Download the scripts

•  If you wish to use any of the scripts described in this presentaNon, they are available for download.

•  Please be aware that they are community created scripts and are available without warranty. They should be used at your own risk.

•  Git Hub -‐ h,ps://github.com/stoeps13/ibmcnx2

•  OpenNTF Project since – AdministraNon Scripts for WebSphere -‐ h,p://www.openn�.org/main.nsf/project.xsp?r=project/AdministraNon%20Scripts%20for%20WebSphere/summary

•  ConContrl – Linux ConnecNons start / stop script from Tim Clark and Sharon Bellamy h,ps://github.com/stoeps13/ibmcnxscripNng/tree/master/ConnecNons/startstop

Install the scripts

• hGp://scrip8ng101.org/resources/installing-‐the-‐scripts/ • Create links to extracted scripts directory

• Linux cd /opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin ln -‐s /opt/IBM/ibmcnx2/ibmcnx ./

• Windows mklink /d <link> <target> mklink /d D:\IBM\WebSphere\AppServer\profiles\Dmgr01\bin\ibmcnx D:\IBM\ibmcnx2\ibmcnx

•  Edit the ibmcnx.proper8es

At the moment there are 3 parts: Generic, Database and Tuning. Generic: You should change the values within Generic of all j2ee.* parameters and put your administraNon users and groups in it. This part is used to set the J2EE Security Roles and you save typing in the values each Nme. Database: The User scripts (checking ExId, DeacNvate) are using the db* parameters and you can put the User, password, host and so on. Tuning: These parameters are used for DataSource performance tuning and are set to the values provided in the IBM ConnecNons Performance Tuning Guide. That’s a good starNng point, but you should check them in your environment.

Install the scripts (cont.)

•  Addi8onal tasks to load JDBC Drivers within Jython Linux Create a WAS_USER_SCRIPT (/opt/IBM/JDBC is my path to the database drivers): wasuserscript.sh export WAS_EXT_DIRS=$WAS_EXT_DIRS:/opt/IBM/JDBC Now you have to set an environment variable each Nme when you want to start the scripts: export WAS_USER_SCRIPT=/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin/ibmcnx/wasuserscript.sh

Windows There is an issue loading the JDBC Driver path through WAS_USER_SCRIPT on Windows. To resolve this edit setupCmdLine.bat in dmgr/bin and add the path to WAS_EXT_DIRS i.e. SET WAS_EXT_DIRS=%JAVA_HOME%\lib;%WAS_HOME%\classes;%WAS_HOME%\lib ……C:\IBM\SQLLIB\java

•  Load the script menu Linux

./wsadmin.sh -‐lang jython -‐username wasadmin –password password -‐f ibmcnx/menu/cnxmenu.py

Windows wsadmin.bat -‐lang jython -‐username wasadmin –password password -‐f ibmcnx\menu\cnxmenu.py

Useful links / Ar8cles

•  IBM ConnecNons Performance Tuning – relevant for Version 4 and 4.5 (see Addendum for 4.5 addiNons and changes) -‐ h,p://www-‐10.lotus.com/ldd/lcwiki.nsf/dx/IBM_ConnecNons_4.0_Performance_Tuning_Guide

•  IBM ConnecNons Performance Tuning Addendum – relevant for 4.5 changes -‐ h,p://www-‐10.lotus.com/ldd/lcwiki.nsf/dx/IBM_ConnecNons_4.5_Performance_Tuning_Guide_Addendum

•  Auto start DB2 on RHEL6 systems -‐ h,ps://www-‐304.ibm.com/support/docview.wss?uid=swg21497220

•  Tuning WebSphere ApplicaNon Server V7 and V8 for performance -‐ h,p://www.ibm.com/developerworks/websphere/techjournal/0909_blythe/0909_blythe.html

•  DB2 Info centre -‐ h,p://pic.dhe.ibm.com/infocenter/db2luw/v10r1/index.jsp •  IBM ConnecNons wiki -‐ h,p://www-‐10.lotus.com/ldd/lcwiki.nsf •  IBM WebSphere 8 info centre -‐ h,p://pic.dhe.ibm.com/infocenter/wasinfo/v8r0/index.jsp

Skype Chats

•  There are a number of community skype chats

If you wish to be added to any of these chats either ping myself or Christoph.

Sharon – dilfechnical Christoph -‐ christophstoe,ner

Giving Credit

• IBM® Notes® • IBM® Domino® • IBM® ConnecNons • IBM® WebSphere® • IBM® DB2 • IBM® AIX® • Tivoli®

• Linux® • Java® • Microsof® Windows® • Red Hat® Linux® • Twi,er® • Skype®

• This presentaNon menNons the following Copyrights and Trademarks.

Technology

Practical solutions for connections administrators