Tivoli Access Manager problem determination using logging and tracing features

8/10/2019 Tivoli Access Manager problem determination using logging and tracing features

1/41

IBM Software Group

IBM Corporation

Tivoli Access Manager problem determination

using logging and tracing features

Jenny Totterdel l - EMEA Securi ty Suppo rt jen ny_to tter dell@u k.ibm .com
mailto:[email protected]:[email protected]


2/41

IBM Software Group | Tivoli software

Topics covered in this workshop

Installation and Configuration Logs

Serviceability logs

Trace Logging

WebSEAL HTTP Trace Logging

Debugging Java Runtime Issues

GSKit Traces

Must Gather Information for Support

Capturing Core Files

System_status script

Question/Answer Session


3/41


Log Files


4/41


Installation Logs If the easy installation programs are used, the log files are written to the temp directory

Windows - %TEMP% (e.g. C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp)

UNIX - typically /tmp or /var/tmp

Component Installation log file name

Policy server msg__ammgr_install.log

Policy proxy server msg__amproxy_install.log

Authorization server msg__amacld_install.log

Runtime msg__amrte_install.log

Java runtime msg__amjrte_install.log

ADK msg__amadk_install.log

Web Portal Manager msg__amwpm_install.log

WebSEAL msg__amweb_install.log

WebSEAL Application Development Kit msg__amwebadk_install.log

Plug-in for Web Servers msg__amwpiismp_install.log

WebSphere Application Svr & BEA WebLogic Svr integration support msg__amismp.log

Attribute retrieval service msg__amars_install.log

Tivoli Identity Manager Provisioning Fast Start msg__ampfs_install.log

IBM Tivoli Directory Server msg__ldaps_install.log


5/41


Configuration Logs

Messages generated during the configuration process are stored

within Tivoli Access Manager configuration log files.

Component Installation log file name

Base msg__config.logWeb Portal manager msg__amwpmcfg.log & amwpmcfg1.log

Java runtime environment msg__PDJrteCfg1.log

WebSEAL msg__amweb_config.log

Plug-in for Web Servers msg__pdwpicfg.log


6/41


Serviceability Logs

Examples of serviceability logs: msg__pdmgrd_utf8.log

msg__webseald-default.log

Message logging (i.e. Error/Warning/Informational logging) is enabled by default

Default log locations

UNIX: /var/PolicyDirector/log/

Windows:pd_dir \log\


7/41


Message Format

A message consists of:

Date

Message Number (unique 32-bit decimal or hexadecimal value)

Process Name

Priority (e.g. WARNING)

Component information (including file name)

A message identifier (ID) and message text.

Example of a failed login captured in WebSEAL server message log

(/var/pdweb/log/msg__webseald-default.log):

2005-07-20-05:54:36.655+00:00I----- 0x132120DDwebsealdWARNING ias authsvc pdauthn.cpp 1435 0x00002526

HPDIA0221W Authentication for user testuser failed. Youhave used an invalid user name, password or clientcertificate


8/41


Types of Messages

Notice (Notice_verbose)

Does not directly require action, such as information about running state

Warning

Results may not be as desired but the program continues to function

normally.

Error

The product continues to function, but some services or functionality might

not be available

Fatal

Unrecoverable error, the process encountering the error usually terminates


9/41


Message Examples:

Notices:

2005-08-09-09:07:31.814+00:00I----- 0x1354A0A0 pdmgrd NOTICE ivc generalivmgrd.cpp 743 0x00000001 Server startup

Server startup message

Warning:

2003-10-31-23:09:45.457+00:00I----- 0x38CF0131 webseald WARNING wwa server listen-ssl.c 167 0x00000044 The 'ssl_writechunk' routine failed for 'gsk_secure_soc_write', errno

= 406 This error is common and normal for webseal and ssl, which is why they're reported as warnings.

Mainly they are due to network connectivity or the customer hitting the "stop" button on their browsers.The reason you see several messages with the same timestamp is because the browsers tend to openmultiple simultaneous connections. Losing the network or hitting the "stop" button will cancel allsimultaneous connections.

406 is a GSKit return code GSK_ERROR_IO

Error:

2003-07-08-12:59:07.032+00:00I----- 0x1354A0B6 pdmgrd ERROR ivc generalLDAPClient.cpp 212 0x00000001 LDAP initialization failed: ira_rgy_init('tarsus', 636,'cn=ivmgrd/master,cn=SecurityDaemons,secAuthority=Default', ***) = 113, 202

Connection to LDAP failed.

Fatal:

2004-12-09-14:42:32.391+01:00I----- 0x14C010A4 pdmgrd FATAL mgr generale:\am510\src\ivmgrd\ivmgrd.cpp 252 0x00000ba4 HPDMG0164E The Policy Server couldnot be started (0x14c01420).


10/41


Message ID FormatThe message ID consists of 10 alphanumeric characters, where the sequence is

XXXYY####Z:

XXX is the product identifier, including the following product codes:

Code Subsystem

HPD Base

DPW/HPW WebSEAL

AWD Plug-in for IBM WebSphere Edge Server

AWL BEA WebLogic Server integration

AWX WebSphere Application Server Integration

AMZ Plug-in for Web Servers

YY is the subsystem code

#### is a unique message id.Z is the severity code indicator, including the following indicators:

Severity Code Description

I Informational message.

W Warning message.

E Error message.


11/41


Routing Message Logging

Routing of serviceability messages is controlled by the routing file.

The contents of the routing file enables control of

Whether message logging is on or off for each class of messages (FATAL, ERROR,

WARNING, NOTICE, or NOTICE_VERBOSE)

Where the message log output for each class of messages is to be directed

If message output is being directed to a file, how many files for each class of messages

should be used, and how many messages should be placed in each file

The routing files for each component are

pdmgrd_routing for the Policy Server

pdacld_routing for the Authorization Server

pdmgrproxyd_routing for the Policy Proxy Server

/opt/pdweb/etc/routing for WebSEAL

/opt/PolicyDirector/etc/routing for Runtime

PDJlog.properties for Java apps


12/41


Routing File Message Logging Entry

The format of a routing file entry that controls message logging is:

severity:destination:location {[;destination:location] ...}

[;GOESTO:other_severity]

Default configuration for FATAL and ERROR messages:

Unix FATAL:STDOUT:-

;UTF8FILE:/var/PolicyDirector/log/msg__pdmgrd_utf8.log:644:ivmgr:ivmgr

ERROR:STDOUT:-

;UTF8FILE:/var/PolicyDirector/log/msg__pdmgrd_utf8.log:644:ivmgr:ivmgr

Windows:

FATAL:STDERR:-;FILE:C:/PROGRA~1/Tivoli/POLICY~1/log/msg__fatal.log

ERROR:STDERR:-;FILE:C:/PROGRA~1/Tivoli/POLICY~1/log/msg__error.log


13/41


WebSEAL Logs

WebSEAL maintains three conventional HTTP log files that record

activity rather than messages:

request.log

logs HTTP requests, such as information on URLs that have been

requested and information on the client (e.g. IP address).

agent.log records contents of the User_Agent:header in the HTTP request. Includes

data about the client browser, such as architecture or version number

referer.log

records the Referer:header of the HTTP request. Records the document

that contained the link to the requested document.

By default, these log files are located under the following directory:

UNIX: /var/pdweb/www/log/

Windows: C:\Program Files\Tivoli\PDWeb\www\log\


14/41


Request.log

Every response sent back by TAM is recorded with a one-line entry inthe request.log

Format: host - authuser [date] request status bytes

host Specifies the IP address of the requesting machine.

authuserIdentity information of the user. The value unauth is used for an

unauthenticated user.dateSpecifies the date and time of the request.

request Specifies the first line of the request as it came from the client.

statusSpecifies the HTTP status code sent back to the requesting machine.

bytesSpecifies the number of bytes sent back to the requesting machine.

130.15.1.90- lmalone [30/Aug/2005: 10:24:11 +0100]"GET /jct/images/IBMLogo.gif HTTP/1.1" 2001979

130.15.1.90- lmalone[30/Aug/2005: 10:24:13 +0100]"GET /jct/images/IBMLogo.gif HTTP/1.1" 3040


15/41


Traces


16/41


Trace Logging

Unlike message logging, trace logging (or tracing) is not enabled bydefault.

Enabled using routing/properties files or pdadmin

Useful for

Recreateable problems

Issues short lived in duration

Blade Startup Failures (including during configuration)

Checking LDAP Return Codes


17/41


Enabling TracingRouting File

Can trace all components, or limit the scope

General format for routing file tracing statement:

component :subcomponent.debuglevel:destination :attributes

Examples:

Entries in /opt/PolicyDirector/etc/pdmgrd_routing (TAM 5.1)Trace all components for the Policy Server at highest trace level

*:*.9:TEXTFILE.10.10000:/var/PolicyDirector/log/trace__%ld.log

Trace the Policy Server's LDAP client calls/LDAP Server return codes

ivc:ira.9:TEXTFILE.10.10000:/var/PolicyDirector/log/trace__

pdmgrd_ira.log


18/41


Enabling TracingTrace Command

Can be activated dynamically using the command:

pdadmin> server task server_name trace set component level

List possible dynamic trace points:

pdadmin sec_master> server task server_name trace list

Particularly useful pdadmin traces:

pdweb.debug, pdweb.snoop, pd.ivc.ira

Examples:

pdadmin> server task webseald-instance trace set pdweb.debug 2

file path=/tmp/pdweb.debug.out

pdadmin> server task webseald-instance trace show

pdweb.debug 2


19/41


Trace Logging Example

Tracing an authentication failure, WebSEAL using auth-using-

compare=yes

Snippet from msg__webseald-default.log:

2005-07-20-07:55:29.772+00:00I----- 0x132120DD webseald

WARNING ias authsvc pdauthn.cpp 1435 0x00002728

HPDIA0221W Authentication for user testuser failed. You

have used an invalid user name, password or client

certificate.

Enable pd.ivc.ira tracing using:

pdadmin> server task default-webseald-amaix51 trace setpd.ivc.ira 9 file path=/tmp/pdweb.ira.out


20/41


Trace Logging Example (continued)

Portion of pd.ivc.ira trace output: 2005-07-20-07:55:29.757+00:00I----- thread(4) trace.pd.ivc.ira:8

/project/am510/build/am510/src/ivrgy/ira_auth.c:1417: CII ENTRY:

ira_auth_passwd_compare() dn: cn=testuser,o=ibm,c=us

2005-07-20-07:55:29.757+00:00I----- thread(4) trace.pd.ivc.ira:7

/project/am510/build/am510/src/ivrgy/ira_entry.c:3053:

ira_ldap_compare_s() DN: cn=testuser,o=ibm,c=us Attr: userPassword


/project/am510/build/am510/src/ivrgy/ira_ldap.c:757:

ira_ldap_compare_s(): No timeout - calling ldap_compare_s


/project/am510/build/am510/src/ivrgy/ira_ldap.c:767:

ira_ldap_compare_s: Returning LDAP rc x5


/project/am510/build/am510/src/ivrgy/ira_entry.c:3060: LDAP rc: x5


/project/am510/build/am510/src/ivrgy/ira_auth.c:1427: CII EXIT

ira_auth_passwd_compare()with rc: 0x00000031 LDAP_ERROR x5 "Acompare operation returned false.".


21/41



pdweb.debug

Advantages:

Smallest trace files available within webseal

HTTP headers in plain text, with time stamp showing arrival/sent

Disadvantages

Only traces HTTP headers

Does not trace responses from WebSEAL or show WebSEAL user or client IP

address

pdweb.snoop

Advantages:

Includes message bodies, responses from WebSEAL and client IP addresses

Decrypts HTTPS traffic Disadvantages

Large trace files (4-5 chars per byte)

Messages are hex encoded (get ascii value for none control char)

Does not show WebSEAL user (unless iv_user header is sent to jnc)

packets do not correspond to network frames in network trace


22/41



Starting traces

pdadmin>server task webseald-instance trace set

pdweb.debug 2 file path=/var/pdweb/log/debug.log


pdweb.snoop 9 file path=/var/pdweb/log/snoop.out

Stopping traces


pdweb.debug 0


pdweb.snoop 0

Path and Filename Issues for the traces

Follow local Operating System Rules

DYNURL mapping shown in traces


23/41


Pdweb.debug and Pdweb.snoop Overview

Typical request breaks down to 4 parts

Browser ===>PD

PD ===>BackEnd

PD


24/41


Pdweb.debug Example2 0 0 5 - 0 8 - 0 9 - 1 4 : 0 4 : 5 7 . 8 7 8 - 0 5 : 0 0 I - - - - - t h r e a d ( 4 ) t r a c e . p d w e b . d e b u g : 2

/ p r o j e c t / a m w e b 5 1 0 / b u i l d / a m w e b 5 1 0 / s r c / p d w e b / w a n d / w a n d / l o g . c : 3 0 9 : - - - - - - - - - - - - - - - - - B r o w s e r = = = > P D - - - - - - - - - - - - - - - - -T h r e a d _ I D : 1 3 3 2 6G E T / t e s t / H T T P / 1 . 1 H o s t : l i n u x U s e r - A g e n t : M o z i l l a / 5 . 0 ( X 1 1 ; U ; L i n u x i 6 8 6 ; e n - U S ; r v : 1 . 0 . 1 ) G e c k o / 2 0 0 2 0 9 0 3 A c c e p t :t e x t / x m l , a p p l i c a t i o n / x m l , a p p l i c a t i o n / x h t m l + x m l , t e x t / h t m l ; q = 0 . 9 , t e x t / p l a i n ; q = 0 . 8 , v i d e o / x -m n g , i m a g e / p n g , i m a g e / j p e g , i m a g e / g i f ; q = 0 . 2 , t e x t / c s s , * / * ; q = 0 . 1 A c c e p t - L a n g u a g e : e n - u s , e n ; q = 0 . 5 0 A c c e p t - E n c o d i n g : g z i p ,d e f l a t e , c o m p r e s s ; q = 0 . 9 A c c e p t - C h a r s e t : I S O - 8 8 5 9 - 1 , u t f - 8 ; q = 0 . 6 6 , * ; q = 0 . 6 6 K e e p - A l i v e : 3 0 0 C o n n e c t i o n : k e e p - a l i v eA u t h o r i z a t i o n : * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

2 0 0 5 - 0 8 - 0 9 - 1 4 : 0 4 : 5 7 . 8 9 6 - 0 5 : 0 0 I - - - - - t h r e a d ( 4 ) t r a c e . p d w e b . d e b u g : 2/ p r o j e c t / a m w e b 5 1 0 / b u i l d / a m w e b 5 1 0 / s r c / p d w e b / w a n d / w a n d / l o g . c : 3 0 9 : - - - - - - - - - - - - - - - - - P D = = = > B a c k E n d - - - - - - - - - - - - - - - - -

T h r e a d _ I D : 1 3 3 2 6G E T / H T T P / 1 . 1 v i a : H T T P / 1 . 1 l i n u x : 4 4 3 u s e r - a g e n t : M o z i l l a / 5 . 0 ( X 1 1 ; U ; L i n u x i 6 8 6 ; e n - U S ; r v : 1 . 0 . 1 ) G e c k o / 2 0 0 2 0 9 0 3i v _ s e r v e r _ n a m e : d e f a u l t - w e b s e a l d - l i n u x a c c e p t - c h a r s e t : I S O - 8 8 5 9 - 1 , u t f - 8 ; q = 0 . 6 6 , * ; q = 0 . 6 6 h o s t : l i n u x . n e t : 8 0 8 0 a c c e p t :t e x t / x m l , a p p l i c a t i o n / x m l , a p p l i c a t i o n / x h t m l + x m l , t e x t / h t m l ; q = 0 . 9 , t e x t / p l a i n ; q = 0 . 8 , v i d e o / x -m n g , i m a g e / p n g , i m a g e / j p e g , i m a g e / g i f ; q = 0 . 2 , t e x t / c s s , * / * ; q = 0 . 1 k e e p - a l i v e : 3 0 0 c o n n e c t i o n : c l o s e a c c e p t - l a n g u a g e : e n - u s ,e n ; q = 0 . 5 0 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

2 0 0 5 - 0 8 - 0 9 - 1 4 : 0 4 : 5 7 . 9 2 8 - 0 5 : 0 0 I - - - - - t h r e a d ( 4 ) t r a c e . p d w e b . d e b u g : 2/ p r o j e c t / a m w e b 5 1 0 / b u i l d / a m w e b 5 1 0 / s r c / p d w e b / w a n d / w a n d / l o g . c : 3 0 9 : - - - - - - - - - - - - - - - - - P D < = = = B a c k E n d - - - - - - - - - - - - - - - - -T h r e a d _ I D : 1 3 3 2 6 H T T P / 1 . 1 2 0 0 O K c o n t e n t - t y p e : t e x t / h t m l l a s t - m o d i f i e d : W e d , 0 6 N o v 2 0 0 2 1 3 : 0 6 : 4 7 G M T d a t e : T u e , 0 9 A u g2 0 0 5 1 9 : 0 4 : 5 7 G M T e t a g : " 2 1 3 7 c - 1 2 5 4 - 3 d c 9 1 3 e 7 " c o n t e n t - l e n g t h : 4 6 9 2 a c c e p t - r a n g e s : b y t e s c o n n e c t i o n : c l o s e s e r v e r :

I B M _ H T T P _ S E R V E R / 1 . 3 . 2 6 . 2 A p a c h e / 1 . 3 . 2 6 ( U n i x ) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

2 0 0 5 - 0 8 - 0 9 - 1 4 : 0 4 : 5 7 . 9 2 9 - 0 5 : 0 0 I - - - - - t h r e a d ( 4 ) t r a c e . p d w e b . d e b u g : 2/ p r o j e c t / a m w e b 5 1 0 / b u i l d / a m w e b 5 1 0 / s r c / p d w e b / w a n d / w a n d / l o g . c : 3 0 9 : - - - - - - - - - - - - - - - - - B r o w s e r < = = = P D - - - - - - - - - - - - - - - - -T h r e a d _ I D : 1 3 3 2 6H T T P / 1 . 1 2 0 0 O K p 3 p : C P = " N O N C U R O T P i O U R N O R U N I " c o n t e n t - t y p e : t e x t / h t m l l a s t - m o d i f i e d : W e d , 0 6 N o v 2 0 0 2 1 3 : 0 6 : 4 7 G M Tt r a n s f e r - e n c o d i n g : c h u n k e d d a t e : T u e , 0 9 A u g 2 0 0 5 1 9 : 0 4 : 5 7 G M T e t a g : " 2 1 3 7 c - 1 2 5 4 - 3 d c 9 1 3 e 7 " a c c e p t - r a n g e s : b y t e s x - o l d -c o n t e n t - l e n g t h : 4 6 9 2 s e r v e r : I B M _ H T T P _ S E R V E R / 1 . 3 . 2 6 . 2 A p a c h e / 1 . 3 . 2 6 ( U n i x )- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


25/41


Pdweb.snoop Example One

Webseal (9.168.13.15) opens up a socket to the Application Server (9.1.131.27)----------------------------------------

2005-08-08-09:47:36.050+02:00I----- thread(263) trace.pdweb.snoop.jct:1/project/amweb510/build/amweb510/src/pdwebrte/webcore/amw_snoop.cpp:100:

----------------------------------------

Thread 251; fd 58; local 9.168.13.15:62113; remote 9.1.131.27:4482

Socket opened.

A GET is performed on IBMabcLogo.gif

----------------------------------------

2005-08-08-09:47:36.053+02:00I----- thread(263) trace.pdweb.snoop.jct:1/project/amweb510/build/amweb510/src/pdwebrte/webcore/amw_snoop.cpp:159:

----------------------------------------


Sending 2652 bytes

0x0000 4745 5420 2e2f 7465 7374 2f69 6d61 6765 GET./test/images

0x0010 732f 4942 4d61 6263 4c6f 676f 2e67 6966 /IBMabcLogo.gif.

0x0020 4854 5450 2f31 2e30 2020 7669 613a 2048 HTTP/1.0..via:.H


26/41


Pdweb.snoop Example Two2005-08-07-13:09:31.588-05:00I----- thread(3) trace.pdweb.snoop.jct:1

/project/amweb510/build/amweb510/src/pdwebrte/webcore/amw_snoop.cpp:159:

----------------------------------------


Sending 60 bytes

0x0000 4845 4144 202f 2048 5454 502f 312e 310d HEAD./.HTTP/1.1.

0x0010 0a68 6f73 743a 206c 696e 7578 2e6e 6574 .host:.linux.net

0x0020 3a38 3038 300d 0a63 6f6e 6e65 6374 696f :8080..connectio

0x0030 6e3a 2063 6c6f 7365 0d0a 0d0a n:.close....

2005-08-07-13:09:31.589-05:00I----- thread(3) trace.pdweb.snoop.jct:1

/project/amweb510/build/amweb510/src/pdwebrte/webcore/amw_snoop.cpp:133:----------------------------------------


Receiving 275 bytes

0x0000 4854 5450 2f31 2e31 2032 3030 204f 4b0d HTTP/1.1.200.OK.

0x0010 0a44 6174 653a 2053 756e 2c20 3037 2041 .Date:.Sun,.07.A

0x0020 7567 2032 3030 3520 3138 3a30 393a 3331 ug.2005.18:09:31

0x0030 2047 4d54 0d0a 5365 7276 6572 3a20 4942 .GMT..Server:.IB

0x0040 4d5f 4854 5450 5f53 4552 5645 522f 312e M_HTTP_SERVER/1.

0x0050 332e 3236 2e32 2020 4170 6163 6865 2f31 3.26.2..Apache/1

0x0060 2e33 2e32 3620 2855 6e69 7829 0d0a 4c61 .3.26.(Unix)..La

0x0070 7374 2d4d 6f64 6966 6965 643a 2057 6564 st-Modified:.Wed

0x0080 2c20 3036 204e 6f76 2032 3030 3220 3133 ,.06.Nov.2002.13

0x0090 3a30 363a 3437 2047 4d54 0d0a 4554 6167 :06:47.GMT..ETag

0x00a0 3a20 2232 3133 3763 2d31 3235 342d 3364 :."2137c-1254-3d

0x00b0 6339 3133 6537 220d 0a41 6363 6570 742d c913e7"..Accept-

0x00c0 5261 6e67 6573 3a20 6279 7465 730d 0a43 Ranges:.bytes..C

0x00d0 6f6e 7465 6e74 2d4c 656e 6774 683a 2034 ontent-Length:.4

0x00e0 3639 320d 0a43 6f6e 6e65 6374 696f 6e3a 692..Connection:

0x00f0 2063 6c6f 7365 0d0a 436f 6e74 656e 742d .close..Content-

0x0100 5479 7065 3a20 7465 7874 2f68 746d 6c0d Type:.text/html.


27/41


Java Issues

IBM S ft G | Ti li ft


28/41


PDJrte Configuration

Verify the pdjrte has been configured properly

Created in

/PolicyDirector

/PolicyDirector/PD.propertiescontains key-value pairs used by the TAM java runtime

/PolicyDirector/PDJLog.propertiescontains key-value pairs used by Java Logging

/PolicyDirector/PDCA.ksCA certificate keystore. Used in subsequent calls to pdmgrd(ie. SvrSslCfg)

Added in /lib/ext/

PD.jaradmin and authorization java classes

ibmjcefw.jarjava cryptography extension

ibmjsse.jarjava secure sockets implementation ibmjcaprovider.jar, US_export_policy.jar, local_policy.jarcryptography

ibmpkcs.jar, ibmpkcs11.jarpublic key cryptography standard support

jaas.jarjava authentication and authorization service

US_export_policy.jar

local_policy.jar



29/41


Debugging Common Java Runtime Issues

Expired certificates in keystore files

Check WebSphere logs for errors

Enable WAS security trace

Viewing certificates in keystore using keytool

keytoollistvkeystore -storetype JCEKS

How to refresh the certificate

java com.tivoli.pd.jcfg.SvrSslCfgaction replcertadmin_id

-admin_pwd -cfg_file



30/41


Debugging Common Java Runtime Issues (contd)

SvrSslCfg not found

Multiple JREs on system?

Outdated TAM JRTE

WAS 5.0.2 must be configured with TAM 5.1 Java Runtime

Incorrect administrator name or password specified

Caused by incompatibility between PD.jar file shipped with WAS and one

shipped with TAM.

To resolve, copy PD.jar file from /java/export/pdjrte/ to

/java/jre/lib/ext directory

Embedded WAS Support Issues



31/41


Tracing and Messaging

PDJLog.properties is a wrapper to the java logging facility

Configuration of logging is done via:

/PolicyDirector/PDJLog.properties

Log files created: trace_amj.log

msg__amj_fatal.log

msg__amj_error.log

msg__amj_warning.log

msg__amj_notice.logmsg__amj_noticeverbose.log



32/41


Enabling Tracing and Logging

To enable logging:

Edit the PDJLog.properties:

For all components, specify:

baseGroup.PDJTraceLogger.isLogging = true

For individual components, specify:

baseGroup.PDJadminTraceLogger.isLogging = true

baseGroup.PDJauthzTraceLogger.isLogging = true



33/41


GSKit Traces



34/41


GSkit Trace

To enable the trace, perform the following steps:

Specify the file in which the trace data is to be stored with the environment

variable GSK_TRACE_FILE. Reference the following example:

export GSK_TRACE_FILE=/tmp/mytracefile

Re-create the error.

The system will append a ".1" to the file name and then accumulate

about 25 megabytes of trace data. It will then close the

"/tmp/mytracefile.1" file, open a "/tmp/mytracefile.2" file which

accumulate 25 more megabytes of trace information. It will then close

that one, erase the first file, and start over.

The trace files are binary



35/41


System Data



36/41


Must Gather Information for Support

The following should be known before calling support

Platform for each component (O/S level including patches)

TAM Version and Fixpack Level from all machine not just the failing machine

Appropriate Log and Configuration files

If core, provide senddata output

If windows failure, provide dr watson

User Registry and version (IDS, Sun, eDirectory)

Integration with other products



37/41


Capturing Core Files

Senddata ScriptWhat does it capture

core

daemon binary

libs.tar


38/41


Core Files cont.

Also on AIX it is possible to use the AIX Command snapcore which

does not require dbx.



39/41


System_status Script

Sample of Information Gathered

O/S and patch levels

Resource and Environment data (Memory, disk space, environment

variables, locales, ulimits)

Network information (/etc/hosts, ip address, network devices, aliases)

TAM Configuration (configuration files, daemon build levels)

TAM Log files

TAM data

ACLs, Users, POPs, Groups, Junctions, ObjectSpace, Servers, Password

Policy, GSO data,

LDAP Data Schema definitions, suffix data

DB2 Data

Instances, db connectivity to databases, table searches



40/41


System_status script (cont)

This script can be run with multiple options

system_status.ksh

Anonymous pdadmin and Anonymous LDAP Requests

system_status.ksh -D 'cn=root' -w 'cn=root_password'

Anonymous pdadmin but LDAP authentication with

cn=root/cn=root_password

system_status.ksh -a 'sec_master' -p 'master_password'

pdadmin authentication with sec_master/master_password, but

Anonymous LDAP bind

system_status.ksh -a 'sec_master' -p 'master_password' -D 'cn=root' -w

'cn=root_password'

Pdadmin and LDAP authentication

Resulting file will be in the format of hostname-mm-dd-yy_hh-mm-ss

This script can take up to and hour or more to run



41/41


Questions

Documents

Tivoli Access Manager problem determination using logging and tracing features