Troubleshoot Enterprise Portal 6.0 SP2 - SAP · Troubleshoot Enterprise Portal 6.0 SP2 Part I: Master the Basics SAP AG 2003, Title of Presentation, Speaker Name / 2 What are EP 6.0

1

Troubleshoot Enterprise Portal 6.0 SP2

Part I: Master the Basics

SAP AG 2003, Title of Presentation, Speaker Name /2

What are EP 6.0 SP2 Troubleshooting Basics?

Essential knowledge, tools and methods which answer the following questions:

nOn which component should I focus troubleshooting effort?

nHow can I get more information about the problem?

nWhat to do when I identify the problem?

2


SAP Enterprise Portal 6.0

EP 6.0 SP2 Software Layers

Operating System

Java Virtual Machine 1.3.1 (SUN)

SAP J2EE Engine 6.20

Portal Framework

Portal Runtime - PRT

Portal Components(e.g. iViews)

Portal Services(e.g. http service)


EP 6.0 SP2 Architecture

3


User Management Architecture


User Management Architecture (II)

4


File System Structure (I)

sap j2eeusr <SID 1> j2ee_00 dispatcher

<SID 2>

j2ee_01

<SID n>

j2ee_n

server2

server

server n

J2EE-EngineFile System

cluster

admin

analyzer

configtool

ume


File System Structure (II)

dispatcher

services servlet_jsp

http

managers

jms

dbms

telnet

ThreadM.

Sys.Thr.M.

ClusterM.

ServiceM.

server

server2

server n

irj

J2EE-Engine

htmlb

work jspTemp

mobilehtmlb

J2EE Cluster Personality

settings

Console_logs

5


File System Structure (III)

root WEB-INF work portalirj lib

logs

portalapps

system

J2EE-Engine


Portal Support iViews

6


Portal Logging Console


What We’ve Learned So Far

nKnowing and understanding the EP 6.0 software layers and architecture helps isolate troubleshooting efforts in the integrated environment.

nObjects to assist troubleshooting exist in the file system:n SAP J2EE, java, and EP 6 output log files

n Property files which control SAP J2EE and EP 6 behavour

n SAP tools for controlling and analyzing error reporting

n SAP tools for visualizing configuration

nThe portal provides troubleshooting pages and iViews:n System Administration > Support

n System Administration > Monitoring > Logging Console

7


Troubleshooting Approach

Portal Error

StartupProblem?

ContentProblem?

WebserverProblem?

ClientProblem?

Auth.Problem?

Connect.Problem?

SAP J2EE Engine/Oper Sys?

Database/PCD?

UME/Persistence?

Browser/Network?

Network Config.?

Backend Sys Down?


Troubleshooting Approach: SAP J2EE Engine

EP inaccessible!http://portal.mycompany.com/irj/ -> ERROR

J2EE-Engine started?-> Java Processes (Nodes) running?

-> J2EE Documentation at http://portalX.mycompany.com:50000 available?

Just 1 Server or all Servers concerned?Connect to each Portal Server directly:

http://portalX.mycompany.com/irj/

Are all Nodes visible in the cluster?-> Login to J2EE-Engine via telnet („lsc“)

or J2EE Administrator Tool

-> Analysis of Console Logs of failed Node

Detailed Logging and Logfile analysis as necessary

8


Portal Error: Test each portal server


J2EE Engine Doc: http://portal.mycompany.com:50000 & java processes

9


Check SAP J2EE Status with Telnet Admin/Visual Admin


SAP J2EE Engine Console Logs

10


Portal/PRT LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/services/servlet_jsp/jspTemp/irj/root/WEB-INF/portal/logsExamples:acl_logger, http_logger, iview_logger, page_logger, ...

Logging in the Portal: Overview

The Portal consists of a number of components that work together.

è Different Components have separate Logging functions.

J2EE-Engine LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/console_logs/Examples:output.log, error.logPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/log/Examples:INFO.log, WARNING.log ERROR.log, EMERGENCY.log, ALERT.log


Logging: J2EE-Engine/Console Logs

J2EE-Engine - Console Logs:

Name: YYYY_DD_MM_at_HH_MM_SS_output.log

Path: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/console_logs/

Activation: console_logs.properties

Contains: Console Output, e.g. Startup messages, Garbage Collection Trace, Thread Dumps

Name: YYYY_DD_MM_at_HH_MM_SS_error.log

Path: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/ managers/console_logs/

Activation: console_logs.properties

Contains: Error and exceptions Messages about all kinds of errors and exceptions

Important: The setting “DaysToKeepLogs” in “console_log.properties” specifies how long (in days) to store the log files (default=7).

11


Logging: J2EE-Engine/Output.log (I)

Example for an output.log of a server node:

SAP J2EE Engine Version 6.20 PatchLevel 67440.20 is starting...

Loading: LogManager ... Loading: SystemThreadManager ... Loading: ThreadManager ... Loading: TimeoutManager ... Loading: MemoryManager ... Loading: PoolManager ... Loading: PolicyManager ... Loading: IpVerificationManager ... Loading: ClusterManager ... Loading: ClassLoaderManager ... Loading: SwapManager ... Loading: LockManager ... Loading: R3StartupManager ... Loading: ServiceManager ...

Loading core services:Starting core service monitor ... done.Starting core service p4 ... done.Starting core service file ... done.Starting core service log ... done.Starting core service dbms ... done.Starting core service security ... done.Starting core service naming ... done.Starting core service admin ... done.Starting core service deploy ... done.

Core services loaded successfully.


Logging: J2EE-Engine/Output.log (II)Loading additional services:

Starting service telnet ... done.Starting service ts ... done.Starting service appclient ... done.Starting service runtimeinfo ... done.Starting service javamail ... done.Starting service pinger ... done.Starting service logviewer ... done.Starting service shell ... done.Starting service eisconnector ... done.Starting service http ... done.Starting service PRTBridge ... done.Starting service jms ... done.Starting service dbpool ... done.Starting service rfcengine ... done.Starting service ejb ... done.

Registering Channels handler...PcdAccessService.init()PcdAccessService.afterInit()Initialization IViewsPersonalizationUpgraderService was successful

Starting service servlet_jsp ... done.Additional services loaded successfully.

SAP J2EE Engine Version 6.20 PatchLevel 67440.20 is running!

PatchLevel 67440.20 December 11, 2003 08:40 GMT

>Login :Element 3002 joined.Element 264395723 joined.Element 264395722 joined.Element 3020 joined.

12


Logging: J2EE-Engine/Log Manager – Log Service

J2EE-Engine – Log Manager:

Names: DEBUG.log, TRACE.log, INFO.log, NOTICE.log, WARNING.log, ERROR.log, CRITICAL.log, ALERT.log, EMERGENCY.log

Path: e.g./usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/cluster//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/memory//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/r3startup//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/service//usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/managers/log/timeout/

Activation: via J2EE Administrator Tool or via properties-file of manager

Contains: Specific information on Manager depending on the log level.

J2EE-Engine – Log Service:


Path:/usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/services/log/work/

Activation: via J2EE Administrator Tool or ConfigtoolContains: Specific information on Service depending on the log level.


Logging: J2EE-Engine/Log Level

The log system recognizes the following types of events:

• EMERGENCY (log level 0) – System is unusable

• ALERT (log level 1) – Immediate action must be taken

• CRITICAL (log level 2) – Critical conditions

• ERROR (log level 3) – Error conditions

• WARNING (log level 4) – Warning conditions

• NOTICE (log level 5) – Normal but significant events for the system

• INFO (log level 6) – Information

• TRACE (log level 7) – Events that occur as a result of application methods execution

• DEBUG (log level 8) – Debug-level messages,

And accordingly creates nine log file types with the these names.

13


Logging: J2EE-Engine/Logging Configuration

1) Select the Manager or the Log Service

2) Set values in Properties


Logging: J2EE-Engine/Built-In Log Viewer

2) Select Type

of Messages

For built-in Log Viewer:

1) Select Tab „Log Viewer“

Works for all Managers, the Services, awhole Node or the whole Cluster!

14


Logging: J2EE Engine/HTTP Service Logging

J2EE-Engine – HTTP Service:


Path:/usr/sap/<SID>/j2ee/j2ee_XX/cluster/[server|dispatcher]/services/http/log/

Configuration: via J2EE Administrator Tool or Configtool

Contains: Information related to the HTTP service on the log level.

The HTTP Service is one of the most important services and it has some special issues:

2004-03-26 10:47:38 | http | INFO | | 10.20.19.37 | 10.20.19.37 - - [26/Mar/2004:10:47:38 +0100] "GET /irj/index.html" 200 0 [187] |

2004-03-26 10:48:00 | http | INFO | | 10.20.19.37 | 10.20.19.37 - - [26/Mar/2004:10:48:00 +0100] "GET /irj/servlet/prt/portal/prtroot/com.sap.portal.navigation.portallauncher.default" 200 8853 [21953] |

Example messages taken from an services/http/log/INFO.log


HTTP Trace

All http communication can be traced from the HTTP Service of the Dispatcher Node for problem analysis.

Configuration is done in: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/dispatcher/services/http/properties

The following parameters have to be adjusted:

HttpTrace =[disable|testFull|textHeaders|hexFull|hexHeaders]

HttpTraceRequestOutput=File name or empty for Standard Output

HttpTraceResponseOutput=File name or empty for Standard Output

The HTTP Trace is disabled by default.

For all details about the HTTP Trace please check Note 657765.

15


Logging: J2EE-Engine/Log Viewer iView

Viewing J2EE Log Files within the Portal:

ð System Administrationð Supportð SAP J2EE Log Viewerð Log Viewer for J2EE 620

(The corresponding Logger has to be activated.)

Portal has to be up and running!


Logging: J2EE-Engine/Log Viewer iView

Searching in the Log File is possible!

16


Logging: J2EE-Engine/Standalone Log Viewer (Server)

The Standalone Log Viewer:è Client/Server Tool to conveniently access Log Files

independently from the J2EE-Engineè Also usable if J2EE-Engine is down!

Installation: Available as “logviewer-standalone.zip” in /usr/sap/<SID>/j2ee/j2ee_00/admin/ or as download from SAP Service Marketplace.

è Extract file on the Portal Serverè Set JAVA_HOME and LV_HOMEè Customize LogViewerServer.properties, especially parameters:

Logviewer_LogDirectoryReadFromFile= andLogviewer_LogDirectoryWriteToFile=

è Execute server.shè Run client.bat or client.sh on your client machine


Logging: J2EE-Engine/Standalone Log Viewer (Client)

With the Standalone Log Viewer can you view, filter and merge log files from a number of nodes in one place!

17


Troubleshooting Approach: Portal Solution Path

EP shows Error Messages!

Check Logs of active Portal Loggers within the Portal or on the File System

Error in Customer Development or SAP Standard Functions?

Activate related Loggers and analyze the messages

Check J2EE-Engine Console Logs


Logging in the Portal: Overview

The Portal consists of a number of components that work together.

è Different Components have separate Logging functions.

J2EE-Engine LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/console_logs/Examples:output.log, error.logPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/managers/log/Examples:INFO.log, WARNING.log ERROR.log, EMERGENCY.log, ALERT.log

Portal/PRT LoggingPath: /usr/sap/<SID>/j2ee/j2ee_XX/cluster/server/services/servlet_jsp/jspTemp/irj/root/WEB-INF/portal/logsExamples:acl_logger, http_logger, iview_logger, page_logger, ...

18


Logging: Portal/PRT Logging - Display Mode

Viewing Portal/PRT Logs from within the Portal:ð System Administrationð Supportð Portal Runtimeð Log Admin


Select NodeSelect LoggerSelect Log LevelClick „Display“


Logging: Portal/PRT Logging - Configuration

Configuration of Portal/PRT Logging within the Portal:ð System Administrationð Supportð Portal Runtimeð Log Adminð Configuration

Alternatively the Portal/PRT Logging can be modified in file “logger.xml” in:

cluster/server/services/servlet_jsp/work/jspTemp/irj/root/WEB-INF/portal/system/xml/

19


Logging: Portal/PRT Logging – Log Levels

Log Levels in Portal/PRT Logging:§ OFF§ SEVERE§WARNING§ INFO§ ALL


Logging: Portal/PRT Logging – Output Types

Writes log messages in a file in XML format.XmlFileLogger

Writes log messages in a file. The format can be specified.SimpleFileLogger

Writes log messages in a file with a simple hash-separated-field format.

ListFileLogger

Writes log messages to the console.ConsoleLogger

DescriptionLogging Class

Loggers can be set up to put out messages in different log levels to different destinations.

This can be easily seen in the logger.xml:

20


Logging: Portal/PRT Log and File Viewer

Viewing files from within the portal:ð System Administrationð Supportð Portal Runtimeð Log Viewer


By browsing the file system any file can be displayed in the browser from this iView!


Logging: Portal/PRT Log and File Viewer

The Log and File Viewer

can make problem

analysis a lot easier!

21


PCD: Object Locking Tool

The Object Locking Tool enables you to locate objects being edited and currently in locked mode.

Path: System Administration -> Monitoring -> Portal -> Object Locking


Preventive Measures: java virtual machine parameters

nProper fine tuning of java memory parameters prevents:n SAP J2EE crashes and instabilityn Poor portal performancen java “out of memory (OOM)” errorsn Implement the SAP-provided How-To Guide “Fine-

Tuning Performance of EP 6.0 SP2” section “Tuning the Java Virtual Machine” immediately after installation.

nReview and understand CSN notes: 552522, 634689, 597187, 610134, 667841, 667711

nImplement java garbage collection output to console logs with “–verbose:gc” in SAP J2EE server (not dispatcher) startup.n Always attach console error log with verbose garbage

collection output to any CSN message.

nRemove java “–server” parameter from server startup to improve stability of SAP J2EE.

22


Summary

Enterprise Portal 6.0 SP2 exists in a constellation of technologies. Understanding these software layers and their implementation leads to faster problem identification and solution.

Troubleshooting a portal error can take a J2EE Engine approach or a portal approach.

SAP provides many tools and settings, especially loggers and logviewers, for analyzing problems in the portal. Fluency in thesetools improves analysis and solution timeframes.

Setting up portal environments that report useful information for problem analysis and submitting the correct information with a CSN message leads to increased problem resolution efficiency.

Preventive measures are better than curatives.


End of Troubleshooting Part I: Master the Basics

Q & A?

Documents

Troubleshoot Enterprise Portal 6.0 SP2 - SAP · Troubleshoot Enterprise Portal 6.0 SP2 Part I: Master the Basics SAP AG 2003, Title of Presentation, Speaker Name / 2 What are EP 6.0