Ewa Content Red Alerts

SAP EarlyWatch Alert

SAP Active Global SupportSAP AG

Agenda

EarlyWatch Alert und EarlyWatch Alert for Solutions

EarlyWatch Alert in your Solution Manager

Chapter by Chapter

How to get to the EWA Configuration:Alternative 1

2

Transaction SOLMAN_WORKCENTER: Tab SAP Solution Manger Administration

How to get to the EWA Configuration:Alternative 2

32

Transaction SOLMAN_WORKCENTER: Tab SAP Solution Manger Administration

EWA Administration

Checkbox does not work, see SAP Note 1362950.Fixed with ST 400 SP 21.

Default will be changed to yes in ST 400 SP23.

Find EWA Reports in Solution ManagerAlternative I

23

Transaction SOLMAN_WORKCENTER: System Monitoring

Find EWA Reports in Solution ManagerAlternative II

2

Transaction SOLMAN_WORKCENTER: Tab SAP Engagement and Service Delivery

3

Monitoring in Solution Manager

SAP Solution Manager

Satellite Systems

System Monitoring

Weekly Evaluation:Service Data Download

(SDCC), E2E Diagnostics

Real-time Monitoring:CCMS Monitoring

Infrastructure (RZ20)

SAP EarlyWatchAlerts

Business ProcessesRepository

SAP EarlyWatchAlert for Solutions

Business Process Monitoring

Next Generation: SAP EarlyWatch Alert for Solutions

SAP EarlyWatch Alert

Pro-active System evaluation at SAP or in Solution Manager

Based on a single system

Gives System KPIs

Statistics on single transactions/reports

Raises System Alerts

Shows long-term histories of KPIs

SAP EarlyWatch Alert for Solutions

is powered by SAP Solution Manager including Diagnostics

Based on system landscape

Gives aggregated Solution KPIs

Performance of core business processes

Classifies alerts in areas for improvement

Shows trends and correlation

Tracks changes in the landscape

PresenterPresentation NotesRequirement for a full scope Service Report:Solution Manager Diagnostics is set-up for satellite systems Core Business Process maintained in Repository of Solution ManagerSee SAP Note 1040343 for SAP components, for which full scope Service Report is available.

In Solution Manager or @SAP


Monitored System

RFC Data transferMonitored System

Monitored System

CUSTOMER

Alternative RFC Data transfer

Service Marketplace

Service Tools and Plug-ins


Monitored System


Monitored System

CUSTOMER


Service Tools: ST-SER 701_2008_2 SP

SolMan Tool: ST 400 SP

ST-PIST-A/PI

ST-PIST-A/PI

ST-PIST-A/PI

*described in SAP Note 1143775

Service Marketplace

PresenterPresentation Notes

Current Releases in August 2009:

ST-PI 2008_1_ST-A/PI 01L_

New concept: Service Content Update


Monitored System


Monitored System

CUSTOMER


Service Tools: ST-SER 701_2010_1 + Service Content Update*

SolMan Tool: ST 400 SP >15

ST-PIST-A/PI

ST-PIST-A/PI

ST-PIST-A/PI

*described in SAP Note 1143775

Service Marketplace

In Solution Manager or @SAP

You should use your local Solution Manager for EWA, because:

Content about Java components is only available there. Prerequisites is Solution Manager Diagnostics.

New EWA content for CRM systems is read from Solution Manager BI

More checks are currently developed to read from SolMan BI, e.g. for ABAP Workload Analysis

Prerequisite for EarlyWatch Alert for Solutions.

SAP Inst

Customer System

Data sources for EWA

Solution Manager

SDCCN

Diagnostic Agent

CCDBWily IntroScope

SM Diagnostics

BI Statistics

SDCCN

EWA localreport

Solution Manager

Configuration andChange DB

Service Data Control Center

EWAprocessed by SAP

CUSTOMER

The EWA report in your local Solution Manager has additional data sources from Solution Manager Diagnostics.

PresenterPresentation NotesSDCC: Service Data Control Center , SAP note 91488


SMD

Integration of Solution Manager and EarlyWatch Alert

Data Basis used for Solution Manager Diagnostics is now integrated into the EarlyWatch Alert

Detailed time profile, peak workload and error analysis is now possible!

BI

CRM:

History of max. & avg. number of BDocs and qRFC and R&R* queue entries

History of CRM Mobile Synchronizations

History of ICM Load

Day of highest Load: 24-hour time profiles of CRM Mobile synchronizations, qRFC and R&R Queues and BDocs

EWA

New Content in EWA due to Integration

*Replication and Realignment Queue

PresenterPresentation NotesBDoc = Business Document. A BDoc is a document that contains information about changes to a business object. BDocs are used to exchange and process data in a CRM solution. There are different classes of BDocs available.

R&R = Replication and Realignment Queue Replication & Realignment: Receiver determination and replication of data to the receivers that have been determined.

The Report: Chapter by Chapter

Determined automatically from table T000: If there is any client with field CCCATEGORY containing the value 'P then Productive, else Non-Productive.

From which rating does the overall rating get the red rating?

SAP BW: KPI's Query Performance Minimum SPS BWA Memory Usage

SAP CRM: Unprocessed BDoc messages BDocs in error status BDocs waiting for external receivers Replication and Realignment Queues Mobile Client Message Recovery -

unextracted messages

SAP PI / XI : Performance Message Errors

Java: Garbage Collection Performance and Fill

Level Old Space Usage Request Bottleneck

Database: Database Performance Missing indexes SQL Server with outdated DLL

General: System Performance (Performance

Evaluation, DB Profile, HW Capacity) Problems in qRFC queue Different Kernel Version or Patch Kernel needs Updating MDMP Support

Service Summary

System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware CapacitySecurity

Performance Indicators

Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis

Report Chapters

Software Change Management

Service Summary

System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating

SecuritySoftware Change Management



Hardware Capacity

Critical area, but not decisive for report rating

Reason for red report

Alert propagation

Critical issues

Service Summary

System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware Capacity




Security

Key Performance Indicators (KPIs)

General tendency compared to the last SAP EWA Report.

< 500 users: small system500-2000 users: medium-sized system> 2000 users: large system


should be < 1 sec, depends on customer requirements

should be < 1 sec, depends on customer requirements

< 5.000 DS/h: small system5.000-100.000 DS/hmedium-sized system> 100.000 DS/h: large system

should be 100%

Performance Indicators and their trend

< 1000ms

< 50% of 1

< 2 ms

Service Summary

System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware Capacity




Security

For information on database and operating system platforms, see

Databases and OS on SDN.To verify this information in your system,

choose System -> Status.

Information taken from transaction ST06.

System Configuration

SAP 2007 / Page 28

Kernel check decisive for report redTo verify which SAP Kernel Release is installed in your system, in your SAP System, call transaction SM51 andchoose Release Notes. The release is indicated in the row Kernel Patch number.

Service Summary





Hardware Capacity

DatabaseLayer

ApplicationLayer

PresentationLayer

Wait queue(memory)

Databasebuffers

Roll buffer

Roll file

Roll memory

. . .

Work process

Dispatcher

Databaseprocess

Extended memory

Response timeRoll-in time

Wait time

Summary: Response Time Components

Network = LAN

Network = LAN / WAN

Database Interface

Database time

Shared Pool Buffers

PresenterPresentation NotesThis slide summarizes the previous slides describing the sequence of actions that contribute to total response time in multi-tier architecture.

Note for instructor:- Show /nstad and /nst03 for parts of time

CPU time

Presentation Server

Rollin

Loadtime

Processing timeWaittime

Database time

Response time

Application Server Database Server

Net

wor

k

Net

wor

k

Workload Statistics (I)

Response time: Start: User request enters dispatcher queue; End: Next screen is returned to the user. Wait time: Request is waiting in dispatcher queue for a free work process. Roll-in time: User context information is rolled into the work process.Load time: Load from the database and generate objects like ABAP source code, CUA, and screen information.Database request time: Starts in database interface on application server; ends when database has delivered resuProcessing time = Response time (wait time + database time + load time + roll time + enqueue time)CPU time: This is the CPU time used by the R/3 work process.

PresenterPresentation NotesWorkload time statistics include:Response time in milliseconds: Starts when a user request enters the dispatcher queue; ends when the next screen is returned to the user. Wait time in milliseconds: This is the time a user request is waiting in the dispatcher queue. It starts when user request is entered in the dispatcher queue; and ends when the request starts being processed. Roll-in time in milliseconds: The amount of time needed to roll user context information into the work process.Load time in milliseconds: The time needed to load from the database and generate objects like ABAP source code, CUA, and screen information.Processing time: This is equivalent to response time minus the sum of wait time, database request time, load time, roll time, and enqueue time.Database request time: Starts when a database request is put through to the database interface; ends when the database interface has delivered the result.CPU time in milliseconds: This is the CPU time used by the R/3 work process

All transactions

affected?

Program analysis

No

Yes

(Cont.)

Performanceproblem?

Check:

Workload on all servers

Problem symptoms:

Large load time

Processing timemuch larger thanCPU time

Large databaserequest times

(Average)Large CPU times

Large roll-wait time

Workload Monitor points us to the cause of the performance problem (I)

PresenterPresentation NotesIf a problem is detected, the data in the Workload Monitor (Transaction ST03N) can be used as follows to identify the area of the system where the problem is located. First check for general performance problems affecting all transactions. Good general performance is normally indicated by: Wait time < 10% response timeMain menu (choose Transaction Profile) < 100 msIn the Workload Monitor, the following values normally indicate good performance:Average roll-in time < 20 ms Average roll wait time < 200 ms Average load (and generation) time < 10 % of response time (

Program buffer, CUA buffer, or screen buffer too smallLarge load time

CPU bottlenecks, network problems, communication problems

Processing timemuch larger thanCPU time

CPU/memory bottleneck on database server, network problems, expensive SQL statements, database locks,missing indexes, missing statistics, small buffers

Large databaserequest times

(Average)Large CPU times Expensive ABAP processing, for example, processing large tables, frequent accessing of R/3 buffers

Communication problem with GUI or external systemLarge roll-wait time

Workload Monitor points us to the cause of the performance problem (II)

May be caused by:

PresenterPresentation Notes

Workload Statistics by Task Type

Should be < 1000ms

Should be < 50% of

response time

PresenterPresentation NotesResponse Time < 1000msCPU Time < 50% of response timeWait Time < 50 msLoad Time < 50 msDB Time < 50 % of Response TimeGUI Time < 200 ms

Task Type RFC: Remote Function Calls between SAP Systems or SAP and external systems, as well as communication inside one SAP System e.g. for parallelization.

Rating details for Performance Evaluation

-RED if at least three time ranges are rated RED- YELLOW if at least two time ranges are rated RED or at least three time ranges are rated YELLOW- Otherwise, GREEN

Transaction Profile I

The Transaction Profileshows the statistical data

aggregated by transaction

Display variant 1:

sorted by total response time (= load caused on the system by this transaction)

Transaction Profile II

The Transaction Profileshows the statistical data

aggregated by transaction

Display variant 2:

sorted by DB time (= load caused on the DB by this transaction)

Service Summary





Hardware Capacity

Workload by Application Module

CPU load and DB load are displayed in a pie chart and table. Rating is always green.

DB Load Profile

The diagram shows how many DB processes are used in parallel in a certain time frame on a weekly average.

The check rating can indicate a DB bottleneck by comparing the used DB processes with the number of CPUs in the DB Server:

A YELLOW rating is set if in at least two time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x no. CPUs on the database server).

A RED rating is set if in at least five time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x No CPUs on the database server).

PresenterPresentation NotesA GREEN rating is set if the number of CPUs is unknown or is marked as not relevant.A YELLOW rating is set if in at least two time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x no. CPUs on the database server).A RED rating is set if in at least five time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x No CPUs on the database server).

Service Summary





Hardware Capacity

SAP System Operating

Update Errors: Rating Scheme: Info from SM13

Table Reorganizations: Top 20 tables and indexes (by size) are checked for their size and monthly growth. For

known issues SAP Notes for reorganization are recommended. No rating propagation to chapter level.

Transports Number of transported objects and programs in the last week are shown (SAP,

Customer or in SP). Rating is always green

Program Errors (ABAP Dumps) Rating Scheme:

Users Green Yellow Red

1000 200

Green Yellow

30

Hardware Capacity

Service Summary


Security




Hardware Capacity

CPU load thresholds: yellow 75 %, red 90% Paging thresholds depend on components, e.g. Java and Database server more sensible

ST06:User avg + Sys avg

EW Report I

CPU utilization < 75% No CPU bottleneck

Avg. Paging/h < 20% of RAM No Memory bottleneck

PresenterPresentation NotesEarlyWatch Alert Reports generated at SAP show 0 utilization from February 2009 until 15th of April 2009. See SAP Note 1326443.

EW Report II

CPU utilization > 75% Temporary CPU bottleneck

Avg. Paging/h > 20% of RAM Temporary Memory bottleneck

Consequences

Hardware bottlenecks (especially long-lasting ones) harm the performance of all users on this one machine

This is especially critical on the DB server, since here a hardware bottleneck will harm the overall system performance

The database server is a unique resource in the R/3 system

All users suffer from increased response times !

Ensure that there never is a CPU bottleneck on the database server (!)

Service Summary





Hardware Capacity

Unsuccessful Security Check

Security Checks are described in SAP Note

863362.

Successful Security Check

User Authorizations

Yellow alert, if:More than 10% of the users (but at least 10) of a client.orMore than 75 users of a client have the authorization.

No red alert!

PresenterPresentation NotesCritical Authorizations:Display all TablesDebug / ReplaceAdminister RFC Connections

Service Summary





Hardware Capacity

SAP 2007 / Page 56

Failed Changes

SAP 2007 / Page 57

Service Summary





Chapter content is DB vendor specific and

will not be covered in this presentation. Hardware Capacity

Service Summary

System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingSecurityHardware Capacity


Database SettingsDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis

Chapter content is DB vendor specific and

will only be covered in an overview.

Service Summary




Hardware Capacity

Database Administration

Database Growth and Free Space Development are shown

Backup and Archive Frequency is checked

Free Space in tablespaces and space critical objects are investigated

DB Release is checked

Service Summary




Service Summary




Hardware Capacity

Expensive Statements

DB time in dialog > 600ms and more than 500 mio. reads/day

DB time in dialog < 400ms or less than 100 mio. reads/day

PresenterPresentation NotesDefinition for Expensive SQL Statement: Top 10 statements that have more than 2% elapsed time from the total elapsed time of all statements.

Expensive Statements

Records/Exe. = 3-> selective!

Buffer gets/ records = 19600-> inefficient access!

Explain plan and Origin of Coding

PresenterPresentation NotesAccess predicates and Filter predicates in Explain plan In Oracle 10g, when we see the explain plan for a SQL, then there are 2 new items that are of interest - the 'Access predicate' and the 'Filter predicate'.We know that depending on the predicate information ('where clause') the Oracle optimizer chooses the optimal path for executing the query.So the 'access predicate' information tells us how the Oracle optimizer is accessing the rows of the table - i.e. if the optimizer is using an index, then what predicate made it use the same.The 'filter predicate' tells us what criteria Oracle is using to filter rows from the returned rowset (the rowset was fetched based on the 'access predicate') .

Service Summary




Service Summary




Hardware Capacity

Trends in Activity and Performance

Top 5 Transactions by total load

System Operating

Availability in EWA is based on Collector protocols only. It is an hourly "life sign monitoring" of the ABAP stack central instance.

In Solution Manager the CCMSPING-based availability measurement allows you to monitor the central instances (ABAP and JAVA), instance availability, instance logon availability, and the SAP logon groups by the minute. For details see SAP Note 1332677.

SAP EarlyWatch Alert SAP Active Global SupportSAP AGAgendaHow to get to the EWA Configuration:Alternative 1How to get to the EWA Configuration:Alternative 2EWA AdministrationFind EWA Reports in Solution ManagerAlternative IFind EWA Reports in Solution ManagerAlternative IIMonitoring in Solution ManagerNext Generation: SAP EarlyWatch Alert for Solutions In Solution Manager or @SAPService Tools and Plug-insNew concept: Service Content UpdateIn Solution Manager or @SAPData sources for EWAIntegration of Solution Manager and EarlyWatch AlertThe Report: Chapter by ChapterFrom which rating does the overall rating get the red rating?Slide Number 18Slide Number 19Slide Number 20Alert propagationSlide Number 22Key Performance Indicators (KPIs)Key Performance Indicators (KPIs)Performance Indicators and their trendSlide Number 26 System ConfigurationKernel check decisive for report redSlide Number 31Summary: Response Time ComponentsWorkload Statistics (I)Workload Monitor points us to the cause of the performance problem (I)Workload Monitor points us to the cause of the performance problem (II)Workload Statistics by Task TypeRating details for Performance EvaluationTransaction Profile ITransaction Profile IISlide Number 40Workload by Application ModuleDB Load ProfileSlide Number 43SAP System OperatingSlide Number 45Hardware CapacityEW Report IEW Report IIKey Performance Indicators (KPIs)ConsequencesSlide Number 51Unsuccessful Security CheckSuccessful Security CheckUser AuthorizationsSlide Number 55 Failed ChangesSlide Number 58Slide Number 59Database AdministrationSlide Number 61Expensive StatementsExpensive StatementsExplain plan and Origin of CodingSlide Number 65Trends in Activity and PerformanceTop 5 Transactions by total loadSystem Operating

Documents

Ewa Content Red Alerts