SAP EarlyWatch Alert
SAP Active Global SupportSAP AG
Agenda
EarlyWatch Alert und EarlyWatch Alert for Solutions
EarlyWatch Alert in your Solution Manager
Chapter by Chapter
How to get to the EWA Configuration:Alternative 1
2
Transaction SOLMAN_WORKCENTER: Tab SAP Solution Manger Administration
How to get to the EWA Configuration:Alternative 2
32
Transaction SOLMAN_WORKCENTER: Tab SAP Solution Manger Administration
EWA Administration
Checkbox does not work, see SAP Note 1362950.Fixed with ST 400 SP 21.
Default will be changed to yes in ST 400 SP23.
Find EWA Reports in Solution ManagerAlternative I
23
Transaction SOLMAN_WORKCENTER: System Monitoring
Find EWA Reports in Solution ManagerAlternative II
2
Transaction SOLMAN_WORKCENTER: Tab SAP Engagement and Service Delivery
3
Monitoring in Solution Manager
SAP Solution Manager
Satellite Systems
System Monitoring
Weekly Evaluation:Service Data Download
(SDCC), E2E Diagnostics
Real-time Monitoring:CCMS Monitoring
Infrastructure (RZ20)
SAP EarlyWatchAlerts
Business ProcessesRepository
SAP EarlyWatchAlert for Solutions
Business Process Monitoring
Next Generation: SAP EarlyWatch Alert for Solutions
SAP EarlyWatch Alert
Pro-active System evaluation at SAP or in Solution Manager
Based on a single system
Gives System KPIs
Statistics on single transactions/reports
Raises System Alerts
Shows long-term histories of KPIs
SAP EarlyWatch Alert for Solutions
is powered by SAP Solution Manager including Diagnostics
Based on system landscape
Gives aggregated Solution KPIs
Performance of core business processes
Classifies alerts in areas for improvement
Shows trends and correlation
Tracks changes in the landscape
PresenterPresentation NotesRequirement for a full scope Service Report:Solution Manager Diagnostics is set-up for satellite systems Core Business Process maintained in Repository of Solution ManagerSee SAP Note 1040343 for SAP components, for which full scope Service Report is available.
In Solution Manager or @SAP
SAP Solution Manager
Monitored System
RFC Data transferMonitored System
Monitored System
CUSTOMER
Alternative RFC Data transfer
Service Marketplace
Service Tools and Plug-ins
SAP Solution Manager
Monitored System
RFC Data transferMonitored System
Monitored System
CUSTOMER
Alternative RFC Data transfer
Service Tools: ST-SER 701_2008_2 SP
SolMan Tool: ST 400 SP
ST-PIST-A/PI
ST-PIST-A/PI
ST-PIST-A/PI
*described in SAP Note 1143775
Service Marketplace
PresenterPresentation Notes
Current Releases in August 2009:
ST-PI 2008_1_ST-A/PI 01L_
New concept: Service Content Update
SAP Solution Manager
Monitored System
RFC Data transferMonitored System
Monitored System
CUSTOMER
Alternative RFC Data transfer
Service Tools: ST-SER 701_2010_1 + Service Content Update*
SolMan Tool: ST 400 SP >15
ST-PIST-A/PI
ST-PIST-A/PI
ST-PIST-A/PI
*described in SAP Note 1143775
Service Marketplace
In Solution Manager or @SAP
You should use your local Solution Manager for EWA, because:
Content about Java components is only available there. Prerequisites is Solution Manager Diagnostics.
New EWA content for CRM systems is read from Solution Manager BI
More checks are currently developed to read from SolMan BI, e.g. for ABAP Workload Analysis
Prerequisite for EarlyWatch Alert for Solutions.
SAP Inst
Customer System
Data sources for EWA
Solution Manager
SDCCN
Diagnostic Agent
CCDBWily IntroScope
SM Diagnostics
BI Statistics
SDCCN
EWA localreport
Solution Manager
Configuration andChange DB
Service Data Control Center
EWAprocessed by SAP
CUSTOMER
The EWA report in your local Solution Manager has additional data sources from Solution Manager Diagnostics.
PresenterPresentation NotesSDCC: Service Data Control Center , SAP note 91488
SAP Solution Manager
SMD
Integration of Solution Manager and EarlyWatch Alert
Data Basis used for Solution Manager Diagnostics is now integrated into the EarlyWatch Alert
Detailed time profile, peak workload and error analysis is now possible!
BI
CRM:
History of max. & avg. number of BDocs and qRFC and R&R* queue entries
History of CRM Mobile Synchronizations
History of ICM Load
Day of highest Load: 24-hour time profiles of CRM Mobile synchronizations, qRFC and R&R Queues and BDocs
EWA
New Content in EWA due to Integration
*Replication and Realignment Queue
PresenterPresentation NotesBDoc = Business Document. A BDoc is a document that contains information about changes to a business object. BDocs are used to exchange and process data in a CRM solution. There are different classes of BDocs available.
R&R = Replication and Realignment Queue Replication & Realignment: Receiver determination and replication of data to the receivers that have been determined.
The Report: Chapter by Chapter
Determined automatically from table T000: If there is any client with field CCCATEGORY containing the value 'P then Productive, else Non-Productive.
From which rating does the overall rating get the red rating?
SAP BW: KPI's Query Performance Minimum SPS BWA Memory Usage
SAP CRM: Unprocessed BDoc messages BDocs in error status BDocs waiting for external receivers Replication and Realignment Queues Mobile Client Message Recovery -
unextracted messages
SAP PI / XI : Performance Message Errors
Java: Garbage Collection Performance and Fill
Level Old Space Usage Request Bottleneck
Database: Database Performance Missing indexes SQL Server with outdated DLL
General: System Performance (Performance
Evaluation, DB Profile, HW Capacity) Problems in qRFC queue Different Kernel Version or Patch Kernel needs Updating MDMP Support
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware CapacitySecurity
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Report Chapters
Software Change Management
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
Critical area, but not decisive for report rating
Reason for red report
Alert propagation
Critical issues
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware Capacity
Software Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Security
Key Performance Indicators (KPIs)
General tendency compared to the last SAP EWA Report.
< 500 users: small system500-2000 users: medium-sized system> 2000 users: large system
Key Performance Indicators (KPIs)
should be < 1 sec, depends on customer requirements
should be < 1 sec, depends on customer requirements
< 5.000 DS/h: small system5.000-100.000 DS/hmedium-sized system> 100.000 DS/h: large system
should be 100%
Performance Indicators and their trend
< 1000ms
< 50% of 1
< 2 ms
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingHardware Capacity
Software Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Security
For information on database and operating system platforms, see
Databases and OS on SDN.To verify this information in your system,
choose System -> Status.
Information taken from transaction ST06.
System Configuration
SAP 2007 / Page 28
Kernel check decisive for report redTo verify which SAP Kernel Release is installed in your system, in your SAP System, call transaction SM51 andchoose Release Notes. The release is indicated in the row Kernel Patch number.
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
DatabaseLayer
ApplicationLayer
PresentationLayer
Wait queue(memory)
Databasebuffers
Roll buffer
Roll file
Roll memory
. . .
Work process
Dispatcher
Databaseprocess
Extended memory
Response timeRoll-in time
Wait time
Summary: Response Time Components
Network = LAN
Network = LAN / WAN
Database Interface
Database time
Shared Pool Buffers
PresenterPresentation NotesThis slide summarizes the previous slides describing the sequence of actions that contribute to total response time in multi-tier architecture.
Note for instructor:- Show /nstad and /nst03 for parts of time
CPU time
Presentation Server
Rollin
Loadtime
Processing timeWaittime
Database time
Response time
Application Server Database Server
Net
wor
k
Net
wor
k
Workload Statistics (I)
Response time: Start: User request enters dispatcher queue; End: Next screen is returned to the user. Wait time: Request is waiting in dispatcher queue for a free work process. Roll-in time: User context information is rolled into the work process.Load time: Load from the database and generate objects like ABAP source code, CUA, and screen information.Database request time: Starts in database interface on application server; ends when database has delivered resuProcessing time = Response time (wait time + database time + load time + roll time + enqueue time)CPU time: This is the CPU time used by the R/3 work process.
PresenterPresentation NotesWorkload time statistics include:Response time in milliseconds: Starts when a user request enters the dispatcher queue; ends when the next screen is returned to the user. Wait time in milliseconds: This is the time a user request is waiting in the dispatcher queue. It starts when user request is entered in the dispatcher queue; and ends when the request starts being processed. Roll-in time in milliseconds: The amount of time needed to roll user context information into the work process.Load time in milliseconds: The time needed to load from the database and generate objects like ABAP source code, CUA, and screen information.Processing time: This is equivalent to response time minus the sum of wait time, database request time, load time, roll time, and enqueue time.Database request time: Starts when a database request is put through to the database interface; ends when the database interface has delivered the result.CPU time in milliseconds: This is the CPU time used by the R/3 work process
All transactions
affected?
Program analysis
No
Yes
(Cont.)
Performanceproblem?
Check:
Workload on all servers
Problem symptoms:
Large load time
Processing timemuch larger thanCPU time
Large databaserequest times
(Average)Large CPU times
Large roll-wait time
Workload Monitor points us to the cause of the performance problem (I)
PresenterPresentation NotesIf a problem is detected, the data in the Workload Monitor (Transaction ST03N) can be used as follows to identify the area of the system where the problem is located. First check for general performance problems affecting all transactions. Good general performance is normally indicated by: Wait time < 10% response timeMain menu (choose Transaction Profile) < 100 msIn the Workload Monitor, the following values normally indicate good performance:Average roll-in time < 20 ms Average roll wait time < 200 ms Average load (and generation) time < 10 % of response time (
Program buffer, CUA buffer, or screen buffer too smallLarge load time
CPU bottlenecks, network problems, communication problems
Processing timemuch larger thanCPU time
CPU/memory bottleneck on database server, network problems, expensive SQL statements, database locks,missing indexes, missing statistics, small buffers
Large databaserequest times
(Average)Large CPU times Expensive ABAP processing, for example, processing large tables, frequent accessing of R/3 buffers
Communication problem with GUI or external systemLarge roll-wait time
Workload Monitor points us to the cause of the performance problem (II)
May be caused by:
PresenterPresentation Notes
Workload Statistics by Task Type
Should be < 1000ms
Should be < 50% of
response time
PresenterPresentation NotesResponse Time < 1000msCPU Time < 50% of response timeWait Time < 50 msLoad Time < 50 msDB Time < 50 % of Response TimeGUI Time < 200 ms
Task Type RFC: Remote Function Calls between SAP Systems or SAP and external systems, as well as communication inside one SAP System e.g. for parallelization.
Rating details for Performance Evaluation
-RED if at least three time ranges are rated RED- YELLOW if at least two time ranges are rated RED or at least three time ranges are rated YELLOW- Otherwise, GREEN
Transaction Profile I
The Transaction Profileshows the statistical data
aggregated by transaction
Display variant 1:
sorted by total response time (= load caused on the system by this transaction)
Transaction Profile II
The Transaction Profileshows the statistical data
aggregated by transaction
Display variant 2:
sorted by DB time (= load caused on the DB by this transaction)
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
Workload by Application Module
CPU load and DB load are displayed in a pie chart and table. Rating is always green.
DB Load Profile
The diagram shows how many DB processes are used in parallel in a certain time frame on a weekly average.
The check rating can indicate a DB bottleneck by comparing the used DB processes with the number of CPUs in the DB Server:
A YELLOW rating is set if in at least two time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x no. CPUs on the database server).
A RED rating is set if in at least five time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x No CPUs on the database server).
PresenterPresentation NotesA GREEN rating is set if the number of CPUs is unknown or is marked as not relevant.A YELLOW rating is set if in at least two time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x no. CPUs on the database server).A RED rating is set if in at least five time periods ("DB Load DIA" + "DB Load RFC" + "DB Load BTC") is larger than (3 x No CPUs on the database server).
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
SAP System Operating
Update Errors: Rating Scheme: Info from SM13
Table Reorganizations: Top 20 tables and indexes (by size) are checked for their size and monthly growth. For
known issues SAP Notes for reorganization are recommended. No rating propagation to chapter level.
Transports Number of transported objects and programs in the last week are shown (SAP,
Customer or in SP). Rating is always green
Program Errors (ABAP Dumps) Rating Scheme:
Users Green Yellow Red
1000 200
Green Yellow
30
Hardware Capacity
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
Security
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Software Change Management
Hardware Capacity
CPU load thresholds: yellow 75 %, red 90% Paging thresholds depend on components, e.g. Java and Database server more sensible
ST06:User avg + Sys avg
EW Report I
CPU utilization < 75% No CPU bottleneck
Avg. Paging/h < 20% of RAM No Memory bottleneck
PresenterPresentation NotesEarlyWatch Alert Reports generated at SAP show 0 utilization from February 2009 until 15th of April 2009. See SAP Note 1326443.
EW Report II
CPU utilization > 75% Temporary CPU bottleneck
Avg. Paging/h > 20% of RAM Temporary Memory bottleneck
Key Performance Indicators (KPIs)
Consequences
Hardware bottlenecks (especially long-lasting ones) harm the performance of all users on this one machine
This is especially critical on the DB server, since here a hardware bottleneck will harm the overall system performance
The database server is a unique resource in the R/3 system
All users suffer from increased response times !
Ensure that there never is a CPU bottleneck on the database server (!)
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
Unsuccessful Security Check
Security Checks are described in SAP Note
863362.
Successful Security Check
User Authorizations
Yellow alert, if:More than 10% of the users (but at least 10) of a client.orMore than 75 users of a client have the authorization.
No red alert!
PresenterPresentation NotesCritical Authorizations:Display all TablesDebug / ReplaceAdminister RFC Connections
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Hardware Capacity
SAP 2007 / Page 56
Failed Changes
SAP 2007 / Page 57
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Database PerformanceDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Chapter content is DB vendor specific and
will not be covered in this presentation. Hardware Capacity
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingSecurityHardware Capacity
Performance Indicators
Database SettingsDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Chapter content is DB vendor specific and
will only be covered in an overview.
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Hardware Capacity
Database Administration
Database Growth and Free Space Development are shown
Backup and Archive Frequency is checked
Free Space in tablespaces and space critical objects are investigated
DB Release is checked
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingSecurityHardware Capacity
Performance Indicators
Database SettingsDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Hardware Capacity
Expensive Statements
DB time in dialog > 600ms and more than 500 mio. reads/day
DB time in dialog < 400ms or less than 100 mio. reads/day
PresenterPresentation NotesDefinition for Expensive SQL Statement: Top 10 statements that have more than 2% elapsed time from the total elapsed time of all statements.
Expensive Statements
Records/Exe. = 3-> selective!
Buffer gets/ records = 19600-> inefficient access!
Explain plan and Origin of Coding
PresenterPresentation NotesAccess predicates and Filter predicates in Explain plan In Oracle 10g, when we see the explain plan for a SQL, then there are 2 new items that are of interest - the 'Access predicate' and the 'Filter predicate'.We know that depending on the predicate information ('where clause') the Oracle optimizer chooses the optimal path for executing the query.So the 'access predicate' information tells us how the Oracle optimizer is accessing the rows of the table - i.e. if the optimizer is using an index, then what predicate made it use the same.The 'filter predicate' tells us what criteria Oracle is using to filter rows from the returned rowset (the rowset was fetched based on the 'access predicate') .
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System OperatingSecurityHardware Capacity
Performance Indicators
Database SettingsDatabase AdministrationDatabase Server Load from Expensive SQL StatementsTrend Analysis
Service Summary
System ConfigurationPerformance OverviewWorkload DistributionSAP System Operating
SecuritySoftware Change Management
Performance Indicators
Hardware Capacity
Trends in Activity and Performance
Top 5 Transactions by total load
System Operating
Availability in EWA is based on Collector protocols only. It is an hourly "life sign monitoring" of the ABAP stack central instance.
In Solution Manager the CCMSPING-based availability measurement allows you to monitor the central instances (ABAP and JAVA), instance availability, instance logon availability, and the SAP logon groups by the minute. For details see SAP Note 1332677.
SAP EarlyWatch Alert SAP Active Global SupportSAP AGAgendaHow to get to the EWA Configuration:Alternative 1How to get to the EWA Configuration:Alternative 2EWA AdministrationFind EWA Reports in Solution ManagerAlternative IFind EWA Reports in Solution ManagerAlternative IIMonitoring in Solution ManagerNext Generation: SAP EarlyWatch Alert for Solutions In Solution Manager or @SAPService Tools and Plug-insNew concept: Service Content UpdateIn Solution Manager or @SAPData sources for EWAIntegration of Solution Manager and EarlyWatch AlertThe Report: Chapter by ChapterFrom which rating does the overall rating get the red rating?Slide Number 18Slide Number 19Slide Number 20Alert propagationSlide Number 22Key Performance Indicators (KPIs)Key Performance Indicators (KPIs)Performance Indicators and their trendSlide Number 26 System ConfigurationKernel check decisive for report redSlide Number 31Summary: Response Time ComponentsWorkload Statistics (I)Workload Monitor points us to the cause of the performance problem (I)Workload Monitor points us to the cause of the performance problem (II)Workload Statistics by Task TypeRating details for Performance EvaluationTransaction Profile ITransaction Profile IISlide Number 40Workload by Application ModuleDB Load ProfileSlide Number 43SAP System OperatingSlide Number 45Hardware CapacityEW Report IEW Report IIKey Performance Indicators (KPIs)ConsequencesSlide Number 51Unsuccessful Security CheckSuccessful Security CheckUser AuthorizationsSlide Number 55 Failed ChangesSlide Number 58Slide Number 59Database AdministrationSlide Number 61Expensive StatementsExpensive StatementsExplain plan and Origin of CodingSlide Number 65Trends in Activity and PerformanceTop 5 Transactions by total loadSystem Operating