Introduction to Resource Efficiency Testing · Resource-Efficiency Testing Introduction (1 of 3)...

Resource-Efficiency Testing

Introduction to Resource‐Efficiency Testing

• Introduction

• Metrics Acquisition - Windows Performance Monitor

• OPC Server test plan model

• OPC Client test plan model

• Network Failure Simulation Batch File

• Test Reports

• Next Steps

Resource-Efficiency TestingContents

Resource-Efficiency TestingIntroduction (1 of 3)

What is it? 36-hour stress-test to verify no resource leaks exist

in your product, e.g. CPU, memory, threads etc.

More info? Windows Performance Monitor is used in the lab; the

performance counter details are available online:www.opcfoundation.org/ua/ctt/help/testcases/server/all/testcases.html#Efficiency

technet.microsoft.com/en-us/library/cc768048.aspx

What is tested? UA Servers/Clients, OPC Classic Servers/Clients

Easy to test? Yes, but several other OPC products are required to

generate a load on the product being tested.

4/19/2012 © 2012 OPC Foundation 3

You

Resource-Efficiency TestingIntroduction (2 of 3)

4/19/2012 © 2012 OPC Foundation 4

How it works: UA and Classic products are tested the same way:

Servers:- multiple device (PLC) connections; 1 is intermittent

- 5 or more OPC Client connections, 1 is intermittent

Clients: - 5 or more OPC Server connections, 1 is intermittent

Load is placed on the product for stress-testing.

Resources are measured over 36-hours.

Test authors? Compliance Working Group

Resource-Efficiency TestingIntroduction (3 of 3)

How do I pass? No resource leaks or performance degradation.

Am I Certified? No. It means that your product is a good candidate

for certification testing in a Test Lab.

Lab tools used? Windows Performance Monitor, network fault

simulation BAT file, also:

Servers: 2 or more PLCs; 5 or more OPC Clients.

Clients: 5 or more OPC Servers.

Test when? As frequently as possible, and on any official build

whether it is alpha, beta, or release etc.

4/19/2012 © 2012 OPC Foundation 5

Resource-Efficiency TestingWindows Performance Monitor (1 of 6)

4/19/2012 © 2012 OPC Foundation6

Configure the Windows Performance monitor to record the performance counters for 36‐hour duration• Click Start, click in the Search box, type perfmon.msc, and then press ENTER.

Perfmon.msc

NP4

Resource-Efficiency TestingWindows Performance Monitor (2 of 6)

4/19/2012 © 2012 OPC Foundation7

Create a Data Collector Set to store the data values recorded for the 36‐hour test.

Specify the directory you want to store the 

file

Create a new Data collector set to record the data

Click finish to create the Data collector 

set

Resource-Efficiency TestingWindows Performance Monitor (4 of 6)

4/19/2012 © 2012 OPC Foundation8

Configure the Process, Processor, Threads, Handles, and other counters for the instances of your product:  

Step 3: Pick the Client/Server  Process 

(must be running to appear in the this list)

Step 4: Choose the performance counters 

from the list

Step 1: Double click “Performance Counter ” to select 

the counters

Step 2: Click “Add” to specify the resources to monitor

Step 5: Select the log format. Test lab uses comma separated format to record the values for the counters 

selected.

Resource-Efficiency TestingWindows Performance Monitor (5 of 6)

4/19/2012 © 2012 OPC Foundation9

Setup a 36‐hour stop condition in the Data Collector Set properties to stop recording:

Setup the duration to 36 

hours 

Click OK to apply the changes

Resource-Efficiency TestingWindows Performance Monitor (6 of 6)

4/19/2012 © 2012 OPC Foundation10

Setup the stop condition in the Data Collector Set properties to stop recording the data values after 36 hours.

Measurements can be displayed in a chart, as well as being logged to a file (as previously configured).

Click the Triangle icon to start monitoring…

Resource-Efficiency TestingTest Plan Model – Server (1 of 2)

4/19/2012 © 2012 OPC Foundation11

Device 1(e.g. PLC)

Device 2(e.g. PLC)

Remote PC1Classic/UA Client 

with max no of Tags Supported

• An OPC Server is configured with 2 devices (PLCs)• Typically, 5 Clients are connected, with approx. 1000+ items each• Make 1 Device and 1 Client go online/offline every 2‐minutes

Remote PC2Classic/UA Client 

with max no of Tags Supported

Remote PC3Classic/UA Client 

with max no of Tags Supported

Remote PC4Classic/UA Client 

with max no of Tags Supported

Remote PC5Classic/UA Client 

with max no of Tags Supported

Local Client (if available)UA/Classic Server 

NP5

Resource-Efficiency TestingTest Plan Model – Server (1 of 2)

4/19/2012 © 2012 OPC Foundation12

Remote PC1Classic/UA Server with max no of Tags 

Supported

• An OPC Client is connected to 5 or more OPC Servers, each approx. 1000+ items• Make 1 Server go online/offline every 2‐minutes

Remote PC2Classic/UA Server with max no of Tags 

Supported

Remote PC3Classic/UA Server with max no of Tags 

Supported

Remote PC4Classic/UA Server with max no of Tags 

Supported

Remote PC5Classic/UA Server with max no of Tags 

Supported

Local Server (if available)UA/Classic Client

NP5

@echo offSET DTIME=120CLSECHO Delay time is: %DTIME%ECHO Network Connection Enabler/Disabler started at: %date% %time%

:startECHO Disabling network adapter at %time%netsh interface set interface "Local Area Connection" DISABLED

timeout /t %DTIME%ECHO Enabling network adapter at %date% %time%netsh interface set interface "Local Area Connection" ENABLED

choice /c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d yECHO %ERRORLEVEL%IF ERRORLEVEL 2 GOTO endIF ERRORLEVEL 1 GOTO start

:endECHO.ECHO Network adapter recycle complete at %date% %time%

@echo offSET DTIME=120CLSECHO Delay time is: %DTIME%ECHO Network Connection Enabler/Disabler started at: %date% %time%

:startECHO Disabling network adapter at %time%netsh interface set interface "Local Area Connection" DISABLED

timeout /t %DTIME%ECHO Enabling network adapter at %date% %time%netsh interface set interface "Local Area Connection" ENABLED

choice /c:yn /m "Repeat network adapter recycle (y/n)?" /t %DTIME% /d yECHO %ERRORLEVEL%IF ERRORLEVEL 2 GOTO endIF ERRORLEVEL 1 GOTO start

:endECHO.ECHO Network adapter recycle complete at %date% %time%

Resource-Efficiency TestingBatch File

4/19/2012 © 2012 OPC Foundation13

• To toggle the enabled state of the “Local Area Connection” network adapter in a windows environment use the below batch file.

• Replicate the following text into file “NIC.BAT” and run it using Administrator/elevated permission.

NP6

Resource-Efficiency TestingReports

4/19/2012 © 2012 OPC Foundation14

In Certification testing, the Test Lab will generate a resource Data report that is generated from the data collected by the Windows Performance Monitor:  

NP7

Resource-Efficiency TestingNext Steps…

If no growth is detected in your Classic/UA product then you are ready for Lab Certification:

• Request Certification in the Test Lab:www.opcfoundation.org/certification/schedule-testing

4/19/2012 © 2012 OPC Foundation 15

Certification: www.opcfoundation.org/certification

Lab Testing: www.opcfoundation.org/certification/schedule-testing

Bug Tracker: www.opcfoundation.org/mantis

Contact: compliance@opcfoundation.org

Resource-Efficiency TestingThe End!