Embed Size (px)
Citation preview
Website monitoring with APM:
handling one-time password
Document version 1.0
Enrico Vannini [email protected]
Rafal Szypulka [email protected]
IBM Hybrid Cloud - Client Advisor and Technical Enablement Team
© Copyright International Business Machines Corporation 2016. All rights reserved.
Table of Contents Table of Contents ........................................................................................................................................................................ 2 Introduction ................................................................................................................................................................................. 3
The goal ................................................................................................................................................................................. 3 The Scenario ......................................................................................................................................................................... 3
Solution Architecture .................................................................................................................................................................. 4 Components and prerequisites .................................................................................................................................................... 4 Details: installation and configuration ........................................................................................................................................ 5
SMS Gateway: ...................................................................................................................................................................... 5 Web service server: .............................................................................................................................................................. 7 APM ..................................................................................................................................................................................... 12
Results ....................................................................................................................................................................................... 12 Appendix A Source code .......................................................................................................................................................... 13
OTP_web_service.pl .......................................................................................................................................................... 13 01-HomePageAndLogin.html ............................................................................................................................................ 13 02-getOTP ........................................................................................................................................................................... 15 04-Logout ............................................................................................................................................................................ 17
Introduction
In this document we want to share an experience we recently had using Application Performance Management (APM) V8.1.3 website monitoring component, solution implemented by APM with the integration with open source tool Selenium IDE
The goal Many application scenarios include logging in to the target application with given user name and password. This is widely supported by any synthetic solution, either from IBM or others. But some business applications present an extra layer of security: after inserting user-id and password, user is asked to insert a one-time password (OTP from now on), that is sent a few seconds after the login from the application to a registered cell phone via SMS. Goal here is to prepare a synthetic script, using Selenium IDE, and upload it to APM server, that can play back this scenario, included the integration with the OTP system, i.e. with the cell phone. The reader should already have basic knowledge of APM website monitoring solution, the document will not describe how to install and use it The Scenario The detailed scenario is:
• reach application login page
• insert user-id and password
• receive the OTP as SMS on your smartphone • transmit the OTP from smartphone to the Selenium script via intermediate web service
• use it to continue the application scenario
• logout Of course the non-standard part is: How to forward the SMS content from the smartphone to the computer running APM synthetic agent and have Selenium reading it ? The rest of this document is all about it.
Solution Architecture
1. The Selenium script, running either form Selenium IDE GUI or from APM Synthetic agent, visits
the target application web page and reach the login page, insert user-id and password and sends them. Then it waits.
2. The application generates the OTP and sends is via SMS to a registered smartphone 3. SMS Gateway (or similar) application running on smartphone forwards any SMS message to a
given web service. Message is stored on web service server and can be retrieved via web service call on specific URL.
4. Selenium script running on the APM agent on a Linux box will continuously check the web service URL and once the password is available, it gets it. Once the password is used once, the web service itself will change to a value that that the script can interpret as 'this is not an OTP'. This prevents Selenium from reusing old OTP in next iterations.
5. Script sends the OTP to the target application and complete the scenario. 6. Availability and performance data are sent to APM server. 7. IT Operator connect to the APM console and check results.
Components and prerequisites
Below is the list of components prerequisites for the full solution to work. • APM Server: APM Server requires a Red Hat Enterprise Linux (RHEL) V6 or 7. Refer to standard APM documentation (http://ibm.co/2a9dfeR) and to compatibility report
APM Synthetic Agent / Selenium IDE script
WebService
12
3 4
5 6
7
SMS gtw App
(http://ibm.co/1yJLeBp) for more details. Otherwise the prerequisites check that is run automatically right before the installation will detect missing rpms or other missing prerequisites. Just a note: in the case you use APM Saas, keep in mind that the agent box must connect to the the web service running on a box you own. If you can't expose publicly that web server or if you need to keep it in a protected network, you will need to install the agent in the same network, and you can't use the IBM provided point of presence. Having APM server on IBM MarketPlace and synthetic agent on-prem is fully supported and possible since APM V8.1.3 • Synthetic agent: For the agent the above bullet is still valid. There is one extra restriction which can't be documented because it depends on Selenium: At today Selenium WebDriver, the engine APM uses on the playback station does not support newer version of firefox (e.g. it does not support FF 45). Be sure to have FF 41 or lower, else any test might fail • Selenium IDE: install the Flow Control plugin on the top of you Selenium IDE • Smartphone SMS Gateway application: Android 1.6 or newer, smartphone wifi network connectivity to web service server. • Web service server: perl 5 (version 5.10 or newer), Mojolicious::Lite perl module.
Details: installation and configuration SMS Gateway: This document refers to specific application found on Google Play store. Most probably there are more applications with similar features. Install SMS Gateway from Google Play on android smartphone: https://play.google.com/store/apps/details?id=eu.apksoft.android.smsgateway
1. Start SMS Gateway application, click on Settings button.
2. Select Forward incoming SMS to HTTP
checkbox.
3. Click on HTTP Settings button. And specify URL of your web service : http://<ip_addr>/test?device=first
4. Go back to the main application screen and click on the Start button to run application in the background. Verify that application status is: RUNNING
5. You may close configuration window. Please note that for commercial use, donate version of the SMS Gateway has to be purchased. Web service server: Web service was implemented using perl Mojolicious:Lite web framework. More information here: www.mojolicious.org. Mojolicious contains a HTTP server that is started together with the perl script.
1. Verify that perl interpreter is installed on Web Service Server using perl –v. Install per if needed. If you deploy solution on RedHat Enterprise Linux it should be installed by default.
2. Install Mojolicious::Lite module. Method described below assumes that server has internet access
a. Install cpanminus using command: curl -L https://raw.githubusercontent.com/miyagawa/cpanminus/master/cpanm| | perl - App::cpanminus
b. Install module using command: cpanm Mojolicious::Lite
3. Copy OTP_web_service.pl script (source code in Appendix A) to the web service server and
make it executable. 4. Start script using command:
./OTP_web_service.pl daemon -l http://*:8080 Script starts web server listening on port 8080 that will both receive information about new OTP from android SMS Gateway and receive queries from Selenium script asking if new OTP was generated. Diagnostic messages are enabled by default, so it is clearly visible on the console where script was started when requests are received from both Android application and Selenium script. [Mon Jul 25 13:27:01 2016] [debug] GET "/test" [Mon Jul 25 13:27:01 2016] [debug] Routing to a callback [Mon Jul 25 13:27:01 2016] [debug] 200 OK (0.000619s, 1615.509/s) Mon Jul 25 13:27:01 2016: OTP was received. OTP content: test message [Mon Jul 25 13:27:09 2016] [debug] GET "/test" [Mon Jul 25 13:27:09 2016] [debug] Routing to a callback [Mon Jul 25 13:27:09 2016] [debug] 200 OK (0.000289s, 3460.208/s) Mon Jul 25 13:27:09 2016: OTP queried by IBM APM. OTP content: test message Mon Jul 25 13:27:09 2016: OTP removed from memory [Mon Jul 25 13:27:16 2016] [debug] GET "/test" [Mon Jul 25 13:27:16 2016] [debug] Routing to a callback [Mon Jul 25 13:27:16 2016] [debug] 200 OK (0.000328s, 3048.780/s) Mon Jul 25 13:27:16 2016: IBM APM request received, but OTP was already queried. Source code of the script is located in the Appendix A.
Selenium IDE script
To achieve better results, the test suite was split to 4 tests:
• 01-HomePageAndLogin
• 02-getOTP
• 03-sendOTP
• 04-Logout The added value for the solution is in 02-getOTP 01-HomePageAndLogin Despite there is nothing specific to OTP handling, worth to comment this part of the test suite:
1. If you are afraid your website is too slow, you can change the default Selenium timeout with the setTimeout command. This applies to the single test case, if you have multiple test cases in the suite, you'll need to repeat it
2. Definitely a good practice to use variables for values like user id and password 3. The storeElementPresent will save true or false if it can or cannot locate the pointed element. This
can be useful in the case the browser was left in logged in status from the previous execution. If the user was logged in already, the Log in link will not be available, and you continue with the next line, else you jump to bullet 6 which is the actual log-in
4. If Log in is not available, we want at least that the Log out button is available, else we really need to throw an error. If available, we log-out
5. Next we verify again that the Log in link is available. If not, at this point we need to throw an error 6. Now is time to actually log-in, by inserting user-id and password, values saved in variables at
bullet #2 7. eventually you verify that the page waiting for OTP has shown The full source code for the above page is located in Appendix.
1
2
3
4
5
6
7
02-getOTP Now our application should send an OTP to the registered smartphone and then wait for the test case to retrieve it and type it in:
1. First, we will need to define variables. The idea is to create an active loop that will iterate a given number of time checking if the OTP is avail in the web service. After then it will fail the test. Counter is the variable we used to save the number of iteration. The other variable, NOOTPvar, is just to save the string returned by the web service when the OTP has not been refreshed (i.e. new OTP has not been received yet)
2. Lines 4, 5, 6, 8, 9 and 15 are the actual implementation for the active loop. At each iteration 1. we first increment the counter, 2. if counter has not reached the limit we pause for 1 second, 3. then we try to get the OTP from the web service 4. If OTP was not received, we iterate once more
3. verifyText will be executed if we reach the limit of iteration without receiving any OTP. In that case we just force a failure by checking for an element that we are sure it does not exist
1
2
3
4
5
4. This is the core part of the solution: 1. openWindow will open a popup page pointing to the URL where the web service can show the
OTP, if available or a given string (NOOTP in our sample) if since last execution no new OTP was received
2. selectPopUp select the newly open page, and all the following Selenium commands will refer to that page
3. we now store the page content in variable OTP 4. next we close the pop up 5. selectWindow will give again focus to the application page, and test can continue
Again, in the appendix the xml source code of this test case.
03-sendOTP Only step left is to click on next on the page:
and the source code is in the appendix 04-Logout Now we only need to verify that the login has completed successful, so we simply verify that the Log out link is available. If so, we click on it to complete the scenario.
Check Appendix A for source code
APM
After carefully testing the full test suite and before uploading it, remember to zip together the OTP, testsuite and each single html file that represents each singe test case Once done, the remaining procedure is the standard one for website monitoring with APM and Selenium. There is just onepoint worth to a comment, about variable usage:
when uploading the testsuite, check the advanced settings tab: all the variables used in the tests are listed on the right side of the page, and repeated for each point of presence where we decided to distribute the test. We also have the possibility to edit them without going back to the Selenium IDE UI. This is useful if we decide to use a different user, or when the password expires, or if we decide to allow a bigger delay to receive the OTP.
Results Results will be available on APM Dashboard some minutes after the test has started under the out of the box 'My Transaction' application. You might need to wait couple of iterations before data shows on the dashboard. At first iterations you can benefit of the web service output to verify iterations
Appendix A Source code
OTP_web_service.pl #!/usr/bin/perl #Author: Rafal Szypulka ([email protected]) use Mojolicious::Lite; my $DEBUG = 1; my $seen=''; get '/test' => sub { my $self = shift; my $device = $self->param('device'); #use $phone variable if you want to filter by sender phone number my $phone = $self->param('phone'); my $text = $self->param('text') if $DEBUG; if($text) { $self->render(text => "OTP stored."); say scalar(localtime).": OTP was received. OTP content: $text" if $DEBUG; $seen = "$text"; } elsif($seen && !$text) { $self->render(text => "$seen"); say scalar(localtime).": OTP queried by IBM APM. OTP content: $seen" if $DEBUG; say scalar(localtime).": OTP removed from memory" if $DEBUG; undef $seen; } else { $self->render(text => "No OTP or OTP already used"); say scalar(localtime).": IBM APM request received, but OTP was already queried." if $DEBUG; } }; app->start;
01-HomePageAndLogin.html <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head profile="http://selenium-ide.openqa.org/profiles/test-case"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <link rel="selenium.base" href="http://test.sssprocess.com/" /> <title>01-HomePageAndLogin</title> </head> <body> <table cellpadding="1" cellspacing="1" border="1"> <thead> <tr><td rowspan="1" colspan="3">01-HomePageAndLogin</td></tr> </thead><tbody> <tr> <td>setTimeout</td>
<td>180000</td> <td></td> </tr> <tr> <td>store</td> <td>[email protected]</td> <td>userId</td> </tr> <tr> <td>store</td> <td>MyP@ssw0rd</td> <td>password</td> </tr> <tr> <td>open</td> <td>/homePage</td> <td></td> </tr> <tr> <td>storeElementPresent</td> <td>link=Log in</td> <td>res</td> </tr> <tr> <td>gotoIf</td> <td>${res} == true</td> <td>doLogin</td> </tr> <tr> <td>verifyText</td> <td>link=Log out</td> <td>Log out</td> </tr> <tr> <td>clickAndWait</td> <td>link=Log out</td> <td></td> </tr> <tr> <td>verifyText</td> <td>link=Log in</td> <td>Log in</td> </tr> <tr> <td>label</td> <td>doLogin</td> <td></td> </tr> <tr> <td>clickAndWait</td> <td>link=Log in</td> <td></td> </tr> <tr> <td>type</td> <td>id=Email</td> <td>${userId}</td> </tr>
<tr> <td>type</td> <td>id=Password</td> <td>${password}</td> </tr> <tr> <td>clickAndWait</td> <td>//input[@value='Log in']</td> <td></td> </tr> <tr> <td>verifyText</td> <td>id=otp</td> <td></td> </tr> </tbody></table> </body> </html>
02-getOTP
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head profile="http://selenium-ide.openqa.org/profiles/test-case"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <link rel="selenium.base" href="http://test.sssprocess.com/" /> <title>02-getOTP</title> </head> <body> <table cellpadding="1" cellspacing="1" border="1"> <thead> <tr><td rowspan="1" colspan="3">02-getOTP</td></tr> </thead><tbody> <tr> <td>setTimeout</td> <td>180000</td> <td></td> </tr> <tr> <td>store</td> <td>10</td> <td>maxIteration</td> </tr> <tr> <td>store</td> <td>0</td> <td>counter</td> </tr> <tr> <td>store</td> <td>NOOTP</td> <td>NOOTPvar</td> </tr> <tr> <td>label</td> <td>tryAgain</td>
<td></td> </tr> <tr> <td>storeEval</td> <td>${counter}+1</td> <td>counter</td> </tr> <tr> <td>gotoIf</td> <td>${counter}<=${maxIteration}</td> <td>nextIter</td> </tr> <tr> <td>verifyText</td> <td>id=notpresent</td> <td>notpresent</td> </tr> <tr> <td>label</td> <td>nextIter</td> <td></td> </tr> <tr> <td>pause</td> <td>1000</td> <td></td> </tr> <tr> <td>openWindow</td> <td>http://<webservice-hostname>:<port-number>/test</td> <td></td> </tr> <tr> <td>selectPopUp</td> <td></td> <td></td> </tr> <tr> <td>storeBodyText</td> <td>OTP</td> <td></td> </tr> <tr> <td>close</td> <td></td> <td></td> </tr> <tr> <td>selectWindow</td> <td></td> <td></td> </tr> <tr> <td>gotoIf</td> <td>storedVars['OTP'] == storedVars['NOOTPvar']</td> <td>tryAgain</td> </tr> <tr>
<td>echo</td> <td>'OTP available'</td> <td></td> </tr> <tr> <td>type</td> <td>id=otp</td> <td>${OTP}</td> </tr> </tbody></table> </body></html>
03-sendOTP <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head profile="http://selenium-ide.openqa.org/profiles/test-case"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <link rel="selenium.base" href="http://test.sssprocess.com/" /> <title>09-SendOTP</title> </head> <body> <table cellpadding="1" cellspacing="1" border="1"> <thead> <tr><td rowspan="1" colspan="3">09-SendOTP</td></tr> </thead><tbody> <tr> <td>setTimeout</td> <td>180000</td> <td></td> </tr> <tr> <td>clickAndWait</td> <td>id=submitBtn</td> <td></td> </tr> </tbody></table> </body> </html>
04-Logout <?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head profile="http://selenium-ide.openqa.org/profiles/test-case"> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<link rel="selenium.base" href="http://test.sssprocess.com/" /> <title>04-Logout</title> </head> <body> <table cellpadding="1" cellspacing="1" border="1"> <thead> <tr><td rowspan="1" colspan="3">04-Logout</td></tr> </thead><tbody> <tr> <td>setTimeout</td> <td>180000</td> <td></td> </tr> <tr> <td>verifyText</td> <td>link=Log out</td> <td>Log out</td> </tr> <tr> <td>clickAndWait</td> <td>link=Log out</td> <td></td> </tr> </tbody></table> </body>
