Testing Web Applications Willem Visser RW334. Overview Testing Theory Testing Web Apps Tools.

  • Published on

  • View

  • Download

Embed Size (px)


  • Testing Web ApplicationsWillem Visser RW334

  • OverviewTesting TheoryTesting Web AppsTools

  • The BasicsA test consists ofInputExpected output (also called the Oracle) White Box TestingConsidering the code when coming up with testsBlack Box TestingOnly consider the input-output interface Automated Testing is the Holy Grail

  • Types of TestingUnit TestingIntegration Testing Systems TestingRegression TestingAcceptance TestingAlpha/Beta TestingUsability TestingPerformance TestingRobustness TestingSecurity Testing

  • Lower Level Code TestingUnit TestingTest individual components of the codeCan be as little as one functionWhite boxIntegration TestingInterface between componentsWhite BoxCan be as much as the whole system (which case it becomes more black box)Code Coverage as Test Adequacy Measure

  • Functional TestingTesting that the systems meets the requirementsSystems TestingIn the deployed environmentUnlike Integration testingAlways black box

  • Cross CuttingRegression TestingWhen code changes (some) old tests must still passCould be at unit, integration or systems levelVery high cost involved in regression testingRegression failure has high cost to fix

  • Customer FacingAcceptance TestingIs the customer happy with the product?Alpha/Beta TestingLet the system out to a few of your customers and see how they feel about itUsability TestingTypically for GUI/Web/Mobile to not just check that the system is correct but also easy to useHarder to create an Oracle

  • Non-Functional TestingPerformanceLoadSee how the system behaves at peak loadStressPush the system beyond its limits to see how far it will surviveSecurity TestingCheck whether there are vulnerabilities that might lead to loss of privacy or other security issues

  • Web App TestingFull Gambit of FeaturesFront End GUIUsability issuesHard to testBrowser Compatibility issuesServer Side (including storage)Performance issuesSecurity issues

  • Unit TestingUtterly important!Finding bugs early saves moneyMakes regression testing much more effectiveWrite once, run oftenWhat do you test?Business logic!No need to test simple code or interactions with 3rd party librariesUntil they fail of course!Use Coverage tools to help you decide if you have tested enough

  • Stubs or MocksIn unit testing you are interested in local behavior and assume other things you might be using behave correctlyMost unit testing frameworks provides stubs for these 3rd party componentsGood example is datastore and memcache stubs provided by GAEOnly problem is that sometimes these stubs dont respect the behavior of the real thing!

  • Unit Testing FrameworksJava JUnitThe most famous of them allPHP PHPUnitPython PyUnitActually just import unittest And many more XUnit tools for language XFor example GAEUnit, but it looks dead nowUse coverage tools along side, examplesCoverage.py for Python(Ecl)Emma for (Eclipse) Java

  • Unit Testing and GUIsWhen doing Unit Testing you try and stay well clear of the GUIClicking and entering text through a GUI is not automated, although we will see later that with Replay Technology it can also be doneIsolating the GUI to allow more efficient testing is the reason to use Model-View-Presenter rather than Model-View-Controller design pattern

  • Webapp2 exampleimport unittestimport webapp2

    # from the app main.pyimport main

    class TestHandlers(unittest.TestCase): def test_hello(self): #input request = webapp2.Request.blank('/) #output response = request.get_response(main.app) #oracle self.assertEqual(response.status_int, 200) self.assertEqual(response.body, 'Hello, world!)import webapp2

    class HelloHandler(webapp2.RequestHandler): def get(self): self.response.write('Hello, world!')

    app = webapp2.WSGIapplication([('/', HelloHandler)])Code to TestTest

  • Integration TestingThin line between unit and integrationStrictly speaking when more than one component is used you are doing integration testingFor example if your web app uses a datastore then the test on the previous slide that came in via a GET request could be an integration test not a unit testThis is not worth worrying aboutUnit/Integration Testing is fine

  • System TestingNow the full round-trip is being tested, including the Browser componentUnit/Integration Testing can be done on a local environment, but System Testing need to be in the deployed environmentState-of-the-PracticeRecord and replay tests

  • Seleniumhttp://docs.seleniumhq.org/The #1 tool in Web System Testingautomates browsers. Thats it!Much like the webapp2 example earlier, just many more options, including browser specific driversIDE for record and replayWith scripts that can be editedWebDriver that allows one to run without a browser at all

  • Selenium Examplefrom selenium import webdriverfrom selenium.common.exceptions import TimeoutExceptionfrom selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

    # Create a new instance of the Firefox driverdriver = webdriver.Firefox()

    # go to the google home pagedriver.get("http://www.google.com")

    # find the element that's name attribute is q (the google search box)inputElement = driver.find_element_by_name("q")

    # type in the searchinputElement.send_keys("cheese!")


    try: # we have to wait for the page to refresh, the last thing that seems to be updated is the title WebDriverWait(driver, 10).until(EC.title_contains("cheese!"))

    # You should see "cheese! - Google Search" print driver.title

    finally: driver.quit()

  • Java Exampleimport com.thoughtworks.selenium.*;// This is the driver's import. You'll use this for instantiating a// browser and making it do what you need.

    public class NewTest extends SeleneseTestCase {

    public void setUp() throws Exception { setUp("http://www.google.com/", "*firefox"); }

    public void testNew() throws Exception { selenium.open("/"); selenium.type("q", "selenium rc"); selenium.click("btnG"); selenium.waitForPageToLoad("30000"); assertTrue(selenium.isTextPresent("Results * for selenium rc")); }}

  • Usability TestingBased on user opinionsManual according to a list of tasksObserver records behaviorCould use things like eye-tracking for more precise resultsAutomated Usability Testing is still a research topic

  • Robustness TestingTry anything and see if something breaksMost famous of these is Android MonkeySends random keystrokes to your Android appgremlins.js does the same thing for web apps


  • Performance TestingSystem performance under specific loadsConcurrent users doing a certain number of transactions for a certain durationThis is a very hard kind of testing to doNeeds lots of infrastructure Often the performance bottleneck is your testing framework and not the system under testIn the end we know where the problem isTHE DATABASE!More generally where something has to wait for something else to finish

  • Web Performance TestingMeasure throughput or transaction rateServer response timeRendering, but that might need additional scripts on the client sideYou might not know what is the expected performance, so often you profile the performanceYou will quickly notice bad performance

  • Web front-end Performance ToolsGoogle Pagespeed toolswww.webpagetest.orgGoogle Chrome Developer ToolsNice summary of tools related to performance can be found at http://samsaffron.com/archive/2012/03/23/sam-s-ultimate-web-performance-tools-and-resources

  • JMeterLoad and Performance TestingServer sideNot a browser, but can simulate some actions typically done by a browserHTTP, etc.No javascript execution

  • Security TestingThis is HUGE! A whole course or even degree can be devoted to itNice checklist of things to do at https://www.owasp.org/index.php/Web_Application_Security_Testing_Cheat_SheetLarge part of security testing is manualPenetration TestingFind security vulnerabilities that can be exploited Tools to find buffer overflows is one of the most effective automatic approachesFuzz TestingSemi-structured random testing

  • Zed Attack Proxy (ZAP)Automated Penetration TestingPoint it to URL and it does the restIncludes Fuzzing and many more Find it here https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project



View more >