601980-SQA Gomez DollarThrifty Webinar QandA

  • View
    213

  • Download
    1

Embed Size (px)

DESCRIPTION

gomez_app tools

Transcript

  • PorposalPPP

    Q:Gomezisstandlonewebapplicationtestingtool?GomezprovidesanondemandplatformthatyoucanuseforbothtestingandmonitoringyourWebapplicationsfromtheoutsideinacrossyourusers,browsers,mobiledevicesandgeographiesusingaglobalnetworkof100,000+locations,168+countriesand2,500+localISPs.RealityLoadisGomezswebloadandperformancetestingsolutionandyoucanreusethesamescriptsforbothloadtestingandproductionmonitoring

    Q:Whatisthedifferencebetweentheotherloadtestingtoolswhichenablesthewanemulation,locationbasedloadtestingandGomezloadtesting?AfewkeydifferencesarethescaleoftheGomezoffering,thenumberoflocationsthatyoucantestfromandthefactthatitisnotemulatingaWAN.Intermsofscale,theGomezLastMilenetworkincludes100,000+peerslocatedincountriesoneverycontinent.(RealityLoadrunsonrealusersmachinesonthelocalISPsandconnectionspeedsavailabletothem).TheGomeznetworkisalreadydeployedandrequiresnoconfigurationotherthantochoosewhichcomputers(wecalledthempeers)youwanttouseforloadtestingormonitoringpurposes..Thisgivesyouanunprecedentedviewintouserexperiencethatcannotbegainedbysimplyemulatingdifferentconfigurations.Ifyouareinterestedinhowyourrealuserswouldexperienceyoursiteunderloadyouneedtorealisticallytestunderthesameconditions,andtheGomezLastMileistheonlywaytodothat.ForexamplewithGomezLastMileyoucanfindgeographicalresponsetimediscrepanciesthatmaysurfaceonlyunderloadWhatwillbetheimpactonthebusinessifaverageresponsetimestaysunder4seconds(yourtargetgoals),butusersfromkeymarketsareseeingresponsetimeof10+seconds?

    Q:HowwouldweuseGomezfortheinternalapplications...nottheexternalapp?RealityLoad,andtherestoftheGomezplatform,isdesignedtotestandmonitoryourapplicationsacrosstheinternet,fromanoutsideincustomerperspective.Buttestinginternalapplicationsisacommonrequest.Atpresentthereareacoupleofoptions.OnewouldbetoallowtheGomeztestmachinesaccesstotheinternalsystem,eitherbyallowingrequestsfromourIPsorbyindentifyingouragentstring.AnotheroptioncurrentlyavailableforourMonitoringofferings,thoughnotourloadtesting,istodeployaGomezprivatepeerbehindthefirewallfromwhichtheperformancemeasurescouldbetaken.

    Q:AlsowhatwillbetheapproachofGomezwhiletestinganywebbasedapplicationrunningonamobilephone?FortestingmobileapplicationsGomezhas2complimentaryofferings,RealityLoad(discussedinourwebinar)andActiveMobileformonitoringmobilewebapplications.UserscouldrunhighvolumeloadtestusingRealityLoad,includingdevicespecificuseragentstringsandHTTPheaders,whileutilizingActiveMobileforthemostaccuratemeasureofenduserexperience.ActiveMobileemulatesdevicesacrossmajormobilecarriersintheUS(Chicago,Boston,Seattle),theUK,GermanyandChina,givinganaccurateviewoftheimpactoftheincreasingloadonthemobileuserexperience.

    Q:Sotheagentactuallymimicsbrowserbehavior,isnotactuallyrunningthescriptsinsidebrowsers?ThebrowsersusedbyGomezsRealityLoadarerealWebbrowsers.Thatmeansthattheyareactuallyrunningscriptsandprocessingthepages,notjustplayingbackHTTPtyperequests.

    Q:doesitsupportVirtualprivatenetworkusage?VPNsarenotsupportedforloadtestingatthistime.

    Gomez,Inc.|10MaguireRoad,Suite330|Lexington,MA024213110|Main:+1781.778.2700|Fax:+1781.778.2799

  • Q:Alsodoesthistoolmonitorsserversduringthetestrunandcorelatethebottlenecks?TodaymostofourRLcustomers(likeJimfromDollarThriftymentionedduringtheWebinar)areusingexistinginhousesystemmgmttoolstocorrelateperformanceinsideandoutsidethefirewallduringaloadtest.TheGomezplatformhasanopenintegrationphilosophy,andwewillcontinuetointegratewithanyEMStoolsthatyoumaybeusing.NowthatGomezisaCompuwareDivisionwearelookingintoofferinganintegratedviewofapplicationperformanceunderloadbyconnectingtheGomezplatformandCompuwareVantagebehindthefirewallinfrastructuremonitoringcapabilities,soyoucanquicklycorrelatebottlenecksandperformanceissuesacrossboththeenterpriseandtheInternet.

    Q:Doyouusea3rdpartyloadtesttoolbehindtheGOMEZRealityloadtool?SomethinglikeHPLR,RPToracombinationofit?TheGomezloadtestingtoolusesourowntoolset.ItusesthesamescriptrecorderandtestagentastherestoftheGomezPlatform.Thismakesitveryeasytomovebetweenloadtestingand24/7monitoringandviceversa.

    Q:CantherawdatabeusedfortheCompuWareLoadtestingtoolsets?TherawdataisexportedinCSVfiles,soitaccessibletoawiderangeoftoolsets.

    Q:Gomezsoftwareisopensource?Isthereatrialversion?itrunsinanyOSorwebexplorer?TheGomezplatformisaSaaSplatformaccessibleondemandthroughawebbrowserthatmeansthatyoudontneedtoallocatehardwareorinstallanysoftwareproducts,andrelieson.TheGomeznetwork100k+testinglocations,168+countries,2,500+ISPs.Itisnotopensource,butifyouareinterestedinatrialjustletusknowandwecansetyouup.

    Q:Onyourinterface,whenandwhywouldyoucheckthecheckboxes,ExcludeJS,CSSorImages?Thiswasafeaturerequestedbyanumberofourcustomers.Basically,itcanservetwopurposes,oneisaroundtestmethodologyandtheotheraroundcost.Methodologywise,wehavecustomerswhoprefertostarttheirtestingwithsimplesteps,focusingveryspecificallyonpiecesoftheirapplications.Thisfocusedapproachstartswithhittingjustthebasicpagesandnotanyoftheancillarycontentlikeimages,JS,CSSthatmaynotbeneeded.Heretheyarefocusingjustontheimpactoftherequestsfortheirapplicationpagesontheserverunderload.Thesecondreasoniscost.ManyofourcustomersuseCDNsthatchargebytheamountofcontentserved.Whenrunningalotofloadteststhecostscangetprettyhigh.Sotheywillexcludetheobjectsfromtheirearlytestsandonlyincludethemforafinaltestortwo.Also,justwantedtohighlightthatweseealotofissueswiththirdpartyprovidersandexternalcomponentsperformingpoorly,especiallyunderload,andregionaldiscrepanciesaswell,sodontforgettoincludetestcasesthatcoverexternalcomponentsandgeographicaltestingintoyourplansWealsoallowuserstheoptiontoexcluderequestsbyhost.

    Q:Areyoutakingcareofthatpagesmaybecachedinatransaction?Ifnot,canyoubesurethatthefrontendisnotgettingtoomuchload?Ourloadtesttoolperformsasarealusersbrowserwouldperformforeachtransaction.Soifimagesarecachedforinstance,eachiterationofeachuserrunwouldrequesttheobjectonce,butnotagainwithinatransaction.

    2

  • Q:IfyouaretestinginthePRODUCTIONenvironmentduringoffpeakhours,youstillcannotknowhowmuchUSERtrafficishittingyoursite.Soyoucannotisolateyourresultstoyourtest.HowdoyouaccountforUNKNOWNloadduringyourtest?Thisisdefinitelyaconcernfortestingofproductionsystems.Hereitisimportanttomonitortheutilizationoftheproductionsystempriortotheloadtest,duringtheloadtestandaftertheloadtest.Thiscangiveyouaprofileofthelivetrafficandassistinunderstandingtheutilizationontheservers.Also,bymonitoringthingslikefirewallbandwidth(atthenetworklevel),andcomparingtheoveralltotalwiththeloadreportedforthetestusers(fromthetesttool)theloadfromliveuserscanbeextrapolated.Forinstance,ifRealityLoadwasshowingthatthetestwasconsuming80Megabitspersecondofbandwidth,andthefirewallstatisticswereshowing90Megabitspersecondofoveralltraffic,thenyoucangetasenseofhowmuchlivetrafficwasonthesite(inthiscase,10mbpsworthoftraffic).Thesamecouldbesaidforpagerequestratesorothermeasureswiththedifferencebetweentheloadtoolmeasuresandthebasicsystemmeasuresprovidingthesenseoflivetraffic.Thebestwaytoensurequalityofexperiencesandthatanapplicationscalesproperlyunderloadistorealisticallytestyourapplicationalltheelementsofthewebapplicationdeliverychainfromtheoutsidein,usingrealusersanddesktopsworldwidetoaccessyourproductionenvironment

    Q:Howdoyoucalculatetransactionspersecond?OneoftheRealityLoadmetricsistransactionsperminute,Iassumethisiswhatthequestionrefersto.Thisiscalculatedasthesumofthetransactions(completerunsofthetestscripts)completeineachminute.

    Q:Doyouhelpcustomersidentifyingworkloadprofile,ifyeswhatisyourapproach?Mostofourcustomersoperateinaselfservicemodelandutilizetheirowntestmethodologies.Howeverwedooffercollateralaroundbestpracticesforthingslikeworkloadprofilingaswellasofferingprofessionalservicesassistancewhereneededorrequested..Ingeneralthough,theapproachistoevaluatethetrafficthewebapplicationiscurrentlyseeingwithfocusonthreekeyareas:themostheavilyhitpages/functionality,themostbusinesscriticalpages/functionalityandanypages/functionsknowntocauseproblems.

    Q:Canyouterminateatestrunwhenthingsstarttobreak?Doestheinitiatorofthetesthaveaccesstothetestadminconsole?Cantheyterminateatestinatimelywaytoavoidcrashingtheapplication?TheGomeztoolisaselfserviceplatform,sousershaveaccesstothecontrolstoscheduleatest,stopit,viewrealtimeresults,etc.Testscanbestoppedwithinafewsecondsjustbyclickingastopbutton.Inaddition,RealityhasaMaxFailureRatesettings(RealityLoadchecksthatthe%ofusersencounteringerrorsatanygivenminuteremainsbelowtheMaxFailureRatevalue)anditcanautomaticallyreducetheloadlevellookingforessentiallyalastknowngoodstateifneeded.

    Q:We'vegotastagingenvironmentthatmatchesproductionandithasawebserverintheDMZandisaccessiblefromtheInternet.WeuseLoadrunnertotestthesystem.WhatistheadvantageofusingGomez?Traditionalinsidethefirewalltestingtools,likeLoadRunner,arebuiltonthephilosophyofgeneratingloadandmeasuringperformancebehindthefirewall.Thisapproachisvaluable,butwillonlytellyoupartofthestory,sincethesetoolsarefocusedontestinginternalsystems.However,yourendusersdon'tliveindatacenterstheyarelocatedattheendofalongandcomplexwebapplicationdeliverychain.Whenyougenerateloadandmeasureperformancebehind the firewallyouhavenovisibility intoenduserexperience,orhow3rdparty

    3

  • components (ISPs, CDNS, etc) will scale under load. And all components and services need to performoptimallyinordertodeliverqualitywebexperiencesacrossallusersandregions.Someexamplesofproblemsthatyoucouldbemissingare:Inconsistentgeoperformanceespeciallyunderload,CDNconfigurationissuesoroversubscribedPOPs,majorISPsoutages,SMSrouting,latencyissues,etc

    Q:Didyousupportflexapplications?Yes,Gomezsupportsflexapplications.

    Q:Whodoesthescripting?Docustomerswriteownscriptsordoyouwritethescripts/scenariosforthecustomer?SpecificallyscriptingwebservicesTheGomezPlatform,includingRealityLoadisaselfserviceplatform,andsomostofourcustomersarebuildingtheirownscriptsforloadtestingandmonitoring.How