DynaTraceWhitepaper - Go Beyond Profiling

8/13/2019 DynaTraceWhitepaper - Go Beyond Profiling

1/13

On the PurePath Go Beyond ProfilingTracing transactions across distributed Java/.NETapplication components at production-safe overhead


2/13

>> rapidly resolve performance problems in distributed java & .net applications

ContentsContentsContentsContents

EXECUTIVE SUMMARY..................................................................................................... 1THE CHALLENGE OF SOFTWARE PERFORMANCE............................................................. 2SOFTWARE PERFORMANCE ANALYSIS ............................................................................. 2OVERHEAD VS. QUALITY AND PRECISION OF MEASUREMENT.......................................... 3TRANSACTIONAL PERFORMANCE DATA COLLECTION..................................................... 4DIAGNOSING PERFORMANCE PROBLEMS IN 7X24 ENVIRONMENTS................................ 6CAPTURING OF DIAGNOSTIC INFORMATION................................................................. 6

ANALYSIS OF DISTRIBUTED ENVIRONMENTS.................................................................... 7DYNATRACE DIAGNOSTICS EDITIONS ............................................................................ 8COMPARISON MATRIX.................................................................................................... 9CONCLUSION ............................................................................................................. 10


3/13

On the PurePath Go Beyond Profiling 1

ExecutiveExecutiveExecutiveExecutive SummarySummarySummarySummary

Modern business runs on the mission-critical applications implemented and sustained by the IT

organization. Efficient automation of business processes is no longer a luxury it is a necessityborne out of a fiercely competitive global marketplace. Business performance is often directlytied to software performance, and unplanned outages or poor application performance canresult in the wrong kind of front page headlines. Application performance analysis is acomplicated, far-reaching exercise that touches many organizations and processes, and requiresthe proper tools and best practices to guarantee consistent service delivery.

One tool commonly utilized as part of this process is a system or application profiler, whichrelies on a number of assumptions that for many situations actually render it inadequate:

The collection of simple statistical performance values may mask systemic performanceproblems and often makes root cause isolation difficult.

Problematic end-user transactions cannot be fully reconstructed from profilinginformation.

High execution overhead and the lack of long-term monitoring capabilities meanprofilers are not suitable for diagnosing problems in production environments.

The high number of influential environmental factors, however, requires diagnosis in production.Missing information beyond basic execution times makes the isolation of context-specificperformance problems nearly impossible. Additionally, the lack of support for diagnosis indistributed, heterogeneous .NET and Java applications makes profilers unsuitable for thediagnosis of performance problems in typical enterprise environments.

dynaTrace Diagnosticssteps in to fulfill all the requirements which profilers miss. dynaTracesPurePath Technologyenables the collection of transactional performance metrics and related

context information. Tracing across technology and server boundaries make it an enterprise levelperformance management solution. Low overhead and hot reconfiguration at runtime enable7x24 diagnosis even in production environments.


4/13


The Challenge of SoftwarePerformanceFulfillment of functional and performance requirements are essential to softwarequality. Both are equal in their relevance to service delivery. Today, mission criticalapplications rely on functional richness as much as on adequate performance.Poorly performing software can potentially cause massive losses in revenue and putyour business at risk. Ensuring software performance is a very challenging task, evenmore challenging than providing the correct functionality, because it depends onmany variables:

Bad code can cause poor algorithmic performance. User behavior and load volume directly impact response times and

resource usage.

Improper configuration of application servers or frameworks may degradeperformance significantly.

Inadequate distribution of software components can introduce bottlenecks.Many of these risk factors are specific to the underlying environment and can onlybe diagnosed in that specific setting, under specific circumstances. Performanceengineers and developers find themselves increasingly confronted with performanceproblems that are difficult or impossible to diagnose. The increasing usage of third-party code and highly distributed software environments adds additional complexity.

Software Performance AnalysisSoftware Performance Analysis ideally adopts a systematic approach that supportsthe rapid isolation, diagnosis and resolution of performance problems in

applications. The most widely used approach for software performance analysis isto sample the execution characteristics of code and collect execution time metrics toprovide statistical data on the performance of individual methods. This is theapproach taken by code profilers. Profilers, which are today among the most widelyused tools for performance analysis, rely on a number of assumptions:

They collect only statistical performance information, assuming thatperformance is independent of the actual user transaction context and flow.

They can significantly alter the behavior of an application in order to collectperformance metrics, assuming that there is no impact on the quality of thecollected measurement (see Figure 1).

They only collect CPU and execution time metrics, assuming that thisinformation is sufficient for identifying performance bottleneck.

They restrict information collection to a single runtime environment (either.NET CLR or Java VM), assuming that performance problems are limitedonly to single tiers of an application even in distributed Java Enterprise orSOA environments.

These assumptions however do not reflect reality for most software applications.Consequently effective performance analysis cannot be provided by such toolsalone.

Sustaining adequate

softwareperformance is adaunting task.

Increasingcomplexity makesperformanceproblems more andmore difficult todiagnose.

Profilers rely on

assumptions that donot hold true inmodern environments


5/13


We will therefore introduce dynaTrace Diagnosticsas a best-of-breed solution forsoftware performance management and diagnosis. dynaTrace Diagnostics providesfunctionality that goes far beyond the capabilities of profilers and thus enables anew level of performance analysis. The following chapters will address all majoraspects of software performance analysis and illustrate dynaTrace Diagnostics

added value.

Overhead vs. Quality and Precision ofMeasurementThe collection of application performance information requires additional code tobe injected into the application. Common injection techniques for Java include theJVM Profiling Interfaces or byte code instrumentation. All these approaches affectsystem performance by adding overhead (see Figure 1).

Uninstrumented Application Instrumented Application

Execution Timeof Diagnosis Code

Figure 1 - Added overhead for performance metric collection

The imposed overhead however is detrimental to the quality of that measurement.The more overhead created by the diagnosis tool the more inaccurate themeasurement inherently becomes. Inadequate collection of performancemeasurements can even lead to the introduction of phantom performanceproblems. Some problems, such as synchronization issues for example, may even

not be detectable as real world application behavior varies too widely from theoriginal system.

Profilers provide only limited support for the adjustment of measurement granularityin order to mitigate and control overhead. Most profilers collect performanceinformation for each and every method that is executed. Others provide the abilityto switch between a set of predefined granularity levels, while some have the abilityto define exclusion rules at code level. However they all lack necessary features tofully support performance engineers who require fully flexible and fine-grained

Profiling overheadinfluences thequality andprecision ofmeasurement.


6/13


configuration of performance data collection in order to minimize overhead whilemaintaining all relevant diagnosis information.

dynaTrace Diagnostics provides powerful, revolutionary features for theconfiguration of measurement collection. An intuitive class browser can be used tonavigate packages, classes and methods in order to easily define flexible data

collection configurations. Interactive, UI-based refinement of configuration and hotdeployment without application restart adds additional convenience to theconfiguration process. Knowledge Sensor Packs allow the collection of relevantperformance metrics for standard technologies like Java EE, ASP .NET, Spring orHibernate (as well as many others) with no manual configuration effort.

Transactional Performance DataCollectionThe ideal starting point for an in-depth analysis of application performance is a

close examination of method execution times within critical user transactions.Performance management tools must strike a delicate balance between thegranularity of collected performance information, which can range from recordingevery single method execution to recording only values aggregated over severalseconds, and the overhead penalty required to collect it.

Profilers present performance measurements only as aggregated values over acouple of seconds. This can easily lead to situations where performance problemsare hidden especially in the case of statistical outliers. Moreover, collectingaverage values creates additional overhead above that required for collection ofraw performance information. As only aggregated values are collected it is notpossible to identify how many transactions experienced problems or to analyze theexact call tree of a problematic transaction. Analysis based on aggregated valuescan easily leave performance problems undiscovered. Figure 2 shows the sampleoutput of a profiler for a portal-based web application using a generic requesthandler.

Count Time s (Total) Time s (Avg)

handleRequest () 750 9500 12.5

buildWebControls () 750 7650 10.2

loadConfiguration () 250 250 1

renderPage () 750 1500 2

Figure 2 - Sample Profiler Output

dynaTrace Diagnostics uses its innovative PurePath Technology to collecttransactional performance metrics. PurePath Technology enables the tracing ofeach individual transaction which has been executed within a system. Performanceengineers get detailed, precise execution information including the complete calltree the PurePath. By recording each individual transaction, outliers can beidentified easily even if there is only a single bad transaction within 100,000executions.

Statistical performancemetrics produced byprofilers often leaveperformance problemsundetected.

PurePath Technologyprovides transactionalperformance metricsenabling betterdiagnosis and outlierisolation


7/13


Figure 3 shows the simplified output for a portal application collected usingPurePath Technology. By capturing all argument values in addition to detailedexecution time metrics for each individual transaction, the performance problemwhich cannot even been seen in the profiler is immediately visible.

Argument Time s

... 500 of type

handleRequest () "start" 9

buildWebControls () 7

renderPage () 2

... 200 of type

handleRequest () "overview" 12

loadConfiguration () 1



renderPage () 2

... 50 of type

handleRequest () "show all details" 50

loadConfiguration () 1


buildWebControls () 10buildWebControls () 10



renderPage () 3

Figure 3 - PurePath based Execution Information


8/13


Diagnosing Performance Problems in7x24 EnvironmentsPerformance problems, unlike many functional problems, can only be analyzed bytaking the runtime environment into consideration. In previous sections wevedescribed the factors influencing software performance, many of which areenvironment-specific. This is in large part the reason why developers are oftenunable to accurately reconstruct performance problems occurring in productionenvironments within their local workstation environment. They require detailedperformance data from the production system in order to perform the necessaryperformance analysis. Without this critical information they must expend significanttime and effort in attempting to reconstruct the problem, or in the worst casescenario they are left to guess what the problem might be and try to fix it in alengthy trial and error process.

The significant performance overhead (10x to 1000x slower) introduced by profilersmake their usage untenable in production environments. Installing a profiler in a

live production environment would result in immediate and significant performancedegradation and likely multiple SLA breaches. Also, as already mentioned,performance problems may occur only under very specific circumstances orconditions. This requires a performance diagnostics tool to run in a productionsystem for hours or even days to ensure capture of these transient, specificproblems. Profilers are designed for only short-term data collection over a couple ofminutes.

In contrast, dynaTrace Diagnostics employs lightweight Diagnostics Agents, whichonly collect performance metrics. Correlation of measurements and execution pathreconstruction is performed by the dynaTrace Diagnostics Server. Performing allresource intensive tasks outside the diagnosed system means that performance

overhead is sustained at typically 3-5%, making dynaTrace Diagnostics ideallysuited to usage within load tests and production environments

Capturing of Diagnostic InformationPractical experience shows that capturing performance metrics is only sufficient foridentifyingapplication components that have performance problems. Exact problemdiagnosis, however, also requires gathering context-specific information. This isespecially true for applications that use customizable frameworks (e.g., portals,Spring), usage of prepared database statements, generic argument-based basedframeworks or Aspect Oriented Programming (AOP) frameworks impose additionalcomplexity to diagnosis process, because their exact execution depends on specificinvocation details(see Figure 3).

Profilers only provide statistical execution time metrics. Therefore they cannotcorrelate additional context information to statistical performance metrics. As aconsequence they fail to analyze performance problems that only occur in a specificscenario, which cannot be distinguished at the plain code level.

Using PurePath Technology, dynaTrace Diagnostics can correlate contextinformation to individual execution paths. Knowledge Sensors allow capturing ofmethod arguments, SQL bind parameters, exceptions, etc. This information is put

Performance Problemscan very often only beanalyzed in productionas they are highlydependent upon theruntime environment.

While profilers intro-duce far too muchoverhead forproduction diagnosis,dynaTrace Diagnosticscan be used to findproblem in 7x24productionenvironments.

Effective root causeanalysis requirescapturing of diagnosticinformation beyondexecution times.


9/13


into to context of a specific transaction allowing the performance engineer ordeveloper to pin down a context-dependent performance problem quickly andeasily.

Analysis of Distributed EnvironmentsTodays application environments have become more and more distributed. Trendslike SOA and EAI create systems that consist of a complex server infrastructure withservices distributed across multiple servers implemented in different technologies.Performance diagnosis therefore requires the ability to follow transactions overserver boundaries, irrespective of whether communication is handled usinglanguage-specific remoting protocols, Web Services or messaging systems.

Profilers are designed to be used in single runtime developer environments. They donot provide any ability to diagnose performance problems in distributedenvironments. The diagnosis of problems in heterogeneous system is alsoimpossible as these tools are limited to either Java or .NET.

Figure 4 Heterogeneous Transactional Diagnosis using PurePath

dynaTrace Diagnostics supports the diagnosis of distributed Java and .NETenvironments. Transactions can be followed across server boundaries. Tracing from.NET to Java and vice versa enables identifying performance problems inheterogeneous hybrid application environments. dynaTrace Diagnostics serves asthe single solution for your company to analyze performance problem in enterpriseenvironments as shown in Figure 4. With just one tool you can investigate yourperformance problems on every platform. Not having to learn multiple toolsprovides an immediate ROI of your tool investment.

Enterpriseenvironments requiretransaction tracingacross systemboundaries and acrosstechnology domains.


10/13


dynaTrace Diagnostics EditionsdynaTrace Diagnostics is offered in several product editions, each providingfeatures for specific use cases. As dynaTrace Diagnostics targets the entireapplication lifecycle, supporting different requirements and usage scenarios helps

differentiate each specific edition. Figure 5 provides an overview of the differenteditions and their target audience.

Figure 5 - dynaTrace Diagnostics Editions scaling with your requirements

dynaTrace DiagnosticsdynaTrace DiagnosticsdynaTrace DiagnosticsdynaTrace Diagnostics Workstation EditionWorkstation EditionWorkstation EditionWorkstation Edition is used for performance analysis duringcode development. Developers immediately understand the dynamics andperformance implications of their code. Development teams rely on dynaTracedynaTracedynaTracedynaTraceDiagnostics ProfessionalDiagnostics ProfessionalDiagnostics ProfessionalDiagnostics Professional EditionEditionEditionEdition to store configurations and Diagnostics Sessions ona central Diagnostics Server improving communication and speeding performanceproblem resolution. Automated build support enables integrating performance

testing in continuous integration and automatic build environments. dynaTracedynaTracedynaTracedynaTraceDiagnosticsDiagnosticsDiagnosticsDiagnostics Professional andProfessional andProfessional andProfessional and EnterpriseEnterpriseEnterpriseEnterprise Edition bothEdition bothEdition bothEdition both enable monitoring anddiagnosis in 24x7 production environments as well as testing and stagingenvironments. Performance engineers and developers can reconstruct performanceissues in production environments making reproduction in testing and stagingenvironments obsolete.

dynaTrace DiagnosticsEditions are tailored tosupport performanceanalysis across thewhole lifecycle.dynaTrace Diagnosticsscales up with youranalysis demands


11/13


Comparison MatrixdynaTrace Diagnostics fulfills or surpasses all the requirements for modernapplication performance diagnostics. The capabilities go far beyond those ofProfilers. The following matrix lists the key capabilities required for efficientperformance diagnosis and how they are supported by Profilers and dynaTraceDiagnostics.

Key Performance Analysis Capabilities Profiler dynaTraceDiagnostics

Easy installation and integration into development andtesting environments; requires no administrationoverhead

Yes Yes

Code level profiling details providing the informationdevelopers require to understand performance problems

Yes Yes

Configuration of profiling rules to limit overhead andrestrict data collection to specific code paths

Limited1 Yes1

Centralized Diagnostics Server for storing and managingconfiguration and monitoring results to supportdevelopment teams

No Yes

Predefined configurations for standard technologies likeJava EE, ASP.NET, etc. to automatically discoverrelevant application components No Yes

Performance metrics for discrete transactions; permitsidentification of outliners

No Yes

Transaction tracing unveils the actual call tree showing

the exact transaction flow

No Yes

Capture contextual diagnosis information (e.g., methodarguments, exceptions, SQL calls, etc.) to accelerateroot cause identification and error resolution

No Yes

Support tracing across technology and physicalboundaries enabling the diagnosis of distributedenterprise applications

No Yes

1Profilers limit configuration to certain level or use configuration files. dynaTrace Diagnostics provides intuitivevisual configuration


12/13


ConclusionTodays mission critical business processes rely on high performance software. Performance problemshowever are much more difficult to find and fix than functional errors. They generally depend on usagecharacteristics or environmental influences. Increasingly complex and heterogeneous enterprise applicationsrepresent an additional challenge to the developer or performance engineer.

Profilers have traditionally been the weapon of choice for analyzing performance problems. However theylack important features required for performance analysis in todays complex application environments.dynaTrace Diagnostics provides a unique set of capabilities, which exceeds profilers by far allowingdevelopers, architects and performance engineers to:

Quickly isolate performance outliers within a population of thousands of transactions Diagnose the root cause of performance problems immediately by analyzing the exact transaction

flow and contextual information

Improve team communication by cooperating and coordinating via a centralized dynaTraceDiagnostic Server

Diagnose performance problems in 7x24 production environments Diagnose performance problems in heterogeneous enterprise environments Trace live transactions across server boundaries to support even large clustered distributed

application environments

dynaTrace Diagnostics PurePath Technology provides the capabilities requiredfor in-depth performance analysis in todays complex heterogeneous

applications making it the first choice for solving performance problems

We invite you to learn more at http://www.dynaTrace.com.


13/13


HeadquarterHeadquarterHeadquarterHeadquarterssss EMEA: dynaTrace software GmbHEMEA: dynaTrace software GmbHEMEA: dynaTrace software GmbHEMEA: dynaTrace software GmbH

Freistdter Str. 313, 4040 Linz, Austria/Europe, T+ 43 (732) 908208, F +43 (732) 210100.008

HeadquarterHeadquarterHeadquarterHeadquarterssss North America: dynaTrace softwarNorth America: dynaTrace softwarNorth America: dynaTrace softwarNorth America: dynaTrace software Inc.e Inc.e Inc.e Inc.

West Street 200, Waltham, MA 02451, USA, T +1 (339) 9330317 F +1 (781) 2075365

E: [email protected]

All rights reservedAll rights reservedAll rights reservedAll rights reserved

dynaTrace software is a registered trademark of dynaTrace software GmbH. All other marks and names

Documents

DynaTraceWhitepaper - Go Beyond Profiling