1. INTROCSOPE AGENT EXTENSIONS- CASE STUDIESINTROSCOPE USER GROUP MEETING18 OCT 2011STOCKHOLMJens Riboe18 Oct 2011jens.riboe@ribomation.com

2. Jens Riboe, Ribomation2Introscope Hacks ValtechSICS KTHRibomationERALondonGoyada Wily (San Francisco) VzB OW RM1990 1995 200020052010 3. Introscope3Introscope Hacks Wily Technology Professional Services 2005 - 2007 London CA-Wily Independent Consultant 2009 - Engagements Crit-sits Installations andcustomizations Developed numerousextensions Contributed to the core product 4. Introscope Experience4 Introscope Hacks Agent Extensions EPA Extensions Expression Tracer MySQL Monitor Dual Methods Tracer Large Logs Reader Large Interval Counter Tracer Introscope Solutions Smart Metrics (SunJVM) SIP Monitor EH Cache Tracer DataPower Monitor Certificate Status Tracer Agent Metrics Router JMX Agent Distributed JMS Message Auth PKITracker JMX Metrics RE Filter Iscope Feature Originator Agent Blocker Introscope clustering EM Extensions DoS Preventions MM Replicator XML TypeViews Shell Command Alert Handler EM/Agent Socket Factories 5. Introscope Customers5Introscope Hacks UK Europe HSBC, Sheffield Dekabank, Frankfurt Barclays, Manchester Air France, Nice Bank of America, Chester Sparebank1, Oslo Legal & General, London Eripol, Krakow Catlin, London ... Royal Bank of Scotland, Edinburgh Sweden British Telecom, Hemel Hempstead Frskringskassan, Sundsvall Atos Origin, Bristol Sandvik, Sandviken Vodafone, Stoke-on-Trent Telia IPTV, Gteborg ... Telia, Lule Systeam, Stockholm Tieto, Stockholm Bilprovningen, Stockholm 6. Besides Introscope6 Introscope Hacks Training C++ and Threads Programming Linux Systema Programming Android Programming Java Programming Groovy and Grails Programming Erlang Programming Seminars Android Programming Cloud Computing Distributed computation with Hadoop The Future of Java Groovy & Grails 7. Android Development7 Introscope Hacks Launched Drive-Journal Android app Open-Source Droid@Screen (GitHub) 8. Agenda8Introscope Hacks Ways to extend the AgentJVM Developing new Tracer TypesPBD Expression Tracer Large Interval Counter TracerJMX Dual Method Tracer MM ReplicatorGC 9. Extending the Agent 10. Many ways to extend an Agent10Introscope Hacks Out-of-the-box Changingthe standard PBD configuration Writing new PBDs Adding supported extensions Own development Usage of the Introscope Metrics API Implementation of a Tracer Type Implementation of an Agent Extensions 11. Introscope Metrics API11 Introscope Hacks For different kinds of non-standard metrics The agent contains a metric factory you can use in your own application to emit metrics based on application oriented data Classes using the Metrics API should not be instrumented via PBD 12. Sample usage of the Metrics API12 Introscope Hacksclass CacheReporter extends TimerTask { private IntAverageDataRecorder metric;{String name = "Fibonacci|Cache:Current Size";try {metric = DataRecorderFactory.createIntAverageDataRecorder(name);} catch (IllegalMetricNameException x) {System.err.println(x); System.exit(1);}}public void run() {metric.recordDataPoint( cache.size() );}} 13. Implementation of a Tracer Type 14. Tracer Type14Introscope Hacks Java code that runs before and after an instrumentation point Used to add different kind of metrics, than BlamePointTracer and other standard tracer types Steps Inheritfrom a tracer factory Create metrics accumulator(s) Do something before and/or after an instrumentation point Pack into JAR 15. Tracer type class skeleton15Introscope Hacks public class MyTracer extends ASingleInstanceTracerFactory { private IntegerAverageDataAccumulator metric; public MyTracer(IAgent agent, AttributeListing params, ProbeIdentification id, Object sample) { super(agent, params, id, sample); metric = getDataAccumulatorFactory().safeGetIntegerAverageDataAccumulator(name); . . . } public void ITracer_startTrace(int i, InvocationData data) { . . . } public void ITracer_finishTrace(int i, InvocationData data) { . . . int value = ; metric. IIntegerAggregatingDataAccumulator_recordDataPoint(value); } } 16. InvocationData16Introscope Hacks Contains all data about the instrumentation point Retrieve the target Object Method Methodargument objects Return value object 17. JAR Manifest17 Introscope Hacks Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven Built-By: jens Build-Jdk: 1.6.0_23 com-wily-Name: Large IntervalCounter Tracer com-wily-Release: 7.0 com-wily-Build: 202011 com-wily-Extension-Name: Large IntervalCounter Tracer com-wily-Extension-Type: Introscope-Agent com-wily-Extension-Version: 1 com-wily-Extension-Plugins-List: xyz com-wily-Extension-Plugin-xyz-Name: Large IntervalCounter Tracer com-wily-Extension-Plugin-xyz-Type: tracer com-wily-Extension-Plugin-xyz-Version: 1 com-wily-Extension-Plugin-xyz-Entry-Point-Class: com.ribomation.large_interval_counter.LargeIntervalCounterTracer 18. Some Agent/EM Extensions18Introscope Hacks Expression Tracer Large Interval Counter Tracer MM Replicator 19. Availability19Introscope Hacks Open Source http://blog.ribomation.com/introscope/ https://github.com/ribomation 20. Expression Tracer 21. ExpressionTracer21 Introscope Hacks Make the result of a reflective call into a metric stream Built upon OGNL Can invoke a Java expression on Any method of a target object Any parameter of a target method The returned value of a target method 22. Sample22 Introscope Hacks SetTracerClassMapping: ET com.wily.ps.expressiontracer.ExpressionTracer ... TraceOneMethodWithParametersOfClass: target.Application bizMethod ET "Tst:Number=this.getNumber()" TraceOneMethodWithParametersOfClass: target.Application bizMethid ET "Tst:Size=$1.size()" 23. Abilities23Introscope Hacks Target this= instrumented object itself $0= returned object from the method $n= n:th argument object of the method Metric types average sum Retained values counter last text Retained last value External OGNL expressions 24. PBD Sample24Introscope Hacks SetTracerClassMapping: ET_avgcom.ribomation.expression_tracer.ExpressionTracercom.wily.introscope.probebuilder.validate.MetricNameValidator SetTracerParameter: ET_avgmetricType average SetTracerClassMapping: ET_sum com.ribomation.expression_tracer.ExpressionTracer com.wily. SetTracerParameter: ET_sum metricType sum SetTracerClassMapping: ET_last com.ribomation.expression_tracer.ExpressionTracer com.wily. SetTracerParameter: ET_last metricType last SetTracerClassMapping: ET_text com.ribomation.expression_tracer.ExpressionTracer com.wily. SetTracerParameter: ET_text metricType text SetTracerClassMapping: ET_no_retain com.ribomation.expression_tracer.ExpressionTracer com.wily. SetTracerParameter: ET_no_retain metricType average SetTracerClassMapping: ET_retain com.ribomation.expression_tracer.ExpressionTracer com.wily. SetTracerParameter: ET_retain metricType average SetTracerParameter: ET_retain retainLastValue true TraceOneMethodWithParametersOfClass: target.Application computeET_avg "ExpressionTracer|Number:Average=this.getNumber()" TraceOneMethodWithParametersOfClass: target.Application computeET_sum "ExpressionTracer|Number:Sum=this.getNumber()" TraceOneMethodWithParametersOfClass: target.Application computeET_last "ExpressionTracer|Number:Last=this.getNumber()" TraceOneMethodWithParametersOfClass: target.Application computeET_avg "ExpressionTracer:List size (avg)=$1.size()" TraceOneMethodWithParametersOfClass: target.Application printET_text "ExpressionTracer:Message=$1" TraceOneMethodWithParametersOfClass: target.Application whatever ET_no_retain "ExpressionTracer:Whatever=$1.intValue()" TraceOneMethodWithParametersOfClass: target.Application whatever ET_retain "ExpressionTracer:Whatever retained=$1.intValue()" 25. Cert Issuer25 Introscope Hacks 26. Cert Issuer26 Introscope Hacks NordicEdge.pbd SetTracerClassMapping: ET com.wily.ps.expressiontracer.ExpressionTracer ... SetTracerParameter:ET metricType counter TraceOneMethodWithParametersOfClass: se.nordicedge.CertificateParsergetCATypeET "NordicEdge|Cert Issuer:Nordea=$0.startsWith("NORDEA")" TraceOneMethodWithParametersOfClass: se.nordicedge.CertificateParsergetCATypeET "NordicEdge|Cert Issuer:BankID=$0.startsWith("BIDT")" TraceOneMethodWithParametersOfClass: se.nordicedge.CertificateParsergetCATypeET "NordicEdge|Cert Issuer:Other=@otherIssuer"ExpressionTracer.propertiesotherIssuer=$0.(#issuer = #this, !(#issuer.startsWith("NORDEA") || #issuer.startsWith("BIDT"))) 27. Large Interval Counter Tracer 28. Large Interval Counter Tracer28Introscope Hacks Like Responses per Interval, but for much longer time periods 29. PBD sample29 Introscope HacksSetTracerClassMapping: LargeInterval_30s com.ribomation.large_interval_counter.LargeIntervalCounterTracer com.wily.introscope.probebuilder.validate.ResourceNameValidatorSetTracerOrdering:LargeInterval_30s-1000SetTracerParameter: LargeInterval_30s interval"30s"SetTracerParameter: LargeInterval_30s metricPattern"Calls / {0}"SetTracerClassMapping: LargeInterval_1m com.ribomation.large_interval_counter.LargeIntervalCounterTracer com.wily.introscope.probebuilder.validate.ResourceNameValidatorSetTracerParameter: LargeInterval_1m interval "1m"SetTracerClassMapping: LargeInterval_5m com.ribomation.large_interval_counter.LargeIntervalCounterTracer com.wily.introscope.probebuilder.validate.ResourceNameValidatorSetTracerParameter: LargeInterval_5m interval "5m"#TraceOneMethodOfClass: ""TraceOneMethodOfClass: test.Application doit LargeInterval_30s "Large Interval Counter|{packagename}|{classname}|{method}|30s"TraceOneMethodOfClass: test.Application doit LargeInterval_1m "Large Interval Counter|{packagename}|{classname}|{method}|1m"TraceOneMethodOfClass: test.Application doit LargeInterval_5m "Large Interval Counter|{packagename}|{classname}|{method}|5m" 30. Number of registrations30Introscope Hacks 31. Number of registrations31 Introscope HacksSetTracerClassMapping: GICT_1h com.wily.ps.generalized_interval_counter.GeneralizedIntervalCounterTracer com.wilySetTracerOrdering:GICT_1h-1000SetTracerParameter: GICT_1h interval "1h"# Half minute. Possible suffixes: h=hour, m=minute, s=second. Default: 1m#SetTracerParameter: GICT_1h type "array" # SlottedBuffer implementation. Possible values: "array", "list". Default: "array"#SetTracerParameter: GICT_1h metricPattern"Calls / {0}" # Metric name to the right of :. The unit will be patched in to {0}SetTracerClassMapping: GICT_24h com.wily.ps.generalized_interval_counter.GeneralizedIntervalCounterTracer com.wilySetTracerParameter: GICT_24h interval "24h"SetTracerClassMapping: GICT_5m com.wily.ps.generalized_interval_counter.GeneralizedIntervalCounterTracer com.wilySetTracerParameter: GICT_5m interval "5m"#TraceOneMethodOfClass: ""TraceOneMethodOfClass: stockholmstad.selfreg.StockholmStadSelfReg register GICT_5m "NordicEdge|{packagename}|{classname}|{method}|5m"TraceOneMethodOfClass: stockholmstad.selfreg.StockholmStadSelfReg register GICT_1h "NordicEdge|{packagename}|{classname}|{method}|1h"TraceOneMethodOfClass: stockholmstad.selfreg.StockholmStadSelfReg register GICT_24h "NordicEdge|{packagename}|{classname}|{method}|24h" 32. MM Replicator 33. MM Replicator33 Introscope Hacks Need to replicate a template MM for many applications Webapp running on the embedded Jetty Operation modes Manualusing WUI Automatic using CURL HTTP GET scripting 34. Create a Template MM34Introscope Hacks (1) Add @ marked keywords, for names, metric groups, alerts and more (2) Choose a MM template 35. Generate35Introscope Hacks (3) Provide values for all @ keywords (4) Generate a new hot deployed MM 36. Summary 37. Thanks & Goodbye37Introscope HacksJens Riboe+46 (0)730-314040jens.riboe@ribomation.comwww.ribomation.com blog.ribomation.com