Slide title minimum 48 pt Slide subtitle minimum 30 pt Tracing Framework Tracing and monitoring Framework (TMF) TCF Agent LTTng integration [email protected]

Tracing Framework

Tracing and monitoring Framework (TMF)TCF AgentLTTng integration

[email protected]

Slide title minimum 32 pt

(32 pt makes 2 rows)

Text and bullet level 1minimum 24 pt

Bullets level 2-5minimum 20 pt

›!"# $%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™−≤≥fifl

Do not add objects or text in the footer area © Ericsson AB 2009 | December 2009

Summary

› Tracing and Monitoring Framework (TMF)› TCF Agent› LTTng Integration› Questions







TMF Requirements (some!)

› Integration of heterogeneous tracing tools› Local and remote tracing tools› Live, concurrent trace streams› Traces that exceed available memory› Generic views› Generic correlation/analysis modules

› Customizable– Support for custom trace parsers– Easy integration of new views and analysis modules (plugins scheme)– Support for external, host-based libraries, analysis modules

› Eclipse based› Performance (what a surprise…)







TMF High-Level Functions

› Tool discovery

› Tool control

› Trace data retrieval

› Trace data visualization

› Trace data analysis







Tool Discovery

› Identify the available trace providers and their capabilities–Used to control tools generically

› Support for local and remote providers

› Integration of existing tracing/monitoring tools







Tool Control

› Control the tool operation–Configuration/Start/Stop/Pause/Resume/…–Conditional trace triggering

› Generic event filtering–By type, context, …

› Manage the resources allocated to tracing– Trace rate regulation (throttling)–CPU/Memory/Bandwidth/…–Per trace, trace type

› Persistence of control settings







Trace Data Retrieval

› Collect and store tracing/monitoring data– File transfer–Continuous stream–Multiple, concurrent, heterogeneous streams–Multiple targets

› Generic trace/log data interface–Support for log-specific parsers–Support for sequential, random access, archives, DB, …–Support for arbitrarily large files (bigger than available memory)







Trace Data Visualization

› Generic monitoring views–Events log (raw, tabular)– Time Line, Sequence Diagram, Logic Analyzer, Gantt Chart–Resource usage (CPU/Memory/Heap/Network/…)–Search filters, pattern matching, saved search queries, …

› Generic graphical widgets (trace agnostic)–Used to build custom views–Charts, histograms, …

› Extendable for application specific contents







Trace Data Analysis› Essentially a plugin-based integration facility

– After all, this is framework

› Generic analysis modules– Statistical modules

› Causal dependency analysis– Event dependency tree, Critical path– Correlation of event data (timestamp, events, …)– Reconstruction of event sequences from related traces– Execution replay

› Log comparisons– Regression testing– Health monitoring– Performance analysis

› Support for host-based, external analysis tools and libraries







TMF Perspective

CBA

Net

wo

rk

Eclipse Host

RSE

TCF

TMF

Application

TCFAgent

AppApp

App

CPP

TCFAgent

AppApp

App

AXE

TCFAgent

AppApp

App







TMF Architecture

TMF .

View A View B View C

View D

Trace 1 Trace 2

An

alys

is X

An

alysis Y

Analysis W

Analysis Z

Request/Event Handler

Experiment

ExternalComponent

ExternalComponent







TMF STatus Feature 0.1.0 0.2.0 1.0.0 2.0.0

Tool Discovery

Tool Control

Trace Data Retrieval

Trace Transfer

Streaming

Trace Data Visualization

Trace Data Analysis

Models

Generic Event Model

Generic Trace Handling Model

Analysis/View Integration Model

Performance Tuning

Bug Fixes







TMF 2.0 (and beyond)

› Trace Parser Wizard (possibly 1.0.0)› Application Wizard› Target Agent protocol (TCF based)› Heterogeneous trace support› Multi-trace timestamp synchronization› User space tracing (C/C++, Java?, Erlang?, …)› Generic statistical views› Generic widgets (more!)› Tracepoints› Source lookup (model?)› Support for “reactivity” (monitoring)

All this is subject to priority changes







Remote System Explorer

› Eclipse “Explorer of the Network Neighborhood”› Integrates any kind of heterogeneous remote resources

under a uniform UI› Pluggable subsystems and adapters–Remote file system –Remote shell access–Remote process handling–Standard adapters for SSH, FTP, …

› Zeroconf-based service discovery› Import/Export RSE profile› Terminal widget with pluggable serial/ssh/telnet connectors







Remote System Explorer

Subsystems manage resources of a particular kind

Filters select resources dynamically







Target Communication Framework

› The Problem–Many tools, each typically using its own agent and protocol– Lots of functional overlap:› How to communicate› How retrieve/model target objects› How to manipulate target› …

› The Solution–Use the same simple, lightweight end-to-end protocol–Allow value-adding services– Transport:› Standard TCP/IP on the client› Transport conversion by the value-adding service

–Auto-discovery of the value-adding services–Works with RSE (same Eclipse sub-project)







Target Communication Framework







LTTng TCF Agent

› Support for LTTng configuration and control

› Support for large binary file transfer–Contributed by Polytechnique–Already integrated in the TCF code base

Strong base for a generic TMF Trace Agent







LTTng Integration

› Scope– LTTv feature parity–Kernel tracing–User space tracing (for free~)–Re-design of the current State Model

› Proving ground for TMF– TMF architecture and models–Support for very large binary traces– Integration of external library (trace parser)– Tool Control– TCF Agent–…







LTTng Status Feature 0.1.0 0.2.0 1.0.0

Project View

Time Frame View

Events View

Control Flow View

Resource View

Statistics View

Integrated Help

Histogram View

Control View

View Synchronization

Multiple Traces (homogeneous)

Performance Tuning

Bug Fixes







LTTng Screenshots







Questions ?







References

Contacts› François Chouinard – [email protected]› Dominique Toupin – [email protected]› Marco Massé – [email protected]

Eclipse› Project: http://www.eclipse.org/linuxtools/projectPages/lttng› Wiki: http://wiki.eclipse.org/Linux_Tools_Project/LTTng

LTTng› http://lttng.org







Documents

Slide title minimum 48 pt Slide subtitle minimum 30 pt Tracing Framework Tracing and monitoring Framework (TMF) TCF Agent LTTng integration [email protected]