DTrace and Drupal

session code S308357

Angelo RajaduraiSr. Technology Evangelist – Sun Microsystems Inc.

Observing Your App and Everything Else It Runs On Using DTrace

Sunday, May 31, 2009

2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone

Learn how to Dynamically observe your application and its environment.

goal


2009 CommunityOne Conference: EAST | developers.sun.com/events/communityone 3

your app...

Native Apps

ruby

C C++

Database MySQL/Oracle/Postgres

Operating System Solaris/Linux/OS X/Windows

Container Glassfish/J2EE

Frameworks

Application Logic

PresentationJSP/Javascript/Ajax/JavaFX

python

phpperl

shell

Apps run in VM

java


Amp Stack


4

OpenSolaris 2009.06

MySQL 5.1.30

Apache 2.2

Drupal 6.12

PHP - Zend 5.2

Our Setupon the menu today



the AMP stackTo build, install, integrate, performance tune and deploy

the amp stack on opensolaris...

# pkg install amp-dev

5

one small easy step



one slide on Drupal



How much time am I spending in Apache/

MySQL/PHP?How much of my load is from anonymous users

What type of database load is caused by anonymous users

What are the Drupal modules that take up

the most time?

What are the most expensive hooks in a given Drupal module?

questions questions questions



the big game hunt

Devel

oper:

“I’d as

k the

datab

ase ad

min” Database Admin:

“May be the middleware guys

will know”

Sysadmin

“I don’t know! Ask the

Developer” Middlew

are Ex

pert:

“Lets

ask th

e Sysa

dmin”



the vicious cycle

Instrument Validate

Run in production

Collect infoProcess

Add Code Compile

Custom Instruments

Stop App Restart w/ flags

Prebuilt Instruments



Please let me observe a live system.

oh please



Native

cc++ph

perl pyth

o

rub

shellDatabase

Operating System Solaris/Linux/OS X/

Container Glassfish/Tomcat

Frameworks

Application Logic

PresentationJSP/Javascript/Ajax

dynamic tracing

12::SELECT id, name, symbol, conversion_rate FROM currencies WHEREstatus = 'Active' and deleted = 012::SELECT category, name, value FROM config12::SELECT id FROM outbound_email WHERE type = 'system'



Native

cc++ph

perl pyth

o

rub

shellDatabase

Operating System Solaris/Linux/OS X/

Container Glassfish/Tomcat

Frameworks

Application Logic

PresentationJSP/Javascript/Ajax

dynamic tracing

-> contentAreaClick -> getPreventDefault <- getPreventDefault<- contentAreaClick



DTrace

You can do it with...



Have you heard about it

Have you used it

Have you written d-scripts

poll



The D - Language

Basicsof

Dynamic Instrumentation



instrumentation format

/ /predicate when

{

}

actions do what

d-script

: : :provider module probe name whereprobe



command line version

dtrace command line

# dtrace -n probe’/predicate/{action}’



demo



aggregation

=> mmap mmap 5 <= mmap mmap 5 => munmap munmap 5 <= munmap munmap 5 => setcontext setcontext 5 <= setcontext setcontext 5 => getrlimit getrlimit 5 <= getrlimit getrlimit 5 => getpid getpid 5 <= getpid getpid 5 => setcontext setcontext 5 <= setcontext setcontext 5 => sysi86 sysi86 5 <= sysi86 sysi86 5 => brk brk 5 <= brk brk 5 => brk brk 5 <= brk brk 5 => xstat xstat 5 <= xstat xstat 5 => resolvepath resolvepath 5 <= resolvepath resolvepath 5 => open open 5 <= open open 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => munmap munmap 5 <= munmap munmap 5 => memcntl memcntl 5 <= memcntl memcntl 5 => close close 5 <= close close 5 => xstat xstat 5 <= xstat xstat 5 => resolvepath resolvepath 5 <= resolvepath resolvepath 5 => open open 5 <= open open 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => mmap mmap 5 <= mmap mmap 5 => munmap munmap 5 <= munmap munmap 5 => memcntl memcntl 5 <= memcntl memcntl 5 => close close 5 <= close close 5 => munmap munmap 5 <= munmap munmap 5 => ioctl ioctl 5 <= ioctl ioctl 5 => ioctl ioctl 5 <= ioctl ioctl 5 => brk brk 5 <= brk brk 5 => brk brk 5 <= brk brk 5 => fsat fsat 5 <= fsat fsat 5 => fcntl fcntl 5 <= fcntl fcntl 5 => fstat64 fstat64 5 <= fstat64 fstat64 5 => getdents64 getdents64 5 <= getdents64 getdents64 5 => getdents64 getdents64 5 <= getdents64 getdents64 5 => close close 5 <= close close 5 => ioctl ioctl 5 <= ioctl ioctl 5 => fstat64 fstat64 5 <= fstat64 fstat64 5 => write write 5 <= write write 5 => close close 5 <= close close 5 => close close 5 <= close close 5 => rexit rexit

rexit 1 fcntl 2 fsat 2 getpid 2 getrlimit 2 sysi86 2 write 2 fstat64 4 getdents64 4 memcntl 4 open 4 resolvepath 4 setcontext 4

xstat 4 ioctl 6

brk 8 munmap 8 close 10 mmap 20

dtrace -P syscall

dtrace -P syscall’{@=count(probefunc)}’



dtrace aggregates - demo



the pid provider

• Dynamically create probes in live running applications.

• No extra work needed from application developer

• Available without any compile or runtime flags

• Can be created on optimized binaries as well

• Probes created at function boundaries and offset

21

provider: pid$<processId>module: name of libraryprobe: function namename: entry

return offset

Pid

ProviderSunday, May 31, 2009


demo



probing MySQL using Pid

23

provider: pid$<mysql-pid>module: not-usedprobe:*dispatch_command*name: entry arg2 - SQL command

MySQ

L

Queries

#!/usr/sbin/dtrace -qspid$1::*dispatch_command*:entry{ printf("%s\n",copyinstr(arg2,arg3));}



dtrace and mysql - demo



php provider

compile-file-entry/-returnfunction-entry/-returnexecution-entry/-return

exception-thrown/-catchederror

request-startup/-shutdown

25

provider: php$<pid>module: not-usedprobe: not-used name: <see detail ->

Apache

Provider

New and improved in OpenSolaris 2009.06

Older releases

function-entry/-return



dtrace and php - demo



Apache provider

received-requestlog-requestcreate-child

accept-connectioncheck-user-credentials

check-accesscheck-authorization

27

provider: apache$<pid>module: not-usedprobe: not-used name: <see detail ->

Apache

ProviderSunday, May 31, 2009


gotcha



dtrace and the webstack



Observing the entire stack

More details on the scripts we saw earlier...

what are the hot modules in Drupal?

what are the hot hooks in Drupal?

how much of my Drupal load is from anonymous users

how much time am I spending in Apache/MySQL/PHP?

what are the SQL statements that are executed by anon users

30



Give me more!

DTrace landing page:http://www.opensolaris.org/os/community/dtrace

DTrace Toolkit:In OpenSolaris - /opt/DTTSolaris download -http://www.opensolaris.org/os/community/dtrace/dtracetoolkit/

DTrace Bigadmin page:http://www.sun.com/bigadmin/content/dtrace/

Lots and lots of info on the web - Google “dtrace”

31


http://www.opensolaris.org/os/community/dtrace

http://www.opensolaris.org/os/community/dtrace

http://www.opensolaris.org/os/community/dtrace/dtracetoolkit/

http://www.opensolaris.org/os/community/dtrace/dtracetoolkit/

http://www.sun.com/bigadmin/content/dtrace/

http://www.sun.com/bigadmin/content/dtrace/


Social

My Blog:http://blogs.sun.com/angelo

Twitter:rajadurai

DTrace hands on labhttp://developers.sun.com/learning/javaoneonline/j1lab.jsp?lab=LAB-9400&yr=2008&track=1

32


http://blogs.sun.com/angelo

http://blogs.sun.com/angelo

http://developers.sun.com/learning/javaoneonline/j1lab.jsp?lab=LAB-9400&yr=2008&track=1




session code S308357

Angelo [email protected]

Observing Your App and Everything Else It Runs On Using DTrace


mailto:[email protected]

mailto:[email protected]


DTrace and JavaScript



Observing javascript using the javascript provider

Enter script details

35



implementation
