NeD: The Network Extensible DebuggerPaul Maybee - Solbourne Computer, Inc.

ABSTRACT

_ N9D is a debugging server with a programmable network interface. NeD is designed tobe flexible and extensible enough to support a wide range of debugging needs. Debuggingclients communicate with NeD by sending it programs to execute. The programminglanguage, NeDtcl, is tcl[l] extended with 30 debugging specifrc functions. NeD can be useãas a traditional debugger with a textual interface, but the user would frnd the languagecumbersome. It is designed to be convenient for communication between programs, ratherthan between program and user. As a demonstration of NeD's víability as a debugging server,the pdb[2] debugger has been retargeted to use NeD as its server.

1.0 Introduction

The nature of UNIX software development con-stantly changes. Over the past several years worksta-tions with graphias displays have largely replacedascii terminals, resulting in ubiquitous windowingsystems, and the enormous growth in user interfacedevelopment. The increasing acceptance of objectoriented programming has resulted in the widespreaduse of new programming languages to support it.Multi-process programming has always been com-mon on UNIX machines, but its use has taken onnew dimensions with networks and the ability toeasily create and communicate with remoteprocesses. Multi-th¡eaded programming is supportednow on many vendors' platforms, with more to comein the near future. All of these developments haveresulted in a steady increase in the size and com-plexity of software systems. Yet the scope ofchanges in debugging technology over the sameperiod of time has been narrow, focusing primarilyon user interface issues. Several products, e.g.,dbxtool [3] and SaberC (now CodeCenter) [4], haveplaced window system interfaces over essentiallycommand driven debuggers. Pi [5] and pdb pushedthis technology further by completely eliminating thecommand interface. SaberC made an initial attemptat data structure visualization, ild FIELD t6ladvanced this technology by the addirion of layouimethods and iconic structure representations.

FIELD also developed an inter-tool communi-cation system (now available in commercial guise)that allows various other softwa¡e development andvisualization tools to interact directlv with thedebugger [7]. Tool-interconnection appeæs to be thenext major step in softwa¡e development environ-ments, with several vendors developing productsalong the lines of the FIELD work.

Several debuggers now support debugging C++programs, at various levels. One of the primary rea-sons for less than complete support in the debuggershas been inadequate compiler generated symboltables. This problem should gradually disappear as

better compilers become available.The ability to debug multiple processes, even

on the same machine, has typically not been sup-ported, and UNIX implementations often haveprevented debuggers from even attaching to newlyforked process. Large program debugging seems tobe an issue that is often ignored in the constructionof debugging systems (with the exceptions of pi [12]and pdb). It is common for users go away and comeback later when a program has finished loading.

NeD addresses all of these issues. NeD doesnot have a user interface to speak of; it is a serverdesigned to work with client user interface tools.NeD was built speciûcally for the support of theC++ language, although it is sr¡itable for use withother languages. However, its capabilities to debugC++ in a completely native mode still suffers some-what for want of compiler support. NeD supports thedebugging of child processes as they fork fromdebugged parents as well as debugging over the net-work and debugging multithreaded proc€sses. NeD isextensible; a client may customize NeD's remoteinterface to more easily and efficiently provideneeded services. Thus NeD will continue to be use-ful in the face of change by allowing clients toexpand the power of the debugging server at runtime. Finally, because of the use of lazy symbolevaluation, NeD is very efficient, especially in itsuse of memory.

In the following discussion, "NeD" will refer tothe NeD server, "client" will refer to a client of theNeD server, "pdb/NeD" will refer to the version ofpdb implemented using NeD, and "subordinate pro-gram" will refer to a program being debugged byNeD.

2.0 Features

2.1 Client-Server DebuggingNeD implements the server side of a

client/server debugging system. The typical clientprovides a user interface, the NeD server providesdebugging services. NeD executes on the same host

Summer '92 USEMX - June 8-June 12,lgg} - San Antonio, TX L45

NeD: The Network Extensible Debugger

as the subordinate program, and uses the operatingsystem provided interface to query and control theprocess. NeD contains an extended tcl[l] commandinterpreter. Tcl provides an embedded, list struc-tured, command language that provides "mechanismsfor variable, procedures, expressions, etc."; theextension consists of a set of additional embeddedfunctions that provide debugging capability. Thisextended language is NeDtcl (see Appendix A). Theclients execute remote procedure calls that sendNeDtcl statements to the interpreter. For example, acall may contain a direct invocation of an embeddeddebugger function, or contain statements definingnew NeDtcl procedures, or it may contain codeinvoking these previously defined procedures.

The execution of the NeDtcl statements pro-duces results that are forwarded back to the client asthe return value of the remote procedure call. Inaddition , the execution of some statements causesthe subordinate to change state, for example to beginexecution. When an execution state change occursNeD generates an event. The event is passed to theNeDtcl interpreter which may handle it, or send it onin a message to the client. NeD generates several

Maybee

classes of events automatically. Clients can alsocause additional, user defined, events to be gen-erated. A client handles the event when the messageis received, or it can download procedures to theinterpreter to catch and process events.

NeD supports a special kind of breakpoint thatis placed only on a fork system call. When thisbreakpoint is hit, NeD modifies the program text fol-lowing the fork call such that executing the newcode will cause the process to suspend. NeD thenallows the process to execute the fork. The newchild process that is created will immediatelysuspend; NeD takes control of the parent again,loads the pid of the child, and opens a new socket.NeD then forks. The child copy of NeD attaches tothe new subordinate child and accepts connectionson the socket. The parent copy of NeD generates a"new3roc" event containing the pid and socket id ofthe child NeD. At this point the NeD client can con-nect to the new NeD and debug the child subordi-nate process (see Figure 1).

NeD clients can extend this picture by attach-ing NeD servers to multiple processes, even whenthose processes are on different machines. Pdb/Ì.{eD,

Figure 1: Debugging Processes that Fork

L46 Summer '92 USENIX - June t.June 12,lgg2 - San Antonio, TX

Maybee

for example, will allow any number of processes tobe debugged simultaneously through one set of win-dows. The user can select the process to view via amenu selection. In addition, a process beingdebugged will automatically come into view when ithits a breakpoint.

Using a client-server architecture has efficiencydrawbacks when more sophisticated tools such asevent stream recognizers or program monitors arebeing used [14,15]. Network latency makes eventsout-of-date by the time they arrive. Data collectionmay take up a great deal of network and machineresource. NeD alleviates these problems through itsextensiblity. Debugging tools can implement eventrecognizers or data filters in the server itself (section2.3) or link additional functionality into the subordi-nate process(section 2.4).2.2 Dlsplaying Data

All communication between NeD and its clientsis textual, including values returned by expressionevaluation requests. NeD supports displaying datastructure values through "templates". Templatesdescribe how fields are to be displayed, includingconditional display based upon the program state. Ifan integer variable "i" with a current value of 10 isevaluated the result returned by NeD would be"SIMPLE NUMBER 10", indicating thar a simplevalue was returned. A template for this expressionwould have a form similar to this result. The tem-plate "SIMPLE {hex tim}" would direct NeD toreturn the value in hexidecimal and as a system timevalue. Also allowed are octal, decimal, binary, ascii,

NeD: The Network Extensible Debugger

unsigned decimal, floating, and special formats. Thespecial option allows the user to customize thedisplay of values in a way that makes sense in thecontext of the subordinate program. This optionrequires that the name of a function to be found inthe subordinate be supplied, e.g., "SIMPLE {hex{spl myfunctionll". When this value is to bedisplayed, myfunction is invoked with the value asinput. It returns a pointer to a static text stringrepresenting the value.

For aggregate values the format matches that ofthe data structure. Given the following structuredefinition:

struct s {in t i ¡struct s *ptr i

) ;when a variable of this type is evaluated NeDreturns:

AGGREGATE S {{ i {srMPr,E NUMBER 10}}{p t r {S IMPLE PTR Oxf7 f f fbO0} }

)

Each field of an aggregate template contains athree element list:: first the field name, then a condi-tional expression, and then the field's template. Thetemplate that matches "struct s", that displays "i" asabove, and displays "ptr" only when "i" is non-zerolooks like:

set_tenplate {unsigned int} {CLASS foo} {SrMpLE oct}

Figure 2: Displaying unsigned integers in class "foo"

O clo¡at O r¡ le I c lass O Funct ion

i . . . . i i : : : i ( . i : ) : i : yí. i i ì î : :Û int type;! XAnyEvent xany:ú XKeyEvent xkei;É XBut tonEventxbut ton;

ÞÞÞÞ

Scope:0l-d-tech

Condit ion: n<>. tvpe : KeyPress | | <>. type El

i :ili: i i i:il i í i:i:::i i :ii:: ¡ í : i ir;i ! ïrii i í,,,,: ¡ í ii :, i i ri,i iÍ'u rr*tÍc n:

c@@Figure 3: Pdb/NeD template editor

Summer '92 USENIX - June 8-June L2,lgg} - San Antonio, TX 147

NeD: The Network Extensible Debugger

AGGREGATE s {{ i { } {S IMPLE {hex t Ín } } }{ p t r { < > . i t = 0 } { s I M p L E h e x } }

)Any boolean expression that can be evaluated in thecontext of the original expression can be used as anevaluation condition. The angle bracket pair (+) inthe example condition refers to the expression beingevaluated, "exp", the brackets will be replaced by"(exp)" before evaluation. This is especially usefulwhen displaying union types with tag fields. Eachvariant of the union can cafty a condition such as"tag == mytag". Then when the union is evaluatedonly the correct union variant appears.

A template can be specified on an individualexpression or for all expressions of a given type (invarious scopes). For example, all unsigned integersin the scope of class "foo" can be displayed in octalby executing the NeDtcl command shown in Figure2, Variables of type "unsigned int" evaluated inIocations other than member functions of class "foo"will not be affected. Scope designations can includeGLOBAL, FILE, CLASS, or FUNCTION scopes.When an expression is evaluated the templates willbe searched for a matching type in the inner mostmatching scope. If no custom ièmplate is found thena default global template is used.

The set of custom templates that are in effectcan be retrieved from NeD and saved. NeD can thenbe reloaded with these the next time that the oro-gram is being debugged. Templates for entire claisesof debugging can be prepared a priori and used bygroups of programmers. For example, the X windowsystem [5] defines a union called an XEvent withthirty-three variants. Thirty-one of the væiants arethe different X event types and one of the va¡iants is

Maybee

the tag field (i.e., the tag is an overlay of the firstfield in each of the events), the remaining variant ispadding. A NeDtcl initialization file could beprepæed for X progr¿rmmers that would include atemplate for the XEvent; displaying rhe proper væi-ant depending upon the tag field. Figure 3 shows thepdb/Ì.{eD user interface for editing type templates2.3 Extensibility

NeDtcl's embedded functions provide, more orless, haditional debugger functionality. There æefunctions that, for example, set and delete break-points, advance execution, and evaluate expressions,The interface is stateless. There is no concept of acurrent function, or current file, as a more traditionaldebugger would supply, Commands tend to takeadditional parameters with which to specify prog¡amlocations or options. For example, in dbx the usercan type "stop at 10" to set a breakpoint at line L0of the current file. When using NeDtcl this operationrequires entering code as shown in Figure 4. Thismakes the language unacceptably cumbersome forhuman use, but not for computer use. The NeDinterface is designed for use by a NeD client thatpresents an alternate, most likely graphical, userinterface, This is not to say that NeD cannot be usedas a textual debugger, only that the interface, bydefault, does not support it well. Since NeDtcl is acomplete programming language a "dbx-like" userinterface could be written in NeDtcl and loaded intothe interpreter. In fact, to facilitate testing NeD, thiswas done.

NeDtcl does not contain functions that are notdirectly debugger related (e.g., dbx's "make") orfunctions that produce information that can otherwisebe computed (e.9., dbx's "where"). Non-debuggingfunctionality, such as recompiling objects, is

se t_break [addr_of { th is f i le .c t0 { } } ] { } { } s top

Figure 4: A breakpoint in NeDtcl

### defínition of traceback(stack_num, max_depth)

proc Èraceback {{stack_nun 0} {nax_frames 100}} {set d [min $max_frames tdepth stack_num] lset t race " "f o r { s e t i 0 } { S i < S d } { s e r Í [ e x p r { $ i + r y ¡ y

set pc lget_pc $stack_nu¡n gi lset loc I locat ion_of gpc]### concat the pc, l-ocatÍon, and evaluatedlappend trace [ l Íst gpc gloc

Ieva?_Jjst $Ioc gstack_num gi

) [ l ist_of_params $loc¡ ¡ ¡

return Strace)

### compute traceback### }:.ot¡ many frames?### ini t ia l ize l ist

{ ### for each frame### get the pc### get source location

parameter l ist

### return completed list

Figure 5: A traceback

Summer '92 USENIX - June 8-June 12,1992 - San Antonio, TX14t

Maybee

assumed to be provided, if necessary, by the NeDclient. Complex functionality, such as displaying astack traceback, can be loaded into the interpreter, Inthe pdb/Ì.[eD implementation, the first operation pdbdirects NeD to do at start-up is load a file of NeDtclprocedure definitions into the server's interpreter;one of which is "traceback"; see Figure 5.

The information returned by traceback is alist with the same information as is contained in astack traceback obtainable from any source leveldebugger. Ttaceback Eaverses the execution stackreturning a desctiption of each stack frame. Eachdescription is a list containing the pc, source loca-tion (including function name), and parameter list forthe frame. A parameter list contains the name andcurent value of each pæameter. In these examplesitialics aÍe used for names (e.g., traceback,get_pc, eval_Jist, and 7is+_of_params)defined in the pdb/Ì.{eD start-up file; names in bold(e.g,, depth and location_of) are embeddedNeDtcl functions. The remainder is tcl code.

Getting a Eaceback requires only one remoteprocedure call. The previous version of pdb requiredone procedure call to get the location and parameternarnes and then another call for the evaluation ofeach parameter. An alternate way to decrease theremote t¡affic would have been to modify the formerinterface to include all the information returned bythis traceback function. However, the problem with afixed interface is that it does not provide enoughflexibility. It may be that a future graphical debuggerinterface would require much less, or much moreinformation for each stack frame (e.g., no parameterevaluation or values for all local variables). NeDprovides this flexibility.

NeD: The Network Extensible Debugger

NeDtcl also provides support for the intercep-tion and generation of NeD events. Figure 6 imple-ments a conditional step function. The procedurestep_untilJ takes an expression (and some con-text information) and steps the program one sourceline at a time until the expression evaluates to true.

Figure 6 implements a function similar to a"watchpoint", i.e,, break execution on a memorylocation content change rather than on a memorylocation being executed. At each execution of etepthe process changes state twice; it stæts executingand then it stops executing. Each of these statechanges will result in an event being sent to theclient. If the client normally processes these eventsto update a display then this may result is a greatdeal of unwanted screen painting and network traffic.This procedure can be easily modified to interceptthe events.

When NeD generates an event it invokes theNeDtcl function process_events,Process_events cycles through a list of eventprocessor functions looking for one that hasexpressed an interest in the event. If one is notfound, the event is sent to the client via theevenÈ3ass function. So to intercept the processstate change messages, the "step until" proceduremust be modified to add an event processor. Thefunction on_event is used to post or remove anevent processor. The first parameter indicates theevent class to catch, the second parameter is a com-mand to execute. If the command returns non-zerothen the event has been handled. Step_untíL2installs a simple handler for all "proc_state" eventsbefore invoking step_until-1, and then removesit aftenvards. SËep_until2 also sends a"proc-state running"

-Ivent to the client before

######

proc

step_unt i I l ( locat ion stacknum expression): no event interception

step_unÈiJJ {1oc stacknum exp} {wh i le { l [p roc_eva l 91oc gs tacknum 0 $exp { } 0 ] } {

s tep 0 { })

Figure 6: A watchpoint

### step_until2 ( location stacknum expression)

proc step_untlt2 {1oc stacknum exp} {event3ase {proc_state running}on_event proc_state {return 1}step_unt iJJ gloc gstacknum gexpon_event proc_statechecketate 1

)

Figure 7: A more complex example

Summer '92 USENIX - June 8.June L2,1992 - San Antonio, TX

######

######

program set runninghandle event

remove handlergenerate final event

149

NeD: The Network Extensible Debugger

beginning execution and causes NeD to generate a"proc_stateil event after stepping has completed.This allows the client's normal event processing tosee the "step until' operation as it would a singlestep; see Figure 7;

The previous examples demonstrate NeDls abil-ity to display the state and control the execution of asubordinate process, the following examples showthat NeD can also perform non-trivial data querieson subordinates. Procedure LÍst_Tengtl¡J willcalculate the length of a linked list. It assumes thatthe list structure contains a next field, by defaultnamed "next", pointing to a structure of the sametype. Given the head of the list, the function iteratesuntil it discovers a next field with a special value, bydefault 0x0. Each succeeding next value is computedby adding ".next" onto the previous expression andreevaluating. See Figure 8 for the listing.

This implementation will work, but the size ofthe expression being evaluated grows with the lengthof the list, making it a less than perfect solution. Analternate implementation makes use of the expres-sion evaluator's "typeof intrinsic function to keepthe length of the expression constant, Instead ofusing the previous expression to get the next expres-sion to evaluate, Tist_Tengtl¡2 uses the value ofthe previous expression. The next expression is gen-erated by casting the value to a pointer to the type ofthe structure and then adding "->next". See Figure9.

Queries such as this can be verv useful inbuilding advanced graphical user intórfaces, forinstance structure browsers. Network latencv can bereduced by filtering data at the server rather thanrequiring network transmission. As with the

Maybee

templates, groups of users working on similar prob-lems can share modifications to enhance a project'sdebugging envi¡onment.2.4 Subordinate Extensibility

NeD allows the subordinate program to beextended at run time. NeD will load dynamiclibraries into the executing subordinate and thenallow functions in the library to be invoked throughthe debugger interface. This can be useful for sup-porting debugging aids, such as tools that traverseprogram data structures looking for inconsistencies,or performance monitoring frrnctions. For example,the list length function of section 2.3, ot a freespace analysis routine could be supplied by the userin a shared library. One of the first uses this was putto at Solbourne was to provide a file desøiptorstatus utility. A function that 'stat's all the processesfile descriptors is linked in and invoked. The fr¡nc-tion dumps a report to a file; the utility reads thereport and displays the information to the user.

Newly linked library code can also be directlycalled by the subordinate process by inserting callsinto the program text. This feature provides the facil-ities needed for less intrusive debugging and moni-toring tools, such as Parasight [13]. The Parasightparadigm involves inserting new functions into arunning process to collect performance and tracedata. It is orders of magnitude faster to have the pro-gram collect its own data, than to have the debuggerstop it continually to do so.

NeD causes the subordinate proc€ss to executedynamic linker calls in order to load new libraries.Under SunOS, dynamic linker routines are alwaysresident with any dynamically linked process. NeDexposes the 4 dynamic library routines necessary

### l ist_Iength1(loc stacknum head f Íeld 1end): length of t ist

proc Jist_Length7 {Ioc sracknum head {f ietd next} { Iend OxO}} {set next IsimpJevalof [proc_eval gloc gstacknum 0$head { i 0 l l se t exp ghead fo r {se t i 0 } {gnexr ¡= g lend} {se t i lexpr$ i + 1 1 ) {

se t exp ($exp¡ .$ f ie ld se t nex t l s inp leva lo f[proc_eval $loc gstacknum 0 $exp {} 0] I } return $i }

Figure 8: Calculating length of a list

### r ist- length2(Ioc stacknum head type f iel .d lend): length of r ist

proc l ist_Length2 { loc stacknum head {f ie ld next} { lend Ox0}} {set next I sinplevaTof [¡lroc_eval gloc gstacknum 0$head {} 0l l set type ls jmp-Uevalof [proc_eval gloc$s tacknum 0 typeof ($head¡ { } 0 l l fo r {se t i 0 } {$nexr t= $ lend} {ser Íl e x p r $ i + 1 1 ) {

se t exp ( ($ type * )$nex t ) ->$ f ie ld se t nex t as inp levaTof[proc_eval $Ioc gstacknum 0 $exp t ] 011 ] return $i ]

Figure 9: An alternate list length calculator

150 Summer '92 USENIX - June 8.June 12, L992 - San Antonio, TX

Maybee

(dlopen, dlclose, dlsym, and dlerror) by placing sym-bols for them into the symbol table. Expressions canthen reference the symbols to access the functional-ity they provide. On systems where the dynamiclibraries are not always present in the subordinatethe debugger must either use an altemate mechanismor require the image to be linked specially. Figure10 is a sample NeDtcl function that invokes an arbi-trary function in an arbitrary shared library.

NeD: The Network Extensible Debugger

3.0 Implementation

NeD is more than a demonstration of concept;it will form the basis for the next generation ofSolbourne's debugger products. As such, it must beefficient and provide a full range of debuggingfeatures. NeD is still under development but isalready robust enough and fi¡nctional enough to sup-port the pdb debugger. NeD is implemented for theSPARC architecture under SunOS 4.1.1. It can

proc dyncalT I library func args) {set 1 simplevaLueof

[proc_evaf {{} 0set f simplevalueof

lproc_evaf {{} 0proc_eval {{} 0 {}} 0proc_eva l { t } 0 { } } 0

{ } } 0 0 d l o p e n l $ I i b r a r y , l ) { } 0 l

0 0 d l s y m ( $ 1 , $ f u n c ) { } 0 l( ( i n t ( ) ) $ f ) ( $ a r s s ) { } 0d l c l o s e ( $ 1 ) { } 0

Figure 10: An arbitræy function evaluator

{ } }00

Figure 11: pdb/Ì.{eD architecture

Summer '92 USEMX - June 8-June LZ,lgg2 - San Antonio, TX 151

NeD: The Network Extensible Debugger

debug C, C++, and Fortran progmms at the sourcelevel, understanding proper target language syntax.Initial tests indicate that NeD uses considerablyfewer resources than conventional debuggers such asdbx and gdb. Table 1 shows a comparison of thetime and memory required to load a C++ prog¡amcontaining approximately 144,000 symbols. Thetests generating these results were run on a Sol-bourne 54000 with 40 meg of memory.

resource NeD edb dbxreal time (sec) 33 29 t40memorv (mess) 4 t6 39

Table 1: Comparison of NeD, gdb, and dbx

4.0 Futures

4.1 Pdb/NeD ArchitecturePdb/NeD clients currently communicates

directly with NeD seryers. However they will even-tually communicate through a FIELD-like messageserver [7,9]. Pdb/Ì.[eD is currently a single client,and that too will change. The pdb user will be ableto invoke multiple, independent clients that commun-icate through the message server. Clients will bespecialized, for example to display and edit anno-tated source, to evaluate expressions, to graph datastructures, or to monitor communications. NeDservers will respond to queries from any client.Clients will extend the servers by downloading pro-cedure definitions, or by having the servers load ini-tialization files, or by mapping shared libraries thatprovide additional functionality into thesubordinate's address space (see Figure 11).4.2 Multi-threaded Process Debugging

NeD contains support for debugging multi-threaded programs, but the current implementationbase (i.e., SunOS 4.1.1) does not implement multi-threaded processes, thus there is never more thanone thread per process. NeD will be ported to amulti-threaded version of the SVR4 operating systemin the near future. The port will also result in aswitch from using the ptrace debugging interface tousing the /proc interface [10]. A benefit of the moveto the /proc interface is that it greatly simplifiesdebugging forked processes.4.3 Grafting NeDtcl to Other Debuggers

A debugger for a modern software developmentenvironment cannot succeed without associated toolsproviding graphical user interfaces. In fact, mosrdebuggers offer very similar underlying capabilities,with the main distinction between them coming inthe interface or higher level analysis capabilities.User interface software written for the X windowsystem, in our experience, has been easy to portbetween platforms. Porting debuggers is a muchmore difficult task slnce the debugger is dependentupon the operating system's debugging interface, the

Maybee

compiler-debugger interface, and machine architec-ture. Projects to standardize UNIX and to standard-ize the symbols generated by compilers, ê.8,DWARF [11], will help make the task easier ifadopted. But the task will not become as easy asporting X library dependent code, or other applica-tion software, in the foreseeable future.

The NeD viewpoint is that the investment ingraphical tools can be most easily capitalized on ifthe debugger doesn't have to be ported. If the graph-ical tools are written to use a defined debugger inter-face such as NeDtcl, then only the interface must beported to a new debugger. There are debuggersavailable, e.g., the Free Software Foundation's gdb,that have already been ported to many platforms. Ifsuch a debugger were to understand the NeDtclinterface then new tools could more rapidly be avail-able across different platforms.

The NeDtcl portion of the NeD server amountsto about 1600 lines of C++ code, most of which is astraightforward translation of the parameters to, andresults generated by, the embedded functions fromcharacter to internal format. The special formatsrequired for expression value output (see "value" inAppendix A, Data Types) is not such a large changeto the traditional value display procedure in anydebugger. This would typically be a recursive rou-tine that traversed an internal structure and prints outthe contents of each field. The modifications consistmainly of printing some additional brackets and typ-ing information that must already be available tosuch a module.

5.0 Acknowledgments and Availability

Andrew Gerber implemented the graphical userinterface for NeD, making sophisticated testing pos-sible. The members of the OI group sewed as guineapigs for early versions of pdbAIeD. Their sugges-tions and comments have helped enormously.

NeD is still under development but will beavailable from Solbourne Computer's software busi-ness unit later this year. The initial version will betargeted for SPARC compatible computers runningUNIX SVR4.

References

[1] Ousterhout, J. K. Tcl: An Embeddable Com-mand Language, Proceedings of the Winter1990 USENIX Conference, Washington D. C.(January 22-26,7990)

[2] Maybee, P. pdb: A Network Oriented SymbolicDebugger, Proceedings of the Winter 1990USENIX Conference, \ilashington D. C. (Janu-ary 22-26,7990)

[3]Adams, E. and Muchnick, S. S. Dbxtool: AWindow-Based Symbolic Debugger for SunWorkstations. Software - Practice and Experi-ence, V16 N7, July 1986, pp.653-669

tsz Summer '92 USENIX - June 8.June 12,1992 - San Antonio, TX

Maybee

[4] Kafer, S. Lopex, R. and Pratap S. Saber-C AnInterpreter-based Programmíng Environmentfor the C Language, Proceedings of the Sum-mer 1988 USENIX Conference, San Francisco,CA (June 20-24, L988)

[5] Cargill, T. A. The Feel of Pi, Proceedings ofthe Winter 1986 USF,NIX Conference, Denver,CO (January 15-17, 1986)

[6] Reiss, S. P. Interøcting with the FIELDEnvironment Brown University DeparEnent ofComputer Science Technical Report CS-89-51(May, 1989)

[7] Reiss, S. P. Integratìon Mechønßms ín theFIELD Environment, Brown University Depart-ment of Computer Science Technical ReportCS-88-18 (October, 1.988)

[8] Scheifler, R. W., Gettys, J., and Newman, R. XWindaw System C Library ønd Protocol Refer-ence, Digital Press, 1988

l9l ToolTalk Programmers Guide, SunSoft, Inc.Revision A of September 1991

[10] Faulkner, R., and Gomes, R, The Process FileSystem and Process Model in UND( System V,Proceedings of the Winter 1991 USENIXConference, Dallas, TX (Jan. 2l-25,lggt)

[1,1] DWARF Debugging Information Format, IJNIXInternational Programming Languages SpecialInterest Group, October 21,1,99i,, DRAFT

[12] Cargill, T. A. Pi; A Case Study in Object-Oriented Programming, C++ WorkihopProceedings, Santa Fe, NM, USENIX Assoc.(Nov 9-10, 1987)

[13] Aral, Z. and Gertrer, 1., High-Level Debuggíngin Parasight, Proceeding of Workshop onParallel and Distributed Debugging, publishedas SIGPLAN Notices, V24, No. l, ppt51.-1.62,January 1989.

[14] Bates, P., Debugging Heterogeneous Dßtri-buted Systems Using Event-BAsed Models ofBehavior, Proceeding of Workshop on paralleland Distributed Debugging, May 5-6, 1988,published as ACM SIGPLAN Norices, V24,No. 1., pp 1.1-22, January 1989.

[15] Spezialetti, M. An Approach to ReducíngDe.lays in Recognizing Distributed EventOccurrences, Proceedings of the ACM/ONRWorkshop on Parallel and Distributed Debug-ging, May 20-21, 1991., published as ACMSIGPLAN Notices, V26, No. 12, December799L.

Author Information

Paul Maybee is a software engineer with Sol-b9u_rne Computer, Inc. He is the principle designerof Solbourne's debugger products including pdbìndNeD. Prior to joining Solboume, Paul attended theUniversity of Colorado where he received a Mastersdegree in Computer Science. Reach him via U.S.

NeD: The Network Extensible Debugger

Mail at Solbourne Computer, Inc., 1900 pikeLongmont, CO 80501, or electronicallypaulm@solbourne.com.

Rd.at

Summer '92 USENIX - June 8-June lZ,1rgg? - San Antonio, TX 153

NeD: The Network Extensible Debugger Maybee

Appendix A: NeDtcl grammar

This grammar defines the NeDtcl extensions to tcl. All non-chain rule productions correspond to tcl lists.Thus a production with n terms on the right hand side refers to a tcl list of length n (unless n = 1). Terminalsappear in uppercase, non-terminals in lowercase. All non-terminals ending in "_name" are chained to STRING,these productions do not appear. Functions can complete successfully or fail. Functions are listed with ttreparameters types each expects. Each function is followed, optionally, by the type of value each returns when itcompletes successfully and the type of event it generates.Embedded Functions

command=proc-cmd I scope-cmd I obj_cmd I expr_cmd I sig_cmd I inst_cmdI exec_cmd I info_cmd

proc_cmd ="s tacks""depth" stackÍd" locat ion_of" ADDRESS"addr-of" focation

scope_cmd ="l ist_of_variables" context" l ist_of_types" context" l ist-of_funcs" context" l ist_of_f i les "" I i s t o f c l a s s e s "

obj_cmd =" load_obj " fíIe_name

### process/stack commands# returns stackid l ist# returns INT# returns location# returns ADDRESS

### scope query commands# returns var_list# returns type_Iist# returns func_l ist# returns f i le_l ist# returns class_l ist

### object command# generates object load

= ### expression evaluation co¡nmands"proc_eval" Iocation stackid stackdepth expression template async

# returns va1ue, generates expr_eval"set_template" type_name context template # returns BOOLEAN"get_template" type_name context # returns template

expr_cmd

Isig-cmd

inst cmd

exec cmct

info

coref i le_name

ihandle" signal-name sighow"signal" s ignal_name" s igs ta te"

"s-et-break"

" set vtatch"

"catchfork""delete_break" break_id"get_breaks"

" l t tach" pid"coreattach"" detach "¡ "s tep Í " I

### insLrumentation commandsADDRESS condÍtion expression break_action

# returns break_idADDRESS size condition expression break_action

# returns break id# returns Urea¡<-id## returns break_list

### ptocess execution commands#

objectfite_name# returns INT#

### signal processing commands### returns signal l ist

### infornational commands# returns conf ig_l ist# returns proc_desc,# generates proc_state#

n_b # returns interp_Iistparam_value

"Next i " | "s tep" | "Nex t " | "con t " | "F in ish" ¡stackid signal_name# generates proc_state

_cmd"conf ig" conf ig_kind"checkstate" async

"eventSass" event"interp" Ínterp_kind address n_a

param_name

154

" set3aram"

Summer '92 USENIX - June 8-June 12,1992 - San Antonlo, TX

Eventsevent=object_load I expr_evaluated I proc_state I traceobject_load = "obj load" t j_meexpr_evaluated =

"expreval" Locat ion stackid

Maybee

stackdepthproc-state = ###

[ "stopped" | "running" | " terminated"nevtjrocess = "ne\,rproc" pid host_name port ###trace = ###

"tracemsg" locat ion stackid stackdepth

normal_break desc =ADDRESS condition expressj.on

pid = INTport = INTprint_condÍtion = expression

NeD: The Network Extensibte Debugger

### object loading is complete### exptession eval is complete

Data Typesaggregate_temp = "AGGREGATE" field_temp ### struct/union templateaggregate_value = ### struct/union expression value

"AGGREGATE" agg_na¡ne fÍel¿l_value*async = BooLEAN ### asynchronous notifÍcation flagbreak_action = "STOPU | "GO" ### stopr==breakpoiDt, go==lracepointbreak desc =

[ 'BREAI(PoINT" normal break desc | -

"WATCHPOINT" watch ãesc l -"FORKPOTNT" ¡ breaE_Íd

break_id'= rNT ### Ínstrumentat ion ident i f ierbreak_Iist = break desc* ### breakpoint description 1istcrass- l ist = crass-n*.* ### crass name l istcondiEion = expresãion ### condltional expressionconf ig kind = "regj-sters" | "s ignals" ### confígurat ion Lyp""context ### scope descriptiòn

scope_type scope_n€rme l "l,ocATroN" Iocationexpr-value = expression value ### result of evaluating an expressionexpression = STRING ### C/C++/Fortran expreãsion

-

field-temp = ### struct/union tiela templatef ield_name print_condition template

fÍeId-value = field- name-value -

### struct/union fieLd expr valuetile_Íist = source_File* ### :J.isl- of source filesfunc-l Íst = ### Iist of functions in source

function_name file_name path_name lj.ne-num ADDRESSÍnterp-desc = inTruction-dãsc ###iemory interpretationinterprkind = "INSTR" ### how mämory sÈould be interpretedinterp_list = j-nterp_desc* ### J,isl- of mãmory interpretationsinstruction_desc - ### instruction däscriptÍon

location instr_tranlation symbolic nameinstr_tranlation = SfnÏuC

-### ínstruction disassembly

Iíne number = INT ### source line numberlocation = ### source location

f ile_name line_number function_namen-a = INT ### number of translations

### af|-et addressn_b = INT ### number of translations

### before address

expr_value type_nameprocess state change| "unattached" Ine̡r process to debugtracepoint executedexpr_va1ue type_name

### breakpoint descriptionbreak_action

print_kind =" d e c " | " h e x " | " o c t , ' | " u n s , ' I" t i m " | " a s c " | " i n s " | " f I t , , i

proc-desc = STRING

### UNIX process íd### inet port number### conditional expression for### templates### simple print formats

" b i n " I" s p e "

### description of process being

Summer '92 USENIX - June 8-June 12,lgg2 - San Antonio, TX 155

NeD: The Network Extensible Debugger Maybee

regclass- l ist = register_crass_name* ### rLsE of register classesregnarne_Iist = register name* ### l-isl" of register namessig-state = signal-name sighow ### how signal wilr be handledsighow - "IGNoRE" | "CATCH" ### aignal handling typessignal_Iist = sig_state* ### l.ist of signal handlÍngssimple_temp = "SIMPLE" print_kind* ### simple type templatesimple_value = "SfMpLE" ### simple type value

[ "NUMBER" | "prR" I UMSG"

| "ERROR" I"vorD" | "ENUM" | "coMpLEx" | "srRrNG"l srRrNG

size = INT ### number of bytesscope_type = ### kinds of scopes

t "GLOBAL" | "CLASS" | "FrLE" | "FUNCTTON', Isource-fÍle - fi le_name path_name ### source file descriptionstackdepth - rNT ### number of frames on call stackstackid = INT ### identifier for cal1 stackstorase-crass =,,1ocal, ,

l "grobar, ' l , ," tut iËf i l" ï : :*1 i t?::g;":*:" i""resparam" | "register" | "regparam" | "unknown" I"classmem" l "common"

template = simple_temp I aggregate_temp ### value print templatestime = rNT ### object file time stamptype_list = type_na¡ne* ### LLsE of program typesvalue = simpl,e_value I aggregate_value ### exptession valuevar-list = variable* ### list of progr¿rm variablesvariable = variable_nane storage_class ### vatiable name and classwatch_desc = ### description of a watchpoint

ADDRESS síze condition expression break_actj.on

156 Summer '92 USENIX - June 8.June L2,1992 - San Antonio, TX