RTOS Debugger for Linux - Run Mode - · PDF fileRTOS Debugger for Linux - Run Mode 2 ©1989-2017 Lauterbach GmbH RTOS Debugger for Linux - Run Mode Version 06-Nov-2017 Debugging Modes

RTOS Debugger for Linux - Run Mode

TRACE32 Online Help

TRACE32 Directory

TRACE32 Index

TRACE32 Documents ......................................................................................................................

RTOS Debuggers ...........................................................................................................................

RTOS Debugger for Linux .........................................................................................................

RTOS Debugger for Linux - Run Mode .................................................................................. 1

Debugging Modes for Embedded Linux ............................................................................ 2

Run Mode Debugging with TRACE32 as GDB Front-end 2

Stop Mode Debugging via JTAG 2

Integrated Run & Stop Mode Debugging via JTAG 3

Supported Debugging Modes sorted by Architectures 4

Basic Concepts .................................................................................................................... 5

t32server (only ARM) 5

gdbserver 5

Ethernet as Communication Interface to the gdbserver 7

Operation Theory 7

Switching to Run Mode Debugging 8

DCC as Communication Interface to the t32server 9

The Debug Communications Channel (DCC) 9

Operation Theory 9

Switching to Run Mode Debugging 10

The Space ID for Run Mode Debugging 11

Process Debugging 12

Quick Start Example for ARM ............................................................................................. 14

Quick Start Example for MIPS ............................................................................................. 16

Quick Start Example for PowerPC ...................................................................................... 18

SYStem.Option NoDebugStop 19

Switching between Run & Stop Mode Debugging ............................................................ 20

Commands for Run Mode Debugging ................................................................................ 23

Breakpoint Conventions ...................................................................................................... 24

Debugging of Multi-threaded Applications ........................................................................ 25

Frequently-Asked Questions .............................................................................................. 26

RTOS Debugger for Linux - Run Mode 1 ©1989-2017 Lauterbach GmbH

RTOS Debugger for Linux - Run Mode

Version 06-Nov-2017

Debugging Modes for Embedded Linux

TRACE32 provides 3 modes for debugging embedded Linux: Run Mode Debugging, Stop Mode Debugging and Integrated Run & Stop Mode Debugging.

Run Mode Debugging with TRACE32 as GDB Front-end

Pure software debugger, no TRACE32 hardware required.

The TRACE32 software is licensed by a USB dongle.

Requires a gdbserver/gdbstub running on the target.

The TRACE32 GDB Front-end works in so-called Run Mode Debugging: At a breakpoint only the selected process is stopped, while the kernel and all other processes continue to run.

The TRACE32 GDB Front End is described in the document “TRACE32 as GDB Front-End” (frontend_gdb.pdf).

Stop Mode Debugging via JTAG

Stop Mode Debugging requires a TRACE32 JTAG debugger hardware.

If debugging is performed via the JTAG interface, TRACE32 works in so-called Stop Mode Debugging. At a breakpoint the CPU and thus the whole target system is stopped.

The main advantages of Stop Mode Debugging are:

• Since the only requirement for Stop Mode Debugging is a functioning JTAG interface, debugging can start at the reset vector.

• Since TRACE32 provides both Linux and MMU support, debugging of the kernel and beyond process boundaries is possible.

Stop Mode Debugging is described in the document “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).


Integrated Run & Stop Mode Debugging via JTAG

Integrated Run & Stop Mode Debugging requires a TRACE32 JTAG debugger hardware.

If debugging is performed via the JTAG interface, TRACE32 can be configured:

• To allow Stop Mode Debugging via JTAG.

• To allow Run Mode Debugging via the t32server/gdbserver running as debug agent on the target.

• To allow, if required, to switch between the two debugging modes.

TRACE32 communicates with:

• The gdbserver via ethernet for all supported architectures (see table on next page) or

• The t32server via DCC (Debug Communications Channel) for the ARM architecture. Since the ARM-JTAG interface provides a DCC, Integrated Run & Stop Mode Debugging can function with JTAG as the only communication interface.


Supported Debugging Modes sorted by Architectures

Architecture GDB Front-end Stop ModeDebugging

Integrated Run & Stop Mode Debugging

ARM Yes, refer to ”TRACE32 as GDB Front-End” (monitor_gdb.pdf).

Yes, refer to “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).

Yes, via DCC (except Cortex-M) or Ethernet. For details refer to this manual.

ColdFire — Yes, refer to “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).

—

I386 Yes — Yes, via Ethernet

MIPS Yes, refer to ”TRACE32 as GDB Front-End” (monitor_gdb.pdf).


Yes, via ethernet. For details refer to this manual.

PowerPC Yes, refer to ”TRACE32 as GDB Front-End” (monitor_gdb.pdf).



SH4 Yes Yes, refer to “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).


XScale Yes Yes, refer to “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf).

—


Basic Concepts

For Integrated Run & Stop Mode Debugging, Stop Mode Debugging via the JTAG interface is extended by:

• t32server/gdbserver as debug agent on the target.

• A communication interface between TRACE32 and the debug agent (ethernet or DCC).

t32server (only ARM)

The t32server is a gdbserver extension provided by LAUTERBACH. Compared to the gdbserver the t32server allows debugging over DCC for the ARM architecture. The t32server starts a gdbserver for every process.

• the gdbserver has to be in the /bin directory of the Linux file system.

• the IP address 127.0.0.1 has to be configured, because the t32server communicates with the separate gdbservers via TCP/IP.

In order to provide Integrated Run & Stop Mode Debugging the t32server has to be started as a Linux process on the target via the terminal window e.g.:

gdbserver

You can display the version of your gdbserver by typing in the terminal

If nothing is printed out, then your gdbserver has a version older than 6.2. Using version 6.2 and older is not recommended.

If run mode debugging over Ethernet is used, we recommend to use gdbserver 7.1 or newer.

./t32server ; Communication via DCC

gdbserver --version


If used together with the t32server (communication over DCC), gdbservers newer than 6.5 have to be patched to disable run length encoding:

If the run length encoding is not disabled in the gdbserver, the communication with the t32server will be too slow.

Patching gdbserver newer than 6.5 (DCC communication only)file: gdb-x.x/gdb/gdbserver/remote-utils.c

static inttry_rle (char *buf, int remaining, unsigned char *csum, char **p){ int n;

/* Always output the character. */ *csum += buf[0]; *(*p)++ = buf[0]; return 1; // add this to disable run length encoding /* Don't go past '~'. */ if (remaining > 97) remaining = 97;...


Ethernet as Communication Interface to the gdbserver

TRACE32 communicates with the gdbserver via ethernet for the MIPS and the PowerPC architecture.

Operation Theory

Linux

gdbserver

SOFTWARE

HARDWARE

MIPSCore

JTAG

ethern

et


Switching to Run Mode Debugging

After TRACE32 was started and configured for Stop Mode Debugging switching to Run Mode debugging is performed as follows:

After the communication is configured, debugging can be performed completely via the TRACE32 GUI.

SYStem.PORT 10.1.2.99:2345

Go.MONitor

SYStem.PORT <ip>:<port> Configure ethernet communication

<ip> is the target IP-Address<port> is the used port number

Go.MONitor Switch to Run Mode Debugging


DCC as Communication Interface to the t32server

TRACE32 communicates with the t32server via DCC for the ARM architecture.

The Debug Communications Channel (DCC)

The JTAG interface of the ARM architecture includes a so-called Debug Communications Channel (DCC). Information exchange via DCC is possible between

• the TRACE32 software running on the host

• and the t32server running as Linux process on the target

On an SMP system, the debugger only communicates with the DCC registers of the first core. Thus, the t32server always run on this core. For details about DCC refer to your ARM Technical Reference Manual.

Operation Theory

Linux

t32server

127.0.0.1:port1gdbserver Process 1

127.0.0.1:port2gdbserver Process 2

127.0.0.1:portngdbserver Process n

SOFTWARE

HARDWARE

ARMCore

JTAG

DC

C


Switching to Run Mode Debugging

After TRACE32 was started and configured for Stop Mode Debugging switching to Run Mode debugging is performed as follows:

After the communication is configured, debugging can be performed completely via the TRACE32 GUI.

SYStem.MemAccess GdbMON

Go.MONitor

SYStem.MemAccess GdbMON Configure DCC as communcation interface to t32server

Go.MONitor Switch to Run Mode Debugging


The Space ID for Run Mode Debugging

Processes of Linux may reside virtually on the same addresses. To distinguish those addresses, the debugger uses an additional space ID, that specifies, to which virtual memory space an address refers. In Run Mode Debugging the space ID is equal to the process ID.

The command SYStem.Option MMUspaces ON enables the additional space ID.

A source code listing for the process sieve is displayed as follows:

For details on the space ID for Stop Mode Debugging, refer to “Training Linux Debugging” (training_rtos_linux.pdf).

Space ID (decimal and hex value)

Space ID


Process Debugging

To debug a process proceed as follows:

1. Check if the process is already running.

2. Load the process for debugging.

If the process is not running, the command TASK.RUN can be used to load the process for debugging.

If the process is already running, the command TASK.SELect can be used to attach to it.

TASK.List.tasks List all running processes

TASK.List.tasks

TASK.RUN <process> Load <process> (process not running)

TASK.select <id> Attach to the process(process already running)

; Load process sieve from the Linux file system and prepare it for; debugging

TASK.RUN /bin/sieve

TASK.SELect /bin/sieve


3. Load the symbol and debug information for the process.

Since processes of Linux may reside virtually on the same addresses, the symbol and debug information has to be loaded for the address space of the process by using the <space_id>.

/NOCODE - load only symbol information.

/NoClear - obtain the symbol information loaded for other processes.

/NOREG - avoid any unintended change to the CPU registers.

Example for a PRACTICE script:

Data.LOAD.<subcommand> <file> <space_id>:0 /NOCODE /NoClear /NOREG

; Data.LOAD.Elf <file> <space_id>:0 /NOCODE /NoClear /NOREGData.LOAD.Elf sieve.elf 0x91:0 /NOCODE /NoClear /NOREG

; Stop sieve at main and display source listingGo mainData.List

TASK.PROC.SPACEID(<process>) This function returns the <space_id> of a process. This is required for PRACTICE scripts.

LOCAL &spaceid&spaceid=TASK.PROC.SPACEID("sieve")

TASK.RUN /bin/sieve

Data.LOAD.Elf sieve.elf &spaceid:0 /NOCODE /NoClear /NOREG


Quick Start Example for ARM

Integrated Run & Stop Mode Debugging requires that Stop Mode Debugging is working properly before Run Mode Debugging can be activated. The following commands represent a basic TRACE32 setup for Stop Mode Debugging. It is assumed that the target setup and Linux booting is performed by the code in the boot FLASH.

SYStem.CPU ARM920 ; Select the target CPU

SYStem.Option MMUspaces ON ; Extend logical addresses by; space ID

TrOnchip.Set DABORT OFF ; Debug mode is not entered at data; abort exception used by Linux; for page miss!

TrOnchip.Set PABORT OFF ; Debug mode is not entered at; prefetch abort exception used by; Linux for page miss!

TrOnchip.Set UNDEF OFF ; Debug mode is not entered at an; UNDEF instruction may be used by; Linux for FPU detection

SYStem.MemAccess GdbMON ; Configure DCC communication; between TRACE32 and t32server

SYStem.Mode Attach ; Establish the communication; between the debugger and the CPU

Data.LOAD.Elf vmlinux /NOCODE ; Load the kernel symbols

TASK.CONFIG linux ; Enable the TRACE32 linux; awareness

MMU.FORMAT LINUX swapper_pg_dir ; Define an MMU page table format; for Linux

TRANSlation.COMMON 0xC0000000--0xffffffff

; Upper memory pages are valid for; all space IDs

TRANSlation.TableWalk ON ; Configure the debugger to proceed; as follows at an conversion; from virtual to physical address:; ; 1. Look-up in the debugger MMU; table; 2. Walk through the Linux MMU; table, if the look-up in the; debugger MMU fails

TRANSlation.ON ; Switch debugger MMU to ON


The target setup and the preparations for Linux debugging might be more complex for your system. It is strongly recommended to refer to “ARM Debugger” (debugger_arm.pdf) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details.

To configure Run Mode Debugging:

1. Make sure you have a gdbserver in the /bin of the Linux file system on your target.

2. Make sure the loop-back address 127.0.0.1 is configured on your target.

3. Start t32server as Linux process via the terminal window:

4. Then continue to enter the following commands in TRACE32:

./t32server

Go.MONitor ; Switch to Run Mode Debugging


Quick Start Example for MIPS


The target setup and the preparations for Linux debugging might be more complex for your system. It is strongly recommanded to refer to “MIPS Debugger and Trace” (debugger_mips.pdf) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details.

SYStem.CPU MIPS24KE ; Select the target CPU

SYStem.Option Endianness Big ; Select big endian mode



SYStem.PORT 10.1.2.99:2345 ; Configure ethernet communication; between TRACE32 and t32server




TRANSlation.COMMON 0x80000000--0xffffffff


TRANSlation.TableWalk ON ; Configure the debugger to proceed; as follows at an conversion; from virtual to physical address:; ; 1. Look-up in the debugger MMU; table; 2. Walk through the Linux MMU; table, if the look-up in the; debugger MMU fails


SYStem.Option gdbNONSTOP ON ; Enable GDB non-stop mode

SYStem.Option gdbEXTENDED ON ; Enable GDB extended mode



1. Start the gdbserver in multi-process mode:

2. Then continue to enter the following commands in TRACE32

gdbsever --multi:2345



Quick Start Example for PowerPC


The target setup and the preparations for Linux debugging might be more complex for your system. It is strongly recommended to refer to your PowerPC Processor Architecture Manual (e.g. “PPC600 Family Debugger” (debugger_ppc600.pdf) ) and “RTOS Debugger for Linux - Stop Mode” (rtos_linux_stop.pdf) for details.

So that ethernet will not be broken in the case of some PowerPCs, please do not stop the CPU after Linux is started. If you would like to debug in stop mode first, please go to run mode with Go.MONitor and back to stop mode with Break.MONitor, so that the ethernet interface could be disabled.

SYStem.CPU MPC8349 ; Select the target CPU



SYStem.PORT 10.1.2.99:2345 ; Configure ethernet communication ; between TRACE32 and t32server




TRANSlation.COMMON 0xC0000000--0xffffffff


TRANSlation.TableWalk ON ; Configure the debugger to proceed; as follows at an conversion; from virtual to physical address:; ; 1. Look-up in the debugger MMU ; table; 2. Walk through the Linux MMU; table, if the look-up in the ; debugger MMU fails


SYStem.Option gdbNONSTOP ON ; Enable GDB non-stop mode

SYStem.Option gdbEXTENDED ON ; Enable GDB extended mode



1. Start the gdbserver in multi-process mode:

2. Then continue to enter the following commands in TRACE32

SYStem.Option NoDebugStop

For PowerPC, software breakpoints and single step are enabled respectively in stop or run mode:

• SYStem.Option NoDebugStop ON: software breakpoints and single step are enabled in run mode.

• SYStem.Option NoDebugStop OFF: software breakpoints and single step are enabled in stop mode.

This option is automatically set/reset by the debugger with every Go.MONitor and Break.MONitor command (except the first Go.MONitor. After the first Go.MONitor command, breakpoints and single step will not work in run mode).

Please note that this option could only be set/reset while the CPU is stopped. If you change this option manually, this could disable breakpoints and single step in you current mode.

gdbsever --multi:2345



Switching between Run & Stop Mode Debugging

The graphic above shows a simple schema of the switching between Run Mode Debugging and Stop Mode Debugging. Not all transitions are covered.

The following commands are used to switch between Run & Stop Mode Debugging:

Go.MONitor If the CPU is stopped, the program execution is started.

Switch to Run Mode Debugging.

In Run Mode Debugging no process is selected for debugging.

Break.MONitor Switch to Stop Mode Debugging and stop the program execution on CPU.

Break.SetMONitor ON Switch to Run Mode Debugging with the next Go.

In Run Mode Debugging no process is selected for debugging.

Break.SetMONitor OFF Switch to Stop Mode Debugging.

If the selected process was running or no process was selected, the CPU stays running in Stop Mode.

If the selected process was stopped, the CPU is stopped in Stop Mode.


If Run Mode Debugging is active, a green M is displayed in the state line of TRACE32.

The following states are possible in Run Mode Debugging:

1. Run Mode Debugging active (green M), no process selected (see TASK.List.tasks).

2. Run Mode Debugging active (green M), selected process (sieve) stopped.


3. Run Mode Debugging active (green M), selected process (sieve) running.


Commands for Run Mode Debugging

If the selected process has been started with TASK.RUN, it will be selected as current process. Otherwise a gdbserver will be started and the selected process will be attached.

Only processes that have been started with a TASK.RUN or that have been attached with TASK.SELect can be killed. After a TASK.KILL, the process and its gdbserver will be killed.

TASK.List.tasks List all running processes

TASK.RUN <process> Load a process for debugging

If the command TASK.RUN is used to load a process for debugging, the process is stopped by the gdbserver at the label _start in ld-linux.

To start process debugging, load the symbol information for the process first and then type Go main.

TASK.select <id> Select a process for debugging

It is only allowed to attach to user process. If you try to attach to a kernel process, an error message will be returned.

TASK.KILL <id> Request GDB agent to end the process

TASK.COPYUP <src> <dest> Copy a file from the target into the host

TASK.COPYDOWN <src> <dest> Copy a file from the host into the target

SYStem.GdbLibPath <path> Set the path to the target libraries on the host


Breakpoint Conventions

For Integrated Run & Stop Mode Debugging please keep the following breakpoint convention:

• Use on-chip breakpoints for Stop Mode Debugging

If an on-chip breakpoint is hit in Run Mode Debugging, the CPU is stopped in Stop Mode debugging (only for ARM).

• Use software breakpoints for Run Mode Debugging

Examples for Stop Mode Debugging:

Examples for Run Mode Debugging:

; Break.Set <space_id>:<address> /Program /OnchipBreak.Set 0x0:0x4578 /Program /Onchip

Break.Set error /Program /Onchip

; Break.Set <space_id>:<address> /Program /SOFTBreak.Set 0x2bc:0x0xd0065789 /Program /SOFT

Break.Set 0x2bc:main /Program /SOFT


Debugging of Multi-threaded Applications

To be able to debug multi-threaded applications, TRACE32 needs the target libraries ld.so.1 (ld-linux.so.2 or ld-linux.so.3 in the case of ARM targets) and libpthread.so.0. The path to this libraries on the host can be set with the TRACE32 command SYStem.GdbLibPat h. For example:

SYStem.GdbLibPath C:\T32\libs


Frequently-Asked Questions

ARM WFI and power down modes

Ref: 0395

After booting Linux, why does the target die or loose connection to the debugger?

Many CPUs have special power saving modes if they spin in the idle loop. Usually these are encoded in pm_idle() - Check this. While the debugger tries to recover from power saving modes, this is not always possible. E.g. the WFI mode of some i.MX CPUs does not work with JTAG, and many Cortex-A CPUs power down the JTAG interface in idle. Check your distribution - In Freescale's Linux distribution for i.MX set the command line parameter "jtag=on" to disable WFI mode. In some other distributions there is a "nohlt" parameter to disable power down modes.

e500 breakpoints

Ref: 0426

Breakpoint are not working properly or stop working completely for the e500 cores

The Linux kernel for e500 cores (PQ3/MPC85XX/QorIQ P10XX/P2020) has to be patched to enable debugging via JTAG. The MSR_KERNEL macro defined in arch/powerpc/include/asm/reg_book.h needs to be changes to include the MSR_DE bit: #define MSR_KERNEL (MSR_ME|MSR_RI|MSR_CE|MSR_DE)

Kernel message "soft lockup"

Ref: 0388

What does the kernel message soft lockup mean?

The kernel detects when there is too much time gone between two timer ticks. This easily happens, if the system is halted with the debugger. Please switch off the soft lockup detection by disabling CONFIG_DETECT_SOFTLOCKUP in the kernel configuration.


Linux trace in simulator

Ref: 0304

What is needed to revise a Linux trace with a TRACE32 instruction set simulator?

Example for ARM926 SAVING IN TRACE32 DEBUGGER:

; save trace recordingTrace.SAVE trace.ad; save whole RAM address rangeData.SAVE.Binary image.bin ASD:<address_range>; save registersSTORE regs.cmm register; save important CP15 registersData.SAVE.Binary p15.bin C15:0x0--0xfff /Long

LOADING IN TRACE32 INSTRUCTION SET SIMULATOR:

SYStem.RESetSYStem.CPU ARM926SYStem.Option MMUspaces ON ; enable space IDSystem.Up; load linux imageData.LOAD.Binary image.bin ASD:address_range; load registersdo regs.cmmData.LOAD.Binary cp15.bin c15:0; load symbol informationData.LOAD.Elf vmlinux /GNU /NOCODE; specifiy MMU table formatMMU.FORMAT LINUX swapper_pg_dir <kernel-range> <RAM phys. address>TRANSlation.COMMON 0xc000000--0xfffffffTRANSlation.TableWalk ONTRANSlation.ON; configure Linux awarenessTASK.CONFIG linuxMenu.ReProgram linuxHelp.Filter.ADD rtoslinux; load trace recordingTrace.LOAD trace.ad

MMU.FORMAT LINUX swapper_pg_dir

Ref: 0255

Why does "MMU.FORMAT LINUX swapper_pg_dir" report "invalid combination"?

Space IDs must be switched on for this command to work: "SYStem.Option MMUSPACES ON".


PPCSIM Trap

Ref: 0425

The error message "PPCSIM Trap <number>" appears when using the Linux awareness

This error message is due to an incompatibility between the TRACE32 software and the loaded Linux awareness.

Problems with Onchip Breakpoints

Ref: 0252

After halting at a function entry point with an on-chip breakpoint, or after stepping into a new function, the Data.List window shows "???"for all assembler lines, even after rescanning the MMU.

You probably fell over the demand paging mechanism of Linux. Please read the "On-Demand Paging" chapter in the Linux Awareness Manual.

SYstem.Option MMUSPACES ON/OFF Error

Ref: 0256

Why does "SYstem.Option MMUSPACES ON/OFF" report "Cannot change from short to long"?

This command executes only, if no symbols are loaded. Delete previous loaded symbols with "sYmbol.Delete".

TASK.MODULE Display Problems

Ref: 0254

After loading a new kernel module, TASK.MODULE shows nothing, dashed lines or errors.

The debugger couldn't resolve the virtual addresses of the new kernel module. Check the MMU and TRANSlation settings in your startup script. See also chapter "MMU Support" in the Linux Awareness Manual.
