Thread Debugging — Overview of eCos Kernel thread-aware debugging
Thread-aware debugging refers to the ability to interrogate the list of threads active within an application when the system is stopped (halted). This is normally when the code has either stopped at a breakpoint, or when execution is interrupted via a hosted debug session (e.g. from GDB).
For eCosPro to aid external host-based debug tools, a set of helper symbols are defined to provide information on the size (width) and offset of useful fields, or relevant constant values, instead of addresses. The majority of these symbols are named to avoid possible namespace clashes with applications, but for historical reasons some architecture specific symbols are valid in the C/C++ namespace. Similarly different architectures export their own symbols.
An external tool that wishes to interpret thread information can check for the presence of the specifically named symbol, and add support accordingly.
These symbols are held in the symbol table of the ELF file, but have no cost impact (code or data size) on the actual binary loaded into the target (either via a debugger, or if an application binary is stored on the target). Obviously a stripped executable will lose the helper symbols, but debugging using a stripped ELF file would always pose some restrictions.
These symbols are required for accessing the list of threads and the currently active thread. If they are not present in the symbol table then it indicates an eCos build without a thread scheduler, and hence there is no need for thread-aware debug support.
- Pointer to the first thread descriptor in the chain of created threads.
- Pointer to the thread context for the currently active thread.
When an eCos scheduler is configured, information describing the thread context is provided to enable generic scanning code to be implemented in an external tool regardless whether some eCos features are enabled or disabled. Since individual eCos configurations can have features present that change the shape of the actual thread descriptor structure, we need the important fields for scanning a list of threads to be available in each ELF file.
The presence of a
field in the thread descriptor structure can be
determined by a non-zero
size symbol being provided. These symbols give the
size, in bytes, of the relevant field at the offset
specified by the corresponding
__ecospro_syminfo.off.*symbol. This allows any host tool to provide features based on the conditional presence of fields.
- Offset to field that points to the next thread descriptor.
- Offsets for useful fields in a thread descriptor structure.
- This symbol is the offset for the field containing the address of the stacked register context for inactive threads.
For Cortex-M targets the following symbols describe the
main CPU register state. The presence of the symbol
can be used as indicator of an eCos Cortex-M
This symbol provides the actual
PCaddress of the point in the code where a switch actually occurs, It may be useful depending on how the external tool interprets the stacked context information.
These symbols provide the values in the type field
used to identify the shape of the stacked
context. Since only threads will be accessed by
Cyg_Thread::thread_listlist the host tools should only ever encounter
__ecospro_syminfo.value.HAL_SAVEDREGISTERS.THREADtype contexts (with the optional FPU register state indicator flag). The values for the other types of Cortex-M contexts are provided for completeness only.
This symbol provides the total size, in bytes, of
a stacked CPU context for contexts of type
- Total size, in bytes, of all the core CPU registers present in a stacked context.
- Size, in bytes, of the field that encodes the type of stacked context. Since for Cortex-M targets the actual context stored (and its size) depends on whether the individual thread has any hardware FPU context saved. This is needed to ensure only valid information is used when dealing with lazy per-thread hardware FPU support.
- Size of individual context fields referenced by the corresponding offset symbol.
- Offsets into the stacked context for the core register values.
When a Cortex-M configuration with hardware FPU support configured is used then the following optional symbols will be present with non-zero values where appropriate.
This value provides the bitmask flag
OR-ed into the type field provided at the
__ecospro_syminfo.off.HAL_SavedRegisters.u.thread.typeoffset used to identify individual thread contexts that contain FPU state.
Size, in bytes, of the
FPSCRregister stacked in the context.
- Total size, in bytes, of the single-precision vector stacked in the context.
FPSCRregister in the stacked context.
- Offset of the single-precision register vector in the stacked context.
For ARM/Cortex-A targets the following symbols describe
the main CPU register state. The presence of the symbol
ARMREG_SIZE can be used as indicator
of an eCos
- This symbol, if non-zero, provides the total size of the stacked context for inactive threads.
These symbols provide the offset of the
corresponding register within the stacked context
referenced from the thread object
Optional ARM FPU symbols.
- If non-zero then this symbol indicates that the eCos applicaton has been configured with hardware FPU support. The value is the total size of the stacked inactive thread context.
Offset with the stacked context of the
ARM/Cortex-A FPU Single-Precision
Symbols present when single-precision ARM FPU is configured.
- The offset to the start of the stacked single-precision register vector.
Number of single-precision registers present from
ARM/Cortex-A FPU Double-Precision
Symbols present when doubled-precision ARM FPU is configured.
- The offset to the start of the stacked double-precision register vector.
Number of single-precision registers present from
Symbols provided by eCos ColdFire targets. The presence of
be used as an identifier for ColdFire targets.
- The overall stacked context size.
- If hardware FPU support is configured this symbol provides the size of the stacked FPU context.
- These symbols provide information on the size (width) of individual registers.
- The offsets within the stacked context for the processor state.
The value, in bytes, if a
PCSRRTE adjustment is used.
Other Useful Symbols
Some other standard eCos symbols may be present that could also be useful for external debug tools.
- This symbol can be used to identify the idle (background) thread descriptor object if useful to the thread-aware host debug tool.
This symbol will not be present in the application
symbol table if the relevant object is not defined.
CYGSEM_LIBC_STARTUP_MAIN_THREADis configured this symbol can be used to reference the thread descriptor object for the
main()C thread created by the run-time.
The following documentation uses the GDB command-line interface for its examples, although the thread-aware debug support is applicable to applications that access GDB via its programmatic interface, e.g. Eclipse.
When a GDB debug session halts the CPU, either from the code
hitting a previously set breakpoint or via the user
requesting a halt, it will display the state of the
currently active CPU state and select the currently
executing thread. When displaying threads via the
info threads command the currently
selected thread is highlighted by an asterisk (
*) character. Therefore, immediately
after a halt this will indicate the active, running, thread.
Examining the CPU register state will report the state of
this active thread.
With thread-aware debugging for the target application available,
GDB will display a list of all known threads when given the
info threads. The command
can be used to switch context to other threads, providing
the ability to examine their CPU register state, call stack,
and local variables.
Depending on the target system being connected, the act of
loading an eCos application into memory will not necessarily
initialise all the memory and hardware state. The eCos
application run-time startup code will normally initialise
memory alongside other I/O requirements. Since it can be
possible to execute thread interrogation commands before any
target code has been executed, it can be useful having
helper macros in your
to minimise misinformation being displayed. The following
clear_ecos_thread_pointers GDB macro is
an example which could be executed after loading an
application and before any
system initialisation code in the loaded application has
been executed. It ensures that debug commands to interrogate
thread state will not parse stale/undefined information from
define clear_ecos_thread_pointers set *((unsigned int *)&Cyg_Thread::thread_list) = 0 set *((unsigned int *)&Cyg_Scheduler_Base::current_thread) = 0 end document clear_ecos_thread_pointers When starting a new debug session from application reset the run-time code that clears BSS will not have been executed, so stale/unitialised state may be present in memory. For RTOS aware thread debugging as provided by external tools the GDB server may be confused and report invalid state if the thread state is interrogated before the initial eCos run-time initialisation has cleared the BSS area. This macro just ensures that the relevant eCos pointers are NULL prior to debugging. end
Of course if a hardware debugger is being used to connect to an existing application session (rather than loading and starting a new application session) then the macro should not be called.
It is useful to wrap the steps needed to connect to a target in a helper macro. e.g.:
define connocd target extended-remote localhost:3333 load break cyg_test_exit break cyg_assert_fail display/i $pc clear_ecos_thread_pointers end
So that all of the normal steps for loading and setting the debug environment for an application can be performed by a single command:
(gdb) connocd 0x080016dc in ?? () Loading section .rom_vectors, size 0x8 lma 0x90000000 Loading section .text, size 0x7b34 lma 0x90000008 Loading section .rodata, size 0x678 lma 0x90007b40 Loading section .data, size 0x180 lma 0x900081b8 Start address 0x90000008, load size 33588 Transfer rate: 94 KB/sec, 6717 bytes/write. Breakpoint 1 at 0x900040f4: file
ecospro-path/packages/infra/current/src/tcdiag.cxx, line 391. Function "cyg_assert_fail" not defined. Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal] (gdb)
The use of such macros from a GDB script file can make the task of debugging less cumbersome.
The Ronetix PEEDI firmware must be updated to at least version 21.2.0 to ensure the correct operation of the thread-aware debugging support.
[TARGET] section option
be used to introduce a thread/context description using the
generic PEEDI support.
The example PEEDI configuration files supplied with eCosPro
releases 4.5.8 and above should already have suitable RTOS support
fragments. For example, the file
referenced from the head of the file via the
With a suitable
PEEDI will parse the eCos thread lists and stacked register
contents when interrogating threads other than the current
thread of executing on the CPU.
OpenOCD provides the
-rtos eCos option
that can be used to configure thread-aware debug support in
the configuration file used for the OpenOCD session.
As of 2021-04-01 the generic
eCos thread-aware debug support is specific to eCosPro
At its simplest the OpenOCD configuration file just needs to specify:
$_TARGETNAME configure -rtos eCos
NOTE: When OpenOCD
-rtos support for eCos
is configured the act of executing
to connect to an OpenOCD instance will cause the configured
OpenOCD RTOS support to perform an update_threads
operation against the current memory state. This is to allow a debug
session to be attached to an active system. However, it does
mean that for an undefined memory state (power-on, CPU reset
with undefined DRAM state, an application with a different
thread context shape to the previous application new
different-configuration application to be subsequently
loaded after connecting to the OpenOCD GDB server) that the
GDB server may report spurious thread information upon
The following is example output when the application is halted in a thread named "busy". The name, state and priority of the other available threads is also shown:
(gdb) info thr Id Target Id Frame * 1 Thread 12 (Name: busy, State: Ready Pri: 20) 0x20009f50 in thread_busy (data=30000) at
ecospro-path/packages/kernel/current/tests/fpint_thread_switch.cxx:566 2 Thread 1 (Name: Idle Thread, State: Ready Pri: 31) Cyg_Scheduler::unlock_inner (new_lock=0) at
ecospro-path/packages/kernel/current/src/sched/sched.cxx:233 3 Thread 2 (Name: Test, State: Sleeping (WAIT) Pri: 3) Cyg_Scheduler::unlock_inner (new_lock=1) at
ecospro-path/packages/kernel/current/src/sched/sched.cxx:233 4 Thread 11 (Name: highpri, State: Sleeping (DELAY) Pri: 10) Cyg_Scheduler::unlock_inner (new_lock=0) at
The OpenOCD GDB server can be left executing between GDB application debug sessions. It does not need to be re-started for every GDB session.
Unlike the hardware debug approaches described above which benefit from bare-metal hardware support (SWD, JTAG, BDM, etc.), eCos also supports the use of GDBstubs which can be built into the application and accessed via an I/O channel (e.g. serial, Ethernet, etc.), or provided via a boot monitor/loader (e.g. RedBoot covering in Chapter 215, Getting Started with RedBoot) that provides an environment for executing applications. Such support has a run-time cost (code+data space as well as CPU cycles), and is only usable after some level of system initialisation has occurred. A hardware debug solution is therefore preferred.
However, if GDBstubs is the only available/possible solution, the eCos GDBstubs implementation supports thread aware debugging.
|2022-10-12||Open Publication License|