While kgdb as an offline debugger provides a very high level of user interface, there are some things it cannot do. The most important ones being breakpointing and single-stepping kernel code.
If you need to do low-level debugging on your kernel, there is an on-line debugger available called DDB. It allows to setting breakpoints, single-stepping kernel functions, examining and changing kernel variables, etc. However, it cannot access kernel source files, and only has access to the global and static symbols, not to the full debug information like kgdb.
To configure your kernel to include DDB, add the option line
options DDBto your config file, and rebuild. (See Kernel Configuration for details on configuring the FreeBSD kernel.
Note: Note that if you have an older version of the boot blocks, your debugger symbols might not be loaded at all. Update the boot blocks; the recent ones load the DDB symbols automagically.)
Once your DDB kernel is running, there are several ways to enter DDB. The first, and earliest way is to type the boot flag -d right at the boot prompt. The kernel will start up in debug mode and enter DDB prior to any device probing. Hence you can even debug the device probe/attach functions.
The second scenario is a hot-key on the keyboard, usually Ctrl-Alt-ESC. For syscons, this can be remapped; some of the distributed maps do this, so watch out. There is an option available for serial consoles that allows the use of a serial line BREAK on the console line to enter DDB (options BREAK_TO_DEBUGGER in the kernel config file). It is not the default since there are a lot of crappy serial adapters around that gratuitously generate a BREAK condition, for example when pulling the cable.
The third way is that any panic condition will branch to DDB if the kernel is configured to use it. For this reason, it is not wise to configure a kernel with DDB for a machine running unattended.
The DDB commands roughly resemble some gdb commands. The first thing you probably need to do is to set a breakpoint:
b function-name b address
Numbers are taken hexadecimal by default, but to make them distinct from symbol names; hexadecimal numbers starting with the letters a-f need to be preceded with 0x (this is optional for other numbers). Simple expressions are allowed, for example: function-name + 0x103.
To continue the operation of an interrupted kernel, simply type:
To get a stack trace, use:
Note: Note that when entering DDB via a hot-key, the kernel is currently servicing an interrupt, so the stack trace might be not of much use for you.
If you want to remove a breakpoint, use
del del address-expression
The first form will be accepted immediately after a breakpoint hit, and deletes the current breakpoint. The second form can remove any breakpoint, but you need to specify the exact address; this can be obtained from:
To single-step the kernel, try:
This will step into functions, but you can make DDB trace them until the matching return statement is reached by:
Note: This is different from gdb's next statement; it is like gdb's finish.
To examine data from memory, use (for example):
x/wx 0xf0133fe0,40 x/hd db_symtab_space x/bc termbuf,10 x/s stringbuffor word/halfword/byte access, and hexadecimal/decimal/character/ string display. The number after the comma is the object count. To display the next 0x10 items, simply use:
x/ia foofunc,10to disassemble the first 0x10 instructions of foofunc, and display them along with their offset from the beginning of foofunc.
To modify memory, use the write command:
w/b termbuf 0xa 0xb 0 w/w 0xf0010030 0 0
The command modifier (b/h/w) specifies the size of the data to be written, the first following expression is the address to write to and the remainder is interpreted as data to write to successive memory locations.
If you need to know the current registers, use:
Alternatively, you can display a single register value by e.g.
p $eaxand modify it by:
set $eax new-value
Should you need to call some kernel functions from DDB, simply say:
call func(arg1, arg2, ...)
The return value will be printed.
For a ps(1) style summary of all running processes, use:
Now you have now examined why your kernel failed, and you wish to reboot. Remember that, depending on the severity of previous malfunctioning, not all parts of the kernel might still be working as expected. Perform one of the following actions to shut down and reboot your system:
This will cause your kernel to dump core and reboot, so you can later analyze the core on a higher level with kgdb. This command usually must be followed by another continue statement. There is now an alias for this: panic.
Which might be a good way to cleanly shut down the running system, sync() all disks, and finally reboot. As long as the disk and file system interfaces of the kernel are not damaged, this might be a good way for an almost clean shutdown.
is the final way out of disaster and almost the same as hitting the Big Red Button.
If you need a short command summary, simply type:
However, it is highly recommended to have a printed copy of the ddb(4) manual page ready for a debugging session. Remember that it is hard to read the on-line manual while single-stepping the kernel.