赞
踩
NAME
SYNOPSIS
bt [-a|-c cpu(s)|-g|-r|-t|-T|-l|-e|-E|-f|-F|-o|-O|-v|-p] [-R ref] [-s [-x|d]]
[-I ip] [-S sp] [pid | task]
DESCRIPTION
Display a kernel stack backtrace. If no arguments are given, the stack
trace of the current context will be displayed.
-a displays the stack traces of the active task on each CPU. (only applicable to crash dumps) -A same as -a, but also displays vector registers (S390X only). -p display the stack trace of the panic task only. (only applicable to crash dumps) -c cpu display the stack trace of the active task on one or more CPUs, which can be specified using the format "3", "1,8,9", "1-23", or "1,8,9-14". (only applicable to crash dumps) -g displays the stack traces of all threads in the thread group of the target task; the thread group leader will be displayed first. -r display raw stack data, consisting of a memory dump of the two pages of memory containing the task_union structure. -t display all text symbols found from the last known stack location to the top of the stack. (helpful if the back trace fails) -T display all text symbols found from just above the task_struct or thread_info to the top of the stack. (helpful if the back trace fails or the -t option starts too high in the process stack). -l show file and line number of each stack trace text location. -e search the stack for possible kernel and user mode exception frames. -E search the IRQ stacks (x86, x86_64, arm64, and ppc64), and the exception stacks (x86_64) for possible exception frames; all other arguments except for -c will be ignored since this is not a context- sensitive operation. -f display all stack data contained in a frame; this option can be used to determine the arguments passed to each function; on ia64, the argument register contents are dumped. -F[F] similar to -f, except that the stack data is displayed symbolically when appropriate; if the stack data references a slab cache object, the name of the slab cache will be displayed in brackets; on ia64, the substitution is done to the argument register contents. If -F is entered twice, and the stack data references a slab cache object, both the address and the name of the slab cache will be displayed in brackets. -v check the kernel stack of all tasks for evidence of stack overflows. It does so by verifying the thread_info.task pointer, ensuring that the thread_info.cpu is a valid cpu number, and checking the end of the stack for the STACK_END_MAGIC value. -o arm64: use optional backtrace method; not supported on Linux 4.14 or later kernels. x86: use old backtrace method, permissible only on kernels that were compiled without the -fomit-frame_pointer. x86_64: use old backtrace method, which dumps potentially stale kernel text return addresses found on the stack. -O arm64: use optional backtrace method by default; subsequent usage of this option toggles the backtrace method. x86: use old backtrace method by default, permissible only on kernels that were compiled without the -fomit-frame_pointer; subsequent usage of this option toggles the backtrace method. x86_64: use old backtrace method by default; subsequent usage of this option toggles the backtrace method. -R ref display stack trace only if there is a reference to this symbol or text address. -s display the symbol name plus its offset. -x when displaying a symbol offset with the -s option, override the default output format with hexadecimal format. -d when displaying a symbol offset with the -s option, override the default output format with decimal format. -I ip use ip as the starting text location. -S sp use sp as the starting stack frame address. pid displays the stack trace(s) of this pid. taskp displays the stack trace the the task referenced by this hexadecimal task_struct pointer.
Multiple pid and taskp arguments may be specified.
Note that all examples below are for x86 only. The output format will differ
for other architectures. x86 backtraces from kernels that were compiled
with the --fomit-frame-pointer CFLAG occasionally will drop stack frames,
or display a stale frame reference. When in doubt as to the accuracy of a
backtrace, the -t or -T options may help fill in the blanks.
EXAMPLES
Display the stack trace of the active task(s) when the kernel panicked:
crash> bt -a PID: 286 TASK: c0b3a000 CPU: 0 COMMAND: "in.rlogind" #0 [c0b3be90] crash_save_current_state at c011aed0 #1 [c0b3bea4] panic at c011367c #2 [c0b3bee8] tulip_interrupt at c01bc820 #3 [c0b3bf08] handle_IRQ_event at c010a551 #4 [c0b3bf2c] do_8259A_IRQ at c010a319 #5 [c0b3bf3c] do_IRQ at c010a653 #6 [c0b3bfbc] ret_from_intr at c0109634 EAX: 00000000 EBX: c0e68280 ECX: 00000000 EDX: 00000004 EBP: c0b3bfbc DS: 0018 ESI: 00000004 ES: 0018 EDI: c0e68284 CS: 0010 EIP: c012f803 ERR: ffffff09 EFLAGS: 00000246 #7 [c0b3bfbc] sys_select at c012f803 #8 [c0b3bfc0] system_call at c0109598 EAX: 0000008e EBX: 00000004 ECX: bfffc9a0 EDX: 00000000 DS: 002b ESI: bfffc8a0 ES: 002b EDI: 00000000 SS: 002b ESP: bfffc82c EBP: bfffd224 CS: 0023 EIP: 400d032e ERR: 0000008e EFLAGS: 00000246
Display the stack trace of the active task on CPU 0 and 1:
crash> bt -c 0,1 PID: 0 TASK: ffffffff81a8d020 CPU: 0 COMMAND: "swapper" #0 [ffff880002207e90] crash_nmi_callback at ffffffff8102fee6 #1 [ffff880002207ea0] notifier_call_chain at ffffffff8152d525 #2 [ffff880002207ee0] atomic_notifier_call_chain at ffffffff8152d58a #3 [ffff880002207ef0] notify_die at ffffffff810a155e #4 [ffff880002207f20] do_nmi at ffffffff8152b1eb #5 [ffff880002207f50] nmi at ffffffff8152aab0 [exception RIP: native_safe_halt+0xb] RIP: ffffffff8103eacb RSP: ffffffff81a01ea8 RFLAGS: 00000296 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff81de5228 RBP: ffffffff81a01ea8 R8: 0000000000000000 R9: 0000000000000000 R10: 0012099429a6bea3 R11: 0000000000000000 R12: ffffffff81c066c0 R13: 0000000000000000 R14: ffffffffffffffff R15: ffffffff81de1000 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 --- <NMI exception stack> --- #6 [ffffffff81a01ea8] native_safe_halt at ffffffff8103eacb #7 [ffffffff81a01eb0] default_idle at ffffffff810167bd #8 [ffffffff81a01ed0] cpu_idle at ffffffff81009fc6 PID: 38 TASK: ffff88003eaae040 CPU: 1 COMMAND: "khungtaskd" #0 [ffff88003ad97ce8] machine_kexec at ffffffff81038f3b #1 [ffff88003ad97d48] crash_kexec at ffffffff810c5da2 #2 [ffff88003ad97e18] panic at ffffffff8152721a #3 [ffff88003ad97e98] watchdog at ffffffff810e6346 #4 [ffff88003ad97ee8] kthread at ffffffff8109af06 #5 [ffff88003ad97f48] kernel_thread at ffffffff8100c20a
Display the stack traces of task f2814000 and PID 1592:
crash> bt f2814000 1592 PID: 1018 TASK: f2814000 CPU: 1 COMMAND: "java" #0 [f2815db4] schedule at c011af85 #1 [f2815de4] __down at c010600f #2 [f2815e14] __down_failed at c01061b3 #3 [f2815e24] stext_lock (via drain_cpu_caches) at c025fa55 #4 [f2815ec8] kmem_cache_shrink_nr at c013a53e #5 [f2815ed8] do_try_to_free_pages at c013f402 #6 [f2815f04] try_to_free_pages at c013f8d2 #7 [f2815f1c] _wrapped_alloc_pages at c01406bd #8 [f2815f40] __alloc_pages at c014079d #9 [f2815f60] __get_free_pages at c014083e #10 [f2815f68] do_fork at c011cebb #11 [f2815fa4] sys_clone at c0105ceb #12 [f2815fc0] system_call at c010740c EAX: 00000078 EBX: 00000f21 ECX: bc1ffbd8 EDX: bc1ffbe0 DS: 002b ESI: 00000000 ES: 002b EDI: bc1ffd04 SS: 002b ESP: 0807316c EBP: 080731bc CS: 0023 EIP: 4012881e ERR: 00000078 EFLAGS: 00000296 PID: 1592 TASK: c0cec000 CPU: 3 COMMAND: "httpd" #0 [c0ceded4] schedule at c011af85 #1 [c0cedf04] pipe_wait at c0153083 #2 [c0cedf58] pipe_read at c015317f #3 [c0cedf7c] sys_read at c0148be6 #4 [c0cedfc0] system_call at c010740c EAX: 00000003 EBX: 00000004 ECX: bffed4a3 EDX: 00000001 DS: 002b ESI: 00000001 ES: 002b EDI: bffed4a3 SS: 002b ESP: bffed458 EBP: bffed488 CS: 0023 EIP: 4024f1d4 ERR: 00000003 EFLAGS: 00000286
In order to examine each stack frame’s contents use the bt -f option.
From the extra frame data that is displayed, the arguments passed to each
function can be determined. Re-examining the PID 1592 trace above:
crash> bt -f 1592 PID: 1592 TASK: c0cec000 CPU: 3 COMMAND: "httpd" #0 [c0ceded4] schedule at c011af85 [RA: c0153088 SP: c0ceded4 FP: c0cedf04 SIZE: 52] c0ceded4: c0cedf00 c0cec000 ce1a6000 00000003 c0cedee4: c0cec000 f26152c0 cfafc8c0 c0cec000 c0cedef4: ef70a0a0 c0cec000 c0cedf28 c0cedf54 c0cedf04: c0153088 #1 [c0cedf04] pipe_wait at c0153083 [RA: c0153184 SP: c0cedf08 FP: c0cedf58 SIZE: 84] c0cedf08: 00000000 c0cec000 00000000 00000000 c0cedf18: 00000000 c0a41fa0 c011d38b c0394120 c0cedf28: 00000000 c0cec000 ceeebf30 ce4adf30 c0cedf38: 00000000 d4b60ce0 00000000 c0cedf58 c0cedf48: e204f820 ef70a040 00000001 c0cedf78 c0cedf58: c0153184 #2 [c0cedf58] pipe_read at c015317f [RA: c0148be8 SP: c0cedf5c FP: c0cedf7c SIZE: 36] c0cedf5c: ef70a040 c0cec000 00000000 00000000 c0cedf6c: 00000001 f27ae680 ffffffea c0cedfbc c0cedf7c: c0148be8 #3 [c0cedf7c] sys_read at c0148be6 [RA: c0107413 SP: c0cedf80 FP: c0cedfc0 SIZE: 68] c0cedf80: f27ae680 bffed4a3 00000001 f27ae6a0 c0cedf90: 40160370 24000000 4019ba28 00000000 c0cedfa0: 00000000 fffffffe bffba207 fffffffe c0cedfb0: c0cec000 00000001 bffed4a3 bffed488 c0cedfc0: c0107413 #4 [c0cedfc0] system_call at c010740c EAX: 00000003 EBX: 00000004 ECX: bffed4a3 EDX: 00000001 DS: 002b ESI: 00000001 ES: 002b EDI: bffed4a3 SS: 002b ESP: bffed458 EBP: bffed488 CS: 0023 EIP: 4024f1d4 ERR: 00000003 EFLAGS: 00000286 [RA: 4024f1d4 SP: c0cedfc4 FP: c0cedffc SIZE: 60] c0cedfc4: 00000004 bffed4a3 00000001 00000001 c0cedfd4: bffed4a3 bffed488 00000003 0000002b c0cedfe4: 0000002b 00000003 4024f1d4 00000023 c0cedff4: 00000286 bffed458 0000002b Typically the arguments passed to a function will be the last values that were pushed onto the stack by the next higher-numbered function, i.e., the lowest stack addresses in the frame above the called function's stack frame. That can be verified by disassembling the calling function. For example, the arguments passed from sys_read() to pipe_read() above are the file pointer, the user buffer address, the count, and a pointer to the file structure's f_pos field. Looking at the frame #3 data for sys_read(), the last four items pushed onto the stack (lowest addresses) are f27ae680, bffed4a3, 00000001, and f27ae6a0 -- which are the 4 arguments above, in that order. Note that the first (highest address) stack content in frame #2 data for pipe_read() is c0148be8, which is the return address back to sys_read().
Dump the text symbols found in the current context’s stack:
crash> bt -t
PID: 1357 TASK: c1aa0000 CPU: 0 COMMAND: "lockd"
START: schedule at c01190e0
[c1aa1f28] dput at c0157dbc
[c1aa1f4c] schedule_timeout at c0124cd4
[c1aa1f78] svc_recv at cb22c4d8 [sunrpc]
[c1aa1f98] put_files_struct at c011eb21
[c1aa1fcc] nlmclnt_proc at cb237bef [lockd]
[c1aa1ff0] kernel_thread at c0105826
[c1aa1ff8] nlmclnt_proc at cb237a60 [lockd]
Search the current stack for possible exception frames:
crash> bt -e
PID: 286 TASK: c0b3a000 CPU: 0 COMMAND: "in.rlogind"
KERNEL-MODE EXCEPTION FRAME AT c0b3bf44:
EAX: 00000000 EBX: c0e68280 ECX: 00000000 EDX: 00000004 EBP: c0b3bfbc
DS: 0018 ESI: 00000004 ES: 0018 EDI: c0e68284
CS: 0010 EIP: c012f803 ERR: ffffff09 EFLAGS: 00000246
USER-MODE EXCEPTION FRAME AT c0b3bfc4:
EAX: 0000008e EBX: 00000004 ECX: bfffc9a0 EDX: 00000000
DS: 002b ESI: bfffc8a0 ES: 002b EDI: 00000000
SS: 002b ESP: bfffc82c EBP: bfffd224
CS: 0023 EIP: 400d032e ERR: 0000008e EFLAGS: 00000246
Display the back trace from a dumpfile that resulted from the execution
of the crash utility’s “sys -panic” command:
crash> bt PID: 12523 TASK: c610c000 CPU: 0 COMMAND: "crash" #0 [c610de64] die at c01076ec #1 [c610de74] do_invalid_op at c01079bc #2 [c610df2c] error_code (via invalid_op) at c0107256 EAX: 0000001d EBX: c024a4c0 ECX: c02f13c4 EDX: 000026f6 EBP: c610c000 DS: 0018 ESI: 401de2e0 ES: 0018 EDI: c610c000 CS: 0010 EIP: c011bbb4 ERR: ffffffff EFLAGS: 00010296 #3 [c610df68] panic at c011bbb4 #4 [c610df78] do_exit at c011f1fe #5 [c610dfc0] system_call at c0107154 EAX: 00000001 EBX: 00000000 ECX: 00001000 EDX: 401df154 DS: 002b ESI: 401de2e0 ES: 002b EDI: 00000000 SS: 002b ESP: bffebf0c EBP: bffebf38 CS: 0023 EIP: 40163afd ERR: 00000001 EFLAGS: 00000246 Display the back trace from a dumpfile that resulted from an attempt to insmod the sample "crash.c" kernel module that comes as part of the Red Hat netdump package: crash> bt PID: 1696 TASK: c74de000 CPU: 0 COMMAND: "insmod" #0 [c74dfdcc] die at c01076ec #1 [c74dfddc] do_page_fault at c0117bbc #2 [c74dfee0] error_code (via page_fault) at c0107256 EAX: 00000013 EBX: cb297000 ECX: 00000000 EDX: c5962000 EBP: c74dff28 DS: 0018 ESI: 00000000 ES: 0018 EDI: 00000000 CS: 0010 EIP: cb297076 ERR: ffffffff EFLAGS: 00010282 #3 [c74dff1c] crash_init at cb297076 [crash] #4 [c74dff2c] sys_init_module at c011d233 #5 [c74dffc0] system_call at c0107154 EAX: 00000080 EBX: 08060528 ECX: 08076450 EDX: 0000000a DS: 002b ESI: 0804b305 ES: 002b EDI: 08074ed0 SS: 002b ESP: bffe9a90 EBP: bffe9ac8 CS: 0023 EIP: 4012066e ERR: 00000080 EFLAGS: 00000246
Display the symbol name plus its offset in each frame, overriding
the current output format with hexadecimal:
crash> bt -sx PID: 1499 TASK: ffff88006af43cc0 CPU: 2 COMMAND: "su" #0 [ffff8800664a1c90] machine_kexec+0x167 at ffffffff810327b7 #1 [ffff8800664a1ce0] crash_kexec+0x60 at ffffffff810a9ec0 #2 [ffff8800664a1db0] oops_end+0xb0 at ffffffff81504160 #3 [ffff8800664a1dd0] general_protection+0x25 at ffffffff81503435 [exception RIP: kmem_cache_alloc+120] RIP: ffffffff8113cf88 RSP: ffff8800664a1e88 RFLAGS: 00010086 RAX: 0000000000000000 RBX: ff88006ef56840ff RCX: ffffffff8114e9e4 RDX: 0000000000000000 RSI: 00000000000080d0 RDI: ffffffff81796020 RBP: ffffffff81796020 R8: ffff88000a3137a0 R9: 0000000000000000 R10: ffff88007ac97300 R11: 0000000000000400 R12: 00000000000080d0 R13: 0000000000000292 R14: 00000000000080d0 R15: 00000000000000c0 ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018 #4 [ffff8800664a1ed0] get_empty_filp+0x74 at ffffffff8114e9e4 #5 [ffff8800664a1ef0] sock_alloc_fd+0x23 at ffffffff8142f553 #6 [ffff8800664a1f10] sock_map_fd+0x23 at ffffffff8142f693 #7 [ffff8800664a1f50] sys_socket+0x43 at ffffffff814302a3 #8 [ffff8800664a1f80] system_call_fastpath+0x16 at ffffffff81013042 RIP: 00007f5720b368e7 RSP: 00007fff52b629a8 RFLAGS: 00010206 RAX: 0000000000000029 RBX: ffffffff81013042 RCX: 0000000000000000 RDX: 0000000000000009 RSI: 0000000000000003 RDI: 0000000000000010 RBP: 000000000066f320 R8: 0000000000000001 R9: 0000000000000000 R10: 0000000000000000 R11: 0000000000000202 R12: ffff88007ac97300 R13: 0000000000000000 R14: 00007f571e104a80 R15: 00007f571e305048 ORIG_RAX: 0000000000000029 CS: 0033 SS: 002b
The following three examples show the difference in the display of
the same stack frame’s contents using -f, -F, and -FF:
crash> bf -f ... #4 [ffff810072b47f10] vfs_write at ffffffff800789d8 ffff810072b47f18: ffff81007e020380 ffff81007e2c2880 ffff810072b47f28: 0000000000000002 fffffffffffffff7 ffff810072b47f38: 00002b141825d000 ffffffff80078f75 #5 [ffff810072b47f40] sys_write at ffffffff80078f75 ... crash> bt -F ... #4 [ffff810072b47f10] vfs_write at ffffffff800789d8 ffff810072b47f18: [files_cache] [filp] ffff810072b47f28: 0000000000000002 fffffffffffffff7 ffff810072b47f38: 00002b141825d000 sys_write+69 #5 [ffff810072b47f40] sys_write at ffffffff80078f75 ... crash> bf -FF ... #4 [ffff810072b47f10] vfs_write at ffffffff800789d8 ffff810072b47f18: [ffff81007e020380:files_cache] [ffff81007e2c2880:filp] ffff810072b47f28: 0000000000000002 fffffffffffffff7 ffff810072b47f38: 00002b141825d000 sys_write+69 #5 [ffff810072b47f40] sys_write at ffffffff80078f75 ...
Check the kernel stack of all tasks for evidence of a stack overflow:
crash> bt -v
PID: 5823 TASK: ffff88102aae0040 CPU: 1 COMMAND: "flush-253:0"
possible stack overflow: thread_info.task: 102efb5adc0 != ffff88102aae0040
possible stack overflow: 40ffffffff != STACK_END_MAGIC
///
NAME
SYNOPSIS
gdb command …
DESCRIPTION
This command passes its arguments directly to gdb for processing.
This is typically not necessary, but where ambiguities between crash and
gdb command names exist, this will force the command to be executed by gdb.
Alternatively, if “set gdb on” is entered, the session will be run in a
mode where all commands are passed directly to gdb. When running in that
mode, native crash commands may be executed by preceding them with the
“crash” directive. To restore native crash mode, enter “set gdb off”.
EXAMPLES
crash> gdb help List of classes of commands: aliases -- Aliases of other commands breakpoints -- Making program stop at certain points data -- Examining data files -- Specifying and examining files internals -- Maintenance commands obscure -- Obscure features running -- Running the program stack -- Examining the stack status -- Status inquiries support -- Support facilities tracepoints -- Tracing of program execution without stopping the program user-defined -- User-defined commands Type "help" followed by a class name for a list of commands in that class. Type "help" followed by command name for full documentation. Command name abbreviations are allowed if unambiguous.
///
NAME
SYNOPSIS
net [[-s | -S] [-xd] [-R ref] [pid | task]] [-a] [ -n [pid | task]] [-N addr]
DESCRIPTION
Displays various network related data.
If no arguments are entered, the list of network devices, names and IP
addresses are displayed. For kernels supporting namespaces, the -n option
may be used to display the list of network devices with respect to the
network namespace of a current context or a task specified by pid or task:
-n the namespace of the current context.
-n pid a process PID.
-n task a hexadecimal task_struct pointer.
The -s and -S options display data with respect to the current context, but
may be appended with an argument to show the socket data with respect
to a specified task:
-s display open network socket/sock addresses, their family and type,
and for INET and INET6 families, their source and destination
addresses and ports.
-s pid same as above, for task with process PID pid.
-s task same as above, for task with hexadecimal task_struct pointer task.
-S displays open network socket/sock addresses followed by a dump
of both data structures.
-S pid same as above, with respect to process PID.
-S task same as above, with respect to hexadecimal task_struct pointer.
The -R option, typically invoked from “foreach net”, and in conjunction
with the -s or -S options, searches for references to a socket address,
sock address, or a file descriptor; if found, only the referenced fd, socket
or sock data will be displayed:
-R ref socket or sock address, or file descriptor.
Other options:
-a display the ARP cache.
-N addr translates an IPv4 address expressed as a decimal or hexadecimal
value into a standard numbers-and-dots notation.
-x override default output format with hexadecimal format.
-d override default output format with decimal format.
EXAMPLES
Display the system’s network device list:
crash> net
NET_DEVICE NAME IP ADDRESS(ES)
ffff8803741c0000 lo 127.0.0.1
fff88037059c0000 eth0 10.226.229.141
ffff8803705c0000 eth1 10.226.228.250
ffff880374ad6000 usb0 169.254.95.120
Display the network device list with respect to the network namespace
of PID 2618:
crash> net -n 2618
NET_DEVICE NAME IP ADDRESS(ES)
ffff880456ee7020 lo 127.0.0.1
ffff8804516a1020 eth0 10.1.9.223
Dump the ARP cache:
crash> net -a
NEIGHBOUR IP ADDRESS HW TYPE HW ADDRESS DEVICE STATE
f38d1b00 10.16.64.14 ETHER 00:16:3e:4b:a5:4a eth1 STALE
f38d1080 0.0.0.0 UNKNOWN 00 00 00 00 00 00 lo NOARP
f38d1bc0 10.16.71.254 ETHER 00:00:0c:07:ac:00 eth1 REACHABLE
f38d1200 10.16.64.21 ETHER 00:16:3e:51:d8:09 eth1 REACHABLE
Display the sockets for PID 2517, using both -s and -S output formats:
crash> net -s 2517 PID: 2517 TASK: c1598000 CPU: 1 COMMAND: "rlogin" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 3 c57375dc c1ff1850 INET:STREAM 10.1.8.20-1023 10.1.16.62-513 crash> net -S 2517 PID: 2517 TASK: c1598000 CPU: 1 COMMAND: "rlogin" FD SOCKET SOCK 3 c57375dc c1ff1850 struct socket { state = SS_CONNECTED, flags = 131072, ops = 0xc023f820, inode = 0xc5737540, fasync_list = 0x0, file = 0xc58892b0, sk = 0xc1ff1850, wait = 0xc14d9ed4, type = 1, passcred = 0 '\000', tli = 0 '\000' } struct sock { sklist_next = 0xc1ff12f0, sklist_prev = 0xc216bc00, bind_next = 0x0, bind_pprev = 0xc0918448, daddr = 1041236234, rcv_saddr = 336068874, dport = 258, num = 1023, bound_dev_if = 0, next = 0x0, pprev = 0xc0286dd4, state = 1 '\001', zapped = 0 '\000', sport = 65283, family = 2, reuse = 0 '\000', ...
Translate the rcv_saddr from above into dotted-decimal notation:
crash> net -N 1041236234
10.1.16.62
From “foreach”, find all tasks with references to socket c08ea3cc:
crash> foreach net -s -R c08ea3cc PID: 2184 TASK: c7026000 CPU: 1 COMMAND: "klines.kss" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2200 TASK: c670a000 CPU: 1 COMMAND: "kpanel" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2201 TASK: c648a000 CPU: 1 COMMAND: "kbgndwm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 19294 TASK: c250a000 CPU: 0 COMMAND: "prefdm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2194 TASK: c62dc000 CPU: 1 COMMAND: "kaudioserver" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2195 TASK: c6684000 CPU: 1 COMMAND: "maudio" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2196 TASK: c6b58000 CPU: 1 COMMAND: "kwmsound" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2197 TASK: c6696000 CPU: 0 COMMAND: "kfm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2199 TASK: c65ec000 CPU: 0 COMMAND: "krootwm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 694 TASK: c1942000 CPU: 0 COMMAND: "prefdm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 698 TASK: c6a2c000 CPU: 1 COMMAND: "X" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0 PID: 2159 TASK: c4a5a000 CPU: 1 COMMAND: "kwm" FD SOCKET SOCK FAMILY:TYPE SOURCE-PORT DESTINATION-PORT 5 c08ea3cc c50d3c80 INET:STREAM 0.0.0.0-1026 0.0.0.0-0
///
NAME
SYNOPSIS
set [[-a] [pid | taskp] | [-c cpu] | -p] | [crash_variable [setting]] | -v
DESCRIPTION
This command either sets a new context, or gets the current context for
display. The context can be set by the use of:
pid a process PID.
taskp a hexadecimal task_struct pointer.
-a sets the pid or task as the active task on its cpu (dumpfiles only).
-c cpu sets the context to the active task on a cpu (dumpfiles only).
-p sets the context to the panic task, or back to the crash task on
a live system.
-v display the current state of internal crash variables.
If no argument is entered, the current context is displayed. The context
consists of the PID, the task pointer, the CPU, and task state. The task
state shows the bits found in both the task_struct state and exit_state
fields.
This command may also be used to set internal crash variables. If no value
argument is entered, the current value of the crash variable is shown. These
are the crash variables, acceptable arguments, and purpose:
scroll on | off controls output scrolling. scroll less /usr/bin/less as the output scrolling program. scroll more /bin/more as the output scrolling program. scroll CRASHPAGER use CRASHPAGER environment variable as the output scrolling program. radix 10 | 16 sets output radix to 10 or 16. refresh on | off controls internal task list refresh. print_max number set maximum number of array elements to print. print_array on | off if on, set gdb's printing of arrays to "pretty" format, with one line per element. console device-name sets debug console device. debug number sets crash debug level. core on | off if on, drops core when the next error message is displayed. hash on | off controls internal list verification. silent on | off turns off initialization messages; turns off crash prompt during input file execution. (scrolling is turned off if silent is on) edit vi | emacs set line editing mode (from .crashrc file only). namelist filename name of kernel (from .crashrc file only). zero_excluded on | off controls whether excluded pages, or pages that are missing from an incomplete dumpfile, should return zero-filled memory when read. null-stop on | off if on, gdb's printing of character arrays will stop at the first NULL encountered. gdb on | off if on, the crash session will be run in a mode where all commands will be passed directly to gdb, and the command prompt will change to "gdb>"; when running in this mode, native crash commands may be executed by preceding them with the "crash" directive. scope text-addr sets the text scope for viewing the definition of data structures; the "text-addr" argument must be a kernel or module text address, which may be expressed symbolically or as a hexadecimal value. offline show | hide show or hide command output that is associated with offline cpus. redzone on | off if on, CONFIG_SLUB object addresses displayed by the kmem command will point to the SLAB_RED_ZONE padding inserted at the beginning of the object. error default | redirect | filename set the destination of error messages. "default": error messages are always displayed on the console; if the output of a command is piped to an external command or redirected to a file, the error messages are also sent to the pipe or file. "redirect": if the output of a command is piped to an external command or redirected to a file, error messages are only sent to the pipe or file; otherwise they are displayed on the console. "filename": error messages are only sent to the specified filename; they are not displayed on the console and are not sent to a pipe or file.
Internal variables may be set in four manners:
1. entering the set command in $HOME/.crashrc.
2. entering the set command in .crashrc in the current directory.
3. executing an input file containing the set command.
4. during runtime with this command.
During initialization, $HOME/.crashrc is read first, followed by the
.crashrc file in the current directory. Set commands in the .crashrc file
in the current directory override those in $HOME/.crashrc. Set commands
entered with this command or by runtime input file override those
defined in either .crashrc file. Multiple set command arguments or argument
pairs may be entered in one command line.
EXAMPLES
Set the current context to task c2fe8000:
crash> set c2fe8000
PID: 15917
COMMAND: "bash"
TASK: c2fe8000
CPU: 0
STATE: TASK_INTERRUPTIBLE
Set the context back to the panicking task:
crash> set -p
PID: 698
COMMAND: "gen12"
TASK: f9d78000
CPU: 2
STATE: TASK_RUNNING (PANIC)
Turn off output scrolling:
crash> set scroll off
scroll: off (/usr/bin/less)
Show the current state of crash internal variables:
crash> set -v scroll: on (/usr/bin/less) radix: 10 (decimal) refresh: on print_max: 256 print_array: off console: /dev/pts/2 debug: 0 core: off hash: on silent: off edit: vi namelist: vmlinux zero_excluded: off null-stop: on gdb: off scope: (not set) offline: show redzone: on error: default
Show the current context:
crash> set
PID: 1525
COMMAND: "bash"
TASK: c1ede000
CPU: 0
STATE: TASK_INTERRUPTIBLE
///
NAME
SYNOPSIS
vm [-p | -P vma | -M mm | -v | -m | -x | -d | [-R reference] [pid | task]]
[-f vm_flags]
DESCRIPTION
This command displays basic virtual memory information of a context,
consisting of a pointer to its mm_struct and page dirctory, its RSS and
total virtual memory size; and a list of pointers to each vm_area_struct,
its starting and ending address, vm_flags value, and file pathname. If no
arguments are entered, the current context is used. Additionally, the -p
option translates each virtual page of each VM area to its physical address.
The -R option, typically invoked from “foreach vm”, searches for references
to a supplied number, address, or filename argument, and prints only the
essential information leading up to and including the reference.
Alternatively, the -m or -v options may be used to dump the task’s mm_struct
or all of its vm_area_structs respectively. The -p, -v, -m, -R and -f
options are all mutually exclusive.
-p translate each virtual page to its physical address, or if the page is not mapped, its swap device and offset, or filename and offset. -P vma similar to -p, but only translate the pages belonging to the specified VM area of a context. -M mm if the mm_struct address has been removed from the task_struct of an exiting task, the virtual memory data cannot be displayed. However, if the address can be determined from the kernel stack, it can be entered manually in order to try to resurrect the virtual memory data of the task. -R reference search for references to this number or filename. -m dump the mm_struct assocated with the task. -v dump all of the vm_area_structs associated with the task. -x override the default output format for the -m or -v options with hexadecimal format. -d override the default output format for the -m or -v options with decimal format. -f vm_flags translate the bits of a FLAGS (vm_flags) value. pid a process PID. task a hexadecimal task_struct pointer.
EXAMPLES
Display the virtual memory data of the current context:
crash> vm PID: 30986 TASK: c0440000 CPU: 0 COMMAND: "bash" MM PGD RSS TOTAL_VM c303fe20 c4789000 88k 1728k VMA START END FLAGS FILE c0d1f540 8048000 80ad000 1875 /bin/bash c0d1f400 80ad000 80b3000 1873 /bin/bash c0d1f880 80b3000 80ec000 77 c0d1f0c0 40000000 40012000 875 /lib/ld-2.1.1.so c0d1f700 40012000 40013000 873 /lib/ld-2.1.1.so c0d1fe00 40013000 40014000 77 c0d1f580 40014000 40016000 73 c0d1f280 4001a000 4004b000 75 /usr/lib/libncurses.so.4.2 c0d1f100 4004b000 40054000 73 /usr/lib/libncurses.so.4.2 c0d1f600 40054000 40057000 73 c0d1f9c0 40057000 40059000 75 /lib/libdl-2.1.1.so c0d1f800 40059000 4005a000 73 /lib/libdl-2.1.1.so c0d1fd00 4005a000 40140000 75 /lib/libc-2.1.1.so c0d1fe40 40140000 40145000 73 /lib/libc-2.1.1.so c0d1f780 40145000 40148000 73 c0d1f140 40148000 40150000 75 /lib/libnss_files-2.1.1.so c0d1fa80 40150000 40151000 73 /lib/libnss_files-2.1.1.so c0d1fb00 40151000 4015a000 75 /lib/libnss_nisplus-2.1.1.so c5f754e0 4015a000 4015b000 73 /lib/libnss_nisplus-2.1.1.so c0d1fec0 4015b000 4016d000 75 /lib/libnsl-2.1.1.so c5f75460 4016d000 4016e000 73 /lib/libnsl-2.1.1.so c5f75420 4016e000 40170000 73 c5f753e0 40170000 40178000 75 /lib/libnss_nis-2.1.1.so c5f753a0 40178000 40179000 73 /lib/libnss_nis-2.1.1.so c0d1f240 bfffc000 c0000000 177
Display the virtual memory data along with page translations for PID 386:
crash> vm -p 386 PID: 386 TASK: c11cc000 CPU: 0 COMMAND: "atd" MM PGD RSS TOTAL_VM c7e30560 c10e5000 104k 1112k VMA START END FLAGS FILE c0fbe6a0 8048000 804b000 1875 /usr/sbin/atd VIRTUAL PHYSICAL 8048000 20e1000 8049000 17c6000 804a000 1f6f000 VMA START END FLAGS FILE c61e0ba0 804b000 804d000 1873 /usr/sbin/atd VIRTUAL PHYSICAL 804b000 254d000 804c000 6a9c000 VMA START END FLAGS FILE c61e04e0 804d000 8050000 77 VIRTUAL PHYSICAL 804d000 219d000 804e000 2617000 804f000 SWAP: /dev/sda8 OFFSET: 24225 VMA START END FLAGS FILE c61e0720 40000000 40012000 875 /lib/ld-2.1.1.so VIRTUAL PHYSICAL 40000000 FILE: /lib/ld-2.1.1.so OFFSET: 0 40001000 FILE: /lib/ld-2.1.1.so OFFSET: 1000 40002000 FILE: /lib/ld-2.1.1.so OFFSET: 2000 40003000 FILE: /lib/ld-2.1.1.so OFFSET: 3000 40004000 FILE: /lib/ld-2.1.1.so OFFSET: 4000 40005000 FILE: /lib/ld-2.1.1.so OFFSET: 5000 ...
Although the -R option is typically invoked from “foreach vm”, it can be
executed directly. This example displays all VM areas with vm_flags of 75:
crash> vm -R 75
PID: 694 TASK: c0c76000 CPU: 1 COMMAND: "crash"
MM PGD RSS TOTAL_VM
c6c43110 c0fe9000 8932k 10720k
VMA START END FLAGS FILE
c322c0d0 40019000 4004a000 75 /usr/lib/libncurses.so.4.2
c67537c0 40056000 40071000 75 /lib/libm-2.1.1.so
c6753d00 40072000 40074000 75 /lib/libdl-2.1.1.so
c6753540 40075000 40081000 75 /usr/lib/libz.so.1.1.3
c6753740 40085000 4016b000 75 /lib/libc-2.1.1.so
One reason to use -R directly is to pare down the output associated with
the -p option on a task with a huge address space. This example displays
the page data associated with virtual address 40121000:
crash> vm -R 40121000
PID: 694 TASK: c0c76000 CPU: 0 COMMAND: "crash"
MM PGD RSS TOTAL_VM
c6c43110 c0fe9000 8928k 10720k
VMA START END FLAGS FILE
c6753740 40085000 4016b000 75 /lib/libc-2.1.1.so
VIRTUAL PHYSICAL
40121000 FILE: /lib/libc-2.1.1.so OFFSET: 9c000
Display the mm_struct for PID 4777:
crash> vm -m 4777 PID: 4777 TASK: c0896000 CPU: 0 COMMAND: "bash" struct mm_struct { mmap = 0xc6caa1c0, mmap_avl = 0x0, mmap_cache = 0xc6caabc0, pgd = 0xc100a000, count = { counter = 0x1 }, map_count = 0x14, mmap_sem = { count = { counter = 0x1 }, waking = 0x0, wait = 0x0 }, context = 0x0, start_code = 0x8048000, end_code = 0x809c6f7, start_data = 0x0, end_data = 0x80a2090, start_brk = 0x80a5420, brk = 0x80b9000, start_stack = 0xbffff9d0, arg_start = 0xbffffad1, arg_end = 0xbffffad7, env_start = 0xbffffad7, env_end = 0xbffffff2, rss = 0xf6, total_vm = 0x1a3, locked_vm = 0x0, def_flags = 0x0, cpu_vm_mask = 0x0, swap_cnt = 0x23d, swap_address = 0x0, segments = 0x0 }
Display all of the vm_area_structs for task c47d4000:
crash> vm -v c47d4000 PID: 4971 TASK: c47d4000 CPU: 1 COMMAND: "login" struct vm_area_struct { vm_mm = 0xc4b0d200, vm_start = 0x8048000, vm_end = 0x804d000, vm_next = 0xc3e3abd0, vm_page_prot = { pgprot = 0x25 }, vm_flags = 0x1875, vm_avl_height = 0x1, vm_avl_left = 0x0, vm_avl_right = 0x0, vm_next_share = 0x0, vm_pprev_share = 0xc3e3abf0, vm_ops = 0xc02392a0, vm_offset = 0x0, vm_file = 0xc1e23660, vm_pte = 0x0 } struct vm_area_struct { vm_mm = 0xc4b0d200, vm_start = 0x804d000, vm_end = 0x804e000, vm_next = 0xc3e3a010, vm_page_prot = { pgprot = 0x25 }, vm_flags = 0x1873, vm_avl_height = 0x2, vm_avl_left = 0xc3e3a810, vm_avl_right = 0xc3e3a010, vm_next_share = 0xc3e3a810, vm_pprev_share = 0xc3699c14 ...
Translate a FLAGS value:
crash> vm -f 3875
3875: (READ|EXEC|MAYREAD|MAYWRITE|MAYEXEC|DENYWRITE|EXECUTABLE|LOCKED)
Display the page translations of the VM area at address f5604f2c:
crash> vm -P f5604f2c PID: 5508 TASK: f56a9570 CPU: 0 COMMAND: "crond" VMA START END FLAGS FILE f5604f2c f5b000 f67000 8000075 /lib/libnss_files-2.12.so VIRTUAL PHYSICAL f5b000 3fec1000 f5c000 3d3a4000 f5d000 FILE: /lib/libnss_files-2.12.so OFFSET: 2000 f5e000 FILE: /lib/libnss_files-2.12.so OFFSET: 3000 f5f000 FILE: /lib/libnss_files-2.12.so OFFSET: 4000 f60000 3fd31000 f61000 3fd32000 f62000 FILE: /lib/libnss_files-2.12.so OFFSET: 7000 f63000 FILE: /lib/libnss_files-2.12.so OFFSET: 8000 f64000 3ff35000 f65000 FILE: /lib/libnss_files-2.12.so OFFSET: a000 f66000 FILE: /lib/libnss_files-2.12.so OFFSET: b000
///
NAME
SYNOPSIS
btop address …
DESCRIPTION
This command translates a hexadecimal address to its page number.
EXAMPLES
crash> btop 512a000
512a000: 512a
///
NAME
SYNOPSIS
help [command | all] [-]
DESCRIPTION
When entered with no argument, a list of all currently available crash
commands is listed. If a name of a crash command is entered, a man-like
page for the command is displayed. If “all” is entered, help pages
for all commands will be displayed. If neither of the above is entered,
the argument string will be passed on to the gdb help command.
A number of internal debug, statistical, and other dumpfile related
data is available with the following options:
-a - alias data -b - shared buffer data -B - build data -c - numargs cache -d - device table -D - dumpfile contents/statistics -e - extension table data -f - filesys table -g - gdb data -h - hash_table data -H - hash_table data (verbose) -k - kernel_table -K - kernel_table (verbose) -L - LKCD page cache environment -M <num> machine specific -m - machdep_table -N - net_table -n - dumpfile contents/statistics -o - offset_table and size_table -p - program_context -r - dump registers from dumpfile header -s - symbol table data -t - task_table -T - task_table plus context_array -v - vm_table -V - vm_table (verbose) -x - text cache -z - help options
///
NAME
SYNOPSIS
p [-x|-d][-u] [expression | symbol[:cpuspec]]
DESCRIPTION
This command passes its arguments on to gdb “print” command for evaluation.
expression an expression to be evaluated.
symbol a kernel symbol.
:cpuspec CPU specification for a per-cpu symbol:
: CPU of the currently selected task.
:a[ll] all CPUs.
:#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3",
or "1,3,5-7,10".
-x override default output format with hexadecimal format.
-d override default output format with decimal format.
-u the expression evaluates to a user address reference.
The default output format is decimal, but that can be changed at any time
with the two built-in aliases “hex” and “dec”. Alternatively, there
are two other built-in aliases, “px” and “pd”, which force the command
output to be displayed in hexadecimal or decimal, without changing the
default mode.
EXAMPLES
Print the contents of jiffies:
crash> p jiffies
jiffies = $6 = 166532620
crash> px jiffies
jiffies = $7 = 0x9ed174b
crash> pd jiffies
jiffies = $8 = 166533160
Print the contents of the vm_area_struct “init_mm”:
crash> p init_mm init_mm = $5 = { mmap = 0xc022d540, mmap_avl = 0x0, mmap_cache = 0x0, pgd = 0xc0101000, count = { counter = 0x6 }, map_count = 0x1, mmap_sem = { count = { counter = 0x1 }, waking = 0x0, wait = 0x0 }, context = 0x0, start_code = 0xc0000000, end_code = 0xc022b4c8, start_data = 0x0, end_data = 0xc0250388, start_brk = 0x0, brk = 0xc02928d8, start_stack = 0x0, arg_start = 0x0, arg_end = 0x0, env_start = 0x0, env_end = 0x0, rss = 0x0, total_vm = 0x0, locked_vm = 0x0, def_flags = 0x0, cpu_vm_mask = 0x0, swap_cnt = 0x0, swap_address = 0x0, segments = 0x0 }
If a per-cpu symbol is entered as a argument, its data type
and all of its per-cpu addresses are displayed:
crash> p irq_stat
PER-CPU DATA TYPE:
irq_cpustat_t irq_stat;
PER-CPU ADDRESSES:
[0]: ffff88021e211540
[1]: ffff88021e251540
[2]: ffff88021e291540
[3]: ffff88021e2d1540
To display the contents a per-cpu symbol for CPU 1, append
a cpu-specifier:
crash> p irq_stat:1 per_cpu(irq_stat, 1) = $29 = { __softirq_pending = 0, __nmi_count = 209034, apic_timer_irqs = 597509876, irq_spurious_count = 0, icr_read_retry_count = 2, x86_platform_ipis = 0, apic_perf_irqs = 209034, apic_irq_work_irqs = 0, irq_resched_count = 264922233, irq_call_count = 7036692, irq_tlb_count = 4750442, irq_thermal_count = 0, irq_threshold_count = 0 }
///
NAME
SYNOPSIS
sig [[-l] | [-s sigset]] | [-g] [pid | taskp] …
DESCRIPTION
This command displays signal-handling data of one or more tasks. Multiple
task or PID numbers may be entered; if no arguments are entered, the signal
handling data of the current context will be displayed. The default display
shows:
1. A formatted dump of the "sig" signal_struct structure referenced by
the task_struct. For each defined signal, it shows the sigaction
structure address, the signal handler, the signal sigset_t mask
(also expressed as a 64-bit hexadecimal value), and the flags.
2. Whether the task has an unblocked signal pending.
3. The contents of the "blocked" and "signal" sigset_t structures
from the task_struct/signal_struct, both of which are represented
as a 64-bit hexadecimal value.
4. For each queued signal, private and/or shared, if any, its signal
number and associated siginfo structure address.
The -l option lists the signal numbers and their name(s). The -s option
translates a 64-bit hexadecimal value representing the contents of a
sigset_t structure into the signal names whose bits are set.
pid a process PID.
taskp a hexadecimal task_struct pointer.
-g displays signal information for all threads in a task's
thread group.
-l displays the defined signal numbers and names.
-s sigset translates a 64-bit hexadecimal value representing a sigset_t
into a list of signal names associated with the bits set.
EXAMPLES
Dump the signal-handling data of PID 8970:
crash> sig 8970 PID: 8970 TASK: f67d8560 CPU: 1 COMMAND: "procsig" SIGNAL_STRUCT: f6018680 COUNT: 1 SIG SIGACTION HANDLER MASK FLAGS [1] f7877684 SIG_DFL 0000000000000000 0 [2] f7877698 SIG_DFL 0000000000000000 0 ... [8] f7877710 SIG_DFL 0000000000000000 0 [9] f7877724 SIG_DFL 0000000000000000 0 [10] f7877738 804867a 0000000000000000 80000000 (SA_RESETHAND) [11] f787774c SIG_DFL 0000000000000000 0 [12] f7877760 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART) [13] f7877774 SIG_DFL 0000000000000000 0 ... [31] f78778dc SIG_DFL 0000000000000000 0 [32] f78778f0 SIG_DFL 0000000000000000 0 [33] f7877904 SIG_DFL 0000000000000000 0 [34] f7877918 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART) [35] f787792c SIG_DFL 0000000000000000 0 [36] f7877940 SIG_DFL 0000000000000000 0 ... [58] f7877af8 SIG_DFL 0000000000000000 0 [59] f7877b0c SIG_DFL 0000000000000000 0 [60] f7877b20 SIG_DFL 0000000000000000 0 [61] f7877b34 SIG_DFL 0000000000000000 0 [62] f7877b48 SIG_DFL 0000000000000000 0 [63] f7877b5c SIG_DFL 0000000000000000 0 [64] f7877b70 804867f 0000000000000000 10000004 (SA_SIGINFO|SA_RESTART) SIGPENDING: no BLOCKED: 8000000200000800 PRIVATE_PENDING SIGNAL: 0000000200000800 SIGQUEUE: SIG SIGINFO 12 f51b9c84 34 f51b9594 SHARED_PENDING SIGNAL: 8000000000000800 SIGQUEUE: SIG SIGINFO 12 f51b9188 64 f51b9d18 64 f51b9500
Dump the signal-handling data for all tasks in the thread group containing
PID 2578:
crash> sig -g 2578 PID: 2387 TASK: f617d020 CPU: 0 COMMAND: "slapd" SIGNAL_STRUCT: f7dede00 COUNT: 6 SIG SIGACTION HANDLER MASK FLAGS [1] c1f60c04 a258a7 0000000000000000 10000000 (SA_RESTART) [2] c1f60c18 a258a7 0000000000000000 10000000 (SA_RESTART) [3] c1f60c2c SIG_DFL 0000000000000000 0 [4] c1f60c40 SIG_DFL 0000000000000000 0 [5] c1f60c54 a258a7 0000000000000000 10000000 (SA_RESTART) [6] c1f60c68 SIG_DFL 0000000000000000 0 [7] c1f60c7c SIG_DFL 0000000000000000 0 [8] c1f60c90 SIG_DFL 0000000000000000 0 [9] c1f60ca4 SIG_DFL 0000000000000000 0 [10] c1f60cb8 a25911 0000000000000000 10000000 (SA_RESTART) ... [64] c1f610f0 SIG_DFL 0000000000000000 0 SHARED_PENDING SIGNAL: 0000000000000000 SIGQUEUE: (empty) PID: 2387 TASK: f617d020 CPU: 0 COMMAND: "slapd" SIGPENDING: no BLOCKED: 0000000000000000 PRIVATE_PENDING SIGNAL: 0000000000000000 SIGQUEUE: (empty) PID: 2392 TASK: f6175aa0 CPU: 0 COMMAND: "slapd" SIGPENDING: no BLOCKED: 0000000000000000 PRIVATE_PENDING SIGNAL: 0000000000000000 SIGQUEUE: (empty) PID: 2523 TASK: f7cd4aa0 CPU: 1 COMMAND: "slapd" SIGPENDING: no BLOCKED: 0000000000000000 PRIVATE_PENDING SIGNAL: 0000000000000000 SIGQUEUE: (empty) ...
Translate the sigset_t mask value, cut-and-pasted from the signal handling
data from signals 1 and 10 above:
crash> sig -s 800A000000000201
SIGHUP SIGUSR1 SIGRTMAX-14 SIGRTMAX-12 SIGRTMAX
List the signal numbers and their names:
crash> sig -l [1] SIGHUP [2] SIGINT [3] SIGQUIT [4] SIGILL [5] SIGTRAP [6] SIGABRT/SIGIOT [7] SIGBUS [8] SIGFPE [9] SIGKILL [10] SIGUSR1 [11] SIGSEGV [12] SIGUSR2 [13] SIGPIPE [14] SIGALRM [15] SIGTERM [16] SIGSTKFLT [17] SIGCHLD/SIGCLD [18] SIGCONT [19] SIGSTOP [20] SIGTSTP [21] SIGTTIN [22] SIGTTOU [23] SIGURG [24] SIGXCPU [25] SIGXFSZ [26] SIGVTALRM [27] SIGPROF [28] SIGWINCH [29] SIGIO/SIGPOLL [30] SIGPWR [31] SIGSYS [32] SIGRTMIN [33] SIGRTMIN+1 [34] SIGRTMIN+2 [35] SIGRTMIN+3 [36] SIGRTMIN+4 [37] SIGRTMIN+5 [38] SIGRTMIN+6 [39] SIGRTMIN+7 [40] SIGRTMIN+8 [41] SIGRTMIN+9 [42] SIGRTMIN+10 [43] SIGRTMIN+11 [44] SIGRTMIN+12 [45] SIGRTMIN+13 [46] SIGRTMIN+14 [47] SIGRTMIN+15 [48] SIGRTMIN+16 [49] SIGRTMAX-15 [50] SIGRTMAX-14 [51] SIGRTMAX-13 [52] SIGRTMAX-12 [53] SIGRTMAX-11 [54] SIGRTMAX-10 [55] SIGRTMAX-9 [56] SIGRTMAX-8 [57] SIGRTMAX-7 [58] SIGRTMAX-6 [59] SIGRTMAX-5 [60] SIGRTMAX-4 [61] SIGRTMAX-3 [62] SIGRTMAX-2 [63] SIGRTMAX-1 [64] SIGRTMAX
///
NAME
SYNOPSIS
vtop [-c [pid | taskp]] [-u|-k] address …
DESCRIPTION
This command translates a user or kernel virtual address to its physical
address. Also displayed is the PTE translation, the vm_area_struct data
for user virtual addresses, the mem_map page data associated with the
physical page, and the swap location or file location if the page is
not mapped. The -u and -k options specify that the address is a user
or kernel virtual address; -u and -k are not necessary on processors whose
virtual addresses self-define themselves as user or kernel. User addresses
are translated with respect to the current context unless the -c option
is used. Kernel virtual addresses are translated using the swapper_pg_dir
as the base page directory unless the -c option is used.
-u The address is a user virtual address; only required
on processors with overlapping user and kernel virtual
address spaces.
-k The address is a kernel virtual address; only required
on processors with overlapping user and kernel virtual
address spaces.
-c [pid | taskp] Translate the virtual address from the page directory
of the specified PID or hexadecimal task_struct pointer.
However, if this command is invoked from "foreach vtop",
the pid or taskp argument should NOT be entered; the
address will be translated using the page directory of
each task specified by "foreach".
address A hexadecimal user or kernel virtual address.
EXAMPLES
Translate user virtual address 80b4000:
crash> vtop 80b4000 VIRTUAL PHYSICAL 80b4000 660f000 PAGE DIRECTORY: c37f0000 PGD: c37f0080 => e0d067 PMD: c37f0080 => e0d067 PTE: c0e0d2d0 => 660f067 PAGE: 660f000 PTE PHYSICAL FLAGS 660f067 660f000 (PRESENT|RW|USER|ACCESSED|DIRTY) VMA START END FLAGS FILE c773daa0 80b4000 810c000 77 PAGE PHYSICAL INODE OFFSET CNT FLAGS c0393258 660f000 0 17000 1 uptodate
Translate kernel virtual address c806e000, first using swapper_pg_dir
as the page directory base, and secondly, using the page table base
of PID 1359:
crash> vtop c806e000 VIRTUAL PHYSICAL c806e000 2216000 PAGE DIRECTORY: c0101000 PGD: c0101c80 => 94063 PMD: c0101c80 => 94063 PTE: c00941b8 => 2216063 PAGE: 2216000 PTE PHYSICAL FLAGS 2216063 2216000 (PRESENT|RW|ACCESSED|DIRTY) PAGE PHYSICAL INODE OFFSET CNT FLAGS c02e9370 2216000 0 0 1 crash> vtop -c 1359 c806e000 VIRTUAL PHYSICAL c806e000 2216000 PAGE DIRECTORY: c5caf000 PGD: c5cafc80 => 94063 PMD: c5cafc80 => 94063 PTE: c00941b8 => 2216063 PAGE: 2216000 PTE PHYSICAL FLAGS 2216063 2216000 (PRESENT|RW|ACCESSED|DIRTY) PAGE PHYSICAL INODE OFFSET CNT FLAGS c02e9370 2216000 0 0 1
Determine swap location of user virtual address 40104000:
crash> vtop 40104000 VIRTUAL PHYSICAL 40104000 (not mapped) PAGE DIRECTORY: c40d8000 PGD: c40d8400 => 6bbe067 PMD: c40d8400 => 6bbe067 PTE: c6bbe410 => 58bc00 PTE SWAP OFFSET 58bc00 /dev/sda8 22716 VMA START END FLAGS FILE c7200ae0 40104000 40b08000 73 SWAP: /dev/sda8 OFFSET: 22716
///
NAME
SYNOPSIS
dev [-i | -p | -d | -D ] [-V | -v index [file]]
DESCRIPTION
If no argument is entered, this command dumps character and block
device data.
-i display I/O port usage; on 2.4 kernels, also display I/O memory usage.
-p display PCI device data.
-d display disk I/O statistics:
TOTAL: total number of allocated in-progress I/O requests
SYNC: I/O requests that are synchronous
ASYNC: I/O requests that are asynchronous
READ: I/O requests that are reads (older kernels)
WRITE: I/O requests that are writes (older kernels)
DRV: I/O requests that are in-flight in the device driver.
If the device driver uses blk-mq interface, this field
shows N/A(MQ). If not available, this column is not shown.
-D same as -d, but filter out disks with no in-progress I/O requests.
If the dumpfile contains device dumps:
-V display an indexed list of all device dumps present in the vmcore,
showing their file offset, size and name.
-v index select and display one device dump based upon an index value
shown by the -V option, shown in a default human-readable format;
alternatively, the "rd -f" option along with its various format
options may be used to further tailor the output.
file only used with -v, copy the device dump data to a file.
EXAMPLES
Display character and block device data:
crash> dev CHRDEV NAME CDEV OPERATIONS 1 mem f79b83c0 memory_fops 4 /dev/vc/0 c07bc560 console_fops 4 tty f7af5004 tty_fops 4 ttyS f7b02204 tty_fops 5 /dev/tty c07bc440 tty_fops 5 /dev/console c07bc4a0 console_fops 5 /dev/ptmx c07bc500 ptmx_fops 6 lp c5797e40 lp_fops 7 vcs f7b03d40 vcs_fops 10 misc f7f68640 misc_fops 13 input f79b8840 input_fops 21 sg f7f12840 sg_fops 29 fb f7f8c640 fb_fops 128 ptm f7b02604 tty_fops 136 pts f7b02404 tty_fops 162 raw c0693e40 raw_fops 180 usb f79b8bc0 usb_fops 189 usb_device c06a0300 usbfs_device_file_operations 216 rfcomm f5961a04 tty_fops 254 pcmcia f79b82c0 ds_fops BLKDEV NAME GENDISK OPERATIONS 1 ramdisk f7b23480 rd_bd_op 8 sd f7cab280 sd_fops 9 md f7829b80 md_fops 11 sr f75c24c0 sr_bdops 65 sd (none) 66 sd (none) 67 sd (none) 68 sd (none) 69 sd (none) 70 sd (none) 71 sd (none) 128 sd (none) 129 sd (none) 130 sd (none) 131 sd (none) 132 sd (none) 133 sd (none) 134 sd (none) 135 sd (none) 253 device-mapper c57a0ac0 dm_blk_dops 254 mdp (none)
Display PCI data:
crash> dev -p
PCI_DEV BU:SL.FN CLASS: VENDOR-DEVICE
c00051c0 00:00.0 Host bridge: Intel 440BX - 82443BX Host
c0005250 00:01.0 PCI bridge: Intel 440BX - 82443BX AGP
c00052e0 00:07.0 ISA bridge: Intel 82371AB PIIX4 ISA
c0005370 00:07.1 IDE interface: Intel 82371AB PIIX4 IDE
c0005400 00:07.2 USB Controller: Intel 82371AB PIIX4 USB
c0005490 00:07.3 Bridge: Intel 82371AB PIIX4 ACPI
c0005520 00:11.0 Ethernet controller: 3Com 3C905B 100bTX
c00055b0 00:13.0 PCI bridge: DEC DC21152
c0005640 01:00.0 VGA compatible controller: NVidia [PCI_DEVICE 28]
c00056d0 02:0a.0 SCSI storage controller: Adaptec AIC-7890/1
c0005760 02:0e.0 SCSI storage controller: Adaptec AIC-7880U
Display I/O port and I/O memory usage:
crash> dev -i RESOURCE RANGE NAME c03036d4 0000-ffff PCI IO c0302594 0000-001f dma1 c03025b0 0020-003f pic1 c03025cc 0040-005f timer c03025e8 0060-006f keyboard c0302604 0080-008f dma page reg c0302620 00a0-00bf pic2 c030263c 00c0-00df dma2 c0302658 00f0-00ff fpu c122ff20 0170-0177 ide1 c122f240 0213-0213 isapnp read c122ff40 02f8-02ff serial(auto) c122ff00 0376-0376 ide1 c03186e8 03c0-03df vga+ c122ff60 03f8-03ff serial(auto) c123851c 0800-083f Intel Corporation 82371AB PIIX4 ACPI c1238538 0840-085f Intel Corporation 82371AB PIIX4 ACPI c122f220 0a79-0a79 isapnp write c122f200 0cf8-0cff PCI conf1 c1238858 dc00-dc7f 3Com Corporation 3c905B 100BaseTX [Cyclone] c122fc00 dc00-dc7f 00:11.0 c12380c8 dce0-dcff Intel Corporation 82371AB PIIX4 USB c1238d1c e000-efff PCI Bus #02 c1237858 e800-e8ff Adaptec AIC-7880U c1237458 ec00-ecff Adaptec AHA-2940U2/W / 7890 c1239cc8 ffa0-ffaf Intel Corporation 82371AB PIIX4 IDE RESOURCE RANGE NAME c03036f0 00000000-ffffffff PCI mem c0004000 00000000-0009ffff System RAM c03026ac 000a0000-000bffff Video RAM area c03026fc 000c0000-000c7fff Video ROM c0302718 000c9800-000cdfff Extension ROM c0302734 000ce000-000ce7ff Extension ROM c0302750 000ce800-000cffff Extension ROM c03026e0 000f0000-000fffff System ROM c0004040 00100000-07ffdfff System RAM c0302674 00100000-0028682b Kernel code c0302690 0028682c-0031c63f Kernel data c0004060 07ffe000-07ffffff reserved c1239058 ec000000-efffffff Intel Corporation 440BX/ZX - 82443BX/ZX Host bridge c1238d54 f1000000-f1ffffff PCI Bus #02 c1239554 f2000000-f5ffffff PCI Bus #01 c1237074 f4000000-f5ffffff nVidia Corporation Riva TnT2 [NV5] c1238d38 fa000000-fbffffff PCI Bus #02 c1237874 faffe000-faffefff Adaptec AIC-7880U c127ec40 faffe000-faffefff aic7xxx c1237474 fafff000-faffffff Adaptec AHA-2940U2/W / 7890 c127eec0 fafff000-faffffff aic7xxx c1239538 fc000000-fdffffff PCI Bus #01 c1237058 fc000000-fcffffff nVidia Corporation Riva TnT2 [NV5] c1238874 fe000000-fe00007f 3Com Corporation 3c905B 100BaseTX [Cyclone] c0004080 fec00000-fec0ffff reserved c00040a0 fee00000-fee0ffff reserved c00040c0 ffe00000-ffffffff reserved
Display disk I/O statistics:
crash> dev -d
MAJOR GENDISK NAME REQUEST_QUEUE TOTAL READ WRITE DRV
2 ffff81012d8a5000 fd0 ffff81012dc053c0 12 0 12 0
22 ffff81012dc6b000 hdc ffff81012d8ae340 2 2 0 0
8 ffff81012dd71000 sda ffff81012d8af040 6 0 6 6
8 ffff81012dc77000 sdb ffff81012d8b5740 0 0 0 0
8 ffff81012d8d0c00 sdc ffff81012d8ae9c0 0 0 0 0
Display the available device dumps:
crash> dev -V
INDEX OFFSET SIZE NAME
0 0x240 33558464 cxgb4_0000:02:00.4
1 0x2001240 33558464 cxgb4_0000:03:00.4
Extract a specified device dump to file:
crash> dev -v 0 device_dump_0.bin
DEVICE: cxgb4_0000:02:00.4
33558464 bytes copied from 0x240 to device_dump_0.bin
Format and display a device’s dump data to the screen using the “rd” command:
crash> rd -f 0x240 -32 8
240: 040b69e2 00000038 000e0001 00675fd4 .i..8........_g.
250: 00000000 21600047 00000000 00000000 ....G.`!........
Display a device’s dump data to the screen using the default format:
crash> dev -v 1
DEVICE: cxgb4_0000:03:00.4
2001240: 00000038040b69e2 00af985c000e0001 .i..8.......\...
2001250: 2150004700000000 0000000000000000 ....G.P!........
2001260: 0000000000000000 0000000000000000 ................
2001270: 0000000000000000 0002fccc00000001 ................
2001280: 00000000000027b0 0000000000000000 .'..............
...
///
NAME
SYNOPSIS
ipcs [-smMq] [-n pid|task] [id | addr]
DESCRIPTION
This command provides information on the System V IPC facilities. With no
arguments, the command will display kernel usage of all three factilities.
-s show semaphore arrays.
-m show shared memory segments.
-M show shared memory segments with additional details.
-q show message queues.
id show the data associated with this resource ID.
addr show the data associated with this virtual address of a
shmid_kernel, sem_array or msq_queue.
For kernels supporting namespaces, the -n option may be used to
display the IPC facilities with respect to the namespace of a
specified task:
-n pid a process PID.
-n task a hexadecimal task_struct pointer.
EXAMPLES
Display all IPC facilities:
crash> ipcs
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ffff880473a28310 00000000 0 0 666 90000 1
ffff880473a28490 00000001 32769 0 666 90000 1
ffff880473a28250 00000002 65538 0 666 90000 1
SEM_ARRAY KEY SEMID UID PERMS NSEMS
ffff88047200f9d0 00000000 0 0 600 1
ffff88046f826910 00000000 32769 0 600 1
MSG_QUEUE KEY MSQID UID PERMS USED-BYTES MESSAGES
ffff8100036bb8d0 000079d7 0 3369 666 16640 104
ffff8100036bb3d0 000079d8 32769 3369 666 12960 81
ffff810026d751d0 000079d9 65538 3369 666 10880 68
Display shared memory usage with detailed information:
crash> ipcs -M
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ffff880473a28310 00000000 0 0 666 90000 1
PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
INODE: ffff88047239cd98
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ffff880473a28490 00000001 32769 0 666 90000 1
PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
INODE: ffff88047239c118
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ffff880473a28250 00000002 65538 0 666 90000 1
PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
INODE: ffff880470503758
Display the shared memory data associated with shmid_kernel ffff880473a28250:
crash> ipcs -M ffff880473a28250
SHMID_KERNEL KEY SHMID UID PERMS BYTES NATTCH STATUS
ffff880473a28250 00000002 65538 0 666 90000 1
PAGES ALLOCATED/RESIDENT/SWAPPED: 22/1/0
INODE: ffff880470503758
///
NAME
SYNOPSIS
ps [-k|-u|-G|-y policy] [-s] [-p|-c|-t|-[l|m][-C cpu]|-a|-g|-r|-S|-A]
[pid | task | command] …
DESCRIPTION
This command displays process status for selected, or all, processes
in the system. If no arguments are entered, the process data is
is displayed for all processes. Specific processes may be selected
by using the following identifier formats:
pid a process PID.
task a hexadecimal task_struct pointer.
command a command name. If a command name is made up of letters that
are all numerical values, precede the name string with a “”.
If the command string is enclosed within “’” characters, then
the encompassed string must be a POSIX extended regular expression
that will be used to match task names.
The process list may be further restricted by the following options:
-k restrict the output to only kernel threads.
-u restrict the output to only user tasks.
-G display only the thread group leader in a thread group.
-y policy restrict the output to tasks having a specified scheduling policy
expressed by its integer value or by its (case-insensitive) name;
multiple policies may be entered in a comma-separated list:
0 or NORMAL
1 or FIFO
2 or RR
3 or BATCH
4 or ISO
5 or IDLE
6 or DEADLINE
The process identifier types may be mixed. For each task, the following
items are displayed:
1. the process PID.
2. the parent process PID.
3. the CPU number that the task ran on last.
4. the task_struct address or the kernel stack pointer of the process.
(see -s option below)
5. the task state (RU, IN, UN, ZO, ST, TR, DE, SW, WA, PA, ID, NE).
6. the percentage of physical memory being used by this task.
7. the virtual address size of this task in kilobytes.
8. the resident set size of this task in kilobytes.
9. the command name.
The default output shows the task_struct address of each process under a
column titled “TASK”. This can be changed to show the kernel stack
pointer under a column titled “KSTACKP”.
-s replace the TASK column with the KSTACKP column.
On SMP machines, the active task on each CPU will be highlighted by an
angle bracket (">") preceding its information. If the crash variable
“offline” is set to “hide”, the active task on an offline CPU will
be highlighted by a “-” preceding its information.
Alternatively, information regarding parent-child relationships,
per-task time usage data, argument/environment data, thread groups,
or resource limits may be displayed:
-p display the parental hierarchy of selected, or all, tasks. -c display the children of selected, or all, tasks. -t display the task run time, start time, and cumulative user and system times. -l display the task's last-run timestamp value, using either the task_struct's last_run value, the task_struct's timestamp value or the task_struct's sched_entity last_arrival value, whichever applies, of selected, or all, tasks; the list is sorted with the most recently-run task (with the largest timestamp) shown first, followed by the task's current state. -m similar to -l, but the timestamp value is translated into days, hours, minutes, seconds, and milliseconds since the task was last run on a cpu. -C cpus only usable with the -l or -m options, dump the timestamp data in per-cpu blocks, where the cpu[s] can be specified as "1,3,5", "1-3", "1,3,5-7,10", "all", or "a" (shortcut for "all"). -a display the command line arguments and environment strings of selected, or all, user-mode tasks. -g display tasks by thread group, of selected, or all, tasks. -r display resource limits (rlimits) of selected, or all, tasks. -S display a summary consisting of the number of tasks in a task state. -A display only the active task on each cpu.
EXAMPLES
Show the process status of all current tasks:
crash> ps PID PPID CPU TASK ST %MEM VSZ RSS COMM > 0 0 3 c024c000 RU 0.0 0 0 [swapper] > 0 0 0 c0dce000 RU 0.0 0 0 [swapper] 0 0 1 c0fa8000 RU 0.0 0 0 [swapper] > 0 0 2 c009a000 RU 0.0 0 0 [swapper] 1 0 1 c0098000 IN 0.0 1096 476 init 2 1 1 c0090000 IN 0.0 0 0 [kflushd] 3 1 1 c000e000 IN 0.0 0 0 [kpiod] 4 1 3 c000c000 IN 0.0 0 0 [kswapd] 5 1 1 c0008000 IN 0.0 0 0 [mdrecoveryd] 253 1 2 fbc4c000 IN 0.0 1088 376 portmap 268 1 2 fbc82000 IN 0.1 1232 504 ypbind 274 268 2 fa984000 IN 0.1 1260 556 ypbind 321 1 1 fabf6000 IN 0.1 1264 608 syslogd 332 1 1 fa9be000 RU 0.1 1364 736 klogd 346 1 2 fae88000 IN 0.0 1112 472 atd 360 1 2 faeb2000 IN 0.1 1284 592 crond 378 1 2 fafd6000 IN 0.1 1236 560 inetd 392 1 0 fb710000 IN 0.1 2264 1468 named 406 1 3 fb768000 IN 0.1 1284 560 lpd 423 1 1 fb8ac000 IN 0.1 1128 528 rpc.statd 434 1 2 fb75a000 IN 0.0 1072 376 rpc.rquotad 445 1 2 fb4a4000 IN 0.0 1132 456 rpc.mountd 460 1 1 fa938000 IN 0.0 0 0 [nfsd] 461 1 1 faa86000 IN 0.0 0 0 [nfsd] 462 1 0 fac48000 IN 0.0 0 0 [nfsd] 463 1 0 fb4ca000 IN 0.0 0 0 [nfsd] 464 1 0 fb4c8000 IN 0.0 0 0 [nfsd] 465 1 2 fba6e000 IN 0.0 0 0 [nfsd] 466 1 1 fba6c000 IN 0.0 0 0 [nfsd] 467 1 2 fac04000 IN 0.0 0 0 [nfsd] 468 461 2 fa93a000 IN 0.0 0 0 [lockd] 469 468 2 fa93e000 IN 0.0 0 0 [rpciod] 486 1 0 fab54000 IN 0.1 1596 880 amd 523 1 2 fa84e000 IN 0.1 1884 1128 sendmail 538 1 0 fa82c000 IN 0.0 1112 416 gpm 552 1 3 fa70a000 IN 0.1 2384 1220 httpd 556 552 3 fa776000 IN 0.1 2572 1352 httpd 557 552 2 faba4000 IN 0.1 2572 1352 httpd 558 552 1 fa802000 IN 0.1 2572 1352 httpd 559 552 3 fa6ee000 IN 0.1 2572 1352 httpd 560 552 3 fa700000 IN 0.1 2572 1352 httpd 561 552 0 fa6f0000 IN 0.1 2572 1352 httpd 562 552 3 fa6ea000 IN 0.1 2572 1352 httpd 563 552 0 fa67c000 IN 0.1 2572 1352 httpd 564 552 3 fa674000 IN 0.1 2572 1352 httpd 565 552 3 fa66a000 IN 0.1 2572 1352 httpd 582 1 2 fa402000 IN 0.2 2968 1916 xfs 633 1 2 fa1ec000 IN 0.2 5512 2248 innd 636 1 3 fa088000 IN 0.1 2536 804 actived 676 1 0 fa840000 IN 0.0 1060 384 mingetty 677 1 1 fa590000 IN 0.0 1060 384 mingetty 678 1 2 fa3b8000 IN 0.0 1060 384 mingetty 679 1 0 fa5b8000 IN 0.0 1060 384 mingetty 680 1 1 fa3a4000 IN 0.0 1060 384 mingetty 681 1 2 fa30a000 IN 0.0 1060 384 mingetty 683 1 3 fa5d8000 IN 0.0 1052 280 update 686 378 1 fa3aa000 IN 0.1 2320 1136 in.rlogind 687 686 2 f9e52000 IN 0.1 2136 1000 login 688 687 0 f9dec000 IN 0.1 1732 976 bash > 700 688 1 f9d62000 RU 0.0 1048 256 gen12
Display the parental hierarchy of the “crash” process on a live system:
crash> ps -p 4249
PID: 0 TASK: c0252000 CPU: 0 COMMAND: "swapper"
PID: 1 TASK: c009a000 CPU: 1 COMMAND: "init"
PID: 632 TASK: c73b6000 CPU: 1 COMMAND: "prefdm"
PID: 637 TASK: c5a4a000 CPU: 1 COMMAND: "prefdm"
PID: 649 TASK: c179a000 CPU: 0 COMMAND: "kwm"
PID: 683 TASK: c1164000 CPU: 0 COMMAND: "kfm"
PID: 1186 TASK: c165a000 CPU: 0 COMMAND: "xterm"
PID: 1188 TASK: c705e000 CPU: 1 COMMAND: "bash"
PID: 4249 TASK: c6b9a000 CPU: 0 COMMAND: "crash"
Display all children of the “kwm” window manager:
crash> ps -c kwm
PID: 649 TASK: c179a000 CPU: 0 COMMAND: "kwm"
PID: 682 TASK: c2d58000 CPU: 1 COMMAND: "kwmsound"
PID: 683 TASK: c1164000 CPU: 1 COMMAND: "kfm"
PID: 685 TASK: c053c000 CPU: 0 COMMAND: "krootwm"
PID: 686 TASK: c13fa000 CPU: 0 COMMAND: "kpanel"
PID: 687 TASK: c13f0000 CPU: 1 COMMAND: "kbgndwm"
Display all threads in a firefox session:
crash> ps firefox
PID PPID CPU TASK ST %MEM VSZ RSS COMM
21273 21256 6 ffff81003ec15080 IN 46.3 1138276 484364 firefox
21276 21256 6 ffff81003f49e7e0 IN 46.3 1138276 484364 firefox
21280 21256 0 ffff81003ec1d7e0 IN 46.3 1138276 484364 firefox
21286 21256 6 ffff81000b0d1820 IN 46.3 1138276 484364 firefox
21287 21256 2 ffff81000b0d10c0 IN 46.3 1138276 484364 firefox
26975 21256 5 ffff81003b5c1820 IN 46.3 1138276 484364 firefox
26976 21256 5 ffff810023232820 IN 46.3 1138276 484364 firefox
26977 21256 4 ffff810021a11820 IN 46.3 1138276 484364 firefox
26978 21256 5 ffff810003159040 IN 46.3 1138276 484364 firefox
26979 21256 5 ffff81003a058820 IN 46.3 1138276 484364 firefox
Display only the thread group leader in the firefox session:
crash> ps -G firefox
PID PPID CPU TASK ST %MEM VSZ RSS COMM
21273 21256 0 ffff81003ec15080 IN 46.3 1138276 484364 firefox
Show the time usage data for pid 10318:
crash> ps -t 10318
PID: 10318 TASK: f7b85550 CPU: 5 COMMAND: "bash"
RUN TIME: 1 days, 01:35:32
START TIME: 5209
UTIME: 95
STIME: 57
Show the process status of PID 1, task f9dec000, and all nfsd tasks:
crash> ps 1 f9dec000 nfsd
PID PPID CPU TASK ST %MEM VSZ RSS COMM
1 0 1 c0098000 IN 0.0 1096 476 init
688 687 0 f9dec000 IN 0.1 1732 976 bash
460 1 1 fa938000 IN 0.0 0 0 [nfsd]
461 1 1 faa86000 IN 0.0 0 0 [nfsd]
462 1 0 fac48000 IN 0.0 0 0 [nfsd]
463 1 0 fb4ca000 IN 0.0 0 0 [nfsd]
464 1 0 fb4c8000 IN 0.0 0 0 [nfsd]
465 1 2 fba6e000 IN 0.0 0 0 [nfsd]
466 1 1 fba6c000 IN 0.0 0 0 [nfsd]
467 1 2 fac04000 IN 0.0 0 0 [nfsd]
Show all kernel threads:
crash> ps -k PID PPID CPU TASK ST %MEM VSZ RSS COMM 0 0 1 c0fac000 RU 0.0 0 0 [swapper] 0 0 0 c0252000 RU 0.0 0 0 [swapper] 2 1 1 c0fa0000 IN 0.0 0 0 [kflushd] 3 1 1 c03de000 IN 0.0 0 0 [kpiod] 4 1 1 c03dc000 IN 0.0 0 0 [kswapd] 5 1 0 c0092000 IN 0.0 0 0 [mdrecoveryd] 336 1 0 c4a9a000 IN 0.0 0 0 [rpciod] 337 1 0 c4830000 IN 0.0 0 0 [lockd] 487 1 1 c4ba6000 IN 0.0 0 0 [nfsd] 488 1 0 c18c6000 IN 0.0 0 0 [nfsd] 489 1 0 c0cac000 IN 0.0 0 0 [nfsd] 490 1 0 c056a000 IN 0.0 0 0 [nfsd] 491 1 0 c0860000 IN 0.0 0 0 [nfsd] 492 1 1 c0254000 IN 0.0 0 0 [nfsd] 493 1 0 c0a86000 IN 0.0 0 0 [nfsd] 494 1 0 c0968000 IN 0.0 0 0 [nfsd]
Display a summary consisting of the number of tasks in a task state:
crash> ps -S
RU: 5
IN: 259
UN: 31
ZO: 1
Display only the active task, on each cpu:
crash> ps -A
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 10 2 1 ffff880212969710 IN 0.0 0 0 [migration/1]
> 0 0 3 ffff884026d43520 RU 0.0 0 0 [swapper]
> 6582 1 2 ffff880f49c52040 RU 0.0 42202472 33368 oracle
> 9497 1 0 ffff880549ec2ab0 RU 0.0 42314692 138664 oracle
Show all tasks sorted by their task_struct’s last_run, timestamp, or
sched_entity last_arrival timestamp value, whichever applies:
crash> ps -l [20811245123] [IN] PID: 37 TASK: f7153030 CPU: 2 COMMAND: "events/2" [20811229959] [IN] PID: 1756 TASK: f2a5a570 CPU: 2 COMMAND: "ntpd" [20800696644] [IN] PID: 1456 TASK: f2b1f030 CPU: 4 COMMAND: "irqbalance" [20617047229] [IN] PID: 2324 TASK: f57f9570 CPU: 5 COMMAND: "flush-253:0" [20617029209] [IN] PID: 49 TASK: f7167030 CPU: 4 COMMAND: "bdi-default" [20438025365] [IN] PID: 345 TASK: f55c7ab0 CPU: 3 COMMAND: "mpt_poll_0" [20103026046] [IN] PID: 728 TASK: f72ba570 CPU: 3 COMMAND: "edac-poller" [20000189409] [IN] PID: 35 TASK: f7153ab0 CPU: 0 COMMAND: "events/0" [20000179905] [IN] PID: 48 TASK: f7167570 CPU: 0 COMMAND: "sync_supers" [19997120354] [IN] PID: 36 TASK: f7153570 CPU: 1 COMMAND: "events/1" [19991059209] [IN] PID: 38 TASK: f715fab0 CPU: 3 COMMAND: "events/3" [19988091608] [IN] PID: 39 TASK: f715f570 CPU: 4 COMMAND: "events/4" [19985076530] [IN] PID: 40 TASK: f715f030 CPU: 5 COMMAND: "events/5" [19982019106] [IN] PID: 41 TASK: f7161ab0 CPU: 6 COMMAND: "events/6" [19982016294] [IN] PID: 29 TASK: f7109ab0 CPU: 6 COMMAND: "ksoftirqd/6" [19838402345] [RU] PID: 2331 TASK: f297f570 CPU: 7 COMMAND: "bash" [19837129436] [IN] PID: 2326 TASK: f2ad5030 CPU: 6 COMMAND: "sshd" [19289476417] [IN] PID: 1772 TASK: f5665570 CPU: 5 COMMAND: "sendmail" ...
Show the most-recently run tasks on cpu 0 using both the -l and the -m
options:
crash> ps -m -C0 CPU: 0 [ 0 00:00:00.003] [RU] PID: 1205 TASK: dee03f20 CPU: 0 COMMAND: "insmod" [ 0 00:00:00.006] [RU] PID: 770 TASK: df9e9940 CPU: 0 COMMAND: "rsyslogd" [ 0 00:00:00.009] [IN] PID: 603 TASK: df9bcbc0 CPU: 0 COMMAND: "udevd" [ 0 00:00:00.010] [IN] PID: 348 TASK: df9ecbc0 CPU: 0 COMMAND: "udevd" [ 0 00:00:00.013] [IN] PID: 934 TASK: df9171a0 CPU: 0 COMMAND: "hald" [ 0 00:00:00.023] [IN] PID: 6 TASK: df443f20 CPU: 0 COMMAND: "events/0" [ 0 00:00:00.029] [IN] PID: 15 TASK: df46b280 CPU: 0 COMMAND: "kblockd/0" [ 0 00:00:00.101] [IN] PID: 1168 TASK: dee01940 CPU: 0 COMMAND: "bash" [ 0 00:00:01.404] [IN] PID: 272 TASK: dfa48ca0 CPU: 0 COMMAND: "flush-8:0" ... crash> ps -l -C0 CPU: 0 [137146164748] [RU] PID: 1205 TASK: dee03f20 CPU: 0 COMMAND: "insmod" [137142534372] [RU] PID: 770 TASK: df9e9940 CPU: 0 COMMAND: "rsyslogd" [137140168469] [IN] PID: 603 TASK: df9bcbc0 CPU: 0 COMMAND: "udevd" [137138826427] [IN] PID: 348 TASK: df9ecbc0 CPU: 0 COMMAND: "udevd" [137135214599] [IN] PID: 934 TASK: df9171a0 CPU: 0 COMMAND: "hald" [137125651275] [IN] PID: 6 TASK: df443f20 CPU: 0 COMMAND: "events/0" [137119564815] [IN] PID: 15 TASK: df46b280 CPU: 0 COMMAND: "kblockd/0" [137047715027] [IN] PID: 1168 TASK: dee01940 CPU: 0 COMMAND: "bash" [135744209052] [IN] PID: 272 TASK: dfa48ca0 CPU: 0 COMMAND: "flush-8:0" ...
Show the kernel stack pointer of each user task:
crash> ps -us
PID PPID CPU KSTACKP ST %MEM VSZ RSS COMM
1 0 0 c009bedc IN 0.0 1096 52 init
239 1 0 c15e7ed8 IN 0.2 1332 224 pump
280 1 1 c7cbdedc IN 0.2 1092 208 portmap
295 1 0 c7481edc IN 0.0 1232 0 ypbind
301 295 0 c7c7bf28 IN 0.1 1260 124 ypbind
376 1 1 c5053f28 IN 0.0 1316 40 automount
381 1 0 c34ddf28 IN 0.2 1316 224 automount
391 1 1 c2777f28 IN 0.2 1316 224 automount
...
Display the argument and environment data for the automount task:
crash> ps -a automount PID: 3948 TASK: f722ee30 CPU: 0 COMMAND: "automount" ARG: /usr/sbin/automount --timeout=60 /net program /etc/auto.net ENV: SELINUX_INIT=YES CONSOLE=/dev/console TERM=linux INIT_VERSION=sysvinit-2.85 PATH=/sbin:/usr/sbin:/bin:/usr/bin LC_MESSAGES=en_US RUNLEVEL=3 runlevel=3 PWD=/ LANG=ja_JP.UTF-8 PREVLEVEL=N previous=N HOME=/ SHLVL=2 _=/usr/sbin/automount
Display the tasks in the thread group containing task c20ab0b0:
crash> ps -g c20ab0b0
PID: 6425 TASK: f72f50b0 CPU: 0 COMMAND: "firefox-bin"
PID: 6516 TASK: f71bf1b0 CPU: 0 COMMAND: "firefox-bin"
PID: 6518 TASK: d394b930 CPU: 0 COMMAND: "firefox-bin"
PID: 6520 TASK: c20aa030 CPU: 0 COMMAND: "firefox-bin"
PID: 6523 TASK: c20ab0b0 CPU: 0 COMMAND: "firefox-bin"
PID: 6614 TASK: f1f181b0 CPU: 0 COMMAND: "firefox-bin"
Display the tasks in the thread group for each instance of the
program named “multi-thread”:
crash> ps -g multi-thread
PID: 2522 TASK: 1003f0dc7f0 CPU: 1 COMMAND: "multi-thread"
PID: 2523 TASK: 10037b13030 CPU: 1 COMMAND: "multi-thread"
PID: 2524 TASK: 1003e064030 CPU: 1 COMMAND: "multi-thread"
PID: 2525 TASK: 1003e13a7f0 CPU: 1 COMMAND: "multi-thread"
PID: 2526 TASK: 1002f82b7f0 CPU: 1 COMMAND: "multi-thread"
PID: 2527 TASK: 1003e1737f0 CPU: 1 COMMAND: "multi-thread"
PID: 2528 TASK: 10035b4b7f0 CPU: 1 COMMAND: "multi-thread"
PID: 2529 TASK: 1003f0c37f0 CPU: 1 COMMAND: "multi-thread"
PID: 2530 TASK: 10035597030 CPU: 1 COMMAND: "multi-thread"
PID: 2531 TASK: 100184be7f0 CPU: 1 COMMAND: "multi-thread"
Display the resource limits of “bash” task 13896:
crash> ps -r 13896
PID: 13896 TASK: cf402000 CPU: 0 COMMAND: "bash"
RLIMIT CURRENT MAXIMUM
CPU (unlimited) (unlimited)
FSIZE (unlimited) (unlimited)
DATA (unlimited) (unlimited)
STACK 10485760 (unlimited)
CORE (unlimited) (unlimited)
RSS (unlimited) (unlimited)
NPROC 4091 4091
NOFILE 1024 1024
MEMLOCK 4096 4096
AS (unlimited) (unlimited)
LOCKS (unlimited) (unlimited)
Search for task names matching a POSIX regular expression:
crash> ps 'migration*'
PID PPID CPU TASK ST %MEM VSZ RSS COMM
8 2 0 ffff8802128a2e20 IN 0.0 0 0 [migration/0]
10 2 1 ffff880212969710 IN 0.0 0 0 [migration/1]
15 2 2 ffff880212989710 IN 0.0 0 0 [migration/2]
20 2 3 ffff8802129a9710 IN 0.0 0 0 [migration/3]
///
NAME
SYNOPSIS
struct struct_name[.member[,member]][-o][-l offset][-rfuxdp]
[address | symbol][:cpuspec] [count | -c count]
DESCRIPTION
This command displays either a structure definition, or a formatted display
of the contents of a structure at a specified address. When no address is
specified, the structure definition is shown along with the structure size.
A structure member may be appended to the structure name in order to limit
the scope of the data displayed to that particular member; when no address
is specified, the member’s offset and definition are shown.
struct_name name of a C-code structure used by the kernel. .member name of a structure member; to display multiple members of a structure, use a comma-separated list of members. If any member contains an embedded structure, or the member is an array, the output may be restricted to just the embedded structure or an array element by expressing the member argument as "member.member" or "member[index]"; embedded member specifications may extend beyond one level deep, by expressing the member argument as "member.member.member...". -o show member offsets when displaying structure definitions; if used with an address or symbol argument, each member will be preceded by its virtual address. -l offset if the address argument is a pointer to a structure member that is contained by the target data structure, typically a pointer to an embedded list_head, the offset to the embedded member may be entered in either of the following manners: 1. in "structure.member" format. 2. a number of bytes. -r raw dump of structure data. -f address argument is a dumpfile offset. -u address argument is a user virtual address in the current context. -x override default output format with hexadecimal format. -d override default output format with decimal format. -p if a structure member is a pointer value, show the member's data type on the output line; and on the subsequent line(s), dereference the pointer, display the pointer target's symbol value in brackets if appropriate, and if possible, display the target data; requires an address argument. address hexadecimal address of a structure; if the address points to an embedded list_head structure contained within the target data structure, then the "-l" option must be used. symbol symbolic reference to the address of a structure. :cpuspec CPU specification for a per-cpu address or symbol: : CPU of the currently selected task. :a[ll] all CPUs. :#[-#][,...] CPU list(s), e.g. "1,3,5", "1-3", or "1,3,5-7,10". count count of structures to dump from an array of structures; if used, this must be the last argument entered. -c count "-c" is only required if "count" is not the last argument entered or if a negative number is entered; if a negative value is entered, the (positive) "count" structures that lead up to and include the target structure will be displayed.
Structure data, sizes, and member offsets are shown in the current output
radix unless the -x or -d option is specified.
Please note that in the vast majority of cases, the “struct” command
name may be dropped; if the structure name does not conflict with any crash
or gdb command name, then the “struct_name[.member]” argument will be
recognized as a structure name, and this command automatically executed.
See the NOTE below.
EXAMPLES
Display the vm_area_struct at address c1e44f10:
crash> struct vm_area_struct c1e44f10 struct vm_area_struct { vm_mm = 0xc2857750, vm_start = 0x8048000, vm_end = 0x80a5000, vm_next = 0xc1e44a10, vm_page_prot = { pgprot = 0x25 }, vm_flags = 0x1875, vm_avl_height = 0x2, vm_avl_left = 0xc30fe200, vm_avl_right = 0xc30fed00, vm_next_share = 0x0, vm_pprev_share = 0xc1e44a30, vm_ops = 0xc0215ca0, vm_offset = 0x0, vm_file = 0xc0bfdc70, vm_pte = 0 }
Display the definition and size of a vm_area_struct structure. This first
example below displays just the structure and size. The second example
uses the -o option to also display member offsets. Both examples were
run with the output radix set to 10 (decimal):
crash> struct vm_area_struct struct vm_area_struct { struct mm_struct *vm_mm; long unsigned int vm_start; long unsigned int vm_end; struct vm_area_struct *vm_next; pgprot_t vm_page_prot; short unsigned int vm_flags; short int vm_avl_height; struct vm_area_struct *vm_avl_left; struct vm_area_struct *vm_avl_right; struct vm_area_struct *vm_next_share; struct vm_area_struct **vm_pprev_share; struct vm_operations_struct *vm_ops; long unsigned int vm_offset; struct file *vm_file; long unsigned int vm_pte; } SIZE: 56 crash> struct vm_area_struct -o struct vm_area_struct { [0] struct mm_struct *vm_mm; [4] long unsigned int vm_start; [8] long unsigned int vm_end; [12] struct vm_area_struct *vm_next; [16] pgprot_t vm_page_prot; [20] short unsigned int vm_flags; [22] short int vm_avl_height; [24] struct vm_area_struct *vm_avl_left; [28] struct vm_area_struct *vm_avl_right; [32] struct vm_area_struct *vm_next_share; [36] struct vm_area_struct **vm_pprev_share; [40] struct vm_operations_struct *vm_ops; [44] long unsigned int vm_offset; [48] struct file *vm_file; [52] long unsigned int vm_pte; } SIZE: 56
Display the definition and offset of the pgd member of an mm_struct:
crash> struct mm_struct.pgd
struct mm_struct {
[80] pgd_t *pgd;
}
Display the pgd member of the mm_struct at address ffff810022e7d080:
crash> struct mm_struct.pgd ffff810022e7d080
pgd = 0xffff81000e3ac000
Display the pgd_t pointed to by the mm_struct.pgd pointer above, forcing
the output to be expressed in hexadecimal:
crash> mm_struct.pgd ffff810022e7d080 -px
pgd_t *pgd = 0xffff81000e3ac000
-> {
pgd = 0x2c0a6067
}
Display the thread_info structure pointed to by the thread_info
member of the task_struct at ffff8100181190c0:
crash> task_struct.thread_info ffff8100181190c0 -p struct thread_info *thread_info = 0xffff810023c06000 -> { task = 0xffff8100181190c0, exec_domain = 0xffffffff802f78e0, flags = 128, status = 1, cpu = 3, preempt_count = 0, addr_limit = { seg = 18446604435732824064 }, restart_block = { fn = 0xffffffff80095a52 <do_no_restart_syscall>, arg0 = 0, arg1 = 0, arg2 = 0, arg3 = 0 } }
Display the flags and virtual members of 4 contigous page structures
in the mem_map page structure array:
crash> page.flags,virtual c101196c 4
flags = 0x8000,
virtual = 0xc04b0000
flags = 0x8000,
virtual = 0xc04b1000
flags = 0x8000,
virtual = 0xc04b2000
flags = 0x8000,
virtual = 0xc04b3000
Display the array of tcp_sl_timer structures declared by tcp_slt_array[]:
crash> struct tcp_sl_timer tcp_slt_array 4 struct tcp_sl_timer { count = { counter = 0x0 }, period = 0x32, last = 0x1419e4, handler = 0xc0164854 <tcp_syn_recv_timer> } struct tcp_sl_timer { count = { counter = 0x2 }, period = 0x753, last = 0x14a6df, handler = 0xc01645b0 <tcp_keepalive> } struct tcp_sl_timer { count = { counter = 0x0 }, period = 0x2ee, last = 0x143134, handler = 0xc016447c <tcp_twkill> } struct tcp_sl_timer { count = { counter = 0x0 }, period = 0x64, last = 0x143198, handler = 0xc0164404 <tcp_bucketgc> }
Without using the “struct” command name, display the the “d_child”
list_head member from a dentry structure:
crash> dentry.d_child 0xe813cb4
d_child = {
next = 0x3661344,
prev = 0xdea4bc4
},
Display the child dentry structure referenced by the “next” pointer above.
Since the “next” address of 0x3661344 above is a pointer to an embedded
list_head structure within the child dentry structure, the -l option
is required:
crash> dentry -l dentry.d_child 0x3661344 struct dentry { d_count = { counter = 1 }, d_flags = 0, d_inode = 0xf9aa604, d_parent = 0x11152b1c, d_hash = { next = 0x11fb3fc0, prev = 0x11fb3fc0 }, d_lru = { next = 0x366133c, prev = 0x366133c }, d_child = { next = 0x36613cc, prev = 0xe813cd4 }, d_subdirs = { next = 0x366134c, prev = 0x366134c }, d_alias = { next = 0xf9aa614, prev = 0xf9aa614 }, d_mounted = 0, d_name = { name = 0x3661384 "boot.log", len = 8, hash = 1935169207 }, d_time = 1515870810, d_op = 0x0, d_sb = 0x11fc9c00, d_vfs_flags = 0, d_fsdata = 0x0, d_extra_attributes = 0x0, d_iname = "boot.log\000" }
Display the virtual address of each member of the task_struct at
ffff8100145d2080:
crash> task_struct -o ffff8100145d2080
struct task_struct {
[ffff8100145d2080] volatile long int state;
[ffff8100145d2088] struct thread_info *thread_info;
[ffff8100145d2090] atomic_t usage;
[ffff8100145d2098] long unsigned int flags;
[ffff8100145d20a0] int lock_depth;
[ffff8100145d20a4] int load_weight;
[ffff8100145d20a8] int prio;
[ffff8100145d20ac] int static_prio;
[ffff8100145d20b0] int normal_prio;
[ffff8100145d20b8] struct list_head run_list;
[ffff8100145d20c8] struct prio_array *array;
...
Display the embedded sched_entity structure’s on_rq member and
the third pid_link structure in the embedded pids[] array of the
task_struct at ffff88011653e250:
crash> task_struct.se.on_rq,pids[2] ffff88011653e250
se.on_rq = 1,
pids[2] = {
node = {
next = 0xffff88011653aff0,
pprev = 0xffff88011653a860
},
pid = 0xffff88010d07ed00
}
For an example of displaying per-cpu variables, consider the
struct hd_struct.dkstats member, which is a percpu pointer to
a disk_stats structure:
crash> struct hd_struct.dkstats
struct hd_struct {
[1232] struct disk_stats *dkstats;
}
Taking an hd_struct at address ffff8802450e2848, display all
of the per-cpu disk_stats structures that it references:
crash> struct hd_struct.dkstats ffff8802450e2848 dkstats = 0x60fdb48026c8 crash> struct disk_stats 0x60fdb48026c8:a [0]: ffffe8fefe6026c8 struct disk_stats { sectors = {451376, 80468}, ios = {6041, 971}, merges = {386, 390}, ticks = {194877, 56131}, io_ticks = 12371, time_in_queue = 309163 } [1]: ffffe8fefe8026c8 struct disk_stats { sectors = {0, 0}, ios = {0, 0}, merges = {7, 242}, ticks = {0, 0}, io_ticks = 23, time_in_queue = 581 } [2]: ffffe8fefea026c8 struct disk_stats { sectors = {0, 0}, ios = {0, 0}, merges = {4, 112}, ticks = {0, 0}, io_ticks = 11, time_in_queue = 305 } [3]: ffffe8fefec026c8 struct disk_stats { sectors = {0, 0}, ios = {0, 0}, merges = {5, 54}, ticks = {0, 0}, io_ticks = 17, time_in_queue = 41 }
NOTE
If the structure name does not conflict with any crash command name, the
“struct” command may be dropped. Accordingly, the examples above could
also have been accomplished like so:
crash> vm_area_struct c1e44f10
crash> vm_area_struct
crash> vm_area_struct -o
crash> mm_struct.pgd ffff810022e7d080
crash> mm_struct.pgd
crash> tcp_sl_timer tcp_slt_array 4
Lastly, the short-cut “*” pointer-to command may also be used to negate
the need to enter the “struct” command name (enter “help *” for details).
//
NAME
SYNOPSIS
waitq [ symbol ] | [ struct.member struct_addr ] | [ address ]
DESCRIPTION
This command walks the wait queue list displaying the tasks which
are blocked on the specified wait queue. The command differentiates
between the old- and new-style wait queue structures used by the kernel.
It can be invoked with the following argument types:
symbol a global symbol of a wait queue.
struct.member struct_addr a structure name and wait queue member combination
followed by the structure’s hexadecimal address.
address a hexadecimal wait queue pointer.
EXAMPLES
Find out if any tasks are blocked on the “buffer_wait” wait queue:
crash> waitq buffer_wait
wait queue "buffer_wait" (c02927f0) is empty
See who is blocked on the “wait_chldexit” queue of task c5496000:
crash> waitq task_struct.wait_chldexit c5496000
PID: 30879 TASK: c5496000 CPU: 0 COMMAND: "bash"
Display the task list waiting on a known task queue:
crash> waitq c3534098
PID: 13691 TASK: c3534000 CPU: 1 COMMAND: "bash"
///
NAME
SYNOPSIS
dis [-rfludxs][-b [num]] [address | symbol | (expression)] [count]
DESCRIPTION
This command disassembles source code instructions starting (or ending) at
a text address that may be expressed by value, symbol or expression:
-r (reverse) displays all instructions from the start of the routine up to and including the designated address. -f (forward) displays all instructions from the given address to the end of the routine. -l displays source code line number data in addition to the disassembly output. -u address is a user virtual address in the current context; otherwise the address is assumed to be a kernel virtual address. If this option is used, then -r and -l are ignored. -x override default output format with hexadecimal format. -d override default output format with decimal format. -s displays the filename and line number of the source code that is associated with the specified text location, followed by a source code listing if it is available on the host machine. The line associated with the text location will be marked with an asterisk; depending upon gdb's internal "listsize" variable, several lines will precede the marked location. If a "count" argument is entered, it specifies the number of source code lines to be displayed after the marked location; otherwise the remaining source code of the containing function will be displayed. -b [num] modify the pre-calculated number of encoded bytes to skip after a kernel BUG ("ud2a") instruction; with no argument, displays the current number of bytes being skipped. (x86 and x86_64 only) address starting hexadecimal text address. symbol symbol of starting text address. On ppc64, the symbol preceded by '.' is used.
(expression) expression evaluating to a starting text address.
count the number of instructions to be disassembled (default is 1).
If no count argument is entered, and the starting address
is entered as a text symbol, then the whole routine will be
disassembled. The count argument is supported when used with
the -r and -f options.
EXAMPLES
Disassemble the sys_signal() routine without, and then with, line numbers:
crash> dis sys_signal 0xc0112c88 <sys_signal>: push %ebp 0xc0112c89 <sys_signal+1>: mov %esp,%ebp 0xc0112c8b <sys_signal+3>: sub $0x28,%esp 0xc0112c8e <sys_signal+6>: mov 0xc(%ebp),%eax 0xc0112c91 <sys_signal+9>: mov %eax,0xffffffec(%ebp) 0xc0112c94 <sys_signal+12>: movl $0xc0000000,0xfffffff0(%ebp) 0xc0112c9b <sys_signal+19>: lea 0xffffffd8(%ebp),%eax 0xc0112c9e <sys_signal+22>: push %eax 0xc0112c9f <sys_signal+23>: lea 0xffffffec(%ebp),%eax 0xc0112ca2 <sys_signal+26>: push %eax 0xc0112ca3 <sys_signal+27>: pushl 0x8(%ebp) 0xc0112ca6 <sys_signal+30>: call 0xc01124b8 <do_sigaction> 0xc0112cab <sys_signal+35>: test %eax,%eax 0xc0112cad <sys_signal+37>: jne 0xc0112cb2 <sys_signal+42> 0xc0112caf <sys_signal+39>: mov 0xffffffd8(%ebp),%eax 0xc0112cb2 <sys_signal+42>: leave 0xc0112cb3 <sys_signal+43>: ret crash> dis -l sys_signal /usr/src/linux-2.2.5/kernel/signal.c: 1074 0xc0112c88 <sys_signal>: push %ebp 0xc0112c89 <sys_signal+1>: mov %esp,%ebp 0xc0112c8b <sys_signal+3>: sub $0x28,%esp 0xc0112c8e <sys_signal+6>: mov 0xc(%ebp),%eax /usr/src/linux-2.2.5/kernel/signal.c: 1078 0xc0112c91 <sys_signal+9>: mov %eax,0xffffffec(%ebp) /usr/src/linux-2.2.5/kernel/signal.c: 1079 0xc0112c94 <sys_signal+12>: movl $0xc0000000,0xfffffff0(%ebp) /usr/src/linux-2.2.5/kernel/signal.c: 1081 0xc0112c9b <sys_signal+19>: lea 0xffffffd8(%ebp),%eax 0xc0112c9e <sys_signal+22>: push %eax 0xc0112c9f <sys_signal+23>: lea 0xffffffec(%ebp),%eax 0xc0112ca2 <sys_signal+26>: push %eax 0xc0112ca3 <sys_signal+27>: pushl 0x8(%ebp) 0xc0112ca6 <sys_signal+30>: call 0xc01124b8 <do_sigaction> /usr/src/linux-2.2.5/kernel/signal.c: 1083 0xc0112cab <sys_signal+35>: test %eax,%eax 0xc0112cad <sys_signal+37>: jne 0xc0112cb2 <sys_signal+42> 0xc0112caf <sys_signal+39>: mov 0xffffffd8(%ebp),%eax /usr/src/linux-2.2.5/kernel/signal.c: 1084 0xc0112cb2 <sys_signal+42>: leave 0xc0112cb3 <sys_signal+43>: ret
Given a return address expression of “do_no_page+65”, find out the
function that do_no_page() calls by using the reverse flag:
crash> dis -r (do_no_page+65) 0xc011ea68 <do_no_page>: push %ebp 0xc011ea69 <do_no_page+1>: mov %esp,%ebp 0xc011ea6b <do_no_page+3>: push %edi 0xc011ea6c <do_no_page+4>: push %esi 0xc011ea6d <do_no_page+5>: push %ebx 0xc011ea6e <do_no_page+6>: mov 0xc(%ebp),%ebx 0xc011ea71 <do_no_page+9>: mov 0x10(%ebp),%edx 0xc011ea74 <do_no_page+12>: mov 0x14(%ebp),%edi 0xc011ea77 <do_no_page+15>: mov 0x28(%ebx),%eax 0xc011ea7a <do_no_page+18>: test %eax,%eax 0xc011ea7c <do_no_page+20>: je 0xc011ea85 <do_no_page+29> 0xc011ea7e <do_no_page+22>: mov 0x18(%eax),%ecx 0xc011ea81 <do_no_page+25>: test %ecx,%ecx 0xc011ea83 <do_no_page+27>: jne 0xc011eab0 <do_no_page+72> 0xc011ea85 <do_no_page+29>: mov $0xffffe000,%eax 0xc011ea8a <do_no_page+34>: and %esp,%eax 0xc011ea8c <do_no_page+36>: decl 0x30(%eax) 0xc011ea8f <do_no_page+39>: jns 0xc011ea9a <do_no_page+50> 0xc011ea91 <do_no_page+41>: lock btrl $0x0,0xc022fb60 0xc011ea9a <do_no_page+50>: push %edi 0xc011ea9b <do_no_page+51>: mov 0x18(%ebp),%esi 0xc011ea9e <do_no_page+54>: push %esi 0xc011ea9f <do_no_page+55>: push %ebx 0xc011eaa0 <do_no_page+56>: mov 0x8(%ebp),%esi 0xc011eaa3 <do_no_page+59>: push %esi 0xc011eaa4 <do_no_page+60>: call 0xc011e9e4 <do_anonymous_page> 0xc011eaa9 <do_no_page+65>: jmp 0xc011eb47 <do_no_page+223>
Disassemble 10 instructions starting at user virtual address 0x81ec624:
crash> dis -u 81ec624 10
0x81ec624: push %ebp
0x81ec625: mov %esp,%ebp
0x81ec627: sub $0x18,%esp
0x81ec62a: movl $0x1,0x8(%ebp)
0x81ec631: mov 0x82f9040,%eax
0x81ec636: mov 0x10(%eax),%edx
0x81ec639: and $0x100,%edx
0x81ec63f: mov 0x14(%eax),%ecx
0x81ec642: and $0x0,%ecx
0x81ec645: mov %ecx,%eax
Override the current decimal output radix format:
crash> dis sys_read 10 -x
0xffffffff8001178f <sys_read>: push %r13
0xffffffff80011791 <sys_read+0x2>: mov %rsi,%r13
0xffffffff80011794 <sys_read+0x5>: push %r12
0xffffffff80011796 <sys_read+0x7>: mov $0xfffffffffffffff7,%r12
0xffffffff8001179d <sys_read+0xe>: push %rbp
0xffffffff8001179e <sys_read+0xf>: mov %rdx,%rbp
0xffffffff800117a1 <sys_read+0x12>: push %rbx
0xffffffff800117a2 <sys_read+0x13>: sub $0x18,%rsp
0xffffffff800117a6 <sys_read+0x17>: lea 0x14(%rsp),%rsi
0xffffffff800117ab <sys_read+0x1c>: callq 0xffffffff8000b5b4 <fget_light>
Disassemble from vfs_read+320 until the end of the function:
crash> dis -f vfs_read+320
0xffffffff8119d4e0 <vfs_read+320>: cmpq $0x0,0x20(%rax)
0xffffffff8119d4e5 <vfs_read+325>: jne 0xffffffff8119d3e8 <vfs_read+72>
0xffffffff8119d4eb <vfs_read+331>: mov $0xffffffffffffffea,%r12
0xffffffff8119d4f2 <vfs_read+338>: jmp 0xffffffff8119d4c3 <vfs_read+291>
0xffffffff8119d4f4 <vfs_read+340>: nopl 0x0(%rax)
0xffffffff8119d4f8 <vfs_read+344>: callq 0xffffffff8119cc40 <do_sync_read>
0xffffffff8119d4fd <vfs_read+349>: mov %rax,%r12
0xffffffff8119d500 <vfs_read+352>: jmpq 0xffffffff8119d44c <vfs_read+172>
0xffffffff8119d505 <vfs_read+357>: nopl (%rax)
0xffffffff8119d508 <vfs_read+360>: mov $0xfffffffffffffff7,%r12
0xffffffff8119d50f <vfs_read+367>: jmp 0xffffffff8119d4c3 <vfs_read+291>
0xffffffff8119d511 <vfs_read+369>: mov $0xfffffffffffffff2,%r12
0xffffffff8119d518 <vfs_read+376>: jmp 0xffffffff8119d4c3 <vfs_read+291>
0xffffffff8119d51a <vfs_read+378>: nopw 0x0(%rax,%rax,1)
Display the source code listing of the mmput() function:
crash> dis -s mmput FILE: kernel/fork.c LINE: 617 612 613 /* 614 * Decrement the use count and release all resources for an mm. 615 */ 616 void mmput(struct mm_struct *mm) * 617 { 618 might_sleep(); 619 620 if (atomic_dec_and_test(&mm->mm_users)) { 621 uprobe_clear_state(mm); 622 exit_aio(mm); 623 ksm_exit(mm); 624 khugepaged_exit(mm); /* must run before exit_mmap */ 625 exit_mmap(mm); 626 set_mm_exe_file(mm, NULL); 627 if (!list_empty(&mm->mmlist)) { 628 spin_lock(&mmlist_lock); 629 list_del(&mm->mmlist); 630 spin_unlock(&mmlist_lock); 631 } 632 if (mm->binfmt) 633 module_put(mm->binfmt->module); 634 mmdrop(mm); 635 } 636 }
The disassembly of dentry_kill() shows an indirect call to a function
whose address is contained within a register. Display the source code
associated with the indirect function call:
crash> dis dentry_kill ... 0xffffffff811dcfb4 <dentry_kill+324>: callq *%rax ... crash> dis -s 0xffffffff811dcfb4 FILE: fs/dcache.c LINE: 276 271 spin_unlock(&dentry->d_lock); 272 spin_unlock(&inode->i_lock); 273 if (!inode->i_nlink) 274 fsnotify_inoderemove(inode); 275 if (dentry->d_op && dentry->d_op->d_iput) * 276 dentry->d_op->d_iput(dentry, inode); 277 else 278 iput(inode); 279 } else { 280 spin_unlock(&dentry->d_lock); 281 } 282 }
///
NAME
SYNOPSIS
irq [[[index …] | -u ] | -d | -b | -a | -s [-c cpu]]
DESCRIPTION
This command collaborates the data in an irq_desc_t, along with its
associated hw_interrupt_type and irqaction structure data, into a
consolidated per-IRQ display. For kernel versions 2.6.37 and later
the display consists of the irq_desc/irq_data address, its irqaction
address(es), and the irqaction name strings. Alternatively, the
intel interrupt descriptor table, bottom half data, cpu affinity for
in-use irqs, or kernel irq stats may be displayed. If no index value
argument(s) nor any options are entered, the IRQ data for all IRQs will
be displayed.
index a valid IRQ index.
-u dump data for in-use IRQs only.
-d dump the intel interrupt descriptor table.
-b dump bottom half data.
-a dump cpu affinity for in-use IRQs.
-s dump the kernel irq stats; if no cpu specified with -c, the
irq stats of all cpus will be displayed.
-c cpu only usable with the -s option, dump the irq stats of the
specified cpu[s]; cpu can be specified as "1,3,5", "1-3",
"1,3,5-7,10", "all", or "a" (shortcut for "all").
EXAMPLES
Display the relevant data for IRQ 18 from a pre-2.6.37 kernel:
crash> irq 18 IRQ: 18 STATUS: 0 HANDLER: c02301e0 <ioapic_level_irq_type> typename: c01f9e0c "IO-APIC-level" startup: c0110234 <unmask_IO_APIC_irq> shutdown: c01101cc <mask_IO_APIC_irq> handle: c0110518 <do_level_ioapic_IRQ> enable: c0110234 <unmask_IO_APIC_irq> disable: c01101cc <mask_IO_APIC_irq> ACTION: c009c6b0 handler: c01ce818 <do_aic7xxx_isr> flags: 4000000 (SA_SHIRQ) mask: 0 name: c0217780 "aic7xxx" dev_id: c0090078 next: c009c770 ACTION: c009c770 handler: c01ce818 <do_aic7xxx_isr> flags: 4000000 (SA_SHIRQ) mask: 0 name: c0217780 "aic7xxx" dev_id: c0091078 next: 0 DEPTH: 0
Display the relevant data for IRQ 21 from a 2.6.37 kernel:
crash> irq 21
IRQ IRQ_DESC/_DATA IRQACTION NAME
21 ffff88003787f780 ffff8800379a8b40 "ehci_hcd:usb2"
ffff8800379cbac0 "uhci_hcd:usb5"
ffff8800379cb140 "uhci_hcd:usb7"
Display the intel interrupt descriptor table entries:
crash> irq -d [0] divide_error [1] debug [2] nmi [3] int3 [4] overflow [5] bounds [6] invalid_op [7] device_not_available [8] double_fault [9] coprocessor_segment_overrun [10] invalid_TSS [11] segment_not_present [12] stack_segment [13] general_protection [14] page_fault [15] spurious_interrupt_bug [16] coprocessor_error [17] alignment_check [18] ignore_int [19] ignore_int [20] ignore_int [21] ignore_int ... [250] IRQ0xda_interrupt [251] IRQ0xdb_interrupt [252] IRQ0xdc_interrupt [253] IRQ0xdd_interrupt [254] IRQ0xde_interrupt [255] spurious_interrupt
Display the bottom half data:
crash> irq -b
SOFTIRQ_VEC ACTION
[0] ffffffff81068f60 <tasklet_hi_action>
[1] ffffffff81071b80 <run_timer_softirq>
[2] ffffffff813e6f30 <net_tx_action>
[3] ffffffff813ee370 <net_rx_action>
[4] ffffffff81211a60 <blk_done_softirq>
[5] ffffffff812122f0 <blk_iopoll_softirq>
[6] ffffffff81069090 <tasklet_action>
[7] ffffffff81058830 <run_rebalance_domains>
[8] ffffffff81087f00 <run_hrtimer_softirq>
[9] ffffffff810ca7a0 <rcu_process_callbacks>
Display the cpu affinity for in-use IRQs:
crash> irq -a IRQ NAME AFFINITY 0 timer 0-23 1 i8042 0-23 8 rtc0 0-23 9 acpi 0-23 16 ehci_hcd:usb2,uhci_hcd:usb3,uhci_hcd:usb6 0,6,18 17 uhci_hcd:usb4,uhci_hcd:usb7 0-23 18 ehci_hcd:usb1,uhci_hcd:usb5,uhci_hcd:usb8,ioc0 0,11,23 24 dmar0 0 35 pciehp 0-23 36 pciehp 0-23 37 pciehp 0-23 38 pciehp 0-23 39 megasas 0-5,12-17 40 lpfc:sp 0-5,12-17 41 lpfc:fp 0,6-11,18-23 42 lpfc:sp 0,6-11,18-23 43 lpfc:fp 0,6-11,18-23 ... 80 ioat-msix 0-23 81 ioat-msix 0-23 82 ioat-msix 0-23 83 ioat-msix 0-23 84 ioat-msix 0-23 85 ioat-msix 0-23 86 ioat-msix 0-23 87 ioat-msix 0-23 88 eth4 0,17
Display the kernel irq stats:
crash>irq -c 0,2 -s
CPU0 CPU2
0: 2068161471 0 IR-IO-APIC-edge timer
1: 9 0 IR-IO-APIC-edge i8042
8: 1 0 IR-IO-APIC-edge rtc0
9: 0 0 IR-IO-APIC-fasteoi acpi
16: 36 0 IR-IO-APIC-fasteoi ehci_hcd:usb2
...
85: 3 0 IR-PCI-MSI-edge ioat-msix
86: 3 0 IR-PCI-MSI-edge ioat-msix
87: 3 0 IR-PCI-MSI-edge ioat-msix
88: 24 295 IR-PCI-MSI-edge eth4
///
NAME
SYNOPSIS
pte contents …
DESCRIPTION
This command translates the hexadecimal contents of a PTE into its physical
page address and page bit settings. If the PTE references a swap location,
the swap device and offset are displayed.
EXAMPLES
crash> pte d8e067
PTE PHYSICAL FLAGS
d8e067 d8e000 (PRESENT|RW|USER|ACCESSED|DIRTY)
crash> pte 13f600
PTE SWAP OFFSET
13f600 /dev/hda2 5104
///
NAME
SYNOPSIS
swap
DESCRIPTION
This command displays information for each configured swap device.
EXAMPLE
crash> swap
SWAP_INFO_STRUCT TYPE SIZE USED PCT PRI FILENAME
ffff880153d45f40 PARTITION 7192568k 1200580k 16% -1 /dev/dm-1
///
NAME
SYNOPSIS
whatis [[-o] [struct | union | typedef | symbol]] |
[[-r [size|range]] [-m member]]
DESCRIPTION
This command displays the definition of structures, unions, typedefs or
text/data symbols:
struct a structure name. The output is the same as if the "struct"
command was used.
union a union name. The output is the same as if the "union" command
was used.
-o display the offsets of structure/union members.
typedef a typedef name. If the typedef translates to a structure or union
the output is the same as if the “struct” or “union” command
was used. If the typedef is a primitive datatype, the one-line
declaration is displayed.
symbol a kernel symbol.
Alternatively, a search can be made for data structures of a given size or
size range, that contain a member of a given type, or contain a pointer to
given type. The -r and -m options may be used alone or in conjunction with
one another:
-r size search for structures of this exact size.
-r range search for structures of a range of sizes, expressed as "low-high".
-m member search for structures that contain a member of this data type, or
that contain a pointer to this data type; if a structure contains
another structure, the members of the embedded structure will also
be subject to the search. The member argument may also be expressed
as a substring of a member's data type.
EXAMPLES
Display the definition of a linux_binfmt structure:
crash> whatis linux_binfmt
struct linux_binfmt {
struct list_head lh;
struct module *module;
int (*load_binary)(struct linux_binprm *);
int (*load_shlib)(struct file *);
int (*core_dump)(struct coredump_params *);
unsigned long min_coredump;
}
SIZE: 56
Display the same structure with member offsets:
crash> whatis -o linux_binfmt
struct linux_binfmt {
[0] struct list_head lh;
[16] struct module *module;
[24] int (*load_binary)(struct linux_binprm *);
[32] int (*load_shlib)(struct file *);
[40] int (*core_dump)(struct coredump_params *);
[48] unsigned long min_coredump;
}
SIZE: 56
Since a kmem_bufctl_t is typedef’d to be a kmem_bufctl_s structure, the
output of the following two commands is identical:
crash> whatis kmem_bufctl_s struct kmem_bufctl_s { union { struct kmem_bufctl_s *buf_nextp; kmem_slab_t *buf_slabp; void *buf_objp; } u; }; crash> whatis kmem_bufctl_t struct kmem_bufctl_s { union { struct kmem_bufctl_s *buf_nextp; kmem_slab_t *buf_slabp; void *buf_objp; } u; }; SIZE: 4 (0x4)
Display the type data of sys_read() and jiffies text and data symbols:
crash> whatis sys_read
ssize_t sys_read(unsigned int, char *, size_t);
crash> whatis jiffies
long unsigned int jiffies;
Display definition of a kdev_t typedef:
crash> whatis kdev_t
typedef short unsigned int kdev_t;
SIZE: 2 (0x2)
Display all structures which have a size of 192 bytes:
crash> whatis -r 192 SIZE TYPE 192 _intel_private 192 blkcg_gq 192 clock_event_device 192 cper_sec_proc_generic 192 dentry 192 dst_ops 192 ehci_itd 192 ethtool_rxnfc 192 fb_ops 192 file_lock 192 inode_operations 192 input_device_id 192 ip_vs_stats 192 numa_group 192 parallel_data 192 pcie_port_service_driver 192 pebs_record_hsw 192 pnp_driver 192 regmap_config 192 sched_entity 192 tcp_timewait_sock 192 timerfd_ctx 192 tpm_vendor_specific 192 urb
Display all structures that contain members that point to
an mm_struct:
crash> whatis -m mm_struct SIZE TYPE 16 tlb_state 24 flush_tlb_info 24 ftrace_raw_xen_mmu_pgd 24 futex_key 24 map_info 32 ftrace_raw_xen_mmu_alloc_ptpage 32 ftrace_raw_xen_mmu_pte_clear 40 ftrace_raw_xen_mmu_flush_tlb_others 40 ftrace_raw_xen_mmu_ptep_modify_prot 40 ftrace_raw_xen_mmu_set_pte_at 40 mm_slot 64 mm_walk 64 rmap_item 104 userfaultfd_ctx 128 mmu_gather 216 vm_area_struct 256 linux_binprm 2616 rq 2936 task_struct
Display all structures sized from 256 to 512 bytes that
contain members that point to a task_struct:
crash> whatis -r 256-512 -m task_struct SIZE TYPE 256 file 256 od_cpu_dbs_info_s 264 srcu_notifier_head 272 protection_domain 288 clk_notifier 288 fsnotify_group 296 quota_info 312 tty_port 320 workqueue_struct 344 trace_array 344 uart_state 352 cpufreq_policy 352 elf_thread_core_info 376 perf_event_context 384 rcu_data 400 cgroup 408 subsys_private 424 hvc_struct 496 psmouse
///
NAME
SYNOPSIS
eval [-b][-l] (expression) | value
DESCRIPTION
This command evaluates an expression or numeric value, and displays its
result in hexadecimal, decimal, octal and binary. If the resultant value
is an integral number of gigabytes, megabytes, or kilobytes, a short-hand
translation of the number will also be shown next to the hexadecimal
value. If the most significant bit is set, the decimal display will show
both unsigned and signed (negative) values. Expressions must of the format
(x operator y), where “x” and “y” may be either numeric values or
symbols. The list of operators are:
+ - & | ^ * % / << >>
Enclosing the expression within parentheses is optional except when the
“|”, “<<” or “>>” operators are used. The single “value” argument may
be a number or symbol. Number arguments must be hexadecimal or decimal.
A leading “0x” identifies a number as hexadecimal, but is not required
when obvious. Numbers may be followed by the letters “k” or “K”, “m”
or “M”, and “g” or “G”, which multiplies the value by a factor of 1024,
1 megabyte or 1 gigabyte, respectively. Numeric arguments may be preceded
by the one’s complement operator ~.
-b Indicate which bit positions in the resultant value are set.
-l Numeric arguments are presumed to be 64-bit values, and the result
will be expressed as a 64-bit value. (ignored on 64-bit processors)
However, if either operand or the resultant value are 64-bit values,
then the result will be also be expressed as a 64-bit value.
The -b and -l options must precede the expression or value arguments.
EXAMPLES
crash> eval 128m hexadecimal: 8000000 (128MB) decimal: 134217728 octal: 1000000000 binary: 00001000000000000000000000000000 crash> eval 128 * 1m hexadecimal: 8000000 (128MB) decimal: 134217728 octal: 1000000000 binary: 00001000000000000000000000000000 crash> eval (1 << 27) hexadecimal: 8000000 (128MB) decimal: 134217728 octal: 1000000000 binary: 00001000000000000000000000000000 crash> eval (1 << 32) hexadecimal: 100000000 (4GB) decimal: 4294967296 octal: 40000000000 binary: 0000000000000000000000000000000100000000000000000000000000000000 crash> eval -b 41dc065 hexadecimal: 41dc065 decimal: 69058661 octal: 407340145 binary: 00000100000111011100000001100101 bits set: 26 20 19 18 16 15 14 6 5 2 0 crash> eval -lb 64g hexadecimal: 1000000000 (64GB) decimal: 68719476736 octal: 1000000000000 binary: 0000000000000000000000000001000000000000000000000000000000000000 bits set: 36
///
NAME
SYNOPSIS
kmem [-f|-F|-c|-C|-i|-v|-V|-n|-z|-o|-h] [-p | -m member[,member]]
[[-s|-S|-r] [slab] [-I slab[,slab]]] [-g [flags]] [[-P] address]]
DESCRIPTION
This command displays information about the use of kernel memory.
-f displays the contents of the system free memory headers. also verifies that the page count equals nr_free_pages. -F same as -f, but also dumps all pages linked to that header. -c walks through the page_hash_table and verifies page_cache_size. -C same as -c, but also dumps all pages in the page_hash_table. -i displays general memory usage information -v displays the mapped virtual memory regions allocated by vmalloc(). -V displays the kernel vm_stat table if it exists, or in more recent kernels, the vm_zone_stat, vm_node_stat and vm_numa_stat tables, the cumulative page_states counter values if they exist, and/or the cumulative, vm_event_states counter values if they exist. -n display memory node, memory section, and memory block data and state; the state of each memory section state is encoded as "P", "M", "O" and/or "E", meaning SECTION_MARKED_PRESENT, SECTION_HAS_MEM_MAP, SECTION_IS_ONLINE and SECTION_IS_EARLY. -z displays per-zone memory statistics. -o displays each cpu's offset value that is added to per-cpu symbol values to translate them into kernel virtual addresses. -h display the address of hugepage hstate array entries, along with their hugepage size, total and free counts, and name. -p displays basic information about each page structure in the system mem_map[] array, made up of the page struct address, its associated physical address, the page.mapping, page.index, page._count and page.flags fields. -m member similar to -p, but displays page structure contents specified by a comma-separated list of one or more struct page members. The "flags" member will always be expressed in hexadecimal format, and the "_count" and "_mapcount" members will always be expressed in decimal format. Otherwise, all other members will be displayed in hexadecimal format unless the output radix is 10 and the member is a signed/unsigned integer. Members that are data structures may be specified either by the data structure's member name, or expanded to specify a member of the data structure. For example, "-m lru" refers to a list_head data structure, and both the list_head.next and list_head.prev pointer values will be displayed, whereas if "-m lru.next" is specified, just the list_head.next value will be displayed. -s displays basic kmalloc() slab data. -S displays all kmalloc() slab data, including all slab objects, and whether each object is in use or is free. If CONFIG_SLUB, slab data for each per-cpu slab is displayed, along with the address of each kmem_cache_node, its count of full and partial slabs, and a list of all tracked slabs. -r displays the accumulated basic kmalloc() slab data of each root slab cache and its children. The kernel must contain the "slab_root_caches" list_head. (currently only available if CONFIG_SLUB)
slab when used with -s, -S or -r, limits the command to only the slab
cache of name “slab”. If the slab argument is “list”, then
all slab cache names and addresses are listed.
-I slab when used with -s, -S or -r, one or more slab cache names in a
comma-separated list may be specified as slab caches to ignore.
-
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。