爲什麼「pstack」只打印一個線程的內容？

Question

我OS是RHEL 7，我運行一個簡單的Go程序：

package main 

import (
    "time" 
) 

func main() { 
    time.Sleep(1000 * time.Second) 
} 

在它的運行，我檢查進程的線程數：

# cat /proc/13858/status | grep Thread 
Threads:  5 

當使用pstack命令出貨RHEL，它只打印一個線程的堆棧：

# pstack 13858 
Thread 1 (process 13858): 
#0 runtime.futex() at /usr/local/go/src/runtime/sys_linux_amd64.s:307 
#1 0x0000000000422580 in runtime.futexsleep (addr=0x4c7af8 <runtime.timers+24>, val=0, ns=999999997446) at /usr/local/go/src/runtime/os1_linux.go:57 
#2 0x000000000040b07b in runtime.notetsleep_internal (n=0x4c7af8 <runtime.timers+24>, ns=999999997446, ~r2=255) at /usr/local/go/src/runtime/lock_futex.go:174 
#3 0x000000000040b1e6 in runtime.notetsleepg (n=0x4c7af8 <runtime.timers+24>, ns=999999997446, ~r2=false) at /usr/local/go/src/runtime/lock_futex.go:206 
#4 0x000000000043e5de in runtime.timerproc() at /usr/local/go/src/runtime/time.go:209 
#5 0x0000000000451001 in runtime.goexit() at /usr/local/go/src/runtime/asm_amd64.s:1998 
#6 0x0000000000000000 in ??() 

爲什麼pstack只能打印一個線程的內容？

P.S：該pstack腳本是在這裏：

#!/bin/sh 

if test $# -ne 1; then 
    echo "Usage: `basename $0 .sh` <process-id>" 1>&2 
    exit 1 
fi 

if test ! -r /proc/$1; then 
    echo "Process $1 not found." 1>&2 
    exit 1 
fi 

# GDB doesn't allow "thread apply all bt" when the process isn't 
# threaded; need to peek at the process to determine if that or the 
# simpler "bt" should be used. 

backtrace="bt" 
if test -d /proc/$1/task ; then 
    # Newer kernel; has a task/ directory. 
    if test `/bin/ls /proc/$1/task | /usr/bin/wc -l` -gt 1 2>/dev/null ; then 
     backtrace="thread apply all bt" 
    fi 
elif test -f /proc/$1/maps ; then 
    # Older kernel; go by it loading libpthread. 
    if /bin/grep -e libpthread /proc/$1/maps > /dev/null 2>&1 ; then 
     backtrace="thread apply all bt" 
    fi 
fi 

GDB=${GDB:-/usr/bin/gdb} 

# Run GDB, strip out unwanted noise. 
# --readnever is no longer used since .gdb_index is now in use. 
$GDB --quiet -nx $GDBARGS /proc/$1/exe $1 <<EOF 2>&1 | 
set width 0 
set height 0 
set pagination no 
$backtrace 
EOF 
/bin/sed -n \ 
    -e 's/^\((gdb) \)*//' \ 
    -e '/^#/p' \ 
    -e '/^Thread/p' 

Answer 1

pstack使用gdb。這是來自golang doc的報價（https://golang.org/doc/gdb）：

GDB不理解Go程序。堆棧管理 線程和運行時包含的方面與 執行模型有很大的不同.GDB希望當程序使用gccgo編譯時，他們可能會混淆調試器，甚至是 。因此，雖然GDB在某些情況下可能很有用，但它不適用於Go程序的可靠調試器 ，特別是高度併發的程序。

即使在程序進入main之前，您在/ proc中看到的5個線程中也有4個被創建。我假設golang運行時創建它們。

爲什麼pstack只打印一個線程的內容？

從strace的輸出爲GDB我看到gdb實際上是嘗試連接到他們，但後來看東西出了問題，並gdb不會嘗試檢查這些線程。這些是GDB對於這些運行的線程，但由於未知的原因發出的系統調用決定立即停止調查他們：

5072 ptrace(PTRACE_ATTACH, 5023, 0, 0) = 0 
5072 --- SIGCHLD (Child exited) @ 0 (0) --- 
5072 rt_sigreturn(0x11)    = 0 
5072 ptrace(PTRACE_ATTACH, 5024, 0, 0) = 0 
5072 --- SIGCHLD (Child exited) @ 0 (0) --- 
5072 rt_sigreturn(0x11)    = 0 
5072 ptrace(PTRACE_ATTACH, 5025, 0, 0) = 0 
5072 --- SIGCHLD (Child exited) @ 0 (0) --- 
5072 rt_sigreturn(0x11)    = 0 

但是你可以自己檢查它們。看來，這些線程屬於golang運行

$ pstack 5094 
Thread 1 (process 5094): 
#0 0x0000000000459243 in runtime.futex() 
#1 0x00000000004271e0 in runtime.futexsleep() 
#2 0x000000000040d55b in runtime.notetsleep_internal() 
#3 0x000000000040d64b in runtime.notetsleep() 
#4 0x0000000000435677 in runtime.sysmon() 
#5 0x000000000042e6cc in runtime.mstart1() 
#6 0x000000000042e5d2 in runtime.mstart() 
#7 0x00000000004592b7 in runtime.clone() 
#8 0x0000000000000000 in ??() 

$ pstack 5095 
Thread 1 (process 5095): 
#0 0x0000000000459243 in runtime.futex() 
#1 0x0000000000427143 in runtime.futexsleep() 
#2 0x000000000040d3f4 in runtime.notesleep() 
#3 0x000000000042f6eb in runtime.stopm() 
#4 0x0000000000430a79 in runtime.findrunnable() 
#5 0x00000000004310ff in runtime.schedule() 
#6 0x000000000043139b in runtime.park_m() 
#7 0x0000000000455acb in runtime.mcall() 
#8 0x000000c820021500 in ??() 
#9 0x0000000000000000 in ??() 

$ pstack 5096 
Thread 1 (process 5096): 
#0 0x0000000000459243 in runtime.futex() 
#1 0x0000000000427143 in runtime.futexsleep() 
#2 0x000000000040d3f4 in runtime.notesleep() 
#3 0x000000000042f6eb in runtime.stopm() 
#4 0x000000000042fff7 in runtime.startlockedm() 
#5 0x0000000000431147 in runtime.schedule() 
#6 0x000000000043139b in runtime.park_m() 
#7 0x0000000000455acb in runtime.mcall() 
#8 0x000000c820020000 in ??() 

---

更新GDB使用GDB 8.0正確打印回溯所有threas 8.0

的pstack。該命令如下：

$ GDB=$HOME/bin/gdb pstack $(pidof main) 

這裏是它的輸出（縮短）：

$ GDB=$HOME/bin/gdb pstack $(pidof main) | egrep "^Thread" 
Thread 4 (LWP 18335): 
Thread 3 (LWP 18334): 
Thread 2 (LWP 18333): 
Thread 1 (LWP 18332): 

Answer 2

當你傳遞LWP /主題ID pstack你只有該線程的堆棧。嘗試將進程的PID傳遞給pstack，您將獲得所有線程的堆棧。您可能會得到進程的PID或Tgid（線程組ID）：cat /proc/13858/status | grep Tgid。要獲得由您的流程創建的所有LWP，您可以運行ps -L <PID>