Posted on 2012-03-14 04:43
S.l.e!ep.￠% 閱讀(2776)
評論(0) 編輯 收藏 引用 所屬分類:
Unix
如何定位死循環或高CPU使用率(linux)
?確定是CPU過高
使用top觀察是否存在CPU使用率過高現象
找出線程
對CPU使用率過高的進程的所有線程進行排序
ps H -e -o pid,tid,pcpu,cmd --sort=pcpu |grep xxx得到如下結果,其中線程2909使用了7.8%的CPU. 2907 2913 0.0 ./xxx 2907 2909 7.8 ./xxx也可以通過查看/proc中的信息來確定高CPU線程. 打印了4列,線程ID,線程名,用戶時間和內核時間(排名未分先后) awk '{print $1,$2,$14,$15}' /proc/2907/task/*/stat
找出調用棧
使用gdb attach nmsagent所在的進程,在gdb中使用 info threads顯示所有線程
gdb
gdb>attach 2907
gdb>info threads
得到如下結果,可以發現2909線程的編號是12
? 13 Thread 0xad5f2b70 (LWP 2908)? 0x004ef0d7 in mq_timedreceive () from /lib/tls/i686/cmov/librt.so.1
? 12 Thread 0xad58eb70 (LWP 2909)? 0x006e0422 in __kernel_vsyscall ()
? 11 Thread 0xad52ab70 (LWP 2910)? 0x006e0422 in __kernel_vsyscall ()
? 10 Thread 0xad4f8b70 (LWP 2911)? 0x006e0422 in __kernel_vsyscall ()
? 9 Thread 0xad4c6b70 (LWP 2912)? 0x006e0422 in __kernel_vsyscall ()
? 8 Thread 0xad3feb70 (LWP 2913)? 0x004ef0d7 in mq_timedreceive () from /lib/tls/i686/cmov/librt.so.1
? 7 Thread 0xace08b70 (LWP 2914)? 0x004ef0d7 in mq_timedreceive () from /lib/tls/i686/cmov/librt.so.1
? 6 Thread 0xac607b70 (LWP 2915)? 0x006e0422 in __kernel_vsyscall ()
? 5 Thread 0xac5e6b70 (LWP 2916)? 0x006e0422 in __kernel_vsyscall ()
? 4 Thread 0xac361b70 (LWP 2917)? 0x006e0422 in __kernel_vsyscall ()
? 3 Thread 0xac2fdb70 (LWP 2918)? 0x006e0422 in __kernel_vsyscall ()
? 2 Thread 0xac1fcb70 (LWP 2919)? 0x004ef0d7 in mq_timedreceive () from /lib/tls/i686/cmov/librt.so.1
* 1 Thread 0xb78496d0 (LWP 2907)? 0x006e0422 in __kernel_vsyscall ()
使用thread 切換線程,使用bt顯示線程棧
gdb>thread 12
gdb>bt
得到如下線程棧
#0? 0x006e0422 in __kernel_vsyscall ()
#1? 0x001cca26 in nanosleep () from /lib/tls/i686/cmov/libc.so.6
#2? 0x001fc2dc in usleep () from /lib/tls/i686/cmov/libc.so.6
#3? 0x0806b510 in OspTaskDelay ()
#4? 0x0805c710 in CDispatchTask::NodeMsgSendToSock() ()
#5? 0x0805cc74 in DispatchTaskEntry ()
#6? 0x0806a8e9 in OspTaskTemplateFunc(void*) ()
#7? 0x00d4780e in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
?#8? 0x002027ee in clone () from /lib/tls/i686/cmov/libc.so.6
?ps + strace
得到進程ID 21465
ps -e |grep cmu
?4996 ???????? 00:00:25 cmu_fjga_sp3
21465 pts/5??? 00:08:10 cmu
得到線程時間, 其中最占CPU的是 EpollRecvTask 21581
ps -eL |grep 21465 21465 21579 pts/5 00:00:00 CamApp 21465 21580 pts/5 00:00:00 TimerMan Task 21465 21581 pts/5 00:09:02 EpollRecvTask 21465 21582 pts/5 00:00:00 使用 strace -p 21581 得到線程棧