|
Home > Archive > Unix administration > June 2006 > cs and sy of vmstat's output is very hight
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
cs and sy of vmstat's output is very hight
|
|
|
| hello, all
IĦĦhave two HP 4410 and each one with 8 cpu. Now one machine system
time is hight
and about 30% and before it would almost be 0. and from the output of
vmstat, the cs is
very high. I don't know why it is so and how to resolv it.
Below is the outout of vmstat of the two machine.
# vmstat 3 100
procs memory page
faults cpu
r b w avm free re at pi po fr de sr
in
sy cs us sy id
11 11 0 3909772 3222627 404 21 0 0 0 0 0
2812
108606 66259 15 20 64
11 11 0 3909772 3222726 94 12 0 0 0 0 0
3710
181999 100208 31 32 37
12 8 0 3912122 3222725 118 26 0 0 0 0 0
3512
182956 102284 24 32 43
12 8 0 3912122 3222619 104 19 0 0 0 0 0
3535
180744 102897 28 32 40
# vmstat 3 100
procs memory page
faults cpu
r b w avm free re at pi po fr de sr
in
sy cs us sy id
2 0 0 890622 4109711 122 27 0 0 0 0 0
3343
30647 1045 2 1 97
1 0 0 890495 4109762 68 10 0 0 0 0 0
4133
18688 1793 3 1 97
1 0 0 890495 4109761 50 6 0 0 0 0 0
4561
19810 2027 5 1 94
1 0 0 890495 4109761 56 6 0 0 0 0 0
4972
23331 2319 6 1 93
the out put of sar is :
# sar -A 1 100
HP-UX fzinms_1 B.11.11 U 9000/800 06/19/06
15:22:21 %usr %sys %wio %idle
device %busy avque r+w/s blks/s avwait avserv
runq-sz %runocc swpq-sz %swpocc
bread/s lread/s %rcache bwrit/s lwrit/s %wcache pread/s pwrit/s
swpin/s bswin/s swpot/s bswot/s pswch/s
scall/s sread/s swrit/s fork/s exec/s rchar/s wchar/s
iget/s namei/s dirbk/s
rawch/s canch/s outch/s rcvin/s xmtin/s mdmin/s
text-sz ov proc-sz ov inod-sz ov file-sz ov
msg/s sema/s
15:22:22 40 26 4 29
c0t6d0 1.96 0.50 6 182 0.00 4.20
c3t6d0 1.96 0.50 5 178 0.00 5.88
c10t0d6 75.49 0.50 148 2416 0.00 5.58
2.6 61 0.0 0
151 8722 98 11 565 98 0 0
0.00 0.0 0.00 0.0 74387
152771 8151 32361 8.82 8.82 71429016 64251
1 4265 0
0 0 0 0 0 0
N/A N/A 472/8212 0 2129/9412 0 5817/76025 0
0.00 168.63
15:22:23 16 37 5 42
c0t6d0 8.00 0.50 6 160 0.00 30.40
c3t6d0 9.00 0.50 6 160 0.00 34.72
c10t0d6 81.00 0.50 163 2704 0.00 4.95
2.0 12 0.0 0
169 8106 98 12 372 97 0 1
0.00 0.0 0.00 0.0 109752
154224 6451 44191 6.00 6.00 66387968 106496
0 2905 0
0 0 0 0 0 0
N/A N/A 472/8212 0 2128/9412 0 5817/76025 0
0.00 824.00
Any ideas?
Thanks,
PanTao
| |
| Juha Laiho 2006-06-19, 7:23 pm |
| "pan" <bandianxing@hotmail.com> said:
> IĦĦhave two HP 4410 and each one with 8 cpu. Now one machine system
>time is hight
>and about 30% and before it would almost be 0. and from the output of
>vmstat, the cs is
>very high. I don't know why it is so and how to resolv it.
A lot of context switches and high system time usage. This sounds like
a fork bomb (though not one of the worst kind, and most possibly
an inadvertent one, just a software behaving badly, not one created with
malicious intent). Could be other reasons, but this would be the most
probable cause.
Now, the trick is to find the process which is creating all these new
processes. "top" (if you even have it installed) will most probably
not help with these (the generated processes are short-lived, and as
such only show as random flashes in 'top', and the parent process
creating all these new processes is consuming only a small fraction
of the system resources; most of the consumption is on these new
processes).
You could try running a few iterations of "ps -ef"; look for processes
which appear in only one of the listings and go away. You should find
several of these. Now the key will be the process which is the parent
for all of these short-lived processes. Kill the parent process, and
restart it if it is needed for system operation (in other words, if it
is not just a leftover from a broken terminal session or something
like that).
If you have Glance+ available, you could try filtering the
process list there to show the processes with highest system call
rate, or perhaps even filter to the specific system call (fork
is what I would expect to see here a lot). The GUI version of
Glance+ (gpm) is somewhat of a video game, but provides a good tool
just for this kind of situations.
--
Wolf a.k.a. Juha Laiho Espoo, Finland
(GC 3.0) GIT d- s+: a C++ ULSH++++$ P++@ L+++ E- W+$@ N++ !K w !O !M V
PS(+) PE Y+ PGP(+) t- 5 !X R !tv b+ !DI D G e+ h---- r+++ y++++
"...cancel my subscription to the resurrection!" (Jim Morrison)
| |
|
|
"Juha Laiho" <Juha.Laiho@iki.fi> ??????:e76s09$qsr$1@ichaos2.ichaos-int...
> "pan" <bandianxing@hotmail.com> said:
>
> A lot of context switches and high system time usage. This sounds like
> a fork bomb (though not one of the worst kind, and most possibly
> an inadvertent one, just a software behaving badly, not one created with
> malicious intent). Could be other reasons, but this would be the most
> probable cause.
>
> Now, the trick is to find the process which is creating all these new
> processes. "top" (if you even have it installed) will most probably
> not help with these (the generated processes are short-lived, and as
> such only show as random flashes in 'top', and the parent process
> creating all these new processes is consuming only a small fraction
> of the system resources; most of the consumption is on these new
> processes).
>
> You could try running a few iterations of "ps -ef"; look for processes
> which appear in only one of the listings and go away. You should find
> several of these. Now the key will be the process which is the parent
> for all of these short-lived processes. Kill the parent process, and
> restart it if it is needed for system operation (in other words, if it
> is not just a leftover from a broken terminal session or something
> like that).
>
Yeah, It seems it is really so. I use ps , awk , sort and comm, I found
the temporay process ( which is "sleep 2") and the Parent process seems be
a script of one hp cluster package.
many thanks,
PanTao
|
|
|
|
|