|
Home > Archive > Data Storage > October 2004 > Intermittent spiking of write queue
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Intermittent spiking of write queue
|
|
| dwayne 2004-09-28, 7:54 am |
| I have a Dell Win2k NAS connected via an emulex hba and a Brocade switch to
an NEC SAN. I'm storing lots of small files and folders on a 14 disk raid
10 set within the san. The disk load that I see is about 60% write & 40 %
read.
Everything runs well and performs as expected, but intermittently I see a
1 -2 sec spike in the write queue, which seems to freeze network and disk
traffic. This seems to happen mostly about once a minute, but not always
and they're not spaced evenly apart.
We're in the process of replacing the SAN at the moment, with the thought
that the SAN is the cause of the problem. But I'm not 100% convinced that
the san is at fault.
Does anyone have any suggestions on what could be the issue and what the
best way to locate the cause of the problem?
| |
|
| I don't think there's any way to isolate which component has the issue
without some investigation...
The components involved seem to include:
1.) Remote, NAS-client Hosts
2.) NAS-client network connection (including ethernet cards, switches,
etc.)
3.) W2k NAS Server OS
4.) W2k NAS Server HBA
5.) W2k NAS Server FC path
6.) Brocade host port
7.) Brocade Switch
8.) Brocade storage port
9.) NEC array FC path
10.) NEC array FC port
11.) NEC array
If the problem extends to multiple hosts, you can probably eliminate 1
& 2 (for now). To eliminate the rest of the issues, you'll want to
check each item top down. I wouldn't suggest trading out gear as your
first action-- you might only need to re-seat a cable!
I'd start by looking at the I/O stats on the NAS server (through
perfmon), and the port stats on the HBA. If you find I/O
errors/latency on the server, but nothing on the HBA skip to component
7. Keep looking for errors or anomalies that coincide with your
errors. If you find nothing, revisit 1 & 2 starting with things you
know to be common between hosts.
If you start by replacing hardware, you may get lucky (I guess a 1/11
chance isn't bad); more likely, you'll spend money with no change.
IMHO proper trouble-shooting will save you time & money.
HTH
Aaron
|
|
|
|
|