Data Storage - Which Combination Is Faster?

Author

Which Combination Is Faster?

Eric Robinson

2006-06-25, 7:17 pm

Your opinions please!

Which is faster:

(6) U320 15K drives in a RAID 10 array, attached to a PCI-X controller.

or

(6) SATA-300 10K drives in a RAID 10 array, attached to a PCI-E controller.

Based on my understanding of the bottlenecks, I would tend to think the 10K
SATA array on the PCI-E controller would be faster than the 15K SCSI array
on the PCI-X controller.

Your thoughts?

Rob Turk

2006-06-25, 7:17 pm

"Eric Robinson" <eric{nospam}@pmcipa.com> wrote in message
news:QsCng.3442$CW.1745@fe04.lga...
> Your opinions please!
>
> Which is faster:
>
> (6) U320 15K drives in a RAID 10 array, attached to a PCI-X controller.
> or
> (6) SATA-300 10K drives in a RAID 10 array, attached to a PCI-E
> controller.
>
> Based on my understanding of the bottlenecks, I would tend to think the
> 10K SATA array on the PCI-E controller would be faster than the 15K SCSI
> array on the PCI-X controller.
>
> Your thoughts?

As always, it depends. Is your application bandwidth or access time
constrained? If you want to run a constant stream of data to/from the RAID
set then you may be right and SATA with PCI-E may be best. If you have a
heavy database on it with many simultaneous queries then the SCSI disks may
be faster due to better seek time and lower rotational latency.

Rob

Eric Robinson

2006-06-26, 1:12 am

That's exactly the kind of answer I was looking for, thanks.

"Rob Turk" <_wipe_me_r.turk@chello.nl> wrote in message
news:7bDng.2718$YI3.1555@amstwist00...
> "Eric Robinson" <eric{nospam}@pmcipa.com> wrote in message
> news:QsCng.3442$CW.1745@fe04.lga...
>
> As always, it depends. Is your application bandwidth or access time
> constrained? If you want to run a constant stream of data to/from the
> RAID set then you may be right and SATA with PCI-E may be best. If you
> have a heavy database on it with many simultaneous queries then the SCSI
> disks may be faster due to better seek time and lower rotational latency.
>
> Rob
>
>

Torbjorn Lindgren

2006-06-26, 7:12 pm

Rob Turk <_wipe_me_r.turk@chello.nl> wrote:
>"Eric Robinson" <eric{nospam}@pmcipa.com> wrote in message
>news:QsCng.3442$CW.1745@fe04.lga...
>
>As always, it depends. Is your application bandwidth or access time
>constrained? If you want to run a constant stream of data to/from the RAID
>set then you may be right and SATA with PCI-E may be best. If you have a
>heavy database on it with many simultaneous queries then the SCSI disks may
>be faster due to better seek time and lower rotational latency.

And details matter a lot, many of which isn't in the original post,
but as a first stab it's a good summary. Assuming current disks, using
figures from Seagate for 15k disk (Cheetah 15k5, but the older 15k4 is
very similar).

The problem starts with the "SATA-300 10K", the only SATA 10k drive I
know of is the WD Raptor, which WD says is SATA-150 :-) Not that I
expect that to actually MATTER, so lets use the latest WD Raptor here.

At one extreme a small random load (high IOPS, very smaller IO,
totally random) the 6x15k array is likely to be able to handle between
30% (avg. seek, it used to be larger) and 50% more (rotational
speed/latency) random ops.

The other extreme is streaming data, where I expected the 10k drives
to be as fast or faster (higher spindle speed but lower density).

However, Seagate properly lists both low and high sustained external
transfer, but WD only lists a single value. The question is WHAT WD is
listing, it could be the lowest figure, the average over the entire
disk or the peak... WD's figure is about 15% higher than Seagate "low"
value and 67% of their "high" value.

Seagate 15k5, 6x15k -> 6x73-125MB/s -> 438-750MB/s
WD Raptor (150GB/16MB), 6x10k -> 6x84MB/s -> 504MB/s (but where!?)

All this is of course assuming enough SCSI channels that *they* don't
get satured first! SATA is less likely to hit this, but there ARE
"port multiplication" features for SATA... :-)

It also assumes that the controller and/or the machine is capable of
handling these kind of bandwiths, which may or may not be the case :-)

Since PCI-E cards is in general newer they may fare slightly better
here (though RAID-10 is usually less affected than RAID5), on the
other hand PCI-X has long been pushed as a server solution and some of
the PCI-E cards are based on their PCI-X cousins.

We also need to figure out what PCI-X is in this case, the smallest
version is 64/66 with a theoretical speed of 528 MB/s. Getting above
450 MB/s on it is likely to be hard, so it could push the 6x15k's top
speed below that of the 6x10k (though it would be faster over almost
all of the disk).

But it's fairly common to have multiple 100MHz or 133MHz PCI-X slots
in a server (with 798MB/s and 1064MB/s theoretical speed),
significantly above that of the 6x10k array above (There's also PCI-X
2.0 for faster speeds).

When it comes to PCI-E, how fast it is depends on the number of lanes
and it's also different in that it provides a set amount of bandwidth
in both directions instead of the shared model used by PCI/PCI-X.

Each PCI-E lane has a raw capacity of 238MB/s in each direction, so
for mostly unidirectional traffic you'd need 2 lanes to match the
slowest PCI-X interface (more balance traffic fares better).

So, a low-end PCI-E SATA controller (1x) would have less bandwidth
available, but I don't think it's likely you'll find one with 6 or
more SATA ports. 4x or 8x seems more likely and even in the worst case
scenario the 4x should have more bandwidth available than 133/64
PCI-X. However, you might be able to do it with two 4-port PCI-E SATA
cards, which IS likely to be 1x cards (I'm ignoring those nearly
mythical port multipliers until I've actually seen them and they
worked!).

My best guess is that the PCI-X/6x15k array is probably going to be
significantly faster in IOPS, but in the transfer rate test it would
depend on details such as exact disk models used and how fast the
"PCI-X" is :-) (and assuming the PCI-E "controller" isn't two 1x
models)

To some extent it's probably better to compare not a set number of
disks but performance against some OTHER metric, such as cost or power
usage! OTOH then Raptor's isn't exactly known for being cheap, and the
cost would have to include all other elements (support, chassi and so
on) which makes it very hard to make a fair comparison.

http://en.wikipedia.org/wiki/PCI-X
http://en.wikipedia.org/wiki/PCI-Express

Eric Robinson

2006-06-26, 7:12 pm

Thanks for your thoughtful and detailed reply. Good information.

For what it's worth, this benchmark article addressed the question to my
satisfaction. Other people might also find it helpful.

http://www.tomshardware.com/2005/11...ss_battles_pci/

"Torbjorn Lindgren" <tl@none.invalid> wrote in message
news:B4Wng.15912$Bh2.5786@fe07.news.easynews.com...
> Rob Turk <_wipe_me_r.turk@chello.nl> wrote:
>
> And details matter a lot, many of which isn't in the original post,
> but as a first stab it's a good summary. Assuming current disks, using
> figures from Seagate for 15k disk (Cheetah 15k5, but the older 15k4 is
> very similar).
>
> The problem starts with the "SATA-300 10K", the only SATA 10k drive I
> know of is the WD Raptor, which WD says is SATA-150 :-) Not that I
> expect that to actually MATTER, so lets use the latest WD Raptor here.
>
> At one extreme a small random load (high IOPS, very smaller IO,
> totally random) the 6x15k array is likely to be able to handle between
> 30% (avg. seek, it used to be larger) and 50% more (rotational
> speed/latency) random ops.
>
> The other extreme is streaming data, where I expected the 10k drives
> to be as fast or faster (higher spindle speed but lower density).
>
> However, Seagate properly lists both low and high sustained external
> transfer, but WD only lists a single value. The question is WHAT WD is
> listing, it could be the lowest figure, the average over the entire
> disk or the peak... WD's figure is about 15% higher than Seagate "low"
> value and 67% of their "high" value.
>
> Seagate 15k5, 6x15k -> 6x73-125MB/s -> 438-750MB/s
> WD Raptor (150GB/16MB), 6x10k -> 6x84MB/s -> 504MB/s (but where!?)
>
> All this is of course assuming enough SCSI channels that *they* don't
> get satured first! SATA is less likely to hit this, but there ARE
> "port multiplication" features for SATA... :-)
>
> It also assumes that the controller and/or the machine is capable of
> handling these kind of bandwiths, which may or may not be the case :-)
>
> Since PCI-E cards is in general newer they may fare slightly better
> here (though RAID-10 is usually less affected than RAID5), on the
> other hand PCI-X has long been pushed as a server solution and some of
> the PCI-E cards are based on their PCI-X cousins.
>
> We also need to figure out what PCI-X is in this case, the smallest
> version is 64/66 with a theoretical speed of 528 MB/s. Getting above
> 450 MB/s on it is likely to be hard, so it could push the 6x15k's top
> speed below that of the 6x10k (though it would be faster over almost
> all of the disk).
>
> But it's fairly common to have multiple 100MHz or 133MHz PCI-X slots
> in a server (with 798MB/s and 1064MB/s theoretical speed),
> significantly above that of the 6x10k array above (There's also PCI-X
> 2.0 for faster speeds).
>
>
> When it comes to PCI-E, how fast it is depends on the number of lanes
> and it's also different in that it provides a set amount of bandwidth
> in both directions instead of the shared model used by PCI/PCI-X.
>
> Each PCI-E lane has a raw capacity of 238MB/s in each direction, so
> for mostly unidirectional traffic you'd need 2 lanes to match the
> slowest PCI-X interface (more balance traffic fares better).
>
> So, a low-end PCI-E SATA controller (1x) would have less bandwidth
> available, but I don't think it's likely you'll find one with 6 or
> more SATA ports. 4x or 8x seems more likely and even in the worst case
> scenario the 4x should have more bandwidth available than 133/64
> PCI-X. However, you might be able to do it with two 4-port PCI-E SATA
> cards, which IS likely to be 1x cards (I'm ignoring those nearly
> mythical port multipliers until I've actually seen them and they
> worked!).
>
>
> My best guess is that the PCI-X/6x15k array is probably going to be
> significantly faster in IOPS, but in the transfer rate test it would
> depend on details such as exact disk models used and how fast the
> "PCI-X" is :-) (and assuming the PCI-E "controller" isn't two 1x
> models)
>
> To some extent it's probably better to compare not a set number of
> disks but performance against some OTHER metric, such as cost or power
> usage! OTOH then Raptor's isn't exactly known for being cheap, and the
> cost would have to include all other elements (support, chassi and so
> on) which makes it very hard to make a fair comparison.
>
> http://en.wikipedia.org/wiki/PCI-X
> http://en.wikipedia.org/wiki/PCI-Express