Hi,

my company is looking at using Clariion with SATA disks for research quality
storage. This means we want cheap, but still good performance. The nature of
the application is to do sequential reads of many average size 3GB flat
files, say 1TB at a time. So cache on the controller is useless. The raw
disk throughput is more important. Write speed is important, but not as
important as read.

There are 2 things I am concerned with this Clariion/SATA solution...

1. Clariion's SATA drives are 5400rpm, compares to Western Digital 7200rpm.

2. The Clariion's internal signal is FC, so the SATA disks attaches to a
"translator" (Can someone explain this), and this further reduces the SATA's
native speed.

Can someone who has experience help me out? Any story of your setup and
experience are deeply appreciated.

Clayton

[ Post a follow-up to this message ]

- C - wrote:
> Hi,
>
> my company is looking at using Clariion with SATA disks for research
> quality storage. This means we want cheap, but still good
> performance. The nature of the application is to do sequential reads
> of many average size 3GB flat files, say 1TB at a time. So cache on
> the controller is useless.

Not necessarily, if the OS/HW combination can't issue read requests fast
enough to saturate the raw bandwidth, then read ahead into the controller
cache may improve things. It'll depend a bit on the data access patterns and
block sizes, but don't rule out the possible benefits of cache even for
sequential reads.

BTW, it might be worth stating the required read performance.

> The raw disk throughput is more important.
> Write speed is important, but not as important as read.


>
> There are 2 things I am concerned with this Clariion/SATA solution...
>
> 1. Clariion's SATA drives are 5400rpm, compares to Western Digital
> 7200rpm.
>
Or even 10K for the WD 36GB. But generally, rotational speed is an important
factor for random IOPs, it has less impact on performance for sequential
I/O.

> 2. The Clariion's internal signal is FC, so the SATA disks attaches
> to a "translator" (Can someone explain this), and this further
> reduces the SATA's native speed.
>
A controller that offers an external I/F that is different to the drives
native I/F (i.e. FC vs. SATA) is nothing new, it's been done with SCSI, SSA
and PATA in the past. Put simply, the incoming I/O request is recieved by
the controller which converts it to the necessary SATA disk I/O request(s)
and passes it on the disk(s). When the data from the disk is recieved by the
controller it re-packages it as an FC I/O and sends the data back to the
host. Given that the FC I/F is faster than the SATA I/F its unlikely that
the FC I/F will be a significant bottleneck.

> Can someone who has experience help me out? Any story of your setup
> and experience are deeply appreciated.


As I said earlier, it would be helpful if you could state your performance
requirements in terms of MB/s as that would help determine whether the
solution workable.

BTW are you talking about the hi-end (CX-200 etc) Clariions that allow you
to use SATA drives in add-on chassis to a first chassis that contains FC
drives or the newly announced AX prduct that is a pure FC-SATA solution.
--
Nik Simpson

[ Post a follow-up to this message ]

Hi Nik,

thanks for the reply. The plan is to attache a few linux boxes to a Clariion
cx700 via 2GB HBA cards. We have large flat files that totals 12TB. The
programs will fire off on the linux boxes and sequentially read a few TBs
each. The apps will read as fast as the data can come in, so the read
requirement is really as fast as the spindles can move. 10TB at 40MB/s will
be 29days, at 80MB/s will be 15days, so faster the better... Although we
don't have the $$ to go FC...

I mentioned the WD 7200rpm 250gb drive because the comparable drive size in
Clariion. To buy 10krpm 36gb drive will be too costly for me... I read on
another thread someone mentioned the translation being a bottleneck, have
you seen this at all?

Thanks!

Clayton

"Nik Simpson" <n_simpson@bellsouth.net> wrote in message
news:yyQtc.13599$ff2.2399@bignews4.bellsouth.net...
> - C - wrote: 
>
> Not necessarily, if the OS/HW combination can't issue read requests fast
> enough to saturate the raw bandwidth, then read ahead into the controller
> cache may improve things. It'll depend a bit on the data access patterns
and
> block sizes, but don't rule out the possible benefits of cache even for
> sequential reads.
>
> BTW, it might be worth stating the required read performance.
> 
>
> 
> Or even 10K for the WD 36GB. But generally, rotational speed is an
important
> factor for random IOPs, it has less impact on performance for sequential
> I/O.
> 
> A controller that offers an external I/F that is different to the drives
> native I/F (i.e. FC vs. SATA) is nothing new, it's been done with SCSI,
SSA
> and PATA in the past. Put simply, the incoming I/O request is recieved by
> the controller which converts it to the necessary SATA disk I/O request(s)
> and passes it on the disk(s). When the data from the disk is recieved by
the
> controller it re-packages it as an FC I/O and sends the data back to the
> host. Given that the FC I/F is faster than the SATA I/F its unlikely that
> the FC I/F will be a significant bottleneck.
> 
>
>
> As I said earlier, it would be helpful if you could state your performance
> requirements in terms of MB/s as that would help determine whether the
> solution workable.
>
> BTW are you talking about the hi-end (CX-200 etc) Clariions that allow you
> to use SATA drives in add-on chassis to a first chassis that contains FC
> drives or the newly announced AX prduct that is a pure FC-SATA solution.
> --
> Nik Simpson
>
>

[ Post a follow-up to this message ]

- C - wrote:
> Hi Nik,
>
> thanks for the reply. The plan is to attache a few linux boxes to a
> Clariion cx700 via 2GB HBA cards. We have large flat files that
> totals 12TB.

How will that be distributed, i.e. will there be one filesystem shared
between all the LINUX boxes or will there be seperate LUNs carved out for
each of the LINUX boxes? BTW, I think you'll find cache much more helpful in
this scenario than you might think because of the need to support multiple
hosts reading from different parts of the array, what looks like sequential
access to each host may well end up looking far from sequential to the
array.

>The programs will fire off on the linux boxes and
> sequentially read a few TBs each. The apps will read as fast as the
> data can come in, so the read requirement is really as fast as the
> spindles can move. 10TB at 40MB/s will be 29days, at 80MB/s will be
> 15days, so faster the better... Although we don't have the $$ to go
> FC...
>

Have you looked at the new Clariion AX-100 announced this week, this is a
pure FC-SATA box and it also supports the 7200RPM drives. The SATA
implementation in the CX-700 requires the controller head & set of FC drives
and its much more expensive. You could probably afford to spread the load
over several AX100s each with dual controllers for the same amount you'd pay
for the CX-700 solution which is probably overkill for what you want.

Raw throughput on the AX100 fully populated is rated at 300MB/s for large
block sequential read, now even if you only see 1/2 of that combined with
the need for four of the arrays (max 3TB for each AX100) you'll able to get
600MB/s overall which isn't bad.

Take a look at:

http://www.emc.com/products/systems...ldv.p
df

Just for the record, I don't work for EMC and have no "axe to grind"

> I mentioned the WD 7200rpm 250gb drive because the comparable drive
> size in Clariion. To buy 10krpm 36gb drive will be too costly for
> me... I read on another thread someone mentioned the translation
> being a bottleneck, have you seen this at all?

I think the bottleneck that people refer to is more in comparison to a pure
FC configuration of the CX-700, either way I don't think the overhead of the
FC-SATA translation is going to be a big problem in this application. But I
do urge you to take a look at the AX if what you want is an FC-SATA
solution.

BTW if you want all the LINUX boxes to share the same array (or storage
pool) and you can't afford pure FC, you really don't have much of an option
other than FC-SATA or FC-SCSI so the overhead involved in the translation is
moot.

[ Post a follow-up to this message ]

Please let us know your experiences with this system.

We are using a CX-400 with both FC-SCSI and S-ATA disks.

The S-ATA disks (10 disks in a RAID group) have a throughput of 4
MB/sec max. Imagine that. I have demanded an investigatin from EMC,
who has setup this thing. They just say that these disks are really
slow. But hey, according to the specs they should reach about 35 up to
60 MB/sec per disk. Something is totally misconfigured here and I have
no help from EMC and no clue about the reason.

Bye
Rick Denoire

[ Post a follow-up to this message ]

"Rick Denoire" <100.17706@germanynet.de> wrote in message
 news:vbrpb0l1enk2v8muvqgjnmadqauu5lifr4@
4ax.com...
> Please let us know your experiences with this system.
>
> We are using a CX-400 with both FC-SCSI and S-ATA disks.
>
> The S-ATA disks (10 disks in a RAID group) have a throughput of 4
> MB/sec max. Imagine that. I have demanded an investigatin from EMC,
> who has setup this thing. They just say that these disks are really
> slow.

There certainly used to be people at EMC far more competent than such an
answer indicates.  If you're talking with them, they're lying to you; if
not, find someone competent to talk with.

But hey, according to the specs they should reach about 35 up to
> 60 MB/sec per disk. Something is totally misconfigured here and I have
> no help from EMC and no clue about the reason.

It used to be that the combination of a disabled disk write-back cache plus
a limitation to smallish request sizes (sometimes as small as 32 KB per
request, depending upon the assumptions the controller made about disk
behavior, since some parallel ATA disks got flakey at larger transfer
sizes - see the Linux ATA driver source code for examples) could cause each
request to miss a complete disk revolution.  With a 7200 rpm disk, at 8+
ms./rev, that works out to just about 4 MB/sec.

But even PATA request size limits got increased (and unnecessary
vendor-specific problems in this area fixed), and SATA request size limits
are at least in the multi-MB range now IIRC.  So there's no excuse for a
properly-configured controller not to achieve something close to the max
theoretical disk bandwidth (which should be in the ballpark you stated).

Now, it's also possible that you're experiencing a pathological embrace
between your operating system and your RAID controller - at least if you're
using RAID-5.  For example, the Windows NT/2K/XP cache (I don't remember if
you specified what OS you're using) usually writes back data in 64 KB
chunks.  If the controller sees such a write, and isn't aware that it can be
performed lazily (and thus potentially coalesced with adjacent writes into a
full-stripe transfer), and the OS also isn't performing the writes lazily
such that it could submit additional 64 KB writes while waiting for the
first to complete, it may well take 2 full disk revolutions before the next
write request can be accepted (again, about 4 MB/sec with 7200 rpm disks).

- bill

[ Post a follow-up to this message ]

Hey Bill,

You seem to know what you're talking about. We have a CX600 Clariion with 40
disks configured as RAID 5 (5-disks/raid group).
We then further stripe over 4 raid groups per SP. From this we offer to the
hosts 2 "devices" (each device being striped over 4 raid groups). On the
host we then stripe over both these devices.
Our application does a read/write of 4MB. On the host our stripe size is set
to 2 MB (thus each request will go to both devices).
The stripe set on the SP is 512KB (thus the 2MB request is split over all 4
raid groups). Then within the raid group we stripe on 128KB (thus the 512KB
request is now split over 4 disks and disk 5 for the parity).

In other words we have all 40 disks (for writes) and 32 disks (for reads)
working for us. Caching is fully enabled.
We notice that once the Clariion needs to start flushing to disk since the
cache is full (happens very quickly) our throughput is down to 40MB/s. This
means we're doing about 1.25MB/s / disk.
We were told that this is due to a bottleneck in the translation from FC to
ATA and that nothing will resolve this unless we switch to FC disks.

What are your thoughts on this?

Thanks.

"Bill Todd" <billtodd@metrocast.net> wrote in message
news:8eedndj_m6EmgyDdRVn_iw@metrocast.net...
>
> "Rick Denoire" <100.17706@germanynet.de> wrote in message
>  news:vbrpb0l1enk2v8muvqgjnmadqauu5lifr4@
4ax.com... 
>
> There certainly used to be people at EMC far more competent than such an
> answer indicates.  If you're talking with them, they're lying to you; if
> not, find someone competent to talk with.
>
>  But hey, according to the specs they should reach about 35 up to 
>
> It used to be that the combination of a disabled disk write-back cache
plus
> a limitation to smallish request sizes (sometimes as small as 32 KB per
> request, depending upon the assumptions the controller made about disk
> behavior, since some parallel ATA disks got flakey at larger transfer
> sizes - see the Linux ATA driver source code for examples) could cause
each
> request to miss a complete disk revolution.  With a 7200 rpm disk, at 8+
> ms./rev, that works out to just about 4 MB/sec.
>
> But even PATA request size limits got increased (and unnecessary
> vendor-specific problems in this area fixed), and SATA request size limits
> are at least in the multi-MB range now IIRC.  So there's no excuse for a
> properly-configured controller not to achieve something close to the max
> theoretical disk bandwidth (which should be in the ballpark you stated).
>
> Now, it's also possible that you're experiencing a pathological embrace
> between your operating system and your RAID controller - at least if
you're
> using RAID-5.  For example, the Windows NT/2K/XP cache (I don't remember
if
> you specified what OS you're using) usually writes back data in 64 KB
> chunks.  If the controller sees such a write, and isn't aware that it can
be
> performed lazily (and thus potentially coalesced with adjacent writes into
a
> full-stripe transfer), and the OS also isn't performing the writes lazily
> such that it could submit additional 64 KB writes while waiting for the
> first to complete, it may well take 2 full disk revolutions before the
next
> write request can be accepted (again, about 4 MB/sec with 7200 rpm disks).
>
> - bill
>
>
>

[ Post a follow-up to this message ]

"Erik Hendrix" <hendrix_erik@hotmail.com> wrote in message
 news:5131e287661111178fb2592dd53975c8@ne
ws.teranews.com...
> Hey Bill,
>
> You seem to know what you're talking about.

Alas, only in a general sense.  I have no direct acquaintance with the
Clariion arrays.

We have a CX600 Clariion with 40
> disks configured as RAID 5 (5-disks/raid group).
> We then further stripe over 4 raid groups per SP. From this we offer to
the
> hosts 2 "devices" (each device being striped over 4 raid groups). On the
> host we then stripe over both these devices.
> Our application does a read/write of 4MB. On the host our stripe size is
set
> to 2 MB (thus each request will go to both devices).
> The stripe set on the SP is 512KB (thus the 2MB request is split over all
4
> raid groups). Then within the raid group we stripe on 128KB (thus the
512KB
> request is now split over 4 disks and disk 5 for the parity).

Unless I need sleep more than I'm aware of right now, that all sounds good.

>
> In other words we have all 40 disks (for writes) and 32 disks (for reads)
> working for us. Caching is fully enabled.
> We notice that once the Clariion needs to start flushing to disk since the
> cache is full (happens very quickly) our throughput is down to 40MB/s.
This
> means we're doing about 1.25MB/s / disk.
> We were told that this is due to a bottleneck in the translation from FC
to
> ATA and that nothing will resolve this unless we switch to FC disks.
>
> What are your thoughts on this?

I once again lean toward incompetence as an explanation:  either the people
who told you this were incompetent, or the people who implemented that
FC-to-ATA translation were (there's no reason it should be anything like
that slow, unless they did something incredibly dumb - or intentionally so,
in order to push their higher-end solutions - and simply used their
tagged-queuing SCSI algorithms unchanged with the SATA disks resulting in
strictly serial/synchronous operation).

- bill

[ Post a follow-up to this message ]

"Erik Hendrix" <hendrix_erik@hotmail.com> wrote:

>You seem to know what you're talking about. We have a CX600 Clariion with 4
0
>disks configured as RAID 5 (5-disks/raid group).
>We then further stripe over 4 raid groups per SP. From this we offer to the
>hosts 2 "devices" (each device being striped over 4 raid groups). On the
>host we then stripe over both these devices.
>Our application does a read/write of 4MB. On the host our stripe size is se
t
>to 2 MB (thus each request will go to both devices).
>The stripe set on the SP is 512KB (thus the 2MB request is split over all 4
>raid groups). Then within the raid group we stripe on 128KB (thus the 512KB
>request is now split over 4 disks and disk 5 for the parity).
>
>In other words we have all 40 disks (for writes) and 32 disks (for reads)
>working for us. Caching is fully enabled.
>We notice that once the Clariion needs to start flushing to disk since the
>cache is full (happens very quickly) our throughput is down to 40MB/s. This
>means we're doing about 1.25MB/s / disk.
>We were told that this is due to a bottleneck in the translation from FC to
>ATA and that nothing will resolve this unless we switch to FC disks.

I wonder if you ever really corroborated this setup. I would not rely
on that kind of calculations alone. You have to somehow "see" what is
happening. RAID devices nowadays are said to have their own
idiosyncracies. For example, I was told to use RAID 5 instead of
mirroring because supposedly the cx-400 is internally specifically
optimized for RAID 5. So it would do something based on a RAID 5 logic
even if the RAID setup is different. Weird.

Bye
Rick Denoire

[ Post a follow-up to this message ]

"Bill Todd" <billtodd@metrocast.net> wrote:

>Now, it's also possible that you're experiencing a pathological embrace
>between your operating system and your RAID controller - at least if you're
>using RAID-5.  For example, the Windows NT/2K/XP cache (I don't remember if
>you specified what OS you're using) usually writes back data in 64 KB
>chunks.  If the controller sees such a write, and isn't aware that it can b
e
>performed lazily (and thus potentially coalesced with adjacent writes into 
a
>full-stripe transfer), and the OS also isn't performing the writes lazily
>such that it could submit additional 64 KB writes while waiting for the
>first to complete, it may well take 2 full disk revolutions before the next
>write request can be accepted (again, about 4 MB/sec with 7200 rpm disks).

Well, since the disks are at the central SAN storage, to which several
different hosts are attached via FC, I am able to do interesting
experiments. I can deassign one LUN from one Sun/Solaris host and
reassign it to an Intel/Linux host or to another Sun/Solaris machine.

In ALL cases, performance was very poor. Even when one LUN was
internally mirrored (no host involved), I was shocked how slow that
was.

I am planning some experiments next time in order to demand to action
from Dell/EMC. I have no clue about how to switch the harddisk own
cache on/off.

My question is, where do I have a chance to improve something? At the
driver level by changing the settings in the driver configuration
file? Using the Navisphere or navicli and changing the element size or
read ahead feature or ..? And as mentioned somewhere else, perhaps by
switching the harddisk's own cache on??

Bye
Rick Denoire

[ Post a follow-up to this message ]