I have alot of NetApp filers and for the most part they serve well.
But my problem is this; I potentially have 1k clients hitting the same
project and a real issue of 1k clients hitting the same filer.

While NetApp and WAFL are optimized for NFS even it gets dropped to
it's knees when this happens.  I'm not talking about low end either,
fully loaded 960's performing 20k+ OPS and moving (in and out) over
130MB/sec.  Admirable performance to say the least but not enough.
Fast forward 1 year and now i'm looking at 1k clients definitely
hitting the same project and 10k clients hitting the same filer.  Not
good.

Does anyone else have this problem and what, if anything, have you
done to alleviate it?

I'm looking for scalable anything, software or appliance, that can
take that kind of punishment.  Sistina/GFS has the potential but
there's alot of management overhead to break the data up properly for
maximum performance.  I'm also looking at Ibrix but have no idea who
their competition might be, which would be nice to know.

Anyone have some pointers?  Thanks.

~F

[ Post a follow-up to this message ]

"Faeandar" <mr_castalot@yahoo.com> wrote in message
 news:ji1re09vm4n4lghtjllnmj2c0u9eqrh2aj@
4ax.com...
> I have alot of NetApp filers and for the most part they serve well.
> But my problem is this; I potentially have 1k clients hitting the same
> project and a real issue of 1k clients hitting the same filer.
>
> While NetApp and WAFL are optimized for NFS even it gets dropped to
> it's knees when this happens.  I'm not talking about low end either,
> fully loaded 960's performing 20k+ OPS and moving (in and out) over
> 130MB/sec.  Admirable performance to say the least but not enough.
> Fast forward 1 year and now i'm looking at 1k clients definitely
> hitting the same project and 10k clients hitting the same filer.  Not
> good.
>
> Does anyone else have this problem and what, if anything, have you
> done to alleviate it?
>
> I'm looking for scalable anything, software or appliance, that can
> take that kind of punishment.  Sistina/GFS has the potential but
> there's alot of management overhead to break the data up properly for
> maximum performance.  I'm also looking at Ibrix but have no idea who
> their competition might be, which would be nice to know.
>
> Anyone have some pointers?  Thanks.

Hmmm.  If you're talking about fine-grained hot-spots (like, large single
files), then you'd need a system that stripes single files across multiple
servers - not a common technology AFAIK.

But for slightly coarser striping, you might look at Z-force:  they
supposedly have some means of striping a single file system across multiple
standard NAS filers (which would let you leverage the hardware that you
already own), though whether their 'switch' would create a bottleneck for
your application could be an issue.  EMC's Celerra, IBM's Storage Tank
(whatever the real name is now that it's shipped), and Lustre are other
products that you might investigate (Spinnaker's as well, though they just
got bought out, I think by Veritas, and I don't know what the status of
their platform currently is).

In any event, let us know what you come up with.

- bill

[ Post a follow-up to this message ]

On Thu, 8 Jul 2004 18:12:40 -0400, "Bill Todd"
<billtodd@metrocast.net> wrote:

>
>"Faeandar" <mr_castalot@yahoo.com> wrote in message
> news:ji1re09vm4n4lghtjllnmj2c0u9eqrh2aj@
4ax.com... 
>
>Hmmm.  If you're talking about fine-grained hot-spots (like, large single
>files), then you'd need a system that stripes single files across multiple
>servers - not a common technology AFAIK.
>
>But for slightly coarser striping, you might look at Z-force:  they
>supposedly have some means of striping a single file system across multiple
>standard NAS filers (which would let you leverage the hardware that you
>already own), though whether their 'switch' would create a bottleneck for
>your application could be an issue.  EMC's Celerra, IBM's Storage Tank
>(whatever the real name is now that it's shipped), and Lustre are other
>products that you might investigate (Spinnaker's as well, though they just
>got bought out, I think by Veritas, and I don't know what the status of
>their platform currently is).
>
>In any event, let us know what you come up with.
>
>- bill
>
>

Good info and ideas, thanks.  Spinnaker was bought by NetApp since
they are the only (imo) NAS appliance that was in the same category as
them.  Actually, I think they were better in several regards.  I mean,
the vldb thing was awesome!  AFS in a can!  Wahooo....
It's still a purchaseable product but it doesn't solve my issue
entirely.  Although it has the potential to greatly reduce it.  the
problem is the cost of the heads you have to put in front of all those
disk drives, it's enormous really.

I'll look into Z-Force, never heard of them.  The idea behind Ibrix is
to use all of the your sim servers local disk as part of a global name
space and clustered file system.  Sounds good but we'll see.

~F

[ Post a follow-up to this message ]