Data Storage - Experiences with Cluster File Systems (ie. GFS) over a SAN

Author

Experiences with Cluster File Systems (ie. GFS) over a SAN

Andrew Gideon

2004-12-07, 5:45 pm

I've been looking at the possibility of putting some NFS servers "over" a
SAN, providing a "home grown" NAS. This would be one solution that'll meet
our NFS requirements, as we can use Linux (or Solaris) NFS servers which
provide ACL support.

But this requires that the underlying Cluster File System provide ACL
support. Does anyone happen to know which do? The semantics I'm seeking
are those found in Solaris's UFS or Linux's XFS or EXT3: the "near POSIX"
ACLs.

Does anyone have experience - good or bad - with this type of solution?
Recommendations? Warnings?

Thanks...

Andrew

Faeandar

2004-12-10, 5:45 pm

On Tue, 07 Dec 2004 11:31:01 -0500, Andrew Gideon
<cR182Driver45@gideon.org> wrote:

>
>I've been looking at the possibility of putting some NFS servers "over" a
>SAN, providing a "home grown" NAS. This would be one solution that'll meet
>our NFS requirements, as we can use Linux (or Solaris) NFS servers which
>provide ACL support.
>
>But this requires that the underlying Cluster File System provide ACL
>support. Does anyone happen to know which do? The semantics I'm seeking
>are those found in Solaris's UFS or Linux's XFS or EXT3: the "near POSIX"
>ACLs.
>
>Does anyone have experience - good or bad - with this type of solution?
>Recommendations? Warnings?
>
>Thanks...
>
> Andrew

I'm not sure I understand what you're trying to do. But as a quick
aside, if you use Solaris as the NFS server and have Solaris clients
ACL's will work. It's when you start mixing server and client OS's
that it gets hairy.

~F

Andrew Gideon

2004-12-10, 8:45 pm

Faeandar wrote:

>
> I'm not sure I understand what you're trying to do. But as a quick
> aside, if you use Solaris as the NFS server and have Solaris clients
> ACL's will work. It's when you start mixing server and client OS's
> that it gets hairy.
>

I currently mix Solaris and Redhat, and these are sharing ACLs (in both
directions) quite well. So that's not really my worry.

http://www.suse.de/~agruen/acl/linux-acls/online/

[For some reason, though, Fedora Core 3 still has ACL support in NFS
disabled. There was even a bug report for this in core 2:

https://bugzilla.redhat.com/bugzill...g.cgi?id=123722

but it's still lacking in 3. Given that it is in the older Enterprise 3,
this is something of a puzzle.]

I have learned - from speaking to a Redhat SE - that the ACLs supported
under ext3 are also supported by GFS. I've some doubts about this, though,
given that - at first - he told me that ACLs weren't supported on ext3.

I'm still having difficulty getting a solid answer on what support Redhat
provides for iSCSI, however. I keep hearing phrases line "not fully
supported". When I ask for more detail, I'm told that Redhat can act as an
initiator (client) but not a target (server). When I then backup and ask
"so I'll be supported by Redhat if I use Redhat as an iSCSI client", I'm
told again "not fully".

<Sigh>

I'm trying to avoid a problem where Redhat won't even listen to a problem
because I've an "unsanctioned" module loaded.

It's frustrating, and I may fall back on simply using a dual-host SCSI RAID.

- Andrew

Faeandar

2004-12-10, 8:45 pm

On Fri, 10 Dec 2004 19:20:00 -0500, Andrew Gideon
<cR182Driver45@gideon.org> wrote:

>Faeandar wrote:
>
>
>I currently mix Solaris and Redhat, and these are sharing ACLs (in both
>directions) quite well. So that's not really my worry.
>
> http://www.suse.de/~agruen/acl/linux-acls/online/
>
>[For some reason, though, Fedora Core 3 still has ACL support in NFS
>disabled. There was even a bug report for this in core 2:
>
> https://bugzilla.redhat.com/bugzill...g.cgi?id=123722
>
>but it's still lacking in 3. Given that it is in the older Enterprise 3,
>this is something of a puzzle.]
>
>
>I have learned - from speaking to a Redhat SE - that the ACLs supported
>under ext3 are also supported by GFS. I've some doubts about this, though,
>given that - at first - he told me that ACLs weren't supported on ext3.
>
>I'm still having difficulty getting a solid answer on what support Redhat
>provides for iSCSI, however. I keep hearing phrases line "not fully
>supported". When I ask for more detail, I'm told that Redhat can act as an
>initiator (client) but not a target (server). When I then backup and ask
>"so I'll be supported by Redhat if I use Redhat as an iSCSI client", I'm
>told again "not fully".
>
><Sigh>
>
>I'm trying to avoid a problem where Redhat won't even listen to a problem
>because I've an "unsanctioned" module loaded.
>
>It's frustrating, and I may fall back on simply using a dual-host SCSI RAID.
>
> - Andrew

The only support for iSCSI by most *nix's right now is client only.
There are a few vendors that handle server side but RedHat is not one
of them atm. In fact, the only one I can think of that fully support
iSCSI server is NetApp. I believe EMC must but I don't know for sure.

We use RedHat iSCSI client initiators and they work quite well, but
server side is NetApp. I can't say we've played with ACL support
beyond user/group/world though.

~F

Andrew Gideon

2004-12-16, 6:26 pm

Faeandar wrote:

> The only support for iSCSI by most *nix's right now is client only.
> There are a few vendors that handle server side but RedHat is not one
> of them atm. In fact, the only one I can think of that fully support
> iSCSI server is NetApp. I believe EMC must but I don't know for sure.
>
> We use RedHat iSCSI client initiators and they work quite well, but
> server side is NetApp. I can't say we've played with ACL support
> beyond user/group/world though.
>

All we need on the UNIX side is the initiator, so that's fine. However, a
Redhat rep told me this morning that their support is "best effort" for
iSCSI. Enterprise 4 is supposed to be "better supported" (and also to
include target capability, but that's not something I need).

That "best effort" is what "not fully supported" meant, apparently.

I'm checking into Solaris to see if it has what I need, as the support is an
important net for us to have in this application.

In the meantime, I've been trying to build iSCSI support for Fedora (core 2
and core 3) but have been having difficulties there as well. But it may
just be my inexperience with source RPMs, as "make" is complaining about
missing files that I see are there...so they're not being found properly.

However, it's a little unfortunate that I found no binary RPM with the
driver and such for iSCSI on a Fedora.

- Andrew