|
Home > Archive > Data Storage > October 2005 > Looking to build storage of about 3TB -- recommendations?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Looking to build storage of about 3TB -- recommendations?
|
|
|
| Hi All
For my research I need large storage for satellite data files. I am
looking to build storage for about 1TB redundant array and >2TB
volatile array. I have looked around at snap servers or other SAN
solutions. I also have looked at scsi solutions but don't think that
scsi is economicable. What I am proposing is to build 2 arrays of SATA
drives with the following setup:
1 3u supermicro chasis with 15 hotswappable sata bays
2 3ware RAID controller (for ex 9550SX-8LP)
12 WD SATA HDS (400GB)
RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
This will all be hooked to dual xeon chips with and internal boot/sys
mount. Will probably use win2k3 server due to software requirements.
The second array is for temporary storage and can crash and we will not
loose much except the need to download again. The first array needs to
be fault tolerant. We envision swapping out drives in array 2 as
larger drives become available.
My questions are:
+ Can anyone propose reasons NOT to do this?
+ Are the better solutions?
+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
MUCH more that I have seen
thanks
daben
| |
| Spindle 2005-10-10, 7:48 am |
|
daben wrote:
> Hi All
>
> For my research I need large storage for satellite data files. I am
> looking to build storage for about 1TB redundant array and >2TB
> volatile array. I have looked around at snap servers or other SAN
> solutions. I also have looked at scsi solutions but don't think that
> scsi is economicable. What I am proposing is to build 2 arrays of SATA
> drives with the following setup:
>
> 1 3u supermicro chasis with 15 hotswappable sata bays
> 2 3ware RAID controller (for ex 9550SX-8LP)
> 12 WD SATA HDS (400GB)
>
> RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
> RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
>
> This will all be hooked to dual xeon chips with and internal boot/sys
> mount. Will probably use win2k3 server due to software requirements.
>
> The second array is for temporary storage and can crash and we will not
> loose much except the need to download again. The first array needs to
> be fault tolerant. We envision swapping out drives in array 2 as
> larger drives become available.
>
> My questions are:
>
> + Can anyone propose reasons NOT to do this?
> + Are the better solutions?
> + NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
> MUCH more that I have seen
>
> thanks
> daben
Nor sure I understand how you're going to connect the server(s) to the
array. Fibre channel could add significantly to the cost. iSCSI could
create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
even slower, but you don't mention performance reqs.
Also, how are you going to back up array 1?
Another point of concern is that recovery from a disk error on array 1
could take hours or days. I would go with raid 6 and with RAID enhanced
SATA drives (WD and Seagate have them).
| |
| Al Dykes 2005-10-10, 5:49 pm |
| In article <1128947603.447478.176960@o13g2000cwo.googlegroups.com>,
Spindle <arthedge@gmail.com> wrote:
>
>daben wrote:
>
>
>
>Nor sure I understand how you're going to connect the server(s) to the
>array. Fibre channel could add significantly to the cost. iSCSI could
>create a 1Gb/sec bottleneck, file serving as in CIFS would probably be
>even slower, but you don't mention performance reqs.
Is there a comodity raid product that allows me to expand the array by
poping larger drives in, one at a time and leting the raid recovery
work?
What's the status of Infiniband?
http://en.wikipedia.org/wiki/Infiniband
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore.
| |
|
| This storage is for server that it is connected to .. it would not be
network storage. In other words, all the processintg and data access
will be on the machine. It will not be a file server in the normal
sense. This is kinda a special case of just needing a lotof storage
for local processing.
Back up will be accomplished with a lot of tapes  .
| |
| Al Dykes 2005-10-10, 5:49 pm |
| In article <1128960466.487265.93070@g44g2000cwa.googlegroups.com>,
daben <dabenpb@yahoo.com> wrote:
>This storage is for server that it is connected to .. it would not be
>network storage. In other words, all the processintg and data access
>will be on the machine. It will not be a file server in the normal
>sense. This is kinda a special case of just needing a lotof storage
>for local processing.
>
>Back up will be accomplished with a lot of tapes .
>
It's possible you can do what you want with dfs, part of the server
ntfs file system if you are loooking to RAID just to aggregate disks
it might be acceptable.
--
a d y k e s @ p a n i x . c o m
Don't blame me. I voted for Gore.
| |
| Steve Cousins 2005-10-12, 6:01 pm |
| daben wrote:
>Hi All
>
>For my research I need large storage for satellite data files. I am
>looking to build storage for about 1TB redundant array and >2TB
>volatile array. I have looked around at snap servers or other SAN
>solutions. I also have looked at scsi solutions but don't think that
>scsi is economicable. What I am proposing is to build 2 arrays of SATA
>drives with the following setup:
>
>1 3u supermicro chasis with 15 hotswappable sata bays
>2 3ware RAID controller (for ex 9550SX-8LP)
>12 WD SATA HDS (400GB)
>
>RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
>RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
>
>This will all be hooked to dual xeon chips with and internal boot/sys
>mount. Will probably use win2k3 server due to software requirements.
>
>The second array is for temporary storage and can crash and we will not
>loose much except the need to download again. The first array needs to
>be fault tolerant. We envision swapping out drives in array 2 as
>larger drives become available.
>
>My questions are:
>
>+ Can anyone propose reasons NOT to do this?
>+ Are the better solutions?
>+ NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
>MUCH more that I have seen
>
>
Hi Daben,
I have created similar servers with similar equipment (3Ware,
Supermicro,Xeon's and Opterons). My preference is Opterons due to
better performance for the money, at least for our ocean models but as
far as the storage goes what you have should work very well. The only
other concern might be the power supply size for that many drives. I
have 11 drives with dual opterons in a Supermicro 933 case and the
triple-redundant 600 watt PS. It works fine.
Good luck,
Steve
| |
|
| Hi Steve
Thanks for your input. Our latest thinking is to go with 2+ servers.
We too have found that the AMD chips are far faster than the Xeon chips
but I have some doubts about the AMD chipsets and their reliability
over long periods of time. Maybe you can comment on this. Therefore
our plan is as follows:
+ Storage server as above with the Xeon chips (we already have them
from a previous purchase).
+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
We would anticipate the ability to add more 1U servers to increase our
computing power over time. I don't think that Blade servers or fiber
is right for our applications or budgets.
Comments?
thanks
daben
| |
|
| Al Dykes wrote:
> Is there a comodity raid product that allows me to expand the array by
> poping larger drives in, one at a time and leting the raid recovery
> work?
Infortrend has that. But it is a dangerous method: drive failure may
kill you.
That said these are rather affordable so why not buy two...
Thomas
| |
| jimwall2000@yahoo.com 2005-10-13, 5:55 pm |
| Be a little careful here on the Opteron vs. Xeon speed comparisons. In
raw compute speeds the Opteron is definately faster. But the Intel
chipsets support a much higher performance in I/O throughput.
So consider your application. If your system has a PCI based RAID
controller, and multiple Gbit or 10gig NICs, then you could be easily
saturating the I/O bandwidth of an Opteron based platform. If you are
doing a OS based RAID, then you probably need all the CPU speed you can
get, so the Opteron will yield the best performance. A hardware based
RAID solution won't need that kind of CPU power, you will be
bottlenecked on the disk subsystem and the CPU will be idle a lot of
the time.
I don't know what your complete appliaction entails, but keep in mind
that CPU speed is always the best thing to optimize.
-Jim
daben wrote:
> Hi Steve
>
> Thanks for your input. Our latest thinking is to go with 2+ servers.
> We too have found that the AMD chips are far faster than the Xeon chips
> but I have some doubts about the AMD chipsets and their reliability
> over long periods of time. Maybe you can comment on this. Therefore
> our plan is as follows:
>
> + Storage server as above with the Xeon chips (we already have them
> from a previous purchase).
> + 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
>
> We would anticipate the ability to add more 1U servers to increase our
> computing power over time. I don't think that Blade servers or fiber
> is right for our applications or budgets.
>
> Comments?
>
> thanks
> daben
| |
| Steve Cousins 2005-10-13, 5:55 pm |
| daben wrote:
>Hi Steve
>
>Thanks for your input. Our latest thinking is to go with 2+ servers.
>We too have found that the AMD chips are far faster than the Xeon chips
>but I have some doubts about the AMD chipsets and their reliability
>over long periods of time. Maybe you can comment on this. Therefore
>our plan is as follows:
>
>
We have had some for close to two years now and haven't had an issue. I
think if you go with a reputable brand you should be fine. We have been
using Tyan with good results although I am glad that I didn't start
using Opterons a little earlier because there were some problems with
some early Tyan boards. At this point I think things are pretty well
established and safe.
>+ Storage server as above with the Xeon chips (we already have them
>from a previous purchase).
>+ 1U dual AMD Opteron 275 server with 2GB RAM, 1 sys disk
>
>We would anticipate the ability to add more 1U servers to increase our
>computing power over time. I don't think that Blade servers or fiber
>is right for our applications or budgets.
>
>
This is similar to what I have done. I started out with a 2 TB storage
server with two 1.6 Ghz Opterons. We use it to run models on too with
good results. It does fine doing both storage and computing. Then we
started adding 1U dual-Opterons and when we needed more storage we
bought a 6.4 TB RAID array and connected it to one of the 1U compute
servers via SCSI. All volumes are NFS mounted to all machines and
performance is acceptable with Gigabit ethernet, even without Jumbo
frames. I originally wanted to set up a SAN with GFS or GPFS instead of
NFS but the FibreChannel hardware was too expensive. As we add compute
servers maybe the NFS performance will become an issue but I'll deal
with that as it comes.
Good luck,
Steve
| |
|
| Jim
The appllications to run on the Opteron chips will actually be all
loaded into memory before run. Tun times are to be about 6hrs so the
IO will be a very small portion of that.
We are doing HW based RAID with 3Ware controllers.
daben
| |
|
| In article <1128890239.363615.207280@z14g2000cwz.googlegroups.com>,
"daben" <dabenpb@yahoo.com> wrote:
> Hi All
>
> For my research I need large storage for satellite data files. I am
> looking to build storage for about 1TB redundant array and >2TB
> volatile array. I have looked around at snap servers or other SAN
> solutions. I also have looked at scsi solutions but don't think that
> scsi is economicable. What I am proposing is to build 2 arrays of SATA
> drives with the following setup:
>
> 1 3u supermicro chasis with 15 hotswappable sata bays
> 2 3ware RAID controller (for ex 9550SX-8LP)
> 12 WD SATA HDS (400GB)
>
> RAID ARRAY 1: RAID 5, 5 drives total, 1 hot spare == 1.2TB
> RAID ARRAY 2: RAID 0 (or none), 7 drives total == 2.8TB
>
> This will all be hooked to dual xeon chips with and internal boot/sys
> mount. Will probably use win2k3 server due to software requirements.
>
> The second array is for temporary storage and can crash and we will not
> loose much except the need to download again. The first array needs to
> be fault tolerant. We envision swapping out drives in array 2 as
> larger drives become available.
>
> My questions are:
>
> + Can anyone propose reasons NOT to do this?
I have seen, for whatever odd reason, a high rate of failure of the SATA
cables. I have also seen backplane failure in these types of boxes.
Ironically, as some will see it, the SATA drives themselves seem to be
quite reliable.
> + Are the better solutions?
A self-contained controller is probably simpler and presumably less
likely to fail.
> + NOTE: We anticipate this will cost $5k-$6k -- the other solutins are
> MUCH more that I have seen
Promise VTrak is not expensive and actually seems to be reliable. It
also has decent management software.
(Someone else mentioned RAID 6; that's the one thing the VTrak lacks.)
| |
|
| In article <1129221599.323718.278400@g47g2000cwa.googlegroups.com>,
jimwall2000@yahoo.com wrote:
> Be a little careful here on the Opteron vs. Xeon speed comparisons. In
> raw compute speeds the Opteron is definately faster. But the Intel
> chipsets support a much higher performance in I/O throughput.
Do you by chance have a reference for this?
| |
| David Magda 2005-10-24, 9:43 am |
| flux <support@fluxsoft.com> writes:
> Promise VTrak is not expensive and actually seems to be reliable. It
> also has decent management software.
The experience I've had with Promise cards is that they're mostly
software RAID (a lot happens in the driver). This is under Linux.
I'm not familiar with the VTrak specifically though.
I've heard many good things about 3ware.
--
David Magda <dmagda at ee.ryerson.ca>
Because the innovator has for enemies all those who have done well under
the old conditions, and lukewarm defenders in those who may do well
under the new. -- Niccolo Machiavelli, _The Prince_, Chapter VI
| |
|
| In article <m27jcddmh7.fsf@gandalf.magda.ca>,
David Magda <dmagda+trace050401@ee.ryerson.ca> wrote:
> flux <support@fluxsoft.com> writes:
>
>
> The experience I've had with Promise cards is that they're mostly
> software RAID (a lot happens in the driver). This is under Linux.
The VTrak is a completely self-contained hardware RAID.
> I'm not familiar with the VTrak specifically though.
>
> I've heard many good things about 3ware.
Ironically, 3ware may be problematic under Linux. I've occasionally
experienced "Out of IOMMU" space errors and such an error generally
spits junk data on the RAID filesystems, which is not good. I don't
know, however, if the source of the problem is the 3ware driver (is it
really "hardware" RAID?), the Linux kernel, the card itself, or
something interaction with the motherboard (Arima HDAMA) or the (Wintec)
memory. But since I don't see this problem on the LSI/mptfusion
card/driver in the same box, I'm inclined to believe it's the 3ware
driver or the card itself.
| |
| jimwall2000@yahoo.com 2005-10-24, 9:44 am |
| This is what I do for a living. But this particular analysis is pretty
easy to come up with.
Just look at the block diagram of the Intel E7520 chipset and then one
for the AMD 8131. What you are looking for are the number of
independent PCI buses and their paths to memory.
Each PCI bus has a maximum burst throughout, and a sustainable
throughput of about half of that (depending on the number of bus
masters). As long as the memory controller is not the bottleneck, then
it is the number of PCI buses times the sustainable throughput.
-Jim
|
|
|
|
|