building on your own a large data storage ...
Web Server forum
Back To The Forum Home!Search!Private Messaging System

Web Server Talk Web Server Talk > WebserverTalk Community > Data Storage > building on your own a large data storage ...




  Last Thread   Next Thread Next
  Show Printable Version Email this Page Subscribe to this Thread      Post New Thread    Post A Reply      

    building on your own a large data storage ...  
lbrtchx@hotmail.com


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
07-04-07 12:14 PM

Hi,
~
I need to store a really large number of texts and I (could) have a
number of ATA100 S.M.A.R.T.-compliant hard drives, which I would like
to use to somehow build a large and safe (RAID-5?) data store
~
Now I am definitely more of a software person (at least
occupationally) and this is what I have in mind:
~
* I will have to use standard (and commercially available (meaning
cheap ;-))) x86-based hardware and Open Source software
~
* AFAIK you could maximally use 4 hard drives in such boxes
~
* heat dissipation could become a problem with so many hard drives
~
* I need a reliable and stable power supply
~
Should I got for ATA or SATA drives and why?
~
You could use firewire and/or USB cards to plug in that many
harddrives. Wouldn't it be faster/better using extra ATA PCI cards?
What else would it entail? How many such cards could Linux take?
~
People in the know use software based RAID. Could you give me links
to these kinds of discussions?
~
What would be my weak/hotspot points in my kind of design?
~
Any suggestions of the type of boxes/racks I should use?
~
Is this more or less feasible? What am I missing here? Any other
suggestions? or intelligent posts in which people have discussed these
issues before? I found two in which some people have said a few right
and some other questionable things:
~
comp.sys.ibm.pc.hardware.storage: "2 TB storage solution"
comp.arch.storage: "Homebuilt server (NAS/SAN) vs the prefab ones?
Peformance"
~
Do you know of any such "do-it-yourself" projects out there?
~
thanks
lbrtchx






[ Post a follow-up to this message ]



    Re: building on your own a large data storage ...  
Kwyjibo


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
07-05-07 06:15 AM


<lbrtchx@hotmail.com> wrote in message
news:1183547526.111134.84700@r34g2000hsd.googlegroups.com...
> Hi,
> ~
> I need to store a really large number of texts and I (could) have a
> number of ATA100 S.M.A.R.T.-compliant hard drives, which I would like
> to use to somehow build a large and safe (RAID-5?) data store
> ~
> Now I am definitely more of a software person (at least
> occupationally) and this is what I have in mind:
> ~
> * I will have to use standard (and commercially available (meaning
> cheap ;-))) x86-based hardware and Open Source software
> ~
> * AFAIK you could maximally use 4 hard drives in such boxes
> ~
> * heat dissipation could become a problem with so many hard drives
> ~
> * I need a reliable and stable power supply
> ~
> Should I got for ATA or SATA drives and why?
> ~
> You could use firewire and/or USB cards to plug in that many
> harddrives. Wouldn't it be faster/better using extra ATA PCI cards?
> What else would it entail? How many such cards could Linux take?
> ~
> People in the know use software based RAID. Could you give me links
> to these kinds of discussions?
> ~
> What would be my weak/hotspot points in my kind of design?
> ~
> Any suggestions of the type of boxes/racks I should use?
> ~
> Is this more or less feasible? What am I missing here? Any other
> suggestions? or intelligent posts in which people have discussed these
> issues before? I found two in which some people have said a few right
> and some other questionable things:
> ~
> comp.sys.ibm.pc.hardware.storage: "2 TB storage solution"
> comp.arch.storage: "Homebuilt server (NAS/SAN) vs the prefab ones?
> Peformance"
> ~
> Do you know of any such "do-it-yourself" projects out there?
> ~

Have a look at OpenFiler. It's a linux based iSCSI, NFS and SMB appliance.
Takes about 10 minutes to install and configure. I'm using it as iSCSI
shared storage (pseudo-SAN) between a couple of ESX Servers and it works
fine.

--
Kwyj.







[ Post a follow-up to this message ]



    Re: building on your own a large data storage ...  
Andre Majorel


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
07-08-07 12:13 AM

["Followup-To:" header set to comp.sys.ibm.pc.hardware.storage.]
On 2007-07-04, lbrtchx@hotmail.com <lbrtchx@hotmail.com> wrote:

>  I need to store a really large number of texts and I (could) have a
> number of ATA100 S.M.A.R.T.-compliant hard drives, which I would like
> to use to somehow build a large and safe (RAID-5?) data store
> ~
>  * I will have to use standard (and commercially available (meaning
> cheap ;-))) x86-based hardware and Open Source software
> ~
>  * AFAIK you could maximally use 4 hard drives in such boxes

On a motherboard with 2 IDE ports, you cannot make a 4-disk
RAID-5 array because doing I/O on two devices on the same IDE
port gives poor performance.

You could make two RAID-1 arrays each having one disk on the
primary IDE port and one on the secondary IDE port. Performance
will still suck when you do I/O on both arrays at the same time
but when one array is idle, the other will work OK.

This is of course not as good as RAID-5 from a disk space/euro
POV.

>  Should I got for ATA or SATA drives and why?

SATA is better because 1) it doesn't have the master/slave
issues of IDE, i.e. if you have 4 SATA ports on your
motherboard, you *can* do a 4-disk RAID-5 array and 2)
motherboards with 8 SATA ports are easy to find.

>  * heat dissipation could become a problem with so many hard drives

I would not want to do it without adequate ventilation.

>  * I need a reliable and stable power supply

Fortron FSP-400-60GLN works for me. We have had issues with
Antec.

>  People in the know use software based RAID. Could you give me links
> to these kinds of discussions?

The archives of the linux-raid mailing list (the administration
tool is called mdadm).

>  What would be my weak/hotspot points in my kind of design?

For me, the time was spent on
- understanding mdadm,
- understanding the trade-offs (partioning an array of disks vs.
making an array of partitions, using LVM or not, optimum
granularity) and
- hardware (how to fit 8 or more disks in a PC case with decent
ventilation).

>  Any suggestions of the type of boxes/racks I should use?

3ware make 3-disks-in-2-5.25"-spaces trays. They are expensive
and the fans they use die after about a year. When a fan goes
bad, the tray helpfully warns you about it by beeping loudly and
constantly. The fans are not the easiest to find (60 mm or some
such). Be prepared to hear a lot of beeping.

I made my own trays. It was a lot of work and they look ugly but
it was cheap and they do the job. The ventilation is superior to
commercial trays (120 mm fan than moves a lot of air quietly and
reliably).

One very important thing about RAID that too many people
overlook : don't make an N-disk array from N disks of the same
make and model bought the same day. Our sysadmin at work did and
both drives on a RAID-1 array failed within days of each
other...

--
André Majorel <URL:http://www.teaser.fr/~amajorel/>
(Counterfeit: elyzekef@sinuous.com abyfiv@demur.com)
"Duty, honor, country" -- Douglas MacArthur
"Travail, famille, patrie" -- Philippe Pétain





[ Post a follow-up to this message ]



    Sponsored Links  




 





   All times are GMT. The time now is 09:06 AM.      Post New Thread    Post A Reply      
  Last Thread   Next Thread Next


Most Popular forums 

Forum Jump:
Rate This Thread:

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is OFF
vB code is ON
Smilies are ON
[IMG] code is OFF
 
Medical and Health forum | Computer Games Reviews | Graphics design forum

Back To The Top
Home | Usercp | Faq | Register