Data Storage - Ideas for multimedia low-cost solution sought

Author

Ideas for multimedia low-cost solution sought

Morten Reistad

2004-12-09, 8:47 pm

A frient threw a problem at me that I want to present to a bigger audience.

He is in the audio/video business. They record lots of audio (and some video)
in digital formats, and use some very expen$ive gear to actually store the data.

He wonders if standard systems can be built from off-the-shelf components that
can equal the performance of these systems; so he can offload less critical work.

The worst case scenario implies streaming of around 18 MB per second for a few hours.
It should be free of SPOF's, but is not extremely critical [2]. If they are cheap enough
he can duplicate them for redundancy. He needs around 4T of actual data storage for this
to be useful. [1]

I went home and did some stress tests on a simple 4-way raid5 using stock 7200 rpm ATA 180G
disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz old Via/C3)
of around 5 MB/sec on write, and around three times that on read. This was via NFS over
a gigabit ethernet.

Simple math tell me that this should be possible with a raid-over-raid. If we make a
software raid on top of hardware raids we may get to the bandwidth, but be stuck on the
raid5 write penalty. We could do stripe-on-raids or raids-on-stripes; but here I am a
little short on theory on what I would recommend. I also foresee interference problems
between stripe sizes if raids/mirrors are run on top of each other.

Also, this should ideally be accessible from Macs and Windows machines. Is such
perfomance (18 MB/SEC) over NFS or SMB even possible? ( I do 5(write) and 15(read) on pretty
low-end hardware without much problems).

Setting up disks to do a lower-scale test is doable.

[1] Actually, having one or two of these systems could alliviate a congested backup window
situation on the existing hardware.

[2] Not used for live performances. Think "Live TV". Could be used as backups for such performances
though.

-- mrr

Tom Petrocelli

2004-12-10, 7:45 am

The demands of digital video and audio are such that the entire system
needs to be specifically tailored to it. Companies like SGI, Pinnacle,
and Avid actual use off the shelf components for their storage systems
but fine tune them to D A/V. In most cases, they add special software to
better manage the streaming data.

Your friend also has to consider the support of these systems. Plunking
in a homegrown solution means not getting support from the tools vendor.
There are a number of resellers that can put a system together as an
aftermarket addition to the major vendors and then support it. That
would seem a better choice.

Tom Petrocelli

"Never send a monster to the work of an evil scientist" - Evil Scientist
while chasing Bugs Bunny

On 12/9/2004 7:50 PM, Morten Reistad wrote:
> A frient threw a problem at me that I want to present to a bigger audience.
>
> He is in the audio/video business. They record lots of audio (and some video)
> in digital formats, and use some very expen$ive gear to actually store the data.
>
> He wonders if standard systems can be built from off-the-shelf components that
> can equal the performance of these systems; so he can offload less critical work.
>
> The worst case scenario implies streaming of around 18 MB per second for a few hours.
> It should be free of SPOF's, but is not extremely critical [2]. If they are cheap enough
> he can duplicate them for redundancy. He needs around 4T of actual data storage for this
> to be useful. [1]
>
> I went home and did some stress tests on a simple 4-way raid5 using stock 7200 rpm ATA 180G
> disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz old Via/C3)
> of around 5 MB/sec on write, and around three times that on read. This was via NFS over
> a gigabit ethernet.
>
> Simple math tell me that this should be possible with a raid-over-raid. If we make a
> software raid on top of hardware raids we may get to the bandwidth, but be stuck on the
> raid5 write penalty. We could do stripe-on-raids or raids-on-stripes; but here I am a
> little short on theory on what I would recommend. I also foresee interference problems
> between stripe sizes if raids/mirrors are run on top of each other.
>
> Also, this should ideally be accessible from Macs and Windows machines. Is such
> perfomance (18 MB/SEC) over NFS or SMB even possible? ( I do 5(write) and 15(read) on pretty
> low-end hardware without much problems).
>
> Setting up disks to do a lower-scale test is doable.
>
> [1] Actually, having one or two of these systems could alliviate a congested backup window
> situation on the existing hardware.
>
> [2] Not used for live performances. Think "Live TV". Could be used as backups for such performances
> though.
>
> -- mrr
>
>
>

Maxim S. Shatskih

2004-12-10, 5:45 pm

> disks, and found a sustainable long-term bandwidth (OpenBSD, Raidframe, 1Ghz
old Via/C3)
> of around 5 MB/sec on write, and around three times that on read. This was
via NFS over
> a gigabit ethernet.

Looks like this is very slow. I would compare several OSes, filesystems and
network protocols. NFS or SMB, FreeBSD/Linux/Windows, FFS/ext2/ext3/FAT/NTFS
and so on.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Morten Reistad

2004-12-10, 8:45 pm

Maxim S. Shatskih wrote:
>
> old Via/C3)
>
>
> via NFS over
>
>
>
> Looks like this is very slow. I would compare several OSes, filesystems and
> network protocols. NFS or SMB, FreeBSD/Linux/Windows, FFS/ext2/ext3/FAT/NTFS
> and so on.
>
That is just my point. A thrown-together pretty lame software-only
solution manages around a third to a quarter of the actual requirement.
A decently designed system should be able to cope pretty nicely.

Now, my question is about getting some pointers BEFORE we embark
on a goose-chase. For something like this I would expect to have to use
a journalling file system like XFS or JFS (possibly ext3, but I don't
know how far that scales well). I would expect to give a lot of tuning
effort to NFS; and I am questioning whether this is at all possible in
SMB. These two are showstoppers, and I would appreciate feedback on the
performance of these. These file system issues I know well from other
projects.

I am also sending a query about guidelines about the effects when
stacking raids.

Have googled, but haven't found anything useful, so I appreciate
insights from this group.

Tom Petrocelli wrote :

>Your friend also has to consider the support of these systems.
Plunking >in a homegrown solution means not getting support from the
tools >vendor. There are a number of resellers that can put a system
together >as an aftermarket addition to the major vendors and then
support it. >That would seem a better choice.

You seem to have a lot more faith in tools vendors than I do. I am here
to get some feedback on the feasibility of the solution before they use
any work on looking for a low(er) cost solution.

A fully digitized record label (which this is) does an astounding amount
of data shuffling; and pay through the nose for systems to do this. In
lots of places this isn't really warranted.

Even a pretty homegrown version can be quite valuable to them, as it can
be used for taking nightly backups; and the cheap system can then
generate tape media in business hours. It can also be used for doing
ad-hoc masters, simple off-line edits, proofs of concept etc without
having to contend for studio time.

(Studio time is around $400/hr).