Data Storage - Any software libraries or techniques for using disks as large FIFO?

Author

Any software libraries or techniques for using disks as large FIFO?

johnfofawn@hotmail.com

2005-06-02, 5:47 pm

Hi,

I have an application where digital data is streamed to me at a rate of
20 MB/sec. I have 6 250GB disks totallying 1.5 TB of storage.

I want to store as much data as possible and age off old data as new
data arrives. I need to allow "readers" to pull from the 20 MB/sec data
stream at any point in the stream. I need to allow for "real-time"
readers that can essentially pull data as it is written into the "disk
buffer".

Are there any software development tools or packages that do this?

My code is in C, compiled using Intel compiler, developed using Visual
Studio on Windows 2003 Server Standard.

My code is portable (it also runs on Linux). If there is a way that I
can do this a preserve portability I would like to do that.

Ideas? Thoughts? Comments?

THANKS!
John

RPR

2005-06-09, 5:47 pm

I'd pre-allocate a number of files with a fixed size and use them as
blocks in a FIFO. Keep the FIFO pointers in shared memory and you're
all set.

Ralf-Peter