|
Home > Archive > Unix Programming > January 2008 > Reading an unclosed file while data is being written into it.
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Reading an unclosed file while data is being written into it.
|
|
| dh..@gmail.com 2008-01-02, 7:23 pm |
| Hi,
I need some expert opinion on this:
There is a C program ( Pwriter), which creates 5 files with an average
size of 2gb each. There is another C program (Preader), which reads
these 5 files. The issue is that Preader needs to wait for almost 2.5
hrs, before Pwriter closes all the 5 files.
I intend to change these processes such that Pwriter creates an entry
into a control table after writing(unbuffered write) 100 records into
the output files. Preader will check control table and then read 100
records( per entry in control table) from the files.
I wrote some dummy code for this parallel read and write and
everything seems to be working fine. Just looking for an expert
opinion on this if there is any hidden potential problem with this
approach.
Regards,
Dh..
| |
| Barry Margolin 2008-01-03, 1:44 am |
| In article
<37ccf937-44c0-4df1-846f-9e664c6fc96b@d4g2000prg.googlegroups.com>,
"dh..@gmail.com" <dheeraj087@gmail.com> wrote:
> Hi,
> I need some expert opinion on this:
>
> There is a C program ( Pwriter), which creates 5 files with an average
> size of 2gb each. There is another C program (Preader), which reads
> these 5 files. The issue is that Preader needs to wait for almost 2.5
> hrs, before Pwriter closes all the 5 files.
>
> I intend to change these processes such that Pwriter creates an entry
> into a control table after writing(unbuffered write) 100 records into
> the output files. Preader will check control table and then read 100
> records( per entry in control table) from the files.
>
> I wrote some dummy code for this parallel read and write and
> everything seems to be working fine. Just looking for an expert
> opinion on this if there is any hidden potential problem with this
> approach.
As long as you're using the control table to tell the reader how much
has been written, this should work fine.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
| |
| Vakayil Thobias 2008-01-03, 1:44 am |
|
"dh..@gmail.com" <dheeraj087@gmail.com> wrote in message
news:37ccf937-44c0-4df1-846f-9e664c6fc96b@d4g2000prg.googlegroups.com...
> Hi,
> I need some expert opinion on this:
>
> There is a C program ( Pwriter), which creates 5 files with an average
> size of 2gb each. There is another C program (Preader), which reads
> these 5 files. The issue is that Preader needs to wait for almost 2.5
> hrs, before Pwriter closes all the 5 files.
>
> I intend to change these processes such that Pwriter creates an entry
> into a control table after writing(unbuffered write) 100 records into
> the output files. Preader will check control table and then read 100
> records( per entry in control table) from the files.
>
> I wrote some dummy code for this parallel read and write and
> everything seems to be working fine. Just looking for an expert
> opinion on this if there is any hidden potential problem with this
> approach.
>
> Regards,
> Dh..
>
You can use multithreading in your program.
One thread can read data from the file and another thread can write data
into another file.
|
|
|
|
|