Hi,

I was wondering if anyone knew how to increase the maximum buffer size
for a FIFO file type.  Right now, it's at 4KB, but I want this to be
much larger - if possible.

As a point of reference, I am trying to do a weird type of
interprocess communication.  I have one program logging - at an
unknown frequency.  I then have another program that reads the FIFO
every 5 mins to scan all the entries.  As you can see, it's possible
that the first program exceeds the LINUX limit of 4KB and I want to
prevent this.

Thanks in advance, any ideas appreciated.
--Sachin

P.S. Any drawbacks to increasing the bufsize or is there any easier
way to do this?

[ Post a follow-up to this message ]

Sachin Doshi wrote:
> Hi,
>
> I was wondering if anyone knew how to increase the maximum buffer size
> for a FIFO file type.  Right now, it's at 4KB, but I want this to be
> much larger - if possible.
..
>
> P.S. Any drawbacks to increasing the bufsize or is there any easier
> way to do this?
Sound better to me just having the reader of the log continously
reading from the pipe.

[ Post a follow-up to this message ]

In article <beb6d765.0410200033.1076a532@posting.google.com>,
ssdas4eva@yahoo.com (Sachin Doshi) wrote:

> Hi,
>
> I was wondering if anyone knew how to increase the maximum buffer size
> for a FIFO file type.  Right now, it's at 4KB, but I want this to be
> much larger - if possible.

It's typically compiled into the kernel, not something you can change on
the fly.  If there's a way to change it, it will be system-dependent.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

[ Post a follow-up to this message ]

Sachin Doshi wrote:

>
>
> P.S. Any drawbacks to increasing the bufsize or is there any easier
> way to do this?

Use syslog.

-- ced


--
Chuck Dillon
Senior Software Engineer
NimbleGen Systems Inc.

[ Post a follow-up to this message ]

Sachin Doshi <ssdas4eva@yahoo.com> wrote:
>I was wondering if anyone knew how to increase the maximum buffer size
>for a FIFO file type.  Right now, it's at 4KB, but I want this to be
>much larger - if possible.

This kind of question should ring an alarm bell.  Sometimes it's needed, but
there's almost always a more reliable/maintainable/portable/efficient soluti
on
that doesn't involve tweaking system internals by orders of magnitude.

>As a point of reference, I am trying to do a weird type of
>interprocess communication.  I have one program logging - at an
>unknown frequency.  I then have another program that reads the FIFO
>every 5 mins to scan all the entries.

Doesn't sound wierd to me - that's pretty standard, except for the 5 minutes
thing.

>As you can see, it's possible that the first program exceeds the LINUX
>limit of 4KB and I want to prevent this.

It's possible that it will exceed any finite amount you set.

>P.S. Any drawbacks to increasing the bufsize or is there any easier
>way to do this?

You can probably find a way to change the bufsize, but it'll vary based on
your flavor of unix, and there's no way to tell what effect it'll have on
other apps, which may (incorrectly, but that's the world for you) depend on
having reasonably small pipe sizes.

A far better solution is to read from the pipe continuously, blocking if
there's no data.  If your main reader is truly restricted to every 5 minutes
,
you could use a helper app that reads from the pipe, has it's own buffer, an
d
writes to a different pipe that the main logger reads from.  This helper
can grow it's buffers as needed, buffer to disk, or whatever it takes.
--
Mark Rafn    dagon@dagon.net    <http://www.dagon.net/>

[ Post a follow-up to this message ]

Thanks for all the suggestions.  Decided to continuously read from the
pipe and just print the data to a file.  ( The five minute thing was
because the data needs to be put in a database and if requests are
coming in continuously, the database would be constanly hammered).

I am trying to implement your suggestion with mixed results.  I am
using trying to use the select() function to block until the program
sees some data on the pipe.  Buf for some odd reason, the select is
constantly returning - even when there is no new data in the FIFO
pipe.  Here's my code if it helps.



fd_set rfds;
int retval;
char *buf;
size_t n = 0;
struct timeval tv;
FILE *output = fopen("out", "a");
FILE *log = fopen("fifo", "r");

FD_ZERO(&rfds);
FD_SET(fileno(log), &rfds);

while(1) {
tv.tv_sec = 300;
tv.tv_usec = 0;
retval = select(fileno(log) + 1, &rfds, NULL, NULL, &tv);

if (retval) {
while(getline(&buf, &n, log) != -1) {
fprintf(out, "%s", buf);
}
}
}


dagon@dagon.net (Mark Rafn) wrote in message news:<i7hi42-8qv.ln1@hydra.dagon.net>...
> Sachin Doshi <ssdas4eva@yahoo.com> wrote: 
>
> This kind of question should ring an alarm bell.  Sometimes it's needed, b
ut
> there's almost always a more reliable/maintainable/portable/efficient solu
tion
> that doesn't involve tweaking system internals by orders of magnitude.
> 
>
> Doesn't sound wierd to me - that's pretty standard, except for the 5 minut
es
> thing.
> 
>
> It's possible that it will exceed any finite amount you set.
> 
>
> You can probably find a way to change the bufsize, but it'll vary based on
> your flavor of unix, and there's no way to tell what effect it'll have on
> other apps, which may (incorrectly, but that's the world for you) depend o
n
> having reasonably small pipe sizes.
>
> A far better solution is to read from the pipe continuously, blocking if
> there's no data.  If your main reader is truly restricted to every 5 minut
es,
> you could use a helper app that reads from the pipe, has it's own buffer, 
and
> writes to a different pipe that the main logger reads from.  This helper
> can grow it's buffers as needed, buffer to disk, or whatever it takes.

[ Post a follow-up to this message ]

Why aren't you checking if fileno(log) is readable using FD_ISSET()?
The select function returns the number of file descriptors readable.
Also there is no real guarantee between invocations of fileno that
the descriptor returned will be the same.

Most importantly perhaps is that for select to block till data arrives
you must not specify a timeout value. Select will also return with EINTR
in case of signal, so you will have to check for this condition and
except this error.



ssdas4eva@yahoo.com (Sachin Doshi) wrote in message news:<beb6d765.0410212344.6300b408@posti
ng.google.com>...
> Thanks for all the suggestions.  Decided to continuously read from the
> pipe and just print the data to a file.  ( The five minute thing was
> because the data needs to be put in a database and if requests are
> coming in continuously, the database would be constanly hammered).
>
> I am trying to implement your suggestion with mixed results.  I am
> using trying to use the select() function to block until the program
> sees some data on the pipe.  Buf for some odd reason, the select is
> constantly returning - even when there is no new data in the FIFO
> pipe.  Here's my code if it helps.
>
>
>
> fd_set rfds;
> int retval;
> char *buf;
> size_t n = 0;
> struct timeval tv;
> FILE *output = fopen("out", "a");
> FILE *log = fopen("fifo", "r");
>
> FD_ZERO(&rfds);
> FD_SET(fileno(log), &rfds);
>
> while(1) {
>   tv.tv_sec = 300;
>   tv.tv_usec = 0;
>   retval = select(fileno(log) + 1, &rfds, NULL, NULL, &tv);
>
>   if (retval) {
>     while(getline(&buf, &n, log) != -1) {
>       fprintf(out, "%s", buf);
>     }
>   }
> }
>
>

[ Post a follow-up to this message ]