|
Home > Archive > Unix Programming > May 2006 > fread - ensure read is always derived from file, not from cache?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
fread - ensure read is always derived from file, not from cache?
|
|
|
| Hi,
I'm looking for help with fread.
Everytime I execute an fread command I would like for it to read
directly
from the hardware device rather than from cache.
I have found a way to ensure fwrite always writes to the device using
sync().
Does anyone know of a way to guarantee fread always reads directly from
the
file opened by fopen?
I've also looked at setvbuf(no buffering option) and __fpurge but
neither solves my problem.
I've also looked at open/read/write/close but the caching affects them
also.
open a file
write a file
seek to 0
read file (read come from buffer, want to come from device).
Is there a way to prevent the filesystem/operating system from
buffering/caching data
for reads? I realize this is not the desired behavior for performance
concerns but have
a special case.
Do not wish to use mount/umount since will have to close files and
reopen them since
I will be performing many reads and need for them all to be derived
from the specified file.
Thank you,
Ed Ruff
| |
| Maxim Yegorushkin 2006-05-29, 5:32 pm |
|
eruff wrote:
> Hi,
>
> I'm looking for help with fread.
> Everytime I execute an fread command I would like for it to read
> directly
> from the hardware device rather than from cache.
> I have found a way to ensure fwrite always writes to the device using
> sync().
> Does anyone know of a way to guarantee fread always reads directly from
>
> the
> file opened by fopen?
Use open() with SYNC flags and read()/write() instead of libc f*
functions.
http://www.opengroup.org/onlinepubs...tions/open.html
http://www.opengroup.org/onlinepubs...html#tag_03_373
| |
| Fred Kleinschmidt 2006-05-29, 5:32 pm |
|
"eruff" <eruffing@gmail.com> wrote in message
news:1148651549.093003.207320@38g2000cwa.googlegroups.com...
> Hi,
>
> I'm looking for help with fread.
> Everytime I execute an fread command I would like for it to read
> directly
> from the hardware device rather than from cache.
> I have found a way to ensure fwrite always writes to the device using
> sync().
> Does anyone know of a way to guarantee fread always reads directly from
>
> the
> file opened by fopen?
> I've also looked at setvbuf(no buffering option) and __fpurge but
> neither solves my problem.
> I've also looked at open/read/write/close but the caching affects them
> also.
>
>
> open a file
> write a file
> seek to 0
> read file (read come from buffer, want to come from device).
>
>
> Is there a way to prevent the filesystem/operating system from
> buffering/caching data
> for reads? I realize this is not the desired behavior for performance
> concerns but have
> a special case.
> Do not wish to use mount/umount since will have to close files and
> reopen them since
> I will be performing many reads and need for them all to be derived
> from the specified file.
>
>
> Thank you,
> Ed Ruff
>
See setbuf() or setvbuf()
--
Fred L. Kleinschmidt
Boeing Associate Technical Fellow
Technical Architect, Software Reuse Project
| |
|
| Have tried both setbuf and setvbuf without any success.
rc = setvbuf(fd, (char *)NULL, _IONBF, 0);
or
setbuf(fd, NULL);
They must be being cached as well.
Did see a mention of fclean but appears it is unavailable.
| |
| Maxim Yegorushkin 2006-05-29, 5:32 pm |
|
eruff wrote:
> Have tried both setbuf and setvbuf without any success.
>
> rc = setvbuf(fd, (char *)NULL, _IONBF, 0);
> or
> setbuf(fd, NULL);
>
> They must be being cached as well.
> Did see a mention of fclean but appears it is unavailable.
You are trying to use a wrong API for your task.
| |
|
| I have tried both
1) open/read/close
and
2) fopen/fread/fclose
approaches
I tried setbuf and setvbuf with 2.
I tried the O_SYNC, O_RSYNC, O_DSYNC, and O_FSYNC flags with 1.
Upon performing:
open
read (generates read requests from hw)
read (hw not accessed)
#define FILE_MODE (S_IRWXU | S_IRWXG | S_IRWXO)
int main()
{
char buf[255];
FILE *fd;
int bytes_read, rc = 0, fh;
char buffer[2000];
fh = open("/mmc/mmca1/abc", O_RDWR | O_RSYNC, FILE_MODE);
printf("fh = %d errno = %d\n", fh, errno);
bytes_read = read(fh, buffer, 1000);
printf("bytes_read = %d errno = %d\n", bytes_read, errno);
rc = close(fh);
printf("rc = %d errno = %d\n", rc, errno);
Someone mentioned usage of the O_DIRECT flag but appears in unavailable
in my flavor of linux.
Thanks!
| |
|
|
eruff wrote:
> Hi,
>
> I'm looking for help with fread.
> Everytime I execute an fread command I would like for it to read
> directly
> from the hardware device rather than from cache.
> I have found a way to ensure fwrite always writes to the device using
> sync().
sync() isn't the right call, btw. 'man fsync'
> Does anyone know of a way to guarantee fread always reads directly from
>
> the
> file opened by fopen?
> I've also looked at setvbuf(no buffering option) and __fpurge but
> neither solves my problem.
> I've also looked at open/read/write/close but the caching affects them
> also.
>
>
> open a file
> write a file
> seek to 0
> read file (read come from buffer, want to come from device).
>
>
> Is there a way to prevent the filesystem/operating system from
> buffering/caching data
> for reads? I realize this is not the desired behavior for performance
> concerns but have
> a special case.
> Do not wish to use mount/umount since will have to close files and
> reopen them since
> I will be performing many reads and need for them all to be derived
> from the specified file.
>
>
> Thank you,
> Ed Ruff
| |
| Maxim Yegorushkin 2006-05-29, 5:32 pm |
|
eruff wrote:
> I have tried both
> 1) open/read/close
> and
> 2) fopen/fread/fclose
> approaches
> I tried setbuf and setvbuf with 2.
setvbuf only affects libc buffer which is in the user space.
> I tried the O_SYNC, O_RSYNC, O_DSYNC, and O_FSYNC flags with 1.
>
> Upon performing:
> open
> read (generates read requests from hw)
> read (hw not accessed)
For block devices the kernel reads entire blocks. So, if you do several
reads which read the same block, the block will only be read once from
the device.
| |
|
| How about using the dd command?
Would require unmounting the device.
Should I still be able to access the device using dd if the device in
unmounted?
Also, I've tried sync with fopen/fwrite/fread/fclose and fsync with
open/write/read/close
but unfortunately sync/fsync only apply to writes. I need for the read
to always access the device (i.e no caching/buffering).
I could use fread with unmount but would require unmounting after every
read which would
be performance wise horrendous.
Wondering is "dd" could work with device unmounted.
Thanks again!
| |
|
| sync() is used with fopen/fwrite/fread/fclose
fsync() is used with open/read/write/close
The "f" character confuses things. Would think the oppostive is true.
|
|
|
|
|