|
Home > Archive > Unix Programming > August 2005 > malloc() failure
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
| Roman Mashak 2005-08-22, 8:49 pm |
| Hello, All!
It's considered a right way to check result of malloc(). If 'malloc' failed
(that's returned NULL) - is there a general way to deal with it or it
strictly depends on application design? On the other hand, the reason memory
wasn't allocated might be far beyond of my program's concern, it may be OS
faulty or similar.
In my case, I just invoke 'exit(EXIT_FAILURE)'.
Anyway, I'd appreciate your comments, thanks.
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Fletcher Glenn 2005-08-23, 2:48 am |
|
"Roman Mashak" <mrv@tusur.ru> wrote in message
news:dedu2v$ph0$1@relay.tomsk.ru...
> Hello, All!
>
> It's considered a right way to check result of malloc(). If 'malloc'
> failed (that's returned NULL) - is there a general way to deal with it or
> it strictly depends on application design? On the other hand, the reason
> memory wasn't allocated might be far beyond of my program's concern, it
> may be OS faulty or similar.
>
> In my case, I just invoke 'exit(EXIT_FAILURE)'.
>
> Anyway, I'd appreciate your comments, thanks.
>
> With best regards, Roman Mashak. E-mail: mrv@tusur.ru
>
Even though your program may not be at fault, you can get a malloc
error if the system runs out of swap space and all of real memory is
full. The only way you can deal with this from within the program
is to either re-use some memory you have already got, or give up.
--
Fletcher Glenn
| |
| Gordon Burditt 2005-08-23, 2:48 am |
| >It's considered a right way to check result of malloc(). If 'malloc' failed
>(that's returned NULL) - is there a general way to deal with it or it
>strictly depends on application design? On the other hand, the reason memory
>wasn't allocated might be far beyond of my program's concern, it may be OS
>faulty or similar.
>
>In my case, I just invoke 'exit(EXIT_FAILURE)'.
There are several things you might consider doing first:
- Print an error message (if possible).
- If you've got temporary files open, close them and delete them.
This comes under the general heading of "clean up your mess".
- If it's an interactive program like an editor, consider aborting
the command and either let the user try another command, or give
the user a chance to save his work (which hopefully doesn't require
more memory to handle).
- Log the failure, including how much memory you tried to allocate.
Some failures to allocate memory happen because the amount of memory
to allocate is very silly (random bits or (unsigned long)-1 or
something like that).
Gordon L. Burditt
| |
| Pascal Bourguignon 2005-08-23, 2:48 am |
| "Roman Mashak" <mrv@tusur.ru> writes:
> It's considered a right way to check result of malloc(). If 'malloc' failed
> (that's returned NULL) - is there a general way to deal with it or it
> strictly depends on application design?
Time to call the garbage collector!
> On the other hand, the reason memory
> wasn't allocated might be far beyond of my program's concern, it may be OS
> faulty or similar.
>
> In my case, I just invoke 'exit(EXIT_FAILURE)'.
--
__Pascal Bourguignon__ http://www.informatimago.com/
Kitty like plastic.
Confuses for litter box.
Don't leave tarp around.
| |
| Roman Mashak 2005-08-23, 2:48 am |
| Hello, Pascal!
You wrote on Tue, 23 Aug 2005 06:54:25 +0200:
??>> failed (that's returned NULL) - is there a general way to deal with it
??>> or it strictly depends on application design?
PB> Time to call the garbage collector!
Is it some type of utility?
??>> On the other hand, the reason memory
With best regards, Roman Mashak. E-mail: mrv@tusur.ru
| |
| Maxim Yegorushkin 2005-08-23, 7:54 am |
|
Roman Mashak wrote:
> Hello, All!
>
> It's considered a right way to check result of malloc(). If 'malloc' failed
> (that's returned NULL) - is there a general way to deal with it or it
> strictly depends on application design? On the other hand, the reason memory
> wasn't allocated might be far beyond of my program's concern, it may be OS
> faulty or similar.
>
> In my case, I just invoke 'exit(EXIT_FAILURE)'.
There are two different cases with malloc.
The first is when RLIMIT_DATA for your process is not RLIM_INFINITY. In
this case malloc can return 0 when brk() or mmap() fails.
The second is when RLIMIT_DATA == RLIM_INFINITY. In this case man
malloc:
BUGS
By default, Linux follows an optimistic memory allocation
strategy.
This means that when malloc() returns non-NULL there is no
guarantee
that the memory really is available. This is a really bad bug.
In case
it turns out that the system is out of memory, one or more
processes
will be killed by the infamous OOM killer. In case Linux is
employed
under circumstances where it would be less desirable to
suddenly lose
some randomly picked processes, and moreover the kernel version
is suf-
ficiently recent, one can switch off this overcommitting
behavior using
a command like
# echo 2 > /proc/sys/vm/overcommit_memory
See also the kernel Documentation directory, files
vm/overcommit-
accounting and sysctl/vm.txt.
| |
| Ulrich Hobelmann 2005-08-23, 7:54 am |
| Pascal Bourguignon wrote:
> "Roman Mashak" <mrv@tusur.ru> writes:
>
> Time to call the garbage collector!
Maybe that's why some Java programs constantly grow in their memory
usage -- they don't invoke the GC until the whole system runs out of
memory ;)
--
I believe in Karma. That means I can do bad things to people
all day long and I assume they deserve it.
Dogbert
| |
|
|
| John Gordon 2005-08-24, 6:11 pm |
| In <dedu2v$ph0$1@relay.tomsk.ru> "Roman Mashak" <mrv@tusur.ru> writes:
> Hello, All!
> It's considered a right way to check result of malloc(). If 'malloc' failed
> (that's returned NULL) - is there a general way to deal with it or it
> strictly depends on application design? On the other hand, the reason memory
> wasn't allocated might be far beyond of my program's concern, it may be OS
> faulty or similar.
> In my case, I just invoke 'exit(EXIT_FAILURE)'.
> Anyway, I'd appreciate your comments, thanks.
It depends on the program and its intended use.
For small unpolished programs, exiting would be perfectly fine. But for
larger programs intended for general use, exiting would be very bad. I'd
be upset if Microsoft Word exited on a malloc failure after I'd just typed
in my entire graduate thesis, for example.
In general, any module that requests the allocation of resources should
be able to gracefully handle failure, and pass an error code back to
its caller.
--
John Gordon "It's certainly uncontaminated by cheese."
gordon@panix.com
| |
| Rich Teer 2005-08-24, 6:11 pm |
| On Wed, 24 Aug 2005, John Gordon wrote:
> be upset if Microsoft Word exited on a malloc failure after I'd just typed
> in my entire graduate thesis, for example.
More fool you for using Word for such an important document! ;-)
--
Rich Teer, SCNA, SCSA, OpenSolaris CAB member
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich
| |
| joe@invalid.address 2005-08-24, 6:11 pm |
| Rich Teer <rich.teer@rite-group.com> writes:
> On Wed, 24 Aug 2005, John Gordon wrote:
>
>
> More fool you for using Word for such an important document! ;-)
Word is a fact of life now, unfortunately. But the specific
application isn't the point. According to the preface in your book,
you wrote it using vi, wouldn't you be upset if vi did something like
that (assuming you wrote the whole book in one setting without saving
changes along the way :-)?
Joe
| |
| Rich Teer 2005-08-24, 8:53 pm |
| On Wed, 24 Aug 2005 joe@invalid.address wrote:
> Word is a fact of life now, unfortunately. But the specific
Not necessarily. I use StarOffice whereever WOrd is "required".
> application isn't the point. According to the preface in your book,
> you wrote it using vi, wouldn't you be upset if vi did something like
> that (assuming you wrote the whole book in one setting without saving
> changes along the way :-)?
Yeah, I guess I would be pretty cheesed off. But I'd also not save that
infrequently (and I always ensure that my personal machines are generously
(some might say over) specified. FOr example, I don't really *need* 2 GB
of RAM on this Ferrari laptop, but that didn't stop me from installing
that much... ;-)
--
Rich Teer, SCNA, SCSA, OpenSolaris CAB member
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich
| |
| joe@invalid.address 2005-08-25, 2:54 am |
| Rich Teer <rich.teer@rite-group.com> writes:
> On Wed, 24 Aug 2005 joe@invalid.address wrote:
>
>
> Not necessarily. I use StarOffice whereever WOrd is "required".
Doesn't work for me. MS has too many hidden features that SO doesn't
deal with. Whenever I have to edit a Word document at work nobody else
can read it if I use SO/OO.
If you can live without it I'm happy for you, but many of the rest of
us have to live with it.
Joe
| |
| William Ahern 2005-08-25, 2:54 am |
| Roman Mashak <mrv@tusur.ru> wrote:
> Hello, All!
> It's considered a right way to check result of malloc(). If 'malloc' failed
> (that's returned NULL) - is there a general way to deal with it or it
> strictly depends on application design? On the other hand, the reason memory
> wasn't allocated might be far beyond of my program's concern, it may be OS
> faulty or similar.
>
> In my case, I just invoke 'exit(EXIT_FAILURE)'.
>
> Anyway, I'd appreciate your comments, thanks.
It depends on the application. For instance, in a network server you might
preallocate a sufficient amount of memory to service a client when you
accept the connection. If malloc(3) fails, you close the socket. You don't
exit the program because you might be handling hundreds of other clients,
and the failure is likely transient so odds are in a few seconds or minutes
you'll be able to accept more clients.
You don't have to preallocate, but handling failures of small allocations
within larger contexts is more difficult and requires a stronger design. You
have to design so that at any point you can unwind and deallocate whatever
context you're working in, rather than just at the end of whatever logical
job you're accomplishing. For instace, in one server I'm writing most
operations servicing a connection allocate memory from an arena,
preallocated on connect with sufficient memory for most cases. If growing
the arena--by malloc(3)--fails, destroying the context isn't as difficult
since most memory is allocated from the arena. I can free the arena (which
handles free'ing the blocks it got from malloc) and there's less cleanup
involved. It's an abtraction that has reduced complexity.
You need to know what type of availability you want to provide. Technically
it's even possible to catch segmentation faults and fail gracefully (or
possibly limp along). But that requires some serious effort and knowledge,
at which point you're not catching SIGSEGV or SIGBUS to cover up coding
mistakes, but to catch hardware errors.
Many GUI applications just abort() when malloc fails. For most that's okay.
glib--not glibc--does it by default, which annoys me when Evolution of
Galeon die on me (if I disabled Linux's OOM killer I'm damned, if I don't
I'm damned).
| |
| William Ahern 2005-08-25, 2:54 am |
| Maxim Yegorushkin <maxim.yegorushkin@gmail.com> wrote:
<snip>
> # echo 2 > /proc/sys/vm/overcommit_memory
<snip>
The worst part about the OOM killer is the culture it encourages. Too many
programmers don't expect nor care to deal with malloc() failing.
Enabled or disabled, most desktop applications (all Gtk/GNOME applications,
possibly KDE) will fail on malloc() failure. It kills me. ;)
| |
| Ulrich Hobelmann 2005-08-25, 7:49 am |
| William Ahern wrote:
> The worst part about the OOM killer is the culture it encourages. Too many
> programmers don't expect nor care to deal with malloc() failing.
That's a problem, maybe. But I think most people wrap malloc in another
function, anyway.
> Enabled or disabled, most desktop applications (all Gtk/GNOME applications,
> possibly KDE) will fail on malloc() failure. It kills me. ;)
But what's the alternative? If malloc fails, I usually print an error
and quit. How are you supposed to even clean up if there's no more
memory? Luckily most systems simply have enough memory that as one
process grows, another shrinks, and it all works out. As soon as the
system starts swapping, the user notices that he hasn't got enough RAM
anyway. It won't crash, but performance alone will encourage him to
change the memory config, or the processes he runs.
--
I believe in Karma. That means I can do bad things to people
all day long and I assume they deserve it.
Dogbert
| |
| Giorgos Keramidas 2005-08-25, 7:49 am |
| joe@invalid.address writes:
>Rich Teer <rich.teer@rite-group.com> writes:
>
> Word is a fact of life now, unfortunately. But the specific
> application isn't the point. According to the preface in your book,
> you wrote it using vi, wouldn't you be upset if vi did something like
> that (assuming you wrote the whole book in one setting without saving
> changes along the way :-)?
That's one assumption too many, if you ask me. Why would one write an
entire thesis/book/insert your favorite long document here and do all
this without ever saving one's work?
If ``saving'' a file is what you were thinking, it may not be possible
at the time malloc() fails. Saving the file itself may require opening
the file, i.e. calling fopen(), and your program might be ``surprised''
to note that fopen() might fail when it calls malloc() too.
The rest about Word documents is irrelevant.
| |
| Eric Sosman 2005-08-25, 6:04 pm |
|
Ulrich Hobelmann wrote:
> William Ahern wrote:
>
>
>
> That's a problem, maybe. But I think most people wrap malloc in another
> function, anyway.
>
>
>
>
> But what's the alternative? If malloc fails, I usually print an error
> and quit. How are you supposed to even clean up if there's no more
> memory?
/* Load an image from `filename' and return an image
* descriptor, or complain and return NULL if unable
* to load the image.
*/
Image_t *loadImageFile(const char *filename) {
...
fid = open(filename, O_RDONLY);
if (fid == -1) {
perror ("Couldn't open");
return NULL;
}
...
buff = malloc(bigBatchOfBytes);
if (buff == NULL) {
perror ("Insufficient memory");
close (fid);
return NULL;
}
...
return imageDescriptor;
}
In other words, a malloc() failure is just like any
other failure: whether it's fatal to the program as a whole
or merely to the operation in progress at the moment is a
property of the program, not of the failure. Should the
program terminate "with prejudice" if it's unable to open
a file? Maybe, maybe not: it depends on the program. The
same goes for malloc() failure, getenv() failure, strtol()
failure, ... The seriousness of a failure is not in the
failure itself, but in its consequences -- and in different
programs the consequences will be different.
--
Eric.Sosman@sun.com
| |
| Brian Raiter 2005-08-25, 6:04 pm |
| > If ``saving'' a file is what you were thinking, it may not be possible
> at the time malloc() fails. Saving the file itself may require opening
> the file, i.e. calling fopen(), and your program might be ``surprised''
> to note that fopen() might fail when it calls malloc() too.
>
> The rest about Word documents is irrelevant.
Well, if we're going to focus on what's relevant to the question ...
If I were using my editor of choice, be it Word or vi or whatever
(come on people), and if I ran out of memory while attempting to save
the file, no matter HOW frequently I had been saving my work up until
that point ... if the editor handled the memory failure by aborting, I
would in fact be cheesed off.
It wouldn't even have to be a save command. Any command that hit an
OOM error. I expect my text editor to be better prepared for OOM
situations than that. For some programs/situations, aborting on OOM is
a perfectly appropriate response. Text editors are not one of them.
b
|
|
|
|
|