Unix Programming - Making malloc() fail -- stuck

Author

Making malloc() fail -- stuck

tweak

2004-07-11, 5:50 pm

I am trying to create a situation in which malloc()
will fail when called to make sure my error code is
working correctly.

Here's my source:

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

#if 0
#include <string.h>
#endif

#include <sys/resource.h>

int
main(void)

{
struct rlimit r;
r.rlim_cur = 8;
r.rlim_max = 8;
char *ptr = NULL;

if( (setrlimit(RLIMIT_DATA, &r) == -1) ) {
perror("setrlimit() error");
exit(EXIT_FAILURE);
}

if( (ptr = malloc( sizeof(long long) * 256 ) ) == NULL) {
perror("malloc() failed"); /* wanna check if errno is set */
exit(EXIT_FAILURE);
}
(void)printf("malloc() worked\n");
free(ptr);
return 0;
}

Do I have to write a program with the malloc() call in it and then
call it from within another program with one of exec functions?

I don't know why malloc() will not fail. Am I not using the
setrlimit() function correctly?

Thanks,

Brian

--
* Remove x's to send me mail *

James

2004-07-11, 5:50 pm

On Sun, 11 Jul 2004 15:24:56 -0700, tweak <xbwaichunasx@cox.net>
wrote:

>
>I am trying to create a situation in which malloc()
>will fail when called to make sure my error code is
>working correctly.
>
>Here's my source:
>
>#define _GNU_SOURCE
>#include <stdio.h>
>#include <stdlib.h>
>#include <errno.h>
>
>#if 0
>#include <string.h>
>#endif
>
>#include <sys/resource.h>
>
>int
>main(void)
>
>{
> struct rlimit r;
> r.rlim_cur = 8;
> r.rlim_max = 8;
> char *ptr = NULL;
>
> if( (setrlimit(RLIMIT_DATA, &r) == -1) ) {
> perror("setrlimit() error");
> exit(EXIT_FAILURE);
> }
>
> if( (ptr = malloc( sizeof(long long) * 256 ) ) == NULL) {
> perror("malloc() failed"); /* wanna check if errno is set */
> exit(EXIT_FAILURE);
> }
> (void)printf("malloc() worked\n");
> free(ptr);
> return 0;
>}
>
>Do I have to write a program with the malloc() call in it and then
>call it from within another program with one of exec functions?
>
>I don't know why malloc() will not fail. Am I not using the
>setrlimit() function correctly?

There was a thread on this subject in this group very recently. On
some systems, malloc() does NOT allocate *memory*, just *address
space*; memory is then put into place when you first try using that
address space.

You'll probably get the results you expect if you compile with
"-lmalloc", which will use a different malloc() implementation.

James.

Bjorn Reese

2004-07-12, 5:57 pm

On Sun, 11 Jul 2004 15:24:56 -0700, tweak wrote:

> I am trying to create a situation in which malloc()
> will fail when called to make sure my error code is
> working correctly.

It is easier to write your own malloc(). For example, do
something like this:

void *failing_malloc(size_t size)
{
static int number_of_calls = 0;

number_of_calls++;

if (number_of_calls == 22) /* Change 22 to appropriate value */
return 0;

return (malloc)(size);
}

#define malloc(x) failing_malloc(x)

--
mail1dotstofanetdotdk

elia Mazzawi

2004-07-12, 5:57 pm

you can also try calloc()

Jens.Toerring@physik.fu-berlin.de

2004-07-12, 5:57 pm

tweak <xbwaichunasx@cox.net> wrote:

> I am trying to create a situation in which malloc()
> will fail when called to make sure my error code is
> working correctly.

I found, by looking setrlimit up in SUSv3 and playing around, that
your program works when you use RLIMIT_AS instead of RLIMIT_DATA
(with libc 2.3.2) But use something more that 8 bytes as the limit
(the limits are specified in bytes), otherwise the call of perror()
will crash the program with a segmentation fault - looks like it is
using malloc() internally and if it can't get enough memory it
crashes. Using RLIMIT_DATA does not seem to work, malloc() only
returns an error when you get near to the 2 GB address space limit.

But you can also limit the memory usage of a program from the shell,
e.g. with bash using

ulimit -v 10000

limits memory consumption of a program to a maximum of 10 MB only.

Regards, Jens
--
\ Jens Thoms Toerring ___ Jens.Toerring@physik.fu-berlin.de
\__________________________ http://www.toerring.de

tweak

2004-07-12, 8:49 pm

Jens.Toerring@physik.fu-berlin.de wrote:
> tweak <xbwaichunasx@cox.net> wrote:
>
>
>
>
> I found, by looking setrlimit up in SUSv3 and playing around, that
> your program works when you use RLIMIT_AS instead of RLIMIT_DATA
> (with libc 2.3.2) But use something more that 8 bytes as the limit
> (the limits are specified in bytes), otherwise the call of perror()
> will crash the program with a segmentation fault - looks like it is
> using malloc() internally and if it can't get enough memory it
> crashes. Using RLIMIT_DATA does not seem to work, malloc() only
> returns an error when you get near to the 2 GB address space limit.

Thanks. I will try using RLIMIT_AS tonight. I was following what
was said in Advanced Unix programming 2nd Ed. On page 324 it says:

"
RLIMIT_DATA Maximum size of data segment, in bytes. Exceeding it
causes memory-allocation function (e.g. malloc) to fail.
"

I guess I need to send an e-mail to the author to correct that line
for Linux. I am running the 2.4 kernel, so I'm not sure if the results
would be different with the 2.6

Are there any defines I need to do to establish SUSv3, or is using
_GNU_SOURCE good enough?

Note: there is nothing special about the 8 bytes. I just wanted to put
any number lower than my call to malloc()

>
> But you can also limit the memory usage of a program from the shell,
> e.g. with bash using
>
> ulimit -v 10000
>
> limits memory consumption of a program to a maximum of 10 MB only.
>
> Regards, Jens

I wanted to avoid using ulimit for now since I did not want to effect
my shell. I wanted to only effect the single process.

Thanks,

Brian
--
* Remove x's to send me mail *

Jens.Toerring@physik.fu-berlin.de

2004-07-13, 2:50 am

tweak <xbwaichunasx@cox.net> wrote:
> Jens.Toerring@physik.fu-berlin.de wrote:
[vbcol=seagreen]
> Thanks. I will try using RLIMIT_AS tonight. I was following what
> was said in Advanced Unix programming 2nd Ed. On page 324 it says:

> "
> RLIMIT_DATA Maximum size of data segment, in bytes. Exceeding it
> causes memory-allocation function (e.g. malloc) to fail.
> "

> I guess I need to send an e-mail to the author to correct that line
> for Linux. I am running the 2.4 kernel, so I'm not sure if the results
> would be different with the 2.6

I don't think you need to write to Marc Rochkind, he's not only some-
one who regularly answers questions here, so he might be even reading
this, but he can't do anything about Linux not working correctly in
this respect. What you're citing is more or less identical to what
SUSv3 (Single Unix Specification, Version 3) writes, and even the
Linux man page for setrlimit() states something similar (but does not
even mention RLIMIT_AS, at least in the version I have).

> Are there any defines I need to do to establish SUSv3, or is using
> _GNU_SOURCE good enough?

You don't even need _GNU_SOURCE, SUSv3 is basically the current ver-
sion of the POSIX standard, see also

http://www.unix.org/version3/online.html

(registration required).
Regards, Jens
--
\ Jens Thoms Toerring ___ Jens.Toerring@physik.fu-berlin.de
\__________________________ http://www.toerring.de

James Antill

2004-07-13, 7:52 am

On Sun, 11 Jul 2004 15:24:56 -0700, tweak wrote:

>
> I am trying to create a situation in which malloc()
> will fail when called to make sure my error code is
> working correctly.

It's not obvious what you are saying. I initially took this to mean you
have code X, that calls malloc() and it has a failure path. You then want
to trigger that failure path externally.
However the example posted wasn't the normal approach to that problem, so
I'm not positive I understood.

Assuming I got it right though, the solution for malloc() is pretty
simple as you are guaranteed to be able to overload the
malloc()/free()/etc. set of functions and there is often a simple way to
get back to the normal libc implementations.

For instance, on Linux/glibc you'd do something like...

#include <stdlib.h>

static int tst_fail = FALSE;

extern void *__libc_malloc(size_t);
void *malloc(size_t len)
{
if (tst_fail)
return (NULL);

return (__libc_malloc(len));
}

int main(void)
{
ret = MY_api(...); /* check it works */

tst_fail = TRUE;

ret = MY_api(...); /* check it fails */

return 0;
}

....you can get more complicated and use counters to get specific calls in
a code path that calls malloc() more than once ... and also track the
memory to free to make sure you aren't losing bits on the error paths etc.

Note however that this only works reliably with the malloc() set of
functions, if you want to do the same for open() etc. then you'll
probably have to insert your own debugging hooks into the code in question.

--
James Antill -- james@and.org
Need an efficient and powerful string library for C?
http://www.and.org/vstr/

Michael Kerrisk

2004-07-28, 6:19 pm

This is way after the fact, but I've only just seen this:

On 13 Jul 2004 08:05:49 GMT, Jens.Toerring@physik.fu-berlin.de wrote:

>tweak <xbwaichunasx@cox.net> wrote:
>
>
>
>
>I don't think you need to write to Marc Rochkind, he's not only some-
>one who regularly answers questions here, so he might be even reading
>this, but he can't do anything about Linux not working correctly in
>this respect. What you're citing is more or less identical to what
>SUSv3 (Single Unix Specification, Version 3) writes, and even the
>Linux man page for setrlimit() states something similar (but does not
>even mention RLIMIT_AS, at least in the version I have).

Okay -- first off, Linux isn't doing anything wrong here. It's just
that malloc() on Linux/glibc is being clever.

RLMIT_DATA sets a limit on the program break, as adjusted by
brk(2)/sbrk(2). Now malloc() may use sbrk() or sbrk() to allocate the
required memory. But if that fails, it will allocate an area in a
different part of VM using mmap(), and mmap() is unaffected by
RLIMIT_DATA (but *is* affected by RLIMIT_AS). (Note also: malloc()
always uses mmap() for "large" blocks of memory (> 128kb by default,
IIRC).)

On Linux, if you set RLIMIT_DATA and then use explicit sbrk()/brk()
calls then you will hit an ENOMEM error.

Now, to defend Marc's statement a little. He did say

(e.g., malloc())
====

On some implementations, that example wll be true (if the malloc()
always uses brk()/sbrk()), but on some others, as we see, it's not.

Jens: The very latest Linux man pages (http://www.tlpd.org/) do
document RLIMIT_AS.

Cheers,

Michael

brian

2004-07-29, 2:52 am

Michael Kerrisk wrote:
> This is way after the fact, but I've only just seen this:
>
> On 13 Jul 2004 08:05:49 GMT, Jens.Toerring@physik.fu-berlin.de wrote:
>
>
>
>
> Okay -- first off, Linux isn't doing anything wrong here. It's just
> that malloc() on Linux/glibc is being clever.
>
> RLMIT_DATA sets a limit on the program break, as adjusted by
> brk(2)/sbrk(2). Now malloc() may use sbrk() or sbrk() to allocate the
> required memory. But if that fails, it will allocate an area in a
> different part of VM using mmap(), and mmap() is unaffected by
> RLIMIT_DATA (but *is* affected by RLIMIT_AS). (Note also: malloc()
> always uses mmap() for "large" blocks of memory (> 128kb by default,
> IIRC).)
>
> On Linux, if you set RLIMIT_DATA and then use explicit sbrk()/brk()
> calls then you will hit an ENOMEM error.
>
> Now, to defend Marc's statement a little. He did say
>
> (e.g., malloc())
> ====
>
> On some implementations, that example wll be true (if the malloc()
> always uses brk()/sbrk()), but on some others, as we see, it's not.
>
> Jens: The very latest Linux man pages (http://www.tlpd.org/) do
> document RLIMIT_AS.
>
> Cheers,
>
> Michael
>
I have been spending alot of time digging deeper into the whole memory
piece, and the implementation stuff is over my head right now.

The biggest problems I run into now is:

1) documentation that differs from source code

and/or

2) source code implementation (such as extra tests as
found in calloc() on BSD) that differs from standard

I am not saying that the author was wrong when he wrote what
was in the man page. I am saying that the man page isn't
exactly right.

I am also saying that you have to test the documentation to make sure
your code is error free. You cannot count 100% on the documentation;
you have to test. That's the learning I walked away from my
attempt to make malloc() fail. Test. Test. Test.

brian