|
Home > Archive > Unix Programming > February 2004 > Available memory
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
|
|
|
| Is there a function call, or a set of calls, that will tell me how much
physical memory is available to grab with malloc?
I'm sure there is, but I can't find it.
| |
| Nick Landsberg 2004-02-28, 6:33 pm |
|
Allo wrote:
> Is there a function call, or a set of calls, that will tell me how much=
=20
> physical memory is available to grab with malloc?
>=20
> I'm sure there is, but I can't find it.
see if there's an getrlimit(2) function in your
distribution.
man 2 getrlimit
It's a Berkleyism as far as I can tell but may
be in some other kernels as well.
--=20
=D1
"It is impossible to make anything foolproof because fools are so=20
ingenious" - A. Bloch
| |
|
| In article <hR80c.113749$hR.2200197@bgtnsc05-news.ops.worldnet.att.net>,
hukolau@NOSPAM.att.net says...
>
>
> Allo wrote:
>
> =20
>
> see if there's an getrlimit(2) function in your
> distribution.
>
> man 2 getrlimit
>
> It's a Berkleyism as far as I can tell but may
> be in some other kernels as well.
Yeah, there is. (I'm using Solaris, by the way.)
And it just occurred to me that I could always look at the source code
for "top" and see how that prog checks for quantities of available
memory. Should've thought of that sooner.
| |
| Rich Teer 2004-02-28, 6:33 pm |
| On Sat, 28 Feb 2004, Allo wrote:
> Is there a function call, or a set of calls, that will tell me how much
> physical memory is available to grab with malloc?
sysconf is a good one - but the amountof physical memory you have
and what's available for use by malloc two entirely different beasts.
All memory allocated to your process comes from a pool of virtual
memory.
--
Rich Teer, SCNA, SCSA
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-online.net
| |
| those who know me have no need of my name 2004-02-28, 11:33 pm |
| in comp.unix.programmer i read:
>Is there a function call, or a set of calls, that will tell me how much
>physical memory is available to grab with malloc?
why would you want to do that -- just allocate what you need. other
programs may need some memory, and if you are hogging it all they may fail.
--
a signature
| |
| David Schwartz 2004-02-28, 11:33 pm |
|
"those who know me have no need of my name" <not-a-real-address@usa.net>
wrote in message news:m1y8qmo8iw.gnus@usa.net...
> in comp.unix.programmer i read:
The 'malloc' function allocates virtual memory, not physical memory, so
your question makes no sense. A typical UNIX machine has a physical memory
availability target (usually 2-32Mb) and there is nearly always that amount
free.
Free memory is only good for certain very specific things, such as
handling a sudden burst of incoming network traffic. Other than that, free
memory is wasted memory and there is no good reason for an operating system
to waste anything.
[color=darkred]
> why would you want to do that -- just allocate what you need. other
> programs may need some memory, and if you are hogging it all they may
> fail.
Or ask the real question rather than how to implement a solution that
won't work. ;)
DS
| |
|
| In article <c1roo3$o75$1@nntp.webmaster.com>, davids@webmaster.com
says...
>
> "those who know me have no need of my name" <not-a-real-address@usa.net>
> wrote in message news:m1y8qmo8iw.gnus@usa.net...
>
>
>
> The 'malloc' function allocates virtual memory, not physical memory, so
> your question makes no sense.
Physical memory is part of virtual memory, so malloc actually does
allocate physical memory. If I determine the amount of unused physical
memory, then a request for a percentage of that amount should result in
an allocation that resides entirely within physical RAM. This should be
true unless the amount of allocated virtual memory exceeds the machine's
physical memory capacity (adjusted for whatever amount of physical
memory has been reserved by the operating system).
> A typical UNIX machine has a physical memory
> availability target (usually 2-32Mb) and there is nearly always that amount
> free.
>
> Free memory is only good for certain very specific things, such as
> handling a sudden burst of incoming network traffic. Other than that, free
> memory is wasted memory and there is no good reason for an operating system
> to waste anything.
If free memory is wasted memory, then a "physical memory availability
target" (i.e., an amount of physical memory that's always free) means
that memory waste is actually part of the system design.
| |
| Nick Landsberg 2004-02-29, 8:33 am |
|
Allo wrote:
> In article <c1roo3$o75$1@nntp.webmaster.com>, davids@webmaster.com=20
> says...
>=20
>=20
ch[color=darkred]
, so=20[color=darkred]
>=20
>=20
> Physical memory is part of virtual memory, so malloc actually does=20
> allocate physical memory. If I determine the amount of unused physical=
=20
> memory, then a request for a percentage of that amount should result in=
=20
> an allocation that resides entirely within physical RAM. This should b=
e=20
> true unless the amount of allocated virtual memory exceeds the machine'=
s=20
> physical memory capacity (adjusted for whatever amount of physical=20
> memory has been reserved by the operating system).
>=20
>=20
ount=20[color=darkred]
[color=darkred]
ree=20[color=darkred]
stem=20[color=darkred]
>=20
>=20
> If free memory is wasted memory, then a "physical memory availability=20
> target" (i.e., an amount of physical memory that's always free) means=20
> that memory waste is actually part of the system design.
Actually, you're both right.
In the "usual world" of *nix, you often exceed physical
memory as processes get created, live for a few seconds
or a few minuts, then go away. (Transients).
Every command typed into a shell command line
is like this. Paging portions of processes to
th swap device is how the O/S takes care of this
condition. In theory, if I had only one process,
it could grow to physical memory size plus
swap partition size. (I said in theory).
This implies that in order to run a process
once it is next in the pecking order, some
or all of the process image has to be
read in from disk again. On systems with
strict response time requirements this
is considered a "BAD THING" because it leads
to unpredictable response times. At least
even more unpredictable than just because
of the scheduling algorithm.
Those kinds of systems (and I have worked on
several of the over the years) generally
have long lived processes to do the majority
of the work (a few maintenance tasks are
transients but carefully controlled), and
the size of each process is also carefully
controlled by the design.
e.g. Total Memory: 2 GB
Database memory: 1,570 MB
O/S & Utilities: 100 MB
Communications Subsystem: 60 MB
Application Logic Handlers: 150 MB
Reserve: 120 MB (space for transients in here)
Our statisics indicate that we generally
page on average of only once or twice per second
and most of this happens during the short time
period when transients pop up to monitor the
health and welfare of the syste and to
gather statistics on the operation.
--=20
=D1
"It is impossible to make anything foolproof because fools are so=20
ingenious" - A. Bloch
| |
| Rolf Magnus 2004-02-29, 10:33 am |
| Allo wrote:
>
> Physical memory is part of virtual memory, so malloc actually does
> allocate physical memory. If I determine the amount of unused
> physical memory, then a request for a percentage of that amount should
> result in an allocation that resides entirely within physical RAM.
Actually, it might result in no real allocation at all. Many kernels do
lazy allocation, which means that the actual memory allocation doesn't
happen before you access the memory. Therefore, it can even happen that
you allocate more memory than is physically or virtually available.
>
> If free memory is wasted memory, then a "physical memory availability
> target" (i.e., an amount of physical memory that's always free) means
> that memory waste is actually part of the system design.
Under some systems, you often have only few memory free, because the
system tries to use memory for buffers and caches instead of letting it
stay unused. However, as soon as a program needs that memory, it's made
available for it.
| |
| Jem Berkes 2004-02-29, 2:34 pm |
| > The 'malloc' function allocates virtual memory, not physical
> memory, so your question makes no sense. A typical UNIX machine has
> a physical memory availability target (usually 2-32Mb) and there is
> nearly always that amount free.
Most modern UNIX systems are virtual memory systems. When you request a
large allocation, sometimes you get physical RAM; sometimes you get swap
space; sometimes both. Also, sometimes you get nothing... by which I mean,
the kernel delays allocating anything until you actually start using the
area. Here's a description of copy-on-write, used during fork and calloc
http://en.wikipedia.org/wiki/Copy-on-write
This all makes it very difficult to determine what exactly is available (in
terms of resources).
--
Jem Berkes
http://www.sysdesign.ca/
| |
| Jerry Feldman 2004-02-29, 3:34 pm |
| On Sat, 28 Feb 2004 17:38:16 -0500
Allo <all@other.s> wrote:
> Is there a function call, or a set of calls, that will tell me how
> much physical memory is available to grab with malloc?
>
> I'm sure there is, but I can't find it.
To add to what has already been discussed, it is very implementation
specific. Depends if you are on a 32 or 64 bit system, BSD or SysV, and
how the OS manages its own memory. Most Unix systems historically used
brk(2) and sbrk(2) to grow the heap, and when it cannot grow any
more,that becomes your limit unless you have already his the artificial
limit. But, newer mallocs may use mmap(2) to allocate pages as they are
needed. This is more flexible since mmap(2) may allocate pages anywhere
in the process' virtual memory space where sbrk(2) can only allocate in
a contiguous space. And, as I mentioned, memory management is very
implementation specific. Other than that, there are tools (like
getrlimit()) that can give you some information to make a guess.
--
Jerry Feldman <gaf-nospam-at-blu.org>
Boston Linux and Unix user group
http://www.blu.org PGP key id:C5061EA9
PGP Key fingerprint:053C 73EC 3AC1 5C44 3E14 9245 FB00 3ED5 C506 1EA9
| |
| Pascal Bourguignon 2004-02-29, 9:33 pm |
| Jerry Feldman <gaf-noSPAM@blu.org> writes:
> On Sat, 28 Feb 2004 17:38:16 -0500
> Allo <all@other.s> wrote:
>
> To add to what has already been discussed, it is very implementation
> specific.
And anyway, you don't grab physical memory, you grab virtual memory:
the OS can swap your process out anytime.
Perhaps there are some platform specific extensions allowing usercode
to wire a page, but the more wired pages there are the more easy it is
to trash the system with swaps, to crash the processes if the system
does overcommiting, or even to panic it with resource exhaustion.
Niceties we all need to have some fun in our dull lifes...
--
__Pascal_Bourguignon__ http://www.informatimago.com/
There is no worse tyranny than to force a man to pay for what he doesn't
want merely because you think it would be good for him.--Robert Heinlein
http://www.theadvocates.org/
|
|
|
|
|