|
Home > Archive > Unix Programming > April 2007 > logical address space vs physical address space
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
logical address space vs physical address space
|
|
| sam_cit@yahoo.co.in 2007-04-29, 7:18 am |
| Hi Everyone,
It is well known that logical address space and physical address
space are different in case of execution time address binding memory
management.
I think this is what virtual memory management is based on, however,
my doubt is what if i have a 32 bit machine with 5 GB RAM with an
operating system and hardware support for supporting virtual memory
management,
(1) The physical address space is just 4 GB (2^32), hence it is
correct that the last 1 GB of RAM will never be used?
(2) In this case, wouldn't the logical address space and physical
address space be the same?
Thanks in advance!!!
| |
| Chris Friesen 2007-04-29, 7:18 am |
| sam_cit@yahoo.co.in wrote:
> I think this is what virtual memory management is based on, however,
> my doubt is what if i have a 32 bit machine with 5 GB RAM with an
> operating system and hardware support for supporting virtual memory
> management,
>
> (1) The physical address space is just 4 GB (2^32), hence it is
> correct that the last 1 GB of RAM will never be used?
Each process has a different set of page tables, so while each process
can only map 4GB of memory the various processes can each map a
different 4GB. The whole available memory can be used.
Also, the physical address space can be much more than 4GB on a 32-bit
cpu. See x86 with PAE, for instance.
> (2) In this case, wouldn't the logical address space and physical
> address space be the same?
No.
Chris
| |
| sam_cit@yahoo.co.in 2007-04-29, 7:18 am |
| >
> Each process has a different set of page tables, so while each process
> can only map 4GB of memory the various processes can each map a
> different 4GB. The whole available memory can be used.
>
> Also, the physical address space can be much more than 4GB on a 32-bit
> cpu. See x86 with PAE, for instance.
>
Can a 32 bit machine address anything more than 4 GB with just 32
bits?
The register which holds the memory address are of 32 bits, isn't it?
| |
| Gordon Burditt 2007-04-29, 7:18 am |
| > It is well known that logical address space and physical address
>space are different in case of execution time address binding memory
>management.
>
> I think this is what virtual memory management is based on, however,
>my doubt is what if i have a 32 bit machine with 5 GB RAM with an
If the bittedness of a machine is defined by the width of the
physical address, you *CAN'T HAVE* a 32 bit machine with 5 GB RAM
(other than some awkward page-switching mechanism like MSDOS used).
For example, a number of the Pentium processors have a 36-bit
physical address bus, permitting physical memory of 64GB. The
memory-mapping mechanism also handles 36-bit physical addresses.
>operating system and hardware support for supporting virtual memory
>management,
>
> (1) The physical address space is just 4 GB (2^32), hence it is
>correct that the last 1 GB of RAM will never be used?
No. You have to change the memory map for more than 4GB to be used.
It's usually changed for each process switch anyway. The maximum address
space for any single process is 4GB (might be smaller if the kernel
occupies some of that space, often half).
> (2) In this case, wouldn't the logical address space and physical
>address space be the same?
No. And even if the physical memory and virtual memory are both 4G,
that doesn't mean the pages are mapped in order.
| |
| Gordon Burditt 2007-04-29, 7:18 am |
| >> Each process has a different set of page tables, so while each process
>
> Can a 32 bit machine address anything more than 4 GB with just 32
>bits?
It uses more than 32 bits if you consider the page tables.
Yes, you could only address 4GB if you don't change page tables, so
any single process is limited to 4GB (and probably less, if the kernel
takes a lot of the address space).
>The register which holds the memory address are of 32 bits, isn't it?
And how many bits is the register which selects the page tables?
| |
| David Schwartz 2007-04-29, 7:18 am |
| On Apr 29, 1:04 am, sam_...@yahoo.co.in wrote:
> Can a 32 bit machine address anything more than 4 GB with just 32
> bits?
Yes.
> The register which holds the memory address are of 32 bits, isn't it?
Yes, but that register holds a virtual address. Nothing prevents more
than 2^32 bytes of *physical* memory being addressed just because
virtual addresses are limited to 32-bits.
The memory management unit takes the 32-bit virtual address, combines
it with information from the page tables, to get the physical address,
which can have more than 32-bits in it.
For example, an implementation might use 32-bit virtual addresses,
4,096 byte pages, and a 32-bit page offset in the page table. This
would allow the page table to specify a starting address from page 0
to page 2^33-1. If there's no wraparound, an all-1's address would be
2^33-1 bytes starting at page 2^33-1. That would make addresses from 0
to 2^45 (roughly). (Realistically, this would mean 2^44 bytes or
16,384GB.)
For x86's, PAE supports a physical memory space of 64GB. The limiting
factor is largely just how many physical address pins the CPU is going
to have. I think there's a trick to get support for 128GB (if you ever
had hardware to support it), not that it matters.
DS
| |
| sam_cit@yahoo.co.in 2007-04-29, 7:18 am |
| On Apr 29, 2:25 pm, David Schwartz <dav...@webmaster.com> wrote:
> On Apr 29, 1:04 am, sam_...@yahoo.co.in wrote:
>
>
> Yes.
>
>
> Yes, but that register holds a virtual address. Nothing prevents more
> than 2^32 bytes of *physical* memory being addressed just because
> virtual addresses are limited to 32-bits.
>
> The memory management unit takes the 32-bit virtual address, combines
> it with information from the page tables, to get the physical address,
> which can have more than 32-bits in it.
>
> For example, an implementation might use 32-bit virtual addresses,
> 4,096 byte pages, and a 32-bit page offset in the page table. This
> would allow the page table to specify a starting address from page 0
> to page 2^33-1. If there's no wraparound, an all-1's address would be
> 2^33-1 bytes starting at page 2^33-1. That would make addresses from 0
> to 2^45 (roughly). (Realistically, this would mean 2^44 bytes or
> 16,384GB.)
>
Sorry, i didn't get the logic of 2^33-1... :-(
Moreover, if the frame size in memory is 4096 bytes, 12 bits (0 to
4095) of the virtual address will act as the offset.
Hence we are left out with 20 MSB bits of the virtual address... hence
2^20 possible index to PMT (page map table) is possible,
lets assume that each entry in PMT is of 32 bits,hence number of
frames is 2^32 and as each frame is of size 4 kb, total memory would
be 2^36 = 64 GB,
but with 32 bit virtual address, the maximum possible entries in PMT
is 2^20.
Now how can 2^20 entries map to 2^32 possible frames? (as mapping is
one is to one)
Hope you got my question.
| |
| Logan Shaw 2007-04-29, 1:17 pm |
| sam_cit@yahoo.co.in wrote:
> Sorry, i didn't get the logic of 2^33-1... :-(
It seems like that should have been 2^32-1. Maybe there is some
x86-specific reason to say 2^33-1, I don't know. (I will ignore
x86 since I don't know much about it and just talk in general terms.)
> Moreover, if the frame size in memory is 4096 bytes, 12 bits (0 to
> 4095) of the virtual address will act as the offset.
Yes, 12 bits will act as the offset within the page. The other
20 bits will act as a virtual page number, indicating which of
the possible 2^20 virtual pages the data lies in.
When a virtual address is mapped to a physical address, this
20-bit virtual page number would then be mapped to a 32-bit
physical page number. Virtual pages and physical pages are
the same size as each other, but they are not the same thing.
Once you have a 32-bit physical page number, you still have
a 12-bit offset within the page. Putting those together, you
now have a 44-bit number that the hardware can put on an address
bus if it wants to (or access a cache, whatever).
This means that each process's virtual address space can map to
any location in the 44-bit physical address space. But it can
only have 2^32 bytes of that mapped at once[1]. However, two
processes can map different areas because the mapping is a
function of both the virtual address and the process. So,
together two processes can see more than 2^32 bytes of memory,
even though each individually can see only 2^32.
- Logan
[1] It could change the mapping over time. This actually happens
in real life when you use mmap() to read and write to files.
| |
| sam_cit@yahoo.co.in 2007-04-29, 1:17 pm |
|
>
> Yes, 12 bits will act as the offset within the page. The other
> 20 bits will act as a virtual page number, indicating which of
> the possible 2^20 virtual pages the data lies in.
>
> When a virtual address is mapped to a physical address, this
> 20-bit virtual page number would then be mapped to a 32-bit
> physical page number. Virtual pages and physical pages are
> the same size as each other, but they are not the same thing.
>
> Once you have a 32-bit physical page number, you still have
> a 12-bit offset within the page. Putting those together, you
> now have a 44-bit number that the hardware can put on an address
> bus if it wants to (or access a cache, whatever).
>
I hope, you mean as to add the 12 bit offset to the 32 bit physical
address, when you say 44-bit member.
Just like adding offset to the base address in relocation register...
| |
| sam_cit@yahoo.co.in 2007-04-29, 1:17 pm |
| On Apr 29, 9:26 pm, Logan Shaw <lshaw-use...@austin.rr.com> wrote:
> sam_...@yahoo.co.in wrote:
>
> It seems like that should have been 2^32-1. Maybe there is some
> x86-specific reason to say 2^33-1, I don't know. (I will ignore
> x86 since I don't know much about it and just talk in general terms.)
>
>
> Yes, 12 bits will act as the offset within the page. The other
> 20 bits will act as a virtual page number, indicating which of
> the possible 2^20 virtual pages the data lies in.
>
> When a virtual address is mapped to a physical address, this
> 20-bit virtual page number would then be mapped to a 32-bit
> physical page number. Virtual pages and physical pages are
> the same size as each other, but they are not the same thing.
>
> Once you have a 32-bit physical page number, you still have
> a 12-bit offset within the page. Putting those together, you
> now have a 44-bit number that the hardware can put on an address
> bus if it wants to (or access a cache, whatever).
>
> This means that each process's virtual address space can map to
> any location in the 44-bit physical address space. But it can
> only have 2^32 bytes of that mapped at once[1]. However, two
> processes can map different areas because the mapping is a
> function of both the virtual address and the process. So,
> together two processes can see more than 2^32 bytes of memory,
> even though each individually can see only 2^32.
>
Exactly, Ok so any process's address space on such a platform will be
4 GB, however
if i have physical memory more than 4 GB, the remaining of the
physical memory could still be used for the other process, assuming
that the actual address bus to and from the physical memory is able to
carry more than 32 bits at a time.
Is this the correct way to sum-up?
> - Logan
>
> [1] It could change the mapping over time. This actually happens
> in real life when you use mmap() to read and write to files.
Can you explain as to what exactly you mean by this case?
| |
| Logan Shaw 2007-04-29, 7:16 pm |
| sam_cit@yahoo.co.in wrote:
> Ok so any process's address space on such a platform will be
> 4 GB, however
> if i have physical memory more than 4 GB, the remaining of the
> physical memory could still be used for the other process, assuming
> that the actual address bus to and from the physical memory is able to
> carry more than 32 bits at a time.
>
> Is this the correct way to sum-up?
Yes, that sounds correct.
>
> Can you explain as to what exactly you mean by this case?
One way to read and write files is to use a bunch of read() and
write() calls. Another way is to open the files but then do
all the reading and writing by doing an mmap() and then reading
and writing to the mapped memory. It is not uncommon for programs
to read the whole contents of a file sequentially by doing an
mmap() of the first (say) 128K, then reading the mapped memory,
then munmap()ing that part and doing another mmap() of the next
128K, and so on.
What's more, if there is not a shortage of RAM, on most Unix(-like)
systems, those physical pages (that are used to hold the file)
will continue to be used for the file even though it is not mapped
anymore. If you use this technique to read an entire 8 GB file,
and if you have 8 GB of RAM available to "waste", the system may
use 8 GB of physical pages. You could then go back and mmap() the
same part of the same file again and get access to those same
physical pages again.
It's almost like having a secret way to access more than 4 GB of
RAM. It wouldn't be like regular addressable memory, of course.
The point is really just that a single 32-bit process can easily
have more than 4 GB of active data in RAM.
- Logan
| |
| sam_cit@yahoo.co.in 2007-04-30, 1:17 am |
| On Apr 30, 1:16 am, Logan Shaw <lshaw-use...@austin.rr.com> wrote:
> sam_...@yahoo.co.in wrote:
>
>
> Yes, that sounds correct.
>
>
>
> One way to read and write files is to use a bunch of read() and
> write() calls. Another way is to open the files but then do
> all the reading and writing by doing an mmap() and then reading
> and writing to the mapped memory. It is not uncommon for programs
> to read the whole contents of a file sequentially by doing an
> mmap() of the first (say) 128K, then reading the mapped memory,
> then munmap()ing that part and doing another mmap() of the next
> 128K, and so on.
>
Isn't one mmap() enough for the entire file? If i'm correct the
single mmap() call works, and as and when the program refers to the
file's content which is not in memory, a page fault occurs and the new
page having the new contents is loaded into physical memory and that
application need not know all this as this is very similar to paging
taken care by the operating system and a single call to mmap() and
munmap() works fine?
> What's more, if there is not a shortage of RAM, on most Unix(-like)
> systems, those physical pages (that are used to hold the file)
> will continue to be used for the file even though it is not mapped
> anymore. If you use this technique to read an entire 8 GB file,
> and if you have 8 GB of RAM available to "waste", the system may
> use 8 GB of physical pages. You could then go back and mmap() the
> same part of the same file again and get access to those same
> physical pages again.
>
> It's almost like having a secret way to access more than 4 GB of
> RAM. It wouldn't be like regular addressable memory, of course.
> The point is really just that a single 32-bit process can easily
> have more than 4 GB of active data in RAM.
>
> - Logan
Im confused here, we just concluded that a 32-bit process can't have
any address space beyond 4gb and i'm sure mmap() is based on paging
concept. So how is it possible that more than 4 GB can be used for
mmap() the file for the 32-bit process. I was thinking that mmap()
would return ENOMEM in such cases, where the file's size is more than
the logical address space of the process. Can you clarify, please?
| |
| Gordon Burditt 2007-04-30, 1:17 am |
| > Isn't one mmap() enough for the entire file? If i'm correct the
No, if the file is only a few terabytes long, it won't fit in 4GB
along with all the code and data for the program that has to be
in there.
>single mmap() call works, and as and when the program refers to the
>file's content which is not in memory, a page fault occurs and the new
>page having the new contents is loaded into physical memory and that
>application need not know all this as this is very similar to paging
>taken care by the operating system and a single call to mmap() and
>munmap() works fine?
>
>
> Im confused here, we just concluded that a 32-bit process can't have
>any address space beyond 4gb
*AT ANY ONE TIME*, yes. But if you unmap some of it, it may stick around
in RAM so if you remap it, it comes back, without ever having left RAM.
>and i'm sure mmap() is based on paging
>concept. So how is it possible that more than 4 GB can be used for
>mmap() the file for the 32-bit process.
To mmap() a multi-terabyte file, mmap() 1GB of it at a time. When
you need a different piece of it, unmap it and mmap() a different
chunk of the file.
>I was thinking that mmap()
>would return ENOMEM in such cases, where the file's size is more than
>the logical address space of the process. Can you clarify, please?
You don't have to mmap() the entire file at once, and in certain cases,
you can't.
|
|
|
|
|