Hi!
I am building a server using many legacy libraries (shared
objects).I have used LD_DEBUG=reloc,symbols,bindings,detail server and
tried to understand how the loader setups process address space.I
observed that even before actual references to functions are made,
linker resolves the symbols. I referred to SUN manual and understood
that loader does this as part of relocation processing. But I am not
still clear about this. SUN is supposed to do lazy binding for
function calls then why should it try to resolve any symbols before
they get called. I observed that the function symbols that get
resolved during relocation processing are of type:
R_SPARC_GLOB_DAT,R_SPARC_WDISP30 (and these happen to the references
(UNDEFINED) in the file whose relocation processing tries to resolve
these symbols and load libraries that define these symbols). I read
SUN manual for description but still its unclear.Does this not
contradict what we expect on SUN's, lazybinding? This effectively
results in memory space explosion and my server crashes even before
entering main(). Any help/ideas/suggestion wuld be really appreciated.
Thanks a lot,
Sunil.

[ Post a follow-up to this message ]

sunilsreenivas2001@yahoo.com (sunil) writes:
quote:

> SUN is supposed to do lazy binding for
> function calls then why should it try to resolve any symbols before
> they get called.


Well, for one thing, *data* symbols can't be resolved in lazy
fashion, only functions can be. And only from PIC code.

To understand how lazy binding works, read the "Linkers an
Loaders" by John Levine. A draft of it is available here:
http://www.iecc.com/linker
quote:

> I observed that the function symbols that get
> resolved during relocation processing are of type:
> R_SPARC_GLOB_DAT,


That's data ...
quote:

> R_SPARC_WDISP30 (and these happen to the references


That's non-PIC code having unresolved function calls. Only
R_SPARC_JMP_SLOT relocations can be lazy-resolved.
quote:

> (UNDEFINED) in the file whose relocation processing tries to resolve
> these symbols and load libraries that define these symbols). I read
> SUN manual for description but still its unclear.Does this not
> contradict what we expect on SUN's, lazybinding?


Nope, doesn't contradict anything.
quote:

> This effectively
> results in memory space explosion and my server crashes even before
> entering main().


What makes you think it is "memory explosion" that causes your crash?
And what makes you think that lazy-relocations will help memory
usage in any way? [I don't believe they would, unless you have a
lot of processes all using these shared libraries, and your server
dies due to a global system memory starvation, rather than idividual
process growing too big.]
quote:

> Any help/ideas/suggestion wuld be really appreciated.


If you really want lazy relocations, be sure your libraries are
built with PIC (position-independent code). Properly-built shared
libraries should not have any R_SPARC_WDISP30 relocations.

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

[ Post a follow-up to this message ]

Thanks a lot for your reply.
Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> wrote in message news:<m3oet3geor.fsf@salmon.parasof
t.com>...
quote:

> sunilsreenivas2001@yahoo.com (sunil) writes:
> R_SPARC_GLOB_DAT,
>
> That's data ...

Its actually a function pointer passed around and so linker puts it in
GOT (absolute address). Hence even though its function, its resolved
before main.
quote:

> 
>
> That's non-PIC code having unresolved function calls. Only
> R_SPARC_JMP_SLOT relocations can be lazy-resolved.

Yeah due to error in my process some code was compiled without KPIC.
quote:

> What makes you think it is "memory explosion" that causes your crash?
> And what makes you think that lazy-relocations will help memory
> usage in any way? [I don't believe they would, unless you have a
> lot of processes all using these shared libraries, and your server
> dies due to a global system memory starvation, rather than idividual
> process growing too big.]

Becoz I link many libraries of which, only some may be used. (There
are reasons for doing this). I use filters concept, which with
lazybinding does optimize run time memory usage (since some libs may
not be called for some exec. path).
I have one more issue:
I am having same glob. var. defind with multiple sizes across diff.
.so's.If we
have two objects or archives that define same variable as int a[2],
char a[16], ld does link them and reports that symbol a is resized to
maximum size of 16.
Now lets consider the case with shared objs. I have a.o, b.so, c.so
and we define a (global var) as int a[2],char a[16],char a[18] in
these three files. In this case, ld reports that it is taking
definition from a.o and resizes it to size 16, but **doesn't** resize
it to 18.(But this effectively useless if we only have .so's and no
object.)I guess only first .so has effect on final size. Now I am
having this kind of situation. I have libraries a.so,b.so,c.so that
define same symbol with different sizes. In this case the definition
that is ultimately used to resolve this symbol by runtime loader
depends on order of loading of these libraries. In default model, it
tries to resolve the symbol by going through executable and .so's in
the order they were loaded (assuming -Bstatic is not being used). This
is esentially order in which libraries appear on the linkline assuming
we have no lazyloading. So all references to var. "a" at run time are
resolved to same .so. However problem occurs if we call code in
diffrent .so that expects it to be of different size. Only solution
appears to be to generate an object file that has all global symbols
from all .so's we are using resized to maximum size and linking this
obj. file directly into executable. This prevents before mentioned
problems. But now we have issue of runtime memory usage. Even though
from point of view of correctness this works fine, all individual
.so's do have their own copy of these global vars (which are
essentially unused). This might not be an issue in general but in my
case it does become an issue because there are so many glob. vars,
common areas(BSS vars) that runtime memory footprint of executable
that has obj. file(having all global vars resized to max size)
linked in into it occupies 1.3GB of memory (since I put in all glob.
var defs from all libs) and added to this, we have all .so's data
sections allocating memory for the same variables. This is exhausting
my runtime memory space. I have no control over libs (repair them).I
am clueless how to solve this issue. Is it possible to load only text
section for a .so?(a stupid question). Any ideas will be of GREAT
help.
Thanks a LOT,
Sunil.

[ Post a follow-up to this message ]

sunilsreenivas2001@yahoo.com (sunil) writes:
quote:
 
> Its actually a function pointer passed around and so linker puts it in


Function pointer *is* data.
quote:

> I have libraries a.so,b.so,c.so that
> define same symbol with different sizes. In this case the definition
> that is ultimately used to resolve this symbol by runtime loader
> depends on order of loading of these libraries.


Correct, provided you load them with "dlopen(..., RTLD_GLOBAL)"
quote:

> Only solution
> appears to be to generate an object file that has all global symbols
> from all .so's we are using resized to maximum size and linking this
> obj. file directly into executable.


The big questions are:
- are the variables 'a' in a.so, b.so and c.so intended to be
private to their libraries, or shared between them?
- if they are intended to be shared (which I suspect they aren't),
then *why* do they have different size?
quote:

> I have no control over libs (repair them).


But you do! You probably can't rebuild them, but you can still
modify their symbol tables.
quote:

> am clueless how to solve this issue.


If in fact the libraries did not intend to "share" their variables,
then you can load them without RTLD_GLOBAL, and they'll each operate
in their own "private" environment. Or you can "trash" variable 'a'
dynamic symbol table entries in them.
quote:

> Is it possible to load only text section for a .so?


No, but you could build a library with the '.data*' section(s)
stripped, and in effect achieve the same thing (man objcopy).
However, there are all kinds of other data in there besides globals,
and you are quite unlikely to end up with something that doesn't
crash ;-(

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

[ Post a follow-up to this message ]

Thanks for your prompt reply.
Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> wrote in message news:<m3zncklhjg.fsf@salmon.parasof
t.com>...
quote:

> sunilsreenivas2001@yahoo.com (sunil) writes:
> The big questions are:
> - are the variables 'a' in a.so, b.so and c.so intended to be
>   private to their libraries, or shared between them?
> - if they are intended to be shared (which I suspect they aren't),
>   then *why* do they have different size?

Yes some of them are shared variables as we want specific addresses
and alignment for them (using bss directives to align them at spec.
addresses in memory). Unfortunately thats how people have written
their libraries, without any standards/practices. So let me put it
this way few variables need to be shared. So lets say I resize them
and put them at specific address. For other variables, lets say each
uses its local copy. People shouldn't be stupid enough to declare a
glob. var with some size in their library and expect it to be of
different size (as defined globally in another library). Now is there
any way by which I can do this at link time? (-Bstatic... But this can
bind *all* data ref to local definitions. I want this to happen to
*some*. May be I should use mapfile?)
Bwn, when dealing with archives, say I define same symbol with two
sizes in a .o and .a. Now ld says it resizes it but finally we have
only one mem. alloc done at runtime by loader right?(unlike .so where
if two .so's define they have their own copy at runtime)
quote:

> 
>
> But you do! You probably can't rebuild them, but you can still
> modify their symbol tables.
> 
>
> If in fact the libraries did not intend to "share" their variables,
> then you can load them without RTLD_GLOBAL, and they'll each operate
> in their own "private" environment. Or you can "trash" variable 'a'
> dynamic symbol table entries in them.

When you are saying load without RTLD_GLOBAL, are you not assuming I
am using dlsymopen (which is not the case with me). What do you mean
by trashing variable "a" dynamic symbol table entries?
quote:

> 
>
> No, but you could build a library with the '.data*' section(s)
> stripped, and in effect achieve the same thing (man objcopy).
> However, there are all kinds of other data in there besides globals,
> and you are quite unlikely to end up with something that doesn't
> crash ;-(

Yes I think its  A VERY BIG AND UNNECESSARY RISK
quote:

>
> Cheers,

Thanks a LOT AGAIN for prompt reply,
Sunil.

[ Post a follow-up to this message ]

sunilsreenivas2001@yahoo.com (sunil) writes:
quote:

> Now is there any way by which I can do this at link time?


I thought you said you can't rebuild your shared libraries?
If you can, the mapfile will certainly allow you to share some
variables, and hide the rest.
quote:

> Bwn, when dealing with archives, say I define same symbol with two
> sizes in a .o and .a. Now ld says it resizes it but finally we have
> only one mem. alloc done at runtime by loader right?


Probably.
quote:

> When you are saying load without RTLD_GLOBAL, are you not assuming I
> am using dlsymopen


I am, since that's the only way (I can think of) to have 'different
library load order' you mentioned. If your exe doesn't do dlopen()s,
then it will always load shared libraries in the same order. [But
perhaps you meant that different executables load libraries in
different order.]
quote:

> (which is not the case with me). What do you mean
> by trashing variable "a" dynamic symbol table entries?


You can go into the .dynstr section of the DSO (dynamic shared
object), and you'll find the name of all variables (and functions),
exported from it.

If you overwrite such name with a different string (of the same
length), you'll effectively hide that name from the dynamic linker [*].

* Since you apparently can relink your DSOs, mapfile is much easier
option to achieve the same result.

* Emacs makes for nice 'binary editor'; try 'Meta-X hexl-mode'

* Note that this operation will hide, not rename, the symbol.
This is because there is also a hash table in the DSO, and the
renamed symbol will not be "moved" to its correct hash chain.

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

[ Post a follow-up to this message ]

Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> wrote in message news:<m3k73om
m2c.fsf@salmon.parasoft.com>...
quote:

> I am, since that's the only way (I can think of) to have 'different
> library load order' you mentioned. If your exe doesn't do dlopen()s,
> then it will always load shared libraries in the same order. [But
> perhaps you meant that different executables load libraries in
> different order.]

Unless we are using lazyloading or filters which can change order of
loading of libs dep. on execution path.
quote:

> * Since you apparently can relink your DSOs, mapfile is much easier
>   option to achieve the same result.
>

Yes I tried it. As I mentioned in last post, I want to "remove" all
the global symbols from all the libraries and put them in an obj. file
that I will be linking at runtime into my server. I want to remove all
the variables as I want to see change in var in one .so to reflect in
another .so , hence all need to use same variable. I tried this with
mapfile option: eliminate: list of glob vars to remove for a .so. This
did work fine and these symbols got removed from symbol table for that
.so. But when I ran the program and used "pmap", I am seeing that
memory is allocated for them. Why is this happening? Why is it
allocating mem. for vars I removed. One more intersting observation I
made was that what pmap is showing as mem. usage for this par. .so is
different from what I see with LD_DEBUG=files,detail. What the latter
reports is atleast twice of what pmap reports. What am I missing
here?
Thanks,
Sunil.
quote:

> Cheers,

[ Post a follow-up to this message ]

sunilsreenivas2001@yahoo.com (sunil) writes:
quote:

> I want to remove all
> the variables as I want to see change in var in one .so to reflect in
> another .so ,


You seem to be going in circles: you said that some small number of
variables need to be shared, and the rest should be kept private,
now you want all of them to be shared. You said that you have no
control over these libraries, but then that you can rebuild them.

Why don't you try to describe consicely what is it that you are
*really* trying to do, and why? Try to describe the situation
in terms of "here is what I have", "here is what I can change",
"here is what I can't change", "here is what doesn't currently work".

Then someone may be able to offer a solution.

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

[ Post a follow-up to this message ]

Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> wrote in message news:<m33cab2rul.fsf@salmon.parasof
t.com>...
quote:

> sunilsreenivas2001@yahoo.com (sunil) writes:
>

quote:

> You seem to be going in circles: you said that some small number of
> variables need to be shared, and the rest should be kept private,
> now you want all of them to be shared. You said that you have no
> control over these libraries, but then that you can rebuild them.
>
> Why don't you try to describe consicely what is it that you are
> *really* trying to do, and why? Try to describe the situation
> in terms of "here is what I have", "here is what I can change",
> "here is what I can't change", "here is what doesn't currently work".
>

Sorry for the confusion. The situation is like this: We were using
archive libraries until sometime back and now plan on using shared
objects. These libraries have the case where same variable is defined
in more than one library with different sizes. This was not an issue
when we were using archive libraries, since it resizes the variable
and we have only *one* memory allocation done for it and all use the
same variable.
We want to have similar behavior with shared objects. There are two
issues here:
All the shared objects loaded will have their own copy of this
variable. If the code from all these shared objects refer to this
variable, they will always refer to same memory location (runtime
loader default symbol resolution) but this might not be of maximum
size. Thus if some shared objects code which expects it to be of
higher size writes to this variable, we are in trouble. We want
everything to refer to same variable since only then change made by
one is seen by others.So it is inevitable to create a C file that has
declaration for all the glob. vars resized to max. size and link this
into executable. But we still have an issue with this solution. If we
have case where we have multiple defns. with different sizes with
these var. being initialized, this solution needs to be modified so
that we initialize them correctly. Is this possible?
Second issue is that every shared object now has its own memory copy
of variable which is unused and we don't want this since it wastes
memory. So what I tried was to use a mapfile when building my shared
object and remove these global vars from them using eliminate
directive. This did remove it from symbol table(used nm -x to verify
this) but still at runtime memory is allocated for variable.(Verified
with pmap command). So how can I acheive this? Hope this is clear.
Thanks a lot,
Sunil.

[ Post a follow-up to this message ]

sunilsreenivas2001@yahoo.com (sunil) writes:

Hmm, nobody offered any advice ;-(
I'll try once more.
quote:

> The situation is like this: We were using
> archive libraries until sometime back and now plan on using shared
> objects. These libraries have the case where same variable is defined
> in more than one library with different sizes. This was not an issue
> when we were using archive libraries, since it resizes the variable
> and we have only *one* memory allocation done for it and all use the
> same variable.


So you were depending on totally undefined behaviour (multiple
incompatible global definitions), and this just happened to "work"
in your environment. But now you are changing that environment,
and the undefined behaviour "doesn't work" anymore.

Perhaps it's time to remove your dependence on undefined behaviour
and *fix* your code?
quote:

> So it is inevitable to create a C file that has
> declaration for all the glob. vars resized to max. size and link this
> into executable.


Nothing is inevitable, but death and taxes.

You are trying to invent a complicated band-aid, instead of fixing
the real problem: badly-structured code.

The proper fix is to decide which data belongs to which library,
and make it so that each datum is defined in exactly one library
(with proper size), and if any other library needs it, it must link
to the "owner" library.

If it is impossible to determine what size the datum should be
until runtime (or until all libraries that need it are loaded),
then allocate that datum dynamically.

Problem solved ;-)

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

[ Post a follow-up to this message ]