|
Home > Archive > Debian Developers > August 2004 > Can't find a library that's right there
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Can't find a library that's right there
|
|
| Ross Boylan 2004-08-22, 6:11 pm |
| I have a library on my system, yet I'm having trouble accessing it:
$ ls -l /usr/lib/R/bin/libR.so
-rw-r--r-- 1 root root 1638320 Jun 22 05:01 /usr/lib/R/bin/libR.so
Here is my link step, and subsequent failures, even after setting
LD_LIBRARY_PATH:
g++ -o test1 mspathCEntry_test.o Manager_test.o TestManager.o
PathGenerator_test.o TestRecorder.o SimpleRecorder_test.o
TimeStepsGenerator_test.o StateTimeClassifier_test.o Environment_test.o
SuccessorGenerator_test.o Model_test.o ConstrainedCoefficients_test.o
Path_test.o Data_test.o TimeSteps_test.o TimePoint_test.o basic_test.o
.../mspath.so -L/usr/lib/debug -lboost_unit_test_framework \
-L/usr/lib/R/bin -lR
../test1 --report_level=detailed
../test1: error while loading shared libraries: libR.so: cannot open
shared object file: No such file or directory
make: *** [runtest] Error 127
My test program is in the build directory; mspath.so is one directory
up, and mspath.so references libR.so.
ross@iron:~/peter/R/mspath/src/test$ ldd test1
../mspath.so => ../mspath.so (0x40018000)
libboost_unit_test_framework-gcc-mt-1_31.so.1.31.0 =>
/usr/lib/libboost_unit_test_framework-gcc-mt-1_31.so.1.31.0 (0x40047000)
libR.so => not found
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x40068000)
libm.so.6 => /lib/libm.so.6 (0x40122000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x40144000)
libc.so.6 => /lib/libc.so.6 (0x4014d000)
libR.so => not found
librt.so.1 => /lib/librt.so.1 (0x40280000)
libpthread.so.0 => /lib/libpthread.so.0 (0x40293000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
ross@iron:~/peter/R/mspath/src/test$ echo $LD_LIBRARY_PATH
ross@iron:~/peter/R/mspath/src/test$ LD_LIBRARY_PATH=/usr/lib/R/bin
ross@iron:~/peter/R/mspath/src/test$ !ldd
ldd test1
../mspath.so => ../mspath.so (0x40018000)
libboost_unit_test_framework-gcc-mt-1_31.so.1.31.0 =>
/usr/lib/libboost_unit_test_framework-gcc-mt-1_31.so.1.31.0 (0x40047000)
libR.so => not found
etc.
Here's how I made mspath.so:
g++ -I/usr/lib/R/include -mieee-fp -I . -fPIC -g -O2 -c mspath.cc -o
mspath.o
g++ -shared -o mspath.so ConstrainedCoefficients.o Data.o Manager.o
Model.o Path.o PathGenerator.o SimpleRecorder.o StateTimeClassifier.o
SuccessorGenerator.o TestManager.o TestRecorder.o TimeStepsGenerator.o
mspath.o -L/usr/lib/R/bin -lR
The .o files in my test program are compiled like
g++ -O2 -I .. -c -o Path_test.o Path_test.cc
They don't, for example, have -fPIC, but they aren't destined for a
shared library. Still, I wonder if the "not found" is a misleading
error message for "incompatible formats".
I'd appreciate any ideas about what's going on. If you could cc me that
would be great, as I'm not subscribed.
I hope this isn't too off-topic; it seemed to make more sense here than
debian-user (it's a Debian system).
Thanks.
--
Ross Boylan wk: (415) 502-4031
530 Parnassus Avenue (Library) rm 115-4 ross@biostat.ucsf.edu
Dept of Epidemiology and Biostatistics fax: (415) 476-9856
University of California, San Francisco
San Francisco, CA 94143-0840 hm: (415) 550-1062
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Josselin Mouette 2004-08-22, 6:11 pm |
| Le vendredi 20 août 2004 Ã_ 18:11 -0700, Ross Boylan a écrit :
> I have a library on my system, yet I'm having trouble accessing it:
> $ ls -l /usr/lib/R/bin/libR.so
> -rw-r--r-- 1 root root 1638320 Jun 22 05:01 /usr/lib/R/bin/libR.so
>
> Here is my link step, and subsequent failures, even after setting
> LD_LIBRARY_PATH:
>
> g++ -o test1 mspathCEntry_test.o Manager_test.o TestManager.o
> PathGenerator_test.o TestRecorder.o SimpleRecorder_test.o
> TimeStepsGenerator_test.o StateTimeClassifier_test.o Environment_test.o
> SuccessorGenerator_test.o Model_test.o ConstrainedCoefficients_test.o
> Path_test.o Data_test.o TimeSteps_test.o TimePoint_test.o basic_test.o
> ../mspath.so -L/usr/lib/debug -lboost_unit_test_framework \
> -L/usr/lib/R/bin -lR
> ./test1 --report_level=detailed
> ./test1: error while loading shared libraries: libR.so: cannot open
> shared object file: No such file or directory
> make: *** [runtest] Error 127
For compilation, LD_LIBRARY_PATH is useless. You should set
LIBRARY_PATH, or use a -L flag.
--
.''`. Josselin Mouette /\./\
: :' : josselin.mouette@ens-lyon.org
`. `' joss@debian.org
`- Debian GNU/Linux -- The power of freedom
| |
| Goswin von Brederlow 2004-08-22, 6:11 pm |
| Josselin Mouette <joss@debian.org> writes:
> Le vendredi 20 août 2004 Ã_ 18:11 -0700, Ross Boylan a écrit :
>
> For compilation, LD_LIBRARY_PATH is useless. You should set
> LIBRARY_PATH, or use a -L flag.
He isn't compiling but running the binary.
What is your LD_LIBRARY_PATH set to and when? Is it maybe cleared by
fakeroot or something? What does "ldd test1" say?
For libraries in non standard dirs you might have to set rpath.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Kevin B. McCarty 2004-08-22, 6:11 pm |
| > g++ -o test1 mspathCEntry_test.o Manager_test.o TestManager.o
> PathGenerator_test.o TestRecorder.o SimpleRecorder_test.o
> TimeStepsGenerator_test.o StateTimeClassifier_test.o Environment_test.o
> SuccessorGenerator_test.o Model_test.o ConstrainedCoefficients_test.o
> Path_test.o Data_test.o TimeSteps_test.o TimePoint_test.o basic_test.o
> ../mspath.so -L/usr/lib/debug -lboost_unit_test_framework \
> -L/usr/lib/R/bin -lR
Since you say later that mspath.so depends upon libR.so, I would put the
-L/usr/lib/R/bin before ../mspath.so in this command. If that still
doesn't help, I'm not sure what causes the problem you describe, but try
this as a workaround:
- add /usr/lib/R/bin (on a separate line) to /etc/ld.so.conf
- run /sbin/ldconfig as root
Does that help any? What does "file /usr/lib/R/bin/libR.so" say?
--
Kevin B. McCarty <kmccarty@princeton.edu> Physics Department
WWW: http://www.princeton.edu/~kmccarty/ Princeton University
GPG public key ID: 4F83C751 Princeton, NJ 08544
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Ross Boylan 2004-08-22, 6:11 pm |
| I've found several solutions, and have a theory of the problem. If
anyone could comment on the theory, that would be great.
SOLUTIONS
The best solution seems to be to include
-Wl,-rpath,/usr/lib/R/bin
as an option to gcc.
Setting LD_LIBRARY_PATH also worked, when I remembered to export
it. (duh)
Editing ld.so.conf also worked.
Only the first meets my needs, since I want to redistribute the work
(as part of an R package), and it needs to work without extra
environment tweaking.
THEORY
I'm still puzzled the the -L/usr/lib/R/bin didn't work. My theory is
that the linker simply sticks in a reference to mspath.so, but does
not attempt to resolve its dependencies. That only happens at
run-time, at which point -L (but not -runpath) is lost. Does that
sound right?
RESPONSES TO EARLIER POSTS
I wasn't sure that LD_LIBRARY_PATH affects links. However the link
command line included -L. Further, in the examples further down in my
original post I ran the program both with my default LD_LIBRARY_PATH
(empty) and with /usr/lib/R/bin in it (but only set locally). As the
examples showed, in neither case was the program able to find libR.so,
nor could ldd.
By the way, as shown in the original post, mspath.so also had
-L/usr/lib/R/bin -lR in its link command.
On Sat, Aug 21, 2004 at 12:08:25PM -0400, Kevin B. McCarty wrote:
>
> Since you say later that mspath.so depends upon libR.so, I would put the
> -L/usr/lib/R/bin before ../mspath.so in this command. If that still
> doesn't help, I'm not sure what causes the problem you describe, but try
> this as a workaround:
That didn't help, though I moved the -lR as well. I think libraries
usually come last in the link line, so any references are picked up.
>
> - add /usr/lib/R/bin (on a separate line) to /etc/ld.so.conf
> - run /sbin/ldconfig as root
Yes, that works. (Well, I get an error about R Home directory not
defined, but that's a separate issue. Further, that error didn't
happen with -runpath).
>
> Does that help any? What does "file /usr/lib/R/bin/libR.so" say?
$ file /usr/lib/R/bin/libR.so
/usr/lib/R/bin/libR.so: ELF 32-bit LSB shared object, Intel 80386,
version 1 (SYSV), stripped
also
$ file test1
test1: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for
GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
Thanks to everyone for their help.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Goswin von Brederlow 2004-08-22, 6:11 pm |
| Ross Boylan <ross@biostat.ucsf.edu> writes:
> I've found several solutions, and have a theory of the problem. If
> anyone could comment on the theory, that would be great.
>
> SOLUTIONS
> The best solution seems to be to include
> -Wl,-rpath,/usr/lib/R/bin
> as an option to gcc.
That has all theproblems associated with rpath and is only valid
within a package. If you use an rpath to link in another package
library you get problems on future updates and version skews.
> Setting LD_LIBRARY_PATH also worked, when I remembered to export
> it. (duh)
This can be done by a wraper script and avoids most problems caused by
rapth. Worst case you can change the wraper script to fix bugs.
But both of those realy are only appropriate for within a single
package. If you (R package exporting a library) have libraries that
are ment to be linked to by other they should follow the FSH and be in
a public place (/usr/lib/). Then you (package linking against R libs)
don't have this problem.
> Editing ld.so.conf also worked.
I think that is a big no-no.
> Only the first meets my needs, since I want to redistribute the work
> (as part of an R package), and it needs to work without extra
> environment tweaking.
>
> THEORY
> I'm still puzzled the the -L/usr/lib/R/bin didn't work. My theory is
> that the linker simply sticks in a reference to mspath.so, but does
> not attempt to resolve its dependencies. That only happens at
> run-time, at which point -L (but not -runpath) is lost. Does that
> sound right?
-L only tells the compile time linker where to look. Without rpath it
is assumed the runtime libarry will be in the search path later. With
rpath you hardcode the full path into the binary and no runtime
searching is done.
> RESPONSES TO EARLIER POSTS
>
> I wasn't sure that LD_LIBRARY_PATH affects links. However the link
> command line included -L. Further, in the examples further down in my
> original post I ran the program both with my default LD_LIBRARY_PATH
> (empty) and with /usr/lib/R/bin in it (but only set locally). As the
> examples showed, in neither case was the program able to find libR.so,
> nor could ldd.
>
> By the way, as shown in the original post, mspath.so also had
> -L/usr/lib/R/bin -lR in its link command.
As it should if the lib is there.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Dirk Eddelbuettel 2004-08-22, 6:11 pm |
|
Hi Ross
I found the thread by accident when browsing a web archive of
debian-devel, I'm no longer subscribed here as the volume is too high
for my available time. Feel free to bug me in private on this too
(given that I'm the R maintainer, that is).
On Sat, Aug 21, 2004 at 10:10:07PM +0200, Goswin von Brederlow wrote:
> Ross Boylan <ross@biostat.ucsf.edu> writes:
>
>
> That has all theproblems associated with rpath and is only valid
> within a package. If you use an rpath to link in another package
> library you get problems on future updates and version skews.
Still, I'd recommend this for a _standalone_ binary.
But you said earlier in the thread that you wanted to distribute this as
part of an R package -- in which case things are different because ...
>
>
> This can be done by a wraper script and avoids most problems caused by
> rapth. Worst case you can change the wraper script to fix bugs.
.... /usr/bin/R is _exactly_ the kind of wrapper script Goswin talks
about here.
E.g., witness ldd on the .so of my RQuantLib packages:
dd@homebud:~> ldd /usr/lib/R/site-library/RQuantLib/libs/RQuantLib.so
libQuantLib-0.3.7.so => /usr/lib/libQuantLib-0.3.7.so (0x400bb000)
libR.so => not found
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x404b5000)
libm.so.6 => /lib/libm.so.6 (0x4056f000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x40591000)
libc.so.6 => /lib/libc.so.6 (0x4059a000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
edd@homebud:~>
The morale here is that iff
a) you want to build a regular, CRAN-style R package that will get
loaded by R using library() or require(), then the normal manual
'Writing R Extensions' fromin the various r-doc-* packages applies,
and everything just works if you build the package using
R CMD build .... as described --- this will set up the linker
calls for you etc pp
b) you want to embed R into another application, things are a little
different but I am not sure if that is the case here.
> But both of those realy are only appropriate for within a single
> package. If you (R package exporting a library) have libraries that
> are ment to be linked to by other they should follow the FSH and be in
> a public place (/usr/lib/). Then you (package linking against R libs)
> don't have this problem.
>
>
> I think that is a big no-no.
FWIW I fully agree with the 'no-no' view. There is one ugly-ish fudge
that also works: Given that /usr/bin/R sets LD_LIBRARY_PATH for you,
/usr/lib/R/bin will in fact searched. So for one package which expects
to link R with another private set of .so libs not otherwise know to ldd
(via ld.so.conf or other means), I simply softlinked from their
/usr/lib/$foo/lib$foo.so into /usr/lib/R/bin/lib$foo.so.
[vbcol=seagreen]
I think you'd be well advised to follow a) above. Bug me is something is
unclear; otherwise follow any of those over 350 CRAN packages that
contain C/C++ code to see how they do. It really is taken care of by R
itself thanks to its build mechanism.
Dirk
--
Three out of two people have difficulties with fractions.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Ross Boylan 2004-08-22, 8:48 pm |
| On Sun, Aug 22, 2004 at 08:41:42AM -0500, Dirk Eddelbuettel wrote:
>
> Hi Ross
>
> I found the thread by accident when browsing a web archive of
> debian-devel, I'm no longer subscribed here as the volume is too high
> for my available time. Feel free to bug me in private on this too
> (given that I'm the R maintainer, that is).
Thanks, Dirk. My initial puzzle was about how the linker was working,
so didn't seem R-specific. By now it's getting a bit more R-specific.
For everyone's benefit, here's the full context. I have avoided
stating it because it's a bit involved:
I am making an R package, which is a different concept from a Debian
package. The source of an R package consists of a directory tree. R
can check this in place, or build it into a tarball. A user then
takes the tarball and runs an R command which builds binaries (if
any--I have them) from it, digests the other files, and puts the
results in a place R can find them. One can then use the package from
R.
The original source includes a src/ directory, containing code to be
compiled. I have a src/test/ directory, which exercises the code in
src/. This is different from the tests/ top level directory for
exercising R code.
My current setup is that I run R CMD check, which builds a .so file
out of the material in src/ (not including the subdirectory). This
..so file is linked to (references?) libR.so.
I want to 'cd src/test/' and then 'make' to run the tests there.
So when I redistribute this, I want people to be able to do the same:
change into the src/test directory, make, and see if the tests pass.
As far as I can tell, R has no built-in way to exercise the src/
code.* The test is intended to be run at installation, rather than
being run from R in standard use of the package.
*(The top level tests/ directory can exercise the whole system from R,
but my tests are unit tests on individual classes in the C++ source.)
The problem arose because the test program, test1, couldn't find
libR.so. It needs to, because the tests exercise the code in
src/libmspath.so, and the latter links to libR. On Debian, at least,
R runs from a wrapper which sets LD_LIBRARY_PATH appropriately, but
since I'm not running R I don't get the benefit of this. Further,
recipients of the package aren't necessarily using Debian.
Since anyone interested in this will be building it on their own
system, and since the test is just a one shot thing, it seems to me
using -rpath $(shell R RHOME)/bin is OK. (Of course, they are out of
luck without GNU make).
>
> On Sat, Aug 21, 2004 at 10:10:07PM +0200, Goswin von Brederlow wrote:
I'm not sure what problems with rpath you are referring to. And what
do you mean by "only valid within a package." First, is that in the
"R package" or "Debian package" sense? Second, I just don't see the
problem. Say that libR got updated, which is an action outside of my
package. The path to it doesn't change, so I think my approach would
still work. Perhaps because my use, as noted above, is a one-shot
test after building, I don't think the problems with updates or
version skews apply anyway.
[vbcol=seagreen]
>
> Still, I'd recommend this for a _standalone_ binary.
>
> But you said earlier in the thread that you wanted to distribute this as
> part of an R package -- in which case things are different because ...
>
This is running from a makefile, which ideally requires no special
setup. I could write it to invoke the test program via a wrapper
script, and put the path in the wrapper, but that seems just an
additional complexity here.
[vbcol=seagreen]
>
> ... /usr/bin/R is _exactly_ the kind of wrapper script Goswin talks
> about here.
The problem is that the test is intended to run outside of R, even
though the package itself is destined for use within R.
>
> E.g., witness ldd on the .so of my RQuantLib packages:
>
> dd@homebud:~> ldd /usr/lib/R/site-library/RQuantLib/libs/RQuantLib.so
> libQuantLib-0.3.7.so => /usr/lib/libQuantLib-0.3.7.so (0x400bb000)
> libR.so => not found
Yep, that's exactly the problem I was getting with the mspath.so file
I built.
> libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x404b5000)
> libm.so.6 => /lib/libm.so.6 (0x4056f000)
> libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x40591000)
> libc.so.6 => /lib/libc.so.6 (0x4059a000)
> /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x80000000)
> edd@homebud:~>
>
> The morale here is that iff
>
> a) you want to build a regular, CRAN-style R package that will get
> loaded by R using library() or require(), then the normal manual
> 'Writing R Extensions' fromin the various r-doc-* packages applies,
> and everything just works if you build the package using
> R CMD build .... as described --- this will set up the linker
> calls for you etc pp
>
> b) you want to embed R into another application, things are a little
> different but I am not sure if that is the case here.
>
The problem arose from case c), which is a) + wanting to run a test
outside of R. I am not embedding R in my application (the test
program), though I am using part of the R library.
The problem I had was not where my library went, but how to find the
standard libR.so. It's in an appropriate place by the FSH, but it is
not found without some help.
[vbcol=seagreen]
>
Agreed, but it was an interesting proof at one point the library could
be found.
> FWIW I fully agree with the 'no-no' view. There is one ugly-ish fudge
> that also works: Given that /usr/bin/R sets LD_LIBRARY_PATH for you,
> /usr/lib/R/bin will in fact searched. So for one package which expects
> to link R with another private set of .so libs not otherwise know to ldd
> (via ld.so.conf or other means), I simply softlinked from their
> /usr/lib/$foo/lib$foo.so into /usr/lib/R/bin/lib$foo.so.
Again, I'm not running from R. But does this mean that my little
trick for finding libR ( $(shell R RHOME)/bin) is not going to work
outside of Debian systems?
>
>
> I think you'd be well advised to follow a) above. Bug me is something is
> unclear; otherwise follow any of those over 350 CRAN packages that
> contain C/C++ code to see how they do. It really is taken care of by R
> itself thanks to its build mechanism.
>
> Dirk
>
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
|
|
|
|
|