`ffi-lib` fails to load an existing library

Questions & Answers
,
1

Hello, everyone!

I create an application and use The Racket Foreign Interface to load foreign libraries.

Here's part of the project after compiling c codes.

./gsl-fork/build/lib/
├── libgsl.a
├── libgslcblas.a
├── libgslcblas.la
├── libgslcblas.so -> libgslcblas.so.0.0.0
├── libgslcblas.so.0 -> libgslcblas.so.0.0.0
├── libgslcblas.so.0.0.0
├── libgsl.la
├── libgsl.so -> libgsl.so.25.1.0
├── libgsl.so.25 -> libgsl.so.25.1.0
├── libgsl.so.25.1.0
└── pkgconfig
    └── gsl.pc

And I have a library.rkt in the root directory.

#lang racket/base
(require racket/runtime-path ffi/unsafe setup/dirs (for-syntax racket/base))

(define-runtime-path lib-dir (build-path "gsl-fork" "build" "lib"))

(define (find-ffi-lib name version)
  (ffi-lib name (list version #f) #:get-lib-dirs (lambda () (cons lib-dir (get-lib-search-dirs)))))

(define libgsl (find-ffi-lib "libgsl" "25.1.0"))
(define libgslblas (find-ffi-lib "libgslcblas" "0.0.0"))

When I execute racket library.rkt, ffi-lib reports an error. It seems that the shared object is not found.

ffi-lib: could not load foreign library
  path: libgsl.so.25.1.0
  system error: libgsl.so.25.1.0: 无法打开共享对象文件: 没有那个文件或目录
  context...:
   /home/hin/racket/collects/ffi/unsafe.rkt:131:0: get-ffi-lib
   body of "/home/hin/repo/racket-gsl/library.rkt"

Maybe there's something I've never learnt about. So I run two tests in racket REPL.

> ,r ffi/unsafe
> (current-directory "gsl-fork/build/lib")
; now in /home/hin/repo/racket-gsl/gsl-fork/build/lib
> (define cblas (ffi-lib "libgslcblas.so" "0.0.0"))
> (define gsl (ffi-lib "libgsl.so" "25.1.0"))
ffi-lib: could not load foreign library
  path: libgsl.so.25.1.0
  system error: libgsl.so.25.1.0: 无法打开共享对象文件: 没有那个文件或目录
 [,bt for context]

As I expect, ffi-lib cannot find libgsl.so.25.1.0. But it does successfully load libgslcblas.so.0.0.0.

Would you please explain the situation to me? What can I do to repair this?

Yours frankly.
Antigen-11

2

It looks like it could be related to the search path for shared libraries.

  • Have you tried an absolute path for testing?

If it works, then there is some other thigs to try:

  • Does it work if you move the shared library to one of the directories listed in (get-lib-search-dirs)?

  • What's the value of lib-dir and what do you see when you run ls in the directory?

3

Yes, I have. racket library.rkt still fails when I use (define-runtime-path lib-dir (path->complete-path (build-path "gsl-fork" "build" "lib"))).

No, it doesn't work.

> ,en "library.rkt"
ffi-lib: could not load foreign library
  path: libgsl.so.25.1.0
  system error: libgsl.so.25.1.0: 无法打开共享对象文件: 没有那个文件或目录
 [,bt for context]
"library.rkt"> lib-dir
#<path:/home/hin/repo/racket-gsl/gsl-fork/build/lib>

~/repo/racket-gsl/gsl-fork/build/lib$ ls .
libgsl.a        libgslcblas.so        libgsl.la     libgsl.so.25.1.0
libgslcblas.a   libgslcblas.so.0      libgsl.so     pkgconfig
libgslcblas.la  libgslcblas.so.0.0.0  libgsl.so.25
4

Have you tried an absolute path for testing?

I meant tried

(ffi-lib absolute-path-to-shared-library)

Does it work if you move the shared library to one of the directories listed in (get-lib-search-dirs) ?
No, it doesn't work.

Can you load your shared library from another programming language, like C?

Another thing to try: Rename it something simple like foo.so without version numbers.
Will it load now? [Maybe it's not the search path that's the problem, maybe it's the version number?]

5

Two suggestions:

First, ffi-lib logs about its library searches on the ffi-lib topic. So if you start racket with, for example,

$ PLTSTDERR="debug@ffi-lib error" racket

then you'll get messages to stderr about what paths ffi-lib tried in order to find the library. Unfortunately, the most common reason for ffi-lib failing is that it finds the right file, but it is unable to load some dependency, and the OS doesn't have a portable interface for finding out why (IIRC). So the logging might not tell you much in this case.

Second, if this is on Linux, you can run the command ldd on your shared object file. It will print the library's dependencies and how they would be resolved given the OS search path. If you have unresolved dependencies, it might (?) work to load them explicitly first with ffi-lib, but I'm not sure. But that's the best way I know to diagnose a failure to load because of unresolved dependencies.

1 Like
6

Python logs an error, too.

>>> from ctypes import *
>>> lib = cdll.LoadLibrary("gsl-fork/build/lib/libgsl.so")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: gsl-fork/build/lib/libgsl.so: undefined symbol: cblas_ctrmv

There is a symbol defined in libgslcblas.so, so I need to load it globally first.

> ,r ffi/unsafe
> ;;Load libgslcblas globally
  (define libgslcblas (ffi-lib "gsl-fork/build/lib/libgslcblas" #:global? #t))
> (define libgsl (ffi-lib "gsl-fork/build/lib/libgsl"))
>

I manage to load libgsl.so now. But the error message of racket really confuses me. It is not that ffi-lib cannot find the library, but that there is an undefined symbol.

7

Thanks! I learnt a lot from your suggestions.

8

I run PLTSTDERR="debug@ffi-lib error" racket library.rkt and then get some messages.

ffi-lib: failed for (ffi-lib "libgsl" '("25.1.0" #f)), tried: 
  #<path:/home/hin/repo/racket-gsl/gsl-fork/build/lib/libgsl.so.25.1.0> (exists)
  #<path:/home/hin/repo/racket-gsl/gsl-fork/build/lib/libgsl.so> (exists)
  #<path:/home/hin/repo/racket-gsl/gsl-fork/build/lib/libgsl> (no such file)
  #<path:/home/hin/.local/share/racket/8.10/lib/libgsl.so.25.1.0> (no such file)
  #<path:/home/hin/.local/share/racket/8.10/lib/libgsl.so> (no such file)
  #<path:/home/hin/.local/share/racket/8.10/lib/libgsl> (no such file)
  #<path:/home/hin/racket/lib/libgsl.so.25.1.0> (no such file)
  #<path:/home/hin/racket/lib/libgsl.so> (no such file)
  #<path:/home/hin/racket/lib/libgsl> (no such file)
  "libgsl.so.25.1.0" (using OS library search path)
  "libgsl.so" (using OS library search path)
  "libgsl" (using OS library search path)
  #<path:/home/hin/repo/racket-gsl/libgsl.so.25.1.0> (no such file)
  #<path:/home/hin/repo/racket-gsl/libgsl.so> (no such file)
  #<path:/home/hin/repo/racket-gsl/libgsl> (no such file)

ffi-lib does find the library.

And according to the documentation, this is expected behaviour.

A library file may exist but fail to load for some reason; the eventual error message will unfortunately name the fallback from the second or third bullet, since some operating systems offer no way to determine why a given library path failed.