# Shared libraries that depend on each other?

**URL:** <https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037>\
**Category:** General\
**Created:** [May 27, 2022, 5:30pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037 "2022-05-27T17:30:37Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [May 27, 2022, 5:30pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/1 "2022-05-27T17:30:37Z")

</div>

WARNING: this question might very well have not a lot to do with Racket. We'll see.

I'm working with a student, Alex MacLean, who's developing racket bindings for OpenFST. You can see the current version on the package manager, actually. Since this is a C++ library, name-mangling make straightforward use of the shared library ... difficult? ... Alex has chosen to create a separate library that re-exports these bindings with simpler names. If these are both shared libraries, then it's not clear how to "tell one of them where the other one is". This is in fact a very general question about shared libraries in Linux (and probably on all other platforms as well): is there a standard way for dynamic libraries to depend on each other? Or is the standard approach to re-link the shared library into a new, larger shared library? Or is there some other solution entirely?

---

<div class="post-metadata">

**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\
**Post date:** [May 27, 2022, 7:06pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/2 "2022-05-27T19:06:54Z")

</div>

I am emphatically not an expert—and I know especially little about C++—but I can try to give a few pointers to things as I understand them, and maybe someone will explain everything I'm wrong about.

One way for dynamic libraries to depend on each other is for the client code to `dlopen` some library and access its exports using `dlsym`. That's basically what `ffi-lib` and `get-ffi-obj` do. My impression is that it's not really feasible to `dlopen` C++ code (given name mangling, classes, templates, etc.), which is why wrapper libraries exist: they provide an `extern "C"` interface, so Racket (or anyone) can `dlopen` the wrapper, and all the C++ complexity happens where it can be managed by the C++ compiler.

So, if your wrapper library needs to _not_ `dlopen` the underlying C++ library, it will be loaded implicitly by the dynamic linker. On Linux, that's `/lib/ld-linux.so.2`:

[https://man7.org/linux/man-pages/man8/ld-linux.so.8.html](https://man7.org/linux/man-pages/man8/ld-linux.so.8.html)

Basically, it boils down to two strategies:

- The ELF file for your wrapper library may embed places to look for its dependencies in a `DT_RUNPATH` (or `DT_RPATH`) entry (see [rpath - Wikipedia](https://en.wikipedia.org/wiki/Rpath)). These entries may use special tokens like `$ORIGIN` to define relative paths. The `patchelf` utility ([GitHub - NixOS/patchelf: A small utility to modify the dynamic linker and RPATH of ELF executables](https://github.com/NixOS/patchelf)) is a nice way to inspect and modify these entries.

- Otherwise, the dynamic linker will look in some general system search path, which might involve built-in default locations like `/lib` and `/usr/lib` or environment variables like `LD_LIBRARY_PATH`.

Mac OS has vaguely similar mechanisms I understand in less detail. Using `install_name_tool` and `llvm-objdump` can help to inspect or modify some of them. I think `@loader_path` is analogous to `$ORIGIN`, and `man 1 dyld` is the best documentation I know of for the dynamic linker.

Racket can manipulate the `$ORIGIN` and `@loader_path` of libraries it installs, e.g.`raco setup` in response to a [`copy-foreign-libs`](https://docs.racket-lang.org/raco/setup-info.html#%28idx._%28gentag._26._%28lib._scribblings%2Fraco%2Fraco..scrbl%29%29%29) definition.

I don't really understand Windows. The only Windows programming I've done since a little Visual Basic 6 a very long time ago is trying to keep my Racket code reasonably portable.

Here is a tangentially interesting blog post about `ld-linux.so.2`: [Taming the ‘stat’ storm with a loader cache — 2021 — Blog — GNU&nbsp;Guix](https://guix.gnu.org/en/blog/2021/taming-the-stat-storm-with-a-loader-cache/)

If you want to distribute the wrapper shared library as a Racket package but use a system-provided OpenFST, I think you may have to just hope that it ends up being in the system's search path. If you are also distributing OpenFST with Racket, I think you want to build your wrapper library so that its `DT_RUNPATH` finds OpenFST relative to `$ORIGIN`.

I would love to find a more streamlined way to build shared libraries for use by Racket. This has in fact been on my mind today: my best attempt is at [GitHub - LiberalArtist/native-libgit2-pkgs at build-scripts](https://github.com/LiberalArtist/native-libgit2-pkgs/tree/build-scripts) (see the extensive `README.md`), which works, but I'd like it to be much more convenient. I find Guix particularly tantalizing for this, since it already has definitions for how to build a whole GNU/Linux distribution's worth of software (in a Scheme DSL, even!). Now that I've got it working for `libgit2`, I'm planning to write up an email to the Guix list with thoughts and questions about how to make this sort of thing work more smoothly.

The build scripts in [racket/racket/src/native-libs at master · racket/racket · GitHub](https://github.com/racket/racket/tree/master/racket/src/native-libs) are a useful reference, as is [Dynext: Running a C Compiler/Linker](https://docs.racket-lang.org/dynext/index.html) (also useful for compiling C that is not a BC extension, as in [bcrypt.rkt/install.rkt at master · samth/bcrypt.rkt · GitHub](https://github.com/samth/bcrypt.rkt/blob/master/bcrypt/private/install.rkt)).

---

<div class="post-metadata">

**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\
**Post date:** [May 27, 2022, 8:05pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/3 "2022-05-27T20:05:44Z")

</div>

An even zanier pipe dream I've sometimes entertained is that we could embed a WASM runtime into Racket, then compile many kinds of foreign libraries to WASM, rather than machine code, avoiding all of the platform-specific issues while also reducing the surface area for memory unsafety.

Apparently Firefox has started doing something like this for some libraries: [Securing Firefox with WebAssembly - Mozilla Hacks - the Web developer blog](https://hacks.mozilla.org/2020/02/securing-firefox-with-webassembly/)

Probably the simplest way would be to write bindings for [Wasmtime](https://wasmtime.dev/), but I could also imagine a Nanopass-based WASM front-end for the Chez Scheme compiler.

As with everything WASM, it seems like procrastination would only make things easier, e.g. as the [WebAssembly System Interface](https://wasi.dev/) stabilizes. It's not something I have any plan to work on, at any rate.

---

<div class="post-metadata">

**Author:** ![simonls](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/simonls/32/170_2.png) [@simonls](https://racket.discourse.group/u/simonls)\
**Post date:** [May 27, 2022, 11:36pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/4 "2022-05-27T23:36:52Z")

</div>

I am not sure about all the details and possible gotchas, I only read about it what I needed to get it to work for my case. I think the first thing you should check is that the wrapper library is actually dynamically linked to the wrapped library. (I write linux commands because that is the only platform I currently use)

In my case I am wrapping the freetype library with a small wrapper that makes it easier to extract gylphs (with an api that is easier for me to work with).  
So when you use a command like `ldd libyourwrapper.so` (also try `ldd -v ...` for more output) you should get a list of dynamic dependencies that includes your wrapped library and also its dependents. If that list does not show those libraries then maybe you are missing some (library) flags while you are building your wrapper library.

I can't give you a general guide towards those flags (I only know what I use, the rest is vague; also other people may know better ways to do things, however I hope it is useful anyway), here is what I use to build my wrapper library:

```bash
gcc $(pkg-config --cflags --libs freetype2) -Lfreetype -Wall -fPIC -c glyphloader.c
gcc -shared -Wl,-soname,libglyphloader.so -lfreetype -o libglyphloader.so glyphloader.o

```

The first line creates the object file, the second creates the shared library out of it.  
In my case the output from `ldd libglyphloader.so` is:

```scheme
linux-vdso.so.1 (0x00007ffe339eb000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x00007f6f7270d000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f6f72500000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007f6f724e6000)
libbz2.so.1.0 => /usr/lib/libbz2.so.1.0 (0x00007f6f724d3000)
libpng16.so.16 => /usr/lib/libpng16.so.16 (0x00007f6f7249c000)
libharfbuzz.so.0 => /usr/lib/libharfbuzz.so.0 (0x00007f6f723ac000)
libbrotlidec.so.1 => /usr/lib/libbrotlidec.so.1 (0x00007f6f7239c000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007f6f72823000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f6f722b5000)
libgraphite2.so.3 => /usr/lib/libgraphite2.so.3 (0x00007f6f72290000)
libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0x00007f6f72152000)
libbrotlicommon.so.1 => /usr/lib/libbrotlicommon.so.1 (0x00007f6f7212f000)
libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007f6f720b6000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f6f720b1000)

```

When I first tried getting this to work I think I didn't have the `-lfreetype` in the second line and the ldd output was:

```scheme
linux-vdso.so.1 (0x00007ffea517d000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fe84817c000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007fe8483d0000)

```

With that missing loading the wrapper library from racket fails with an error message (... inserted by me):

```scheme
ffi-lib: could not load foreign library
  path: /home/sze/development/.../libglyphloader.so
  system error: /home/sze/development/.../libglyphloader.so: undefined symbol: FT_Done_Face
  context...:
   /usr/share/racket/collects/ffi/unsafe.rkt:131:0: get-ffi-lib
   body of "/home/sze/development/.../glyphloader.rkt"

```

Hopefully it is a simple error like this and this information helps.  
If it is something more complicated you should look into using other tools that can provide insights, like @LiberalArtist suggested, I think there are a lot of tools, but I only know of a few.

In cases like this everything that can give us hints towards what actually is happening is helpful.  
Do you / the student get any kind of error when trying to load the library?

> [@jbclements](#):
>
> is there a standard way for dynamic libraries to depend on each other?

I think it is a standard feature of dynamic libraries to depend on other dynamic libraries, but as far as I know every compiler has different ways of being told to add that link information for those other libraries into the created library file, so they end up as dependencies in the shared libraries. Basically with c and c++ you have to tell the compiler what it should do, because there is no proper module system that keeps track of the dependencies. I imagine that there are other languages / build-systems that automatically manage this.

Another thing is, that with dynamic libraries the actual dynamic linking of the other dependencies only happens when the library is loaded, so that is why you can build a library, that looks as if it compiled correctly, but actually has missing dependencies that you only discover when you try to load it.

If you don't get any error (which seems surprising to me) it may make sense to use a debugger (I mean something like gdb or lldb here, not the racket one, but this mostly if racket doesn't give any insight/error message) to start the racket program that tries to load the library and see whether that can give any insight into what is happening and where the program fails.

But take all of this with a grain of salt, the last time I read something more in depth about this stuff was probably something like 15 years ago. So there may be a lot I have forgotten, or different new tools to handle things like this.

---

<div class="post-metadata">

**Author:** ![jjsimpso](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jjsimpso/32/602_2.png) [@jjsimpso](https://racket.discourse.group/u/jjsimpso)\
**Post date:** [May 28, 2022, 2:43am UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/5 "2022-05-28T02:43:12Z")

</div>

As @simonls says above, I think the standard approach is to link your wrapper library against openFST. With a standard C/C++ app, as long as openFST is installed on the target system at runtime, you should be good to go. Modifying the rpath and such isn't normally necessary. I'm not sure what complications result when using the library from a Racket app though.

If the library links fine but doesn't load, i.e. there is a runtime error loading openFST for example, on Linux you can use strace and look for `open` syscalls. You should see it attempt to load the library from various places. That may give you a clue what went wrong.

---

<div class="post-metadata">

**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\
**Post date:** [May 28, 2022, 9:27am UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/6 "2022-05-28T09:27:55Z")

</div>

FWIW here are some useful tools when you are on macOS (quoting [1]):

> After you have built a library you can check that it links to the correct libraries with otool:
> 
> ```
> otool -L foo.dylib
> 
> ```
> 
> The command will show the paths of the libaries that foo.dylib depends on.  
> If you see any /opt/local paths here, you have made a mistake (you have linked with a macport library).
> 
> During the build the dylibs in ${BuildRacketLibs}/lib will contain full paths.  
> If you need to distribute the libraries, they need to be changed to relative paths with `install_name_tool` .

I would do the following:

1. Place both shared libraries in the same folder
2. Add the folder to the environment variable controlling the paths used when linking.
3. Link using -lfoo [where `foo.so` is the name of the shared library]
4. Use `otool` to check that the path is correct inside the shared library.

For more information on shared libraries on Linux, I can't recommend [2] enough.

[1] [GitHub - soegaard/racket-osx-libs: Source files and build instructions for the dynamic libraries used by Racket on OS X](https://github.com/soegaard/racket-osx-libs)  
[2] [https://www.cs.dartmouth.edu/~sergey/cs258/ABI/UlrichDrepper-How-To-Write-Shared-Libraries.pdf](https://www.cs.dartmouth.edu/~sergey/cs258/ABI/UlrichDrepper-How-To-Write-Shared-Libraries.pdf)

---

<div class="post-metadata">

**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\
**Post date:** [May 28, 2022, 9:32pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/7 "2022-05-28T21:32:27Z")

</div>

I took a look at the specific issue at hand with OpenFST, and the problem was that the installed filename of `libfst.so` didn't match the ~~`RPATH`~~ `DT_NEEDED` of `openfst_wrapper.so`, which expected `libfst.so.25`: [fix library loading by LiberalArtist · Pull Request #1 · AlexMaclean/racket-openfst · GitHub](https://github.com/AlexMaclean/racket-openfst/pull/1)

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [May 29, 2022, 9:36pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/8 "2022-05-29T21:36:58Z")

</div>

Okay, well _that_ was way way above and beyond. You'd better watch out, you're going to get a reputation for fixing people's code gratis! 🙂

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [May 29, 2022, 9:37pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/9 "2022-05-29T21:37:26Z")

</div>

Yes, I love this idea. That's exactly what wasm was designed for.

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [May 29, 2022, 9:38pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/10 "2022-05-29T21:38:35Z")

</div>

Of course, having said that, the killer app is self-contained computation-heavy code; the more interdependencies you have, the more this works only if everyone else has already ported all of their other stuff, too.

---

<div class="post-metadata">

**Author:** ![LiberalArtist](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/liberalartist/32/151_2.png) [@LiberalArtist](https://racket.discourse.group/u/LiberalArtist)\
**Post date:** [August 11, 2022, 7:42pm UTC](https://racket.discourse.group/t/shared-libraries-that-depend-on-each-other/1037/11 "2022-08-11T19:42:19Z")

</div>

I was reminded of this today when I stumbled across this paper, which (among many other things) presents a compiler from wasm to safe Rust with good performance:

[https://www.usenix.org/conference/usenixsecurity22/presentation/bosamiya](https://www.usenix.org/conference/usenixsecurity22/presentation/bosamiya)

It's an interesting paper in a number of respects, and some of the suggested techniques for compiling wasm to a high-level language might be applicable to Chez, but what particularly struck me was that they say, in §&nbsp;6.1.3, that it "took one person-month" to develop. That sounds a lot more tractable than I would have guessed!