Unloading a C library / reinitializing (FFI)

[hnmdijkema]

Suppose I would like to unload a library (QtWebEngine) as to re-initialize the complete structure as if loaded for the first time. Would that be possible with racket?

Otherwise are there handlers to make it possible to cleanup libraries, before the complete racket / drracket environment quits? That would not be the same as calling '(exit)' in a drracket tab. It really be just before quitting drracket entirely.

[soegaard]

As I understand it the answer is no.

Unloading a dynamic C library from a process is as far as I know,
not possible - in any language [1].

As a practical note, when I write bindings, I don't run the test programs from DrRacket / racket-mode, but runs a test script in the terminal. This solves the unloading problem for me.

[1]

dlclose doesn't really unload shared object, no matter how many times it is called

[hnmdijkema]

Oke, thanks, I was already afraid this would be the case. However, I still would like to be able to use this FFI binding from the DrRacket IDE. Would it be possible to install something of an "atexit" handler on the "exit drracket" level? Than I could do the cleanup of the library at that point. It can however not be the same as a C atexit handler, because cleaning up QApplication in an atexit() function in C crashes.

[soegaard]

It used to be the case that there were only one system process running both DrRacket and the user programs. In that scenario, you still have the problem of not being able to unload a dynamic library.

But - recently it's possible to have parallel OS level processes.
So, I would experiment with loading the bindings in a parallel thread.
Then kill the parallel thread to "unload" the library.

[hnmdijkema]

Thanks for your answer. Unfortunately this is an binding to a library that encapsulates Qt and needs a QApplication, which can only be used in the main thread.

[LiberalArtist]

@hnmdijkema

uppose I would like to unload a library (QtWebEngine) as to re-initialize the complete structure as if loaded for the first time.

You actually can do this using the #:custodian argument to ffi-lib:

If custodian is 'place or a custodian, the library is unloaded when a custodian is shut down—either the given custodian or the place’s main custodian if custodian is 'place. When a library is unloaded, all references to the library become invalid. Supplying 'place for custodian is consistent with finalization via ffi/unsafe/alloc but will not, for example, unload the library when hitting in the Run button in DrRacket. Supplying (current-custodian) for custodian tends to unload the library for eagerly, but requires even more care to ensure that library references are not accessed after the library is unloaded.

It is true that the extent to which unloading and reloading works as hoped will depend on the platform: in terms of “The Separate Compilation Guarantee,” interactions with foreign libraries are firmly “external effects.” I think there are even linker options whereby the foreign library can influence how it might be un/reloaded. Nonetheless, even with reasonably complex libraries (e.g. libvlc), I’ve found that #:custodian (current-custodian) works well enough to be useful, especially for development.

@soegaard

It used to be the case that there were only one system process running both DrRacket and the user programs. In that scenario, you still have the problem of not being able to unload a dynamic library.

But - recently it's possible to have parallel OS level processes.
So, I would experiment with loading the bindings in a parallel thread.
Then kill the parallel thread to "unload" the library.

This isn’t right. The new parallel threads involve multiple OS-level threads, but they are in the same OS-level process, with the same address space.

(If for some reason you want multiple OS processes, look at distributed places or loci.)

Relatedly, one of the ways that “unsafe” code is unsafe is that DrRacket cannot reliably protect itself from buggy user programs: it is possibly to use unsafe features to crash DrRacket (at best!).

@hnmdijkema

Would it be possible to install something of an "atexit" handler on the "exit drracket" level? Than I could do the cleanup of the library at that point. It can however not be the same as a C atexit handler, because cleaning up QApplication in an atexit() function in C crashes.

There are a lot of features for cleanup, mostly related to custodians and plumbers. It can be a bit tricky to work out what feature to use for a specific kind of cleanup. If you can make things work with just ffi/unsafe/alloc, that is a well-trodden path. For interaction with custodians and cleanup not related to the reachability of a particular value, ffi/unsafe/custodian can help. But feel ask lots of questions about design approaches.

[soegaard]

@LiberalArtist

The new parallel threads involve multiple OS-level threads, but they are in the same OS-level process, with the same address space.

Thanks for pointing this out.

I'll look forward to trying the custodian trick the next I need to experiment with dynamic libraries.

[hnmdijkema]

Wow, this is next level. I may try some of these possibilities, if I can understand them . The Qt WebEngine behavior does not leave me with many options. I submitted an issue to the Qt community, but the way it works seems to be expected, not a bug.