Suppose I would like to unload a library (QtWebEngine) as to re-initialize the complete structure as if loaded for the first time. Would that be possible with racket?
Otherwise are there handlers to make it possible to cleanup libraries, before the complete racket / drracket environment quits? That would not be the same as calling '(exit)' in a drracket tab. It really be just before quitting drracket entirely.
As I understand it the answer is no.
Unloading a dynamic C library from a process is as far as I know,
not possible - in any language [1].
As a practical note, when I write bindings, I don't run the test programs from DrRacket / racket-mode, but runs a test script in the terminal. This solves the unloading problem for me.
[1]
Oke, thanks, I was already afraid this would be the case. However, I still would like to be able to use this FFI binding from the DrRacket IDE. Would it be possible to install something of an "atexit" handler on the "exit drracket" level? Than I could do the cleanup of the library at that point. It can however not be the same as a C atexit handler, because cleaning up QApplication in an atexit() function in C crashes.
It used to be the case that there were only one system process running both DrRacket and the user programs. In that scenario, you still have the problem of not being able to unload a dynamic library.
But - recently it's possible to have parallel OS level processes.
So, I would experiment with loading the bindings in a parallel thread.
Then kill the parallel thread to "unload" the library.
Thanks for your answer. Unfortunately this is an binding to a library that encapsulates Qt and needs a QApplication, which can only be used in the main thread.
You actually can do this using the #:custodian argument to ffi-lib:
If
custodianis'placeor a custodian, the library is unloaded when a custodian is shut down—either the given custodian or the place’s main custodian ifcustodianis'place. When a library is unloaded, all references to the library become invalid. Supplying'placefor custodian is consistent with finalization viaffi/unsafe/allocbut will not, for example, unload the library when hitting in theRunbutton in DrRacket. Supplying(current-custodian)forcustodiantends to unload the library for eagerly, but requires even more care to ensure that library references are not accessed after the library is unloaded.
It is true that the extent to which unloading and reloading works as hoped will depend on the platform: in terms of “The Separate Compilation Guarantee,” interactions with foreign libraries are firmly “external effects.” I think there are even linker options whereby the foreign library can influence how it might be un/reloaded. Nonetheless, even with reasonably complex libraries (e.g. libvlc), I’ve found that #:custodian (current-custodian) works well enough to be useful, especially for development.
This isn’t right. The new parallel threads involve multiple OS-level threads, but they are in the same OS-level process, with the same address space.
(If for some reason you want multiple OS processes, look at distributed places or loci.)
Relatedly, one of the ways that “unsafe” code is unsafe is that DrRacket cannot reliably protect itself from buggy user programs: it is possibly to use unsafe features to crash DrRacket (at best!).
There are a lot of features for cleanup, mostly related to custodians and plumbers. It can be a bit tricky to work out what feature to use for a specific kind of cleanup. If you can make things work with just ffi/unsafe/alloc, that is a well-trodden path. For interaction with custodians and cleanup not related to the reachability of a particular value, ffi/unsafe/custodian can help. But feel ask lots of questions about design approaches.
Thanks for pointing this out.
I'll look forward to trying the custodian trick the next I need to experiment with dynamic libraries.
Wow, this is next level. I may try some of these possibilities, if I can understand them 
. The Qt WebEngine behavior does not leave me with many options. I submitted an issue to the Qt community, but the way it works seems to be expected, not a bug.
I am responding to this as an example of focusing solely on Linux and not looking to other operating systems.
In Windows, FreeLibrary will unload memory and there is even a convention in which a function in the DLL is called to free resources and the like before being unloaded from memory.
In OS/X dlclose is much more aggressive in actually removing the library unless certain conditions are true that make it impractical to do so.
Nathan Dykman
But the claim isn’t generally true on Linux, either! The linked Stack Overflow thread just established that POSIX permits dlclose() to be a no-op, and that some compiler and linker options can end up with unloading not happening, or at least with static data not being re-initialized on reloading.
Since v7.4.0.7, Racket has supported unloading foreign libraries on all platforms, to the fullest extent implemented by each platform.
Since v7.4.0.7 ...
That's very specific!
I completely missed this.
Good news.
I distinctly remember a situation, where I recompiled a C library on disk,
and each time I tried reopening it in DrRacket, I would get the old version.
Whether it was the DrRacket process or the OS that had cached the version, I don't know.
And I can't even remember which OS I was on.
This absolutely will happen if you do not use #:custodian (current-custodian), and it is a memorably vexing experience!
Oke, I've tested loading the library with #:custodian (current-custodian) on Linux and as expected, (given the Qt Bug, reported earlier) it doesn't work.
(define libraries-to-preload
(cond
([eq? os 'windows]
'(Qt6Core.dll
Qt6Positioning.dll
Qt6Gui.dll
Qt6Widgets.dll
Qt6Svg.dll
Qt6Network.dll
Qt6OpenGL.dll
Qt6PrintSupport.dll
Qt6Qml.dll
Qt6QmlModels.dll
Qt6QmlWorkerScript.dll
Qt6QmlMeta.dll
Qt6Quick.dll
Qt6QuickWidgets.dll
Qt6WebChannel.dll
Qt6WebEngineCore.dll
Qt6WebEngineWidgets.dll
))
([eq? os 'linux]
'(libQt6XcbQpa
;libQt6WaylandClient
;libQt6EglFSDeviceIntegration
libQt6Core
libQt6Positioning
libQt6Gui
libQt6Widgets
libQt6Svg
libQt6Network
libQt6OpenGL
libQt6PrintSupport
libQt6Qml
libQt6QmlModels
libQt6QmlWorkerScript
libQt6QmlMeta
libQt6Quick
libQt6QuickWidgets
libQt6WebChannel
libQt6WebEngineCore
libQt6WebEngineWidgets
))
)
)
(define ffi-library
(cond
([eq? os 'windows] 'rktwebview_qt.dll)
([eq? os 'linux] 'librktwebview_qt.so)
)
)
(define os-lib-dir (build-path lib-dir (symbol->string os) (symbol->string arch)))
(define (libname lib-symbol)
(build-path os-lib-dir (symbol->string lib-symbol)))
; c:\qt\6.10.2\msvc2022_64\bin\windeployqt.exe rktwebview_qt_test.exe
(define quiet-call #t)
(set! quiet-call
(when (or (eq? os 'windows) (eq? os 'linux))
(putenv "QT_PLUGIN_PATH"
(path->string (build-path os-lib-dir)))
(putenv "QTWEBENGINEPROCESS_PATH"
(path->string (build-path os-lib-dir "QtWebEngineProcess.exe")))
(putenv "QTWEBENGINE_RESOURCES_PATH"
(path->string (build-path os-lib-dir "resources")))
(putenv "QTWEBENGINE_LOCALES_PATH"
(path->string (build-path os-lib-dir "translations" "qtwebengine_locales")))
(when (eq? os 'linux)
(putenv "QT_QPA_PLATFORM" "xcb")
(putenv "LD_LIBRARY_PATH"
(string-append
(path->string (build-path os-lib-dir)) ":"
(path->string (build-path os-lib-dir "platforms"))
)
)
)
)
)
;;; Preload libraries
(for-each (λ (lib-symbol)
(let* ((libn (if (list? lib-symbol) (car lib-symbol) lib-symbol))
(versions (if (list? lib-symbol) (cons (cadr lib-symbol) '(#f)) (list #f)))
(load-lib (if (list? lib-symbol)
(if (eq? (caddr lib-symbol) #f)
(symbol->string libn)
(libname libn))
(libname libn)))
)
;(displayln (format "loading ~a" load-lib))
(ffi-lib load-lib versions
#:custodian (current-custodian))
)
)
libraries-to-preload)
;;; Actual FFI integration
(define webview-lib-file (libname ffi-library))
(define webview-lib
(with-handlers ([exn:fail?
(λ (exp)
(cond
([eq? os 'linux]
(error (format
(string-append "Cannot load ~a.\n"
"Make sure you installed Qt6on your system\n"
"e.g. on debian 'sudo apt install libqt6webenginewidgets6'\n"
"\n"
"Exception:\n\n~a")
ffi-library exp)))
(else (error
(format "Cannot load ~a for os ~a\n\nException:\n\n~a"
ffi-library os exp))))
)
])
(ffi-lib webview-lib-file '("6" #f)
#:get-lib-dirs (list os-lib-dir)
#:custodian (current-custodian)
)
)
)
Cleanup code in C(++):
void rkt_webview_cleanup()
{
if (handler != nullptr) {
// Does nothing.
// See QTBUG-145033
// QtWebEngine cannot be started as part of QApplication more than once in an application run.
QApplication *app = handler->app();
if (app != nullptr) {
// testing with #:custodian (current-custodian)
QTimer t;
QObject::connect(&t, &QTimer::timeout, app, [app]() {
fprintf(stderr, "Quitting\n");
app->quit();
});
t.setSingleShot(true);
t.start(250);
fprintf(stderr, "Executing app\n");
app->exec();
fprintf(stderr, "App exited\n");
delete handler;
handler = nullptr;
}
}
}
(...)
Rktwebview_qt::~Rktwebview_qt()
{
QList<int> win_keys = _views.keys();
int i, N;
for(i = 0, N = win_keys.size(); i < N; i++) {
WebviewWindow *w = _views[win_keys[i]];
delete w;
}
QList<int> c_keys = _contexts.keys();
for(i = 0, N = c_keys.size(); i < N; i++) {
QWebEngineProfile *p = _contexts[c_keys[i]];
delete p;
}
delete _app;
}
However, as has been determined in several issue reports (QTBUG-70519, QTBUG-87460, QTBUG-145033) at Qt, QtWebEngine cannot be restarted (it fires up several threads, those are not released). Also cleaning up 'atexit' is not possible (QTBUG-145033).
So preferably, I would like to cleanup everything just before racket or drracket exits, but not in an atexit procedure.