FFI define-cstruct recursive dependencies on types

ffi bindings / experience report / what I did

first attempt: manual bindings for freetype

In my particular case I wanted to use freetype from racket via ffi, but after a while I gave up on trying to create bindings for it directly. The freetype library has deeply nested structs and it is very annoying having to replicate all the type declarations, then eventually you get some type wrong somewhere and the struct alignments don't work anymore and you get a segfault trying to use your faulty bindings.

Overall this can be quite tedious and demotivating, on the c side you are confronted with types where sometimes you aren't exactly sure what their size is, so sometimes you have to write a small piece of code that prints its size with sizeof. Then on the racket side it is sometimes not the easiest to find the right racket type. When there are a lot of typedefs or even pre-processor macros on the c side this makes it that you are flipping through the (freetype) documentation from one place to the next and occasionally hunting down things in the header files. When you crash you don't really know what is wrong, because you are dealing with structs with sub-structs with 20+ entries and you don't really know which one you got wrong. I was almost starting to analyze all the structs interactively with a debugger, but then decided to go another route.

second attempt: c wrapper library to simplify api

All that manual binding creation above, I tried that for a bit, but then I switched gears.
I thought well if the libraries api is difficult to create bindings for, why don't I write a small wrapper around the library, creating a simple api, for what I actually want to get from that library.

So I wrote a small piece of c code compiled that as a dynamic library, that is dynamically linked with the original freetype library, now my wrapper can make the calls to freetype and give me a few more constrained, easier to call function calls. So now it was way easier to create bindings for my wrapper because it exposed simpler c types.

With that I relatively quickly had a working racket program that used the ffi to make calls to my wrapper and load glyphs from some font with font size x. Success.

That wrapper tries to do the minimal stuff to load gray scale glyph data using freetype.
(so far completely without complex hinting stuff, only the basic advance info that is attached to the glyph)

native racket libraries

Except now I have a new problem, because now I have to build that wrapper on every platform where I want to use it. I don't think that this is necessarily difficult, it is more another annoyance.

For example, when I want to make it easy for people to use that wrapper too, then it is nicer if I provide it for them pre-built and packaged as a racket package. So far I don't have a setup / images / (virtual-)machines to do that.


closing thoughts

For now I am closing this, because I had a look at how define-cstruct is implemented, but apparently that was pre syntax-parse and I don't want to reverse engineer it to figure out how to build a version where you can define mutually recursive structs.

And my experience with freetype has taught me that sometimes it doesn't make sense to try to create bindings 1 to 1, especially not if you just use 5% of what a library offers. If you just need 5% then you can often greatly simplify how you cross that ffi boundary making it easier to write that ffi-code and also possibly more performant, because you don't have to cross back and forth over that boundary a lot of times.

Another approach might be using dynamic-ffi to generate bindings, but so far I haven't tried a lot in that direction, it seems you need a relatively old llvm version, I couldn't find these versions as packages for my system. I have the impression that llvm is quite big and maybe it is quite the task to compile it yourself, but I haven't actually tried it so I could be wrong.

Overall the idea of being dependent upon llvm is something I am not particularly fond of, which is another reason why I have avoided that particular route so far.