So I updated my racket git checkout (sepparate local racket) to version Racket v8.3.0.8 [cs] commit 795c2bd1b17e337a6f6a99e6229304fdd6c6123a and recompiled it.
config returned 0
("COMPILER=gcc-11.1.0" "ENABLE_COLUMN_METADATA" "ENABLE_DBSTAT_VTAB" "ENABLE_FTS3" "ENABLE_FTS3_TOKENIZER" "ENABLE_FTS4" "ENABLE_FTS5" "ENABLE_JSON1" "ENABLE_MATH_FUNCTIONS" "ENABLE_RTREE" "ENABLE_STMTVTAB" "ENABLE_UNLOCK_NOTIFY" "HAVE_ISNAN" "MAX_EXPR_DEPTH=10000" "MAX_VARIABLE_NUMBER=250000" "SECURE_DELETE" "SYSTEM_MALLOC" "TEMP_STORE=1" "THREADSAFE=1")
flags = 65538, db = #<cpointer>, status = 0
0
done
All programs working without a crash, so I am confused because I expected it to not work.
I do raco pkg install libsqlite3 and I get the error again raco pkg remove libsqlite3 and I still get the error. So now I am thinking maybe the error only happens if you had libsqlite3 installed at some point.
Honestly at this point I am just confused 
Took a bit of a brake and reran the compile option one.
config returned 0
("COMPILER=gcc-5.5.0 20171010" "ENABLE_FTS3" "ENABLE_FTS3_PARENTHESIS" "ENABLE_FTS4" "ENABLE_FTS5" "ENABLE_JSON1" "ENABLE_LOAD_EXTENSION" "ENABLE_MATH_FUNCTIONS" "ENABLE_RTREE" "ENABLE_STAT4" "ENABLE_UPDATE_DELETE_LIMIT" "HAVE_ISNAN" "SOUNDEX" "SYSTEM_MALLOC" "TEMP_STORE=3" "THREADSAFE=1" "USE_URI")
invalid memory reference. Some debugging context lost
What I find troubling is that even after I removed libsqlite3 it seems its library stays installed.
And further that gcc-5.5.0 20171010 seems very old and could explain the different and strange behavior.
What I don't know yet is how I can get back to the state as if I had never installed libsqlite3.
Cause found?
So it really seems to be the libsqlite3 package.
I looked at its package definition racket-libsqlite3/info.rkt at 9e5c5696c65ddd3f8359e234fd4ebd93ad1bfc3c · Bogdanp/racket-libsqlite3 · GitHub
and then executed raco pkg remove libsqlite3-x86_64-linux and it actually removed something.
After that it now works again.
Bug:
To me it seems there is a bug with the package manager here: platform specific package dependencies aren't marked as automatically installed / dependent on the meta package and thus aren't removed when the meta package is removed?
Thanks
Thank you @ryanc for your help and example program, without it I wouldn't have found out that there was still the platform specific sub package of libsqlite3 hanging around after removing the package.
Where to go from here?
I am still not sure what is going on with the libsqlite3 package if that is a problem others could have? So far nobody but me could reproduce this error...
Maybe it would disappear if the system that builds the libraries used a less old gcc?
(@bogdan it seems the crash in this post is somehow caused by libsqlite3, but unless someone can reproduce it, this is simply to let you know in case someone has the same problem)
For now it seems uninstalling the not auto-removed sub-package fixes the problem for me.