Best way to integrate "schemesh" written in Chez Scheme, into Racket ecosystem?

[LiberalArtist]

@Cosmos721

My question is: what's the best way to extend schemesh in order to integrate it within Racket ecosystem?

This sounds very exciting! I have a few off-hand thoughts.

@Cosmos721

c. take the existing schemesh, compiled as a Chez Scheme library, and load it from Racket
No idea if that's even possible, if it can be implemented by extending Racket, etc.

This is definitely an option you could use for all or part of the code. You can use the ffi/unsafe/vm library to access Chez Scheme functionality from Racket: there’s more discussion here on how to load additional Chez Scheme libraries.

However, as the name of ffi/unsafe/vm implies, Chez Scheme functionality corresponds to the “unsafe” layer of Racket. There are some documented caveats, particularly about potential uses of dynamic-wind and Racket procedures that may not be Chez Scheme procedures.

I think it would probably work out better to write a compatibility layer in Racket that would let most of your code be shared between both implementations.

@Cosmos721

(register-signal-handler) and (keyboard-interrupt-handler)
needed for installing signal handlers for POSIX signals SIGINT, SIGCHLD, SIGQUIT
and quickly reacting to them

($primitive $event)
if a POSIX signal was received, calls the corresponding signal handler.
by default, Chez Scheme periodically calls ($primitive $event), but I need to call it immediately after C functions return with errno = -EINTR
because it means some POSIX signal has been received and I need to call the corresponding signal handler,
before retrying the C function that may block for an arbitrarily long time. Examples: read() or write() on a pipe file descriptor

To the extent you need them, ffi/unsafe/vm is probably the best way to get these. You could also consider the unix-signals package.

However, with respect to blocking C functions, note that Racket’s IO functions are non-blocking at the level of the OS process (though some block at the level of the green Racket thread). You might want to use Racket’s process control functionality instead of going through C. In particular, some variants can handle bridging the Racket-thread-specific current-directory parameter and wiring up arbitrary Racket ports to stdin/out/err.

@Cosmos721

(read-token) and (unread-char)
used to parse a single token of Scheme syntax - otherwise I would need to reimplement a Scheme syntax parser from scratch.
(read) is not a suitable alternative because it does not recognize the syntax extension tokens added by schemesh for switching from Scheme syntax to shell syntax: #!shell { }

Racket’s reader is highly extensible: you can add dispatch and delimiter macros to the readtable to cover #!shell, {, }, and everything else. Extending the Racket reader can also help you cooperate with syntax coloring and REPL support that works with both DrRacket and Racket’s command-line expeditor (and maybe racket-mode in Emacs, and/or other editors via the LSP?).

@Cosmos721

(interaction-environment) and (eval form environment)
the mutable Chez Scheme environment containing all top-level R6RS bindings plus Chez Scheme extensions,
and the (eval) procedure to implement a REPL.
Since schemesh is a REPL, expressions evaluated at REPL must be able to access top-level bindings, and may also create new ones.

The needed functionality definitely exists, and may be as simple as calling (read-eval-print-loop) after setting up an appropriate namespace and such. If you need finer-grained control, all of the pieces are also provided.

@Cosmos721

(top-level-bound?) (top-level-value) (meta-cond) (library-exports)
used to check for some Chez Scheme bindings that are not always present, such as:
(make-thread-parameter) (make-flvector) (flvector-set!)

There are ways to do conditional compilation and reflection in Racket, though you may need less of it: Racket always has flvectors, for example.

@Cosmos721

(foreign-procedure) (lock-object) and (unlock-object)
the core of Chez Scheme C FFI, schemesh also uses it for bidirectional exchange of Scheme objects with C functions
such as vectors, bytevectors and lists.

If I understand correctly, Racket C FFI can only exchange C types with C functions, i.e. one needs to malloc(), copy a Racket string or byte string into the allocated memory, and pass such memory to C functions. It may be enough, but the porting will be somewhat painful.

The Racket FFI is a layer on top of the Chez Scheme FFI, and vectors, bytevectors (a.k.a. byte strings), and lists are the same in Racket as in Chez Scheme (except IIRC for impersonated vectors, including chaperones), so all of this should work. In particular, a byte string is a Racket cpointer?. This might be an area where ffi/unsafe/vm would be useful.

@Cosmos721

(environment-symbols)
used for autompletion with TAB key: needed to retrieve the top-level bindings present in (interaction-environment)
and match them against user-entered text.

If my suggestions above work out, the autocompletion in expeditor should do this for you. If you end up needing to reimplement more functionality, all the reflective operations you need exist, e.g. namespace-mapped-symbols.

@Cosmos721

(generate-temporaries)
used for hygienic macros that need to introduce a variable number of symbols into their expansion

We’ve got it: generate-temporaries

(Also, this one is standardized in (rnrs syntax-case).)