Calling into a Chez Scheme library from Racket -- imported symbol rewritten in vm-eval?

Questions & Answers
, ,
1

I have a Chez Scheme library "foo.ss" that I'd like to call from Racket:

(library (foo)
   (export bar)
   (import (rnrs))

   (define (bar) "hello"))

From Chez Scheme:

$ chezscheme
Chez Scheme Version 9.5.4
Copyright 1984-2020 Cisco Systems, Inc.
> (eval '((lambda () (import (foo)) (bar))))
"hello"

And my Racket program "call.rkt" which attempts to use the library:

#lang racket

(require ffi/unsafe/vm)

(vm-eval
 '((lambda () (import (foo)) (bar))))

$ racket call.rkt
variable bar554 is not bound
  context...:
   body of "/home/andrew/prj/audit/compile/call.rkt"

What would be the right way to do this?

Thank you!

Andrew

2

I don't know the solution, but I think at least part of the problem would be that Racket CS does not consult the default Chez Scheme search path for Scheme libraries, because it does not expect to use any Scheme libraries other than those in its bootfiles.

3

The code isn't using Racket to load the Chez library... the default Chez Scheme library path is the current directory, and this doesn't appear to have been changed in the VM environment:

$ racket
Welcome to Racket v8.10 [cs].
> (require ffi/unsafe/vm)
> (vm-eval '(library-directories))
'(("." . "."))

And loading the library from Chez appears to be working (at least there isn't an error):

$ racket
Welcome to Racket v8.10 [cs].
> (require ffi/unsafe/vm)
> (vm-eval '(import (foo)))
>

I imagine the Chez Scheme library import might be doing some kind of hygienic macro renaming of symbols. I don't know how to get around that.

4

Interesting, it seems like it isn't some quirk of vm-eval, but maybe something about Chez libraries not working in the VM environment?

script.ss:

((lambda ()
  (import (foo))
  (write (bar))
  (newline)))

$ chezscheme 
Chez Scheme Version 9.5.4
Copyright 1984-2020 Cisco Systems, Inc.

> (load "script.ss")
"hello"

$ racket
Welcome to Racket v8.10 [cs].
> (require ffi/unsafe/vm)
> (vm-eval '(load "script.ss"))
; variable bar585 is not bound [,bt for context]
5

Could be! I tried this and it manages to run, but I don't understand Scheme enough to tell what's going on:

;; go.ss
(import (foo) (rnrs))
(write (bar))
(newline)

$ racket
> (require ffi/unsafe/vm)
> (vm-eval '(load-program "go.ss"))
"hello"

So that vm-eval is wrapped inside some library? Racket's vm-primitive is implemented via primitive-lookup here, I'm not sure how this call to eval interacts with the environment.
(In case it's relevant, I believe Racket's entry point is this https://github.com/racket/racket/blob/06f10d51d11ded84df9805578245bfd80391b96a/racket/src/cs/main.sps.)

https://github.com/racket/racket/blob/06f10d51d11ded84df9805578245bfd80391b96a/racket/src/cs/linklet.sls#L154-L163

(library (linklet)
  (export ... primitive-lookup ...)
  (define (primitive-lookup sym)
    (unless (symbol? sym)
      (raise-argument-error 'primitive-lookup "symbol?" sym))
    (call-with-system-wind
     (lambda ()
       (guard
        (c [else #f])
        (eval sym)))))
6

In Racket's main program, the exposed eval is the one that goes through Racket's expander, not Chez Scheme's eval. I wonder if that has affected the result of (vm-primitive 'eval) and therefore affected vm-eval, or that some current-eval parameter is different so the lambda in your example actually went through Racket's expander.

These tests seem to suggest otherwise, though, so I don't know how to make sure to run raw Scheme expressions without going through Racket's frontend.

Welcome to Racket v8.10.0.3 [cs].
> (require ffi/unsafe/vm)
> (equal?  (vm-eval (quote call-with-current-continuation))  (vm-eval (quote ($primitive call-with-current-continua
tion)))  )
#f
> (equal?  (vm-eval (quote eval))  (vm-eval (quote ($primitive eval)))  )
#t

In particular, I don't understand why the second expression is #t. That suggests that the eval obtained by running vm-eval is the same one as the primitive Chez Scheme eval, but in main.sps the eval is shadowed by the eval from the expander (https://github.com/racket/racket/blob/06f10d51d11ded84df9805578245bfd80391b96a/racket/src/cs/main.sps#L1-L27).

7

That's promising! Now all I need is a way to get a value from the loaded program back up into Racket.

load-program doesn't return a value (it doesn't return the last value in go.ss), so I can't get a value that way...

The loaded program also doesn't seem to have access to "globals" defined with vm-eval:

$ racket
Welcome to Racket v8.10 [cs].
> (require ffi/unsafe/vm)
> (vm-eval '(define abc 123))
> (vm-eval 'abc)
123
> (vm-eval '(load-program "go.ss"))
; attempt to reference unbound identifier abc [,bt for context]

where go.ss is:

(import (foo) (rnrs))
(write abc)
(newline)

:thinking:

8

Okay something sketchy is going on. I don't know how this went through :joy:

$ racket
Welcome to Racket v8.10.0.3 [cs].
> (require ffi/unsafe/vm)
> (vm-eval '(top-level-program (import (foo)) bar))
> (vm-eval '(import (foo)))
> (vm-eval 'bar)
#<procedure:bar>
> ((vm-eval 'bar))
"hello"

Perhaps posting the question to ＃internal on Racket discord can help.

9

I think the problem is how Racket adjusts import expansion via the expand-omit-library-invocations parameter. You can restore it while evaluating like this:

#lang racket
(require ffi/unsafe/vm)

(define call-with-import-working
  (vm-eval '(lambda (thunk)
              (call-with-system-wind
               (lambda ()
                 (parameterize ([expand-omit-library-invocations #f])
                   (thunk)))))))

(call-with-import-working
 (lambda ()
   (vm-eval
    '((lambda () (import (foo)) (bar))))))
3 Likes
10

Success!

call.rkt:

#lang racket

(require ffi/unsafe/vm)

(define call-with-import-working
  (vm-eval '(lambda (thunk)
              (call-with-system-wind
               (lambda ()
                 (parameterize ([expand-omit-library-invocations #f])
                   (thunk)))))))

(define bar
  (call-with-import-working
   (lambda ()
     (vm-eval
      '((lambda () (import (foo)) bar))))))

(printf "bar: ~s~n" (bar))

$ racket call.rkt
bar: "hello"

Thanks @mflatt!