Pyffi - Use Python from Racket

Ran diagnostics:

open-input-file: cannot open module file
module path: pyffi/configure-pyffi
path: /Users/saludes/pyffi/configure-pyffi.rkt
system error: no such file or directory; rkt_err=3

I have pyffi cloned into /Users/saludes/ but configure-pyffi.rkt is at
pyffi-lib/pyffi/configure-pyffi.rkt inside the cloned repo;
So I guess it should be:
/Users/saludes/pyffi/pyffi-lib/pyffi/configure-pyffi.rkt in my case.

Is Python only necessary to build some examples in the docs? How difficult is to make those examples generate some comment explaining that Python is not available?

Another possibility is that the examples save the results of the Python code to a .txt file that is committed to the repository along the docs, and is used in case Python is not available. Racket has a few of these pregenerated files, mostly to break cycles where the compiler would need a working copy of racket to bootstrap.

1 Like

Note that the make-log-based-eval evaluator is useful for doing exactly this.

1 Like

That looks promising indeed. I'll try it out.

@samth

Today I tried to use make-log-based-eval rather than the normal make-base-eval.

The logging version logs the result of an evaluation: it write the serialized value to a file.
When replayed the value is deserialized from disk.

Sounds easy enough, so I added the property prop:serializable to the obj struct.
The obj struct represents a Python object and consist of the name of the object class and a C-pointer to the Python object.

Serializing the name only gets me to this result (using replay):


Compare this to the original:

The missing part is stored in the Python object.

Now Python also have (de)serialization called pickling and unpickling.
But ... that can't be used, since the reason we are doing this is to build the documentation
on a machine without Python.

The idea of logging the evaluation results are good, but the logging happens too early.
What is needed is a logging version of examples where the scribble structures is logged.

At least I think so - but maybe I am missing an alternative solution?

I think you would either need to have a different version of run that produced the full printout, or a different version of make-log-based-evaluator that did the printing a logging time.

I think you would either need to have a different version of run that produced the full printout, or a different version of make-log-based-evaluator that did the printing a logging time.

The function run is just one of many functions exported by pyffi, so I have looked at
the other option. Given an evaluator returned from make-log-based-evaluator the function wrap
below will do the printing at evaluation time. Combined with make-log-based-evaluator I can record and replay the evaluations.

However, the examples form doesn't know, that the values it receives have been pretty printed already.

The wrap function was tricky. The examples form calls the evaluator multiple times for each example. Besides the value(s) of each example expression, it also needs to know about any activity on output and debug ports. This is done by sending "evaluator messages", but evaluator-message? is not exported from racket/sandbox.

@(require racket/match rackunit)
@(require/expose racket/sandbox (evaluator-message?))
@(define (wrap ev)
   (lambda (x)
     (call-with-values
      (λ ()
        (if (evaluator-message? x)
            (ev x)
            (match x
              ; these non-expressions are used in the manual
              [(cons (or 'require 'define 'import) _)
               (ev x)]
              [else
               (ev `(with-output-to-string (λ () (print ,x))))])))
      (λ vs (apply values vs)))))

Example output:

How do I get examples to format these pretty printed values correctly?

I think that instead of producing a string directly, you need to produce something whose printed representation is that string. Maybe like (totally untested and buggy):

(struct v (s) #:property prop:custom-write (lambda (v p m) (write (v-s v) p))

and then wrap (v ...) around the with-output-to-string.

1 Like

@samth

The latest commit of pyffi uses recorded results from examples.log to avoid invoking pyffi in the examples form. That part works.

But ... since I need to document the forms in pyffi my Scribble document manual-pyffi.scbrl has an for-label dependency on pyffi. And that of provokes an error, since libpython is not present.

At this point I am beginning to think, that the path of least resistance is to ship pyffi with a minimal python installation (maybe just the shared library), so the build server doesn't complain.

The documentation for the package server suggests delaying the use a the native libary to runtime.
So I went looking for examples on the package server without much luck.
Is there a nice example of this somewhere?

1 Like

I think the trick is to install from the pyffi/pyffi folder not the top level pyffi folder. At least that was the case for me. You'll have to remove the existing install first raco pkg remove pyffi and then cd to pyffi\pyffi and run raco pkg install

@wluker
You are/were right when installing from the Github repo.

Yesterday I put pyffi on the package server, so (crossed fingers) now you can install pyffi with:

raco pkg install --no-docs pyffi

After configuring pyffi with

raco pyffi configure

it is possible to build the documentation with:

raco setup pyffi

A confirmation that the above steps work on macOS and Linux would be nice.
I expect the bindings doesn't work on Windows.

1 Like

@soegaard
I can confirm it installs on Linux with your command sequence given above.

raco setup pyffi is triggering something, but the documentation is not showing up under pyffi with raco docs. I noticed it is also not showing up at https://docs.racket-lang.org/

Neat package!

1 Like

Yes! Now it works for me on Mac OSX 13.0.1 with Python 3.10.6

1 Like

I think you want to use some combination of the #:fail argument to ffi-lib and make-not-available with define-ffi-definer to make the relevant functions error when called rather than when the modules are required.

1 Like

@samth

I think you want to use some combination of the #:fail argument to ffi-lib and make-not-available with define-ffi-definer to make the relevant functions error when called rather than when the modules are required.

I did try that line of attack. I can get ffi-lib to return a value that indicates, that the library is not loaded.
However define-ffi-definer signals an error, when it is passed a non-lib.
The make-not-available option is for the individual functions.

It's possible to cheat - i.e. pass a different library to define-ffi-definer but that's at best a hack.

Also, in regular use, I'd expect an error when the library is not present.
Since pyffi is require for-label the loading happens before the code in the Scribble file runs - so I don't see a way for the documentation to set a flag suppressing the error.

And i can import sympy. Thanks!

#lang racket
(require pyffi)
(initialize)
(post-initialize)

(import sympy)
(define x (sympy.Symbol "x" #:real #t))
(define pe sympy.parse_expr)
(for ([e (list "sin(x)" "cos(x)" "exp(x)" "x**3")])
    (let ([e~ (pe e)])
      (displayln
       (cons e~ (sympy.diff e~)))))
2 Likes