Error from dynamic-require collection written in typed racket in a new namespace

Questions & Answers
1

I have a project, which has a main program that generates an exe file using raco exe in advance, and distributed it to clients. When the main program is running, it will dynamically require multiple compiled zo and dep files from the server side and run them. And if the module on the server is modified, the zo and dep files will be automatically reloaded. But if it happens in the same namespace, reloading will not take effect, until the main program restarted. So I use make-base-namespace to create a new namespace which is named new-ns, but the new-ns lackes modules information of the main namespace. I has to use namespace-attach-module to add the missing module information one by one to new-ns. But a new problem emerges, which originates from the typed racket. When I refer to a collection written in typed racket, such as math/statistics, an error will appear, indicating that a certain submodule cannot be found.

If I don't require collections written in typed racket, no error occurs.

This is a simplified example：

contents in main.rkt:

#lang racket/base
(require "a.rkt")
(define-namespace-anchor a)
(define ns (namespace-anchor->namespace a))
(define ROOTDIR "/home/duhaibo/projects/test")
(parameterize ([current-namespace (make-base-namespace)])
  (namespace-attach-module ns 'math/statistics)
  (let ([my-mean (dynamic-require (make-resolved-module-path (build-path ROOTDIR "a.rkt"))
                                  'my-mean)])
    (my-mean '(1 2 3))))

contents in a.rkt:

#lang racket/base
(require math/statistics)
(provide (all-defined-out))
(define (my-mean lst) (mean lst))

Compile the main.rkt and run：

raco make main.rkt
raco exe --embed-dlls main.rkt
./main

Then an error occurs：

instantiate: unknown module
module name: (submod '#%embedded:math/private/statistics/expected-values: #%contract-defs)
context...:
.../test/main.rkt:35:0
body of '#%mzc:main

I don't know how to attach the submodule to the new namespace, and I think the submodule will be attached to the new namespace with math/statistics automatically. Is there a walkaround?

2

The immediate cause of your problem isn't dynamic-require or namespace-attach-module per se: it's the way you are using raco exe. You can tell by comparing the result from your example's ./main with racket main.rkt:

philip@bastet:/tmp/rkt$ raco make main.rkt 
philip@bastet:/tmp/rkt$ racket main.rkt 
2
philip@bastet:/tmp/rkt$ raco exe --embed-dlls main.rkt
philip@bastet:/tmp/rkt$ ./main 
instantiate: unknown module
  module name: (submod '#%embedded:math/private/statistics/expected-values: #%contract-defs)
  context...:
   .../rkt/main.rkt:6:0
   body of '#%mzc:main

(Note that --embed-dlls only has effect on Windows, though.)

When raco exe embeds modules, it only includes (sub)modules that are actually required by your compiled program. This is related to the way that a package converted to binary form can be installed without build-deps dependencies, for example.

I don't know how to fix your example, but I also don't know how close your simplified example is to what you actually want to do, so I have a few more thoughts.

One possibility is to make a #lang that encapsulates everything you want to be available to dynamic dependencies, then provide use raco exe ++lang:

++lang ‹lang› — include modules needed to load modules starting #lang ‹lang› dynamically. The ‹lang› does not have to be a plain language or module name; it might be a more general text sequence, such as at-exp racket/base to support language constructors like at-exp. The initial require for a module read as ‹lang› must be available though the language reader’s get-info function and the 'module-language key; languages implemented with syntax/module-reader support that key automatically.

If you end up needing namespace-attach-module, you probably should also use define-runtime-module-path or one of the related forms. (If you need many, consider define-runtime-path-list with the (list 'lib str ...+) or (list 'module module-path var-ref) variants documented under define-runtime-path.)

But I think there may be a deeper misconception:

Some of this may by a simplification for your test, but I wanted to make sure you know that raco exe will embed "a.rkt" in your program as a built-in primitive module with a name like '#%embedded:a:. The module referenced written as (require "a.rkt") in your source code won't have anything to do with any file on disk when you run the result of raco exe main.rkt.

Furthermore, depending on what exactly you mean by "reloading", dynamic-require may not do it regardless of the current-namespace.

One thing you might mean by "if it happens in the same namespace, reloading will not take effect" is that, if a module has already been declared, the existing declaration will be used and the filesystem will never be consulted. Creating a new namespace can help by creating a new module registry. (See the reference section on Module Names and Loading for more specifics, especially the documentation for module name resolvers.) There are many types of plug-in designs for which this can be a good definition of "reloading"!

However, many people use "reloading" to mean something more like redeclaration. If this is what you want, the answer is going to be more complex, and I'd encourage you to reply with more specifics. One option that might be similar to what you're doing now might be dynamic-rerequire, but there are many caveats, including the fact that turning off compile-enforce-module-constants disables optimizations that we usually take for granted. For a high-level solution to code reloading, the reloadable package is one option (it is used by GitHub - racket/racket-pkg-website: A frontend for the Racket Package Catalog., for example):

3

Thanks for your help.

Yes, I made a mistake in the main.rkt, the real module I want require is (require math/statistics), not (require "a.rkt"). The a.rkt won't compiled into main.rkt, actually the main.rkt even don't know it's existence. The path of a.rkt come from user's input when interacting with main.rkt, then the a.rkt file downloaded from server dynamically. It's like this:
contents in main.rkt:

(require math/statistics)
(define-namespace-anchor a)
(define ns (namespace-anchor->namespace a))
(define ROOTDIR "/home/duhaibo/projects/test")
(parameterize ([current-namespace (make-base-namespace)])
  (namespace-attach-module ns 'math/statistics)
;; the path "a.rkt" comes from user's input, not consolidated in the main.rkt
  (let ([my-mean (dynamic-require (make-resolved-module-path (build-path ROOTDIR "a.rkt"))
                                  'my-mean)])
    (my-mean '(1 2 3))))

and a.rkt will be compiled before main.rkt.

I doubt that the error comes from typed racket just because if I don't call the mean function of math/statistics, for example, I change a.rkt like this:

#lang racket/base
(require math/statistics
         racket/vector)
(provide (all-defined-out))
(define (my-mean lst)
  (if (null? lst) 0 (/ (apply + lst) (length (vector->list (list->vector lst))))))

in a.rkt functions in rakcet/vector was called, but no error occurs.

The error is about #%contract-def, which only occurs when using function written in typed racket, and #%contract-def is not a submodule in the source code of expected-values.rkt.