Need to use module registry lock with `read-language`?

General
1

I have a bug against Racket Mode that was very surprising and confusing to me: #768. A situation with many instantiate-linklet: mismatch errors in color-lexers.

I believe I finally boiled it down concurrency when using read-language, as well as calling the returned get-info procedure with e.g. a color-lexer key, which loads modules.

At first I fixed (?) this using a general purpose semaphore. Then I remembered something about a module registry lock, i.e. namespace-call-with-registry-lock, which also fixes it (AFAICT).

Here's a minimal-ish example:

#lang racket/base

(define source "#lang rhombus\ndef value = 1\n")

(define (read-language-and-get-color-lexer)
  (define get-info (read-language (open-input-string source)))
  (get-info 'color-lexer #f))

(define (read-language-and-get-color-lexer/with-lock)
  (namespace-call-with-registry-lock (current-namespace)
                                     read-language-and-get-color-lexer))

;; Change this to see many errors, or not. Note that, after running
;; with #t, a given REPL session may produce errors even with
;; #f, until you refresh the session and get a new namespace.
(define want-to-see-many-instantiate-linklet-errors? #t) ;; <====

(for ([_ 5])
  (thread (if want-to-see-many-instantiate-linklet-errors?
              read-language-and-get-color-lexer
              read-language-and-get-color-lexer/with-lock)))
(sleep 1)

;; Background story:
;; https://github.com/greghendershott/racket-mode/issues/768

The concurrency vulnerable spot seems to be between calling read-language and calling the returned get-info procedure value -- another thread could muck with the namespace's module registry, there?

Beside sharing the experience, I guess my questions are:

  1. Is this as-expected -- use of read-language by multiple threads should be guarded by the module registry lock? (Or, using a distinct current-namespace for each thread.) IOW does it sound like I finally arrived at the likely solution?

  2. If so, should I do a doc PR for read-language to point this out, as a hint (or is that considered too obvious, and/or the consumers of this documentation too few, to make it worthwhile)?

2

I believe (after some AI-assisted investigation) that the calls to dynamic-require in the rhombus get-info code should use the registry lock themselves, along these lines: Comparing racket:master...samth:get-info-registry-lock · racket/rhombus · GitHub

3

It's true that Rhombus uses dynamic-require in get-info, but Datalog, 2D Syntax, Algol 60 and Racklog all do this too. So it looks like not worrying about module registry in get-infos has become a convention.

4

This FrosthavenManager commit using the registry lock may be of interest. I later was able to change strategies entirely (see the commits linked to the corresponding issue](Is it possible to record AoE specs as dependencies of bestiaries? · Issue #108 · benknoble/frosthaven-manager · GitHub))), but I found using the registry lock to be important in multi-thread-ed code.

1 Like
5

I agree, and the docs actually suggest doing this without a lock: 17.3.5 Source-Handling Configuration

However, the docs for dynamic-require say that a lock is needed: "Beware that concurrent dynamic-requires in namespaces that share a module registry can create race conditions; see also namespace-call-with-registry-lock."