Running a procedure from 'delay' does not capture modified parameters

I'm having an issue where changes to parameters are not being noticed in a new thread. I think I know what's happening but would appreciate confirmation and suggestions on the best way to handle it.

  • In server.rkt I parse configuration data from the command line and other sources, combine them, and use them to set some parameters such as db-config-filepath which tells the system which database to use.

  • For testing I am running two instances of the server in parallel, one named 'alice' and the other named 'bob'. Each is run in a separate terminal with separate command line arguments.

  • The default DB path is db.conf but for testing I have db.alice.conf and db.bob.conf, each of which points the relevant peer to its own database.

  • When messages come into the server from another peer (e.g. when alice sends a message to bob), the message gets handled in a thread. These threads try to conclude as rapidly as possible in order to minimize the delay of responding to the other peer; long-running tasks get pushed into a job queue to be run separately as soon as a worker is available .

  • Tasks are added to the queue via a promise. Here is a simplified version of the relevant code from files.rkt:

       #lang racket/base
       (define jarvis (start-majordomo #:max-workers 5)) ; task manager instance
       (define (handle-file-upsert-message args)
          (log-files-debug "db conf is: ~a" (db-config-filepath))
          (add-task (delay ...code goes here...) args)
          (log-files-debug "db conf is: ~a" (db-config-filepath)))
  • I'm defining the task manager at the top of the file because I want there to be a single task manager instance shared across multiple message-handling threads. (Recall that each thread is created in response to a different message received by the server as alice and bob talk to each other.)

  • Both of the logging lines shown above have the correct db-config-filepath value (in this case db.alice.conf) but the code inside the promise has the default db.conf

(Note: As mentioned, this is simplified; although db-config-filepath is the parameter that led me to notice the issues, there are other parameters that are also not being updated. It's therefore not as simple as passing an already-initialized db connection.)

I think what's happening is this:

  1. jarvis is created at startup time before the configuration data has been parsed and the parameters updated. jarvis therefore starts with the default values for all the parameters and for whatever reason it does not notice the updates.
  2. Inside add-task, a new thread is created and the promise is forced inside it. As standard, this thread receives copies of all parameters as known to the surrounding context. Since 'the surrounding context' is the jarvis instance that is using the unconfigured default values, that means that the thread has the default db-config-filepath value and so it ends up talking to the default DB instead of alice's db.

Does this sound right? If so, what would y'all suggest for how to fix it -- would it be as simple as using a thunk instead of a promise, or is there a better solution?

Does the problem occur in both CS and BC?

I have a problem with parameters in CS.

This problem does not occur in BC.

The program (a toy interpreter) is to large to show here,

It is at

My 2 cents

Jos Koot

Can you fill a but report so this problem is not forgoten?

It should be easier to fill because all your code is public.To see the error, it's enough to tun simplisp/trace-source-code.rkt at main · joskoot/simplisp · GitHub ? Is it possible to run it with a smaller expression instead of source-code so the output is not huge?

(It's nice to have a minimized example instead of a full repo, but for me it's good enough to get a reproducible foolproof instructions.)

Thanks gus-massa

Currently trace-source-code.rkt disables some trace options when run on Racket CS.

I can undo the disabling in order to reproduce the problem.

The module requires simplisp.rkt.

If you don’t want to install the package, but simply download the two files,

line (require simplisp/simplisp) must be replaced by (require “simplisp.rkt”).

It is not easy to make a minimized example,

for the problem occurs only when the interpreter is tracing its own source code.

This requires that the interpreter is capable of interpreting the code that yields the problem.

Nevertheless I’ll have a look what parts of the code can be removed.

I’ll do that in the coming days.

The output can be reduced by replacing (trace-option ‘all) by (trace-option ‘(value)).

Anyway, with (trace-option ‘all) Racket CS produces no output.

It simply quits within a second or two without leaving any message

and without using much resources.

BTW, what is a but report? A bug report? How do I post it?

DrRacket no longer has an option to send a bug report from its help menu.

Thanks again, Jos Koot

I tried this program:

#lang racket

#;(require simplisp/simplisp)
(require "simplisp.rkt")

; Running this module produces almost 100000 lines of output.

 '(trace-align '5)
 '(trace-width 90)
 '(trace-option 'all)

In DrRacket I got a loooooooooooooot of lines in 8.2[3m] . But in 8.2[cs] I got a few lines and a segfault. I don't have both versions of 8.4 to try now.

Just post an issue in Issues · racket/racket · GitHub explaining the problem and with a link to your repo and this program, so it's possible to use cut and paste see the problem.

It would be nice if you replace in the issue

'(trace-option 'all)

with a variant of

'(trace-option '(start finis selfi))

that prints less lines but still shows the difference between cs and bc.

Yes, that's right. Parameters are wrappers around thread-local state. Using a thunk instead of a promise wouldn't make a difference; when you run the code, it will see the parameter values of the thread it's running in.

One solution would be to replace

(delay ...code goes here...)


(let ([saved-db-config-filepath (db-config-filepath)])
  (delay (parameterize ([db-config-filepath saved-db-config-filepath])
    ...code goes here...))

And if your code wants to carry over other parameters, you have to save and restore them too. You can of course create a function or macro that does that for you, but you need to tell it what parameters to capture.

(In my RacketCon talk I said that I thought parameters were overused. This is one reason why.)


[Edit: Ryan's reply already covered this while I was typing it in.]

I'm not sure I follow, but it sounds like you're describing a situation like this:

(define param (make-parameter 'p-default))
(define var 'v-default) ; just to contrast with `param`

(define th
  (thread (lambda ()
            (sync (system-idle-evt)) ; waits until main thread is done
            (printf "~s vs. ~s\n" (param) var))))

(param 'p-new)
(set! var 'v-new)

(sync th)

This example will print "p-default" and "v-new", because parameter mutation is thread-specific. Is that what you mean?

If this example is the kind of problem you're seeing, using a thunk instead of a promise won't change anything, because it's about how parameters and threads interact. If the goal is to share state across threads, you may have to use a mutable variable, mutable box, or something like that instead of a parameter.

1 Like

I'm seeing two bugs in Racket CS when running simplisp:

  • An applicable structure as a prop:object-name value is not handled correctly. That's the source of the crash.

  • A structure type's prop:object-name procedure is called to get a printing name for the structure type itself (which is not a correct use of the property value). This bug only show up after the other one is fixed.

I'll push repairs for these problems.


Indeed, after commenting out prop:object-name clauses all goes well, both in CS and BC.
I look forward to the next snapshot.
Thanks, Jos Koot

Alternatively, if you want to indiscriminately capture and restore all parameters, you could write:

(let ([saved-paramz (current-parameterization)])
  (delay (call-with-parameterization saved-paramz
           (λ ()
             ...code goes here...))))

I've also come to at least a soft version of this view. Two examples that particularly get to me:

  1. The way the json library uses (json-null) means that even the result of jsexpr? depends on a relatively obscure piece of mutable state, which makes it difficult/expensive to check as a contract. The typed/json library works around this with wrapper functions that supply #:json-null 'null.
  2. I once wrote some #lang web-server code that had a rather bad bug until Jay explained to me on the mailing list that:
    • A continuation captures the values of parameters at the time it is
      captured. So, if your parameterize is outside of the dispatcher that
      calls the continuation, then the code inside the continuation will not
      see the parameter change. That's a pretty important feature of

    • Web parameters would work the same way as normal parameters and not help you.

    • The stateless Web language guarantees that the parameters will have
      values they had when the servlet was initialized.

    • I think that you may want to use a thread-cell as the storage place.

1 Like

That's a good point, and it works as long as you only parameterize parameters instead of setting them directly. But if you mix the two modes, you can still get surprises. Here's an example:

(define param (make-parameter 'original))
(define chan (make-channel))
(void (thread (lambda () (let loop () ((sync chan)) (loop)))))

(parameterize ((param 'changed))
  (channel-put chan (lambda () (printf "#1 ~v\n" (param)))))
;; prints "#1 'original" -- this is the original problem

;; capture-params : (-> X) -> (-> X)
(define (capture-paramz thunk)
  (define paramz (current-parameterization))
  (lambda () (call-with-parameterization paramz thunk)))

(parameterize ((param 'changed))
  (channel-put chan (capture-paramz (lambda () (printf "#2 ~v\n" (param))))))
;; prints "#2 'changed" -- fixed!

(parameterize ((param 'changed-once))
  (param 'changed-twice)
  (channel-put chan (capture-paramz (lambda () (printf "#3 ~v\n" (param))))))
;; prints "#3 'changed-once" -- doesn't see update after parameterize!

And for fun:

(parameterize ((param 'changed-here))
  (define continue (make-semaphore 0))
  (channel-put chan
                (lambda ()
                  (printf "#4.1 ~v\n" (param))
                  (param 'changed-over-there)
                  (semaphore-post continue))))
  (semaphore-wait continue)
  (printf "#4.2 ~v\n" (param))
  (channel-put chan (capture-paramz (lambda () (printf "#4.3 ~v\n" (param))))))

Those are good examples. The second one, especially, illustrates that sometimes when you mix two different kinds of magic (aka effects), you have to think carefully about how they interact.

1 Like

To gus-massa and with thanks to Matthew Flatt.

With the info of Matthew Flatt I was able to reduce the code.
The following reveals the problem as explained by Matthew Flatt and does not produce much output.
Uncomment the subexpression marked by arrow to produce the problem.
I am aware of the fact that this problem is not related to the original post in this thread.

#lang racket

(require simplisp/simplisp)

(simplisp '(trace-option '(varef)) '(trace-width 50) 

'(letrec-values ; Notice the quote.
  ((($trace-option) (make-parameter #t))
   ((inspector) (make-sibling-inspector))
   ((super-type super-type? super-type-name set-super-type-name!)
     (((descr constr pred acc mut)
        'super-type ; type-name
        #f          ; no super-type
        1           ; fields: name (other fields to be provided by sub-types)
        0           ; no auto fields
        #f          ; auto value n.a.
       ; Uncommenting the following subexpr reveals the problem.
      #; (cons prop:object-name
          (λ (obj) (parameterize* (($trace-option #f)) (super-type-name obj)))))
        #f          ; no procedure properety
        '()         ; mutable name
        #f)))       ; no guard
     (values descr pred
      (make-struct-field-accessor acc 0 'name)
      (make-struct-field-mutator  mut 0 'name))))

   ((make-closure $closure?)
       (λ (obj port mode)
        (parameterize* (($trace-option #f))
         (fprintf port "#<closure:~s>" (super-type-name obj)))))
      ((descr constr pred acc mut)
        'closure ; type-name
        1        ; nr of fields: procedure, name already in super-type
        0        ; no auto fields
        #f       ; auto-value n.a.
        (list (cons prop:custom-write printer))
        0        ; procedure property
        '(0)     ; immutable proc
        #f)))    ; no guard
     (values constr  pred)))

   ((noot) (make-closure 'aap (lambda (x) x))))
  (displayln (noot "this is displayed"))

It's nice that both problem were fixed. Remember that if you find a bug, you can post it in github (or here). They are very helpful to fix corner cases that have errors.

I'm embarrassed to say I'm overdue catching up on talks from that RacketCon, including yours. :frowning_face: So apologies if this is an impoverished version of something you already said.

I think sometimes people just like the signature of parameters, without wanting or remembering the thread-local aspect. In other words they want something like:

(define my-signature-is-like-a-parameter
  (let ([val 'some-default])
    (case-lambda [() val]
                 [(new) (set! val new)])))

I have sometimes done this. Sometimes wrapped in a contract, sometimes not.

I freely admit the objection, "Using a set! everywhere would be clearer and/or more honest".

In any case it's a thing that some people sometimes do.

That reminded me of `fluid-let’s.

Also a variant without the threadlocal aspect..

Thank you all for helping me get my head around this. For the benefit of anyone else who stumbles over this thread, here's what I've taken away:

#lang racket

(define name (make-parameter 'bob))
(define db-path (make-parameter 'orig))
(define orig-parms (current-parameterization))

;  Pointless use of call-with-parameterization since no params have changed
; outputs: 1 orig. bob
 (thunk (displayln (~a 1 " " (db-path) ". " (name)))))

; parameterize db-path, leave name untouched
; outputs: 2 new. bob
(parameterize ([db-path 'new])
  (displayln (~a 2 " " (db-path) ". " (name))))

; outside of the scope of the parameterize, so db-path has been automatically reset
; outputs: 3 orig. bob
(displayln (~a 3 " "  (db-path) ". " (name)))

; set (as opposed to parameterize) db-path
(db-path 'new)

; parameterize name, then call with the original parameterization which did not include 
; this and will therefore show the original value for name.  Essentially, we update the
; value of name and then immediately override it back to its original value.
; outputs: 4 new. bob
(parameterize ([name 'alice])
   (thunk   (displayln (~a 3 " " (db-path) ". " (name))))))
1 Like

Oh, hey, here's a question:

Given that assigning directly to a parameter (e.g. (db-path "db.conf")) causes issues, what's the right way to handle command line arguments? The examples for command-line all work via assigning to parameters.

Ohhhhkay. Thought I had a handle on this and apparently I do not.

#lang racket

(define users (make-parameter '()))
(users (list "bob")) ; assign
(define outer (current-parameterization))

(parameterize ([users (list "alice")]) ; parameterize
  (define inner (current-parameterization))
  (println (users)) ; should print '("alice"). Works as expected.                                                              
    ; should print '("alice") because that was parameterized in                                               
    (println (format "inner users: ~v" (users))))) ; Works as expected.

    ; should print '("alice") because assignment is not captured by                                           
    ; current-parameterization so the (users (list "bob") should not                                          
    ; be seen                                                                                                 
    (println (format "outer users: ~v" (users)))))) ; prints '("bob") ??!?!?!?!

The second call-with-parameterization prints '("bob"), meaning that it did in fact capture the assignment. I thought it wasn't supposed to do that -- what am I missing?

Given that assigning directly to a parameter (e.g. (db-path "db.conf")) causes issues, what's the right way to handle command line arguments? The examples for command-line all work via assigning to parameters.

When I use parameters for command-line arguments, they aren't
application-level; rather, they are scoped very specifically to the
invocation of the command-line parser and the subsequent actions in
the main (sub)module that is the program.

For application-level parameters, I think parameterize is to be
preferred. But of course the greatest challenge is handling dynamic
scope: there's a reason PL has moved away from that by-and-large.
Parameters exist (I think) to re-enable it. I think this may be the
cause of the confusion in the original question; without reading too
closely, it sounds like there was confusion about which "dynamic
extent" (am I using this correctly?) some code in a thread had (or
"was under"?). And of course "dynamic extent" is far more than just
the immediate body forms of parameterize; it includes all code
that executes as a result of those body forms, excepting any

Perhaps this complexity is why some think they are over-used?


I haven't checked the reference, but I'm assuming current-parameterization does in fact capture the current values of all parameters, whether they were set via parameterize or the setter-procedure. If you move the (define outer before the line marked ; assign I would expect the empty list to be printed at the end.