Running a procedure from 'delay' does not capture modified parameters

Here is an implementation of parameters and parameterizations in Racket. This is a simplified version of the code in racket/src/cs/rumble/parameter.ss.

;; A MyParamz is (hasheq MyParam ThreadCell)

;; Rather than having one huge, changing hash for the initial
;; parameterization, the initial thread cell for a parameter is
;; stored in the parameter itself.

;; A MyParam is (myparam ThreadCell)
(struct myparam (init-cell)
  #:property prop:procedure
  (case-lambda
    [(self) (thread-cell-ref (get-param-cell self))]
    [(self newval) (thread-cell-set! (get-param-cell self) newval)]))

;; make-myparam : Any -> MyParam
(define (make-myparam init-val)
  (myparam (make-thread-cell init-val #t)))

;; The 'myparamz continuation mark key is mapped to MyParamz values.

;; current-myparamz : -> MyParamz
(define (current-myparamz)
  (or (continuation-mark-set-first #f 'myparamz) (hasheq)))

;; call-with-myparamz : MyParamz (-> Any) -> Any
(define (call-with-myparamz mypz proc)
  (with-continuation-mark 'myparamz mypz (proc)))

;; (myparameterize ((Expr[MyParam] Expr) ...) Body ...) : Expr
(define-syntax-rule (myparameterize ((myp val) ...) body ...)
  (call-with-myparamz
   (hash-set* (current-myparamz) (~@ myp (make-thread-cell val #t)) ...)
   (lambda () body ...)))

;; get-param-cell : MyParam -> ThreadCell
(define (get-param-cell myp)
  ;; If the current myparamz has an updated mapping, use that;
  ;; otherwise, use the myparam's original thread cell.
  (or (hash-ref (current-myparamz) myp #f)
      (myparam-init-cell myp)))

And here is the recent example translated to use that implementation. I've also added some more printing.

(define users (make-myparam '()))
(users (list "bob"))
(define outer (current-myparamz))
(printf "1: outer = ~v\n" outer) ;; => (hasheq)

(myparameterize ([users (list "alice")]) ; myparameterize
  (define inner (current-myparamz))
  (printf "2: inner = ~v\n" inner) ;; => (hasheq users #<thread-cell>)
  (printf "3: users = ~v\n" (users)) ;; => '("alice")

  (call-with-myparamz
   inner
   (lambda ()
     ;; => #<thread-cell>
     (printf "4a: (current-myparamz) maps users to ~v\n"
             (hash-ref (current-myparamz) users #f))
     ;; => '("alice")
     (printf "4b: inner users = ~v\n" (users))))

  (call-with-myparamz
   outer
   (lambda ()
     ;; => #f
     (printf "5a: (current-myparamz) maps users to ~v\n"
             (hash-ref (current-myparamz) users #f))
     ;; => '("bob"), value in initial thread cell
     (printf "5b: outer users = ~v\n" (users)))))

No, it doesn't capture the values of the parameters. It only captures the mapping of parameters to thread cells.

2 Likes

Thanks Ryan. I'm going to have to stare at that a bit longer to be sure I actually grok it, but it makes sense at a surface level.

I see; so in fact the reason to print the result of the set is multi-part:

  1. The outer parameterization is empty because none is in effect, and
  2. When a parameter's value is retrieved, if the current parameterization does not map it to a specific thread cell, the original is used (presumably this is what allows nesting to work as expected).

This has been instructive.

1 Like

After digging through this it makes a lot more sense to me; thank you very much.

I do still have one thing I'm confused about. In your example code, get-param-cell retrieves the relevant thread cell, preferring the copy that's stored in the parameterization but defaulting to the original one in the parameter itself. Why then do I see this?:

#lang racket
(define ch (make-channel))

(define (show-users n parms)
  (call-with-parameterization
   parms
   (thunk
    (displayln (~a n ": users is: " (users))))))

(thread (thunk  (let loop ()    (apply show-users (channel-get ch))    (loop))))

(define users (make-parameter '(alice)))

(channel-put ch (list 1 (current-parameterization)))

(users '(tom))
(channel-put ch (list 2 (current-parameterization)))

(parameterize ([users '(bob)])
  (channel-put ch (list 3 (current-parameterization)))
  (users '(charlie))  ; XXXX
  (channel-put ch (list 4 (current-parameterization)))
  (sleep 0)
  (displayln (~a 5 ": users is: " (users))))

; output:
;  1: users is: (alice)
;  2: users is: (alice)
;  3: users is: (bob)
;  4: users is: (bob)
;  5: users is: (charlie)

1, 2, 3, make sense to me but I'm confused about 4 and 5.

  • In 1 and 2 there was as yet an empty parameterization and so it defaulted to using the param's thread cell.
  • In 3 it used the value that the call to parameterize had installed in the parameterization.
  • I would have expected that the line marked 'XXXX' would update the the thread cell installed in the parameterization, leaving the original parameter unchanged. Instead, 4 makes clear that it updated the original parameter, leaving the parameterization untouched.
  • In 5 I would have expected it to use the value from the parameterization the way it did on line 3, but instead it's pulling from the original parameter.

I dug around in racket/src/cs/rumble/parameter.ss to see if that would clarify but it did not. What am I missing?

Right. To be more precise, in 3 it used the new thread cell installed by the call to parameterize. That thread cell has the value '(bob) for all threads. The parameter's original thread cell has the value '(tom) in the main thread and '(alice) in all other existing threads. Remember, it's a thread cell, not a box.

It does mutate the new thread cell. That thread cell has the value '(charlie) in the main thread and it still has the value '(bob) in all other existing threads, including the thread listening to ch. That's why that thread prints "4: users is: (bob)".

It uses the thread cell from the parameterization, which has the value '(charlie) in the main thread.

I'm sorry for being so dense here, and I appreciate your patience. In particular, I was being sloppy before when I said 'value of the parameter' when I meant 'value stored in the thread cell associated with the parameter procedure'.

Let me break this down and hopefully you can tell me exactly which bit(s) I'm getting wrong. This is what I understand after reading 1.1 Evaluation Model, the simplified code you were kind enough to send me, and the contents of parameter.ss and thread-cell.ss:

  1. A thread can have zero or more thread cells
  2. A thread cell contains a value and a boolean specifying whether or not the thread cell is preserved
  3. By default, thread cells are marked preserved
  4. A thread cell exists in exactly one thread
  5. A parameter is a wrapper procedure around a parameter-data struct and an associated thread cell. The procedure can be used to retrieve or mutationally update the contents of the thread cell. When I do (user 'alice), I am effectively set!-ing the contents of the thread cell associated with that parameter
  6. After mutationally updating a parameter (e.g. (user-name 'bob)), the new value becomes the 'initial value' for purposes of what gets used to initialize the thread cells of a new thread
  7. A thread may have zero or more parameterizations
  8. A parameterization is a struct containing a hasheq that maps parameters to thread cells
  9. The thread cells stored in a parameterization are copies of the ones associated with the relevant parameter
  10. Parameterizations are stored in a continuation frame
  11. Using parameterize functionally updates the current parameterization -- i.e. it creates a new thread cell using the value that was specified in the parameterize call, it does not mutate the referenced parameter's thread cell
  12. When accessing or mutating a parameter, Racket will use the thread cell from the current parameterization if there is one. If there is not, it will use the 'free floating' one that exists but is not in the parameterization
  13. I couldn't find the exact code for (thread thunk), but my understanding is that when thread X spawns thread Y, thread Y will be instantiated with:
    A. A copy of X's current parameterization (which might be empty)
    B. A copy of every thread cell that is not in the current parameterization
    C. In each case, the copy of the thread cell will be instantiated with thread X's thread cell's current value (if that thread cell is marked preserved) or the initial value of that thread cell (if it is not).
    D. Thread Y will also receive a copy of each parameter procedure, set to point at the appropriate copied thread cell

I'm sure some of this is wrong; which bits?

No problem. I'm glad you're persistent, and the concrete examples are useful.

Points 1, 3, and 4 are wrong. In particular, a thread cell is a Racket value. How could a Racket value only exist in one thread? In fact, the point of a thread cell is to exist in multiple threads and to potentially contain different values in each thread.

Conceptually, a thread cell contains a mutable mapping from threads to values, and there is an entry in this mapping for every existing thread. Code running in a specific thread can only get and set that thread's entry in the thread cell. That isn't how thread cells are actually implemented, but I recommend ignoring the implementation, at least for now.

The preserved flag determines the value for new threads. Your example has a fixed number of threads and all of the threads exist before you start changing parameters, though, so the preserved flag doesn't matter. Let's ignore it for now.

Your program has two threads, so you can think of every thread cell it creates as a mutable mapping containing two entries. When a new thread cell is created with an initial value, both threads are mapped to that value. When a thread cell is mutated, only one entry is updated. See my previous message, where I describe the state of thread cells after updates.

Re 5: The procedure mutates a thread cell, but not necessarily the parameter's initial thread cell. The mutation is done with thread-cell-set!, which is different from set! (it only updates one entry in the conceptual thread->value mapping).

Re 9: No, they aren't copies. I agree with the way you say it in 11.

Re 13: (A) Why copy? Y's initial parameterization is just a reference to X's current parameterization (I think). (B) No copies. Conceptually, the mapping of every thread cell is extended with a new entry for the new thread. The new value is determined as you describe in C. (D) No. Every thread that has a reference to users sees exactly the same parameter object. Parameters behave differently in different threads because of the parameterization and thread-cell indirections.

2 Likes

How is it that English has a word for "the smell of the first rain on warm dust after a long dry spell" (petrichor), but it does not have a word for "the feeling of frustration+hope derived from discovering that you thought you had a reasonably solid understanding of something, which turned out to be completely wrong, but now you have the opportunity to learn the correct version"?

3 Likes