Previous value of a syntax parameter? / block with non-local exits

Previous value a syntax parameter

The block form from ISLisp

Recently I came across the specification of ISLisp.
What Scheme is to Racket, ISLisp is to Common Lisp.

That is, the ISLisp specification is closer in style and size to RnRS
than it is to the Common Lisp specification.

The construct block from ISLisp is like the Racket block but
also supports non-local exit. It is close to let/ec but the
interface is different.

The ISLisp specification of block is as follows:

---
`(block name form*) -> <object>`
`(return-from name result-form)`     *transfers control and data*
---

The `block` special form executes each `form` sequentially from left
to right. If the last `form` exits normally, whatever it returns is
returned by the `block` form.

The `name` in a `block` form is not evaluated; it must be an
identifier. The scope of `name` is the body `form*` — only a
`return-from` textually contained in some `form` can exit the
block. The extent of `name` is dynamic.

If a `return-from` is executed, the `result-form` is evaluated. If
this evaluation returns normally, the value it returns is immediately
returned from the innermost lexically enclosing `block` form with the
same `name`.

`return-from` is used to return from a `block`. `name` is not
evaluated and must be an identifier. A `block` special form must
lexically enclose the occurrence of `return-from`; the value produced
by `result-form` is immediately returned from the block. The
`return-from` form never returns and does not have a value.

Implementing block in Racket

We will call our extension of block for block*.

A simple version of block* is short:

(define-syntax (block* stx)
  (syntax-parse stx
    [(_block* name form ...)
     (syntax/loc stx
       (let/ec name-k
         (syntax-parameterize
             ([return-from (syntax-rules (name) 
                             [(_return-from name e) (name-k e)])])
           (block form ...))))]))

The expansion of block* binds name-k to a procedure we can
call later to exit from the block. If one of the forms
in (block form ...) is (return-from name e) it must expand
into (name-k e). In order to adjust the meaning of return-from
we use syntax-parameterize and bind return-from to the
transformer:

(syntax-rules (name) 
  [(_return-from name e) (name-k e)])

Since name occurs in the list of literals, the pattern
(_return-from name e) only matches the name of block.

At this point we have a block* that works like this:

> (block* here 'a (return-from here 'b) 'c)
c
> (block* here 'a (return-from not-here 'b) 'c)
... an error ...

The problem of nested block* expressions

The simple version of block* has a problem with nested uses.
Our expansion works only for a nested block*-expression, if
it uses return-from with the inner-most name.

> (block* there (list (block* here (return-from here 'a))))
'(a)  ; okay
> (block* there (list (block* here (return-from there 'a))))
'(a)  ; oops, should have been 'a

The culprit is the rule:

(syntax-rules (name) 
  [(_return-from name e) (name-k e)])

The rule only matches the inner-most block name. We need it to match outer
names too. If we could use the name old-return-from for the transformer
for return-from before the new syntax parameterization kicks in, then
we could write:

(syntax-rules (name) 
  [(_return-from name e)       (name-k e)]
  [(_return-from outer-name e) (old-return-from outer-name e)]

An attempt of getting this idea to work failed:

(define-syntax-parameter old-return-from (syntax-rules ()))

(define-syntax (block* stx)
  (syntax-parse stx
    [(_block* name form ...)
     (syntax/loc stx
       (let/ec name-k
         (syntax-parameterize
             ([old-return-from (syntax-rules ()
                                 [(_old-return-from to-name e) (return-from to-name e)])])
           (syntax-parameterize
               ([return-from     (syntax-rules (name) 
                                   [(_return-from name    e) (name-k                  e)]
                                   [(_return-from to-name e) (old-return-from to-name e)])])
             (block form ...)))))]))

The reason this fails is that the expansion of old-return-from expands into a use
of the syntax-parameter return-from. The inner syntax-parameterize takes effect before
the return-from form produced by old-return-from is expanded. This leads to an
infinite expansion loop.

Other attempts of getting hands of the previous transformer for return-from
and using it to define a new version of return-from failed.

Question

How can I use a previous transformer for a syntax parameter to define a new?

A different solution

The solution below introduces a helper form do-block*

(do-block* ((name name-k) ...) form ...))

It works like block* but the first subform is a list
of pairs of block names and associated exit procedures.

Since a nested block* needs to pass along the current list
of block names and exit procedures, block* was made into a
syntax parameter.

#lang racket/base
(require (for-syntax racket/base)
         racket/stxparam
         racket/block)

(define-syntax-parameter return-from
  (syntax-rules ()
    [(_ . _) (error 'return-from "used outside block* form")]))

(define-syntax-parameter block*
  (syntax-rules ()
    [(_block* name form ...)
     (let/ec name-k
       (do-block* ((name name-k)) form ...))]))

(define-syntax do-block*
  (syntax-rules ()
    [(_do-block* ((name name-k) ...) form ...)   
     (syntax-parameterize 
         ([return-from (syntax-rules (name ...) 
                         [(_return-from name e) (name-k e)]
                         ...)]
          [block*      (syntax-rules ()
                         [(_block* -name -form (... ...))
                          (let/ec -name-k
                            (do-block* ((-name -name-k) (name name-k) ...) 
                                       -form (... ...)))])])
       (block form ...))]))

(list 
 (equal? 'a   (block* here  'a))
 (equal? 'b   (block* here  'a 'b))
 (equal? 'b   (block* there 'a (block* here 'b)))
 (equal? 'c   (block* there 'a (block* here 'b) 'c))
 (equal? 'b   (block* here  'a (return-from here 'b) 'c))
 (equal? '(a) (block* there (list (block* here (return-from here  'a) 'b))))
 (equal? 'a   (block* there (list (block* here (return-from there 'a) 'b))))
 (equal? 'a   (block* here (return-from here 'a) (define b (/ 0 0))))
 (equal? 'a   (block* here (return-from here 'a) (error)))
 (equal? 'a   (block
               (define (f1) 
                 (block* b
                   (let ([f (λ () (return-from b 'a))])
                     'big-computation
                     (f2 f))))
               (define (f2 g)
                 'another-big-computation
                 (g))
               (f1))))

I personally probably wouldn't use syntax parameters at all for this, but rather let-syntax with a "block tag" transformer binding, since it's perfectly capable of binding identifiers that the user provides.

#lang racket/base

(require (for-syntax racket/base
                     racket/syntax)
         syntax/parse/define)

(begin-for-syntax
  (struct block-tag (escape-procedure)
    #:property prop:procedure
    (λ (tag stx) (raise-syntax-error #f "block name used outside of context" stx))))

(define-syntax-parser block*
  [(_ name:id body ...)
   #'(let/ec escape-procedure
       (let-syntax ([name (block-tag #'escape-procedure)])
         body ...))])

(define-syntax-parser block-returner
  [(_ name)
   #:declare name (static block-tag? "block name")
   #:with escape-procedure (block-tag-escape-procedure (attribute name.value))
   (with-disappeared-uses
     (record-disappeared-uses #'name)
     #'escape-procedure)])

(define-syntax-parser return-from
  [(_ name:id args ...) #'((block-returner name) args ...)]
  [(_ name:id args ... #:rest restarg) #'(apply (block-returner name) args ... restarg)])

(module+ test
  (require rackunit)
  (check-equal? 'a   (block* here  'a))
  (check-equal? 'b   (block* here  'a 'b))
  (check-equal? 'b   (block* there 'a (block* here 'b)))
  (check-equal? 'c   (block* there 'a (block* here 'b) 'c))
  (check-equal? 'b   (block* here  'a (return-from here 'b) 'c))
  (check-equal? '(a) (block* there (list (block* here (return-from here  'a) 'b))))
  (check-equal? 'a   (block* there (list (block* here (return-from there 'a) 'b))))
  (check-equal? 'a   (block* here (return-from here 'a) (define b (/ 0 0)) (void)))
  (check-equal? 'a   (block* here (return-from here 'a) (error))))
2 Likes

Why not simply:

#lang racket (require racket/block)

(define-syntax (block* stx)
(syntax-case stx ()
((_ return body0 body ...)
#'(let/ec return
body0 body ...))))

(and
(equal? 'a (block* here 'a))
(equal? 'b (block* here 'a 'b))
(equal? 'b (block* there 'a (block* here 'b)))
(equal? 'c (block* there 'a (block* here 'b) 'c))
(equal? 'b (block* here 'a (here 'b) 'c))
(equal? '(a) (block* there (list (block* here (here 'a) 'b))))
(equal? 'a (block* there (list (block* here (there 'a) 'b))))
(equal? 'a (block* here (here 'a) (define b (/ 0 0)) (void)))
(equal? 'a (block* here (here 'a) (error)))
(equal? 'a (block
(define (f1)
(block* b
(let ([f (λ () (b 'a))])
'big-computation
(f2 f))))
(define (f2 g)
'another-big-computation
(g))
(f1)))) ; -> #t

1 Like

Inspired by the solutions of @Eutro and @joskoot :

#lang racket/base
(require (for-syntax racket/base racket/syntax)
         racket/block)

(define-syntax-rule (return-from name e)   (name e))
(define-syntax-rule (block* name form ...) (let/ec name (block form ...)))

(and (equal? 'a   (block* here  'a))
     (equal? 'b   (block* here  'a 'b))
     (equal? 'b   (block* there 'a (block* here 'b)))
     (equal? 'c   (block* there 'a (block* here 'b) 'c))
     (equal? 'b   (block* here  'a (return-from here 'b) 'c))
     (equal? '(a) (block* there (list (block* here (return-from here  'a) 'b))))
     (equal? 'a   (block* there (list (block* here (return-from there 'a) 'b))))
     (equal? 'a   (block* here (return-from here 'a) (define b (/ 0 0)) (void)))
     (equal? 'a   (block* here (return-from here 'a) (error))))

But - I am more interested in a fix for the syntax-parameterize solution.

Why not simply:

Getting rid of return-from is "cheating".
The game is to implement the ISLisp form as-is.

That said, the proposed solution is elegant.

If you really want to, you could use syntax-parameter-value to get the old binding?

(define-syntax (block* stx)
  (syntax-parse stx
    [(_block* name form ...)
     (syntax/loc stx
       (let/ec name-k
         (let-syntax
             ([old-return-from (syntax-parameter-value #'return-from)])
           (syntax-parameterize
               ([return-from (syntax-rules (name) 
                               [(_return-from name e)    (name-k e)]
                               [(_return-from to-name e) (old-return-from to-name e)])])
             (block form ...)))))]))

Though if you're going for an unhygienic version, this is too hygienic, due to how syntax-rules matches literals by checking their binding.

(block*
   there
   (define there 0)
   (return-from there 'a))

If you want it properly unhygienic, this is the closest to how I'd implement it in a more CL-like Lisp, keeping an association of the keys and values on the side:

(define-syntax-parameter block-tags #hash())

(define-syntax-parser return-from
  [(_return-from name value)
   #:with tag-value
   (hash-ref (syntax-parameter-value #'block-tags)
             (syntax-e #'name)
             (λ () (raise-syntax-error #f "no such block in scope" this-syntax #'name)))
   (syntax/loc this-syntax
     (tag-value value))])

(define-syntax (block* stx)
  (syntax-parse stx
    [(_block* name form ...)
     (syntax/loc stx
       (let/ec name-k
         (syntax-parameterize
             ([block-tags (hash-set (syntax-parameter-value #'block-tags)
                                    'name #'name-k)])
           (block form ...))))]))
1 Like

That's it!

         (let-syntax
             ([old-return-from (syntax-parameter-value #'return-from)])

I was so close. I had tried with syntax-local-value.

Your example

(block* there
             (define there 0)
             (return-from there 'a))

shows that there as block tag and there as a variable share the same "namespace".
[Here "namespace" is used in the ISLisp sense.]

Since the game is to implement the ISLisp form, I looked at the spec and it says that block tags have their own namespace.

Maybe we can use make-syntax-introducer make a block tag "namespace".


And later:

When I implemented a Common Lisp style block/return/return-from in my soup-lib package, I used delimited continuations with a different prompt tag for each block name, and stored them in an immutable hash table kept in a regular parameter. Your tests cases (After renaming block* back to block and a few syntax changes) all return true with that approach. Code here if you're interested in comparing the different approaches.

1 Like

It was interesting to see both the implementation and the library.
You have implemented quite a few Common Lisp constructs.

To fix the problem eutro noted yesterday, I implemented a helper let-tag.
The form (let-tag ([name e]) body ...) binds the tag name to the result of e in the body forms. To reference the value associated to the tag name one uses (tag name).

Given this helper, one can implement block* like this:

(define-syntax-parameter return-from (syntax-rules ()))
        
(define-syntax-rule (block* name form ...)
  (let/ec name-k
    (let-tag ([name name-k])
      (syntax-parameterize
          ([return-from (syntax-rules () [(_ name e) ((tag name) e)])])
        (block form ...)))))

The helper is a bit involved. If there is a simpler method (that avoids a maintaining a separate table), I am all ears.

(begin-for-syntax
  (define (make-flip key)
    (let ([mark (make-interned-syntax-introducer (syntax-e key))])
      (λ (id) (mark id 'flip)))))

(define-syntax-parameter tag (syntax-rules ()))

(define-syntax (let-tag stx)
  (syntax-case stx ()
    [(_let-tag ([tag-id e]) body0 body ...)
     (let* ([flip (make-flip #'tag-id)])
       (with-syntax ([tag-id (flip #'tag-id)])
         (flip
          #'(syntax-parameterize
                ([tag (λ (s)
                        (syntax-case s ()
                          [(_tag tid) ((make-flip #'tid)
                                       (syntax-local-identifier-as-binding #'tid))]))])
              (let ([tag-id e])
                body0 body ...)))))]))

The entire source for the block* form with test suite:

#lang racket
(require (for-syntax racket/base) racket/stxparam racket/block)

;;;
;;; let-tag
;;;

; (let-tag ([tag-id e] ...) bodies ...)
;   Evalaute bodies ... where
;   the identifiers tag-id ... are bound in the tag namespace.

; (tag tag-id)
;  If tag-id is bound in the tag namespace, then evaluate to the corresponding value.
;  Otherwise, signal an error.

(begin-for-syntax
  (define (make-flip key)
    (let ([mark (make-interned-syntax-introducer (syntax-e key))])
      (λ (id) (mark id 'flip)))))

(define-syntax-parameter tag (syntax-rules ()))

(define-syntax (let-tag stx)
  (syntax-case stx ()
    [(_let-tag ([tag-id e]) body0 body ...)
     (let* ([flip (make-flip #'tag-id)])
       (with-syntax ([tag-id (flip #'tag-id)])
         (flip
          #'(syntax-parameterize
                ([tag (λ (s)
                        (syntax-case s ()
                          [(_tag tid) ((make-flip #'tid)
                                       (syntax-local-identifier-as-binding #'tid))]))])
              (let ([tag-id e])
                body0 body ...)))))]))

;;;
;;; block*
;;;

; (block* block-name form ...)
;   Like `block` but each block is given a name that can be used
;   for non-local exit.
; (return-from block-name e)
;   Evaluate `e` and return from the block named `block-name`.
;   The result of e is used as the result of the `block*` form..
;   It is an error to use `return-from` outside a `block*` form.
;   It is an error to use `return-from` with a non-block name.

(define-syntax-parameter return-from (syntax-rules ()))
        
(define-syntax-rule (block* name form ...)
  (let/ec name-k
    (let-tag ([name name-k])
      (syntax-parameterize
          ([return-from (syntax-rules () [(_ name e) ((tag name) e)])])
        (block form ...)))))


;;;
;;; Test Suite
;;; 

(and 'let-tag
     (equal? 2          (let-tag ([here 1]) 2))
     (equal? 1          (let-tag ([here 1]) (tag here)))
     (equal? '(1 2)     (let-tag ([here 1])
                          (let ([here 2])
                            (list (tag here) here))))
     (equal? '(1 2 3 4) (let-tag ([there 1])
                          (let-tag ([here 2])
                            (let ([there 3] [here 4])
                              (list (tag there) (tag here) there here)))))
    ;; without `return-from`
    (equal? 'a   (block* here  'a))
    (equal? 'b   (block* here  'a 'b))
    (equal? 'b   (block* there 'a (block* here 'b)))
    (equal? 'c   (block* there 'a (block* here 'b) 'c))
    ;; unnested `return-from`
    (equal? 'b   (block* here  'a (return-from here 'b) 'c))
    (equal? 'a   (block* here  (return-from here 'a) (define b (/ 0 0)) (void)))
    (equal? 'a   (block* here  (return-from here 'a) (error)))
    ;; nested `return-from`
    (equal? '(a) (block* there (list (block* here (return-from here  'a) 'b)))) ; inner exit
    (equal? 'a   (block* there (list (block* here (return-from there 'a) 'b)))) ; outer exit
    ;; separate namespace for block tags 
    (equal? 'a   (block* here (define here 0) (return-from here 'a)))
    (equal? 'b   (block* here (let ([here 'a]) (return-from here 'b) 'c)))
    (equal? 'a   (block* here (let ([here 'a]) (return-from here here) 'c)))
    (equal? 'b   (block* there
                   (list (block* here
                           (let ([there 'a])
                             (return-from there 'b)
                             'c))
                         'd)))
    ; macro generated return-from
    (equal? 42   (let-syntax ([m (syntax-rules ()
                                   [(_ name) (return-from name 42)])])
                   (block* here (m here))))
    (equal? 42   (let-syntax ([m (syntax-rules ()
                                   [(_ name) (return-from name 42)])])
                   (block* there 
                     (list (block* here
                             (m there)))))))

; Note:
;   This gives an "unbound identifier" error.
;       (block* here (return-from there 42))
;   Would be nice to have an "unbound block name" instead.

After thinking about the implementation of let-tag and how the set of scopes model work,
it dawned upon me that tag needs to discard any scopes between the binding of x and the use of tag.

So in short:

  • to bind x in a binding space block-tag we add a scope 'block-tag
    using (make-interned-syntax-introducer 'block-tag)
  • references to x are made through (tag x)
  • since the meaning of tag changes for each new binding, we make it a syntax parameter
  • in (tag x) the only relevant scopes of x are the ones in the block tag binding space,
    so make-syntax-delta-introducer is used to remove any scopes between the binding of x and the reference to x.

Compared to the previous solution, this one uses a single key for the binding space.

#lang racket
(require racket/stxparam)

(begin-for-syntax
  (define introduce (make-interned-syntax-introducer 'block-tag))
  (define (add   stx)      (introduce stx 'add))
  (define (delta ext base) (make-syntax-delta-introducer ext base))
  (define (clean y x)      ((delta y x) y 'remove)))


(define-syntax-parameter tag (syntax-rules ()))

(define-syntax (let-tag stx)
  (syntax-case stx ()
    [(_ x e b)
     (with-syntax ([x* (add #'x)])
       #'(let ([x* e])
           (let-syntax ([old-tag (syntax-parameter-value #'tag)])
             (syntax-parameterize
                 ([tag (λ (s) (syntax-case (clean s #'x) (x) 
                                [(_ x) #'x*]
                                [(_ y) #'(old-tag y)]))])
               b))))]))

; The test returns 1 or (1 2) or (1 2 3) on success. 

(let-tag x 1 (tag x))
(let-tag x 2 (let ([x 1]) x))
(let-tag x 1 (let ([x 2]) (tag x)))
(let-tag x 1 (let ([x 2]) (list (tag x) x)))
(let ([x 1]) (let-tag x 2 (list x (tag x))))
(let-tag x 1 (let-tag y 2 (list (tag x) (tag y))))
(let-tag x 2 (let-tag x 1 (tag x)))
(let-tag x 1 (let ([x 2]) (let-tag y 3 (list (tag x) x (tag y)))))
1 Like