# Ambiguous identifier binding (adding a pass to the reader)

**URL:** <https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298>\
**Category:** Questions & Answers\
**Tags:** macro, reader, binding\
**Created:** [November 28, 2021, 12:54pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298 "2021-11-28T12:54:50Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\
**Post date:** [November 28, 2021, 12:54pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/1 "2021-11-28T12:54:51Z")

</div>

Hi All,

I am experimenting with adding a new pass to the Skething language.  
Instead of

```
read -> expand -> compile -> evaluate 

```

I want to add an `adjust` pass:

```
read -> adjust -> expand -> compile -> evaluate 

```

It's relatively simple to do:

```
(module reader syntax/module-reader
  sketching/main
  #:read read-sketching
  #:read-syntax read-sketching-syntax

  (define (read-sketching [in (current-input-port)])
    (adjust (read in)))

  (define (read-sketching-syntax [source-name (object-name (current-input-port))]
                                 [in (current-input-port)])
    (adjust (read-syntax source-name in)))

```

Now the goal of `adjust` is to detect `id[expr ...]` in the source and rewrite it to `(ref id expr ...)`.  
Whitespace is not allowed between the identifier and the expression.  
Using the source location information and the `paren-shape` syntax property is was pretty simple to detect this situation.

My current problem is that I have run into the following error:

```
; #%top: identifier's binding is ambiguous
; in: #%top
; context...:
; #(496045 module) #(496048 module reader) #(497571 module)
; #(497574 module reader) #(520923 module)
; #(520931 module test-sketching-reader) #(521041 local)
; #(521042 intdef)
; matching binding...:
; #(sketching-top #<module-path-index:"exports-no-gui.rkt" "exports-all.rkt" sketching/main> 0)
; #(520923 module) #(520931 module test-sketching-reader)
; matching binding...:
; #(#%top #<module-path-index:'#%core> 0)
; #(496045 module) #(496048 module reader) #(497571 module)
; #(497574 module reader)
; matching binding...:
; #(#%top #<module-path-index:'#%core> 0)
; #(496045 module) #(496048 module reader)
; Context (plain; to see better errortrace context, re-run with C-u prefix):
; /Users/soegaard/.emacs.d/elpa/racket-mode-20211018.1717/racket/syntax.rkt:66:0

```

The definition of `adjust` is as follows:

```
  (require racket/runtime-path racket/syntax
           (except-in syntax/parse char))
  
  (define (adjust stx)
    (syntax-parse stx
      [(a . d) (adjust-dotted-list stx)]
      [_ stx]))

  (define (ref . xs) ; placeholder
    (cons 'ref xs))
  
  (define (adjust-dotted-list stx)
    (syntax-parse stx
      [(id:id (~and [e:expr ...] brackets) . more)
       (cond
         [(and (eqv? (syntax-property #'brackets 'paren-shape) #\[)
               (= (+ (syntax-position #'id) (syntax-span #'id))
                  (syntax-position #'brackets)))
          (with-syntax ([adjusted-more (adjust #'more)])
            (syntax/loc #'id
              (ref (id e ...) . adjusted-more)))]
         [else
          (with-syntax* ([(_ . rest) stx]
                         [adjusted-rest (adjust-dotted-list #'rest)])
            (syntax/loc stx
              (id . adjusted-rest)))])]
      [(a . more)
       (with-syntax ([adjusted-more (adjust #'more)])
         (syntax/loc stx
           (a . adjusted-more)))]
      [_
       (raise-syntax-error 'adjust-dotted-list "expected a dotted list" stx)]))

```

Where should I look?

It's relevant to mention that `#lang sketching` uses `sketching-top` as `#%top`.  
It is defined in [1].

The full source:

```
(module reader syntax/module-reader
  ; 1. Module path of the language.
  sketching/main
  ; The module path `sketching/main` is used in the language position
  ; of read modules. That is, reading `#lang sketching` will produce a
  ; module with `sketching/main` as language.

  ; 2. Reader options (#:read, #:read-syntax, etc. ...)
  ; Note: When #:read and #:read-syntax are used, they both need to be supplied.
  #:read read-sketching
  #:read-syntax read-sketching-syntax

  ; 3. Forms as in the body of racket/base 

  ; After standard reading, we will rewrite
  ; id[expr ...]
  ; to
  ; (#%ref id expr ...).

  ; We will use this to index to vectors, strings and hash tables.
  
  (define (read-sketching [in (current-input-port)])
    (adjust (read in)))

  (define (read-sketching-syntax [source-name (object-name (current-input-port))]
                                 [in (current-input-port)])
    (adjust (read-syntax source-name in)))

  ; Since adjust is called after reading, we are essentially working with
  ; three passes.
  ; - read-syntax
  ; - adjust
  ; - expand
  
  ; Let's define our `adjust` pass.

  (require racket/runtime-path racket/syntax
           (except-in syntax/parse char))
  ; (require (only-in sketching/main #%top))
  
  (define (read-string str #:source-name [source-name #f])
    (define in (open-input-string str))
    ; (port-count-lines! in)
    (read-syntax source-name in))
  
  (define (adjust stx)
    (syntax-parse stx
      [(a . d) (adjust-dotted-list stx)]
      [_ stx]))

  (define (ref . xs) ; placeholder
    (cons 'ref xs))
  
  (define (adjust-dotted-list stx)
    (syntax-parse stx
      [(id:id (~and [e:expr ...] brackets) . more)
       (cond
         [(and (eqv? (syntax-property #'brackets 'paren-shape) #\[)
               (= (+ (syntax-position #'id) (syntax-span #'id))
                  (syntax-position #'brackets)))
          (with-syntax ([adjusted-more (adjust #'more)])
            (syntax/loc #'id
              (ref (id e ...) . adjusted-more)))]
         [else
          (with-syntax* ([(_ . rest) stx]
                         [adjusted-rest (adjust-dotted-list #'rest)])
            (syntax/loc stx
              (id . adjusted-rest)))])]
      [(a . more)
       (with-syntax ([adjusted-more (adjust #'more)])
         (syntax/loc stx
           (a . adjusted-more)))]
      [_
       (raise-syntax-error 'adjust-dotted-list "expected a dotted list" stx)]))

  ; > (displayln (adjust (read-string "(foo[bar])")))
  ; #<syntax:string::2 (ref (foo bar))>

  ; (displayln (adjust (read-string "(foo [bar])")))
  ; > #<syntax:string::1 (foo (bar))>
  
  )

```

And the definition of `sketching-top` is here:

[1] [sketching/exports-no-gui.rkt at main · soegaard/sketching · GitHub](https://github.com/soegaard/sketching/blob/main/sketching-lib/sketching/exports-no-gui.rkt)

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [November 28, 2021, 2:24pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/2 "2021-11-28T14:24:58Z")

</div>

Normally, a language reader should produce a syntax object with no lexical context, because context is added by the expansion step.

When your language rewrites `id[expr ...]` to `(ref id expr ...)`, what does `ref` mean there? Is it just a `ref` identifier that gets a binding from its context, similar to the implicit `#%app`? Or is it meant to refer always to a specific `ref` binding?

The former is conceptually simpler. The latter can work, and in that case your reader will produce `ref` syntax objects that have context. Still, you want to avoid adding context on other things, such as the parentheses wrapping the `ref` call, since that could lead to an ambiguous `#%app`.

You're getting an ambiguous `#%top` instead of an ambiguous `#%app`, though, so I may not have this quite right. Still, my best guess is that it's something about creating more syntax objects that already have context, and you'll probably need to use more `(datum->syntax #f ....)` than `syntax/loc`.

---

<div class="post-metadata">

**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\
**Post date:** [November 28, 2021, 2:29pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/3 "2021-11-28T14:29:48Z")

</div>

> [@mflatt](#):
>
> When your language rewrites `id[expr ...]` to `(ref id expr ...)` , what does `ref` mean there? Is it just a `ref` identifier that gets a binding from its context, similar to the implicit `#%app` ? Or is it meant to refer always to a specific `ref` binding?

It's meant to work like the implicit `#%app`.

> [@mflatt](#):
>
> You're getting an ambiguous `#%top` instead of an ambiguous `#%app` , though, so I may not have this quite right. Still, my best guess is that it's something about creating more syntax objects that already have context, and you'll probably need to use more `(datum->syntax #f ....)` than `syntax/loc` .

I'll try a version with `datum->syntax`.

---

<div class="post-metadata">

**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\
**Post date:** [November 28, 2021, 4:28pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/4 "2021-11-28T16:28:39Z")

</div>

The operation was successful - and the patient lived.

Since `read` and `read-syntax` work on individual expressions, the right place to apply `adjust` was the module wrapper.

The problem of the ambiguous `#%app` was solved with `(datum->syntax #f ...)` instead of `syntax/loc`.

Thanks for the pointer.

For reference:

```
(module reader syntax/module-reader
  ; 1. Module path of the language.
  sketching/main
  ; The module path `sketching/main` is used in the language position
  ; of read modules. That is, reading `#lang sketching` will produce a
  ; module with `sketching/main` as language.

  ; 2. Reader options (#:read, #:read-syntax, etc. ...)
  #:module-wrapper (λ (thunk) (adjust (thunk)))

  ; 3. Forms as in the body of racket/base 

  ; After standard reading, we will rewrite
  ; id[expr ...]
  ; to
  ; (#%ref id expr ...).

  ; We will use this to index to vectors, strings and hash tables.
  

  ; Since adjust is called after reading, we are essentially working with
  ; three passes.
  ; - read-syntax
  ; - adjust
  ; - expand
  
  ; Let's define our `adjust` pass.

  (require racket/runtime-path racket/syntax
           (except-in syntax/parse char))
  
  (define (read-string str #:source-name [source-name #f])
    (define in (open-input-string str))
    ; (port-count-lines! in)
    (read-syntax source-name in))
  
  (define (adjust stx)
    (syntax-parse stx
      [(a . d) (adjust-dotted-list stx)]
      [_ stx]))
  
  (define (adjust-dotted-list stx)    
    (syntax-parse stx
      [(id:id (~and [e:expr ...] brackets) . more)
       (cond
         [(and (eqv? (syntax-property #'brackets 'paren-shape) #\[)
               (= (+ (syntax-position #'id) (syntax-span #'id))
                  (syntax-position #'brackets)))
          (let ([adjusted-more (adjust #'more)]
                [arguments (syntax->list #'(id e ...))])
            (datum->syntax #f
                           `((#%ref ,@arguments) . ,adjusted-more)
                           stx))]
         [else
          (with-syntax ([(_ . rest) stx])
            (let ([adjusted-rest (adjust-dotted-list #'rest)])
              (datum->syntax #f
                             `(,#'id . ,adjusted-rest)
                             stx)))])]
      [(a . more)
       (let ([adjusted-a (adjust #'a)]
             [adjusted-more (adjust #'more)])
         (datum->syntax #f
                        `(,adjusted-a . ,adjusted-more)
                        stx))]
      [_
       (raise-syntax-error 'adjust-dotted-list "expected a dotted list" stx)]))

  ; > (displayln (adjust (read-string "(foo[bar])")))
  ; #<syntax:string::2 (ref (foo bar))>

  ; > (displayln (adjust (read-string "(foo [bar])")))
  ; #<syntax:string::1 (foo (bar))>

  ; > (displayln (adjust (read-string "(foo v[1] bar)")))
  ; #<syntax:string::1 (foo (#%ref v 1) bar)>

  )

```

---

<div class="post-metadata">

**Author:** ![SamPhillips](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/samphillips/32/15_2.png) [@SamPhillips](https://racket.discourse.group/u/SamPhillips)\
**Post date:** [November 28, 2021, 9:49pm UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/5 "2021-11-28T21:49:33Z")

</div>

In some cases [`strip-context`](https://docs.racket-lang.org/syntax/syntax-helpers.html#%28def._%28%28lib._syntax%2Fstrip-context..rkt%29._strip-context%29%29) can probably be used too.

---

<div class="post-metadata">

**Author:** ![soegaard](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/soegaard/32/19_2.png) [@soegaard](https://racket.discourse.group/u/soegaard)\
**Post date:** [November 29, 2021, 8:39am UTC](https://racket.discourse.group/t/ambiguous-identifier-binding-adding-a-pass-to-the-reader/298/6 "2021-11-29T08:39:48Z")

</div>

I had forgotten about `strip-context`!

At least the `datum->syntax` solution saves a tree traversal.