Macro Monstrosity: Some Questions

Questions & Answers
1

Hi, Racket Discourse.

I am busy writing an "interface" which allows me to use a macro-defined template to access the content of the equivalents of JSON objects, i.e., hasheq tables, returned by http-easy's response-json from API endpoint responses.

This is mostly for my own use, so I am just playing around, but nonetheless I came across some interesting things that I'd like to ask about.

The code below demonstrates the usage of the macro, called define-hash-interface, which itself defines a macro called with-hash-interface used to access the template defined by the interface.

The macros are invoked using the forms:

(define-hash-interface NAME
  FETCH: TREE ...+ WHERE: (PNAME means PRED?) ...)
;; and
(with-hash-interface (NAME HASH)
  BODY ...)

The FETCH: TREE ...+ specifies the structure of the expected hash-table and the WHERE: (PNAME means PRED?) ... specifies any extra predicates that you might want to "bundle" with the interface.

#lang racket

(require (for-syntax racket/list
                     racket/string
                     racket/match
                     racket/syntax
                     syntax/parse))

(begin-for-syntax
  (define (walk-key-tree key-tree)
    (match key-tree
      [(list (? symbol? c) 'is type?)
       (list (list c type?))]
      [(list (? symbol? head) content ..1)
       (map (lambda (x) (cons head x)) (append* (map walk-key-tree content)))]
      [_
       (error 'walk-key-tree "expected valid key-tree, got ~v." key-tree)]))

  (define (format-ids lctx symbols)
    (for/list ([id symbols])
      (format-id lctx "~a" id)))

  (define (dot-keys key-symbols)
    (string->symbol (string-join (map symbol->string key-symbols) "."))))

(define (get-in h keys)
  (define (getter key)
    (lambda (g) (hash-ref g key)))
  ((apply compose1 (map getter (reverse keys))) h))

(define-syntax (define-hash-interface stx)
  (syntax-parse stx
    #:datum-literals (FETCH: WHERE: means)
    [(define-hash-interface NAME
       FETCH: TREE ...+ WHERE: (PNAME means PRED?) ...)
     (with-syntax* ([((key ... pred?) ...)
                     (append* (map walk-key-tree (syntax->datum #'(TREE ...))))]
                    [(dot-id ...)
                     (format-ids stx (map dot-keys (syntax->datum #'((key ...) ...))))]
                    [interface-handle
                     (format-id stx "~a" 'with-hash-interface)])
       #'(define-syntax (interface-handle stx)
           (syntax-parse stx
             [(interface-handle (NAME HASH) BODY (... ...))
              (with-syntax ([(dot-id ...)
                             (format-ids stx (map dot-keys (syntax->datum #'((HASH dot-id) ...))))])
                #'(let ([PNAME PRED?] ...)
                    (define (dot-id)
                      (define val (get-in HASH (list 'key ...)))
                      (unless (pred? val)
                        (error 'with-hash-interface
                               "~a expected value for which ~a is true, got ~v." dot-id pred? val))
                      val) ...
                    BODY (... ...)))])))]
    [(define-hash-interface NAME FETCH: TREE ...+)
     #'(define-hash-interface NAME FETCH: TREE ... WHERE:)]))

(define-hash-interface Simulation-Run-Result
  FETCH:
  [simulation_id     is uint64?]
  [simulation_run_id is uint64?]
  [template_id       is uint64?]
  [started_at        is int64?]
  [completed_at      is int64?]
  [status            is string?]

  (status_details
   (prevention
    [total_threat_count     is int64?]
    [completed_threat_count is int64?])
   
   (detection
    [total_threat_count     is int64?]
    [completed_threat_count is int64?]))

  WHERE:
  (int64?  means integer?)
  (uint64? means integer?))

(define h
  (hasheq 'simulation_run_id 1
          'status            "COMPLETE"
          'status_details    (hasheq 'detection
                                     (hasheq 'total_threat_count 12))))

(define (ctxt)
  (with-hash-interface (Simulation-Run-Result h)
    (h.status_details.detection.total_threat_count)))

(ctxt)

The accessor ids are bound to procedures, so they only attempt to return the value specified by the chained keys, if any exists and assuming it matches the predicate, once they are called.

What caught me off guard, because I only got to a working solution through trial and error, is:

  • the usage of (... ...); I only happened upon this when searching for the error message syntax: no pattern variables before ellipsis in template, which led me to this Github issue. Why is this necessary?
  • the fact that I cannot bind the identifiers (dot-id ...) in the let (...) form as I did with [PNAME PRED?] ...; I ended up using (define ...) in the body of the let, because if I do place [dot-id (lambda () ...)] ... in the arguments of let, the identifiers for [PNAME PRED?] ... become unbound. Is this because of the outer and inner uses of stx? Solved.

Thanks for reading all that!

2

I solved my own problem with the let. At first I thought I might need to use let* because of the references between the different bindings and values, but the answer was letrec.

3

As an aside, a problem I had not considered at first is that, as the code stands, there is no way to provide these "interfaces" to use in other sources.

Changing it to instead make the interface-handle be something like with-hash-interface/NAME so that the NAME is not just syntax for with-hash-interface but the invocation of the macro, solves this problem.

4

In a string "\n" will give you a string with a single newline character
and "\\n" will give you a string with a slash followed by an n.

Since \ has a special meaning inside the double quotes, it needs to be escaped.

The same phenomenon occurs with syntax and ....
Inside a syntax template the template ... will insert the elements
bound to the pattern variable template. It will not insert a literal ....
Therefore the ellipsis needs to be escaped.

The syntax for escaping is (... ...).

A common technique is to give this template a better name like ooo:

(with-syntax ([ooo #'(... ...)])
   <use ooo here>)

Bonus: For macro generating macros that generate macros, the ooo also needs to be escaped. You can use either ((... ...) (... ...)) or (ooo ooo).

See also:

2 Likes
5

Thank you, @soegaard, that makes sense.

Question: would it be possible to write a macro that automatically replaced something like (ellipse n) with the appropriate level of nested escapes, say

(ellipse 0) ≡ ...
(ellipse 1) ≡ (... ...)
(ellipse 2) ≡ ((... ...) (... ...))
etc.

Would it be a reader macro? I don't know much about them, but when I think of the problem, I assume it must happen before any actual pattern matching and application occurs, at least for the contexts where it [the "macro"] appears.

6

I made a tiny edit to your post to use backticks around the strings with \ so they'd appear differently when viewed here: "\n" vs. "\\n".

The markdown parser here was interpreting the latter as "\n".

Needing to escape a discussion about escaping... is the software world we've built and in which we must now live. :smile:

4 Likes