What kind of `with`s have you come up with?

Hi, Racket Discourse.

Lol, I accidentally posted the deleted entry prematurely trying to start this one. Such embarrass. Much wow.

Never the mind. I read a post the other day regarding a with-gunzip helper, which I ended up modifying a bit because I am trying to read gzip data from emails, and I didn't like all of the explicit thunkery. I ended up with:

#lang racket

(require file/gunzip)

(provide with-gunzip)

; adapted from:
; https://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/
(define-syntax with-gunzip
  (syntax-rules ()
    [(with-gunzip #:path gzip-path
       body ...)
     (with-input-from-file gzip-path
       (thunk (with-gunzip () body ...)))]

    [(with-gunzip #:pipe gzip-proc
       body ...)
     (let-values ([(pipe-read-gzip pipe-write-gzip) (make-pipe #false)]
                  [(pipe-read-data pipe-write-data) (make-pipe #false)])
       (dynamic-wind
        void
        (thunk (thread
                (thunk (gzip-proc pipe-write-gzip)
                       (gunzip-through-ports pipe-read-gzip pipe-write-data)
                       (close-output-port pipe-write-gzip)
                       (close-input-port  pipe-read-gzip)
                       (close-output-port pipe-write-data)))
               (parameterize ([current-input-port pipe-read-data])
                 body ...))
        (thunk
         (unless (port-closed? pipe-write-gzip) (close-output-port pipe-write-gzip))
         (unless (port-closed? pipe-read-gzip)  (close-input-port  pipe-read-gzip))
         (unless (port-closed? pipe-write-data) (close-output-port pipe-write-data))
         (unless (port-closed? pipe-read-data)  (close-input-port  pipe-read-data)))))]
    
    [(with-gunzip () body ...)
     (let-values ([(pipe-read-gzip pipe-write-data) (make-pipe #false)])
       (dynamic-wind
        void
        (thunk (thread
                (thunk (gunzip-through-ports {current-input-port} pipe-write-data)
                       (close-output-port pipe-write-data)))
               (parameterize ([current-input-port pipe-read-gzip])
                 body ...))
        (thunk
         (unless (port-closed? pipe-write-data) (close-output-port pipe-write-data))
         (unless (port-closed? pipe-read-gzip)  (close-input-port  pipe-read-gzip)))))]))

;;
; example of a usage shape (I actually haven't tested this, the horror)
(with-gunzip #:pipe '(output -> void)
  (define gunzipped (port->bytes))
  (with-output-to-file 'unzipped
    (thunk (write-bytes gunzipped))))

I also have a little syntax-rule I like to use for logging over "blocks" of code:

; I think I also saw this `me:*` in a @bogdan snippet, if I recall correctly
(define (me:log-message logger [topic #false])
  (if topic
      (lambda (#:data [data #false] level message . fmt-args)
        (log-message
         logger level topic (format message . apply . fmt-args) data))
    #;(not topic)
      (lambda (#:data [data #false] level topic message . fmt-args)
        (log-message
         logger level topic (format message . apply . fmt-args) data))))

(define-syntax-rule
  (with-logging ([logging (logger topic ...)] ...)
    body ...)
  #;becomes
  (let ([logging (me:log-message logger topic ...)] ...)
    body ...))

;;
; which might look something like this
(with-logging ([mailbox-clean-up-logs {database-logger 'clean-up}])
  (mailbox-clean-up-logs 'info "starting")
      
  (try-use-mailbox
   'clean-up
   (thunk (mailbox-clean-up-logs 'info "searching messages")
              
          (match/values* (find-redundant-messages events-mailbox)
            [(_)
             (mailbox-clean-up-logs
              'warning "search returned an unexpected result")]
                  
            [(total-fetched total-found positions)
             (mailbox-clean-up-logs
              'info "found ~a redundant messages in ~a total messages"
              total-found total-fetched)
                   
             (cond
               [((allow-delete?) . nand . (pair? positions))
                (mailbox-clean-up-logs 'info "skipping delete")]
               [else
                (mailbox-clean-up-logs
                 'info "deleting redundant messages from mailbox")
                (send events-mailbox delete-messages positions)])])))
        
  (mailbox-clean-up-logs 'info "finished"))

I also have one for database connections in the same app, which looks something like this:

(define-syntax (with-db-connection stx)
  (syntax-parse stx
    [(_ connection:expr #:as conn:id
        body ...)
     #'(let ([conn connection])
         (define result (call-with-values (thunk body ...) list))
         (disconnect conn)
         (apply values result))]

    ; implicit `conn` identifier in scope
    [(_ connection:expr body ...)
     #:with conn (format-id stx "~a" #'conn)
     #'(with-db-connection connection #:as conn
         body ...)]))

;;
; looking something like this
(with-db-connection (app-db:connect)
  (app-db:insert-log conn now (~a level) (~a topic) message))

So, what kinds have you come up with, besides all of the great ones already in the standard libraries?

2 Likes

This is tangetial, but I like to start with a call-with-* procedure before I make a with-* macro that expands to it. This makes the macro generate a lot less code, and sometimes a procedure is more convenient than a macro. For example, koyo provides both call-with-database-{connection,transaction} procedures and with-database-{connection,transaction} macros.

9 Likes

This is a great tip. I could even see--although I don't know if it would necessarily be as generally applicable--a form like define-sequence-syntax which takes a call-with-* procedure-variant, and the with-* syntax-variant of the with expression.

1 Like

this remind me a bad experience with database in Clojure accessed by macros. Starting from a working example with macros , i was not really able to adapt it to my needs because there was so much hidden undocumented expansion that i did not know what was expanding in what and the way to use it. Just to say to try avoid macro if you can do it with procedure and always try to be simple in code.But it was in Clojure...

1 Like

One never knows when your code might cause trauma-flashes to stream before the eyes of others :sweat_smile:.

Jokes aside, I agree with your sentiment. The happy path should allow for enough freedom of expression, or at least follow some conventions, to help guide the user to its attractor.

I tend to look at shapes more than I "read" the code. So, I often resort to "shaping" for visual accordance, as opposed to its composability, for example. Although, I find that this sometimes leads to "uncovering" the right shape precisely because it makes things awkward.

As Paul said, "Beproef alle dinge; behou die goeie." :grin:

3 Likes

I would say that with-disposable is one of the more interesting cases of this that I've written. It's intended to generalize over the pattern, after all.

4 Likes

Would you look at that... Thanks for bringing this to my attention!

I made a version of the Common Lisp with-slots macro for Racket structs recently that I'm proud of.

(require soup-lib/struct)
(struct demo (a b c) #:transparent #:mutable)
(define x (demo 1 2 3))
(with-slots demo (a b) x
  (set! b (+ a b))
  (print x)) ; (demo 1 3 3)

Vital to the implementation are make-variable-like-transformer and the struct-id syntax class.

2 Likes

Very nice :heart:. The pattern before the body seemed odd at first, but it makes sense once you've read it a couple of times.

+1 for the links, thank you.

A small and simple macro for evaluating code within the namespace of a module:

(define-syntax (with-namespace stx)
  (syntax-case stx ()
    [(_ filepath exprs ...)
     #'(call-with-namespace filepath
                         (thunk exprs ...))]))

(define (call-with-namespace filepath proc)
  (namespace-require filepath)
  (parameterize ([current-namespace (module->namespace filepath)])
    (proc)))

We use this in our autograder library for an HTDP course when we want to evaluate code within the context of a student's submission.

We also have a with-wrapped-language macro that depends on with-namespace and exists for two main reasons:

  • Evaluating student code in restricted student languages (e.g. preventing filesystem access and arbitrary module requires)
  • Creating a new temporary file with the student's code -- this is to get around namespace-require not instantiating/loading modules twice, which doesn't work well with how we use test-engine.

Here's one snippet (below) where we use the with-wrapped-language macro, for testing a student's submission. It runs our checks (check-expects) against the student's code and gives us the number of tests that passed.

(with-wrapped-language filepath
  (initialize-test-object!)
  (when setup
    (setup))
  (for ([check checks])
    (eval check))
  (define test-obj (run-tests!))
  (debug-test-obj test-obj)
  (define passing-tests-amt (length (test-object-successful-tests test-obj)))
  passing-tests-amt)

I have plans to cover this and more in deeper detail in a blog post, once I get around to open-sourcing our autograder library. We made interesting design decisions that are distinct from Gregor Kiczales's autograder, but we did take some inspiration from it. :smile:

2 Likes

Splendid, thank you for sharing!

Advent of Code reminded me of another one (Well, set of macros really) shamelessly lifted from Common Lisp's uiop library - with-input-file:

(define-values (left right)
  (with-input-file (in "day01.txt")
    (for/lists (a b)
               ([line (in-lines in)])
      (register-groups-bind ((string->number l r)) (#px"(\\d+)\\s+(\\d+)" line)
                            (values l r)))))

Basically an inline call-with-input-file that doesn't require an explicit lambda around the body. There's also a with-output-file and others in my soup-lib library. (register-groups-bind is taken from cl-ppcre; I've stolen a bunch of ideas from popular Common Lisp libraries)

1 Like

The baby was up all night, so I've had time to mess around.

I have a project at work which requires me to edit some xlsx files. Although I have been quite thrilled with @simmone's simple-xlsx, since it is teaching me a great deal (yay, no COM-objects), it has made me curious about editing xexprs as is, so to speak--as opposed to turning them into other data structures first.

How cool are the se-path utility procedures? I found it frustrating, though, that I cannot directly use them to edit xexprs, so I reckoned a zipper-like thing (only halfway so, really) might do the trick.

#lang racket

(require
  threading racket/dict
  (for-syntax
   syntax/parse))

(define attr?
  (match-lambda
    [`[,(? symbol?) ,(? string?)] #true]
    [_
     #false]))

(define kw->symbol
  (Ī»~> (keyword->string) (string->symbol)))

(define (with-xe key app xe)
  (match xe
    ; in a tag with attributes, and, zero or more sub-xes
    [`(,(? symbol? tag) (,(? attr? attrs) ...) ,xes ...)
     (match key
       [(== tag)
        (define push (app xes))
        `(,tag ,attrs . ,push)]
       
       [`(,(== tag))
        (define push (filter-map app xes))
        `(,tag ,attrs . ,push)]

       [`(,(== tag) #:)
        (define push (app attrs))
        (if (null? push)
            `(,tag . ,xes)
            `(,tag ,push . ,xes))]
       
       [`(,(== tag) ,(? keyword? (app kw->symbol attr)))
        (define gets (dict-ref attrs attr '(#false)))
        (define puts (app (car gets)))
        (define push
          (if (not puts)
              (dict-remove attrs attr)
              (dict-set    attrs attr `(,puts))))
        (if (null? push)
            `(,tag . ,xes)
            `(,tag ,push . ,xes))]

       [`(,(== tag) ,(? keyword? (app kw->symbol attr*)) ...)
        (define push
          (app (for/list ([attr (in-list attr*)])
                 `(,attr . ,(dict-ref attrs attr '(#false))))))
        (if (null? push)
            `(,tag . ,xes)
            `(,tag ,push . ,xes))]
       
       [_ null])]

    ; in a tag with no attributes, and, zero or more sub-xes
    [`(,(? symbol? tag) ,xes ...)
     (match key
       [(== tag)
        (define push (app xes))
        `(,tag . ,push)]
       
       [`(,(== tag))
        (define push (filter-map app xes))
        `(,tag . ,push)]
       
       [`(,(== tag) ,(? keyword?) ...)
        (with-xe key app `(,tag () . ,xes))]
       
       [_ null])]
    
    [_ null]))

(define page
  '(html
    (body (p ([class "baz"])
             "foo" "bat")
          (p "bar" (br) "qux"))))

(with-xe 'html
  ; listof matches
  (conjoin println values)
  page)

(with-xe '(html)
  ; single match
  (conjoin println values)
  page)

(define (compose-left . fs)
  (compose1 . apply . (reverse fs)))

(with-xe '(html)
  (Ī» (html)
    (with-xe '(body)
      (compose-left
       (Ī» (body)
         (with-xe '(p)
           (Ī» (p.inner)
             (if (not (string? p.inner))
                 p.inner
                 (string-upcase p.inner)))
           body))
       (Ī» (body)
         (with-xe '(p #:class)
           (Ī» (p.class)
             (and p.class (~a p.class " pop!")))
           body)))
      html))
  page)

;=>
; '((body (p ((class "baz")) "foo" "bat") (p "bar" (br) "qux")))
; '(html (body (p ((class "baz")) "foo" "bat") (p "bar" (br) "qux")))

; '(body (p ((class "baz")) "foo" "bat") (p "bar" (br) "qux"))
; '(html (body (p ((class "baz")) "foo" "bat") (p "bar" (br) "qux")))

; '(html (body (p ((class "baz pop!")) "FOO" "BAT") (p "BAR" (br) "QUX")))

I tried to constrain myself mostly to the conventions used by se-path, although I extended it a bit to allow for multiple attributes to be returned from a tag, as well as using a different mechanism to indicate multiple vs. a single matched result.

The "keys" used to search the xexpr can be of the form:

'tag-name
-> matches on the entire body of the corresponding tag

'(tag-name)
-> matches on each of the expressions in the body of the tag

'(tag-name #:)
-> matches on all of the attributes of the tag

'(tag-name #:attr)
-> matches on the attribute named `attr` of the tag

'(tag-name #:attrā‚€ #:attrā‚ ...)
-> matches on (all of) the attributes named `attrā‚€`, `attrā‚`, etc.

Furthermore, the attribute matches may return #false if no such attribute exists. Similarly, if one sets an attribute (or expression in the body of a tag) to #false, it is removed from the context.

And then, with the power of some sugar to make the medicine go down:

(begin-for-syntax
  (define-syntax-class xe-key
    #:attributes (name quote)
    
    (pattern name:id
      #:with quote #''name)
    
    (pattern (name:id attr:keyword ...)
      #:with quote #''(name attr ...)))

  (define-syntax-class xe-keys
    #:attributes (name next)
    
    (pattern (name:id next:xe-keys))

    (pattern :xe-key
      #:with next #false))

  (define-splicing-syntax-class xe-alias
    #:attributes (aka)
    (pattern {~seq #:as aka:id})))

(define-syntax (<xe> stx)
  (syntax-parse stx
    [(_ (~optional :xe-alias) :xe-key
        body:expr ...
        xe:expr)
     #'(with-xe quote
         (Ī» ((~? aka name)) body ...)
         xe)]

    [(_ (~optional alias:xe-alias) :xe-keys
        body:expr ... xe:expr)
     #'(<xe> (name)
         (<xe> (~? (~@ . alias)) next
           body ... name)
         xe)]))

(define-syntax-rule
  (<xe*> xe* ... xe)
  #;becomes
  (~>> xe xe* ...))

(<xe> (html)
  (<xe> (body)
    (<xe*>
      (<xe> #:as p.inner (p)
        (if (not (string? p.inner))
            p.inner
            (string-upcase p.inner)))
      
      (<xe> #:as p.class (p #:class)
        (and p.class (~a p.class " pop!")))
      body)
    html)
  page)

(<xe> (html (body)) ; maps over each thing in body's content
  (<xe*>
    (<xe> #:as p.inner (p)
      (if (not (string? p.inner))
          p.inner
          (string-upcase p.inner)))
    
    (<xe> #:as p.class (p #:class)
      (and p.class (~a p.class " pop!")))

    (<xe> #:as p-attrs (p #:)
      (and (println p-attrs) p-attrs))
    body)
  page)

(<xe> #:as body* (html body) ; applies to the entirety of body's content
  (append
   (for/list ([body (in-list body*)])
     (<xe*>
       (<xe> #:as p.inner (p)
         (if (not (string? p.inner))
             p.inner
             (string-upcase p.inner)))
       
       (<xe> #:as p.class (p #:class)
         (and p.class (~a p.class " pop!")))
       body))
   '((div ([class "last"]) "done")))
  page)

(<xe> (html (body))
  (<xe> #:as p-attrs (p #:id #:class) ; multiple attributes
    (and (println p-attrs) p-attrs)
    body)
  page)

;=>
; '(html (body (p ((class "baz pop!")) "FOO" "BAT") (p "BAR" (br) "QUX")))

; '((class "baz pop!"))
; '()
; '(html (body (p ((class "baz pop!")) "FOO" "BAT") (p "BAR" (br) "QUX")))

; '(html (body (p ((class "baz pop!")) "FOO" "BAT") (p "BAR" (br) "QUX") (div ((class "last")) "done")))

; '((id #f) (class "baz"))
; '((id #f) (class #f))
; '(html (body (p ((id #f) (class "baz")) "foo" "bat") (p ((id #f) (class #f)) "bar" (br) "qux")))

As you may notice, the macro allows one to "nest" more tag-names than the procedure in one go, but I haven't decided if I should extend the functionality downwards, or whether this makes sense.

The introduction of the aliases is because the tags' ids are used as the symbols for matching, so one must explicitly indicate if one wishes to use a different identifier. The alias also only applies to the final tag in a nested tag sequence.

As with the procedure, the keys take the form:

tag:id
-> matches on the entire body of the corresponding tag

(tag:id)
-> matches on each of the expressions in the body of the tag

(tag:id #:attr)
-> matches on the attribute named `attr` of the tag

(tagā‚€:id tagā‚:id)
-> matches on the tag named `tagā‚€` and then `tagā‚`, specifically the entire body of `tagā‚`

(tagā‚€:id (tagā‚:id))
-> matches on the tag named `tagā‚€` and then `tagā‚`, specifically each expression in the body of `tagā‚`

etc.

Probably needs a lot more thought to really be useful, but the toolkits are such fun to work with.

Edit: missed an ellipsis there while posting.

Narrator:
It was not, in fact, the case.

Luckily fixed, now. :sweat_smile:

P.S. thank heavens for @countvajhula's How to Organize Your Racket Library. So useful when the bug finally bites.

1 Like

:rofl: (need twenty character to get an answer validated)