# A Multi-Language-Oriented Macro System by Michael Ballantyne at the (fourteenth RacketCon) is now available **URL:** \ **Category:** General\ **Tags:** racketcon, video, racketcon2024\ **Created:** [October 25, 2024, 6:00pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254 "2024-10-25T18:00:05Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![spdegabrielle](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/spdegabrielle/32/95_2.png) [@spdegabrielle](https://racket.discourse.group/u/spdegabrielle)\ **Post date:** [October 25, 2024, 6:00pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/1 "2024-10-25T18:00:05Z")
A Multi-Language-Oriented Macro System by [Michael Ballantyne](https://github.com/michaelballantyne) at the (fourteenth RacketCon) is now available on the Racket YouTube channel at [https://youtu.be/KHxZ3mT9BSo](https://youtu.be/KHxZ3mT9BSo) [![image](https://global.discourse-cdn.com/free1/uploads/racket/original/2X/a/a6fe24d0000ec85470c17dedc28a74682f9ebf67.jpeg)](https://youtu.be/KHxZ3mT9BSo) -- [(fourteenth RacketCon) playlist](https://youtube.com/playlist?list=PLXr4KViVC0qKNvsSH9nmZlm_YwvX_TXRW&si=qI7ZawbkdTBaQ_8H) ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 10, 2025, 3:22pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/2 "2025-02-10T15:22:56Z")
This seems very cool. In the related paper ["Macros for Domain-Specific Languages"](https://dl.acm.org/doi/pdf/10.1145/3428297), the first examples are the PEG DSL. It seems like `(require peg)` gives a different grammar than the paper (documented [here](https://docs.racket-lang.org/peg/index.html)). That documentation is incorrect in its syntax for name binding, lacks features from the paper, gives contract violations rather than relevant syntax errors, and goes into an infinite loop on `arith-expr-leftrec` rather than rejecting left-recursive non-terminals as described in the paper, so is there a different package that should be used while reading the paper? ---
**Author:** ![michaelballantyne](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/michaelballantyne/32/252_2.png) [@michaelballantyne](https://racket.discourse.group/u/michaelballantyne)\ **Post date:** [February 11, 2025, 3:53am UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/3 "2025-02-11T03:53:56Z")
Yeah, the package you link is an unrelated project. The PEG DSL developed in the paper is available [here](https://github.com/michaelballantyne/racket-peg-ee). Runnable versions of all the code in the paper are packaged up as an artifact archived [here](https://zenodo.org/records/4067001) or available in source form [here](https://github.com/michaelballantyne/extdsl-oopsla2020-artifact/). You might also be interested in the [syntax-spec](https://docs.racket-lang.org/syntax-spec-v2/index.html) version of the PEG DSL which is adapted from the implementation in that previous paper. It's checked in [here](https://github.com/michaelballantyne/syntax-spec/tree/main/tests/dsls/peg). And finally, the more recent [paper](https://dl.acm.org/doi/10.1145/3674627) on syntax-spec. Thanks for reading, and I'd love to hear your thoughts on any of it. ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 11, 2025, 2:08pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/4 "2025-02-11T14:08:33Z")
> [@michaelballantyne](#): > > Thanks for reading, and I'd love to hear your thoughts on any of it. I definitely want to try to work my way through your research as I have the time. You put your finger on an issue which I've toyed with ever since I first learned about macros: you don't always want to expand in terms of the host language's primitives. When you look at the excellent papers, blog posts, Inside Racket videos, and patient mailing list posts from those who have produced major extensions within Racket, like Robby Findler, @samth, Alexis King, Stephen Chang, etc., it seems like the basics of producing a DSL is pretty straightforward, but the nitty gritty details still bloom into a level of complexity which isn't quite contained with a perfect abstraction. These people have pioneered different approaches. Their artifacts and strategies are fascinating. Each seems like a hard won increment of progress toward the ultimate goal of finding the best way for systems to cooperate at the level of syntax. The mechanisms for cooperation which exist inside Racket reflect these different approaches, and they aren't perfectly synthesized and abstracted yet, so studying these systems can lead to the feeling that if you have a practical goal to achieve, it might be easier to implement your own expander, just so you know exactly what is happening. The beauty of Dybvig (et. al.'s) psyntax expander, is that it was the perfect midpoint of source code you could print, read, and run, on that spectrum that ranges from a very abstract model which might fit in the appendix of a paper, to a colossal system which you could never read in full. If you could completely understand psyntax, then you could adapt or clone it for your own language, and I imagine MzScheme must have started as something like that. Connecting to this thought, it is interesting that you describe part of the work in MfDSLs as providing the tools to implement your own expander for a DSL. Anyway, the PLT research program just seems to keep on making progress figuring out how to take the principal of syntactic abstraction up one level from _intra-_ to _inter-_ language cooperation. I love how PLT has stayed focused on these questions over the decades even as the topic moves in and out of the fashion! ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 24, 2025, 2:11am UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/5 "2025-02-24T02:11:11Z")
I haven't got to read much of your functional pearl yet, but I started looking at the state machine example from the docs. I've got as far as _1.2.1 Simple binding_, and ~~as expected we see the error message:~~ ```scheme ————— run main.rkt ————— main.rkt:30:14: on: not bound as event-var at: x in: (on (event x) (goto green)) main.rkt:30:14 ``` ~~Nice error "for free". An improvement would be if it mentioned `green`, since that is the unbound reference.~~ Also in this section, there is a bit: `#:binding (scope (bind arg) body)` which seems to be at the wrong level of nesting in the example. Maybe it should be part of `transition-spec`. Also `body` would be unbound, I think. I left that out for now. Maybe these issues become more clear later. Very nice work. ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 24, 2025, 2:14am UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/6 "2025-02-24T02:14:08Z")
Oh, actually maybe I misinterpreted that error, since I was _expecting_ an error with green. My understanding now is that this section is dealing with scoping `arg:event-var`. My best guess, under my feeble 30 minutes of playing with syntax-spec is that the form should look like this: ```scheme (nonterminal transition-spec (on (event-name:id arg:event-var ...) action:action-spec ... ((~datum goto) next-state:id)) #:binding (scope (bind arg) action)) ``` So `body` -\> `action` and fix the nesting. _edit 3_ So I see in the newer version of the docs in the repo, that you have fixed this to: ```scheme (nonterminal transition-spec (on (event-name:id arg:event-var ...) action:action-spec ... ((~datum goto) next-state:id)) #:binding (scope (bind arg) ... action ...)) ``` Adding the ellipsis (versus my guess), which makes sense, although this doesn't seem to work, at least in my version of syntax-spec. ---
**Author:** ![michaelballantyne](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/michaelballantyne/32/252_2.png) [@michaelballantyne](https://racket.discourse.group/u/michaelballantyne)\ **Post date:** [February 24, 2025, 3:09am UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/7 "2025-02-24T03:09:37Z")
I recently released the v3 version of the package to the package server and docs server: [syntax-spec: A Metalanguage for Hosted DSLs](https://docs.racket-lang.org/syntax-spec-v3/index.html) It has some changes to the syntax as compared to that in the paper like the newly required ellipses, documented here: [1.3 Release Notes](https://docs.racket-lang.org/syntax-spec-v3/Release_Notes.html#%28part._.Version_3%29) I also fixed some bugs in the tutorial as part of the v3 release, so if you use that version of the package and docs you'll likely have a better time. Thanks again for giving it a try! ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 25, 2025, 3:16pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/8 "2025-02-25T15:16:09Z")
Version v3 is correctly giving nice error messages: ```scheme ————— run main.rkt ————— main.rkt:41:13: on: not bound as state-name at: green in: (on (event x) (goto green)) main.rkt:41:13 ``` I can't say I'm in love with the syntax for scopes and binding yet, but this work is very impressive. I'm still at the beginning of the first example, and it already looks like a giant step forward. ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 26, 2025, 4:28am UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/9 "2025-02-26T04:28:30Z")
For some reason, I'm encountering a scope error in the state machine example. Here is my version: [https://gist.github.com/acarrico/89590bcbf82556440404f2e1ec8b336d](https://gist.github.com/acarrico/89590bcbf82556440404f2e1ec8b336d) For some reason the `select-item` argument `item` referenced on line 115 is not correctly scoped, even though we have `#:binding (scope (bind arg) ... (import body) ...))` on line 30. I've checked my source against the more full featured DSL in "~/.racket/8.13/pkgs/syntax-spec-v3/test/dsls/state-machine-for-tutorial.rkt", and I can't see my mistake. I know that I've made an error, since the example runs fine using that version of the machine DSL. I can also see the scope issue stepping backward from the error in the macro stepper, but otherwise, expanding a DSL gives way too much detail for me to get a clue with the macro stepper. I suspect that a solution to this issue will help me build understanding of the binding spec tree defined by syntax-spec before "compiling" the forms. I'm beginning to see that this stands in for the scopes added by core syntax (lambda, etc) in a normal macro expander, although I'm still hazy about this. Incidentally, my version of the machine DSL works fine if you don't refer to the event argument, as in line 116 (vs. 115). ---
**Author:** ![acarrico](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/acarrico/32/2135_2.png) [@acarrico](https://racket.discourse.group/u/acarrico)\ **Post date:** [February 26, 2025, 1:09pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/10 "2025-02-26T13:09:09Z")
Wrapping my vending machine in `(module+ test ...)` solves the issue. Can anyone explain what is going on? I must be the perfect example tester: I seem to fall into every trap! ---
**Author:** ![michaelballantyne](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/michaelballantyne/32/252_2.png) [@michaelballantyne](https://racket.discourse.group/u/michaelballantyne)\ **Post date:** [February 26, 2025, 7:14pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/11 "2025-02-26T19:14:55Z")
Hey, sorry you ran into this! I'm pretty sure I ran into the same issue but it turns out to be a bug with the Racket expander, so I couldn't immediately fix it. The ticket for the bug as it appears in syntax-spec is here: [Unbound error in definition RHS, probably involving use-site scopes? · Issue #71 · michaelballantyne/syntax-spec · GitHub](https://github.com/michaelballantyne/syntax-spec/issues/71) And the Racket expander ticket is here: [quote-syntax pruning and syntax-local-get-shadower don't include certain use-site scopes · Issue #5212 · racket/racket · GitHub](https://github.com/racket/racket/issues/5212) I should really add the workaround of using module+ test to the tutorial for now though! ---
**Author:** ![michaelballantyne](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/michaelballantyne/32/252_2.png) [@michaelballantyne](https://racket.discourse.group/u/michaelballantyne)\ **Post date:** [March 3, 2025, 11:11pm UTC](https://racket.discourse.group/t/a-multi-language-oriented-macro-system-by-michael-ballantyne-at-the-fourteenth-racketcon-is-now-available/3254/12 "2025-03-03T23:11:09Z")
Matthew came up with a fix for the expander bug! It'll be in the v8.17 release though, which isn't for a while. In the meantime, I've added a [note](https://github.com/michaelballantyne/syntax-spec/commit/61ef260c5b94b457d73355a417e2f6ac5f379bdc#diff-9e5d9703ef8c4fe2cd7d64e36951c4597cd548d4e06d0a53ea81a9ed1e47bee9) on the workaround to the tutorial, so hopefully nobody else will run into the problem. Thanks for the report.