Hello,
This is a topic that's come up from time to time in Qi meetings. I'm moving it out into a separate thread so that others can follow and chime in. It'd be nice to understand:
Could we have the same macro system we use in Racket today but without the need for two distinct kinds of bindings (value bindings and transformer bindings)? And could this provide some advantages?
I want to clarify that this is not about eliminating phases, which are great. It's about whether we can eliminate the dichotomy of bindings so that we just have simple variable binding and reference (at a given phase), and nothing more complicated than that.
For instance, in writing macros, we sometimes use syntax-local-value to look up the transformer binding of a particular identifier.
How many macrologists use syntax-local-value? It feels like an advanced concept.
But I would also ask: how many Racketeers evaluate variables? Why, everybody does!
Here's what I'm wondering: in every case where we use syntax-local-value, are we really just trying to write a variable name? Could it be that this is a basic programming facility that is made complicated by the presence of the value binding / transformer binding dichotomy?
Consider this example:
(begin-for-syntax
(define abc #'5))
(define-syntax-parser mac
[(_ q) q])
(mac abc)
This is a made-up example and it doesn't work.
But maybe it should!
Here, the identifier abc is bound in phase 1 to the value #'5.
In the use of the macro mac, we supply this same identifier, knowing that it has a phase 1 binding. As expansion of this macro is happening in phase 1, the expander should have access to this binding, we reason, and should be able to resolve it.
But this code issues the familiar complaint, "q: pattern variable cannot be used outside of a template."
Now, consider this very slight variation:
(begin-for-syntax
(define abc #'5))
(define-syntax-parser mac
[(_ q) abc])
(mac abc) ;=> 5
The only change is q to abc in the macro template expression, and it works.
Compare that to this:
(define abc 5)
(match abc
[q q]) ;=> 5
... which also seems to support that q ought to be bound to abc in the first version.
Even though the second version works, the problem is, the reason we might be trying to do this is that we hope to indicate a phase 1 binding by name at the use site of the macro. But in this second version, during expansion, we're ignoring the input syntax entirely and just referring to the phase 1 binding independently. There is a disconnect here and it doesn't do the same thing as the first version, though it serves to illustrate the issue and one expectation of what should happen here.
The actual way to do this today is the following:
(define-syntax abc #'5)
(define-syntax-parser mac
[(_ q)
#:with val (syntax-local-value #'q)
#'val])
(mac abc) ;=> 5
Note that, instead of an ordinary binding here, we define abc as a special binding -- a transformer binding, via define-syntax. Additionally, instead of simply referring to the variable by name, we introduce a new concept in the form of syntax-local-value to indicate that we are trying to get the transformer binding and not the ordinary binding of the variable.
Would it be possible instead to allow macros to refer to ordinary phase 1 bindings (and not raise a "pattern variable outside template" error)?
I know there are probably decades of Scheme tradition behind these choices and I'm sure there are good reasons, but sometimes, also, the reason is simply historical, so it would be good to know if that's the case here (and whether there's anything we can do about it).
Of course, the main actual purpose for define-syntax isn't to hold arbitrary values but specifically to refer to macros (i.e. syntax -> syntax functions)! The "Qi meeting notes" thread discusses one possible design to achieve that using the one-binding approach, via a (require (for-lang ...)) module-level directive. I don't know how feasible it is. There may be other ways.