@scolobb thanks for collecting those answers in that way - very useful! I tend to get define-syntax and phases mixed up myself, and your example gave me an opportunity to concretize some concepts.
I think we can think about it this way:
Each phase of execution has its own set of bindings that are available for use during evaluation of that phase. These are called "value bindings," and they refer to ordinary data like numbers, strings, and functions. In order to define bindings for the runtime phase (phase 0) we simply define them. For the syntax phase, we wrap these same definitions in a (begin-for-syntax ...), and if we wanted to define in a preceding phase to that, we can simply nest another (begin-for-syntax ...), or use (require (for meta ...)). All of these result in normal (value) bindings in a particular phase.
Okay, now, we know that "macros are just functions operating at compile time to transform your source code." So that should mean that we can define a function in the syntax phase (phase 1), and it would be a macro, right?
(begin-for-syntax
(require syntax/parse)
(define (helper stx)
(syntax-parse stx
[(_ e ...) #'(list e ...)])))
(helper 1 2 3) ;=> error helper undefined
This doesn't work because although we have a syntax-phase function that transforms syntax, i.e. a "macro," our naive use of helper is treated by the compiler as a runtime binding -- which it looks for in the bindings for the runtime phase and doesn't find. Added to that, all we have here is a function defined at phase 1 and an application to be run at phase 0 -- there's nothing here linking these two things. We need a way to tell the compiler that helper refers to a value in the preceding phase, and furthermore, that the value of this binding is expected to be a function that should be used to transform the code for use in the current phase.
I think this could probably be done in many different ways, but we already have one way to associate identifiers with values at this point -- bindings. So Racket introduces a separate category of bindings which can be used in one phase (phase n) to refer to values in the preceding phase (phase n+1 -- note the numbers increase going backwards in time). These cross-phase bindings are called "transformer" bindings. The way to define value bindings is to use define, and the way to define transformer bindings is to use define-syntax. Using these forms in a particular phase creates value or transformer bindings for that phase. Modifying the earlier code to use define-syntax instead:
(begin-for-syntax
(require syntax/parse))
(define-syntax (helper stx)
(syntax-parse stx
[(_ e ...) #'(list e ...)]))
(helper 1 2 3) ;=> '(1 2 3)
So (define-syntax (mac stx) ...) appears to be essentially (but not quite - see below) equivalent to:
(begin-for-syntax (define (mac stx) ...))
(define-syntax mac mac) ; creating a transformer binding in phase n to refer to a value binding in phase n+1
We can test this theory:
(begin-for-syntax
(require syntax/parse)
(define (helper stx)
(syntax-parse stx
[(_ e ...) #'(list e ...)])))
(define-syntax helper helper)
(helper 1 2 3) ;=> '(1 2 3)
and also:
(begin-for-syntax
(require syntax/parse)
(define (helper stx)
(syntax-parse stx
[(_ e ...) #'(list e ...)])))
(define-syntax (main-macro stx)
(helper stx))
(main-macro 1 2 3) ;=> '(1 2 3)
But this isn't a complete description because if we just do (define-syntax ...) without having already defined a syntax-phase binding via (begin-for-syntax (define ...)), then we have a transformer binding in phase n without the corresponding value binding being accessible in phase n+1. For this case though, we can use syntax-local-value.
I'm not sure how canonical the above descriptions are but they appear to be consistent with the docs on Transformer Bindings, and at least for the moment, they make me feel like I understand what's going on here :). Hope it helps!
Also, re: @capfredf 's suggestion to use a macro, in your code example, what you've done is define a macro in the syntax phase, that is, it is a transformer binding at phase 1, and therefore it is used in phase 2 to expand the code that will be run in phase 1. That's cool! Sometimes that kind of thing is necessary. But I think what @capfredf was getting at was simpler than this - in each phase, macro expansion is invoked repeatedly until the code produced reaches steady state. So macros can expand to other macros, and if that happens then expansion will continue to expand those macros in the same phase. E.g. if you define three macros mac, mac2, and mac3, then mac can expand to a use of mac2 which can expand to a use of mac3 which can expand to pure Racket, and all of these would happen as part of Phase 1 expansion. So you could do:
(begin-for-syntax
(require racket syntax/parse))
(define-syntax (helper stx)
(syntax-parse stx
[(_ stuff ...)
#'(list stuff ...)]))
(define-syntax (main-macro stx)
(syntax-parse stx
[(_ stuff:expr ...)
#'(helper stuff ...)]))
(main-macro 1 2 3) ;=> '(1 2 3)
Finally, since (define-syntax (mac stx) (syntax-parse stx ...)) is such a common pattern, there's a macro for that: define-syntax-parser that could save you some typing.
