I'm currently planning to write a DSL for a project, and am evaluating different ways to do this. Racket came up as a good language for writing languages, and I have some questions I couldn't find answers to from simple searching.
If I write a DSL with Racket, my end-users will need to install Racket on their machines as a runtime dependency, correct? There's no way to create a standalone compiler/interpreter for my custom language with Racket.
Since there's a hard dependency on Racket, then my DSL can only be deployed on platforms where Racket has been ported to, right? I know Windows/Mac/Linux is supported, but are there more? Has anyone ported it to, for example, Android ARM 64-bit? Web assembly? Or maybe even some bare metal ARM/embedded platforms? Are there any particular challenges with porting Racket?
Are there any examples of someone successfully deploying Racket in a multi-platform application? For example, as an internal DSL in a larger desktop/mobile app, etc?
(... which is actually just a pointer to three different pages.)
You also ask about platforms: Racket 8.5 supports a new "--enable-pb" mode that may allow compilation to platforms that are not currently supported by native code generation. The 8.5 release notes say:
"See “README.txt” in the source distribution for more information on the —enable-pb flag to configure."
Note that IIUC, this is an area of ongoing work, and that currently, you could run into some challenges in supporting other platforms.
I was about to answer that. However, as I understand the question, Alex plans to implement the DSL, but other users may need to develop in/with this DSL. When they do so, I think they have to develop in the Racket ecosystem (say, with DrRacket), not just use an executable.
On the other hand, if the question refers to implementing a DSL and distributing a distinct standalone program written in/with this DSL, the usual approaches with raco exe etc., which you mentioned, apply.
That said, I'm curious now if it's possible to create a standalone interpreter/compiler for a DSL. (*) I guess that comes down to wrapping racket and its dependencies plus the modules which implement the DSL in a single executable so you can run
standalone-dsl-racket program-written-in-the-dsl
where program-written-in-the-dsl can change between invocations.
(*) Given how I understand Racket works, it would be on-the-fly-compiling to machine code (I guess that's meant by native support?), or compiling to byte code and interpretation of it.
When they do so, I think they have to develop in the Racket ecosystem (say, with DrRacket), not just use an executable.
Why would that be the case? Any editor should work just fine, though the quality of support might vary.
That said, I'm curious now if it's possible to create a standalone interpreter/compiler for a DSL. (*) I guess that comes down to wrapping racket and its dependencies plus the modules which implement the DSL in a single executable so you can run […]
This is what raco exe and raco distdo. So if the program is a compiler like, say, cc, it produces a compiled output. If it's an interpreter like, say, awk, it can interpret without requiring compiled bytecode/JIT/etc. (though those may be good things to have).
Either way, you just have to write a program with a main of the right kind, rather than writing a #lang. OTOH, it is possible to write a #lang that, when programs in it are run, actually emit compiled code—for that, you probably would want to stick more in the Racket ecosystem.
I didn't express myself clearly. I meant you'd still need Racket and libraries installed while developing. It wasn't specifically about DrRacket.
My understanding is, if you write a single program in #lang typed/racket, you can use racket to execute it, or you can use raco exe to convert the program into a standalone binary. What I'm talking about is a generalization: If you have, say, a hundred programs that use Typed Racket, you could create a hundred programs with raco exe. What I want to know is how you create a single standalone binary that can execute the source code of any of the hundred programs?
But I realized now it's relatively easy. I guess you could just load and execute each of the programs with dynamic-require or eval and the single program that does this loading and executing could be created with raco exe.
It sounds like you are planning on creating an external DSL. That is, you would implement the language as a Racket program which takes textual input---a program written in your language---and then does lexing, parsing, interpreting/compiling, etc. The other responses are focusing on this approach because your questions suggest that is what you want.
Racket's strengths for writing languages are primarily oriented around embedded DSLs, ones that live within the Racket ecosystem. Examples include Typed Racket and pattern matching. They utilize Racket's macro and module system to allow writing "Racket" programs with meanings that would have otherwise been impossible (Typed Racket) or tedious (match) without the DSL.
An embedded DSL can make use of all the existing tools and support within the Racket ecosystem, allowing you to focus on implementing the domain-specific portion of the language and letting Racket take care of the rest. If your DSL has numeric operations, you can define them in terms of Racket's number system. If your DSL has modules, you can define them in terms of Racket's module systems. And so on.
When people refer to Racket "as a good language for writing languages," I believe it is typically within the context of embedded DSLs.
Racket is a great language for creating DSLs. Period.
For a stand-alone example, Jesse Alma (organizer of the 2022 RacketCon) has produced a test-script language for the web. When his colleagues write scripts. they don't even know that they are running Racket. ~~ Two of my former undergraduates -- both '21 grads -- produced stand-alone DSLs as the first thing on the job last year: one for managing pharma simulations on large machine farms and one for managing contracts for distributed micro-services. Again, no colleague experiences Racket.
For an example of an embedded DSL, see Typed Racket. A programmer can act as if it TR is the only thing that exists or link together typed and untyped modules. The Racket code base contains dozens of other languages like TR (though for "smaller" domains). Lazy Racket is the only one that comes to mind with a similarly large app domain.
For an example of a hosted DSL, see match. The language of patterns is something that (1) re-interprets existing elements to have a different (though related) meaning and (2) cannot be used in any other context. Similar DSLs are syntax/parse (the best language for writing embedded and hosted DSL ... Yes, of course there are DSLs for writing DSLs.)
Thanks for the replies everyone, it's helped me understand a bit better, although I'm still pretty lost I think I'm going to bite the bullet and start studying this more since it seems very interesting to me.
Personally I don't really care about the time savings of not having to implement, say, a lexer/parser for a custom DSL. However, being able to build upon an existing ecosystem, and maybe have my custom DSL be extensible as well do seem like really valuable features. I've never really thought about doing that kind of thing for a programming language/DSL.
@samc brought up an interesting point about external vs embedded DSL. My use case does seem to match the description of an external DSL. Here are some examples off the top of my head that are maybe similar:
Build script, like CMake, Make, Ant, Bazel, Meson (I noticed Racket uses something called "Zuo")
Data query language for a custom database, kind of like SQL, GraphQL or Unqlite's Jx9 language
Data processing language to do transformation, filtering, etc used as part(s) of a pipeline
Schema languages like Protocol/Flat Buffers, Cap'n Proto, Apache Thrift, etc
All of those are things that come to mind when I think "DSL". Would those be practical applications for Racket?
A build system would probably be fine since that typically only targets desktop platforms, but something like Unqlite, which is an embedded database (like SQLite) would need to support many more platforms and archs. Racket seems to be using a custom build system, and in my experience that tends to make porting/cross compiling difficult.
IIUC, Zuo the build system is straightforward to compile on purpose (a single C file?).
Where things get more complicated is building Chez Scheme in order to build Racket CS (the default). I don't know how easy or hard that is, but there are several threads on Discourse about it, and I suspect that Chez Scheme is relatively well-supported.
There's also the BC variant, which might extend portability?
Probably you should trust the expert opinion of Matthew Flatt over my hesitant understanding, though
However, as I understand the question, Alex plans to implement the DSL, but other users may need to develop in/with this DSL. When they do so, I think they have to develop in the Racket ecosystem (say, with DrRacket), not just use an executable.
I think I'm going to bite the bullet and start studying this more since it seems very interesting to me.