I’ve been saving this question for the RacketCon town hall, but the more I think about it, it’s probably worth asking ahead of time. Hopefully I can ask it in a way that generates productive discussion.
What improvements (of any significance large or small) to the Racket language and ecosystem do the core team currently have planned, or even just envisioned? Is there a roadmap? Or is the Racket ecosystem (core libraries, language, package system, package server, documentation facilities, etc) essentially in maintenance/life support mode for the foreseeable future, except for community contributions?
I firmly believe everyone, Racket core devs of course included, ought to work on what they find interesting and fulfilling. So this is not a rhetorical question intended to make a point about the state of Racket; I really would appreciate any answer that is honest. Or maybe the core team can just think about it awhile and address it at the town hall in October.
Here are two contributions to expect from Northeastern:
— syntax-spec, a layer atop syntax-parse for specifying hosted DSLvia grammars enriched with binding rules (and perhaps IDE support)
— trace-contracts : a significant addition to the contract library, which enable developers to specify relationships among several calls to (possibly distinct) functions/methods
I was hoping that UMB would launch RacketScript .. but the effort has gotten stuck. It really needs a full-time caretaker.
Someone mentioned a couple of years ago that Racket(the language) might be approaching a local maxima, so I only half jest when I ask: what is missing from Racket that you would put on a roadmap?
I really think RacketScript is important. I know it is ‘experimental’ but it works now - try http://racketscript.org/ playground - and RacketScript deserves more attention. The most popular programming language is Microsoft Excel, but the second most popular is Javascript.
PS and build more apps with Racket, webapps, spa’s, desktop apps, bots, games.
Someone mentioned a couple of years ago that Racket(the language) might be approaching a local maxima, so I only half jest when I ask: what is missing from Racket that you would put on a roadmap?
I am asking the question because I would like to know what the core dev team specifically has in mind, absent any suggestions from anyone else. That is, do they have plans, or don’t they? If so, what are the plans? I appreciate that you approach every question from a community-building perspective, but the community’s perspective is exactly what I am not trying to discuss here.
Racket has also become a tool for doing analytic PL research. Examples are Ben Greenman’s work on performance evaluations or Christos Dimoulas’s work on studying language pragmatics. As such we have slowly moved away from working on Racket per se and use Racket as a tool. So my response was focused on what some on the team will contribute to Racket’s main distribution. (I have also reduced the number of PhD students I train so that’s a secondary reason for fewer developments.)
Matthew (with the help pf several other core members) has spent a lot of energy on Rhombus, a language design that synthesizes a number of his ideas into one bundle: (1) paren-free syntax, (2) macro extensible, (3) static-semantics propagating. And probably some more ideas that I don’t recall right now. My take is that this is Racket development (and I am pretty sure Matthew sees it that way) because it is an extremely large-scale project in Racker’s LOP spirit. In this sense Racket is a family of some 4 or 5 dozen languages already in the main distribution, and Rhombus will be one more. (But if you mean racket/base or just racket you might not be satisfied with the answers you’ll get.)
I don't think there's a development roadmap — or that there has ever really been a roadmap, except maybe in the early days. I'm even skeptical of the idea of "core dev team". There is such an entity as Racket management, and management has sometimes given its endorsement for specific directions, but that has been more of a post-hoc coordination and broadcasting role than a development-planning role.
So, that's why Matthias answers about things happening at Northeastern, and why I'll answer below about my own plans — because whatever "core dev team" is, we figure we're probably included. I worry that others might not answer on the grounds that they were never issued a "core team" badge, though, and that doesn't seem right.
Here are things on my radar in the near-ish (year or so) time scale:
Get the Racket branch of Chez Scheme merged to an official repo as supported by the current maintainers of Chez Scheme. I've had an agreement in principle with the Chez Scheme maintainers since last October, and much of the activity in the Racket (the part mirrored to racket/ChezScheme) in late 2022 was aimed at addressing their concerns. There has been movement recently — recent commits in cisco/ChezScheme were things the maintainers wanted to finish first — so maybe this will keep moving along. I'm not sure whether you'd put this in the category of Racket plans, though.
Refesh the third-party libraries that are used by racket/draw — and some other packages, but it's mostly the racket/draw layer that's tricky. We've gotten by for a while using old versions with a pile of Racket-specific patches, but it's obviously better to sync and get the benefits of modern versions. The work is not just figuring out how to compile and link and distribute for relevant platforms, but also sorting out what needs to change in Racket code that uses the libraries (e.g., where workarounds in that layer become counterproductive after upgrading). The drawing libraries are prominent enough that this seems like it counts as "Racket".
Reduce startup for things built on Racket. Maybe this takes the form of more work along the lines of raco demod, maybe it involves changes to the way the lower layers are put together (even starting from inside Chez Scheme?), or maybe it's a combination. I'm not sure what will happen, but it's a direction I expect to investigate. This seems clearly "Racket".
WebAssembly as a target. Not sure how this goes, either, but I did some work in this direction in the last year, and Adam Perlin did a lot more. Something might line up so that I see a place to contribute.
Rhombus. I don't think this an answer you're looking for, so I put it last. But as Matthias says, I have trouble drawing a distinction between my own work on Racket from (on one end) work on Chez Scheme or (on the other end) applications like Scribble, Rhombus, or a language for my PL class. I plan in terms of the stack, but I also aim to preserve and improve each layer as a coherent system on its own, whether that's Chez Scheme or Racket or Rhombus or even Zuo.
Here are things I've been thinking about that might be answers to this question:
Compiling to the web. I wrote a (sadly declined) grant proposal about this; that will probably get resubmitted, and in general I'm interested in pushing both compiling to WebAssembly and JavaScript as directions for Racket.
Typed Racket type inference. My student Fred Fu and I have been looking at improving Typed Racket's support for type inference, and have developed interesting prototype ideas. This is far away from being something that would be deployed in Typed Racket itself, though, and it might not even work at that scale or might have serious backwards-compatibility problems.
Incremental macro expansion. I'm interested in the problem of combining incremental compilation and macros, but this is still very much a research idea rather than something with a path forward to inclusion in Racket.
Hopefully you've seen this already, but, at the risk of getting slightly off-topic, I wanted to leave a pointer to spritely / Guile Hoot · GitLab, the Spritely Institute's (funded) project to compile Guile to WebAssembly (plus JavaScript or WASI). While Guile is LGPL, the Hoot repository is Apache-2.0, and parts of what they've produced so far (WASM representations for primitive Scheme types and tools for code generation and disassembly: Lisp Game Jam - "Wireworld" - Hoot's low level WASM tooling in action -- Spritely Institute) seem general enough to potentially be reused.
This is my attempt to summarise in the form of a Product Roadmap.
Any errors below are mine.
all this work is strictly volunteer - and we must respect that.
this does not include
roadmap entries for the developers that shepherd/contribute to packages in the main distribution (e.g. Quickscript)
roadmap entries for packages in Racket Ecosystem (e.g. Koyo, Urlang, Qi)
As a software artefact, Racket is an Open Source Software Project, and like most volunteer endeavours it is a 'DoOcracy' in the sense that if you do something you get to make the decision what you do.
Task/enhancement/library/lang/etc.
Status
Owner (may change)
syntax-spec, a layer atop syntax-parse for specifying hosted DSLvia grammars enriched with binding rules (and perhaps IDE support)
planned?
NE
trace-contracts : a significant addition to the contract library, which enable developers to specify relationships among several calls to (possibly distinct) functions/methods
planned?
NE
Get the Racket branch of Chez Scheme merged to an official repo as supported by the current maintainers of Chez Scheme.
future aspirations
mflatt
Refesh the third-party libraries that are used by racket/draw
I'm writing mostly to affirm Matthew's comments about idea of a "core dev team" and to say a little more about that, building on @spdegabrielle 's observation about volunteers. As this is a fairly public discussion and I guess that google will turn up results from this for a while to come, it seems important to contextualize these comments so people who are new to Racket understand the situation a bit better.
So: there is no question that those who have replied with plans in this thread have had substantial benefits to our careers over the years because of Racket, but there has never been any income stream that directly connects Racket to us. This means that we've made decisions about what to work on only partly based on what Racket needs to succeed and thrive. You can probably see this in the replies already, in fact. Along these lines, I have long had a daydream that a giant pile of money might appear from which I could draw a salary to just focus on improving Racket itself but have never seriously tried to make it happen -- I don't really know where to begin and anyway it is hard to give up the professor's life to try seriously! Well, this paragraph is leading in a place that's best a conversation for another day the point I'm trying to make here is that Racket does not really have the resources to fund anyone whose full-time job might include doing things like making roadmaps, let alone carrying them out.
My plans:
I hope to continue the slow (but steady?) stream of minor new features and bugfixes that have characterized my output for the last N years. Not exactly roadmap material but likely to be my largest contributions to Racket in practice.
I have been trying to get research funding that would also improve DrRacket's support for language-oriented programming, bringing IDE support to all of our new languages with as little effort as possible from the language implementer. Alas, unsuccessfully so far but I hope to find time to work on this anyway.
My main focus recently has been on building Esterel in Racket. This seems to be turning into a classic high risk, low reward effort, however, so please don't expect too much.
By ways of comparison, Rust has a roadmap, but this is a much larger project with long standing significant levels of corporate funding, including paid developers dedicated to rust both on their payroll, and on corporate supporters payrolls.
Most of my contributions are in the optimization step in the Chez Scheme version used by Racket. Now they are advancing slowly because my two small kids are causing too much mess. Anyway, some plans:
Refactor the code, and improve some parts: For example, in (lambda (x) (unless (list? x) (unless (list? x) (something x)))) now the compiler should eliminate the second check . The number of predicates that can be tracked is slowly increasing.
More support for fixnums: For example, in (lambda (x) (when (fx< 0 x 10) (+ x 5))) I'd like the compiler to transform + into unsafe-fx+ . I have a minimal half backed version, but it has like a million of cases so it's a tricky feature. And each additional operation adds more and more cases.
Better support impersonators and chaperones: Many predicates in Racket like vectors? can recognize impersonator too, not only real vector. The compiler has a pass call schemify that transform the Raccket code into Chez Scheme code. My idea is to make schemify more optimizer friendly and the optimizer more schemify friendy. But I have to understand a lot of the technical details first, last time I checked the implementation of impersonators and chaperones I got a few surprises.
Better support for loops: For example, in (lambda (x) (for ([x 10]) (+ x 5))) I'd like the compiler to transform + into unsafe-fx+. The optimizer receives an expanded version of this, so it's quite difficult to rediscover that x is a small fixnum. Loops are very tricky in the expanded versions, so this is more a wish than a planed feature. Anyway, my idea is to try to add the easy useful parts (like the second bullet point) and hope that the hard parts will get solved later.
All of these changes should be invisible for the end user. The programs may run a few percents faster magically.
Bonus: Add some methods to talk with the optimizer. Something like is-the-optimizer-sure-this-is-a-boolean-function? or can-the-optimizer-calculate-this-value-at-compile-time?. Internally, the compiler has a lot of special cases for Chez Scheme functions, and schemify has some special cases for Racket functions, but it would be nice that languages build on top of Racket can peek and write special cases too. Macros are a possible solution, but they run before constant propagation/folding. It would be nice that the macros can add some special cases that are guaranty to disappear if the optimizer can't calculate them.
Thanks all for the responses. I’m glad I posted the question now, rather than waiting for RacketCon. All things considered, this thread has so far ended up being a decent (maybe better) alternative to an official page or blog post.
For my simplistic purposes, anyone who has write-level access to the main repos at Racket · GitHub and makes or approves commits to those repos is in the “core dev team”, and everyone else is not. This is the set of people that actually governs Racket’s direction in a hard practical sense, however loosely or benevolently they see themselves doing so.
Reduce startup for things built on Racket. Maybe this takes the form of more work along the lines of raco demod, maybe it involves changes to the way the lower layers are put together (even starting from inside Chez Scheme?), or maybe it's a combination. I'm not sure what will happen, but it's a direction I expect to investigate. This seems clearly "Racket".
WebAssembly as a target. Not sure how this goes, either, but I did some work in this direction in the last year, and Adam Perlin did a lot more. Something might line up so that I see a place to contribute.
Are there more details about this? issue tracker or other links? It seems Adam Perlin did a master's thesis about this. Is it online somewhere? Thank you!
I agree that there really is a core group of developers (many of whom have posted in this thread) but the people with push access to racket/racket is a quite large set of people that doesn't accurately line up with it. The Racket Project Leadership listed at the bottom here Racket is a somewhat more accurate list, although it's not exactly right either.
No surprise, it looks like Cal Poly hasn't yet put that master's thesis up publicly. I've written to Adam to make sure he's okay with me sharing it. Related: Adam will be talking about this work at RacketCon in Chicago this year, you should consider attending!
Typed Racket type inference. My student Fred Fu and I have been looking at improving Typed Racket's support for type inference, and have developed interesting prototype ideas. This is far away from being something that would be deployed in Typed Racket itself, though, and it might not even work at that scale or might have serious backwards-compatibility problems.
The only type inference Racket seriously lacks is determining types of parameters and results when polymorphic functions are involved.
For the rest, explicit type specifiction is just fine.
