I see your effort of writing down arguments we have been aware of for some decades, and we have tried to respond to those over time.
Most of the time when I have seen image-based programming discussed in the Racket context, the only one of the above points that has been addressed is the increased development efficiency (as being true but not significant enough to justify a language re-write).
Could you point me to some resources so I could get an idea on the thinking that's been done already on the other issues that arise from this gap?
On a separate note, I suspect I'm not the only one who's not aware of any responses for the other points. This may be indicative of improvement opportunities in how those responses are communicated to people considering ramping-up on Racket.
where you're going with your write-up
I'm working under the assumption that Racket has among its goals the four following ones:
- Support research across as wide a subset of the "programming language design space" as possible. This is a hard problem, but not an impossible one; while a programming language must select defaults behaviors of some sort and certain behavior are incompatible with one another, if there is sufficient extensibility and configurability then it seems almost any possible area of programming language space would be reachable from the default configuration without excessive boilerplate. Even if some subsets of PL space cannot be reached simultaneously in the same execution, it's still beneficial to have a single common substrate which can be used for both fields of research. However, image-based languages in general currently cannot be reached from Racket without significant boilerplate to copy the language's innate functionalities in a compatible format, and supporting such languages natively would not reduce the support for non-image-based programming, so this seems like an open "win" for the language to take.
- Support the use-cases of as much of the Lisp community as possible without compromising the above priority re: PL research. As you can see in e.g. this recent discussion, the lack of support for image-based programming is one of the primary blockers pushing people to learn Common Lisp rather than Racket, despite the myriad other advantages Racket has. Either addressing this directly or finding a workaround to provide the same features would help them significantly in their ability to build interesting things, as well as providing more traffic to bolster the Racket library ecosystem
- Maintain a stable and growing library ecosystem. As mentioned above, modules having hard rather than soft constraints on modifiability seems harmful to the language library ecosystem, and at the very least eliminates one of the primary mechanisms by which the Common Lisp ecosystem has stayed stable despite the language's age and archaic idioms scaring people away. Again, the solution here doesn't necessarily have to be "make modules modifiable", but there needs to be some ergonomic way to change code you don't own and have those changes propagate through the entirety of your program (including other libraries dependent on the modified code) in order to better fulfill this priority.
- Provide a centralized standard for programming language work in both the PL research and Lisp space. This to my understanding is a lesser priority, but does exist as an indirect benefit of the first point, so calling it out for completeness.
Given the above priorities, this feature gap is a limitation in Racket's ability to fully fulfill its mission.
giving an extensive answer
I had a few questions which were either explicitly or implicitly called out in the above post. If you don't have anything else to bring up, could you answer those (assuming you have the required context)?
- What benefits does Racket gain from fully encapsulated modules that cannot in-practice be gained via merely-encapsulated-by-default modules with escape-hatches of some sort to allow modification?
- In what ways does the Racket ecosystem mitigate bus factor, given the current behavior of full encapsulation disallows Common Lisp's approach to the problem? (Note: IME most mainstream language ecosystems don't have a solution and do deal with the negative consequences of that, so positively comparing Racket to them in this field doesn't eliminate the issue)
- When writing complex DSLs or general purpose languages requiring partial-recompilation of the computation-graph on top of Racket, what kinds of design patterns are used to work around Racket's encapsulation model? Do designers just give up on reusing the existing machinery, and instead implement their own function-table/package systems instead so that code-forms remain open to redefinition?
- New: "Each member has chosen a different point, some more dynamic some more static." - I'm curious if you could elaborate on why you view the dynamic-static axis as relevant? My model of image-based programming is that it is both compatible with static analysis and with executing static programs, which would make it less an element of the dynamic-static axis and more an orthogonal feature which can either be available or not, with no clear benefits from "not".
- New: More from a "marketing" perspective, aside from the library ecosystem, are there any fundamental benefits of Racket's structure and runtime which cannot (unlike hygenic macros or the
#langreader system) be implemented via macros and reader macros in other Lisps? (And for completeness, are there any constraints on this level which the community is working to fix and could benefit from input from those experienced with other languages who may end up reading this thread?) - New (and potentially off-topic for this thread): I've been modeling the other problems I briefly mentioned (no effect-modeling system for errors, lower introspection into entities in a running Racket instance compared to CL) as not currently blocked from being on Racket's roadmap, unlike image-based programming where there have been explicit responses on the topic. Given you mentioned earlier that some of the communities responses re: image-based programming aren't easily found, I'm curious whether that assumption is correct?