Draft of 8.12 release notes

Here's a first draft of the release notes for version 8.12. Please let me know about edits you'd like to make, missing information, and other important things!


Racket automatically sets the close-on-exec flag when opening a file, on systems where this is available. This is a lower-cost way to avoid problems that can occur when file descriptors become accidentally shared between processes.

Match includes hash and hash* patterns.

The vector-set/copy function allows creation of a new vector that differs
at only one index.

The pregexp-quote function brings the functionality of regexp-quote
to pregexps.

The C FFI convention-based converter supports PascalCase and CamelCase in
addition to an underscore-based convention.

The racket/case library allows case-like forms that use different
equality comparisons, such as eq? and equal-always?.

The for/fold form has a more consistent treatment of the rare situation
in which an iteration clause shadows an accumulator.

Scribble rendering to HTML includes hidden buttons on heading titles.

The ''Die Macht der Abstraktion'' language levels are no longer present,
replaced by the "Schreibe dein Programm'' language levels which have been
available for several years.

The interval-map data structure supports iterator functions in the style
of gen:ordered-dict.


1 Like

It was suggested here https://github.com/racket/racket/issues/4843 that the release announcement could include raco pkg migrate upgrade instructions.

@spdegabrielle , I think you're planning to start with the release notes bullets and add instructions about upgrading, is that right?

@shhyou @jbclements
Thanks for the reminder to close that issue as done

Some use platform specific repos to install racket so I’d be in favour of keeping the raco pkg migrate 8.11.1 advice text at the end of the announcement for the immediate future.

I think we will eventually get to a place where you can type raco up to get new releases or nightlies depending on the ‘channel’ you have subscribed and avoid depending on platform specific package installation or needing to run raco pkg migrate. (If anyone has strong feelings about this please start a new topic because it isn’t applicable to 8.12)

In the meantime - and for 8.12 I feel it is good to help the user.

I am planning a new racket survey - it is an opportunity to get feedback about the install and upgrade experiences; Survey planning

  1. I think we should move the incompatible changes (removing DMdA and changing for/fold) higher up and note that they may break things.
  2. The vector changes should also mention optimization of other operations such as vector-append.
  3. The "lower-cost way" phrasing is confusing -- what is it lower-cost than?

@samth: (1) thanks, done. (2), done. (3) Here's the commit message I'm condensing:


Matthew Flatt mflatt@racket-lang.org 2023-12-10T02:28:32Z
529f14dd3467ac4262878a6b5792d30b39950b43

set FD_CLOEXEC on all file descriptors and change subprocess handling

Intended to address #4836, this is a relatively small change to the
implementation, and I think it will work right for nearly all uses of
Racket. Still, changing the rules here might create a file-descriptor
leak in a program that uses both foreign libraries or embeds Racket
and that also uses subprocess. Setting
current-subprocess-keep-file-descriptors to null (which is an
existing part of the Racket API) should restore the old behavior in
that case.

The root of the problem is that a fork+exec on Unix keeps all file
descriptors from the original process open in the new process. Racket
has long used a traditional correction of closing all file descriptors
in a forked process, specifically by trying close on every file
descriptor up to the maximum value. That worked fine when the
file-descriptor limit was something like 256 or 1024. These days, it
can be 1M, and 1M erroring close system calls can take a while. Many
OSes offer an enumeration of open file descriptors through a
"/proc/self/fd" or /dev/fd" device that acts like a directory, but
accessing that information is not completely reliable (due to the
possibility of chroot, for example), and it's not so easy in the
phase between fork and exec that permits only async-signal-safe
functions.

A modern alternative is to set the FD_CLOEXEC flag on descriptors so
that they are automatically closed on the exec step, and leave
FD_CLOEXEC off on the file descriptors that you intend to be
comunicated to the subprocess. This strategy works as long as everyone
plays along, because FD_CLOEXEC is not the default. Also, if you have
fork calls that might happen concurrently with file-descriptor
creation, the two-step process of creating a file descriptor and then
setting its FD_CLOEXEC flag is another problem. Linux helps to avoid
the race by having a variant of each descriptor-creating system call
with an extra flag to set FD_CLOEXEC, but that's not portable.

The change here adapts Racket at the rktio level to set FD_CLOEXEC on
all created file descriptors, but mostly through portable APIs (i.e.,
not Linux-specific) plus an extra lock to prevent a subprocess-based
fork concurrent to a file descriptor's creation. The extra lock
works because rktio controls the call to fork. Foreign libraries are
expected to play along as well as they can, and libraries like glib,
sqlite3, and Gtk+ do seem to work that way (i.e., they set FD_CLOEXEC,
and they tend to use the atomic interface where available).

Meanwhile, the old behavior for subprocess is still available by
setting the current-subprocess-keep-file-descriptors parameter to
null --- an API that was already in place, because Racket's behavior
on Windows was already closer to the FD_CLOEXEC world. Also, if
FD_CLOEXEC is not available from the operating system (as determined
at compile time), everything works as before.

There's no Racket-level or even rktio-level way here to create file
descriptors without FD_CLOEXEC when FD_CLOEXEC is supported, except
for pipes at the rktio level. Adding a way to do that would make
sense, if it one day seems useful. Even better, subprocess could
take a list of file descriptors to share, which seems more
declarative; the null value of
current-subprocess-keep-file-descriptors was meant to support an
extension in that direction.

The Windows implementation of subprocess was closer to right before,
mainly because non-sharing of file handles is the default there. The
way subprocess created inherited stdin, stdout, and stderr pipes
created a race among calls to subprocess in parallel places,
however, and that's fixed here.


My reading of this (and a quick read of the commit suggests that I could be right) is that it replaces a potentially-very-expensive scan of all open file descriptors with an automatic close-on-exec flag whenever it's available. So the change should make the new version of racket lower-cost than the old one. I've rephrased this bullet to hopefully make that more clear. Here's my new proposed text:


The Die Macht der Abstraktion'' language levels are no longer present, replaced by the Schreibe dein Programm'' language levels which have been
available for several years.

The for/fold form has a more consistent treatment of the rare situation
in which an iteration clause shadows an accumulator. This could break code
that depended on the old behavior.

Racket automatically sets the close-on-exec flag when opening a file, on
systems where this is available. This change lowers the cost of avoiding
problems that can occur when file descriptors become accidentally shared
between processes.

Match includes hash and hash* patterns.

The vector-set/copy function allows creation of a new vector that differs
at only one index. This change also adds vector-append and vector-copy
primitives.

The pregexp-quote function brings the functionality of regexp-quote
to pregexps.

The C FFI convention-based converter supports PascalCase and CamelCase in
addition to an underscore-based convention.

The racket/case library allows case-like forms that use different
equality comparisons, such as eq? and equal-always?.

Scribble rendering to HTML includes hidden buttons on heading titles.

The interval-map data structure supports iterator functions in the style
of gen:ordered-dict.


The FFI identifier convention change should note that the old convention, which misbehaves (named camelcase but actually does PascalCase), is kept as is, while two new conventions are added.

Edit: Also note that camel case is stylized as “camelCase”, if it is to be stylized.

@usao My thinking was that existing users are unlikely to be burned by the fact that the existing wrongly-named converter is kept as is, and indeed that people might be confused by mentioning it. I also think the docs are pretty clear. Insofar as the release notes are attracting new users to a feature, I think the new modes (and the existing correctly-named one) are the ones to draw attention to. It's perhaps also worth noting that the final release notes will include links to documentation.

I totally agree that the string "CamelCase" should instead be "camelCase", and I've made that change in my copy. Let me know if I'm misunderstanding you, or if I've missed something!

Was the acmart.cls update in the previous release, or this one? If not the previous one, how about adding this bullet:

  • The scribble/acmart language uses v2.01, which avoids errors about the hyperref package in some latex installations

?

Um... it looks to me like that commit (39f96fc58baba913b4) was done after branch day, and therefore will be a part of the next release... ? But I'll put this bullet aside for the 8.13 release notes!

https://pre-release.racket-lang.org/status/?repo=racket/scribble

Oh! Okay. I guess that was intentional, but one of the changes in acmart is a bugfix and, without it, acmart papers won't build on, at least "TeX Live 2023/Debian", due to the following error:


! Package hyperxmp Error: hyperref must be loaded before hyperxmp.

See the hyperxmp package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                             

l.276 }

Are you suggesting maybe we should add a bullet about the known problem?

No, I was suggesting taking the commit. But I'm okay not to. Maybe it is too late at this point.

I'm ... optimistic that our power users will be able to handle this? I would really prefer not add anything to the release at this point, even though a scribble change is probably relatively unlikely to cause problems.

That makes sense to me.

Hi John,

It appears that my email to you went to the spam folder or got lost, so let me repeat my reply here:

  • For Scribble, I think "Scribble rendering to HTML adds linking and information buttons on hovering over heading titles." would work better.
  • Could you also compile the contributor list ("The following people contributed to this release:") now?

Is there a way to make sure I can communicate with you? I also tried Discord, but that didn't work either.

Thanks!
Sorawee

Many thanks for your persistence, and please accept my apologies. Here's the updated release notes; I replaced the word "on" with "when" in one place.


The Die Macht der Abstraktion'' language levels are no longer present, replaced by the Schreibe dein Programm'' language levels which have been
available for several years.

The for/fold form has a more consistent treatment of the rare situation
in which an iteration clause shadows an accumulator. This could break code
that depended on the old behavior.

Racket automatically sets the close-on-exec flag when opening a file, on
systems where this is available. This change lowers the cost of avoiding
problems that can occur when file descriptors become accidentally shared
between processes.

Match includes hash and hash* patterns.

The vector-set/copy function allows creation of a new vector that differs
at only one index. This change also adds vector-append and vector-copy
primitives.

The pregexp-quote function brings the functionality of regexp-quote
to pregexps.

The C FFI convention-based converter supports PascalCase and camelCase in
addition to an underscore-based convention.

The racket/case library allows case-like forms that use different
equality comparisons, such as eq? and equal-always?.

Scribble rendering to HTML adds linking and information buttons when
hovering over heading titles.

The interval-map data structure supports iterator functions in the style
of gen:ordered-dict.

The following people contributed to this release:

Alex Harsányi, Alex Knauth, Alex Muscar, Alexis King, Ben Greenman, Bert
De Ketelaere, Bob Burger, Bogdan Popa, Chris Payne, Fred Fu, J. Ryan
Stinnett, Jamie Taylor, Jared Forsyth, Jarhmander, Jens Axel Søgaard,
Joel Dueck, John Clements, Jordan Johnson, Ken Harris, Laurent Orseau,
Mao Yifu, Marc Nieper-Wißkirchen, Matteo d’Addio, Matthew Flatt,
Matthias Felleisen, Micah Cantor, Mike Sperber, naveen srinivasan, Oscar
Waddell, Philip McGrath, Philippe Meunier, Robby Findler, Rocketnia, Sam
Phillips, Sam Tobin-Hochstadt, Sarthak Shah, Shu-Hung You, Sorawee
Porncharoenwase, Stephen De Gabrielle, Tom Price, ur4t, Wing Hei Chan,
and ZhangHao


(‘more’ comes w/o comparison; see Strunk and White. And frankly for/fold
was buggy before or violated common-sense binding structure.)

“”"
The release fixes a problem with the binding structure of the for/fold form
concerning the rare situation in which an iteration clause identifier shadowed
an accumulator identifier. This change may break that depended on the
old binding structure.
“””

Sure. How about this?


The "Die Macht der Abstraktion" language levels are no longer present,
replaced by the "Schreibe dein Programm" language levels which have been
available for several years.

The release fixes a problem with the binding structure of the for/fold
form in the rare situation when an iteration clause identifier shadowed
an accumulator identifier. This change may break code that depends on
the old binding structure.

Racket automatically sets the close-on-exec flag when opening a file, on
systems where this is available. This change lowers the cost of avoiding
problems that can occur when file descriptors become accidentally shared
between processes.

Match includes hash and hash* patterns.

The vector-set/copy function allows creation of a new vector that differs
at only one index. This change also adds vector-append and vector-copy
primitives.

The pregexp-quote function brings the functionality of regexp-quote
to pregexps.

The C FFI convention-based converter supports PascalCase and camelCase in
addition to an underscore-based convention.

The racket/case library allows case-like forms that use different
equality comparisons, such as eq? and equal-always?.

Scribble rendering to HTML adds linking and information buttons when
hovering over heading titles.

The interval-map data structure supports iterator functions in the style
of gen:ordered-dict.

The following people contributed to this release:

Alex Harsányi, Alex Knauth, Alex Muscar, Alexis King, Ben Greenman, Bert
De Ketelaere, Bob Burger, Bogdan Popa, Chris Payne, Fred Fu, J. Ryan
Stinnett, Jamie Taylor, Jared Forsyth, Jarhmander, Jens Axel Søgaard,
Joel Dueck, John Clements, Jordan Johnson, Ken Harris, Laurent Orseau,
Mao Yifu, Marc Nieper-Wißkirchen, Matteo d’Addio, Matthew Flatt,
Matthias Felleisen, Micah Cantor, Mike Sperber, naveen srinivasan, Oscar
Waddell, Philip McGrath, Philippe Meunier, Robby Findler, Rocketnia, Sam
Phillips, Sam Tobin-Hochstadt, Sarthak Shah, Shu-Hung You, Sorawee
Porncharoenwase, Stephen De Gabrielle, Tom Price, ur4t, Wing Hei Chan,
and ZhangHao