Racket v8.5 release thread

Yes, I agree that the existence of BC muddies the waters a bit. I'm guessing that even with full support for racket on pb and the improvements that come from pbchunks, that BC would be a generally better choice on those platforms. That's a total guess, though.

I'm also curious to know whether BC "still works right" for all of the packages. I don't think we have a server running pkg-build tests using BC.

I must be misunderstanding. Don't you have to pay that cost anyway, to compile the code before running it? Is there substantial work involved in storing the compiled representation, or am I missing something else?

It's hard to get the words right here... I would say that the reduction occurs for the unchanged files. That is: if I have ten files, and I'm changing one each time, but I don't know about compilation, then it's the price of the unchanged files that I could be saving every time. No?

I think just dropping the word dramatically is good.

If you have the following files, where A depends on B which depends on C:

- A.rkt
- B.rkt
- C.rkt
- compiled/B_rkt.zo

If you change C.rkt, it will in many cases be possible to use B_rkt.zo anyway. But raco make and racket -y will recompile everything. So it can take much longer to run raco make x.rkt than racket x.rkt.

That's dangerous! You need to know a lot for that not to go wrong (especially if TR is involved)!

That can certainly go wrong, but right now, if you edit some part of collects/racket and then run a file, it runs pretty fast, but if you run racket -y thatfile.rkt it will take a long time. I tried the following:

  • create x.rkt with just #lang racket
  • run it. takes about .8 seconds on my laptop.
  • edit racket/private/cond.rkt.
  • run x.rkt again. About 1.4 seconds.
  • run racket -y x.rkt. Took over 2 minutes.

I agree that this is potentially dangerous but it's something that works now and people do similar things regularly, and it will take a long time with -y. So it's a thing people should be aware of.

I'm unreasonably excited for this

1 Like

I'm almost there. I'm having trouble imagining under what circumstances B_rkt.zo could be used after a change to C.rkt. Isn't it possible that the change to 'C.rkt' introduces some new bindingā€”especially a macro bindingā€”that will change the meaning of B.rkt, and wouldn't it be almost impossible to figure that out without compiling C.rkt? Does racket have a system that can do enough of compilation to determine that the changes to C.rkt don't affect the meaning of B.rkt? I see from your example that this is a real effect, I'm just trying to figure out how it works.

Taking "dramatically" out, I now have these two possible wordings for this item:

  • Racket's new -y flag automatically keeps compiled files up to date,
    reducing load times.

  • Racket includes a new flag (-y) that automatically stores a compiled
    version of the code being run.

I agree that the first one does a better job of communicating why you might want this, but I'm slightly concerned by Sam's examples of outlier examples where it might hurt.

Historically, we've used the word "now" way too much. The word "new" hasn't been as much of a problem, I believe.

Either way, I added your bullet and just deleted the word "now", hope that's okay.

1 Like

Ah - I wondered if it was for a deeper philosophical reason. Either way, thatā€™s great, many thanks!

1 Like

Maybe ā€œpotentially reducing load timesā€? :sunglasses:

ā€” Sharon Tuttle

2 Likes

I think "reducing subsequent load times" is the right change.

1 Like

Yes, it's definitely possible that this produces entirely the wrong behavior. And there is no protection against it. That's what @robby means by "dangerous". But the alternatives are somewhat unpalatable (my experiment is an error, or takes multiple minutes, or skips using compiled code and takes potentially even longer) so that's what we've stuck with.

I also think the release notes should have a note about Zlib and whatever users need to know.

Do you have time to draft a note on this?

The intent here is to allow Racket CS to run on any platform where Racket BC runs ā€” including platforms where Racket CS has no machine-code backend, which is a subset of platforms where Racket BC has no JIT. Run time for Racket code in CS without a machine-code backend is similar to run time in BC without a JIT, although compilation times are still slower in the CS variant (because the Racket BC bytecode compler is implemented directly in C, instead of compiled from Scheme to "portable bytecode" to C).

Here's a suggested rewording to say more:

  • Racket CS runs on platforms where native-code generation is not currently supported (e.g., s390x or ppc64). See "README.txt" in the source distribution for more information on the --enable-pb flag to configure.

Here's an attemptā€”the key question is whether this all is, in fact, true, and then whether there is a shorter or clearer way to say it.

  • Racket CS incorporates Zlib 1.2.11, which is affected by CVE-2018-25032. If Racket CS is configured to use Zlib compression for compiled codeā€”which is not the defaultā€”compiling a specially crafted source file could potentially cause memory corruption, denial of service, or arbitrary code execution. The vulnerability does not affect decompression (i.e. loading compiled code). Racket BC does not use Zlib, and thus is unaffected. Likewise, the file/gzip module implements ā€œdeflateā€ compression in pure Racket, without using Zlib. When building Racket from source, Racket CS can be linked with a fixed copy of Zlib, instead of the included copy, by supplying appropriate flags to configure. We expect to update Racket's copy of Zlib in the next Racket release.

Some parts that are a bit squishy:

  • Racket CS incorporates Zlib: My current tentative understanding is that it always links to Zlib, whether statically or dynamically, even if it does not use it.
  • If Racket CS is configured to use Zlib compression: The main sense is configuring with configure, and it might be more clear to just talk about that. I suspect (but do not know) that there may be some way to use ffi/unsafe/vm to set parameters at run time, but if an attacker can use ffi/unsafe/vm, why bother with Zlib?
  • which is not the default: The aim is to communicate that Racket is not configured to use Zlib compression by default, but also to say so without confusing anyone who has followed this bug too closely. (Initially, it was thought to affect only compression with Z_FIXED, which is not Zlib's default, but it turned out to also affect Z_DEFAULT_STRATEGY.)

Here's my shortened attempt:

  • Those who manually configure Racket CS to use Zlib compression for compiled code should be aware of CVE-2018-25032, which could potentially open the door to a host of attacks.
2 Likes