When I install minimal-racket 8.5 on macOS Monterey on my M1 MacBook Pro, launching the racket REPL takes 15 seconds between printing "Welcome to Racket v8.5 [cs]." to displaying the prompt ">". After raco pkg install compiler-lib, it becomes very fast. Is this expected? Or could it be a side effect of installing something with raco that causes it to compile whatever code is used in REPL startup and make it faster?
3 Likes
A slow startup for racket usually means that timestamps are out of sync so that sources appear to be newer than compiled forms. Installing a package can indeed partially fix that, because compiling the package triggers transitive checks on its dependencies.
What installation method did you use for minimal Racket?
2 Likes
I installed with brew install minimal-racket.
$ time echo | racket -I
Welcome to Racket v8.5 [cs].
>
real 0m15.261s
user 0m14.819s
sys 0m0.216s
It's consistently ~15s on multiple runs. After raco pkg install compiler-lib (which takes several minutes), it drops to <200ms, consistently. Then if I brew rm --force minimal-racket && brew install minimal-racket, it's back to 15s. It seems that installing any package does the trick: scheme-lib and racket-lib worked too. (I couldn't find a package with zero dependencies to try; seems everything relies on base, and there is a circular dependency between base and racket-lib?)
I cannot reproduce with https://download.racket-lang.org/releases/8.5/installers/racket-minimal-8.5-aarch64-macosx-cs.dmg, nor with https://download.racket-lang.org/releases/8.5/installers/racket-minimal-8.5-aarch64-macosx-cs.tgz. They're fast right away. So it seems like a Homebrew issue.
It's installing from a bottle:
==> Pouring minimal-racket--8.5.arm64_monterey.bottle.tar.gz
==> Caveats
This is a minimal Racket distribution.
If you want to build the DrRacket IDE, you may run:
raco pkg install --auto drracket
The full Racket distribution is available as a cask:
brew install --cask racket
==> Summary
🍺 /opt/homebrew/Cellar/minimal-racket/8.5: 1,969 files, 116.1MB
==> Running `brew cleanup minimal-racket`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
I tried brew install --build-from-source minimal-racket and didn't see the issue. So it must be something to do with the Homebrew bottling process. Are there any particular files' mtimes that would shed light on this?
I see this in bottle.rb:
# Ensure tar is set up for reproducibility.
# https://reproducible-builds.org/docs/archives/
gnutar_args = [
"--format", "pax", "--owner", "0", "--group", "0", "--sort", "name",
# Set exthdr names to exclude PID (for GNU tar <1.33). Also don't store atime and ctime.
"--pax-option", "globexthdr.name=/GlobalHead.%n,exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime"
].freeze
However, it doesn't seem to be passing the --time flag recommended in the reproducible-builds.org article. Anyway, I think the next step is to confirm whether the bottle's mtimes are messed up, and then see if we can get minimal-racket to be bottled non-reproducibly, or have the formula run raco pkg install ... or equivalent after unbottling.
I personally never download installers from websites if Homebrew can do it for me, so I think this could affect a lot of people.
Just to note that I, too, saw exactly this behaviour of very slow start-up times after a homebrew install of minimal-racket. (I am also on an M1 Macbook Pro.) I don't have anything helpful to contribute beyond that observation -- I "cheated" and installed the cask version.
1 Like
Just to let you both know you are not alone in the problems with brew on macOS. It has come up before. I briefly looked at how to distribute via homebrew but decided it was more effort than it was worth for me. YMMV.
I don't believe installing Racket via homebrew / brew (or apt) is part of the release testing process. This make me nervous.
Installation guidance has come up before, so I took the liberty of updating the issue Document "main-distribution" and "Minimal Racket" · Issue #4082 · racket/racket · GitHub (by @sorawee):
- referencing a comment by @notjack that I agree with and
- adding some draft wikipages Minimal Racket · racket/racket Wiki · GitHub & Installing Racket · racket/racket Wiki · GitHub
You only need a GitHub account to update the wiki pages so I say please add your tips/guidance/etc. and these can be edited into a short 'Installation guide' for the docs.
bw. Stephen
What I do:(but this may not be right for you)
- use the official full installer at https://download.racket-lang.org
- only use the minimal installer when needed . as @soegaard suggested 'doing a deployment' (e.g. to a web server, DO droplet etc.)
- use the installer at Racket: Racket 8.5 to install Minimal Racket
FWIW building from source also works very well, and both building from source and using the official installers are very well behaved in that they result in self-contained installs that can be removed by deleting the folder.
(I have built from source on my x86 Mac, but I haven't build on M1 yet as I have not needed to)
One useful piece of information would be the output of tree --timefmt "%FT%T%z" /opt/homebrew/Cellar/minimal-racket/8.5 (which will list the mtimes for the Racket installation) immediately after installation vs. after getting to the good state. You can install tree via Homebrew.
If I'm right, raco setup with no arguments should work to get you to the good state instead of raco pkg install ....
1 Like
The script used by brew for minimal-racket is (I think) here [1].
The script fetches racket-minimal-8.5-src.tgz from the racket-lang.org and then
runs the equivalent of:
./configure
make
make install
So it seems raco setup is missing.
This was my approach when I started really using Racket, too, and I agree it's something that ought to be supported, but I no longer recommend it for Racket. Among other reasons:
- When I'm using a package manager, I tend to blindly
brew update(orsudo apt update && sudo apt upgrade, etc.) fairly routinely, but I want to make a more explicit choice about updating Racket, especially because no package manager that I know of deals withraco pkg migrate. - Exacerbating the above, on Mac, I typically configure the default package scope to
installationso that Racket packages providing GUI apps (e.g.drracket) end up in/Applications/Racket v8.5/rather than~/Library/Racket/8.4/, andraco pkg migratedoesn't handle installation scope. (ETA: Apparently Homebrew'sminimal-racketsets installation scope by default as part of the packaging.) - The last time I wrote advice on installing Racket for non-Racketeers, 'the “minimal-racket” Homebrew formula [was] unmaintained and should be avoided.' (The “cask” was fine.) I understand the situation has improved since then—certainly so, if they've updated to 8.5 already!—but I haven't followed the Homebrew packaging closely recently.
2 Likes
./configure && make && make install should itself run raco setup.
1 Like
As a distribution package contributor, I generally think making sure my package of Racket installs correctly is my responsibility (ideally, something for me to test when the release candidate becomes available). But I'm certainly not against Racket making distro packagers' lives easier!
1 Like
Yes, when I run brew install --build-from-source minimal-racket it works fine. The problem is only when building from a bottle (which is the default in most cases).
I've confirmed raco setup is sufficient (no need to install anything). At the start it prints:
raco setup: ignoring compiled files, rebuilding from source...
triggered by use of non-".zo" file
path: /opt/homebrew/Cellar/minimal-racket/8.5/share/racket/collects/compiler/private/cm-minimal.rkt
It seems that the compiled cm-minimal.rkt is actually missing, rather than existing with an incorrect mtime. I ran cd /opt/homebrew/Cellar/minimal-racket/8.5/ && find . -name "*.zo" before and after raco setup and this is the diff: Diff of `find . -name "*.zo"` in /opt/homebrew/Cellar/minimal-racket/8.5/ before and after `raco setup` · GitHub.
So perhaps the bottling process is somehow omitted some compiled files, and it's not messing up mtimes?
One short term improvement would be to add a note about running raco setup in the "Caveats" notice Homebrew prints after installing. I could make a PR for that.
2 Likes
That's very interesting! It looks like some .zo files do exist that expect the source files to be under /usr/local/Cellar/minimal-racket/8.5/share/racket/collects/, but your source files are actually under /opt/homebrew/Cellar/minimal-racket/8.5/share/racket/collects/.
Do you know anything about when Homebrew started using /opt/homebrew? (I haven't used it much in a while, and apparently the brew command on the mac I have ready access to is broken for some other reason, though some homebrew stuff is still around under /usr/local.)
The symptoms, though maybe not the root cause, are related to unix-style install: ".zo"s in "lib" instead of "share" · racket/racket@51b8606 · GitHub, so I would expect passing --enable-sharezo to configure to fix things as a short-term workaround.
Oh sorry I think that's a red herring — I've mostly been testing this on my M1 MacBook Pro at home (Homebrew uses /opt/homebrew for Apple Silicon), but I'm currently at work so my last comment was from my work laptop, which is an Intel MacBook Pro (Homebrew uses /usr/local for Intel). I tried to simplify things by glossing over that and writing /opt/homebrew instead but evidently that was a bad idea 
I have confirmed that the same issue (15s slowdown until raco setup) happens on the Intel laptop though.
When I get home tonight I'll try it on my M1 laptop.
Ah, ok. On the Intel system, do files like /usr/local/Cellar/minimal-racket/8.5/lib/racket/compiled/usr/local/Cellar/minimal-racket/8.5/share/racket/collects/compiler/private/cm-minimal.rkt exist? (I.e., the subdirectory of lib/racket/compiled/ matches the absolute path to share/racket.)
I still think it's related to that commit, but it might be a bug in --enable-macprefix or something about the way Homebrew packs up the built files.
The value of racket -e "(find-compiled-file-roots)" would also be useful.
Ok, back on my M1 Mac. Right after installing from the bottle (before raco setup), there is indeed a file
/opt/homebrew/Cellar/minimal-racket/8.5/lib/racket/compiled/opt/homebrew/Cellar/minimal-racket/8.5/share/racket/collects/compiler/private/compiled/cm-minimal_rkt.zo
and denoting the inner opt/homebrew/.../private as $X, that's:
/opt/homebrew/Cellar/minimal-racket/8.5/lib/racket/compiled/$X/compiled/cm-minimal_rkt.zo
The absolute path /$X/cm-minimal.rkt does exist. Not /$X/compiled/cm-minimal.rkt.
Here is the output of racket -e "(find-compiled-file-roots)":
'(same #<path:/opt/homebrew/Cellar/minimal-racket/8.5/lib/racket/compiled>)
However, I just realized I can't reproduce the 15s delay anymore. It's fast right after installing the bottle, without running raco setup. And after running raco setup there is no diff in the set of .zo files. This is puzzling since it looks like the bottle hasn't changed and I was able to reproduce it consistently yesterday.
That said, if I run
cd /opt/homebrew/Cellar/minimal-racket/8.5
find . -name "*.rkt" -exec touch {} +
Then time echo | racket -i is back to exactly 15 seconds. So I think that's good evidence the original problem was caused by mtime issues.
Practically, I've only used the cask version. Never tried the homebrew minimal-racket, but it is probably an issue with their build (a bottle is basically a pre-built formula; with --from-source you build from source locally). Perhaps they are somehow missing the compiled files in the bottle? (Edit, we may have come to similar conclusions.)
I've created a PR here: minimal-racket: run raco setup post install by mk12 · Pull Request #100990 · Homebrew/homebrew-core · GitHub
Even if using Homebrew isn't the officially recommended way of getting Racket, or minimal-racket is only recommended for deployment scenarios, I think it's inevitable people will do what's most convenient. In my case I wasn't really trying to learn Racket or use it directly, it's just a dependency I need in order to do something else. Typing brew install x y z is a lot easier than looking up the recommended installation procedure for x, y, and z. And minimal-racket appeals to me because I don't want or need the GUI apps.
Of course, I'm more likely to want to explore Racket further if my first impression is a 200ms startup time rather than 15s. So I hope this PR helps provide a smoother on-ramp for macOS users.
2 Likes
Thank you so much. This contribution will make a real difference to many beginners. I can’t wait to try it myself. I’m sorry for being discouraging.
Best regards, Stephen.
This is what I was expecting (just another lesson in the dangers of writing enormous paths by hand.
This is also right.
I definitely agree that making this work is important to giving a good first impression! I used Homebrew's minimal Racket myself for several Racket release cycles when I was relatively new to Racket.
Adding an explicit raco setup step will definitely avoid the immediate problem, but I think it may be doing so by producing a duplicate set of .zo files.
I'll try to describe more of what seems to be happening from the Racket side, in the hope that someone else can figure out, if this is right, how Homebrew might be getting into this state.
Since the Racket commit I mentioned above (February 2021, included in Racket since 8.1), running Racket's ./configure with --enable-origtree=no and --enable-macprefix, as Homebrew does, installs Racket with source files under ${PREFIX}/share/racket and the corresponding .zo files (which are architecture-specific) under ${PREFIX}/lib/racket/compiled/ (plus path elements for the absolute path to the source file). It then arranges for the result of (find-compiled-file-roots) to be something like the one you got:
'(same #<path:/opt/homebrew/Cellar/minimal-racket/8.5/lib/racket/compiled>)
where 'same means a relative path, e.g. foo/info.rkt compiled to foo/compiled/info_rkt.zo.
Racket will look for compiled files in both locations, but, if the compiled files are missing or outdated, it will write compiled files in the first location.
From the diffs you posted, it looks like raco setup is not using the existing compiled files under lib/racket/compiled, very possibly due to mtimes, and is instead generating additional compiled files in compiled directories relative to the source files.
Adding the ./configure flag --enable-sharezo would have Racket install compiled files relative to the source files, so (find-compiled-file-roots) would just produce '(same). That at least would mean that raco setup would use or update the existing .zo files, rather than generating a second set. The downside is architecture-specific files under share/, for those who care about that sort of thing.
Of course, it would be even better to figure out what's going wrong (very possibly with mtimes) that's making Racket not use the compiled files that already exist.
From the Racket side, I wonder if we should use mtimes only as a shortcut to hashing the source file, as Zuo does.