Rationale for per-version user scope packages?

Yes.

I'd be fine personally with a key (name . <str>) in this file affecting all installations, but maybe some people may want version exceptions, in which case the entry could be

(name . ((default . <str>)
         ("8.2" . <str>) ; this one is a release
         ("8.3.0.10" . <str>)) ; this one is a snapshot

Edit: or maybe even just

(name . ("default" "8.2" "8.3.0.10"))

Probably the environment variable would support only a single name though.

I've implemented this, but I'm not enthusiastic about the end result. It seems like only a small improvement over creating a link from one name to another within the addon directory (i.e., ln -s <old-name> <new-name> within the (find-system-path 'addon-dir) directory).

A separate mapping file supports a default mapping, and it avoids link trouble on Windows. Are those benefits enough, given that it would work only for v8.4.0.3 and later?

2 Likes

Thank you for spending time on this!

A separate mapping file supports a default mapping, and it avoids link trouble on Windows. Are those benefits enough, given that it would work only for v8.4.0.3 and later?

Yes. The most important benefit to me is to smooth the UX for the (I suspect) 90% of users would only ever use their latest install of Racket, and fear upgrading. (And it would simplify my own workflow, making it easier to upgrade the Racket install more often than currently.)

However, possibly a better and simpler approach just occurred to me:
No need for even a new file, just use the names of the directories within (find-system-path 'addon-dir):

  • if a directory with (version) exists, use it;
  • otherwise, if the directory default exists, use it;
  • otherwise, create the default directory and use it.
  • raco pkg migrate <from-version> (where <from-version> can now also be default) would work just as before, and copy from from-version to a new directory with the current version name, then creating a version-specific user scope.
  • raco pkg config --create-user-scope would merely create the version-specific directory without migrating anything. Maybe raco pkg config --delete-user-scope would delete the directory.
  • Possibly (format "~a-~a" (version) (system-type 'gc)) would be more helpful than a mere (version) for the directory, for some of the core devs.

This should solve all the issues, while (almost) not changing anyone's workflow, and avoiding a new config file:

  • Users used to raco pkg migrate would see no difference.
  • Users having always a single installation of Racket would not need to migrate anymore, but a raco setup would need to be performed at some point (either at the end of the installer, or the first time racket is run). Since the previous version number is not necessary anymore, this can be automated.
  • Snapshots would still have a shared folder by default.
  • The one difference will be for (experienced) people who reinstall all their packages after each new release install, and often switch between various release versions. They would need to run raco pkg config --create-user-scope before doing so, or use raco pkg migrate, if they want separate user scopes per version. Or they can also rely on default and just run raco setup when they switch versions, if they switch rarely enough.

How does it sound?

I like this idea, except for having default be the default if no directories exist. Running raco setup on install (to accommodate installing a new version after a previous version) is not a trivial requirement to add, either for users or for the installer/DrRacket implementation.

Especially for that reason, maybe default isn't the right name. How about any-version?

I like this idea, except for having default be the default if no directories exist. Running raco setup on install (to accommodate installing a new version after a previous version) is not a trivial requirement to add, either for users or for the installer/DrRacket implementation.

It shouldn't be too hard to make DrRacket (or racket) detect that it's running for the first time since installation. For a simple example, it could read/write to a file named last-run-version, and if different from the current version it asks whether to run raco setup (or just do it), and then write the current version to last-run-version.

That would even accommodate switching between versions with a shared user-scope.

Especially for that reason, maybe default isn't the right name. How about any-version ?

SGTM. Or 'other-versions'.

Even just starting DrRacket might not work, because DrRacket itself might be installed as a package, or something that it depends on might be installed. That can happen easily if you start from Minimal Racket; maybe few people do that. But it also happens when a package moves into the main distribution for use by DrRacket, but the package was installed by users (perhaps as a dependency of something else) with older versions.

This is probably only the tip of the iceberg. Racket installations work in many different configurations, and various defaults and constraints are all balanced against each other.

Even just starting DrRacket might not work,

For the non-power user, the DrRacket launcher could be a shell script that does the last-run-version thing, and then starts DrRacket. (edit: on Unix at least, it is already a shell script.)

I'm not really worried about the other users. Command-line users won't be afraid to run a simple raco setup.

This is probably only the tip of the iceberg. Racket installations work in many different configurations, and various defaults and constraints are all balanced against each other.

I trust you on this.

But when do you envision the any-version directory to be created, if not automatically when none exists?

I've pushed support for an other-version directory, so we can see how that works.

There could be some tool to help users opt in — maybe raco, maybe raco pkg config, or maybe something different. I don't really know the best way to do that, but it needs to be opt-in for a user who is up to the complexity of managing packages at that level. The complexity is not just raco setup, but also uninstalling packages that conflict with an installation, and probably more.

For more automatic/default handling of previously installed packages, something based on package migration still seems more promising to me. Someone could try adding that to the Unix and Windows installers (and maybe just not do anything for Mac OS). Or maybe adjusting DrRacket offer to try a migration is the right idea.

2 Likes

I haven't been able to follow all the details, but am wondering a bit about the naming.

  • any-version would mean to me that this directory is always used (possibly in addition to others).
  • other-version would mean to me that this directory is only used as a fallback if there's no version-specific directory.

Does the currently-anticipated directory name and its usage match this understanding?

other-version would mean to me that this directory is only used as a fallback if there's no version-specific directory.

Yes, that's the intent.

Awesome, thank you so much! This will appear in the next snapshot I guess?

There could be some tool to help users opt in — maybe raco , maybe raco pkg config , or maybe something different. I don't really know the best way to do that, but it needs to be opt-in for a user who is up to the complexity of managing packages at that level. The complexity is not just raco setup , but also uninstalling packages that conflict with an installation, and probably more.

Well, that's a start at least, we may find our way up after playing some more with the new setup.

For more automatic/default handling of previously installed packages, something based on package migration still seems more promising to me. Someone could try adding that to the Unix and Windows installers (and maybe just not do anything for Mac OS). Or maybe adjusting DrRacket offer to try a migration is the right idea.

We should now be in a better place to do these in any case. I may get to modify the Unix installer at some point at least (can't really test the others).

Yes, now available from the Utah site.

Thanks!

  1. Downloaded and installed racket-8.4.0.3-x86_64-linux-xenial-cs.sh from Utah, as a Unix-style installation
  2. Moved ~/.local/share/racket/snapshot to ~/.local/share/racket/other-version.
  3. raco setup went fine, apart from some errors like this one
delete-file: cannot delete file
  path: /usr/bin/raco
  system error: Permission denied; errno=13
  context...:
   /usr/share/racket-8.4.0.3/collects/launcher/launcher.rkt:177:0: install-template

but I believe they also appeared in previous versions (I usually don't use raco setup but only raco setup <user-scope-packages> with this script which doesn't error)
4. raco setup --doc-index warns

raco setup: WARNING: failed to find info for path: /home/laurent/.local/share/racket/snapshot/pkgs/shlex/doc/shlex/out0.sxref

and fails with

raco setup: 0 rendering: <pkgs>/racket-index/scribblings/main/search.scrbl
start-transaction: attempt to write a readonly database
  error code: 8
  SQL: "BEGIN IMMEDIATE TRANSACTION"
  database: #<path:/usr/share/racket-8.4.0.3/doc/docindex.sqlite>
  mode: 'create
  file permissions: (read)
  context...:
   /usr/share/racket-8.4.0.3/collects/db/private/sqlite3/connection.rkt:515:0: handle-status*

Is there something I need to configure to tell it to use other-version instead of snapshot?


Edit: after moving back to snapshot, I get the same error for raco setup --doc-index

After reinstalling racket-8.3.0.5-x86_64-linux-xenial-cs.sh, I also get this error on raco setup:

raco setup: launcher: <console-bin>/raco
delete-file: cannot delete file
  path: /usr/bin/raco
  system error: Permission denied; errno=13
  context...:
   /usr/share/racket-8.3.0.5/collects/launcher/launcher.rkt:177:0: install-template

and this error on raco setup --doc-index:

raco setup: 0 rendering: <pkgs>/racket-index/scribblings/main/user/search.scrbl
dynamic-require: name is not provided
  name: 'deserialize-info:cls/intf-v1
  module: #<resolved-module-path:"/usr/share/racket-8.3.0.5/pkgs/scribble-lib/scribble/private/manual-class-struct.rkt">
  context...:
   /usr/share/racket-8.3.0.5/collects/racket/private/serialize.rkt:697:2: deserialize

so it may not be related to the recent change

The errors you get from not being able to write to the installation make sense. You could try raco setup --avoid-main.

The warning where "snapshot" appears in the path is probably because the previous build's documentation-index information embedded the "snapshot" path, which no longer exists. I'm not clear on why that information didn't get rebuilt with raco setup, though.

on racket-8.3.0.5-x86_64-linux-xenial-cs.sh, raco setup --avoid-main then raco setup --doc-index both worked.

Reinstalled racket-8.4.0.3-x86_64-linux-xenial-cs.sh without changing snapshot. raco setup --avoid-main works, but raco setup --doc-index still fails with:

start-transaction: attempt to write a readonly database
  error code: 8
  SQL: "BEGIN IMMEDIATE TRANSACTION"
  database: #<path:/usr/share/racket-8.4.0.3/doc/docindex.sqlite>
  mode: 'create
  file permissions: (read)

But raco setup --doc-index --avoid-main succeeds.

Then
mv snapshot other-version
both raco setup --avoid-main and raco setup --doc-index --avoid-main work and the docs page is indeed ~/.local/share/racket/other-version/doc/index.html

:tada:

After installing the 8.4 release today, it seems the release does not look for other-version, is that correct?

$ raco setup --avoid-main && raco setup --avoid-main --doc-index
raco setup: version: 8.4
raco setup: platform: x86_64-linux [cs]
raco setup: target machine: ta6le
raco setup: installation name: 8.4
raco setup: variants: cs
raco setup: main collects: /usr/share/racket/collects/
raco setup: collects paths: 
raco setup:   /home/laurent/.local/share/racket/8.4/collects
raco setup:   /usr/share/racket/collects/
raco setup: main pkgs: /usr/share/racket/pkgs
raco setup: pkgs paths: 
raco setup:   /usr/share/racket/pkgs
raco setup:   /home/laurent/.local/share/racket/8.4/pkgs
raco setup: links files: 
raco setup:   /usr/share/racket/links.rktd
raco setup:   /home/laurent/.local/share/racket/8.4/links.rktd

That's right. The user-specific installation-name mapping is 8.4.0.3 and later.

Ah of course. Is there a way to test whether it works properly when installing a release without having to wait for the next actual release? (No big deal though)

On a different machine, I confirm this worked smoothly!

Right after installing 8.10,

  • cd ~/.local/share/racket.
  • mv 8.10 8.10-old (I suppose it can just be deleted too)
  • raco setup --avoid-main ; raco setup --doc-index

Voilà, seamless transition!

Thanks @mflatt !