Currently user-scope packages for releases are installed in version-specific directories.
The main advantage I see is that if someone has several versions of racket installed, there is no need to recompile when switching between versions.
Are there other pros?
There are important cons IMO:
Need to run raco pkg migrate <from-version> after each new update of Racket, where from-version can be annoying to find out.
Snapshots packages are not installed in the same way as release ones, leading to even more discrepancy, and more headaches when updating from a snapshot to a release, or conversely. Packages may even need to be reinstalled manually.
I've seen a few people refraining from upgrading Racket in fear of all the works that may need to be done to migrate/update packages. (edit: see also this issue.)
However, I expect that only very few people use multiple versions of Racket on the same user account (the core devs would be my best guess), and assuming instead by default that there is always only a single Racket installation (unless otherwise specified), and thus installing user-scope packages in a global user-scope directory, would solve all this issues, and would make upgrading Racket significantly smoother, and raco pkg migrate would become superfluous.
For people who use multiple versions of Racket, there is already the --scope-dir, but it may be even better to tell raco pkg to subsequently install all future user-scope packages in special directory, such as raco pkg install set-user-scope-dir <dir>.
Could one argument for separate directories be that already installed packages may break when the Racket version is updated? I don't have long-term experience with Racket and don't use many packages, so I don't know how often something like that happens.
Sounds scary to me. I understand that I probably switch version of DrRacket more than most people, so in some sense my convenience is not what we should optimize for, but I find that it's pretty easy to re-install the packages that I need.
I think the key, perhaps, is having packages that require all of the other packages that I want. When I install a new version of Racket, I require one or two of my own packages, and they pull in all of the other packages that I use. Is that not the case for others?
I think the key, perhaps, is having packages that require all of the other packages that I want. When I install a new version of Racket, I require one or two of my own packages, and they pull in all of the other packages that I use. Is that not the case for others?
I almost never switch between versions of DrRacket, but I often reinstall a new snapshot or release. I have 3 machines, all with different package setups (with significant overlap though), some with some packages installed with --link, others with the same packages installed more directly.
When updating from a snapshot, I merely need to use this script to update my packages, but when updating release->snapshot or snapshot->release I need to find out which packages I need to reinstall.
It may be possible to solve all my problems without changing the default:
Assume that raco pkg install --set-global-user-scope creates a user-scope directory in the racket config subdirectory (on my machine that would be ~/.local/share/racket/user-scope), and that, subsequently, all packages installed in user scope use this directory, as if using raco pkg install --scope-dir ~/.local/share/racket/user-scope.
Option 1: This directory is added to the Racket preferences so that I don't have to call the line above after every new installation. No more headaches, upgrading Racket is a breeze.
Option 2: This command must be called after each new Racket install. It's easy to script anyway, and can be integrated with the script mentioned above. However, forgetting to call this command will lead to headaches.
Although that would solve my own problems, I still worry about the remaining 90% of users who just want to keep all their previously installed packages when they upgrade Racket. Forcing them to do the above dance would still not be so helpful because they probably didn't call the magic --set-global-user-scope command before installing their packages.
Right now, if you migrate from DrRacket, it shows a nice list of available installations:
If it's annoying to find from-version out, perhaps we could fancify the raco migrate interface a bit to include something like this? Right now it's like this:
crystal@wings:~$ raco pkg migrate
raco pkg migrate: expects 1 <from-version> on the command line, given 0 arguments
This could, instead, provide a list of migrate targets:
crystal@wings:~$ raco pkg migrate
raco pkg migrate: expects 1 <from-version> on the command line, given 0 arguments
Available versions for migration:
7.6
7.7
7.8
7.9
8.0
8.1
8.1-bc
8.2
8.3
snapshot-bc
snapshot-old
Right now, if you migrate from DrRacket, it shows a nice list of available installations
This helps a little, but:
it requires a manual action, while this should be automated or scriptable,
many people don't use DrRacket,
I still don't see a vaguely compelling reason why most users should have their installed packages duplicated each time they upgrade their Racket version, nor why the mechanism for snapshots is different from the one for releases.
Although I'm not a Racket core developer I tend to have a dozen versions of Racket installed locally.
Why: Mainly so I can "smoke test" my own packages as well as Racket Mode against older versions of Racket. Although that's a great job for CI, and I do set that up, I still sometimes need to dog-food or debug something locally for an older version of Racket.
(I've sometimes also seen situations where something broke in a specific version of Racket, but was fine before/after.)
So I just wanted to point out that (possibly tiny) space in the set of user scenarios.
Having said all that, I never install packages in user scope, so, I might be misunderstanding some key aspect of what you're talking about.
Sometimes I do raco pkg migrate. But sometimes like @jbclements I just let raco pkg update get only things actually needed transitively by my own packages. This is kind of like housecleaning.
I agree it would be nice if raco pkg migrate (perhaps with a "only-actually-needed" option?) would do this automatically after install. However I think the root issue here is that Racket only has an "installer" on Windows? Assuming that is added in some platform-appropriate way, then a package migrate could ride on that?
@Laurent.O: Apologies if you've covered this and I missed it, but have you tried configuring your release installations' names with raco pkg config -i --set name release? That would make release installations behave like snapshot installations, which share because they're all configured with the installation name snapshot. You could even use the same name for releases and snapshots to make moving between them easier. In other words, raco pkg config -i --set name seems close already to raco pkg install --set-global-user-scope.
Sharing by default or auto-migrating packages would require more steps to happen in an installer. In that case, drag-and-drop install wouldn't work on macOS, and there would be room for more things to go wrong while installing on Windows and other platforms.
The different sharing default for snapshots was partly intended as an experiment to see what could go wrong. I think few people use snapshots and mostly they're experts, though, so I'm not sure the experiment has told us all that much.
have you tried configuring your release installations' names with raco pkg config -i --set name release?
I had no idea this was already possible, thank you! Probably what I want is raco pkg config -u --set name snapshot if I want to share all user installed packages with snapshots, is that correct? I'll play a little with it.
Edit: This appears to be only for installation-scope packages, so doesn't apply to user-scope packages installed on releases. Or I don't understand what's happening.
Edit 2:
$ raco pkg config --set name snapshot
raco pkg config: setting `name` makes sense only ininstallation scope
current package scope: user
I don't get why it doesn't make sense for user.
There's still an outstanding UX issue for the general user: because this shared behaviour is not the default (and is not easy to find), the less experienced user encounters road blocks when upgrading racket, and ends up reinstalling many packages manually the first time, then ends up not reinstalling racket the next time. I wouldn't be surprised if many people didn't know that ones needs to do raco pkg migrate or similar in DrRacket, since the installers and the install pages don't mention having to do this.
The different sharing default for snapshots was partly intended as an experiment to see what could go wrong. I think few people use snapshots and mostly they're experts, though, so I'm not sure the experiment has told us all that much.
To me the experiment is a success, because this default behaviour is much simpler to deal with for a more casual user.
Another issue: a package that uses "version exceptions" can have different contents on different versions of Racket.
This is a very good point indeed. For these, I would expect raco setup to report a warning or error, and raco pkg update to fetch the correct version from the catalog(s). Is this what's currently happening?
The installation name determines where "user scope" information resides, including user-scope preferences. So, it only makes sense to set that at the installation level, even though its effect is only on user-scope things. It really should affect user-scope packages, not installation-scope packages; it's the mechanism that snapshots use to share.
We'd need more tool or installation support than currently to make this work for the release. Otherwise, installing a new version would start reporting errors if any package had been installed with the old version. (I expect that few users of a release know about running raco setup.)
Or maybe that extra tool support could instead perform a migrate on install, as @plane suggests.
The installation name determines where "user scope" information resides, including user-scope preferences. So, it only makes sense to set that at the installation level, even though its effect is only on user-scope things. It really should affect user-scope packages, not installation-scope packages; it's the mechanism that snapshots use to share.
I see. So this is an installation-specific configuration then. It seems to me that a user-specific configuration would make more sense, since several users may share the Racket installation but may not want to share the scope structure. More importantly to me, with a user-specific configuration, I wouldn't have to set this preference after each installation. (Possibly the user preferences could have installation exceptions on top of a default value.)
Edit: Having this value in the user prefs would also avoid the need for sudo when modifying it.
We'd need more tool or installation support than currently to make this work for the release. Otherwise, installing a new version would start reporting errors if any package had been installed with the old version. (I expect that few users of a release know about running raco setup.)
Clearly a raco setup should be done right after installation, or the first time racket is launched.
Just to make sure were on the same page, the installation name can't be stored in user preferences in the sense of raco pkg config -u, because the location of that user-specific information is itself determined by the installation name.
It could be in preferences in the sense of get-preference from racket/file, but that's probably the wrong layer, since the installation name is needed early in Racket startup. Also, the preferences file tends to be large enough that reading it on every startup wouldn't be good.
A new configuration file that is stored in (find-system-path 'addon-dir) could work. Also, an environment variable could work. Would one of those be better than the other in your case?
FWIW, I'm currently updating my migration script (mentioned above) to do as before when (get-installation-name) is either snapshot or release, and propose migration otherwise (with similar options to the GUI in DrRacket) with an option to set the new name to either snapshot (to share packages with snapshots and releases) or release (to share packages between releases, but not between snapshots and releases).
Being able to modify name without sudo would be very helpful, as otherwise I'd need to run the script several time with different rights, and/or ask the user to run some commands manually.
Making sure we have the same idea: If there's a file in (find-system-path 'addon-dir) or an environment variable, then it affects all installations. That wouldn't support differentiating between "snapshot" and "release" installations. Does that match what you have in mind?
