# Rationale for per-version user scope packages?

**URL:** <https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543>\
**Category:** General\
**Created:** [January 10, 2022, 11:33am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543 "2022-01-10T11:33:08Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 10, 2022, 11:33am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/1 "2022-01-10T11:33:09Z")

</div>

Currently user-scope packages for `releases` are installed in version-specific directories.

The main advantage I see is that if someone has several versions of racket installed, there is no need to recompile when switching between versions.

Are there other pros?

There are important cons IMO:

- Need to run `raco pkg migrate <from-version>` after each new update of Racket, where from-version can be annoying to find out.
- Snapshots packages are not installed in the same way as release ones, leading to even more discrepancy, and more headaches when updating from a snapshot to a release, or conversely. Packages may even need to be reinstalled manually.
- Packages are installed multiple times, but are not uninstalled, leading to package-specific configuration issues (e.g. [quickscript-extra being registered multiple times](https://github.com/Metaxal/quickscript-extra/issues/27)).

I've seen a few people refraining from upgrading Racket in fear of all the works that may need to be done to migrate/update packages. ( **edit:** see also [this issue](https://github.com/racket/racket/issues/3887).)

However, I expect that only very few people use multiple versions of Racket on the same user account (the core devs would be my best guess), and assuming instead by default that there is always only a single Racket installation (unless otherwise specified), and thus installing user-scope packages in a global user-scope directory, would solve all this issues, and would make upgrading Racket significantly smoother, and `raco pkg migrate` would become superfluous.

For people who use multiple versions of Racket, there is already the `--scope-dir`, but it may be even better to tell raco pkg to subsequently install all future user-scope packages in special directory, such as `raco pkg install set-user-scope-dir <dir>`.

What do people think?

---

<div class="post-metadata">

**Author:** ![sschwarzer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/sschwarzer/32/1940_2.png) [@sschwarzer](https://racket.discourse.group/u/sschwarzer)\
**Post date:** [January 10, 2022, 10:52pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/2 "2022-01-10T22:52:24Z")

</div>

Could one argument for separate directories be that already installed packages may break when the Racket version is updated? I don't have long-term experience with Racket and don't use many packages, so I don't know how often something like that happens.

---

<div class="post-metadata">

**Author:** ![jbclements](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/jbclements/32/11_2.png) [@jbclements](https://racket.discourse.group/u/jbclements)\
**Post date:** [January 11, 2022, 1:37am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/3 "2022-01-11T01:37:00Z")

</div>

Sounds scary to me. I understand that I probably switch version of DrRacket more than most people, so in some sense my convenience is not what we should optimize for, but I find that it's pretty easy to re-install the packages that I need.

I think the key, perhaps, is having packages that require all of the other packages that I want. When I install a new version of Racket, I require one or two of my own packages, and they pull in all of the other packages that I use. Is that not the case for others?

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 11, 2022, 9:52am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/4 "2022-01-11T09:52:25Z")

</div>

@schwarzer :

> Could one argument for separate directories be that already installed packages may break when the Racket version is updated?

Given how much effort the core devs put into keeping Racket backward compatible, I don't think this argument has a lot of weight 🙂

@jbclements :

> I think the key, perhaps, is having packages that require all of the other packages that I want. When I install a new version of Racket, I require one or two of my own packages, and they pull in all of the other packages that I use. Is that not the case for others?

I almost never switch between versions of DrRacket, but I often reinstall a new snapshot or release. I have 3 machines, all with different package setups (with significant overlap though), some with some packages installed with `--link`, others with the same packages installed more directly.

When updating from a snapshot, I merely need to use [this script](https://gist.github.com/Metaxal/13c4effc483c5ef9f5797ace4950f97a) to update my packages, but when updating release-\>snapshot or snapshot-\>release I need to find out which packages I need to reinstall.

**It may be possible to solve all my problems without changing the default:**

- Assume that `raco pkg install --set-global-user-scope` creates a `user-scope` directory in the racket config subdirectory (on my machine that would be `~/.local/share/racket/user-scope`), and that, subsequently, all packages installed in user scope use this directory, as if using `raco pkg install --scope-dir ~/.local/share/racket/user-scope`.
- Option 1: This directory is added to the Racket preferences so that I don't have to call the line above after every new installation. No more headaches, upgrading Racket is a breeze.
- Option 2: This command must be called after each new Racket install. It's easy to script anyway, and can be integrated with the script mentioned above. However, forgetting to call this command will lead to headaches.

Although that would solve my own problems, I still worry about the remaining 90% of users who just want to keep all their previously installed packages when they upgrade Racket. Forcing them to do the above dance would still not be so helpful because they probably didn't call the magic `--set-global-user-scope` command _before_ installing their packages.

---

<div class="post-metadata">

**Author:** ![sschwarzer](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/sschwarzer/32/1940_2.png) [@sschwarzer](https://racket.discourse.group/u/sschwarzer)\
**Post date:** [January 11, 2022, 10:04am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/5 "2022-01-11T10:04:58Z")

</div>

> [@Laurent.O](#):
>
> Given how much effort the core devs put into keeping Racket backward compatible, I don't think this argument has a lot of weight

It's not only about backward compatibility; a new version could also introduce bugs.

But I agree, this should be rare. Then again, the more packages you use, the more likely it becomes. 😉

---

<div class="post-metadata">

**Author:** ![plane](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/plane/32/67_2.png) [@plane](https://racket.discourse.group/u/plane)\
**Post date:** [January 11, 2022, 11:26am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/6 "2022-01-11T11:26:48Z")

</div>

> [@Laurent.O](#):
>
> - Need to run `raco pkg migrate <from-version>` after each new update of Racket, where from-version can be annoying to find out.

Right now, if you migrate from DrRacket, it shows a nice list of available installations:

![image](https://global.discourse-cdn.com/free1/uploads/racket/original/1X/bd691577c90b55e1076a45bab0a411675d7e0684.png)

If it's annoying to find `from-version` out, perhaps we could fancify the raco migrate interface a bit to include something like this? Right now it's like this:

```scheme
crystal@wings:~$ raco pkg migrate
raco pkg migrate: expects 1 <from-version> on the command line, given 0 arguments

```

This could, instead, provide a list of migrate targets:

```scheme
crystal@wings:~$ raco pkg migrate
raco pkg migrate: expects 1 <from-version> on the command line, given 0 arguments

Available versions for migration:

    7.6
    7.7
    7.8
    7.9
    8.0
    8.1
    8.1-bc
    8.2
    8.3
    snapshot-bc
    snapshot-old

```

Something like that.

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 11, 2022, 11:35am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/7 "2022-01-11T11:35:42Z")

</div>

> Right now, if you migrate from DrRacket, it shows a nice list of available installations

This helps a little, but:

- it requires a manual action, while this should be automated or scriptable,
- many people don't use DrRacket,
- I still don't see a vaguely compelling reason why most users should have their installed packages duplicated each time they upgrade their Racket version, nor why the mechanism for snapshots is different from the one for releases.

---

<div class="post-metadata">

**Author:** ![greghendershott](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/greghendershott/32/98_2.png) [@greghendershott](https://racket.discourse.group/u/greghendershott)\
**Post date:** [January 11, 2022, 2:10pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/8 "2022-01-11T14:10:17Z")

</div>

Although I'm not a Racket core developer I tend to have a dozen versions of Racket installed locally.

Why: Mainly so I can "smoke test" my own packages as well as Racket Mode against older versions of Racket. Although that's a great job for CI, and I do set that up, I still sometimes need to dog-food or debug something locally for an older version of Racket.

(I've sometimes also seen situations where something broke in a specific version of Racket, but was fine before/after.)

So I just wanted to point out that (possibly tiny) space in the set of user scenarios.

* * *

Having said all that, I never install packages in user scope, so, I might be misunderstanding some key aspect of what you're talking about.

* * *

Sometimes I do `raco pkg migrate`. But sometimes like @jbclements I just let `raco pkg update` get only things actually needed transitively by my own packages. This is kind of like housecleaning.

I agree it would be nice if `raco pkg migrate` (perhaps with a "only-actually-needed" option?) would do this automatically after install. However I think the root issue here is that Racket only has an "installer" on Windows? Assuming that is added in some platform-appropriate way, then a package migrate could ride on that?

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [January 11, 2022, 2:27pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/9 "2022-01-11T14:27:40Z")

</div>

@Laurent.O: Apologies if you've covered this and I missed it, but have you tried configuring your release installations' names with `raco pkg config -i --set name release`? That would make release installations behave like snapshot installations, which share because they're all configured with the installation name `snapshot`. You could even use the same name for releases and snapshots to make moving between them easier. In other words, `raco pkg config -i --set name` seems close already to `raco pkg install --set-global-user-scope`.

Sharing by default or auto-migrating packages would require more steps to happen in an installer. In that case, drag-and-drop install wouldn't work on macOS, and there would be room for more things to go wrong while installing on Windows and other platforms.

The different sharing default for snapshots was partly intended as an experiment to see what could go wrong. I think few people use snapshots and mostly they're experts, though, so I'm not sure the experiment has told us all that much.

---

<div class="post-metadata">

**Author:** ![ryanc](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/ryanc/32/71_2.png) [@ryanc](https://racket.discourse.group/u/ryanc)\
**Post date:** [January 11, 2022, 4:26pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/10 "2022-01-11T16:26:07Z")

</div>

Another issue: a package that uses "version exceptions" can have different contents on different versions of Racket.

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 11, 2022, 6:09pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/11 "2022-01-11T18:09:57Z")

</div>

@mflatt

> have you tried configuring your release installations' names with raco pkg config -i --set name release?

I had no idea this was already possible, thank you! Probably what I want is `raco pkg config -u --set name snapshot` if I want to share all user installed packages with snapshots, is that correct? I'll play a little with it.

* * *

**Edit:** This appears to be only for installation-scope packages, so doesn't apply to user-scope packages installed on releases. Or I don't understand what's happening.

* * *

**Edit 2:**

```scheme
$ raco pkg config --set name snapshot
raco pkg config: setting `name` makes sense only ininstallation scope
  current package scope: user

```

I don't get why it doesn't make sense for `user`.

* * *

There's still an outstanding UX issue for the general user: because this shared behaviour is not the default (and is not easy to find), the less experienced user encounters road blocks when upgrading racket, and ends up reinstalling many packages manually the first time, then ends up not reinstalling racket the next time. I wouldn't be surprised if many people didn't know that ones needs to do `raco pkg migrate` or similar in DrRacket, since the installers and the install pages don't mention having to do this.

> The different sharing default for snapshots was partly intended as an experiment to see what could go wrong. I think few people use snapshots and mostly they're experts, though, so I'm not sure the experiment has told us all that much.

To me the experiment is a success, because this default behaviour is much simpler to deal with for a more casual user.

@ryanc

> Another issue: a package that uses "version exceptions" can have different contents on different versions of Racket.

This is a very good point indeed. For these, I would expect `raco setup` to report a warning or error, and `raco pkg update` to fetch the correct version from the catalog(s). Is this what's currently happening?

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 11, 2022, 6:11pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/12 "2022-01-11T18:11:30Z")

</div>

> [@greghendershott](#):
>
> Although I'm not a Racket core developer I tend to have a dozen versions of Racket installed locally.

You're not exactly the typical racket user either I'd say 😉

> Having said all that, I never install packages in user scope, so, I might be misunderstanding some key aspect of what you're talking about.

Interesting.

---

<div class="post-metadata">

**Author:** ![plane](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/plane/32/67_2.png) [@plane](https://racket.discourse.group/u/plane)\
**Post date:** [January 11, 2022, 6:12pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/13 "2022-01-11T18:12:10Z")

</div>

greghendershott wrote:

> Although I'm not a Racket core developer I tend to have a dozen versions of Racket installed

I have at least that many installed, myself.

For what it's worth, I've previously suggested including migration in the installation process with a feature request over on GitHub:

> <https://github.com/racket/racket/issues/3887>
>
> \*\*Is your feature request related to a problem? Please describe.\*\*
> When I was a… new user, I had no idea \`raco pkg migrate\` existed, and I didn't know about DrRacket's equivalent "Copy from Version" feature. I wanted to keep Racket up-to-date by installing new versions, but whenever I updated Racket, it always broke my packages! I was frustrated.
> 
> Once I learned I could easily migrate packages, I was happy! But other new users may have the same problem I did. In fact, on the Racket Discord server, jimpjorps wrote:
> 
> \> it'd be a nice thing to prompt for, my first version was 8.0 and I flailed for a little bit when I upgraded to 8.1 because I thought I had to manually reinstall everything in raco before noticing Copy from Version  
> 
> So I'm not the only one who had trouble!
> 
> \*\*Describe the solution you'd like\*\*
> The installation process should prompt the user when at least one existing Racket installation is detected, asking them if they'd like to migrate the packages from any of those installations. If they say no, the script can mention that the packages can be migrated at any time if the user wishes through a \`raco pkg migrate\` command; the graphical installer can mention "Copy from Version" in DrRacket.
> 
> \*\*Describe alternatives you've considered\*\*
> DrRacket could have a pop-up which appears on the first run after upgrading, asking users if they'd like to migrate their packages from the previous installation. However, this would not help people who install using the command-line installation script.  
> 
> \*\*Do you want to contribute to this feature\*\*
> Yes, I could help!

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 11, 2022, 6:14pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/14 "2022-01-11T18:14:01Z")

</div>

> [@plane](#):
>
> I have at least that many installed, myself.

May I ask why? Do you switch often between these?

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [January 11, 2022, 9:57pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/15 "2022-01-11T21:57:16Z")

</div>

The installation name determines where "user scope" information resides, including user-scope preferences. So, it only makes sense to set that at the installation level, even though its effect is only on user-scope things. It really should affect user-scope packages, not installation-scope packages; it's the mechanism that snapshots use to share.

> [@Laurent.O](#):
>
> > The different sharing default for snapshots was partly intended as an experiment to see what could go wrong. I think few people use snapshots and mostly they're experts, though, so I'm not sure the experiment has told us all that much.
> 
> To me the experiment is a success, because this default behaviour is much simpler to deal with for a more casual user.

We'd need more tool or installation support than currently to make this work for the release. Otherwise, installing a new version would start reporting errors if any package had been installed with the old version. (I expect that few users of a release know about running `raco setup`.)

Or maybe that extra tool support could instead perform a migrate on install, as @plane suggests.

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 12, 2022, 9:22am UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/16 "2022-01-12T09:22:39Z")

</div>

> The installation name determines where "user scope" information resides, including user-scope preferences. So, it only makes sense to set that at the installation level, even though its effect is only on user-scope things. It really should affect user-scope packages, not installation-scope packages; it's the mechanism that snapshots use to share.

I see. So this is an installation-specific configuration then. It seems to me that a user-specific configuration would make more sense, since several users may share the Racket installation but may not want to share the scope structure. More importantly to me, with a user-specific configuration, I wouldn't have to set this preference after each installation. (Possibly the user preferences could have installation exceptions on top of a default value.)

**Edit:** Having this value in the user prefs would also avoid the need for `sudo` when modifying it.

> We'd need more tool or installation support than currently to make this work for the release. Otherwise, installing a new version would start reporting errors if any package had been installed with the old version. (I expect that few users of a release know about running raco setup.)

Clearly a `raco setup` should be done right after installation, or the first time racket is launched.

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [January 12, 2022, 2:12pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/17 "2022-01-12T14:12:45Z")

</div>

> [@Laurent.O](#):
>
> (Possibly the user preferences could have installation exceptions on top of a default value.)
> 
> **Edit:** Having this value in the user prefs would also avoid the need for `sudo` when modifying it.

Just to make sure were on the same page, the installation name can't be stored in user preferences in the sense of `raco pkg config -u`, because the location of that user-specific information is itself determined by the installation name.

It could be in preferences in the sense of `get-preference` from `racket/file`, but that's probably the wrong layer, since the installation name is needed early in Racket startup. Also, the preferences file tends to be large enough that reading it on every startup wouldn't be good.

A new configuration file that is stored in `(find-system-path 'addon-dir)` could work. Also, an environment variable could work. Would one of those be better than the other in your case?

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 12, 2022, 2:17pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/18 "2022-01-12T14:17:04Z")

</div>

> [@mflatt](#):
>
> A new configuration file that is stored in `(find-system-path 'addon-dir)` could work. Also, an environment variable could work. Would one of those be better than the other in your case?

Yes to either or both, with a preference for the first option.

---

<div class="post-metadata">

**Author:** ![Laurent.O](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/laurent.o/32/18_2.png) [@Laurent.O](https://racket.discourse.group/u/Laurent.O)\
**Post date:** [January 12, 2022, 2:25pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/19 "2022-01-12T14:25:15Z")

</div>

FWIW, I'm currently updating my migration script (mentioned above) to do as before when `(get-installation-name)` is either `snapshot` or `release`, and propose migration otherwise (with similar options to the GUI in DrRacket) with an option to set the new `name` to either `snapshot` (to share packages with snapshots and releases) or `release` (to share packages between releases, but not between snapshots and releases).

Being able to modify `name` without `sudo` would be very helpful, as otherwise I'd need to run the script several time with different rights, and/or ask the user to run some commands manually.

---

<div class="post-metadata">

**Author:** ![mflatt](https://yyz2.discourse-cdn.com/free1/user_avatar/racket.discourse.group/mflatt/32/6_2.png) [@mflatt](https://racket.discourse.group/u/mflatt)\
**Post date:** [January 12, 2022, 2:47pm UTC](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543/20 "2022-01-12T14:47:18Z")

</div>

Making sure we have the same idea: If there's a file in `(find-system-path 'addon-dir)` or an environment variable, then it affects all installations. That wouldn't support differentiating between "snapshot" and "release" installations. Does that match what you have in mind?

[Next page](https://racket.discourse.group/t/rationale-for-per-version-user-scope-packages/543.md?page=2)