Simple overview of how packages/dependencies work in Racket?

Questions & Answers
1

I'm trying to get a grasp of how packages / library dependencies work in racket. I'm going through documentation, however a few pieces are still puzzling.

Let's say I'm trying to develop two different racket projects, each one with their own set of dependencies. There may be a few packages required by both the projects, but the versions may differ. The following are the questions I'm trying to find answers for:

  1. How do I declare dependencies for each of my projects?
    The answer is, I can list my dependencies in info.rkt, and use raco to install them. (However, this wasn't apparent when glancing through the documentation.)

  2. How do I specify versions for my dependencies?
    info.rkt does seem to support version constraints, but the feature doesn't seem to be used widely. I checked info.rkt of many packages in https://pkgs.racket-lang.org/, they list dependencies without version numbers. Am I missing something here?

  3. Where do the packages get installed and how different versions are managed?
    From what I understood so far, raco installs packages to either a system level or a user level directory. How does it work for different projects requiring different version of the same package? Is there a python virtual-environment kind of concept here? (or because each version has a different hash, one central location (user/system level) works fine for all projects?)

Thanks for the responses in advance. I would appreciate if any one can point to a quick Getting Started kind of tutorial regarding project setup, deps, development cycle.

7 Likes
2

Question 1: you answered it! In case it isn't clear, though, package dependencies are of course installed automatically when a package depending on those packages is installed.

Questions 2: Racket had an earlier system, PLaneT, that (IIRC) by default specified versions in package dependencies. The result seemed to be a system in which downstream packages very rarely picked up bug fixes and performance improvements from upstream packages. With the new package system, the idea is that if you want to make breaking changes to a package, you should probably come up with a new package name.

But wait! There's an escape hatch (or maybe an "anti-escape" hatch?). Some people work in environments where it's critical that software not change without warning, and that every update be carefully scrutinized. This is available too; Racket makes it relatively easy for you to construct your own "catalog" of packages. By doing this, you can fix the packages at the versions you want.

Question 3: I'm not answering this question.... I need to go empty the dishwasher!

5 Likes
3

That's not quite right, but I think it's close enough for this discussion....

4

How does it work in reality? Do all packages strive to be backward compatible all the time? Is there the concept of semantic versioning where the package manager accepts minor and patch version changes but restricts major version changes?

2 Likes
5

Here's a link to a conversation that took place during the transition, in 2013. I had forgotten that the planet->pkg transition also made it possible to move a whole lot of code out of the core of racket.

https://lists.racket-lang.org/dev/archive/2013-May/012364.html

6

One of the best explanations I know—I think it was in Jay's RacketCon 2013 talk about the design of the package system, though I didn't re-watch it just now—is to think of raco pkg as less like npm and more like apt.

For example, Debian has packages for libgtk-4-1, libgtk-3-0, and libgtk2.0-0. Since they have different names, they can coexist peacefully. A package that depends on one of them—say, gnubg—will always get the one it specifies (libgtk2.0-0 (>= 2.24.0)).

Similarly, in Racket, we have both #lang scribble/lp2 and #lang scribble/lp. For that matter, we now write e.g. (require racket/unit), but we (collectively—this was before my time) used to write (require scheme/unit), and, before that, (require mzlib/unit), and programs written with any of them still work today. In fact, they can all reasonably be used together in the same program. (That's usually true, though it's technically possible to write a set of libraries where something surprising could happen, e.g. if each one relies on a lot of mutable internal state.)

Those examples are at the level of modules and collections, but the same principle applies to packages: racket/unit comes from the base package, scheme/unit from scheme-lib, and mzlib/unit from compatibility-lib. Some of the most direct examples are packages containing native libraries, e.g. draw-x86_64-macosx-3, draw-x86_64-macosx-2, and draw-x86_64-macosx.

To understand the role of the version in package metadata, lets return to Debian for a moment. Here we might see that Debian Bullseye has the packages libcairo2 at version 1.16.0-5, libcurl4 at version 7.74.0-1.3+deb11u1, and libgtk2.0-0 at version 2.24.33-2. Note how the version is distinct from the number that might be part of the package name! Thus, Debian's gnubg package can depend on libcairo2 (>= 1.2.4), libcurl4 (>= 7.16.2), and libgtk2.0-0 (>= 2.24.0).

For a Racket example, consider the package typed-racket-more, which provides the module typed/web-server/http, a wrapper around web-server/http from the package web-server-lib. When typed-racket-more added type definitions for binding:file/port, it updated its list of deps to include ["web-server-lib" #:version "1.6"], which was the version when web-server-lib added binding:file/port. The effect is that, when installing typed-racket-more, raco-pkg will check that web-server-lib has a version ≥ 1.6 and, if not, update it, cancel the transaction, or whatever your command-line arguments have told it to do. (The only version constraint is ≥.) This lets a package add features in a compatible way and other packages depend on those features being present.

On the other hand, the distinction between package versions and checksums means that you do not need to increment the version for every trivial change. Someone who explicitly installs or updates a package will always get the latest release.

4 Likes
7

My impression is that many packages don't make backward compatibility guarantees, but I don't know if they try to maintain backward compatibility nonetheless.

What worries me a bit is that most package sources on the package catalog just use the main/master branch, which could mean that a less-than-careful change and push could break clients that depend on the package.

Personally, I use semantic versioning and specify the latest stable version as a tag on the package server, for example at https://pkgd.racket-lang.org/pkgn/package/todo-txt .

Of course, this still has (at least) two downsides:

  • Users who want an earlier version need to install it explicitly from the Git repo and specify the tag as part of the URL. (You can use raco pkg install git+https://repo-url#tag for that.)
  • Sometimes I've created a new tag for a new version in the Git repo, but forgot to update the tag on the package server, so users wouldn't get the newest improvements in my package if they installed it with raco pkg install package-name.
8

Thanks for the great example. It makes sense now, but I'm still surprised. Well, racket is full of surprises :wink:

1 Like
9

It's been many years so I might be mis-remembering, but I think Jay also expressed the idea that package compatibility is as much social as technological.

1 Like
10

Indeed, the slides make for a useful window into the thinking at the time. From page 175 of that PDF:

image
image2334×1806 274 KB

1 Like