Moving third-party packages to Codeberg

General
1

Hi. Does anyone knowledgeable about the Racket package system and Git want to comment on this plan?

I want to move 37+ Racket packages I wrote to Codeberg (which is like GitHub or GitLab, but warmer and fuzzier). The Racket package server is currently getting most of them from my Web site, not from Git. Some of these packages are in use by others. You can see most of them here: neilvandyke.org - Scheme & Racket

I'd like to have this migration work the first time, without breaking anyone's dependencies, and with a minimum of effort.

Example plan for package mcfly:

  1. I have releases: mcfly--2-0.zip mcfly--2-1.zip mcfly--2-2.zip

  2. Make repo: codeberg.org/neilvandyke/racket-mcfly

  3. Extract .zips to git tags: v2.0 v2.1 v2.2

  4. Add homepage/doc link in Codeberg.

  5. Change Racket pkg system entry to use repo.

  6. Some kind of measure to discourage the recent flood of LLM-generated pull requests that are drowning some other open source projects.

Then I would do the other few dozen packages like that.

If that exact plan should work smoothly, I can maybe do it in a couple hours. Putting on my software engineer hat, what I want to avoid is doing it but then later having to abruptly do urgent firefighting because something broke. Or having to go back and do something additional to each package.

Comments appreciated. Please feel free to reply here, or (I'll see it sooner) on https://mastodon.online/@neilvandyke/116756820844767929 or via email.

P.S., I also have a few unreleased packages, which I might polish up and release if this goes smoothly.

Thanks,
Neil V.

2 Likes
2

I don't see any particular issues with that plan.

Obviously this step is easier if the package sources are already tracked by Git (or another VCS that Git knows how to import from), but this also seems a reasonable way to map the existing archives into version control.

I don't know that the tags will matter too much to the Racket package server (though as above they'll be nice for humans) if you plan on pointing it at the "main" branch; if you plan to use another strategy[1], then you should be able to have the package server do something with the tags.

  1. Recall that Raco and Git and humans sometimes have differing notions of versions—and unless library consumers use a specific-enough URL to pin to a particular source revision, Raco will happily given them anything newer than the #:version constraint in their info.rkt (if any). ↩︎

1 Like
3

Good point about versions. One of the reasons for the Git tags (and for me increasing the priority to move to Codeberg) is that there was a recent headache involving versioning and the Racket package server.

It's also unclear to me whether the Racket package server is preserving old versions of packages.

4

Not to jinx myself but so far I have not experienced any automated (a.k.a. "agentic" ... pardon me I throw up in my mouth a little while typing that :smile:) pull requests.

I did get one human submitting a PR, the code for which was "AI" assisted. But just once. I updated the README to explain why I couldn't accept such PRs (TL;DR: no provenance).

Hopefully this factor won't turn out to be a big practical problem for you, either!

5

This plan sounds great to me!

I've thought some about these sorts of things in the context of (very, very slowly) integrating Racket with Guix and archival preservation. I won't have time to write a whole lot of detail this week, but the bottom line is this sounds like an excellent direction, and many additional tools are able to help with archiving software in Git repositories than in ad hoc zip/tar/etc. archives on a website.

While it is possible for a library consumer to use a sufficiently-specific URL in-band to achieve pinning, it is usually better to do any needed pinning out-of-band through a catalog that maps the package name to the pinned URL. (And maybe that's what you had in mind, after all: the in-band way is fairly obscure, I think.)

The package server doesn't do anything about old versions of packages.

A good mental model of the package catalog protocol is a dictionary mapping “package names” to concrete, non-name “package sources.” That one layer of indirection turns out to be very useful for a variety of purposes. The Racket release process involves creating a catalog of all the packages in the release at the released version, which lets e.g. “minimal Racket” 9.2, configured to try the 9.2 release catalog first, install the drracket package and get the same libraries as the Racket 9.2 “main distribution.” The same catalog mechanism works for off-line builds, since the catalog can map names to local filesystem locations. In fact, that also gets used in the release process: a source distribution finds its bundled packages for initial installation through an off-line catalog.

I am enthusiastic about the fact that the catalog protocol does one simple job well. Nonetheless there are additional features one reasonably might want in a package system, like preserving old versions. My view is that the features are best approached by layering on top of the catalog protocol, rather “piling feature on top of feature” in the core protocol.

This is where I think it is very helpful, @neilvandyke, that you are planning to move your packages to Git. While a human can look at the URL for mcfly--2-1.zip in the catalog and infer the existence of mcfly--2-0.zip (and then hope that either you or e.g. the Internet Archive has preserved it), it would be less obvious to a tool, let alone if some less experienced and conscientious author had just published my-great-package.zip with no contextual clues. Using a Git repository ties the latest code to older and alternate versions, with the Racket package “checksum” recording the Git commit hash. It then becomes much more viable to leverage the work other communities have been doing to archive software in Git repositories.

(Other kinds of version-control repositories would have similar technical merits: Git is just the lingua franca at the moment.)