Use Git directly for versioning?

General
, ,
1

I recently posted the discussion from a recent Qi meeting on the subject of versioning in the Racket package management ecosystem, and whether it would be useful to be able to indicate a Git version directly on the user side (either via raco pkg install or in a deps declaration in info.rkt files).

There was some private feedback expressed on a couple of different channels (from Jay and Sage) and it felt like a common place for the discussion would be valuable, so I'm starting this topic for the purpose.

The context is:

An Early Release?

... and more specifically the discussion in "Racket's Superpower is that It Doesn't Need One" and "Leveraging Git Versioning in the Package Catalog."

cc @jeapostrophe @slg and also @greghendershott (you are mentioned in the notes!)

2

I had similar issues with Polyglot 3. As user, I don't want to play supply chain manager for other programmers. My in-field liabilities for licensed materials seems to be hand-editing version metadata for the rest of my life.

I get why you go for Git names to solve this, but I might not be a Git user forever. Also, the "Git has three kinds of names" is incorrect. Git has refs. Tags and branches are refs treated in particular ways. Commits are a Git object type, which can be referenced by... refs! Also, HEAD is not a tag. HEAD can be a ref to a ref, so it doesn't work the same way.

The Git internals page shows that Git is not actually that complicated. It's content addressed files with names you make up acting as pointers. That's it. Denxi is the same. All the interfaces are just on top of files that feel like a hash table, but link in storage as a trie.

If you are allowing repository-wide access to Qi implementations, then I'd expect your "raw" release address format to be the SHA-1 or SHA-256 used for a Git tree object representing your team's shared worktree. The Git trees you happen to tag are the ones you are pointing out to people as releases, but tags/branches aren't special to users.

What if you see Git names as yet another thing to make flow? If I am a Qi user, measure how I distinguish Qi releases, then give me Qi-powered flow that prints what I put in info.rkt to target the Qi I mean. If it's a Git ref or object name that gets printed, that's up to you. Maybe an online bot evaluator can help so I don't have to install anything, and you can use the age of the old Qi release to market its stability.

Also: If I am a good user, I want you to have space and time to do work. 996 is a problem, and we need FOSS developers resting. Don't let my expectations of release schedules add too much work to you. If I want to use Qi, then I will use an expression evaluated by an OLD QI RELEASE to computer any future release name, right? That way you know I can always download an old version, then "swim" to the one I want in your version selection flow.

1 Like
3

"computer any future release"

fml

1 Like
4

Assuming Racket packages are backward-compatible:

In practice the package version number only matters when something new is added: The package author wants to bump the number -- so that using packages can say they depend on >= that number.

IMHO this works OK and doesn't need to be improved.

Although this is nicely simpler than some other package versioning approaches, it's still indirect (ergo busywork and no-meaning numbers).

As a user of some other package, I don't think in terms of depending on "version N". I think in terms of, I need/use the "foo" functionality that was added at some date I don't care about.

If that package currently on the system lacks "foo", it must update (unless I can "polyfill" some substitute if possible).

So I could imagine trying to make this nicer.

  • A weak idea: The package info.rkt declares a "features" set of symbols. This changes the busywork from bumping a number to adding a feature name. No less busywork. Just better labels than numbers.

  • A maybe too clever idea: FWIW, Racket Mode's mix of Emacs and Racket code is delivered as an Emacs package. So I need to cope with whatever's on the system. Basically this comes down to using dynamic-require, plus sometimes a runtime check. When something is unavailable, sometimes I can "polyfill" an adequate substitute, but sometimes I need to warn the user that feature X is disabled until/unless they raco pkg update _.

    A Racket package could do something like this. But it would happen on every module load time.

    What if it could be done at dependency check time. So the deps syntax grows to include values like:

    • a symbol for dynamic-require
    • arity-at-least (e.g. not just "foo" but the one where they added an optional argument)
    • maybe arbitrary lambdas for even more careful checks

Again, I don't feel like this must or even should happen. Also I'm being hand-wavy as hell here. :smile: I just feel like it's interesting moving in the direction of more direct statement of intent (and less busywork with arbitrary numbers).

2 Likes
5

Forgive me, I don't mean to discount any of these very interesting ideas, and I am interested in exploring them further! But I'd also like to keep this discussion focused on the current proposal of pass-through versioning. It would be valuable to develop it to see if it makes sense as an immediate step we can take that may bring us further along some of the more ambitious directions that you've brought up.

Technically, the proposal isn't anything new, as every aspect of it is already supported today (as Jay pointed out separately). For instance, you could just create a package qi that points to the main branch, and separate packages qi1, qi2, that point to the v1 and v2 branches (and so on, including for minor versions like 1.1). This is just what we're talking about here, except with a simpler, user-facing interface that allows us to go to the versioning provider directly (and e.g. use branch names) instead of via the package catalog intermediary (where we would expect dedicated packages that the developer may or may not have created). It eliminates the middleperson (without impacting existing uses), and makes less work for everyone.

We also get >= semantics for free with branch names, and the implicit flexibility to set upper bounds (e.g. release branch vs v1 branch).

Racket's concept of "package sources" also includes support for Git sources (including Git names) today, but the support isn't quite finished, as for instance, your docs won't build on the Racket server if you use this format.

While Git is a no-brainer to support today as it has all the features we need and is also used almost universally, it doesn't preclude supporting other versioning providers in the future.

FWIW, I think that moving from no-meaning version numbers to a versioning structure can bring us closer to your capability-based idea @greghendershott , or at least give us a more expressive language between developers and users with which we might be able to approximate it.

Separately, @slg , I'd gladly include parallel installation instructions in the Qi docs for Denxi if you could show me how. I'm keen to explore it further as I know it addresses many versioning considerations already in a parallel paradigm.

6

It looks like I got distracted by all the topics in the meetings notes, and misunderstood the scope of the discussion.

Instead you were seeking feedback on the specific proposed change:

Today, Raco does support providing Git names in dependencies, but doing so does not receive first class treatment, since the dependency must be indicated via full URL ("git://github.com/michaelballantyne/syntax-spec.git#v0.1") instead of simply a package name and a version name (like ("syntax-spec" #:version "v0.1")), and also, using such a dependency prevents the docs for the package from being built.

Is that right?

1 Like
7

Sorry, yes, that is what I had in mind initially. Although upon further reflection, I'm happy for the discussion to go in other directions as well as there are many other interesting ideas here, and I definitely don't want to get in the way of exploring them.

re: a set of feature symbols and a lambda to check an arbitrary condition on an installed dependency, that sounds a bit like a generalized form of version-case but at the dependency level.

I guess it would operate something like (unless (run-check check package) (upgrade-package package))?

I think this would play well with (is orthogonal to) Git-backed versions, where "upgrade package" would translate to simply git pull or "fast-forward," which IIUC @slg referred to as "swimming" :slight_smile:

I'm not sure if I've understood the other ideas you've put forth so I'll let others chime in, and thanks again!