@ryanc asked at RacketCon over the weekend (IIUC) about package server security checks, such as verifying that the downloaded artifact matches a known checksum (most package managers do this integrity verification today; different uses of the checksums and their computation are discussed by Andrew Nesbitt).
We got 2 different answers that I want to tease apart, I think
Pinning
@samth said we could provide a hash in info.rkt, which sounded to Ryan (and me) like version pinning.
I don't see how to provide a hash in info.rkt; a dependency can have a version string or a platform spec, but the version is a restricted format. (cf. package metadata). Was this a thinko, or did I miss something?
I suppose the equivalent today of pinning is to use custom catalogs, including racksnaps.defn.io.
Integrity-checking
Meanwhile, Package concepts says each package has a SHA-1 checksum which must match the package's archive (if distributed that way). But this checksum isn't given security properties by the docs (it's only used to detect changes for updating the package).
I mention this last bit because governments and security policies world-round are moving on from SHA-1 (I think the US targets 2030); I don't think this use of SHA-1 falls afoul of those ideas, since it's not a security use by my read?
Anyway, I don't read these docs as saying that even the SHA-1 checksum is used for integrity verification of downloads (despite needing to match the package's archive when distributed via archive). It also leaves out Git-distributed packages entirely, even though I thinkraco uses the commit ID as the checksum in those cases.
So I guess that leaves me with: do we need more integrity checks in Racket package management? Or do we already have them, and I've misread? (If so, should they move on from SHA-1?)
The goal I had in mind can be phrased as the question: When I install a package, how do I know that I am getting the package contents that the package author intended? Or from the perspective of a package author, how can I be sure that when a user installs one of my packages, they get the content that I intended?
The rough shape of the answer I had in mind is to allow package authors to tell the package server the cryptographic hash of the intended package contents, and for clients to check that the contents they receive match the expected hash. Details TBD. That would protect against malicious/compromised package hosts. It would not protect against malicious package authors, compromised author accounts on the package server, or compromise of the package server itself.
It sounds to me like you think the answer to the question from
"Integrity-checking" (yuck, why'd I put a hyphen there…) is "we don't
already have them, and we probably should" ?
My point was that you can specify a dependency on a particular version using its hash when naming a dependency using the package source form like git://github.com/foo/bar.git/#abcd. Ryan however correctly points out that we currently do not check that the actual content provided by github matches that hash.
My point was that you can specify a dependency on a particular version using its hash when naming a dependency using the package source form like git://github.com/foo/bar.git/#abcd.
Ah, right. That does give a form of pinning, which I've recommended
before for folks that want it (though it only works for Git-based
package sources).
Ryan however correctly points out that we currently do not check that the actual content provided by github matches that hash.
Indeed. The docs might be somewhat misleading there, as we make a
claim about what the checksum of such a package is.
But, as I wrote earlier, that checksum is not a security (integrity)
check; it's only used, per our docs, to decide if an update is
available.
So, I think we agree there isn't an integrity mechanism. Should there be?
I doubt it is of general interest but I thought it was worth capturing somewhere.
TL;DR: Racket does compare package contents against the SHA-1 checksum, but only when the package is served as a .zip file and has a .CHECKSUM file next to it. That check cannot serve any security purpose (because anyone who can mess with the zip can mess with the checksum); at best it can only help assure you that no bits were flipped in error during the download.
I’m much more enthusiastic about integrity checking than pinning.
I haven’t tried it to confirm, but I think you can currently specify a particular hash in the package catalog, the same way you can with a branch or a tag.
An obvious downside is that you have to manually update the catalog every time you update your package. Also, while I don’t know of any specific vulnerabilities in our catalog, it certainly does not have a security team the size of Github’s: I don’t think the package host is often the weakest link in the chain.
Checking that the content of a Git checkout matches the commit hash is more complicated than verifying the checksum of a regular file like a .zip or .tar.gz archive.
The commit hash is specifically the hash of a “commit object,” which (decoded from binary by git cat-file -p ce81585153a7ce5266790c23b2748ea654fa0d8e) contains something like:
tree e68dde175792d4f6dcb7b31aaa9649129d5de2e6
parent 246e997050ee8135dcb2f33f726f91725b5eafa2
author Matthew Flatt <mflatt@racket-lang.org> 1791467102 -0600
committer Matthew Flatt <mflatt@racket-lang.org> 1791571121 -0600
Racket HISTORY.txt notes for v9.4
The actual content on disk is the tree object identified by e68dde175792d4f6dcb7b31aaa9649129d5de2e6, which consists of references to other tree and blob objects:
Admittedly net/git-checkout must already somehow be getting the tree out of the commit, but I haven't checked how easy or hard it might be save the commit object itself for verification.
A subtler point is that the commit object also includes the hash of zero-or-more parent commits. With the huge caveat that it’s been a while since I thought about this (maybe there is some well-known solution I’m forgetting—in particular, I’d want to see what git itself does for shallow clones), I believe fully verifying the hash would require downloading and checking its full chain of ancestors.
In Nix and Guix, where we want to put the contents of a Git commit into content-addressed storage as well as verify integrity, we use our own hashes of the content on the side, which will be different from the Git hash.
Sets of Guix package definitions are themselves delivered as Git depositories, so the equivalent of sudo apt update is guix pull, a layer over git pull. These face a slightly different authentication challenge, as they are delivering the expected hashes for the packages in the set. For guix pull, we need to authenticate that the latest commit is signed by an authorized committer, a set which will change over time. Ludo’ designed a very elegant solution for that notion of authentication that is applicable to any Git repository (i.e. it is not tied to Guix): see blog post, paper, and talk.