Hi, I'm looking for a practical way to "vendor" dependencies in a project.
We are building a Docker container with Racket and some packages (eg, quickcheck). During the docker build process, we would need to download the packages with raco, but raco does not seem to support HTTP proxies. So the second option would be to "vendor" the dependencies by downloading them (and their dependencies, recursively) and commit that in our repo. Is there a simple way to do that? Manually running git over all dependencies is not very convenient specially if packages depend on other packages.
I have a similar problem with my application, and I added all the packages that I needed as git submodules. The build process sets up a package catalog on the directory with the packages, then installs the top-level packages. raco will pick up other packages from the local repository as well, so there is no need to install dependencies "in order".
The only difficult part is identifying the initial packages you need and adding them as submodules. This only needs to be done once and, after this is done, updating packages is a simple submodule update command. To detect new packages added as dependencies, the build process removes all other package catalogs from racket, leaving only the directory catalog from the application repository. If one of the packages tries to "sneak in" a new dependency when updating it, it is caught during the build.
I had some issues with raco pkg install over proxies a while back, but they eventually went away. I'd be curious what your problems were and if more experienced Racketeers could weigh in on them.
Isn't this what raco setup --check-pkg-deps is for? That and --fix-pkg-deps are how I keep my info.rkt up to date, which would help identify exactly the packages you need. I imagine it would be fairly easy to write a script to identify the Git repo for each package in that list by querying the package server (though of course Git is not the only protocol the package server speaks, so what do you do if a dependency is not in Git?).
@alexh i am curious if a combination of raco cross and using your censoring approach would allow fixing the “installed packages are global” issue you’ve called out.
I was thinking if one used raco cross pointed to a specific workspace for all operations?
raco cross did not exist when I last looked at this problem and wrote the blog post, 3 years ago -- that explains why it is not mentioned in the blog post
Currently I simply install all packages in user scope, this works for me, since I only use a small number of packages, and switching package versions is rare and manageable. However, isolated environments are important for those who work with multiple Racket projects and sometimes conflicting package requirements.
Thanks a lot for this blog post! For now we have just a few dependencies, so we can use a submodule or curl the tar.gz from github and install them, but once we have more dependencies, I think we'll use the catalog approach.
It was very surprising to me that raco doesn't take https_proxy into account. I can easily download the .tar.gz packages with curl from github. Perhaps raco could simply use curl to fetch packages?! The error messages would also be easier to follow
