Test environment on package server

Questions & Answers
, ,
1

For one of my packages, there are failing tests in the package server environment, although the tests succeed on my machine.

From the package server's error log it looks like (find-executable-path "raco") doesn't find raco in the PATH environment variable.

A few questions:

  • Can someone confirm that raco isn't in the PATH? (If not, it would be great to add it.)
  • Is there any documentation about the package server test environment, for example, which OS is used, which OS packages are installed and what's available in PATH?
  • What about displaying some important environment variables as part of the test log?
  • What kind of internet connection is available in the test environment? (I ask because my integration tests clone a Git repository that I run tests on.)

There seems to be some information about the environment in the log,

(/usr/bin/env DISPLAY=:1 PLT_PKG_BUILD_SERVICE=1 PLTUSERHOME=/home/root//user PLT_PKG_BUILD_SERVICE=1 CI=true PLTSTDERR=debug@pkg error PLT_INFO_ALLOW_VARS=;PLT_PKG_BUILD_SERVICE PLTCOMPILEDROOTS=/home/root//zo: /usr/bin/xvfb-run -n 1 /bin/sh -c cd "/home/root/"/racket && bin/racket -MCR "/home/root/"/zo: -l- raco pkg install -u --auto raco-exe-multitarget && bin/racket -MCR "/home/root/"/zo: -l- raco test --drdr --package raco-exe-multitarget)

This looks like the shell command that's used to run the tests, which includes setting some environment variables on the fly, in addition to the already-set ones.

The part

cd "/home/root/"/racket && bin/racket

looks like the Racket binaries are in /home/root/racket/bin. I guess I can add this to the PATH in my internal test module, but before I add this workaround, I'd like to suggest that this is added to PATH anyway. Also, many of the tests will fail without internet connection (see above).

2

This doesn't have information on PATH, but it is the documentation for the build environment: Racket: About Package Builds

1 Like
3

Thank you, this document is quite helpful. :slight_smile: For my particular use case, these sentences are especially relevant:

Packages are built on a 64-bit Linux virtual machine (VM) that is isolated from the network.

There’s no way for a package to opt out of testing, but a package author can implement a test suite that skip tests under adverse conditions. In case there’s no other way for a test suite to determine that it can’t run, the package-build service sets the PLT_PKG_BUILD_SERVICE and CI environment variables when running tests; a test suite can explicitly check for the environment variable and skip tests that can’t work.

My plan is now to check PLT_PKG_BUILD_SERVICE. If it's set, I modify the PATH variable for the tests that only need to run a local raco cross and skip the tests that require network access. Further down the road, I may try to get the "test data" from an installed package instead of cloning it from the network.

1 Like
4

To find raco I think you can do this using (find-system-path 'exec-file) and path manipulation, or you can just run the equivalent of racket -l raco -- cross.

2 Likes
5

In the end, I deactivated the test suite that uses raco cross. The "problem" is that I use raco cross --browse, and this also needs network access. :upside_down_face: So for now, I run most of the tests only on my development machine.

Eventually, I'll probably set up automated tests at Sourcehut, as I did for another project.

1 Like