Chez for architectures without native backends

Questions & Answers
,
21

A Guix user, Thiago Bauermann, reported a problem bootstrapping with "rktboot" on powerpc64le-linux:

I tried building the zuo branch from your gitlab repo (commit
00975c823227 “gnu: chez-scheme-for-racket: Suport all systems.” from
August 8th) on powerpc64le-linux and had this build failure in
chez-scheme-for-racket-bootstrap-bootfiles-9.5.9.2:

starting phase `build'
Assuming current directory has Chez Scheme sources
Use /tmp/guix-build-chez-scheme-for-racket-bootstrap-bootfiles-9.5.9.2.drv-0/source/racket/src/ChezScheme/rktboot/.
Use /tmp/guix-build-chez-scheme-for-racket-bootstrap-bootfiles-9.5.9.2.drv-0/source/racket/src/ChezScheme/nanopass
Use /tmp/guix-build-chez-scheme-for-racket-bootstrap-bootfiles-9.5.9.2.drv-0/source/racket/src/ChezScheme/s
Check /tmp/guix-build-chez-scheme-for-racket-bootstrap-bootfiles-9.5.9.2.drv-0/source/racket/src/ChezScheme/boot/tpb64l
Load nanopass
Apply nanopass patch
Load cmacros parts
Load enum
Load cprep
Load expander
Install evaluator
Load cmacros using expander
Continue loading expander
Initialize system libraries
Load nanopass using expander
Load priminfo and primvars
Load expander using expander
Initialize system libraries in bootstrapped expander
Declare nanopass in bootstrapped expander
Load some io.ss declarations
Load some strip.ss declarations
Load some 7.ss declarations
Load most front.ss declarations
Define $filter-foreign-type
Load mkheader
Generate headers
Load mkgc
Generate GC
Load ftype.ss
Load fasl.ss
Load reloc.ss
Load format.ss
Load cp0.ss
Load cpvalid.ss
Load cpcheck.ss
Load cpletrec.ss
Load cpcommonize.ss
Load cpnanopass.ss
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "/gnu/store/f72x3mdyagp67ybwdy9cqqsid9v8jk9l-racket-vm-bc-8.6/opt/racket-vm/bin/racket" arguments: ("rktboot/main.rkt" "--machine" "tpb64l") exit-status: 1 term-signal: #f stop-signal: #f> 
phase `build' failed after 707.9 seconds
command "/gnu/store/f72x3mdyagp67ybwdy9cqqsid9v8jk9l-racket-vm-bc-8.6/opt/racket-vm/bin/racket" "rktboot/main.rkt" "--machine" "tpb64l" failed with status 1

I didn't have the chance yet to dig into why rktboot/main.rkt is
failing. If you have any tips on how to find more details on what is
going on (e.g., is there some verbose flag that can be passed to it?) I
can try to investigate.

I've confirmed that racket rktboot/main.rkt --machine tpb64l works for me (on x86_64-linux) both on the v8.6 tag and on current master, so I'm not sure what else to try. I did give them instructions for testing this outside of the Guix build environment, and I pointed them to this thread.

22

The problem with Racket BC on ppc64le seems to be in stack-overflow handling. I was able to get rktboot to complete by configuring the BC build with CPPFLAGS=-DSTACK_SAFETY_MARGIN=200000, but I don't know whether that's a complete or general solution.

1 Like
23

Thanks! I've adjusted my Guix package definition, and I'll ask Thiago to give it a try. This sounds like it may also explain my issue when I tried QEMU:

so I'll try that again, too.

(Once the update to Racket 8.6 is merged, Guix's build farm will also test ppc64le.)

24

I haven't heard back from anyone with actual hardware yet, but QEMU failed for me the same way as before both with CPPFLAGS=-DSTACK_SAFETY_MARGIN=200000 and, as an experiment, 10 times that. Of course, it could be that the issue under qemu-user-static is actually something different.

25

Thiago says that adding -DSTACK_SAFETY_MARGIN=200000 for BC did get rktboot working on ppc64le and apparently the Chez Scheme build succeeded. However, minimal Racket CS failed during make install by trying to do a raco setup using the BC Racket that had been supplied to --enable-racket=, but incorrectly passing it --cross-compiler. Here's part of the log:

starting phase `install'
/gnu/store/i9h1vc67h9148xvn8djk8j3smlkhaf09-zuo-1.0-racket8.6/bin/zuo . install DESTDIR=""
cp cs/c/racketcs /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/bin/racket
: /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/bin/racket
cp ../src/start/starter-sh /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/starter-sh
cp cs/c/starter /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/starter
: /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/starter
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/start/collects-path.rkt ../src/ /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/starter ../collects ../etc
compiler/cm:   start-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/start/collects-path.rkt
compiler/cm:   finish-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/start/collects-path.rkt
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/cs/c/gen-system.rkt /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/system.rktd tpb64l tpb64l machine ../src/cs/c/ ""
compiler/cm:   start-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/cs/c/gen-system.rkt
compiler/cm:   finish-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/cs/c/gen-system.rkt
cp ../src/cs/c/api.h /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/include/racketcs.h
cp ../src/cs/c/boot.h /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/include/racketcsboot.h
cp cs/c/ChezScheme/tpb64l/boot/tpb64l/scheme.h /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/include/chezscheme.h
cd cs/c/repack && ar x ../rktio/librktio.a
cd cs/c/repack && ar x ../ChezScheme/tpb64l/boot/tpb64l/libkernel.a
ar rc cs/c/libracketcs.a cs/c/repack/expeditor.o cs/c/repack/schsig.o cs/c/repack/compress-io.o cs/c/repack/rktio_flock.o cs/c/repack/gc-par.o cs/c/repack/rktio_shellex.o cs/c/repack/flushcache.o cs/c/repack/new-io.o cs/c/repack/rktio_signal.o cs/c/repack/print.o cs/c/repack/rktio_fd.o cs/c/repack/symbol.o cs/c/repack/rktio_poll_set.o cs/c/repack/schlib.o cs/c/repack/rktio_fs.o cs/c/repack/scheme.o cs/c/repack/rktio_wide.o cs/c/repack/rktio_main.o cs/c/repack/rktio_dll.o cs/c/repack/prim.o cs/c/repack/gc-oce.o cs/c/repack/fasl.o cs/c/repack/ffi.o cs/c/repack/segment.o cs/c/repack/rktio_convert.o cs/c/repack/rktio_error.o cs/c/repack/thread.o cs/c/repack/number.o cs/c/repack/rktio_process.o cs/c/repack/gc-ocd.o cs/c/repack/gc-011.o cs/c/repack/rktio_sha1.o cs/c/repack/rktio_network.o cs/c/repack/rktio_file.o cs/c/repack/random.o cs/c/repack/prim5.o cs/c/repack/io.o cs/c/repack/rktio_syslog.o cs/c/repack/rktio_hash.o cs/c/repack/vfasl.o cs/c/repack/rktio_console.o cs/c/repack/intern.o cs/c/repack/rktio_pipe.o cs/c/repack/rktio_sleep.o cs/c/repack/gcwrapper.o cs/c/repack/stats.o cs/c/repack/rktio_fs_change.o cs/c/repack/rktio_envvars.o cs/c/repack/rktio_ltps.o cs/c/repack/statics.o cs/c/repack/rktio_sha2.o cs/c/repack/foreign.o cs/c/repack/pb.o cs/c/repack/alloc.o cs/c/repack/rktio_time.o cs/c/repack/rktio_cpu.o cs/c/boot.o
cp cs/c/libracketcs.a /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/libracketcs.a
: /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/libracketcs.a
cp cs/c/gracketcs /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/gracket
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/start/collects-path.rkt ../src/ /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/bin/racket ../collects ../etc
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/start/collects-path.rkt ../src/ /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/gracket ../collects ../etc
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/cs/c/add-terminator.rkt cs/c/petite-v.boot /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/petite.boot
compiler/cm:   start-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/cs/c/add-terminator.rkt
compiler/cm:   finish-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/src/cs/c/add-terminator.rkt
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/cs/c/add-terminator.rkt cs/c/scheme-v.boot /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/scheme.boot
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../src/cs/c/add-terminator.rkt cs/c/racket-v.boot /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib/racket.boot
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -O info'@'compiler/cm -l- setup --chain ../src/setup-go.rkt cs/c/compiled ignored cs/c/ignored.d ../collects/setup/unixstyle-install.rkt make-install-copytree ../ /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/bin /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/collects /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/share/pkgs /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/doc /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/include /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/lib /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/share /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/etc /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/share/applications /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/man yes
compiler/cm:   start-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/collects/setup/unixstyle-install.rkt
compiler/cm:   finish-compile: /tmp/guix-build-racket-vm-cs-8.6.drv-0/source/racket/collects/setup/unixstyle-install.rkt
Copying collects -> /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/collects
Copying share/pkgs -> /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/share/pkgs
  missing source path "share/pkgs", skipping...
Copying share -> /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/share
  missing source path "share", skipping...
Copying doc -> /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/doc
  missing source path "doc", skipping...
Copying etc -> /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/etc
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket -MCR cs/c/compiled: --cross-compiler tpb64l cs/c -X /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/collects -G /gnu/store/cb84hlf8gb0nspc8f6n7qhihsflj7k6x-racket-vm-cs-8.6/opt/racket-vm/etc -N raco -l- setup --no-user
/gnu/store/w675i4y454bgcyr69fp672h51p24l1fg-racket-vm-bc-8.6/opt/racket-vm/bin/racket: bad switch: --cross-compiler
Use the --help or -h flag for help.
failed
 in build-one
 in loop
 in module->hash
make: *** [Makefile:16: install] Error 1
error: in phase 'install': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("install" "ZUO=/gnu/store/i9h1vc67h9148xvn8djk8j3smlkhaf09-zuo-1.0-racket8.6/bin/zuo") exit-status: 2 term-signal: #f stop-signal: #f> 
phase `install' failed after 26.8 seconds
command "make" "install" "ZUO=/gnu/store/i9h1vc67h9148xvn8djk8j3smlkhaf09-zuo-1.0-racket8.6/bin/zuo" failed with status 2
26

It looks like the problem might be the assumption in https://github.com/racket/racket/blob/ab6dc20f78ff4e75655f245e28de44fa62a4b82d/racket/src/cs/c/configure.ac#L447-L456:

# For a pb build where Racket is supplied, force cross-build
# mode on the assumption that the host is not a pb build
# (because it should be created with default configure options)
if test "${enable_pb}" = "yes" ; then
  if test "${enable_racket}" != "" ; then
    if test "${enable_target}" = "" ; then
      enable_target="${MACH}"
    fi
  fi
fi
27

Maybe that check should move before the preceding block that sets MACH, an then add check here that "${MACH}" = ""?

28

I thought this seemed right, but, when I started looking at changing the code, I was less sure. If the check I pointed to:

moved before the preceding block beginning with if test "${enable_mach}" != "", I think the result would be the same, because the preceding case "$MACH_HOST_CPU" statement sets MACH="" for powerpc64*, and the value from --enable-mach=tpb64l won't have been consulted yet.

Tangentially, I think:

  powerpc64*)
    MACH=""
    pb_machine_name="${thread_prefix}pb64b"
    # MACH="${thread_prefix}ppc64${MACH_OS}"
    ;;

is wrong. POWER8 and POWER9 chips can be run in either big-endian or little-endian mode: config.sub normalizes them as e.g. powerpc64-unknown-linux-gnu and powerpc64le-unknown-linux-gnu. IIUC, this is set in the firmware or something: a given operating system runs in big-endian or little-endian mode, not both, so Debian has ports for both ppc64 (big-endian) and ppc64el (little-endian). (Guix only supports little-endian mode.)

More generally, I think it would be better to use the standard Autoconf macros AC_C_BIGENDIAN and AC_CHECK_SIZEOF rather than having to explicitly recognize architecture and os names.

As I looked around more, it seems there's a related assumption in this block slightly later in the file:

if test "${enable_racket}" != "" ; then
  if test "${enable_racket}" != "auto" ; then
    SETUP_BOOT_MODE=--chain
  fi
  # In non-cross mode, we interpret `--enable-racket` to supply a
  # Racket used only for generating Chez Scheme boot files
  if test "${CROSS_MODE}" = "cross" ; then
    RACKET="${enable_racket}"
  elif test "${enable_racket}" != "auto" ; then
    BOOTFILE_RACKET="${enable_racket}"
  fi
fi

Until now, Guix has been configureing our racket-vm-cs package with both --enable-racket and --enable-scheme. The intention was that the provided scheme would be used to generate the bootfiles and racket (CS when cross-compiling, 3M otherwise) would be used for other purposes: in particular, because Guix policy takes an especially firm stance on bootstrapping, we want to be able to schemify the regexp, io, and thread layers. However, it seems we were not actually doing that yet, so, as I workaround, I tried not providing --enable-racket for non-cross builds: it didn't break existing builds, and I've asked Thiago to try it on hardware.

For a different approach, could we just delete this block?

If I'm following correctly, it seems like it's meant to handle a "cross" build for the current system, but for a pb or pbarch machine type: is there another way to handle that case?

29

For now, I pushed a change that I think fixes the immediate problem. I'd rather not just delete the troublesome block, because its there to help with building pb much more quickly on a machine where a native build is supported.

I agree that using AC_C_BIGENDIAN and AC_CHECK_SIZEOF would be better.

30

Thiago tried building with a cherry-picked 70e484e885637c495be5481983dae2207fdd67bb and reports that:

For some reason I didn't understand, the problem I first reported
about rktboot/main.rkt failing during the build of
chez-scheme-for-racket-bootstrap-bootfiles came back. I had to increase
STACK_SAFETY_MARGIN again. I simply increased it 10x to make sure it
would work. If you want I can experiment with different values to find
an appropriate one.

So, as you suggested in:

it seems like there's still some underlying problem. I don't think I know enough about BC internals to have any useful ideas.

For now, unless you have a better idea, I'll propose using the 10x larger STACK_SAFETY_MARGIN for Guix and hope that works well enough to bootstrap CS. Whatever the problem with BC on ppc64le is, I expect it would also be triggered by other Racket programs, so Guix using CS by default on all platforms still seems like an improvement.