Mysterious library deletion

jbclements · November 4, 2023, 12:45am

Fun! I now have a locally-reproducible core dump that looks like this on the command line:

ENG-CLEMENTS-1:/tmp/bad clements> drracket &
[1] 55570
ENG-CLEMENTS-1:/tmp/bad clements> cairo_quartz_get_cg_context_with_clip: implementation not found
  arguments...:
   #<cpointer>
  context...:
   /Applications/Racket v8.10.900/share/pkgs/draw-lib/racket/draw/private/emoji.rkt:168:10: draw-loop
   /Applications/Racket v8.10.900/share/pkgs/draw-lib/racket/draw/private/dc.rkt:1461:4: do-text method in dc%
   /Applications/Racket v8.10.900/share/pkgs/draw-lib/racket/draw/private/dc.rkt:1319:4: draw-text method in dc%
   /Applications/Racket v8.10.900/share/pkgs/draw-lib/racket/draw/private/record-dc.rkt:451:15: draw-text method in record-dc-mixin
   .../private/arrow-higher-order.rkt:375:33: ...row-higher-order.rkt:375:33
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:5597:22: process-snips
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:5567:16: lloop
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:5346:2: refresh method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:648:2: redraw method in editor-canvas%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:1295:4: all-in-chain method in canvas-editor-admin%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:792:2: end-edit-sequence method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/framework/private/text-port.rkt:547:6: do-insertion method in ports-mixin
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/framework/private/text-port.rkt:537:11
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:436:6
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:487:32
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:639:3

ENG-CLEMENTS-1:/tmp/bad clements> 
ENG-CLEMENTS-1:/tmp/bad clements> 
ENG-CLEMENTS-1:/tmp/bad clements> internal error: attempt to deschedule the current thread in atomic mode
  context...:
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor.rkt:1394:2: wait-sequence-lock method in editor%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:769:2: begin-edit-sequence method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/framework/private/color.rkt:856:6: match-parens method in text-mixin
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:476:2: on-event method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:412:2: on-event method in editor-canvas%
   /Applications/Racket v8.10.900/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization
   /Applications/Racket v8.10.900/collects/ffi/unsafe/atomic.rkt:73:13
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/cocoa/window.rkt:899:4: dispatch-on-event method in window%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:436:6
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:487:32
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:639:3
exception raised by error display handler: internal error: tried to deschedule a descheduled thread; original exception raised: internal error: attempt to deschedule the current thread in atomic mode
  context...:
   /Applications/Racket v8.10.900/collects/racket/private/misc.rkt:146:2: channel-put
   /Applications/Racket v8.10.900/share/pkgs/drracket/drracket/private/init.rkt:96:3
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor.rkt:1394:2: wait-sequence-lock method in editor%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:769:2: begin-edit-sequence method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/framework/private/color.rkt:856:6: match-parens method in text-mixin
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/text.rkt:476:2: on-event method in text%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:412:2: on-event method in editor-canvas%
   /Applications/Racket v8.10.900/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization
   /Applications/Racket v8.10.900/collects/ffi/unsafe/atomic.rkt:73:13
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/cocoa/window.rkt:899:4: dispatch-on-event method in window%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:436:6
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:487:32
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:639:3
sequence-contract-violation: negative: method set-position cannot be called, except in states (unlocked write-lock), args 0 7
  context...:
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:739:2: call-function method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:587:2: handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor.rkt:206:2: on-local-event method in editor%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:412:2: on-event method in editor-canvas%
   /Applications/Racket v8.10.900/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization
   /Applications/Racket v8.10.900/collects/ffi/unsafe/atomic.rkt:73:13
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/cocoa/window.rkt:899:4: dispatch-on-event method in window%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:436:6
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/common/queue.rkt:487:32
   ...
exception raised by error display handler: internal error: attempt to deschedule the current thread in atomic mode; original exception raised: sequence-contract-violation: negative: method set-position cannot be called, except in states (unlocked write-lock), args 0 7
  context...:
   /Applications/Racket v8.10.900/collects/racket/private/misc.rkt:146:2: channel-put
   /Applications/Racket v8.10.900/share/pkgs/drracket/drracket/private/init.rkt:96:3
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:739:2: call-function method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:625:2: other-handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:651:6: step2
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/keymap.rkt:587:2: handle-mouse-event method in keymap%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor.rkt:206:2: on-local-event method in editor%
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wxme/editor-canvas.rkt:412:2: on-event method in editor-canvas%
   /Applications/Racket v8.10.900/collects/racket/private/more-scheme.rkt:148:2: call-with-break-parameterization
   /Applications/Racket v8.10.900/collects/ffi/unsafe/atomic.rkt:73:13
   /Applications/Racket v8.10.900/share/pkgs/gui-lib/mred/private/wx/cocoa/window.rkt:899:4: dispatch-on-event method in window%
   ...
internal error: tried to reschedule a scheduled thread
internal error: terminated in atomic mode!

[1]  + exit 1     drracket
ENG-CLEMENTS-1:/tmp/bad clements>

It seems plausible to me that this is associated with bad disk state somehow. I will try to reproduce it at home.

FYI, this is mostly just a JFYI.

jbclements · November 6, 2023, 6:56pm

Offline, @soegaard pointed me to a bug report filed by @sorawee I believe that highlighted a problem with search paths for libraries. This problem is definitely related, but not quite the same. My config.rktd file usually contains (lib-search-dirs . (#f "/opt/local/lib")) which searches first in default locations, and then in /opt/local/lib.

But here's the interesting thing!

When I update this to change the lib-search-dirs back to (#f), and try to start drracket, I get this:

ENG-CLEMENTS-1:~/brinckerhoff-source (git)-[master]- clements> ffi-lib: could not load foreign library
  path: libpng16.16.dylib
  system error: dlopen(libpng16.16.dylib, 0x0006): tried: 'libpng16.16.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file, not in dyld cache), 'libpng16.16.dylib' (no such file), '/usr/local/lib/libpng16.16.dylib' (no such file), '/usr/lib/libpng16.16.dylib' (no such file, not in dyld cache)
  context...:
   /Applications/Racket v8.10.900/collects/ffi/unsafe.rkt:131:0: get-ffi-lib
   body of "/Applications/Racket v8.10.900/share/pkgs/draw-lib/racket/draw/unsafe/cairo-lib.rkt"

[1]  + exit 1     drracket

This would make sense if I had a tool installed that required a weird library at startup, but ... I don't believe I do. It appears to me that ... this binary has somehow gotten "fixed" somehow to a version of a draw library (libpng16) that apparently only appears in my macports installation. I'm still not sure what's going on, but other versions of DrRacket on my machine still start up fine.

One other possibly relevant thing; when I first discovered the problem, I realized that I had another copy of DrRacket running at the same time. It seems somehow possible that DrRacket latched onto a different version of the library somehow... but I see now that it appears that libpng16 is the first choice in the list, so that doesn't seem likely. Hmm.

sorawee · November 6, 2023, 7:52pm

I experienced the very same issue two weeks ago. Notably, my raco was totally borked. Only basic tools that come with Minimal Racket like raco pkg worked, but other tools like raco docs didn’t. All these issues are resolved after a raco setup.

The issue occurs after I git pull that incorporates https://github.com/racket/racket/pull/4784, so that might be related.

See https://discord.com/channels/571040468092321801/893314076346826852/1162721154490957854 for full details.

EDITED: actually the FFI change is not irrelevant, since it's not included in 8.11?

jbclements · November 10, 2023, 12:59am

I "finished" tracking this down... the bundle I was using was somehow missing a whole pile of libraries from the "lib" subdirectory of the application install directory. This meant that drracket still worked fine with my enriched library search path because all of these libraries appeared in /opt/local/lib, but did not work without that supplementary help. I have absolutely no idea how those files could have been deleted or not copied. Ugh. I'm going to close this issue.

Oh wait.

It's not an issue.

Ah well.

jbclements · February 20, 2024, 2:24am

Okay, I guess I'm re-opening this non-issue... it just happened again. I believe I now know the fix, but I can't guess at how those libraries got deleted. One interesting thing; I see that the file "libs.rktd" dated to February 18, rather than to February 4th, the date on the rest of the files. The index in libs.rktd contained only the files that were present, a much shorter list that didn't include libpng16.16.dylib. In other words, it seems possible that yesterday, February 18, somehow those libraries got deleted and the index got re-built. I'd be curious to track down the code that performed that re-indexing, to see if it gives any clues about how the files got deleted. This seems very likely related to my need to customize my lib-search-dirs to include /opt/local/lib.

jbclements · February 20, 2024, 2:28am

I should add: re-downloading a fresh copy solved the problem, as before. I was not able to reproduce the error, and the problem is now gone again. Ugh. I guess on the bright side I now have a nice easy workaround.

shhyou · February 21, 2024, 6:12pm

A possible scenario is that some installed package A has duplicate dependencies on some package B (e.g. listing "B" in both deps and build-deps), and that at some point you are raco pkg update package A and racket-lib at the same time.

In that case, raco would remove package A, package B and racket-lib to prepare for re-installation, but then the duplicate dependencies make raco try to remove package B twice and fails, interrupting the re-installation process. Then, racket-lib is forever gone.

Package A and racket-lib need not be related in any way, just that they are updated at the same time, e.g. when running just $ raco pkg update<EOL> and both A and racket-lib have updates.

joeld · February 22, 2024, 8:01pm

If this is indeed possible, it seems like a bug in raco pkg update.

jbclements · November 10, 2024, 6:05pm

Followup on this: it turns out that the problem is much simpler than I'd thought, and it definitely is related to my unusual setting for lib-search-dirs; just using my setting and running raco setup is enough to trigger the library deletion. I've filed a bug report here:

Overlapping libs get deleted from racket libs directory · Issue #5100 · racket/racket · GitHub

jbclements · November 14, 2024, 6:45pm

Problem solved in raco setup: avoid discarding moved libraries that appear elsewhere · racket/racket@f1fc53a · GitHub ! Many thanks to @mflatt

Topic		Replies	Views
Racket v8.15 is now available Announcements release	3	129	November 8, 2024
Package removed after trying to update them Questions & Answers question	4	178	January 5, 2024
Release Announcement for v8.15 Internals release-management	21	303	November 5, 2024
Testing changes to a plugin Questions & Answers contributing	2	207	October 7, 2023
New installltion of Drracket does not work Questions & Answers	1	127	November 26, 2023

Mysterious library deletion

Related topics