Pyffi with numpy (<1.26) low-level lib and Python 3.1x

I'm testing. I think your code is good and my 3.12 version of yesterday was good too but it does not works today with your new version and 3.12 (i did not test with 3.10 or 3.14) because of a problem on my host related to Python, i just noticed a lot of strange path that i can check bad in my my previous post too, also differents and i do not understand what is wrong on my system. Apple ships the host with python 3.9 , i installed with brew (perheaps that is part of the problem) python 3.12, then python 3.10 because it is so easy with brew. All the libs are installed with pip3.x the classic way.

The error now , still the same as yesterday before it worked ,but then also fails in my account, not even test on web engine:

Python path configuration:
  PYTHONHOME = '/Users/mattei/my-env'
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/Users/mattei/my-env/lib/python3.10/python3.12'
  sys._base_executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.base_prefix = '/Users/mattei/my-env'
  sys.base_exec_prefix = '/Users/mattei/my-env'
  sys.platlibdir = '/Users/mattei/my-env/lib/python3.10'
  sys.executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.prefix = '/Users/mattei/my-env'
  sys.exec_prefix = '/Users/mattei/my-env'
  sys.path = [
    '/Users/mattei/my-env/lib/python3.10/python312.zip',
    '/Users/mattei/my-env/lib/python3.10/python3.12',
    '/Users/mattei/my-env/lib/python3.10/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00000001fcfee080 (most recent call first):
  <no Python frame>

look at those bad paths :

stdlib dir = '/Users/mattei/my-env/lib/python3.10/python3.12'
sys.path = [
    '/Users/mattei/my-env/lib/python3.10/python312.zip',
    '/Users/mattei/my-env/lib/python3.10/python3.12',
    '/Users/mattei/my-env/lib/python3.10/python3.12/lib-dynload',
  ]

a real mess.

Then i checked the previous post of mine:

/Library/WebServer/myvenv3.10/lib/python3.10/lib-dynload

did not exists , but exist

/Library/WebServer/myvenv3.10/lib/python3.10/site-packages

the same for this one yesterday or friday:

/Users/mattei/my-env/lib/python3.12/python3.12/lib-dynload

/Users/mattei/my-env/lib/python3.12/python3.12

does not exists python3.12 has been append twice.

If i solve that, for sure it will works fine.

The problem is that i do not know which code is generating those path, is it part of Python ,Racket, FFI in C ?

i have different environment, i checked the shell variable ,there is nothing that can not even confuse between 3.10 and 3.12 or melt them in the same path.

But i'm just at the beginning of my search.

update: just to see my-env is a python 3.12 vitualenv not 3.10 :

((my-env) ) mattei@mac ~ % ls -la /Users/mattei/my-env/lib/
total 0
drwxr-xr-x  3 mattei  staff   96  7 nov.  11:58 .
drwxr-xr-x  7 mattei  staff  224  7 nov.  12:04 ..
drwxr-xr-x  3 mattei  staff   96  7 nov.  11:58 python3.12
((my-env) ) mattei@mac ~ % ls -la /Users/mattei/my-env/bin 
total 120
drwxr-xr-x  18 mattei  staff   576 14 nov.  12:06 .
drwxr-xr-x   7 mattei  staff   224  7 nov.  12:04 ..
-rw-r--r--   1 mattei  staff  2164  7 nov.  11:58 activate
-rw-r--r--   1 mattei  staff   920  7 nov.  11:58 activate.csh
-rw-r--r--   1 mattei  staff  2195  7 nov.  11:58 activate.fish
-rw-r--r--   1 mattei  staff  9033  7 nov.  11:58 Activate.ps1
-rwxr-xr-x   1 mattei  staff   208  7 nov.  12:13 f2py
-rwxr-xr-x   1 mattei  staff   209  7 nov.  12:04 fonttools
-rwxr-xr-x   1 mattei  staff   213  7 nov.  11:58 pip
-rwxr-xr-x   1 mattei  staff   213  7 nov.  11:58 pip3
-rwxr-xr-x   1 mattei  staff   213  7 nov.  11:58 pip3.12
-rwxr-xr-x   1 mattei  staff   206  7 nov.  12:04 pyftmerge
-rwxr-xr-x   1 mattei  staff   207  7 nov.  12:04 pyftsubset
lrwxr-xr-x   1 mattei  staff    10  7 nov.  11:58 python -> python3.12
lrwxr-xr-x   1 mattei  staff    10  7 nov.  11:58 python3 -> python3.12
lrwxr-xr-x   1 mattei  staff    44  7 nov.  11:58 python3.12 -> /opt/homebrew/opt/python@3.12/bin/python3.12
-rwxr-xr-x   1 mattei  staff   204  7 nov.  12:04 ttx
-rwxr-xr-x   1 mattei  staff   230 14 nov.  12:06 virtualenv

FWIW I downloaded Python from the official website:

The way pyffi configure works is:

  1. It uses the python3 in the current path.
  2. It runs python3 -m sysconfig to get the location of folders.
  3. It stores the configuration in the Racket preferences.

So after installing a new version of Python, check that your terminal has added it to the current path. Use which python3 to double check. This is especially relevant if you have Python from several sources installed.

When you run a Racket program, it loads the preferences and uses the stored paths.
I can't remember whether you need to restart DrRacket or not when preferences change.

Here is what I see:

soegaard@Mac scribblings % raco pyffi show
Current configuration for 'pyffi'.

    libdir = "/Library/Frameworks/Python.framework/Versions/3.14/lib"
    data   = "/Library/Frameworks/Python.framework/Versions/3.14"

Meaning:

    libdir:  location of the shared library 'libpython'
    data:    location of bin/ lib/ share/ etc.

i removed python3.10 ,even python 3.12 ,reinstalled python3.12 with brew ,
searched all the reference to python3.10 , there was a few (brew did not de installed all) but i still have the error.

It seems it is ffi-lib related when python is launched .

When i launch python from a shell in virtual env all is ok.

I do not know how it can find some python3.10 info as there is no more sucha file on system. i rebooted. i suppose in a file it should be somewhere python3.10 string but even mdfind can not find it now.

i will continue searching.

i tested to reinstall python3.10 aside python3.12 with brew install python@3.10 and all works again.

i tried again to install it on another Mac that used python3.11 without using virtual environment and got the same type of error , it crashed racket with this same message:

mattei@macbookpro pyffi % drracket 
2025-12-06 23:38:36.329 DrRacket[87218:40202684] The class 'NSSavePanel' overrides the method identifier.  This method is implemented by class 'NSWindow'
2025-12-06 23:39:05.969 DrRacket[87218:40202684] The class 'NSOpenPanel' overrides the method identifier.  This method is implemented by class 'NSWindow'
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = ''
  sys._base_executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.base_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.base_exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.platlibdir = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.path = [
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000020cc52140 (most recent call first):
  <no Python frame>

i had configured pyffi this way:

mattei@macbookpro pyffi % raco pyffi configure
Configuration tool for `pyffi`.
-------------------------------
This tool attempts to find the shared library `libpython3` 
with the help of the `python3` executable.

The executable
    /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
will be used to find the location of the shared library.

The preference for LIBDIR is now set to:
    /Library/Frameworks/Python.framework/Versions/3.11/lib

The preference for DATA is now set to:
    /Library/Frameworks/Python.framework/Versions/3.11

and had installed pyffi via the GUI package manager with the option 'do what i mean' 3 times , on pyffi-lib , pyffi-doc and pyffi directory from github.

On this host i had installed python with brew too.

I have no more idea for now. After all the web apps works with racket and python3.12 by default for the math interpolation, and i will get the meta data from racket via pyffi with python 3.10 lib. That not a big constraint.

update : if i set the python path i have no more the 2 initial warning about missing prefix and exec_prefix but it crash too.

I can confirm something is broken.
I must have accidently changed something after running tests.
I'll investigate.

strange. because i checked your code , at least libpython.rkt (by memory) and it seems ok.Seems to adapt to any python3.1x. So why would it works only with python 3.10?

I just reinstalled Python 3.11 on the Mac book (now python3.11.9) pro where it was not a brew install but an official.dmg from official python.org, the old web app with no pyffi is working.

when i ran into drracket to check the pyffi i got the same error:

mattei@macbookpro ~ % drracket 
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = ''
  sys._base_executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.base_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.base_exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.platlibdir = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.path = [
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000020834e140 (most recent call first):
  <no Python frame>

my opinion is that there is something incoherent in Python start (probably not related to your code , perheaps racket ffi-lib ? or Python on Mac OS ?)

because the variable :
sys.platlibdir = '/Library/Frameworks/Python.framework/Versions/3.11'

seems to be wrong :

for this reason

sys.path = [
  ]

remains empty

if i force in a shell

export PYTHONPLATLIBDIR=/Library/Frameworks/Python.framework/Versions/3.11/lib

it is ok for Python in shell :

but in Racket it fail back to the wrong previous value

how i find that is because if i set something wrong in this variable i can make crash python in shell,example:

mattei@macbookpro ~ % export PYTHONPLATLIBDIR=/Library/Frameworks/Python.framework/Versions/3.11/lib-false
mattei@macbookpro ~ % python3.11
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = '/Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python3.11'
  sys._base_executable = '/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11'
  sys.base_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.base_exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.platlibdir = '/Library/Frameworks/Python.framework/Versions/3.11/lib-false'
  sys.executable = '/Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11'
  sys.prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.path = [
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python311.zip',
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python3.11',
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python3.11/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000020834e140 (most recent call first):
  <no Python frame>

strangely sys.path is set but with my forced wrong value:

sys.path = [
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python311.zip',
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python3.11',
    '/Library/Frameworks/Python.framework/Versions/3.11/lib-false/python3.11/lib-dynload',
  ]

when called from Drracket it is not only false but empty, i think for this reason the encoding can not be found, is it a problem with Python on Mac OS or with Racket ffi-lib ,i do not know.Or perhaps from my system , on the Mac book air i used brew , on the older mac book pro it seems to be from python.org, i should have do the same on mac book air, but as both are professional host i want to keep at least one running fine, if we find it is related to brew i then can safely upgrade the other laptop with official Python later.(have a meeting at work (university), tomorrow about this project and i do not want to be with a broken config :sweat_smile: )

Sorry. False alarm.
Turns out I forgot (initialize) and (post-initialize) before calling pylist.
Sigh.

Welcome to Racket v8.12 [cs].
> (require pyffi)
> (initialize)
> (post-initialize)
> (pylist 1 2 3 4)
(obj "list" : [1, 2, 3, 4])

That looks suspiciously as the values set by set-environment-variables.

Pyffi always uses the settings that raco pyffi configure captures.
So rerun raco pyffi configure each time you change the configuration.

Note: When DrRacket (all gui programs actually) load a dynamic library, it is cached and used until DrRacket is closed again. If DrRacket has cached, say, libpython from the Brew installation, and you change the configuration to use Python from their website, you need to restart DrRacket. [This is just how C dynamic libraries work - so it is no fault of DrRacket].

For that reason, I like to test everything related to dynamic libraries in the terminal.

if it is ok on your host ,then there should only be a configuration problem of my python system on my hosts.I had installed a lot different versions due to scientific program bundled without info on the requirements, i installed a lot of python version. I also cleaned a lot. When i will install all again on Linux i will see what happens.

I reinstalled Python from the official site. Reboot many times.
Start Drracket from command line, and as it crashes ,i restarted it a lot of times.

I also rerun:

mattei@macbookpro pyffi % raco pyffi configure  
Configuration tool for `pyffi`.
-------------------------------
This tool attempts to find the shared library `libpython3` 
with the help of the `python3` executable.

The executable
    /Library/Frameworks/Python.framework/Versions/3.11/bin/python3
will be used to find the location of the shared library.

The previous value of LIBDIR was:
    /Library/Frameworks/Python.framework/Versions/3.11/lib
The preference for LIBDIR is now set to:
    /Library/Frameworks/Python.framework/Versions/3.11/lib

The previous value of DATA was:
    /Library/Frameworks/Python.framework/Versions/3.11
The preference for DATA is now set to:
    /Library/Frameworks/Python.framework/Versions/3.11

I modifed a bit your code to display some critical paths, It crash on (intialize) so if i comment it i got the variables displayed:

#lang racket
(require pyffi); pyffi/numpy)
(set-environment-variables)
;(initialize)

Welcome to DrRacket, version 8.14 [cs].
Language: racket, with debugging; memory limit: 8192 MB.
(libpython-path /Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib)
(libpython-folder /Library/Frameworks/Python.framework/Versions/3.11/lib)
(libpython-path /Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib)
(libpython-folder /Library/Frameworks/Python.framework/Versions/3.11/lib)
(libpython-path /Library/Frameworks/Python.framework/Versions/3.11/lib/libpython3.11.dylib)
(libpython-folder /Library/Frameworks/Python.framework/Versions/3.11/lib)
> 

The path lib seems to be good? the problem seems after in the python launch process. It is strange because the other python programs ,the math interpolation code that use a lot of libraries, numpy,scipy, matplotlib,works fine.

If i use python3 from shell all is ok with the libs:

mattei@macbookpro pyffi % python3
Python 3.11.9 (v3.11.9:de54cf5be3, Apr  2 2024, 07:12:50) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.__version__
'1.26.4'
>>> import sys
>>> sys.version
'3.11.9 (v3.11.9:de54cf5be3, Apr  2 2024, 07:12:50) [Clang 13.0.0 (clang-1300.0.29.30)]'

even with variable set:

mattei@macbookpro pyffi % drracket              
	Python path configuration:
  PYTHONHOME = '/Library/Frameworks/Python.framework/Versions/3.11'
  PYTHONPATH = (not set)
  program name = 'python3.10'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = ''
  sys._base_executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.base_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.base_exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.platlibdir = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.executable = '/Applications/Racket/bin/../DrRacket.app/Contents/MacOS/DrRacket'
  sys.prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.exec_prefix = '/Library/Frameworks/Python.framework/Versions/3.11'
  sys.path = [
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x000000020834e140 (most recent call first):
  <no Python frame>

it is more and more strange,i tried on my personal linux box, so not Mac OS this time and i got the same error:

mattei@acer:~/Téléchargements/Python-3.14.2$ /usr/racket-8.18.0.13/bin/drracket 
Python path configuration:
  PYTHONHOME = '/usr/local'
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/usr/local/lib/python3.10/python3.12'
  sys._base_executable = '/usr/local/bin/python3'
  sys.base_prefix = '/usr/local'
  sys.base_exec_prefix = '/usr/local'
  sys.platlibdir = '/usr/local/lib/python3.10'
  sys.executable = '/usr/local/bin/python3'
  sys.prefix = '/usr/local'
  sys.exec_prefix = '/usr/local'
  sys.path = [
    '/usr/local/lib/python3.10/python312.zip',
    '/usr/local/lib/python3.10/python3.12',
    '/usr/local/lib/python3.10/python3.12/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x0000772981e34b80 (most recent call first):
  <no Python frame>

again the confusion about path /usr/local/lib/python3.10/python3.12
it seems i installed python3.12 on this system , i can find a snap from gnome referencing python3.10 somewhere: /snap/gnome-42-2204/226/usr/lib/python3.10/

my opinion is that the initialisation schema of python is pretty dumb if it so silly to create a sting in path containing python3.10/python3.12

also the snap system already caused problem accessing Racket doc, i would install a linux not based on snap next time...

then i installed python3.14 ,recompiled from source , but it seems there is no more libpython3.14.so (only libpython3.14.a static library,not dynamic)needed to be loaded by ffi-lib , i see an error message from racket unsafe/ffi stating it searched a .so file ,by appending .so , when i force path in pyffi.

it is strange there is no more libpython3.14.so on this version.The compilation and install was good.I can start python3.14 ,install numpy with pip3.14, import it , all is good.
This is weird.

For now work on my Linux box by recompiling Python 3.10 with the option:

./configure --enable-shared

to have the shared library libpython3.1x.so

ffi/unsafe of Racket will load shared library , not static, which is logic.

i have set LD_LIBRARY_PATH to the place where are the shared libraries on my system :

export LD_LIBRARY_PATH=.:/usr/local/lib

i have tested many Python version , so i have many Python libraries:

mattei@acer:~$ ls -la /usr/local/lib
total 153388
drwxr-xr-x  9 root root     4096 déc.  10 10:18 .
drwxr-xr-x 13 root root     4096 nov.   5  2024 ..
drwxr-xr-x  3 root root     4096 nov.   5  2024 bigloo
drwxr-xr-x  3 root root     4096 janv. 11  2025 clojure
-rwxr-xr-x  1 root root 34095326 déc.  10 09:27 libpython3.10.a
lrwxrwxrwx  1 root root       20 déc.  10 10:18 libpython3.10.so -> libpython3.10.so.1.0
-rwxr-xr-x  1 root root 16389808 déc.  10 10:18 libpython3.10.so.1.0
-rwxr-xr-x  1 root root 72591400 déc.  10 08:02 libpython3.14.a
lrwxrwxrwx  1 root root       20 déc.  10 09:56 libpython3.14.so -> libpython3.14.so.1.0
-rwxr-xr-x  1 root root 33923080 déc.  10 09:56 libpython3.14.so.1.0
-rwxr-xr-x  1 root root    15096 déc.  10 10:18 libpython3.so
drwxr-xr-x  6 root root     4096 oct.  17  2024 NsCDE
drwxr-xr-x  2 root root     4096 déc.  10 10:19 pkgconfig
drwxr-xr-x 36 root root     4096 déc.  10 10:18 python3.10
drwxr-xr-x  3 root root     4096 avril 24  2024 python3.12
drwxr-xr-x 43 root root     4096 déc.  10 09:56 python3.14

note that it seems that the pyffi module will only load the lower version number even if raco configure is made with the good python version.

So i modified the libpython.rkt of pyffi to force it on Python 3.14 and test it with numpy > 2,

(set! libpython-path "/usr/local/lib/libpython3.14")

it works, but there is another error later:

searching about this error give few advices...

i will try testing other Python versions

update:

uncommenting and adding :

(define (initialize-main-and-builtins)
  ; (displayln (list 'initialize-main-and-builtins))
  ; AddModule doesn't load or impor the module, it just returns it.
  (set! main          (PyImport_AddModule "__main__"))     ; top-level environment
  (set! builtins      (PyImport_AddModule "builtins"))
  ; (PyImport_AddModule "operator")  ; this leads to an error on Ubuntu
  (PyImport_AddModule "traceback")
  (PyImport_AddModule "inspect")


in python-environment.rkt make the program go far but it later fails on:

Welcome to DrRacket, version 8.18.0.13-2025-08-26-0bb5c26e85 [cs].
Language: racket, with debugging; memory limit: 8192 MB.
'(libpython-path "/usr/local/lib/libpython3.14")
'(libpython-path "/usr/local/lib/libpython3.14")
'(libpython-path "/usr/local/lib/libpython3.14")
initialize passed
import-numpy passed
. . get: not bound: yy format_exception in traceback.format_exception

in finish-initialization

i see what is wrong , at least for numpy <2 and python <3.14, it now works for python 3.10,3.11 i will test with 3.12,3.13 and on Mac .

I find only python 3.10 to 3.12 with numpy 1.26 can be used.AFAIK

I forked a version that just remove the above deprecated API.

Under Linux with multiple Python installs the lib dir is not enough to find the used lib because there can be multiple libpython files in the same dir , so i added an option in configure to specify the libfilename:

raco pyffi configure python3.12 /usr/local/lib/libpython3.12

it is saved in the racket preference file.

note LD_LIBRARY_PATH could be required, example:

export LD_LIBRARY_PATH=.:/usr/local/lib

It has worked with multiple Python installs on my Linux system, for now on Mac i can only test with Python 3.11 but should works with other versions as described above

the source code is available here.

i just realize today ,as i got an error going back from Mac OS to Linux using source code shared over cloud,the good library path for my Linux Ubuntu system is indeed:

raco pyffi configure python3.12 /usr/lib/x86_64-linux-gnu/libpython3.12.so

i do not know why i said /usr/local/lib the libpython are indeed in /usr/lib/x86_64-linux-gnu , i probably too much swapping from a system to another :person_facepalming:
or i was on a Mac using homebrew, but the best way to use Python with pyffi and in general is to install it from python.org . Note on Mac OS M1 or M4 i find that some python installer installed some x64 python library for intel :open_mouth: on my M1 or M4 cpus, but the good lib is the arm64 version for Apple M1,M2,M3,M4 made by ARM that is provided by Xcode tools from Apple.

note: @soegaard if you need to put back my pyffi modif in your source code, tell me, i rarely or never used the github procedure and i have no real knowledge about it.

The thread is long with several false ends.

Is it correct, that the problem on Linux was:

  • several version of libpython.so are in the same folder

i think it was part of the problem.
At some point my idea was that it could be helpful to specify the full path to the library.
At least on my linux system where i had many versions, if i locate libpython :

/usr/lib/x86_64-linux-gnu/libpython3.12.a
/usr/lib/x86_64-linux-gnu/libpython3.12.so
/usr/lib/x86_64-linux-gnu/libpython3.12.so.1
/usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0
/usr/lib/x86_64-linux-gnu/libpeas-1.0/loaders/libpython3loader.so
/usr/local/lib/libpython3.10.a
/usr/local/lib/libpython3.10.so
/usr/local/lib/libpython3.10.so.1.0
/usr/local/lib/libpython3.11.so
/usr/local/lib/libpython3.11.so.1.0
/usr/local/lib/libpython3.13.so
/usr/local/lib/libpython3.13.so.1.0
/usr/local/lib/libpython3.14.a
/usr/local/lib/libpython3.14.so
/usr/local/lib/libpython3.14.so.1.0

another part of problem was with the deprecated API.In fact it seems the deprecated functions do not need even to be replaced. By chance commenting them just was ok. I have no knowledge of FFI of Python,not even C API for python, and have no time to learn that.

On Ubuntu I see:

  1. Which Linux distribution are you using?

  2. Which paths do python3 -m sysconfig display?

Under the headning "Variables" I see:

  1. What do you have under LDLIBRARY and LIBDIR ?
mattei@acer:~$ uname -a
Linux acer 6.14.0-33-generic #33~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 19 17:02:30 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

mattei@acer:~$ python3 -m sysconfig
python3: error while loading shared libraries: libpython3.13.so.1.0: cannot open shared object file: No such file or directory

mattei@acer:~$ python3.13 -m sysconfig
python3.13: error while loading shared libraries: libpython3.13.so.1.0: cannot open shared object file: No such file or directory




python3.12 -m sysconfig
....
LDLIBRARY = "libpython3.12.so"
....
LIBDIR = "/usr/lib/x86_64-linux-gnu"
.....
ls -la /usr/local/bin
...
lrwxrwxrwx  1 root root   10 déc.  10 18:12 python3 -> python3.13
-rwxr-xr-x  1 root root  18K déc.  10 10:18 python3.10
-rwxr-xr-x  1 root root 3,0K déc.  10 10:19 python3.10-config
-rwxr-xr-x  1 root root  18K déc.  10 12:14 python3.11
-rwxr-xr-x  1 root root 3,0K déc.  10 12:15 python3.11-config
-rwxr-xr-x  1 root root  18K déc.  10 18:12 python3.13
-rwxr-xr-x  1 root root 3,1K déc.  10 18:12 python3.13-config
-rwxr-xr-x  1 root root  18K déc.  10 09:56 python3.14
-rwxr-xr-x  1 root root 3,1K déc.  10 09:56 python3.14-config
...

i reinstalled python3.12, so the python3 link point to it as it is the last compatible with pyffi. This on my personal laptop , my pro desktop is with an old python version, i should upgrade the linux this year, i will take care to put only one python version, seems the last installed get the python3 link to it.

update: i haven't test with pyffi but this sould works, because as the pyffi dir is shared on cloud file systems with the Mac ...... each time i configure on box it breaks the other as Racket see binary for the other box....

Are you able to run Python programs without Racket, or even start a Python REPL? If your python3.13 executable cannot load libpython3.13.so.1.0, it seems like your Python installation is broken, with nothing to do with Racket.

This seems like a recipe for trouble.