duplicate rpath in LDSHARED sysconfig

[minrk]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

Related to duplicate LC_RPATH problems on macOS 15.4 showing up elsewhere, the current setup results in Python extensions guaranteeing the prefix rpath is set at least 3 times for Python extensions, which produces unloadable executables by default starting with 15.4 if built with a recent SDK (e.g. runtime compilation in user environments, as opposed to conda-forge infrastructure). This is coming up in e.g. dolfinx users which use runtime compilation via cffi.

The default sysconfig.get_config_var("LDSHARED") gives:

clang -bundle -undefined dynamic_lookup -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib

while LDFLAGS also sets some of the same args:

-Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib

As a result, -Wl,-rpath,$PREFIX/lib -L$PREFIX/lib shows up by default 3 times when linking Python extensions. For a simple CFFI example:

import cffi
import logging

logging.basicConfig(level=logging.DEBUG)
ffibuilder = cffi.FFI()

src = """
int test_func() {
    return 1;
}
"""
ffibuilder.set_source(
    "_test_ffi",
    src,
    include_dirs=["include-dir"],
    extra_compile_args=["-g"],
    libraries=[],
)
ffibuilder.cdef("int test_func();")
ffibuilder.compile(verbose=True)

import _test_ffi

print(_test_ffi.lib.test_func())

Run in an env with c-compiler python cffi. With the default environment variables, this produces a module which fails to load with:

Traceback (most recent call last):
  File "/Users/minrk/dev/simula/fenics/dolfinx/python/demo/test_ffi.py", line 21, in <module>
    import _test_ffi  # noqa
    ^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/minrk/dev/simula/fenics/dolfinx/python/demo/_test_ffi.cpython-312-darwin.so, 0x0002): tried: '/Users/minrk/dev/simula/fenics/dolfinx/python/demo/_test_ffi.cpython-312-darwin.so' (duplicate LC_RPATH '/Users/minrk/dev/simula/fenics/dolfinx/python/demo/.pixi/envs/default/lib'), '/System/Volumes/Preboot/Cryptexes/OS/Users/minrk/dev/simula/fenics/dolfinx/python/demo/_test_ffi.cpython-312-darwin.so' (no such file), '/Users/minrk/dev/simula/fenics/dolfinx/python/demo/_test_ffi.cpython-312-darwin.so' (duplicate LC_RPATH '/Users/minrk/dev/simula/fenics/dolfinx/python/demo/.pixi/envs/default/lib')

Running with

export LDSHARED="$CC -bundle -undefined dynamic_lookup"

results in success.

Suggestion: remove linker flags like rpath (and -L$PREFIX/lib?) from LDSHARED, relying on their presence in LDFLAGS.

Related reports:

• BUG: Duplicate RPATH's on macOS>=15.4 causing installation failures conda/conda-build#5671
• duplicate LD_RPATH for numpy on MacOS 15.4 prefix-dev/pixi#3479
• Numpy import error when using conda-forge build of numpy 2.2.3 // no issue with pip numpy-feedstock#347

Installed packages

python                     3.12.8        hc22306f_1_cpython       conda-forge
  python_abi                 3.12          5_cp312                  conda-forge

Environment info

libmamba version : 2.0.8
          mamba version : 2.0.8
           curl version : libcurl/8.13.0 OpenSSL/3.4.1 (SecureTransport) zlib/1.3.1 zstd/1.5.7 libssh2/1.11.1 nghttp2/1.64.0
     libarchive version : libarchive 3.7.7 zlib/1.3.1 liblzma/5.2.6 bz2lib/1.0.8 liblz4/1.9.3 libzstd/1.5.6
       envs directories : /Users/minrk/conda/envs
          package cache : /Users/minrk/conda/pkgs
                          /Users/minrk/.mamba/pkgs
            environment : hypre (active)
           env location : /Users/minrk/conda/envs/hypre
      user config files : /Users/minrk/.mambarc
 populated config files : /Users/minrk/.condarc
                          /Users/minrk/conda/.condarc
       virtual packages : __unix=0=0
                          __osx=15.4=0
                          __archspec=1=arm64
               channels : https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/conda-forge/osx-arm64
       base environment : /Users/minrk/conda
               platform : osx-arm64