LoadCppInterOp ignores CPPINTEROP_LIBRARY_PATH — dispatch backend can't be redirected to an existing CppInterOp (e.g. an installed ROOT) #99

Summary

LoadCppInterOp() always loads the bundled libclangCppInterOp chosen by cppinterop_paths() and never consults the CPPINTEROP_LIBRARY_PATH environment variable. As a result the dispatch backend cannot be redirected at runtime to an already-present CppInterOp — for example the copy compiled into an installed ROOT's libCling. This forces loading a second ~128 MB CppInterOp+LLVM into the process even when a compatible one is already resident.

Where

src/interop/interop_wrapper.cxx:

extern "C" int LoadCppInterOp() {
  std::call_once(Once, [] {
    const InterOpPaths Paths = cppinterop_paths();   // bundled path only
    if (!loadDispatchAPI(Paths))                      // Cpp::LoadDispatchAPI(Paths.Library)
      return;
    ...

cppinterop_paths() anchors CPPINTEROP_LIBRARY next to the extension (or the install prefix) and returns that as Paths.Library; CPPINTEROP_LIBRARY_PATH is never read on this path. (_cpython_cppjit._preload_backend_library() calls LoadCppInterOp() directly, so there is no Python-side override either.)

Reproduce

With an external libclangCppInterOp (here, an installed ROOT's libCling, same CppInterOp commit) and the env var pointed at it:

export CPPINTEROP_LIBRARY_PATH=$ROOTSYS/lib/libCling.so
python -c "import cppjit; print(open('/proc/self/maps').read())" | grep libclangCppInterOp

The bundled .../cppjit/interop/lib/libclangCppInterOp.so is mapped; the external library is not — even when it is preloaded (RTLD_GLOBAL) before import cppjit. So the override has no effect.

(The compiled objects contain CPPINTEROP_LIBRARY_PATH and a "CPPINTEROP_LIBRARY_PATH overrides the captured path" string, suggesting the override was intended but is not wired into LoadCppInterOp's path.)

Why it matters

It blocks reusing an already-loaded CppInterOp/interpreter. Concretely, letting cppjit share an installed ROOT's libCling (one CppInterOp, one LLVM, one interpreter) requires only redirecting the dispatch source — the dispatch ABI is compatible (both pin the same CppInterOp commit; all dispatch entries resolve across the two builds), so no rebuild is needed, if the library path can be overridden.

Proposed fix

In cppinterop_paths() (or loadDispatchAPI), honor an explicit CPPINTEROP_LIBRARY_PATH when it is set and exists, using it as Paths.Library in place of the captured/bundled path. A few lines, backward compatible (unset env → current behavior).

Environment

• cppjit 0.1.0a1 (PyPI wheel, bundled CppInterOp built against LLVM 21).
• Reproduced on Linux (manylinux wheel) against an external CppInterOp from
  ROOT (LLVM 22, same pinned CppInterOp commit).