Repository navigation
[cuda.core] Add support for Multicast Objects #2057
Description
Feature Request
Add Python bindings and a cuda.core abstraction for CUDA Multicast Objects
(the cuMulticastCreate / cuMulticastAddDevice / cuMulticastBindMem /
cuMulticastBindAddr / cuMulticastUnbind family of driver APIs).
Motivation
Multicast Objects enable a single virtual address to multicast memory accesses
across multiple devices, which is foundational for efficient multi-GPU
collectives and SHARP-style reductions on NVLink-connected systems (Hopper+
and beyond). They are already exposed via the low-level cuda.bindings driver
API, but there is no high-level cuda.core object wrapping them.
This is one of the last major CUDA driver features without a cuda.core
counterpart. Internal teams building multi-GPU libraries (RAPIDS, Triton,
cuBLASMp, etc.) currently have to drop to raw driver calls.
Proposed Scope
- A
MulticastObject(or similarly named) class with:- Constructor / factory taking granularity + participating-device list
add_device(device)/bind_memory(buffer)/bind_address(...)/
unbind(...)methods- Context-manager lifecycle (auto-release on exit)
- Integration with existing
Device,Buffer, andVirtualMemoryResource
- Query helpers for multicast granularity (
cuMulticastGetGranularity) - Example(s) under
cuda_core/examples/demonstrating a simple 2-GPU
multicast allreduce-style pattern - API reference entry in
cuda_core/docs/source/api.rst
Related
- Tracking gaps identified in cuda.core feature audit (Nov 2025) — other
untracked gaps includehost_launchandLibrary(cuLibrary) APIs; those
can be filed separately. - Relevant driver APIs:
cuMulticastCreate,cuMulticastAddDevice,
cuMulticastBindMem,cuMulticastBindAddr,cuMulticastUnbind,
cuMulticastGetGranularity.
Activity
leofang commentedon May 11, 2026
I have mixed feelings in this. On the one hand it's good to have a centralized, pythonic multicast object, on the other hand we simply don't have any hw that could test this feature (which requires at least Hopper+ GPUs, NVLink, and NVSwitch all together in one system, such as NVL72).
Also, from the CUDA perspective this is considered as a central part of VMM, for example
cuMulticastBindMemtakes a VMM handleCUmemGenericAllocationHandleso as to map VA across devices. Furthermore, the CUDA doc suggestswhich is inline with our user/developer guidance: Always start with a high-level library first. So I would wait until someone comes and asks us for coverage.
cc @benhg for thoughts from nvSHMEM & VMM perspectives
benhg commentedon May 13, 2026
I also have mixed feeling son this. In NVSHMEM, we bind multicast address ranges to VMM memprops (moral equivalent of VirtualMemoryResource-allocated objects) and actually can register VirtualMemoryResource buffers against the heap and use for Multimem access.
We are doing all this in the CPP library today, with Python bindings, rather than re-exposing in pure Python so far.
I do know of many users who build this functionality themselves (e.g. TorchSymmetric's CUDA backend)
rparolin commentedon Jun 2, 2026
Did we want to close this as "won't implement"?
cc: @leofang @benhg
kkraus14 commentedon Jun 9, 2026
I'm reopening this issue, because I strongly suspect no one will come and ask us for this as their agents will likely just identify that it's missing and work around the issue in various suboptimal ways.