Skip to content

fix(mir-lower): support bounded shared-pointer arrays in enum payloads - #750

Merged
nihalpasham merged 4 commits into
NVIDIA:mainfrom
uurl:fix/mir-lower-bounded-shared-pointer-enum-arrays
Aug 10, 2026
Merged

nihalpasham merged 4 commits into
NVIDIA:mainfrom
uurl:fix/mir-lower-bounded-shared-pointer-enum-arrays

Conversation

@uurl

@uurl uurl commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Closes #749

Summary

Add bounded recursive support for arrays containing shared-memory pointers in enum payloads.

Enum payload lowering already uses CUDA generic pointers as the target-stable physical representation for direct and struct/tuple-nested address-space-3 pointer leaves. This extends the same representation and reconstruction model to arrays when the recursive expansion contains at most 16 shared-pointer leaves.

Changes

  • Track the number of shared-pointer leaves during enum payload storage type rewriting.
  • Allow recursive array reconstruction when the resulting expansion is at most 16 shared-pointer leaves.
  • Reject larger arrays with an explicit diagnostic reporting the required conversion count and supported bound.
  • Extend enum pointer-storage overlap analysis to bounded pointer arrays so niche layouts such as Option<SharedPointerArrayWrapper> can be represented correctly.
  • Keep pointer vectors fail-closed because they require separate ABI and address-space-cast handling.
  • Add mir-lower coverage for bounded shared-pointer arrays in niche enum payloads.
  • Add positive CUDA runtime coverage using Option with a two-element shared-pointer array.
  • Change error_enum_shared_pointer_layout to exercise the oversized-array boundary with 17 shared-pointer leaves.
  • Update STATUS.md to document bounded array support and the remaining vector/oversized-array limitation.

The bound counts recursive pointer leaves rather than only the outer array length. This keeps generated aggregate reconstruction and verifier work explicitly bounded for nested shapes as well.

Validation

cargo test -p mir-lower
  179 unit tests passed
  100 integration tests passed
  0 failures

cargo clippy -p mir-lower --all-targets -- -D warnings
  PASS

scripts/check-error-example-status.sh
  PASS

cargo oxide run addressof_sharedarray
  PASS
  bounded shared-pointer array enum round-tripped on GPU

cargo oxide build error_enum_shared_pointer_layout
  expected failure:
  rewrite requires 17 pointer conversions, supported bound is 16

cargo clippy --workspace --all-targets -- -D warnings
  PASS

Remaining boundary

Pointer vectors remain unsupported. Their ABI/alignment behavior and address-space-cast semantics should be handled separately rather than implicitly treating them as fixed arrays.

uurl and others added 3 commits August 9, 2026 00:41
Signed-off-by: Raul Estrada <raulestradaa@gmail.com>
…oad root

The 16-leaf bound was checked per array node, so a payload holding
several individually in-bound arrays, e.g. struct([&shared; 9],
[&shared; 8]), passed the storage gate with 17 conversions in direct
layouts while the same shape died in niche layouts inside the overlap
walk with an unrelated pointer-provenance error. Count array-expanded
shared-pointer leaves through the whole rewrite and compare the total
against the bound once, at the payload root, so every layout kind
reports the same 'rewrite requires N pointer conversions, supported
bound is 16' diagnostic for the same over-bound payload.

Struct nesting outside arrays stays unbounded on purpose: its leaf
count is proportional to the source text, while [&shared; N] expands
from three tokens into N rebuild sequences. This matches what the
overlap walk in build_enum_slot_map already budgets.

Single oversized arrays keep the exact diagnostic the
error_enum_shared_pointer_layout example documents (N = 17).
…eeds it

Exhausting the pointer-storage walk's array-expansion budget used to
propagate as a generic walk failure, so build_enum_slot_map reported
'refusing to erase LLVM pointer provenance' for a shape whose only
problem is the documented 16-leaf bound (for example a generic-pointer
array of 17 elements sitting over a niche carrier). Count the
array-expanded pointer leaves up front and return a typed rejection,
PointerOverlapRejection::OverArrayLeafBound versus ProvenanceLoss, so
the slot map surfaces 'rewrite requires N pointer conversions,
supported bound is 16' for over-bound arrays while genuine pointer
puns keep the provenance diagnostic.

Pre-counting also means the walk never allocates a single per-leaf
record for a type it is going to refuse, which replaces the previous
mid-walk budget threading.
@nihalpasham nihalpasham added bug Something isn't working IR-lowering Lowering between dialects (dialect-mir to LLVM dialect) codegen Device code-generation pipeline (Rust MIR to IR to PTX) labels Aug 10, 2026
@nihalpasham

Copy link
Copy Markdown
Collaborator

Gist: this is the representation clang and rustc_codegen_nvvm use for aggregate-held pointers, generic in memory with the address space recovered later, extended to arrays under our two extra constraints: Rust niche encodings, and one storage layout that must hold under both the 64-bit legacy and p3:32 modern NVVM modes. Framework-conformant and fails closed everywhere we probed.

I pushed two commits closing the review's consistency gaps:

  • the 16-leaf budget is now enforced at the payload root, so struct-of-arrays shapes can no longer slip past the gate in direct layouts, and
  • an exhausted overlap walk now reports the bound instead of a generic provenance error.

Verified on an RTX 5090:

  • addressof_sharedarray round-trips the bounded array on GPU, the error example still fails with the exact bound
    diagnostic under both layout modes, mir-lower and full workspace suites green.

Merging. This closes #749.

Related to #306 but doesn't close it.

# Conflicts:
#	crates/rustc-codegen-cuda/STATUS.md
@nihalpasham
nihalpasham merged commit 2088ec0 into NVIDIA:main Aug 10, 2026
38 checks passed
@uurl
uurl deleted the fix/mir-lower-bounded-shared-pointer-enum-arrays branch August 10, 2026 06:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codegen Device code-generation pipeline (Rust MIR to IR to PTX) IR-lowering Lowering between dialects (dialect-mir to LLVM dialect)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support bounded arrays of shared-memory pointers in enum payloads

2 participants