Skip to content

feat: support array-api - #2071

Merged
ilan-gold merged 204 commits into
scverse:mainfrom
amalia-k510:ig/array_api_continue
Feb 6, 2026
Merged

ilan-gold merged 204 commits into
scverse:mainfrom
amalia-k510:ig/array_api_continue

Conversation

@amalia-k510

@amalia-k510 amalia-k510 commented Aug 11, 2025 •

Copy link
Copy Markdown
Contributor

fixes #1731, fixes #1195, fixes #697

First step in getting anndata concat and test generation to work properly with JAX, (and Cubed potentially), without just converting everything into NumPy.

Random data creation and shape handling use xp.asarray so arrays stay in their original backend where possible. I also updated concat paths to actually check types before converting, added helpers for sparse detection and array API checks, and made sure backend arrays only get turned into NumPy when absolutely necessary. This fixes a bunch of concat-related test failures.

It’s still not perfect. Some pandas calls in concat still force conversion to NumPy, so the data gets copied instead of being used directly. Cubed support is only a placeholder right now. Type detection might still be a bit too broad, which can lead to extra conversions. Works for NumPy and JAX in tests, but I haven’t tried other backends.

@flying-sheep flying-sheep left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I just went over general code style, nothing JAX-related

Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/tests/helpers.py Resolved
Comment thread tests/test_helpers.py Resolved
Comment thread src/anndata/_core/anndata.py Outdated
Comment on lines +670 to +675
dest = self._adata_ref._X
# Handles read-only NumPy views from backend arrays like JAX by
# making a writable copy so in-place assignment on views can succeed.
if isinstance(dest, np.ndarray) and not dest.flags.writeable:
dest = np.array(dest, copy=True) # make a fresh, writable buffer
self._adata_ref._X = dest

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would actually just let the error be thrown in this case. If something isn't writeable, I don't think that's our responsibility to handle

Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/_core/merge.py Resolved
Comment thread src/anndata/_core/merge.py Resolved

@flying-sheep flying-sheep left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, I just have some clarifying questions!

Comment thread .github/workflows/test-gpu.yml Resolved
Comment thread src/anndata/_core/anndata.py Resolved
Comment thread src/anndata/_core/index.py Resolved
Comment thread src/anndata/_io/specs/registry.py Resolved
Comment thread src/anndata/compat/__init__.py Resolved
Comment thread .github/workflows/test-gpu.yml Resolved
# As of 2023 dlpack, it must be possible for a library to export to this, see: https://data-apis.org/array-api/latest/API_specification/generated/array_api.array.__dlpack__.html#array_api.array.__dlpack__
# However, https://github.com/numpy/numpy/issues/20742 means we can't roundtrip jax arrays using dlpack so better to just let numpy do its thing in asarray.
self.add_array(np.asarray(self.get_default()))
res = np.from_dlpack(self._manager[(1, 0)])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
res = np.from_dlpack(self._manager[(1, 0)])
res = np.from_dlpack(self._manager[1, 0])

@ilan-gold
ilan-gold merged commit ed34f6b into scverse:main Feb 6, 2026
24 checks passed
katosh added a commit to settylab/anndata that referenced this pull request Feb 10, 2026
Two-tier detection: tier 1 uses the canonical has_xp() protocol check
from anndata.compat (catches JAX, numpy >=2.0); tier 2 falls back to
duck-typing (shape/dtype/ndim) for arrays that don't yet implement the
full protocol (PyTorch, TensorFlow). Also uses __array_namespace__()
for backend label resolution and updates stale PR scverse#2063 → scverse#2071.
@ilan-gold ilan-gold modified the milestones: 0.13.0, 0.14.0 Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for sparse arrays (pydata) to .varp and .obsp fields General array-api support Support JAX as a backend

3 participants