Skip to content

[python] Require full table access in Ray merge_into - #10902

Merged
JingsongLi merged 1 commit into
apache:masterfrom
plusplusjiajia:fix/python-ray-merge-into-query-auth
Nov 1, 2026
Merged

JingsongLi merged 1 commit into
apache:masterfrom
plusplusjiajia:fix/python-ray-merge-into-query-auth

Conversation

@plusplusjiajia

Copy link
Copy Markdown
Member

Purpose

The Ray Data Evolution merge_into did not check query auth before merging, unlike the local and native merge and the other update paths, which require full access to the table (_authorize_update).

  • Matched updates and deletes were refused later. But a merge with only WHEN NOT MATCHED read the target keys through the filtered and masked read, so keys hidden by the row filter, or masked on the ON column, looked unmatched and were inserted again as duplicates.
  • A source given as a table name was read through its own filtered and masked read, so masked values were written into the target as real data.

merge_into now requires full access to the target, and to a source given as a table name, before anything is read or written, as the local merge does.

Tests

  • test_insert_only_merge_authorizes_target_before_source: an insert-only merge into a target that is denied, row filtered or masked on the ON key, with and without existing rows, is refused before the source, the snapshot or any assignment is read, and no files are written.
  • test_merge_authorizes_named_source_separately: a row filtered or masked source table is refused after the target is authorized, and nothing is written.
  • test_merge_with_full_source_and_target_grants: with full access to both tables the merge still updates and inserts.
  • The 10 new test cases fail on master. The related tests pass locally: 89 passed in ray_data_evolution_merge_into_test.py, and 223 passed in native_authorized_update_test.py, table_merge_into_test.py and table_query_auth_test.py. flake8 is clean.

@plusplusjiajia
plusplusjiajia marked this pull request as ready for review October 31, 2026 02:32
The Ray Data Evolution merge_into did not check query auth before
merging. Matched updates and deletes were refused later, but a merge
with only WHEN NOT MATCHED read the target keys through the filtered
and masked read, so keys hidden by the row filter or masked on the ON
column looked unmatched and were inserted again. A source given as a
table name was also read through its filtered and masked read, so the
masked values were written into the target as real data.

As in the local and native merge, merge_into now requires full access
to the target, and to a source given as a table name, before anything
is read or written.
@plusplusjiajia
plusplusjiajia force-pushed the fix/python-ray-merge-into-query-auth branch from 023488d to 326ef74 Compare October 31, 2026 03:28
@JingsongLi
JingsongLi merged commit d9f24dd into apache:master Nov 1, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants