[FEATURE]: Studio: rules that differ only in filter are treated as duplicates (and the second one silently stays in draft) #1576

Is there an existing issue for this?

• I have searched the existing issues

Problem statement

Found while building a regulatory data-quality demo with DQX Studio (studio-v0.1.0-rc.4; the code below is unchanged on main as of b645540).

Studio's duplicate detection ignores the filter of a check. Two rules that run the same check on the same column but on different row subsets are treated as the same rule. Example:

- criticality: error
  check:
    function: is_in_list
    arguments: {column: status, allowed: [OK, PENDING]}
  filter: error_code = 100

- criticality: error
  check:
    function: is_in_list
    arguments: {column: status, allowed: [OK, PENDING]}
  filter: error_code = 200

These are two different rules, and DQX core and the Studio Registry both treat them that way. Two Studio code paths do not:

1. Submit/publish guard. _check_no_duplicate_pending_or_approved compares only the inner check block (function + arguments), so the top-level filter is not part of the comparison:

When a monitored table is published, its checks are moved draft → pending_approval one at a time. The second check fails with ValueError, which is only logged and skipped. The check silently stays in draft, and the UI only receives a count of submitted checks:

2. Batch import with skip_duplicates. Reuse is keyed on shape_key, which hashes mode, polarity, body, slots and parameters, but not definition.filter:

The second rule is reported as "reused" and replaced by the first, so it ends up with the first rule's filter (error_code = 100 instead of error_code = 200).

In contrast, the other identity functions include the filter:

• DQX core compute_rule_fingerprint:

  dqx/src/databricks/labs/dqx/rule.py

  Lines 672 to 679 in 0699653

  	fingerprint_data = {
  	"name": check_dict.get("name"),
  	"criticality": check_dict.get("criticality", "error"),
  	"function": check_inner.get("function"),
  	"arguments": check_inner.get("arguments"),
  	"filter": check_dict.get("filter"),
  	"for_each_column": sorted(for_each_column) if for_each_column else None,
  	}

• Studio Registry fingerprint:

  dqx/app/src/databricks_labs_dqx_app/backend/registry_fingerprint.py

  Line 55 in 0699653

  "filter": rule.definition.filter,

Ignoring the rule name in the duplicate check is intentional and fine. As the _check_signature docstring explains, profiler-generated and manually created rules with the same check should match:

The problem is only the filter.

Proposed Solution

1. Include the normalized filter in the duplicate comparison of _check_no_duplicate_pending_or_approved (and in _check_signature, which find_duplicates uses and which also ignores it). Keep ignoring name, criticality, etc.
2. Include definition.filter in shape_key, so skip_duplicates only collapses rules that also share the same filter. This would match registry_fingerprint.
3. When a check of a monitored table is not submitted/published (duplicate guard or any other per-row failure in _transition_binding_checks), return the skipped checks and the reason in the response and show them in the UI, instead of only logging a warning.

Suggested tests: two checks that differ only in filter both reach pending_approval on table submit; a batch import with skip_duplicates=true of two rules that differ only in filter creates two rules; two checks that differ only in name are still flagged as duplicates.

Additional Context

filter is part of what a rule checks: it selects the rows the check runs on. A common pattern in regulatory data-quality checks is to apply the same domain check to different record types selected by a code column.