Skip to content

ATLAS-5419: Skip ignored entity types on partial update and delete - #768

Open
kalayciburak wants to merge 2 commits into
apache:masterfrom
kalayciburak:atlas-5419-ignore-partial-delete
Open

kalayciburak wants to merge 2 commits into
apache:masterfrom
kalayciburak:atlas-5419-ignore-partial-delete

Conversation

@kalayciburak

@kalayciburak kalayciburak commented Sep 24, 2026 •

Copy link
Copy Markdown

Summary

  • atlas.notification.consumer.preprocess.entity.type.ignore.pattern drops ENTITY_CREATE_V2 for matching types, but partial updates and deletes of the same type were still applied.
  • A partial update of an ignored type then failed later as an unknown typename (for example trino_table).
  • Reuse GenericEntityPreprocessor and skip the store call when the type and qualified name match, for V2 partial update/delete and the V1 equivalents.
  • V2 partial update checks entityId first. A parameters-only patch (ALTER TABLE SET TBLPROPERTIES) has no qualifiedName, so patch-only ignore skipped the check and updateEntity failed with INSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND.

Test plan

  • JAVA_HOME=17 mvn -pl webapp -DskipEnunciate=true -Dtest=SerialEntityProcessorIgnoreTest -Dsurefire.failIfNoSpecifiedTests=false test
    • Before the fix: ignored partial update called updateEntity; ignored delete called deleteByUniqueAttributes
    • After the fix: 6 tests, 0 failures

The entity type ignore pattern dropped create notifications, but partial
updates and deletes of the same type were still applied.
}

if (isIgnoredByConfiguredPattern(partialUpdateShell(partialUpdateRequest.getTypeName(), partialUpdateRequest.getAttribute(), partialUpdateRequest.getAttributeValue()), kafkaMsg)
|| (entities != null && CollectionUtils.isNotEmpty(entities.getEntities()) && isIgnoredByConfiguredPattern(entities.getEntities().get(0), kafkaMsg))) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this case possible ?
please elaborate the use case you thought ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that second check is the case where the lookup attribute is not qualifiedName.

partialUpdateShell only has typeName plus that attribute. shouldIgnore needs both typeName and qualifiedName, so the shell never matches if the unique attr is something else, even when the type is in the ignore pattern. the v1 body can still have qualifiedName, so I check that before createOrUpdate. otherwise the same type that was dropped on create still hits the store and fails as an unknown typename.

if your hooks always key partial updates by qualifiedName, the first check is enough and I can drop the second.

final AtlasObjectId entityId = partialUpdateRequest.getEntityId();
final AtlasEntityWithExtInfo entity = partialUpdateRequest.getEntity();

if (isIgnoredByConfiguredPattern(entity != null ? entity.getEntity() : entityId, kafkaMsg)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ENTITY_PARTIAL_UPDATE_V2, run ignore on entityId, not on entity.getEntity() alone. We hit this on ALTER TABLE … SET TBLPROPERTIES after a CREATE that was already ignored ,the patch often has only parameters and no qualifiedName, so patch-only ignore did not run, updateEntity was called, and we got INSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND. entityId carries the target type and QN — check that first; the patch is optional.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, entityId is checked first now. a parameters-only patch was skipping the ignore, so updateEntity still ran and failed with INSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND.

The patch may only have parameters, so qualifiedName lives on entityId.
Checking the patch first skipped the ignore and updateEntity failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants