Repository navigation
ATLAS-5419: Skip ignored entity types on partial update and delete - #768
kalayciburak wants to merge 2 commits into
Conversation
The entity type ignore pattern dropped create notifications, but partial updates and deletes of the same type were still applied.
| } | ||
|
|
||
| if (isIgnoredByConfiguredPattern(partialUpdateShell(partialUpdateRequest.getTypeName(), partialUpdateRequest.getAttribute(), partialUpdateRequest.getAttributeValue()), kafkaMsg) | ||
| || (entities != null && CollectionUtils.isNotEmpty(entities.getEntities()) && isIgnoredByConfiguredPattern(entities.getEntities().get(0), kafkaMsg))) { |
There was a problem hiding this comment.
how is this case possible ?
please elaborate the use case you thought ?
There was a problem hiding this comment.
that second check is the case where the lookup attribute is not qualifiedName.
partialUpdateShell only has typeName plus that attribute. shouldIgnore needs both typeName and qualifiedName, so the shell never matches if the unique attr is something else, even when the type is in the ignore pattern. the v1 body can still have qualifiedName, so I check that before createOrUpdate. otherwise the same type that was dropped on create still hits the store and fails as an unknown typename.
if your hooks always key partial updates by qualifiedName, the first check is enough and I can drop the second.
| final AtlasObjectId entityId = partialUpdateRequest.getEntityId(); | ||
| final AtlasEntityWithExtInfo entity = partialUpdateRequest.getEntity(); | ||
|
|
||
| if (isIgnoredByConfiguredPattern(entity != null ? entity.getEntity() : entityId, kafkaMsg)) { |
There was a problem hiding this comment.
For ENTITY_PARTIAL_UPDATE_V2, run ignore on entityId, not on entity.getEntity() alone. We hit this on ALTER TABLE … SET TBLPROPERTIES after a CREATE that was already ignored ,the patch often has only parameters and no qualifiedName, so patch-only ignore did not run, updateEntity was called, and we got INSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND. entityId carries the target type and QN — check that first; the patch is optional.
There was a problem hiding this comment.
yes, entityId is checked first now. a parameters-only patch was skipping the ignore, so updateEntity still ran and failed with INSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND.
The patch may only have parameters, so qualifiedName lives on entityId. Checking the patch first skipped the ignore and updateEntity failed.
Summary
atlas.notification.consumer.preprocess.entity.type.ignore.patterndropsENTITY_CREATE_V2for matching types, but partial updates and deletes of the same type were still applied.trino_table).GenericEntityPreprocessorand skip the store call when the type and qualified name match, for V2 partial update/delete and the V1 equivalents.entityIdfirst. A parameters-only patch (ALTER TABLE SET TBLPROPERTIES) has no qualifiedName, so patch-only ignore skipped the check andupdateEntityfailed withINSTANCE_BY_UNIQUE_ATTRIBUTE_NOT_FOUND.Test plan
JAVA_HOME=17 mvn -pl webapp -DskipEnunciate=true -Dtest=SerialEntityProcessorIgnoreTest -Dsurefire.failIfNoSpecifiedTests=false testupdateEntity; ignored delete calleddeleteByUniqueAttributes