Skip to content

Speed up PVC teardown by removing pvc-protection finalizer #326

Open

Description

@mclasmeier

Problem

PVC deletion during teardown is slow (~2 minutes) because kubectl delete blocks on the kubernetes.io/pvc-protection finalizer, which waits for all pods mounting the PVC to fully terminate.

Even though the CR is deleted with --wait before PVC cleanup begins, the operator's pod termination may not complete before PVC deletion starts. Since resource deletions run in parallel, the slowest PVC dominates the total teardown time.

Proposal

Remove the kubernetes.io/pvc-protection finalizer from PVCs before deleting them during teardown. This would allow PVC deletion to return immediately instead of blocking.

Why this is safe

  • By this point in teardown, the pause-reconcile annotation has been removed and the CR has been deleted with --wait, so the operator has had the chance to reconcile the deletion and start terminating pods. All pods consuming these PVCs should be terminating.
  • Removing the finalizer on a PVC whose consumer is already terminating has no practical effect — the pod keeps its volume mount until it exits regardless.
  • Even in the worst case (a pod is still briefly writing), the data is about to be destroyed anyway — this is a teardown operation on a dev environment.
  • The underlying volume mount stays live until the pod process exits; only the PVC API object is removed early.

Scope

Applies to all PVCs deleted during teardown: central-db, central-db-backup, scanner-v4-db.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions