[johntomcat7408-cmyk]

Why are the changes needed?

AMS currently constructs optimizing commits with tables whose Hadoop FileIO runs as the catalog service identity. In secured, table-owner-controlled warehouses, optimizer tasks can succeed while the AMS commit fails to write metadata or clean up files.

This is the first scoped part of #4268. It covers AMS-side optimizing commits only; optimizer task writes and other maintenance operations are unchanged.

Brief change log

• Add opt-in hdfs.impersonation.enabled configuration, disabled by default. An explicit table value overrides the direct catalog value and the catalog table. default.
• Scope owner impersonation to table loading for AMS optimizing commits so the resulting FileIO retains the owner identity for commit-time reads, writes, deletes, and executor-thread cleanup.
• Resolve the current owner from Hive Metastore for Hive-backed Iceberg, Mixed-Iceberg, and Mixed-Hive tables, and from refreshed Iceberg metadata for other Iceberg, Mixed-Iceberg, and internal tables.
• Require SIMPLE or KERBEROS authentication, HadoopFileIO, and a non-empty owner. Proxy and configuration failures are propagated without falling back to the catalog service user.
• Keep ordinary table loads unchanged and document the Hadoop proxy-user and authorization requirements.

How was this patch tested?

• Added positive and negative tests for configuration precedence, owner resolution, caller and executor identities, concrete FileIO entry points, unsupported configurations, missing owners, and no-fallback behavior.
• Screenshots are not applicable because this change has no UI surface.
• Ran focused tests and validation locally.

Commands and results:

• .\mvnw.cmd test -pl amoro-format-iceberg -am -Dtest=TestAuthenticatedHadoopFileIO,TestTableOwnerResolver -Dsurefire.failIfNoSpecifiedTests=false — 17 tests passed.
• .\mvnw.cmd test -pl amoro-format-mixed/amoro-mixed-hive -am -Dtest=TestMixedHiveTablesImpersonation -Dsurefire.failIfNoSpecifiedTests=false — 2 tests passed.
• TestInternalIcebergHandlerImpersonation — 1 test passed in the AMS-focused run.
• .\mvnw.cmd validate -pl amoro-format-iceberg,amoro-format-mixed/amoro-mixed-hive,amoro-ams -am — all 15 reactor modules passed Spotless and Checkstyle validation.
• git diff HEAD^ HEAD --check — passed.

The AMS-focused run could not initialize TestOptimizingQueue's Hive metastore on Windows because HADOOP_HOME/winutils.exe is unavailable. A real Kerberos/MiniDFS/HMS environment and the full test suite were not run locally.

Compatibility and risks

• The feature is opt-in and disabled by default.
• There are no Thrift, REST, database schema, or dependency changes.
• When enabled, an invalid owner, unsupported authentication/FileIO, or Hadoop proxy authorization failure intentionally fails the optimizing commit instead of retrying as the service user.
• The conventional hdfs owner is not special-cased; cluster proxy-user policy remains authoritative.

Provenance and contribution boundaries

The implementation is original to this PR. It builds on the existing TableMetaStore#doAsImpersonating and authenticated FileIO patterns already present in upstream commit 96a4d781aac60c4080b3f4a41322848e2308649a. No code was copied from open or unmerged pull requests or third-party sources.

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs/admin-guides/managing-catalogs.md

[sivakumar-mahalingam]

Could we avoid unconditional icebergTable.refresh() when HDFS impersonation is not enabled?

buildCommit() now loads the table inside withOptimizingCommitImpersonation(), so the TableOwnerResolver.resolve() appears to refresh the Iceberg table on each optimizing commit before checking hdfs.impersonation.enabled

It will add extra metadata I/O/failure point even when impersonation is disabled, which is the default behavior. Could we move the refresh after the hdfs.impersonation.enabled check so it will only run when impersonation is enabled?

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions.