Skip to content

[release-7.4] Preserve committed global configuration versions across protocol upgrades - #14363

Open
nos-a wants to merge 2 commits into
apple:release-7.4from
nos-a:backport-global-config-version-release-7.4
Open

nos-a wants to merge 2 commits into
apple:release-7.4from
nos-a:backport-global-config-version-release-7.4

Conversation

@nos-a

@nos-a nos-a commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Backport of #13841 to release-7.4. rdar://186610406

Problem

After a protocol upgrade, global-configuration history can contain entries serialized by an older protocol. The cluster controller replaced that history with a maximum-version sentinel, allowing a refresh to accept an older cached GRV-proxy snapshot and permanently miss the latest committed configuration update.

Change

The cluster controller publishes the committed version (decoded from the durable versionstamp) in the replacement history entry, and GlobalConfig::refresh rejects snapshots older than the largest seen version. Includes both unit tests from the original change.

release-7.4 differences: GlobalConfig is still an ACTOR here, so the refresh change and the stale-snapshot regression test are ported to ACTOR/wait/state syntax. The conflict was confined to GlobalConfig.actor.cpp.

Testing

  • Release build in the Okteto pod; /fdbclient/GlobalConfig/RejectsStaleRefreshSnapshot and /fdbserver/clustercontroller/globalConfigProtocolChangeUsesCommittedVersion pass via fdbserver -r unittests -f.
  • clang-format-19: clean.
  • 100k Joshua ensemble 20261026-210837-kostia-ea7b2fae5d5db9d7 on this commit (-F 100): pass=99997 fail=3 runtime=3:24:59.
    • 2 failures are exit -15 with no test file or error (agent infrastructure noise).
    • 1 failure is tests/slow/BackupOldAndNewRestore.toml seed 1033057815 (-b off), timed_out at 5401 simulated seconds; it reproduces on the same binary. The new backup stays Running and never becomes restorable after heavy recovery churn (275 backup worker recruitments, injected io_error BackupFailed events, repeated NoPrimaryPeekLocationForLR). The run contains no GlobalConfigRefreshStaleSnapshot events, and the changed cluster-controller path only runs on a protocol change, which this test does not exercise. I did not run a baseline release-7.4 build, so I have not proven it is pre-existing; at 1 in 100k it is not attributed to this change.

@nos-a
nos-a requested a review from spraza as a code owner October 26, 2026 22:05
@nos-a nos-a self-assigned this Oct 26, 2026
@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@nos-a
nos-a force-pushed the backport-global-config-version-release-7.4 branch from a1d9a2a to ec17142 Compare October 27, 2026 00:39
@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@spraza

spraza commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

@saintstack can one of you also review?

@spraza
spraza requested a review from saintstack October 27, 2026 17:47
…ades

Backport of apple#13841 to release-7.4.

After a protocol upgrade, global-configuration history can contain entries
serialized by an older protocol. The cluster controller replaced that history
with a maximum-version sentinel, allowing a refresh to accept an older cached
GRV-proxy snapshot and permanently miss the latest committed configuration
update. Stale profiling settings could then keep server roles writing profiling
data after sampling was disabled.

The cluster controller now decodes the committed version from the durable
versionstamp and publishes it in the replacement history entry, and
GlobalConfig::refresh rejects snapshots older than the largest seen version.
Includes the two unit tests of the original change.

Release-7.4 differences: GlobalConfig is still an ACTOR in this branch, so the
refresh change and the stale-snapshot regression test are ported to
ACTOR/wait/state syntax.

(cherry picked from commit 178c4e9)

Testing:
- Built fdbserver in the Okteto pod (Release); unit tests
  /fdbclient/GlobalConfig/RejectsStaleRefreshSnapshot and
  /fdbserver/clustercontroller/globalConfigProtocolChangeUsesCommittedVersion
  pass via `fdbserver -r unittests -f`.
- clang-format-19: clean.
@nos-a
nos-a force-pushed the backport-global-config-version-release-7.4 branch from ec17142 to 799e46e Compare October 27, 2026 18:33
@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

This comment has been minimized.

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang on Linux RHEL 9

  • Commit ID: 5bed1a4
  • Duration 0:39:03
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-clang-arm on Linux RHEL 9

  • Commit ID: 5bed1a4
  • Duration 0:43:21
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr on Linux RHEL 9

  • Commit ID: 5bed1a4
  • Duration 0:56:11
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos-m1 on macOS 14.x

  • Commit ID: 5bed1a4
  • Duration 1:00:23
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-cluster-tests on Linux RHEL 9

  • Commit ID: 5bed1a4
  • Duration 1:43:38
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)
  • Cluster Test Logs zip file of the test logs (available for 30 days)

@foundationdb-ci

Copy link
Copy Markdown
Contributor

Result of foundationdb-pr-macos on macOS 14.x

  • Commit ID: 5bed1a4
  • Duration 1:58:56
  • Result: ✅ SUCCEEDED
  • Error: N/A
  • Build Log terminal output (available for 30 days)
  • Build Workspace zip file of the working directory (available for 30 days)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants