Skip to content

feat(openfeature): improve feature flag debug logging - #10889

Open
ethan-romero-dg wants to merge 1 commit into
masterfrom
ethan.romero/FFL-3409/improve-debug-logging
Open

ethan-romero-dg wants to merge 1 commit into
masterfrom
ethan.romero/FFL-3409/improve-debug-logging

Conversation

@ethan-romero-dg

Copy link
Copy Markdown

What does this PR do?

Fills gaps in the Feature Flags DD_TRACE_DEBUG output, so a customer can follow a flag result from setup to delivery.

  • Setup: logs the provider's configuration source, SDK env, and initialization timeout, plus the agentless endpoint kind, poll interval, request timeout, and whether the API key header is sent. Warns when remote_config is selected but Remote Configuration is disabled. Logs when feature flagging is disabled.
  • Configuration delivery: logs agentless retry delays, terminal poll failures and whether the prior configuration is kept, and recovery. The applied-configuration line names the SDK env, the requested dd_env, and the delivered configuration environment separately, and includes createdAt with the ETag or Remote Config ID. Remote Config failures to apply are logged before being rethrown.
  • Readiness: logs the initialization wait and elapsed time, whether configuration was already present, and recovery after a timeout or a removed configuration.
  • Evaluation: logs the flag key, type, default, merged evaluation context, and the details seen by the finally hook, without util.inspect depth or length limits and without invoking custom inspectors. Logs the provider resolution separately when a later hook replaced it, and labels configuration identity as "observed at logging time". A one-time warning says these logs contain raw, possibly sensitive data.
  • Exposures: describes accepted batches as receiver acceptance (Agent/local EVP handoff vs. HTTP receiver), states that failed batches are dropped, logs pending events queued once delivery is enabled or dropped at shutdown, and redacts exposure request errors.
  • Docs: adds a short "Feature Flag Debug Logging" section to docs/API.md.

Lazy log callbacks fall back to a fixed message when formatting fails, so diagnostics can't prevent configuration from being applied.

Motivation

FFL-3409. Brings Node.js in line with the cross-SDK debug-logging scenario matrix, where DD_TRACE_DEBUG should expose raw evaluation inputs and outputs and explain configuration delivery, readiness, and exposure outcomes.

Additional Notes

Testing

  • npm run test:openfeature: 766 passing. log.spec.js and proxy.spec.js: 156 passing. integration-tests/openfeature/openfeature-worker-logging.spec.js: 3 passing. Lint is clean.
  • Dogfooding (ffe-dogfooding, local tarball, staging CDN and Agent Remote Config), evaluating all flag types, the experiment flag, and a missing flag:
    • Debug on, agentless and Remote Config: every log above appears, with no fallback or error lines.
    • Debug off: no Feature Flags lines, same results.
    • Debug on with DD_TRACE_LOG_LEVEL=warn: no evaluation lines and no raw-data warning.

Known gaps, not addressed here

  • A custom logger that throws or rejects still crashes the process. Diagnostics channels rethrow subscriber errors on the next tick, so this affects every tracer log, not just Feature Flags, and can only be fixed in log/writer.js. It should be a separate change for the log owners.
  • The writer logs use route.url.origin, which prints null for Unix-socket Agent URLs and omits the endpoint path.
  • Shutdown logs come only from the beforeExit handler, which doesn't run on SIGTERM or process.exit(). OpenFeature.close() doesn't flush exposures.
  • No usable/rejected flag counts: validity is checked at evaluation. No exposure suppression reasons (dedupe, doLog=false): the upstream provider returns silently.
  • The recovery log doesn't fire for a remove-and-restore when configuration was present before initialize().

🤖 Generated with Claude Code

Make DD_TRACE_DEBUG output explain a flag result end to end:

- Setup: log the provider's configuration source, SDK env and init
  timeout; the agentless endpoint kind, poll interval, request timeout and
  whether the API key header is sent; warn when remote_config is selected
  while Remote Configuration is disabled.
- Delivery: log agentless retry delays, terminal poll failures, whether
  the prior configuration is kept, recovery, and the delivered
  configuration identity (environment, createdAt, ETag or RC id,
  requested dd_env).
- Readiness: log the initialization wait and elapsed time, and recovery
  after a timeout or a removed configuration.
- Evaluation: log the flag key, type, default, merged context and final
  details without inspect truncation, the provider resolution when a later
  hook replaced it, and the configuration observed at logging time. Warn
  once that these logs contain raw evaluation data.
- Exposures: report accepted batches as receiver acceptance, failed
  batches as dropped, and pending events queued on enable or dropped at
  shutdown; redact request errors.

Lazy log callbacks fall back to a fixed message when formatting fails, so
diagnostics never prevent configuration from being applied.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@ethan-romero-dg

Copy link
Copy Markdown
Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T16:02:07.620229Z 8804bb4 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: 8804bb40b5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@pr-commenter

pr-commenter Bot commented Oct 22, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-10-22 15:53:12

Comparing candidate commit 8804bb4 in PR branch ethan.romero/FFL-3409/improve-debug-logging with baseline commit 7c1c0a6 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2387 metrics, 16 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-20

  • unstable max_rss_usage [-7.005MB; +27.152MB] or [-2.393%; +9.275%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-20

  • unstable max_rss_usage [-13.243MB; +26.060MB] or [-3.693%; +7.268%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-24

  • unstable max_rss_usage [-22.169MB; +17.673MB] or [-6.164%; +4.914%]

scenario:encoders-0.4-immediate-flush-24

  • unstable max_rss_usage [-7.473MB; +11.944MB] or [-8.936%; +14.282%]

scenario:plugin-aws-sdk-eventbridge-inject-detail-24

  • unstable cpu_usage_percentage [-8.545%; +5.484%]
  • unstable execution_time [-356.520ms; +561.959ms] or [-7.222%; +11.384%]
  • unstable throughput [-119371.819op/s; +76098.034op/s] or [-9.765%; +6.225%]

scenario:plugin-aws-sdk-extract-response-body-26

  • unstable execution_time [-226.156ms; +238.488ms] or [-5.277%; +5.565%]

scenario:plugin-claude-agent-sdk-delayed-noisy-stream-scan-26

  • unstable execution_time [-118.080ms; +215.281ms] or [-3.550%; +6.472%]

scenario:plugin-elasticsearch-get-20

  • unstable cpu_usage_percentage [-6.254%; +5.944%]
  • unstable execution_time [-208.810ms; +217.709ms] or [-8.525%; +8.888%]
  • unstable throughput [-145161.620op/s; +140518.323op/s] or [-6.490%; +6.282%]

scenario:plugin-elasticsearch-search-20

  • unstable cpu_user_time [-233.842ms; +161.801ms] or [-6.332%; +4.381%]
  • unstable execution_time [-234.802ms; +161.595ms] or [-6.353%; +4.372%]

scenario:plugin-graphql-long-with-depth-on-max-26

  • unstable max_rss_usage [-37.833MB; +23.881MB] or [-14.695%; +9.276%]

scenario:plugin-kafkajs-large-26

  • unstable execution_time [-133.791ms; +233.371ms] or [-4.034%; +7.037%]

@ethan-romero-dg
ethan-romero-dg marked this pull request as ready for review October 22, 2026 15:58
@ethan-romero-dg
ethan-romero-dg requested review from a team as code owners October 22, 2026 15:59
@ethan-romero-dg
ethan-romero-dg requested review from khanayan123 and tlhunter and removed request for a team October 22, 2026 15:59

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

Full OpenFeature execution remains unverified because dependencies and generated vendor bundles were unavailable. The completed static review raised no reportable concern, and human PR discussion supplied no additional technical context.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 8804bb4

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant