Skip to content

[kafka] Add Produce request metrics by reusing RequestsMetrics #4918

Open

Description

@wuchong

Search before asking

  • I searched in the issues and found no existing issue covering this scope.

Motivation

Kafka Produce requests need request-level metrics consistent with native Fluss RPC requests. Reuse the existing RequestsMetrics implementation so that both protocols use the same metric names, definitions, and recording methods.

Solution

  1. Extend request-name registration in RequestsMetrics and add Kafka entries to RequestsMetrics#metricsByRequest, for example kafkaProduce. A default group is sufficient for other Kafka requests; dedicated metrics for every Kafka API are not required.
  2. Allow the Fluss and Kafka Netty handlers to own separate RequestsMetrics instances. Sharing an instance is not required, but their reported metric namespaces must remain distinct.
  3. Reuse the existing request-level metric set: requestsPerSecond, errorsPerSecond, requestBytes, requestQueueTimeMs, requestProcessTimeMs, responseSendTimeMs, and totalTimeMs. Keep their definitions, sampling boundaries, and calculation/recording methods aligned with the existing Fluss request metrics, without changing the existing Fluss metrics semantics.
  4. Record the necessary timing information on KafkaRequest, including requestDequeTimeMs and requestCompletedTimeMs, and feed it into the same metric-recording code used by native Fluss requests. A shared request-tracking state machine is not needed.
  5. Keep the scope at the request level. Do not add new metrics, per-table metrics, or a separate Kafka metrics implementation.

Focused tests should verify the reused metric names and recording behavior, the kafkaProduce/default request grouping, and namespace isolation between the two protocols.

Anything else?

Parent issue: #4185.
Related implementation PR: #4896.

Willingness to contribute

  • I'm willing to submit a PR!

Activity

  1. pbanakar commented on Oct 27, 2026

    @pbanakar
    Contributor

    @wuchong @gyang94 I'm intrested to work on this, can i take this one ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions