Skip to content

New RAG overview keeps polling metrics after indexing tasks finish #46699

Open

Description

@hyoban

Summary

On the knowledge space overview (new RAG), metric queries keep polling after all indexing tasks have finished.

Details

web/features/new-rag/overview/state.ts decides whether to poll from overviewHasActiveTasksAtom:

  • statsQueryAtom, outcomesQueryAtom and inventoryQueryAtom call get(overviewHasActiveTasksAtom) inside the refetchInterval callback. That callback runs later, outside the atom's evaluation, so it can read a stale value instead of the current one.
  • attentionQueryAtom and activityPreviewQueryAtom compute refetchInterval from the same atom when the options are built.

With realistic response latency, polling continues at OVERVIEW_REFRESH_INTERVAL (2 s) after hasActiveTasks becomes false. That means extra requests while the page stays open.

How to reproduce

Run web/features/new-rag/overview/__tests__/page-indexing.browser.spec.tsx with service responses served over HTTP (as in #46695). The test "keeps refreshing visible metrics until the snapshot catches up" observes stats requests continuing after the tasks finish. It passed with module-level service mocks only because those responses resolved synchronously.

Expected

Every overview query stops polling once no task is active, and resumes when one starts.

Activity

  1. dosubot commented on Nov 18, 2026

    @dosubot

    One acceptance-test detail to preserve: stats, outcomes and inventory deliberately keep polling after tasks finish until their snapshots catch up. overviewRefreshInterval stops only when there are no active tasks and generated_at >= latestTaskUpdatedAt (or there is no task timestamp). Stopping immediately on hasActiveTasks === false would lose that behavior. [1]

    The existing browser test explicitly checks both phases: metrics still refresh after completion, then stats requests stop once generated_at advances. For the HTTP-backed regression, I’d extend that assertion to all five queries and add an idle-to-active transition to cover resuming polling. Attention and activity also have completion-triggered reconciliation, which cancels pre-completion requests before fetching fresh data, so those final requests should be distinguished from recurring polling. [2] [3] [4]

    To reply, just mention @dosu.


    Share context across your team and agents. Try Dosu.

    Leave Feedback Learn about dify with Dosu

  2. Johnx-w commented on Nov 19, 2026

    @Johnx-w

    I’ll take #46699. I’ll make the overview metric queries observe the current background-task state while preserving the snapshot catch-up rule, then add a browser regression with delayed HTTP responses to verify polling stops after the snapshots catch up and resumes for a new task. I checked that #46695 covers the test transport and does not change the overview polling logic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions