Skip to content

test(api): avoid redundant Sandbox calls in Jinja2 tests - #46851

Merged
hyoban merged 1 commit into
mainfrom
fix/ci-flaky-20261120-1917
Nov 20, 2026
Merged

hyoban merged 1 commit into
mainfrom
fix/ci-flaky-20261120-1917

Conversation

@hyoban

@hyoban hyoban commented Nov 20, 2026

Copy link
Copy Markdown
Member

Summary

The API integration job for #46221 failed in TestJinja2CodeExecutor.test_jinja2. Sandbox returned HTTP 200 with error: timeout and signal: killed before the stdout assertion. The same test passed in the PR #46847 rerun. Two earlier queue entries hit the same Sandbox timeout in a different Jinja2 case, whose coverage moved to unit tests in #46694. Failure log · Passing integration job

Check variable substitution and the exact stdout framing through the production-generated runner in the existing unit suite. Move the runner placeholder assertions there as well. The integration file retains the workflow template case, which sends a real Sandbox request and checks the parsed result.

Verification

  • Ran the full Jinja2 transformer unit file at nice -n 10 in four parallel lanes, five runs each. The baseline had 0 failures in 20 runs of seven tests. The updated file had 0 failures in 20 runs of eight tests.
  • Ruff lint, Ruff format, and git diff --check passed. The integration baseline comes from CI, and the remaining Sandbox request is exercised by the API integration job.
  • The other six merge queue runs created in the scan window passed. On feat(api): file transfers, and files over the openapi MCP endpoint #46847, autofix-ci cancelled attempt 1. The four failed summary jobs reported that their dependencies were cancelled, and attempt 2 passed on the same SHA. Cancelled attempt

Screenshots

Backend test coverage change.

Checklist

  • This change requires a documentation update, included: Dify Document
  • I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
  • I've verified the change and added or updated tests where meaningful regression risk justifies coverage.
  • I've updated the documentation accordingly.
  • I ran make lint && make type-check (backend) and vp staged (frontend) to appease the lint gods

@github-actions github-actions Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Nov 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Pyrefly Type Coverage

Metric Base PR Delta
Type coverage 87.05% 87.06% +0.00%
Strict coverage 86.71% 86.72% +0.00%
Typed symbols 96,167 96,168 +1
Untyped symbols 14,358 14,354 -4
Modules 4641 4641 0

@codecov

codecov Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.29%. Comparing base (63b9827) to head (d258f4d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #46851   +/-   ##
=======================================
  Coverage   90.29%   90.29%           
=======================================
  Files        6463     6463           
  Lines      408190   408190           
  Branches    81856    81856           
=======================================
+ Hits       368569   368575    +6     
+ Misses      33213    33207    -6     
  Partials     6408     6408           
Flag Coverage Δ
api 90.12% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hyoban
hyoban marked this pull request as ready for review November 20, 2026 11:45
@hyoban
hyoban requested a review from QuantumGhost as a code owner November 20, 2026 11:45
@hyoban
hyoban added this pull request to the merge queue Nov 20, 2026
Merged via the queue into main with commit bdbe2d0 Nov 20, 2026
56 checks passed
@hyoban
hyoban deleted the fix/ci-flaky-20261120-1917 branch November 20, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants