Skip to content

[fix] Refuse an agent config the runtime cannot run, on every save - #7411

Merged
mmabrouk merged 2 commits into
feat/cross-agent-config-editingfrom
fix/agent-revision-runnable-gate
Oct 11, 2026
Merged

mmabrouk merged 2 commits into
feat/cross-agent-config-editingfrom
fix/agent-revision-runnable-gate

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Stacked on #7409. It replaces a gate that exists only on that branch, so it targets feat/cross-agent-config-editing and moves to release/v0.122.3 when #7409 merges.

Context

An agent saved with a skill that has no body saved fine. Every later run of that agent then failed with HTTP 500 Invalid skill configuration ... 'body' Field required. #7409 added a check for this, but only for the agent tools (commit_revision, edit_agent_config, create_agent). A save from the playground, the SDK or the API still stored the broken config.

Changes

The check now runs where every agent revision is built. _reject_unrunnable_agent runs the runtime's own AgentTemplate.from_params on the result. Two places call it: _build_revision_commit, which builds every stored revision, and simple workflow and application create and edit, before their first write. The second call keeps a refused create from leaving an empty artifact behind. The check runs only when the uri key is agent, which is the same rule that sets the is_agent flag.

The refusal is a 422 on every route, and the same envelope comes back in the agent tools' result:

{
  "code": "invalid_agent_configuration",
  "message": "The agent configuration cannot run: parameters.agent.skills[0].body is required.",
  "retryable": false,
  "next_step": "Correct the fields listed in `details.issues` ...",
  "details": {"issues": ["parameters.agent.skills[0].body is required"]}
}

Before, the agent tools answered final_validation_failed with skills[0].body is required. Paths now start at the root, so they match the targets the operations use.

What this replaces:

  • The agent-context validate hook in _apply_delta (_agent_template_issues). The build covers it.
  • _reject_unreadable_harness_kind, which ran only on the checked commit. The runtime parse refuses the same values. The error still maps to invalid_harness_kind with the allowed list, and handle_workflow_exceptions now maps it to 422 too, because simple create and edit can raise it now.
  • _reject_unreadable_agent_instructions stays. The runtime reads a bare-string instructions as "no prompt" and does not refuse it, so this rule is the API's own.

An agent that is already broken. The check reads the result, not the change. A stored agent with a skill that has no body refuses every change until the same change fixes that skill. The issue names the field, and next_step says to fix it in the same change. I chose this over letting unrelated edits through. Those edits would keep saving an agent that cannot run, and the field is easy to fix once it is named.

Parts that hold an embed are skipped, as before, because embeds resolve before a run. A list entry that holds an embed keeps its index in the reported path.

SDK. The pydantic location now lives on the SDK errors. SkillValidationError, MCPConfigurationError and ToolConfigurationError share a new base, ConfigEntryError, in agents/config_errors.py. It carries issues (the location inside the entry, with a tool's union tag removed) and describe(field). The issues survive every re-raise, including the collected tool diagnostics. Before, the API walked __cause__, and the tools path lost the cause, so a bad tool was reported only by its message.

Tests

  • New api/.../test_commit_agent_template_validation.py covers one test per writer: a full-data commit (playground and SDK), a delta commit, the plain commit, simple workflow create, idempotent create, simple workflow edit, simple application create and edit, and 422 on the routes. It also covers the broken-agent case in both directions: an unrelated change is refused and names the stored field, and a change that fixes the field is accepted.
  • test_agent_config_handlers.py still covers the three agent tools. It now expects the new code and path.
  • New SDK test_config_entry_errors.py.
  • API unit suite: 7587 passed. SDK unit/agents: 2005 passed.
  • The acceptance test test_agent_config_tools.py is updated, but I did not run it. I did not deploy this branch to a live stack.

What to QA

  • In the playground, add a skill to an agent with a name and a description but no body, then save. The save is refused and the message names parameters.agent.skills[0].body.
  • Commit the same config through the SDK or POST /workflows/revisions/commit. You get a 422 with invalid_agent_configuration.
  • Use a stored agent that already has such a skill. Change only the instructions: the save is refused and names the skill. Add the body in the same save: the save succeeds.
  • Regression: save an agent whose skills are embedded references (@ag.embed). The save succeeds.

Review follow-up (Codex gpt-6-astra and CodeRabbit)

  • The rule finds the template the way the runtime does: parameters.agent when it is an object, else the bare parameters or the prompt fallback. Before, an agent saved in the bare shape (parameters.skills[...]) skipped the check.
  • A single entry given as an object (skills: {...}, which the runtime reads as a list of one) is named, for example parameters.agent.skills.body is required. Before, it raised a KeyError.
  • A simple workflow or application edit with no data commits a copy of the stored data. When that data cannot run, the edit is now refused before the artifact write, not after it. Renaming an unrunnable agent is refused until its configuration is corrected, as the rule above says.
  • Simple evaluator create and edit check before their first write. The evaluator write routes and the template loader answer 422 with the envelope. Before, they answered 500.
  • Only the parse's own refusals become invalid_agent_configuration. A parser defect stays a 500.
  • The instructions rule skips embedded instructions, because the embed resolves before a run. This was a false refusal already on the base branch.

Run the runtime's own AgentTemplate.from_params on every is_agent revision in
_build_revision_commit and before the first write of simple create and edit,
instead of only on agent-tool commits. The refusal is invalid_agent_configuration
with the field paths in details.issues (parameters.agent.skills[0].body is
required). A bad harness kind keeps its invalid_harness_kind envelope and now
reaches every route as a 422.

The SDK's skill, MCP and tool entry errors now carry their pydantic issues, so the
location survives every re-raise and the API no longer walks __cause__.
@coderabbitai

coderabbitai Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

📝 Summary

Summary by CodeRabbit

  • Bug Fixes
    • Agent configuration is now validated across workflow and application create, edit, and revision changes. Invalid configurations are rejected before they are saved.
    • Validation responses now identify the specific fields and entries that need correction, including skill, tool, and MCP configuration issues.
    • Workflow APIs return consistent validation errors for invalid agent configurations and harnesses.
📝 Summary
📝 Summary

Walkthrough

Agent configuration validation now runs across workflow revision commits and simple workflow and application create and edit paths. SDK parsers carry structured field issues for skill, tool, and MCP configurations. Configuration and harness errors map to structured tool refusals and HTTP 422 responses.

Changes

Agent configuration validation

Layer / File(s) Summary
Structured SDK configuration issues
sdks/python/agenta/sdk/agents/config_errors.py, sdks/python/agenta/sdk/agents/{skills,tools,mcp}/*, sdks/python/agenta/sdk/agents/dtos.py, sdks/python/oss/tests/pytest/unit/agents/test_config_entry_errors.py
SDK parsers now attach structured field issues to skill, tool, and MCP configuration errors. Tool diagnostics preserve those issues, and AgentTemplate forwards tool diagnostic issues into its error.
Validation before workflow and application writes
api/oss/src/core/workflows/{service.py,types.py}, api/oss/src/core/applications/service.py, api/oss/tests/pytest/unit/workflows/*, openspec/changes/cross-agent-config-editing/*
_reject_unrunnable_agent validates agent instructions and applicable runtime configuration. Revision commits and simple workflow and application create and edit operations run validation before writes. Delta commits validate the resulting data. Tests cover embedded entries, invalid stored fields, and validation before writes.
HTTP and tool refusal handling
api/oss/src/apis/fastapi/workflows/{exceptions.py,router.py}, api/oss/src/core/tools/platform_handlers.py, api/oss/tests/pytest/{acceptance/tools/test_agent_config_tools.py,unit/tools/test_agent_config_handlers.py,unit/workflows/test_commit_agent_template_validation.py}
Workflow exception handling maps configuration, harness, and instruction errors to HTTP 422. Agent tool handlers convert configuration and harness errors to structured refusals. Tests assert the invalid_agent_configuration code and field paths in details.issues.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SimpleWorkflowCreate
  participant RejectUnrunnableAgent
  participant AgentTemplate
  SimpleWorkflowCreate->>RejectUnrunnableAgent: Validate requested data before writing
  RejectUnrunnableAgent->>AgentTemplate: Parse applicable agent configuration
  AgentTemplate-->>RejectUnrunnableAgent: Return parser validation error
  RejectUnrunnableAgent-->>SimpleWorkflowCreate: Raise InvalidAgentConfigurationError
Loading




Merge Risk: 🟡 Moderate · up to 282bb

A rejected edit can still change workflow or application metadata. Validate the effective revision before either edit writes metadata, and preserve server-error handling for unexpected parser failures before merging.

Pre-merge checks | Passed 4 | Failed 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 20 files. (3 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly states the main change: refusing agent configurations that the runtime cannot execute on every save.
Description check Passed The description directly explains the validation changes, affected write paths, error responses, SDK updates, tests, and QA scope.

Full details: Docstring Coverage

Explanation

Docstring coverage is 20.00% which is insufficient. The required threshold is 60.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 85 functions across 20 files. (3 skipped: 3 unsupported.)


  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Commit to this branch
  • Create a new PR





🛠️ Fix failing CI checks 💡
  • Commit to this branch
  • Create a new PR



🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR





  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Item Value
Status stopped
Commit 20b0cd79f62648c6068f1828269638662f2777fd
Manual expiry (UTC) Not granted
CI deadline (UTC) Not granted
Workflow https://github.com/Agenta-AI/agenta/actions/runs/38156239799

PR closed or converted to draft. Environment absence verified; retained storage requires separate provider accounting.

Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.

@mmabrouk mmabrouk added lgtm This PR has been approved by a maintainer merge queue In the release merge queue; an agent is landing it labels Oct 11, 2026
@mmabrouk

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a341238d-2c3c-44cd-95bb-27ee83991ff4
📥 Commits

Reviewing files that changed from the base of the PR and between 843db37 and 282bb06.

📒 Files selected for processing (24)
  • api/oss/src/apis/fastapi/workflows/exceptions.py
  • api/oss/src/apis/fastapi/workflows/router.py
  • api/oss/src/core/applications/service.py
  • api/oss/src/core/tools/platform_handlers.py
  • api/oss/src/core/workflows/service.py
  • api/oss/src/core/workflows/types.py
  • api/oss/tests/pytest/acceptance/tools/test_agent_config_tools.py
  • api/oss/tests/pytest/unit/tools/test_agent_config_handlers.py
  • api/oss/tests/pytest/unit/workflows/test_commit_agent_template_validation.py
  • api/oss/tests/pytest/unit/workflows/test_commit_harness_validation.py
  • api/oss/tests/pytest/unit/workflows/test_new_agent_template.py
  • openspec/changes/cross-agent-config-editing/design.md
  • openspec/changes/cross-agent-config-editing/specs/agent-config-management/spec.md
  • openspec/changes/cross-agent-config-editing/tasks.md
  • sdks/python/agenta/sdk/agents/config_errors.py
  • sdks/python/agenta/sdk/agents/dtos.py
  • sdks/python/agenta/sdk/agents/mcp/errors.py
  • sdks/python/agenta/sdk/agents/mcp/parsing.py
  • sdks/python/agenta/sdk/agents/skills/errors.py
  • sdks/python/agenta/sdk/agents/skills/parsing.py
  • sdks/python/agenta/sdk/agents/tools/compat.py
  • sdks/python/agenta/sdk/agents/tools/errors.py
  • sdks/python/agenta/sdk/agents/tools/parsing.py
  • sdks/python/oss/tests/pytest/unit/agents/test_config_entry_errors.py
💤 Files with no reviewable changes (1)
  • api/oss/src/apis/fastapi/workflows/router.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 0 remain after this review.

Comment thread api/oss/src/core/workflows/service.py Resolved
Comment thread api/oss/src/core/workflows/service.py Resolved
…y route

- Find the agent template the way the runtime does: parameters.agent when it
  is an object, else the bare parameters or the prompt fallback.
- Name a single entry given as an object (skills: {...}) instead of a KeyError.
- Refuse a metadata-only simple edit of an unrunnable stored agent before the
  artifact write.
- Check simple evaluator create and edit before their first write, and map the
  refusal to 422 on the evaluator write routes and the template loader.
- Let only the parse's own refusals become invalid_agent_configuration; a parser
  defect stays a 500.
- Skip embedded instructions in the instructions rule.
@mmabrouk
mmabrouk merged commit 9c596d5 into feat/cross-agent-config-editing Oct 11, 2026
71 of 74 checks passed
@mmabrouk mmabrouk removed the merge queue In the release merge queue; an agent is landing it label Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm This PR has been approved by a maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant