Skip to content

[feat] Port the optimized builder instructions and add the request tools - #8380

Open
mmabrouk wants to merge 1 commit into
feat/builder-agent-catalogfrom
feat/builder-agent-instructions
Open

mmabrouk wants to merge 1 commit into
feat/builder-agent-catalogfrom
feat/builder-agent-instructions

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Context

The builder agent (PR #8345) has a short first set of instructions. This PR replaces them with the optimized text from the builder optimization work, and gives the builder the three tools that the new text relies on: request_input, request_connection and request_secret.

What changes in the builder's behavior

  • Decisions in one form. For a new agent or a large change, the builder proposes a short scope in text and asks its decisions in one request_input form: at most three decisions, the recommended option first, plus a free-text field. Accepting the form is the ok to build. Declining it builds nothing.
  • Connections and secrets with tools. The builder checks what is already connected first. It asks for a missing connection with request_connection, or for a custom secret with request_secret, right after the form and before the build. It never asks for a secret in chat.
  • Check without a run. The builder cannot run another agent yet. It reads the saved configuration back, writes out what the agent would answer for two or three inputs, and fixes the instructions when an answer fails. It tells the person it could not run the agent, gives two sample inputs to try, and never says "tested".
  • Learning drive rules. The builder reads its project notes silently before a job, keeps dated entries where the newer one wins, and asks at most once per conversation, at the end of a job, before it saves a lesson for the team. Personal facts about one person never go in the drive.
  • Playbooks. The manage-project-agents skill gets a new reference file, references/playbooks.md, with the tricky parts and good defaults of common kinds of jobs.

The text is copied as written. It is not measured on the bench yet. The bench check tested-what-it-built fails by design until a run tool ships, because the builder cannot run the agent it built.

Changes in code

  • builder_agent.py: new text for BUILDER_AGENT_INSTRUCTIONS, the two skill bodies, the manage skill's description, and the new playbooks SkillFile. The drive path in the learning skill still comes from BUILDER_LEARNING_DRIVE_SLUG.

  • The builder's tool list keeps its platform tools and their ask and allow defaults. It now also holds the three client tools as @ag.embed references to their static catalog entries, the same entries the playground build kit uses:

    {"@ag.embed": {"@ag.references": {"workflow": {"slug": "__ag__request_input"}},
                   "@ag.selector": {"path": "parameters.tool"}},
     "name": "Request input"}
    

    These resolve at run time like any other embed. build_kit.py makes its embed helper public (reserved_static_tool_embeds) so both use one list.

Tests

  • test_builder_agent.py: a new test checks that the entry embeds all three client tools. The parse test and the tool resolver test now resolve the embeds through the static catalog (no database) first, and check that the resolver returns the platform tools with their approvals plus the three client tools.
  • pytest oss/tests/pytest/unit/workflows oss/tests/pytest/unit/applications: 925 passed.

Not release-ready on its own

Not release-ready on its own. This PR is one part of the builder agent. It becomes release-ready only together with the builder agent's UI and its first behavior, which other PRs build. Do not merge it alone.


Devin Review

…ilder the request tools

Replace the builder's instructions, its manage-project-agents and builder-learning skills
with the optimized text, and add the playbooks reference file to the manage skill. Embed
request_connection, request_input and request_secret in the builder entry the way the build
kit embeds them, beside the unchanged platform tools.
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Summary

Summary by CodeRabbit

  • New Features
    • The agent-building experience now guides users through clarifying requests, agreeing on scope, reviewing project context, and checking results.
    • Added practical playbooks for common agent-building tasks, with guidance on configuration, connections, and secrets.
    • Builder workflows now support requesting user input, connections, and secrets, and include post-build checks and configuration review.
    • Learning guidance now covers selective, date-aware reading and approval before saving shared lessons.

The Builder template now embeds three client tools alongside its platform tools. Its instructions and management and learning skills now define a detailed agent-building process, task playbooks, and updated rules for shared learning files.

Changes: Builder workflow and client tools

Layer / File(s) Summary
Embed and resolve client tools
api/oss/src/core/workflows/build_kit.py, api/oss/src/core/workflows/builder_agent.py, api/oss/tests/pytest/unit/workflows/test_builder_agent.py
The static embed helper is public, and the Builder template includes the input, connection, and secret client tools. Tests check embed contents, runtime resolution, and resolved permissions.
Define the build process and playbooks
api/oss/src/core/workflows/builder_agent.py, api/oss/tests/pytest/unit/workflows/test_builder_agent.py
The Builder instructions and management skill describe scope approval, agent creation and edit checks, and task-specific playbooks. A test checks the bundled playbooks reference.
Update shared learning rules
api/oss/src/core/workflows/builder_agent.py
The learning skill updates how the Builder reads, saves, and edits shared lessons. It adds guidance on dates, team time zones, and schedule conversions.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • Agenta-AI/agenta#7918: Removes tool catalog entries that are still listed in the build-kit operations defined by this PR.
Merge Risk: Moderate · up to 39225

The Builder's instructions now ask for a secret before the agent exists, so a custom secret cannot reach the new agent. Several instruction conflicts can also make the Builder skip an approval or misreport connections and schedules. These should be fixed, or the PR released together with the planned builder UI work, before merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: porting the optimized builder instructions and adding the request tools. The raw summary confirms the builder instructions, skills, playbooks, and the three client too…
Description check Passed The description is closely related to the changeset. It explains the replacement of the builder instructions, the three client tools, the playbooks reference file, the build_kit.py helper made public,…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: porting the optimized builder instructions and adding the request tools. The raw summary confirms the builder instructions, skills, playbooks, and the three client tools (request_input, request_connection, request_secret) embedded in the builder entry. The title is specific and concise. It uses a bracketed prefix ([feat]), which is a minor style point and does not affect relevance.

Full details: Description check

Explanation

The description is closely related to the changeset. It explains the replacement of the builder instructions, the three client tools, the playbooks reference file, the build_kit.py helper made public, and the test changes. It also gives the test result and a release caveat. Level of detail is more than the lenient check requires.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
Fix failing CI checks

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 3 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

@mmabrouk

Copy link
Copy Markdown
Member Author

🤖 The AI agent says:

Review loop result for head 3922570ae09baadd5e287b9d0f1880b991d0d88c (base 701d8189cdddc25ecb992e5401b367fcd2d4069e), 1 round.

Lens Final rating
Slop Medium
Simplify Medium

The code (the shared embed helper and the resolved-template tests) drew no code findings. Every reviewer raised the same text finding.

Fixed: nothing.
Left on purpose:

  • The playbooks tell the builder to run an agent once after a small change, to run a new agent by hand before a schedule, and to read recent sessions and runs in an audit. The builder has no tool for these yet. The text is a verbatim port by design, so this PR does not edit it. The contradiction goes back to the owner of the text.
  • The playbooks text keeps its source line breaks, so its wrapping differs from the other text bodies. It is a verbatim copy.
  • The stored-entry embed test stays. It is the only test that pins the slugs and the selector before resolution.

@mmabrouk

Copy link
Copy Markdown
Member Author

🤖 The AI agent says:

Review loop result for head 3922570ae09baadd5e287b9d0f1880b991d0d88c (base 701d8189cdddc25ecb992e5401b367fcd2d4069e), 1 round.

Lens Final rating
Correctness Medium
Security Low

Fixed: nothing.
Left on purpose: request_secret attaches the saved secret to the agent of the current chat. SecretRequestDock commits the binding to the chat's own revision. For the builder that is the static builder entry, which has no stored revision, so the attach fails and the request stays open. The secret should go to the agent the builder creates or changes. This needs a change in the secret dock or in the tool's input, which belongs with the builder's UI work. It goes back to the program owner as a difference between the plan and the code.

@mmabrouk

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Item Value
Status ready
Commit 3922570ae09baadd5e287b9d0f1880b991d0d88c
Manual expiry (UTC) Not granted
CI deadline (UTC) 2026-11-19T22:46:01Z
Workflow https://github.com/Agenta-AI/agenta/actions/runs/44609803303
Preview https://gateway-pr-8380.up.railway.app/w

Post /preview as a new comment to request a one-hour preview. Repeating it while active does not extend the hour.
Preview data is disposable. Expiry is enforced by the next successful cleanup sweep, not an exact timer.
Environment deletion stops its compute. Provider-retained storage and permanent services may still be billed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 701d818 and 3922570.

Files selected for processing: 3

  • api/oss/src/core/workflows/build_kit.py
  • api/oss/src/core/workflows/builder_agent.py
  • api/oss/tests/pytest/unit/workflows/test_builder_agent.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 0 remain after this review.

Comment on lines +117 to +120
Read every existing agent's instructions closely: they tell you what the person already \
has. If another agent reads a mailbox, a drive folder or a channel, that connection exists for \
this person: the new agent should use the same source, and the default option is to reuse it. \
Do not tell the person that something is not connected when another agent already uses it. \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Do not treat a connection named in another agent's instructions as available.

Saved instructions can name a source after its connection is removed. The reuse rule at lines 117-120 states that the connection "exists", and it forbids telling the person that a connection is missing when another agent uses it. This can hide a missing connection that list_connections shows. Use the other agent's instructions only to identify the candidate connection. Confirm it with list_connections before reuse. If it is not listed, follow the missing-connection rule in "Connections and secrets".

Suggested fix

-has. If another agent reads a mailbox, a drive folder or a channel, that connection exists for \
+has. If another agent reads a mailbox, a drive folder or a channel, that connection may exist for \
-Do not tell the person that something is not connected when another agent already uses it. \
+If `list_connections` does not list that connection, do not reuse it. Name the missing connection and follow "Connections and secrets". \

Comment on lines +219 to +220
- Do not set `"x-ag-stepper": true`. The person sees every decision on one screen and can \
accept the whole proposal with one click.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Give request_input one layout rule for Builder proposals.

With two decisions, these instructions prohibit x-ag-stepper: true, but the resolved request_input description tells the Builder to set it for several questions. If the Builder follows the tool description, the proposal does not appear on one screen as intended. Make the stepper optional in the shared description and let each flow choose its layout. (raw.githubusercontent.com)

them. A list answer stays under about 100 words: one line per item. Say what changed and that \
you checked it. No ids, no versions, no list of your steps, no offer of more work.

When the person only says thanks, ok or bye, answer in a few words and offer nothing new.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Treat ok as approval when a proposal is pending.

When all decisions are settled, Line 231 asks for approval in text. If the person replies ok, this rule directs the Builder to give a brief reply instead of starting the approved work. Limit this rule to turns with no pending approval. Make text approval enter the same build step as an accepted form. (raw.githubusercontent.com)

Comment on lines +309 to +310
- Name it in the plan of the proposal. Ask for it (`request_connection`, or `request_secret` \
for a custom secret) right after the person accepts the form, before the build.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major · Functional Correctness · Heavy lift

Bind the secret to the agent being built, not to the Builder chat.

The Builder instruction asks for request_secret "before the build" (api/oss/src/core/workflows/builder_agent.py:309-310). The new agent does not exist at that point. The secret dock attaches the secret to the entity the chat was opened with (revisionId={entityId}), and the request_secret tool has no target-agent field. The request therefore cannot reach the agent being created. Depending on how the chat entity loads, the attach either binds the secret to the Builder's own variant or fails and leaves the request open. In both cases the new agent runs without the secret, and the user sees a configured secret.

Required behavior: a secret attachment must target the agent the build creates or changes. Sequence the request_secret call so the target exists first, or give the tool an explicit target. Keep the instruction consistent with whichever choice is made.

Comment on lines +547 to +549
- **Look first, then report**: read every agent, schedule, subscription, connection and the \
recent deliveries
and sessions before you answer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Limit the audit to evidence the Builder can read.

The audit requires the Builder to inspect recent sessions and runs, but its tools expose schedules, subscriptions, and deliveries, not session history or run results. A dispatched delivery cannot establish whether the run succeeded. Use the available records and tell the person which results the Builder could not verify. (raw.githubusercontent.com)

Also applies to: 566-567

Comment on lines +568 to +572
about when it runs. Report what you found. If an agent's instructions say it runs hourly but \
you see no
schedule, take the instructions as the person's setup and say once, briefly, that you could \
not see the
schedule itself. Do not tell the person that their agents do not run.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Do not treat a stated cadence as a configured schedule.

If an agent's instructions say it runs hourly and a complete schedule lookup finds no match, this playbook tells the Builder to treat the instructions as the setup. Report that the instructions specify an hourly cadence but that no matching schedule was found. This leaves room for manual runs or other triggers without implying that hourly execution is configured. (raw.githubusercontent.com)

- **Clean every mention**: when you remove a skill, a tool or a rule, also remove the lines of \
the instructions
that refer to it, or the agent keeps trying to use it.
- **Check**: run the agent once and confirm the change shows in its answer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Use configuration readback to verify small edits.

After removing a skill, this playbook tells the Builder to run the edited agent. The Builder cannot do that. Follow the management skill's readback instead: confirm that the skill and its instruction references are absent. If the person also needs to check the agent's output, give them a manual test input. (raw.githubusercontent.com)

Comment on lines +773 to +775
- Use the team's time zone for every time you propose. Name the zone (for example "New York \
time"), not a region. If you do not know it, assume it from the context and list it as an \
assumption.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Functional Correctness · Quick win

Use the team's time zone only as a fallback.

If a person in Los Angeles requests “8 a.m. my time” and the team's notes say New York, this skill requires New York time while the main instructions require Los Angeles time. The two rules produce different UTC times. Use the time zone specified in the request. If the request specifies none, use the team default and state it. (raw.githubusercontent.com)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant