Repository navigation
feat(tools): add confidence-aware route selector for Jev Choice results - #9437
Rosman Cariño (Rosman-Carino) wants to merge 1 commit into
Conversation
Fixes crewAIInc#8067. Part of crewAIInc#8065. Written with assistance from Claude.
Strix Security Review✅ No open security findings. PR summary: Adds a new `crewai_tools.tools.jev_decision_tool.route_selector` module containing a small pure function, `select_route`, that maps a Jev Choice label plus its confidence onto one of a caller-supplied set of Flow route labels, falling back when the label or confidence cannot be trusted. It also adds an empty package `__init__.py` and a test module covering threshold, payload and configuration behaviour plus two Flow runs. Review summaryReviewed the three added files: Review 1 on this PR · Reviewed by Strix |
|
This PR was written with AI assistance (Claude Code). Per CONTRIBUTING.md it needs the |
Related issue
Fixes #8067
Part of #8065
Summary
Adds
select_route, a small pure function increwai_tools.tools.jev_decision_tool.route_selectorthat maps a Jev Choiceanswer (label + TypeSafe confidence) onto a caller-provided set of Flow route labels.
min_confidenceselects that route.non-numeric confidence selects the explicit
fallback. A malformed payload neverraises from inside a router.
allowed_routes, fallback not in routes —which also covers empty routes — or threshold not a number in [0, 1]) raises
ValueError, checked before the answer so misconfiguration can never hide behinda fallback.
Three new files (291 lines), nothing modified. No new dependency; stdlib only.
The example and Flow test consume dicts shaped like
JevDecisionTool's result in #8076(
{"success", "choice", "probabilities", "confidence"}), so the two compose directly.Design notes
allowed_routes/fallback/min_confidencecan't be transposed.
min_confidenceis required, no default: the issue calls the threshold"configurable" and names none, and a routing cutoff should be an explicit choice.
Happy to add a default if preferred.
boolis rejected as a confidence/threshold (True == 1would read as 1.0).isinstance(choice, str)both narrows the type for strict mypy and keeps anunhashable payload from raising against a set of routes.
self.state); the Flow test asserts the confident path actually reaches therefundlistener, not just that the fallback path works.jev_decision_tool/matches feat(tools): add jev decision tool #8076 so the two PRs compose. Both PRs addjev_decision_tool/__init__.py; ours is empty so the add/add conflict for whichevermerges second is trivial (take feat(tools): add jev decision tool #8076's). Happy to rebase onto feat(tools): add jev decision tool #8076 if it lands first.
crewai_toolsre-export and no alias.The package
__init__re-exports tool classes and their config/schema types, nothelper functions, and this helper is internal to the Jev integration rather than a
tool. (
JevDecisionToolin feat(tools): add jev decision tool #8076 is the public surface.)Verification
28 behavioral tests, no credentials or network: below/equal/above/just-below threshold,
inclusive bounds, int values, unknown/None/non-string labels,
None/NaN/inf/out-of-range/bool/string confidence, each invalid-config path, config
validated before the answer for every config error, and two real
Flowrunsthrough a
@start → @router → @listenchain with synthetic decisions shaped likeJevDecisionTool's result (including its{"success": False, "error": ...}form).uv run pytest lib/crewai-tools/tests/tools/test_jev_route_selector.py→ 28 passed on Python 3.10–3.13uv run mypy lib/crewai-tools/src/crewai_tools/tools/jev_decision_tool/→ clean (strict)uv run ruff check/ruff format --check/pre-commit run→ cleanAdditional context
Docs. No
docs/edge/page added: the helper has no user-facing tool surface, theJev tool (#8076) has no page yet to hang a section on, and comparable helper modules
in crewai-tools are undocumented. Happy to add one (with ar/ko/pt-BR sync) if
maintainers want it.
Relationship to #8156. #8156 (open) addresses the same issue from a separate
jev_tool/package and edits the two package__init__s to re-export the helper.This PR is the assignee's implementation; it is written to compose with #8076's
package and modifies nothing existing. Happy to defer to maintainers on which to
take forward.
🤖 Generated with Claude Code
Note
Low Risk
New isolated helper and tests under
jev_decision_tool; no changes to existing tools or runtime paths.Overview
Adds a
select_routehelper underjev_decision_tool/route_selectorso Flow@routersteps can turn Jev Choice outputs (choice+ TypeSafeconfidence) into a route name with an explicit confidence floor and safe fallback.Callers pass
allowed_routes, afallback, and requiredmin_confidence. Known labels at or above the threshold return that route; bad or low-trust inputs (missing/wrong type, unknown label, invalid confidence including NaN/bool) returnfallbackwithout raising. Misconfiguration (allowed_routesas a bare string, fallback not in routes, invalid threshold) raisesValueErrorbefore evaluating the decision.Adds 28 behavioral tests, including a small Flow (
@start→@router→@listen) that mirrorsJevDecisionTool-shaped dicts (success and error payloads).Reviewed by Cursor Bugbot for commit 1e742c4. Bugbot is set up for automated code reviews on this repo. Configure here.