[Amy Chang (chang-amy)]

Related issue

Fixes #8122

Stacked on #8076 (Jev Choice tool). Until #8076 merges, this diff also shows its commits. The Score changes are the last three commits: 93616d6b6, d48a37bb3, and 47633e470.

Summary

Adds JevScoreTool, which rates state against one ordered TypeSafe Jev Score rubric.

• Input: state, question, and criteria, an ordered list of level descriptions with the lowest first. A level's position is its score, starting at 0. This mirrors the Choice tool's state / question / choices shape.
• SDK types: the request is built with the SDK's own Score type rather than a parallel schema. The tool adds only the checks the SDK does not make: it rejects an empty level list, blank or None levels, and a blank question.
• Output: {"success": True, "score", "probabilities", "confidence", "legend"}, plus model and usage when the response includes them.
• Score semantics: score is the SDK's probability-weighted expected score, passed through unchanged, so it can fall between levels. success only reports whether a usable answer came back. It is never a pass/fail verdict on the score.
• Failures: a missing answer (including scores being None), an answer without a score, invalid input, a missing SDK, and SDK errors all return {"success": False, "error": ...} instead of a made-up score.
• Exports: JevScoreTool is available from crewai_tools and crewai_tools.tools. The JevDecisionTool import moves only because ruff sorts imports.
• README: uses the standard Description / Installation / Example / Arguments sections and installs with uv add "crewai-tools[typesafe-sdk]". The missing-SDK error message gives the same command.

Choice tool behaviour is unchanged.

Verification

• Tests added or updated for the changed behavior

• Relevant tests and quality checks pass locally

• uv run pytest tests/tools/jev_score_tool_test.py tests/tools/jev_decision_tool_test.py: all pass. Score tests use a fake SDK module and mock client, so they make no paid API calls. They cover returned scores, ordered criteria sent as an SDK Score, fractional scores reported as-is, missing answers, invalid input that never reaches the SDK, SDK failure, and a missing dependency.

• One test round-trips a real typesafe_sdk.SystemOneResponse through the tool when the SDK is installed. It ran locally against typesafe-sdk 0.7.x.

• ruff check, ruff format --check, and mypy (strict) are clean on the new files.

Additional context

• Coordination with the Choice tool author: I'd like a review of the input and output shape from the feat(tools): add jev decision tool #8076 author. The tools share about 60 lines: env vars, client setup and closing, and model and usage metadata. I left them duplicated so this PR doesn't edit the Choice tool, and can extract a shared helper here or in a follow-up if you prefer.
• Different install hints: the Score tool's install hint now uses the crewai-tools[typesafe-sdk] extra, while the Choice tool still says uv add --optional typesafe-sdk typesafe-sdk. Happy to align them once feat(tools): add jev decision tool #8076 lands.
• SDK fallback: with an injected client, the Choice tool falls back to a plain dict question when the SDK is missing. The Score tool always requires the SDK.

🤖 Generated with Claude Code

---

Note

Low Risk
Isolated new optional tools behind an extra; behavior is guarded by validation, structured error returns, and tests, with no changes to existing tool logic beyond exports and lockfile.

Overview
Adds a typesafe-sdk optional extra (crewai-tools[typesafe-sdk]) and wires JevDecisionTool and JevScoreTool into the public crewai_tools exports. Both tools call TypeSafe System One with TYPESAFE_* env vars, validate inputs locally, and return structured {success, error, …} payloads (choice label + probabilities, or expected score + level probabilities/legend).

JevScoreTool is the primary feature here: it builds an SDK Score question from state / question / ordered criteria, passes through the SDK’s fractional expected score without treating it as pass/fail, and surfaces API failures and missing answers explicitly. JevDecisionTool mirrors the same pattern for labeled Choice questions. Each ships with a README and pytest coverage (mocked client plus optional real SDK round-trip for score).

uv.lock is updated for typesafe-sdk and its HTTP stack dependencies.

^{Reviewed by Cursor Bugbot for commit d586493. Bugbot is set up for automated code reviews on this repo. Configure here.}

[strix-security]

Strix Security Review

Warning

This pull request has 129 commits after the last Strix review (47633e4). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds a `JevScoreTool` that rates task state against an ordered TypeSafe Jev Score rubric and returns the expected score, per-level probabilities, confidence and legend, alongside the stacked `JevDecisionTool`; both are exported from `crewai_tools` and gated behind a new optional `typesafe-sdk` extra.

Review summary

Reviewed the new JevScoreTool and JevDecisionTool (plus their exports, the typesafe-sdk extra, and the lockfile additions). Both tools are thin, lazily-imported wrappers around the TypeSafe SDK: inputs are re-validated through their pydantic schemas, the only sink is an HTTP POST to the TypeSafe API, and the API key never appears in tool arguments, URLs, returned metadata, or the sanitized SDK error strings. The new dependency chain (typesafe-sdk, httpx2, httpcore2, truststore) is legitimate and hash-verified against PyPI. No security issues found in the changed code.

Ruled out (4)

• Can the new tools leak the TypeSafe API key through error strings, logs, or returned metadata?
  The key is only placed in the Authorization header, the SDK redacts credential headers and exception text, and the tools return only score/choice/probabilities/confidence/legend/model/usage or a sanitized error string.
• Do the Any-typed state/criteria inputs reach an unsafe sink such as eval, pickle, a shell, or a filesystem path?
  The only sink is a JSON HTTP request body; the SDK encoder fails closed on non-JSON objects and every coercion error is caught inside `_run`.
• Does the new typesafe-sdk extra pull in a typosquatted or tampered dependency?
  typesafe-sdk, httpx2, httpcore2, httpx2-jsfetch and truststore are legitimate published packages whose lockfile hashes match the PyPI artifacts, and httpx2/httpcore2 install under their own top-level names so they cannot shadow httpx.
• Can a config-supplied tool name plus a TYPESAFE_BASE_URL override exfiltrate the API key to an attacker host?
  The tools never read TYPESAFE_BASE_URL themselves, no tool input influences the endpoint, and the claimed attacker would already control crewai.json or the process environment, which is full compromise.

_{Review 1 on this PR · 47633e4 · requested Nov 8 17:59 UTC when the PR was opened}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 7bca0ab. Configure here.}