[Prince Maurya (PRINCE2-AI)]

Closes #8068 (part of #8065).

What this adds

create_jev_noul_guardrail(question, threshold, *, client=None, model=None, question_name="output_passes") in crewai_tools.tools.jev_noul_guardrail. It returns a callable task guardrail that asks TypeSafe's Jev model one Noul question about the task output (state=output.raw) and compares Jev's probability with the threshold.

• Pass: probability >= threshold returns (True, output.raw), so the output is kept unchanged. A probability equal to the threshold passes; this is documented.
• Reject: returns (False, reason). The reason includes the question, the probability and the threshold, and it goes back to the agent through the task's existing guardrail_max_retries loop. The guardrail adds no retry loop of its own.
• Missing or malformed answers never pass: no answer under the question name, a non-Noul answer, a missing value, a non-number or bool, NaN or infinity, or a value outside [0, 1] all reject with an explicit reason.
• SDK failures stay explicit: exceptions from system_one (authentication, network, API, response validation) propagate unchanged. They never count as a pass and are not turned into a retry.
• Config validation: the threshold must be a finite real number in [0, 1] (bools and strings are rejected), and the question and question name must be non-empty. Invalid config raises ValueError when the guardrail is created.
• Client: pass any client with a compatible system_one method (a SystemOneClient protocol; TypeSafeClient satisfies it under mypy). With no client, a typesafe_sdk.TypeSafeClient is created from the TYPESAFE_* env vars for each check and closed in a finally block.
• The guardrail only reads Noul probabilities. Choice confidence is not used, so the two are not treated as interchangeable.

Design notes

• A function, not a callable class. LLMGuardrailStartedEvent calls inspect.getsource() on callable guardrails that are not LLMGuardrail/HallucinationGuardrail. That raises TypeError for an instance of a class with __call__, so the factory returns a closure. A test runs it through Task(guardrail=...) and process_guardrail.
• No from __future__ import annotations in the module. Task.validate_guardrail_function checks the return annotation at runtime, and a string annotation would be rejected.
• Optional dependency. The SDK is imported lazily via importlib, so crewai_tools imports without it and mypy passes whether or not typesafe-sdk is installed. I didn't add a typesafe-sdk extra or touch pyproject.toml/uv.lock, to avoid conflicting with feat(tools): add jev decision tool #8076, which adds that extra for [FEATURE] Add a minimal Jev Choice decision tool using the TypeSafe Python SDK #8066. Questions are sent in the SDK's raw-dict form ({"type": "noul", "instructions": ...}), so tests need neither the SDK nor credentials. Once feat(tools): add jev decision tool #8076 lands, I'm happy to move this next to jev_decision_tool or share a package if maintainers prefer.
• It is not exported from crewai_tools.__init__ for now, so it doesn't conflict with feat(tools): add jev decision tool #8076's export lines. I can add the export if wanted.

Tests

lib/crewai-tools/tests/tools/test_jev_noul_guardrail.py: 41 tests with a fake client, no API keys or paid calls.

• Accept and reject paths: pass and reject, including the request payload (state, question, model), the rejection reason, and a custom question name.
• Threshold boundaries: equal, just below, 0 and 1.
• Inputs that must never pass: 12 malformed or missing answer shapes.
• Errors and config: SDK exceptions propagate; invalid config raises ValueError.
• Default client: it is created and closed for each check, it is still closed when the call fails, and a missing SDK raises an install hint.
• Inputs and real SDK types: LiteAgentOutput works, and the guardrail reads real typesafe_sdk.SystemOneResponse objects. That last test is skipped when the SDK isn't installed.
• Task integration: Task(guardrail=...) accepts it, and process_guardrail and its events work. Through Task._invoke_guardrail_function, a rejection retries the agent once and then passes, and the task fails after guardrail_max_retries are used up.

Local checks on Python 3.12:

• pytest on the new test file: 41 passed with typesafe-sdk 0.7.3 installed, and 38 passed and 2 skipped without it.
• ruff check and ruff format --check (0.15.1) are clean on the new files.
• mypy (strict, 1.19.1) is clean on the new module, with and without the SDK installed.
• bandit reports no issues.

I have not run it against the live TypeSafe API, because I don't have a key.

AI disclosure

I prepared this with an AI coding assistant (Claude Code), as stated in my comment on the issue, and I reviewed and tested it. I can't apply the llm-generated label myself; could a maintainer please add it?

🤖 Generated with Claude Code

https://claude.ai/code/session_01HyD8jEWKJEV4qoziJRYLmM

---

Note

Medium Risk
New task guardrail path can block or retry agent output and may call an external TypeSafe API when enabled, though behavior is opt-in and SDK failures fail loudly rather than silently passing.

Overview
Adds an opt-in create_jev_noul_guardrail factory under crewai_tools.tools.jev_noul_guardrail, plus README and broad tests. Tasks can attach a guardrail that sends output.raw to TypeSafe Jev via system_one with one Noul question and passes when the returned probability is ≥ a configured threshold (inclusive); otherwise it returns (False, reason) and uses CrewAI’s existing guardrail_max_retries.

Rejections also cover missing or invalid Noul answers; SDK errors propagate instead of being treated as pass or soft retry. Optional client/model/question_name are supported; without a client, typesafe_sdk is lazy-imported and a per-check TypeSafeClient is opened and closed. The factory returns a plain closure (not a callable class) for guardrail introspection compatibility, and the module avoids postponed annotations so Task’s runtime return-type check accepts tuple[bool, Any]. Not exported from top-level crewai_tools yet.

^{Reviewed by Cursor Bugbot for commit 90f79ed. Bugbot is set up for automated code reviews on this repo. Configure here.}

[strix-security]

Strix Security Review

Warning

This pull request has 20 commits after the last Strix review (42d562c). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

No security issues found.

Review summary

Reviewed the four added files in this PR, which introduce an opt-in, callable task guardrail backed by TypeSafe's Jev Noul model. The factory validates all configuration (threshold is a finite non-bool value in [0,1]; question and question_name are non-empty strings), and the guardrail validates the SDK response probability before use, so malformed or out-of-range answers can never pass. The only external call sends the task's raw output to the TypeSafe SDK, which is the documented, opt-in purpose of the module and not a code-level vulnerability. Rejection reasons only incorporate developer-supplied configuration and validated numeric probabilities, so no untrusted input is reflected back into the agent. Static analysis (bandit, semgrep) returned no findings, and no hardcoded secrets, unsafe deserialization, or injection sinks were found. No security issues require changes.

_{Updated for 42d562c.}

---

Reviewed by Strix
_{Re-run review · Configure security review settings}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 27b46c45-5f6d-4191-b9d6-6161a45f1fde

📥 Commits

Reviewing files that changed from the base of the PR and between 42d562c and 90f79ed.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: baa5de76-d0b0-4bec-a635-b480b3eadc38

📥 Commits

Reviewing files that changed from the base of the PR and between 753d192 and 42d562c.

📒 Files selected for processing (4)

• lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/README.md
• lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/__init__.py
• lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/jev_noul_guardrail.py
• lib/crewai-tools/tests/tools/test_jev_noul_guardrail.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

📝 Walkthrough

Adds an opt-in Jev Noul callable guardrail for task outputs. The guardrail validates a Noul probability, compares it with a configured threshold, and returns acceptance or rejection. The package exports the factory and documents setup and task usage.

Changes: Jev Noul guardrail

Layer / File(s)	Summary
Guardrail contract and evaluation lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/jev_noul_guardrail.py, lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/__init__.py, lib/crewai-tools/tests/tools/test_jev_noul_guardrail.py, lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/README.md	Adds the exported guardrail factory and client protocol. The callable validates configuration and Noul responses, applies the inclusive threshold, and uses a supplied client or creates and closes a default client. Tests cover threshold handling, invalid responses, SDK errors, and client lifecycle. The README documents installation and response behavior.
Task retries and usage example lib/crewai-tools/tests/tools/test_jev_noul_guardrail.py, lib/crewai-tools/src/crewai_tools/tools/jev_noul_guardrail/README.md	Adds tests for task rejection and retries, plus an optional test using TypeSafe response objects. The README includes a task usage example.

Sequence Diagram(s)

sequenceDiagram
  participant Task
  participant Guardrail as create_jev_noul_guardrail callable
  participant Client as SystemOneClient
  Task->>Guardrail: Provide task output
  Guardrail->>Client: system_one(output.raw, named question)
  Client-->>Guardrail: Return Noul response
  Guardrail-->>Task: Return pass or rejection

Loading

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 42d56

No actionable issue remains before merge. Live TypeSafe API behavior has not been verified, so normal integration validation remains advisable.

Pre-merge checks | 4 | 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 17.24% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 29 functions across 3 files. (1 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check		The title is concise, specific, and accurately identifies the addition of an opt-in Jev Noul task guardrail.
Description check		The description provides the linked issue, implementation summary, behavior details, design context, test coverage, verification results, limitations, and additional context. It does not use every tem…
Linked Issues check		The changes satisfy the coding requirements in directly linked issue #8068. create_jev_noul_guardrail validates a finite threshold in [0, 1], validates the question fields, sends one named Noul ques…
Out of Scope Changes check		The changes stay within #8068 scope. They add the Jev Noul guardrail package, its focused README, and behavioral tests. The package initializer exports only the new package API. The changes do not add…

Full details: Docstring Coverage

Explanation

Docstring coverage is 17.24% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 29 functions across 3 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}