Skip to content

fix(anthropic): stop sending temperature=0.7 by default - #870

Open
chengwudi1 wants to merge 1 commit into
Cinnamon:mainfrom
chengwudi1:fix-anthropic-default-temperature
Open

chengwudi1 wants to merge 1 commit into
Cinnamon:mainfrom
chengwudi1:fix-anthropic-default-temperature

Conversation

@chengwudi1

Copy link
Copy Markdown

Closes #867

Problem

LCAnthropicChat hardcoded temperature=0.7 and always put it on the wire. Current Claude models (Sonnet 5, Opus 4.7+, the Fable family) removed the sampling parameters entirely and reject any of them:

400 - {'type': 'error', 'error': {'type': 'invalid_request_error',
      'message': '`temperature` is deprecated for this model.'}}

So "Test connection" fails with the default config, before the conversation even starts — the request authenticates fine and dies on the body.

Root cause

libs/kotaemon/kotaemon/llms/chats/langchain_based.py passed temperature=0.7 down unconditionally. langchain_anthropic's own ChatAnthropic.temperature defaults to None and strips None from the payload, so 0.7 was kotaemon's opinion on top of the library's own default.

Change

LCAnthropicChat now defaults temperature to None:

  • models that removed the parameter no longer see it;
  • older models that accept it fall back to their server-side default;
  • an explicit temperature=0.3 is still forwarded unchanged.

Deliberately scoped to LCAnthropicChat only — the other provider classes keep 0.7, because OpenAI-compatible endpoints accept the parameter.

Tests

Two payload-inspection tests added to test_llms_chat_models.py, both offline (no API key, no network), mirroring the issue's reproduction:

  • test_anthropic_chat_does_not_send_temperature_by_default — the payload must not contain temperature. Fails before this change (temperature: 0.7 was present).
  • test_anthropic_chat_forwards_explicit_temperature — an explicit value still reaches the provider.

Verified payload before: ['max_tokens', 'messages', 'model', 'temperature'] → after: ['max_tokens', 'messages', 'model'].

Verification

$ pytest libs/kotaemon/tests/test_llms_chat_models.py -q
3 passed, 1 skipped   (skip = llama_cpp not installed, pre-existing)

$ black --check <touched files>     # 22.3.0, per .pre-commit-config.yaml
$ isort --profile black --check
$ flake8 --max-line-length 88 --extend-ignore E203
All clean.

Disclosure: an AI assistant was used to prepare this change; I reviewed the diff and ran the tests myself.

LCAnthropicChat hardcoded temperature=0.7 and always put it on the wire.
Current Claude models (Sonnet 5, Opus 4.7+) removed the sampling
parameters outright and answer HTTP 400 for any of them, so the default
configuration fails with "temperature is deprecated for this model"
before the conversation starts. Reproduced offline by inspecting the
outgoing payload: keys were [max_tokens, messages, model, temperature].

langchain_anthropic defaults temperature to None and strips None from
the payload, so making None the default aligns the wrapper with its own
library: models that removed the parameter no longer see it, and older
models that accept it fall back to their server-side default.

An explicitly passed temperature is still forwarded unchanged, and the
other provider classes keep 0.7 because OpenAI-compatible endpoints
accept it. This change is deliberately scoped to LCAnthropicChat.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LCAnthropicChat hardcodes temperature=0.7, which current Claude models reject with HTTP 400

1 participant