Skip to content

Support Aleph Alpha's Kolibri model #75

Open

Description

@SommerEngineering

Idea

On 2026-10-03, Aleph Alpha released Kolibri, an open-weights model from Germany that focuses on the German language. AI Studio should know the model and handle it correctly, for example when it runs on a self-hosted vLLM server.

The model

Checked on 2026-10-04 against the website and the model card:

  • Model IDs: Aleph-Alpha/Kolibri-1 (FP8) and Aleph-Alpha/Kolibri-1-BF16, plus several quantized variants.
  • Architecture: Mixture of experts with 78B parameters in total and about 3.5B active per token. Trained from scratch, not derived from another model.
  • Languages: German and English.
  • License: Apache 2.0.
  • Modality: Text only.
  • Context window: 1,048,576 tokens. Aleph Alpha recommends serving it with 262,144 tokens.
  • Tool calling: Yes, in Hermes style. vLLM needs --tool-call-parser kolibri1 --enable-auto-tool-choice.
  • Reasoning: Controllable through chat_template_kwargs with enable_thinking and reasoning_effort (none, low, medium, high). vLLM needs --reasoning-parser kolibri1.
  • Serving: vLLM with Aleph Alpha's inference plugin. Runs on one H200, B200, or B300, or on two A100 80 GB or H100.

What AI Studio needs

Today, the model registry does not know Kolibri, so it falls back to the global default profile: text in and out, Chat Completion API, and function calling. That part fits, but reasoning and the context window are missing.

  • Model family: A new vendor Aleph Alpha and a Kolibri family with text input and output, function calling, the Chat Completion API, and controllable reasoning. The rule has to match the names servers use, such as Aleph-Alpha/Kolibri-1, Kolibri-1-BF16, and the quantized variants.
  • Context window: Decide how the 1M tokens and the recommended 262k tokens map onto the profile.
  • Reasoning: Check that turning reasoning on and off reaches Kolibri on vLLM. The vLLM dialect reads enable_thinking and thinking inside chat_template_kwargs today, but not reasoning_effort.
  • Token count: Kolibri brings its own tokenizer. Check whether we can count tokens with it.
  • Test: End to end against a vLLM server: chat in German and English, tool calling, and reasoning on and off.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestproviderAdd an AI provider

    Type

    No type

    Projects

    • Status
      Accepted

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions