Skip to content

tinyBLAS on by default: the CPU matrix kernel of F32, F16, BF16, Q8_0 and Q4_0 weights - #133

Open
timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:tinyblas-default
Open

timoncool wants to merge 1 commit into
ServeurpersoCom:masterfrom
timoncool:tinyblas-default

Conversation

@timoncool

@timoncool timoncool commented Nov 12, 2026 •

Copy link
Copy Markdown

Standalone ggml builds GGML_LLAMAFILE off; llama.cpp builds it on. This turns it on by default the same way GGML_CUDA_GRAPHS is (still overridable with -DGGML_LLAMAFILE=OFF).

Measured on the processor with the same ggml in a sibling LM + DiT music engine (16 cores, the same 30 s request): the DiT step on Q8_0 weights about 1.24x faster (9736 -> 7830 ms/step); the LM on Q8_0 about 10% slower per frame (290 -> 318 ms); K-quants (Q3_K_M) unchanged, tinyBLAS does not take them. Results on the processor differ from the non-tinyBLAS build for the same seed (different summation order).

Summary by CodeRabbit

  • Chores
    • Builds now enable llamafile support by default when no setting is specified. Explicit configuration continues to be respected. This may make the feature available in builds without requiring an additional configuration step.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

CMake now sets GGML_LLAMAFILE_DEFAULT to ON when GGML_LLAMAFILE is undefined. An existing definition remains unchanged.

Changes: CMake Option Default

Layer / File(s) Summary
Set the CMake option default
CMakeLists.txt
When GGML_LLAMAFILE is undefined, CMake sets GGML_LLAMAFILE_DEFAULT to ON. Comments describe tinyBLAS and its override option.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Feature

Suggested reviewers: serveurpersocom

Merge Risk: Minimal · up to 77dc9

This enables tinyBLAS by default for standalone builds and can be overridden with -DGGML_LLAMAFILE=OFF. No merge-blocking risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: tinyBLAS, the CPU matrix kernel in llamafile, is enabled by default. The raw summary confirms that CMakeLists.txt sets GGML_LLAMAFILE_DEFAULT to ON when GGML_LLAMAFILE…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check

Explanation

The title names the main change: tinyBLAS, the CPU matrix kernel in llamafile, is enabled by default. The raw summary confirms that CMakeLists.txt sets GGML_LLAMAFILE_DEFAULT to ON when GGML_LLAMAFILE is undefined. The objectives state the same goal, enabling GGML_LLAMAFILE by default, with an override via -DGGML_LLAMAFILE=OFF. The title lists the affected weight types (F32, F16, BF16, Q8_0, Q4_0), which the CMake change does not show directly but which the objectives describe. The title is long, but it is specific and matches the change.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: bc76df65-0226-4127-b1ad-65badc58749b
Commits

Reviewing files that changed from the base of the PR and between 07abf07 and 77dc913.

Files selected for processing: 1

  • CMakeLists.txt

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant