You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
tinyBLAS on by default: the CPU matrix kernel of F32, F16, BF16, Q8_0 and Q4_0 weights - #133
Standalone ggml builds GGML_LLAMAFILE off; llama.cpp builds it on. This turns it on by default the same way GGML_CUDA_GRAPHS is (still overridable with -DGGML_LLAMAFILE=OFF).
Measured on the processor with the same ggml in a sibling LM + DiT music engine (16 cores, the same 30 s request): the DiT step on Q8_0 weights about 1.24x faster (9736 -> 7830 ms/step); the LM on Q8_0 about 10% slower per frame (290 -> 318 ms); K-quants (Q3_K_M) unchanged, tinyBLAS does not take them. Results on the processor differ from the non-tinyBLAS build for the same seed (different summation order).
Summary by CodeRabbit
Chores
Builds now enable llamafile support by default when no setting is specified. Explicit configuration continues to be respected. This may make the feature available in builds without requiring an additional configuration step.
This enables tinyBLAS by default for standalone builds and can be overridden with -DGGML_LLAMAFILE=OFF. No merge-blocking risk was identified.
Pre-merge checks: 4
Check name
Status
Explanation
Title check
The title names the main change: tinyBLAS, the CPU matrix kernel in llamafile, is enabled by default. The raw summary confirms that CMakeLists.txt sets GGML_LLAMAFILE_DEFAULT to ON when GGML_LLAMAFILE…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Full details: Title check
Explanation
The title names the main change: tinyBLAS, the CPU matrix kernel in llamafile, is enabled by default. The raw summary confirms that CMakeLists.txt sets GGML_LLAMAFILE_DEFAULT to ON when GGML_LLAMAFILE is undefined. The objectives state the same goal, enabling GGML_LLAMAFILE by default, with an override via -DGGML_LLAMAFILE=OFF. The title lists the affected weight types (F32, F16, BF16, Q8_0, Q4_0), which the CMake change does not show directly but which the objectives describe. The title is long, but it is specific and matches the change.
Finishing Touches
Generate unit tests (beta)
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: bc76df65-0226-4127-b1ad-65badc58749b
Commits
Reviewing files that changed from the base of the PR and between 07abf07 and 77dc913.
Files selected for processing: 1
CMakeLists.txt
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standalone ggml builds GGML_LLAMAFILE off; llama.cpp builds it on. This turns it on by default the same way GGML_CUDA_GRAPHS is (still overridable with -DGGML_LLAMAFILE=OFF).
Measured on the processor with the same ggml in a sibling LM + DiT music engine (16 cores, the same 30 s request): the DiT step on Q8_0 weights about 1.24x faster (9736 -> 7830 ms/step); the LM on Q8_0 about 10% slower per frame (290 -> 318 ms); K-quants (Q3_K_M) unchanged, tinyBLAS does not take them. Results on the processor differ from the non-tinyBLAS build for the same seed (different summation order).
Summary by CodeRabbit