feat: add MiniMax-M3 model support#10837
Open
nandanadileep wants to merge 1 commit into
Open
Conversation
Updates the llama.cpp pinned version to include preliminary MiniMax-M3 support via upstream PR #24523, and adds inference parameter defaults for the minimax-m3 model family. Changes: - backend/cpp/llama-cpp/Makefile: update checkout logic to support fetching PR refspecs from the upstream repo. The target now fetches all tags plus the specific LLAMA_VERSION refspec, and falls back to checking out the version directly if FETCH_HEAD is unavailable. - core/config/inference_defaults.json: add minimax-m3 family entry (temperature=1.0, top_p=0.95, top_k=40, min_p=0.01, repeat_penalty=1.0, matching the existing minimax defaults) and register it in the patterns list before the shorter minimax-m2.7 entry for correct longest-match-first ordering. Upstream: depends on ggml-org/llama.cpp#24523 Closes: mudler#10820
Owner
|
@nandanadileep instead of moving the pin, I'd rather suggest to include it as a patch, that way we keep pointing to latest llama.cpp |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #10820
MiniMax-M3 GGUF models cannot be loaded because the pinned llama.cpp version does not recognize the
minimax-m3architecture.This PR updates the llama.cpp dependency to include preliminary MiniMax-M3 support via upstream PR ggml-org/llama.cpp#24523, and adds inference parameter defaults for the minimax-m3 model family.
Changes
backend/cpp/llama-cpp/Makefile: Updated the
llama.cpptarget to fetch a specific refspec (PR ref) in addition to tags. This allows pinning to an unmerged PR branch until upstream merges. Once Preliminary MiniMax-M3 support ggml-org/llama.cpp#24523 is merged, theLLAMA_VERSIONcan be updated to the merged commit hash.core/config/inference_defaults.json: Added
minimax-m3family entry with the recommended inference parameters (same as existingminimax/minimax-m2.7entries: temperature=1.0, top_p=0.95, top_k=40). Added to patterns list beforeminimax-m2.7for correct longest-match-first ordering.Verification
go test ./core/config/ -vminimaxpattern in inference_defaults.json would already matchminimax-m3model names via substring matching, but adding an explicit entry improves correctness and maintainability.Notes
LLAMA_VERSIONin the Makefile to the merged commit hash and revert the Makefile changes.Assisted-by: Claude:claude-opus-4-8 [WebFetch] [WebSearch] [Task] [Bash] [Edit]