Qwen3.5-35B-A3B-GGUF:UD-Q4_K_M does not work in Ollama 0.17.4
#14
by Arete7 - opened
Qwen3.5-35B-A3B-GGUF:UD-Q4_K_M does not work in Ollama 0.17.4
Error: 500 Internal Server Error: unable to load model: C:\Users\user.ollama\models\blobs\sha256-223138866b87b12e68ffb43a1d45afb572921e9cd4c594e6a736df94c5130466
At the moment newer GGUFs don't work in Ollama anymore due to chat template incompatibility. Use llama.cpp supporter backends
Actually... You can get this to work with a caveat - You'll have to remove the vision model until Ollama updates.
https://github.com/ollama/ollama/issues/14503#issuecomment-3986898959
I followed the above steps for 'UD-Q4-K-XL' - and generated the following qwen35modelfile.txt
FROM /usr/share/ollama/.ollama/models/blobs/sha256-1b0ac637dfa092bbba2793977db9485a40c4f8b42df5fe342f0076d61b66ae83
TEMPLATE {{ .Prompt }}
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER min_p 0.0
PARAMETER presence_penalty 0.0
PARAMETER repeat_penalty 1.0
PARAMETER num_ctx 262144
PARAMETER num_predict 32565
Then simple create the model using:
ollama create Qwen-3.5-35B-Tool-256:Q4-K-XL -f qwen35modelfile.txt