[Request] Availability of Non-GGUF Weights for vLLM/SGLang Support
Hi there,
Thanks for the great work on providing the GGUF weights!
I was wondering if you could also share the model weights in a format other than GGUF (such as the standard Hugging Face format)?
Currently, inference engines like vLLM and SGLang do not fully support the GGUF version of Qwen 3.5 yet. Providing the original weights would allow us to utilize these engines for better performance and compatibility.
Thanks again for your help!
Same here, would love to have the Safetensors or an AWQ 8 bits !
+1 on non-gguf. The finetune seems great, but being only gguf is a deal breaker. If there's no fp8/nvfp4 it might as well not exist lol.
This ablitered model is better than all other abliterated model I tested. Would love a safetensor release for this one.
+1 on getting safetensors model that we could host in vLLM :)
非常感谢您的付出，是否可以支持MLX格式呢？mac电脑上面使用