Comparing against Unsloth UD_Q4_K_XL
Would it be possible to add UD_Q4_K_XL to the comparison charts? (I'd do it, but on a Mac, so can't run ik)
I'll see if I can squeeze that in before the weekend pulls me away! Also, if you can't run ik, I'm curious why you're curious? BTW ik does run on mac (arm neon).
Fair question re: curiosity.
With many models I'm able to run Q8 (of BF16) quants, which from my understanding are very close to the original models.
With the size of GLM5.1 I have to run a smaller version, and there usually seems to be a lot of variability between the different ones at this level of quantization - and I saw that you had a few of the Q3 unsloth models in your chart, so I thought why not ask if you could include the UD-Q4_K_XL 😀
oh you must have one of those big macs with a lot of RAM then? have you tried ik_llama.cpp with it or tried any of my quants before? I think I've seen you around either here, on GH, or maybe discord?
the UD-Q4_K_XL clocks in about 28.456 GiB larger than my largest released quant and is still "above pareto line" looking at the trend.
if you're just trying to pick between the various UD quants, I suppose you could ask them for their own comparison data? ;p