Jackrong

/

Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled

like 1.7k

Thanks for reporting this issue!
The model was exported in a Transformers 5.2.0 environment, so the tokenizer configuration may rely on components that are not included in the transformers <5 versions currently required by vLLM.

same issue. Any suggestions for vLLM user?

I tried to update the transformer in vllm v0.17.1's docker image with the following docker file:

FROM vllm/vllm-openai:v0.17.1
RUN pip install --no-cache-dir "transformers>=5.2.0,<6" "accelerate>=1.0.0" "huggingface_hub>=0.25.0"
RUN pip install --no-cache-dir "qwen-vl-utils" || true

the starting process seems fine with ray's extra config --tokenizer-mode hf
but it OOM with my 2 4090s when the worker node joined the cluster. so I'm not able to verify the work around e2e. hope this helps.
Also, it would be nice if we could have the FP8 version of this excellent 27B model.

vllm 0.17.1 works with transformers 4, this model was made with 5, but you can work around it with setting the tokenizer_class in tokenizer_config.json from TokenizersBackend to Qwen2TokenizerFast to make it work.

overwrite the tokenizer_config.json file, replace tokenizer_class": "TokenizersBackend" withtokenizer_class": "Qwen2Tokenizer".

overwrite the tokenizer_config.json file, replace tokenizer_class": "TokenizersBackend" withtokenizer_class": "Qwen2Tokenizer".

修改一以后可以成功运行了 谢谢

overwrite the tokenizer_config.json file, replace tokenizer_class": "TokenizersBackend" withtokenizer_class": "Qwen2Tokenizer".

I've made the change but still get the error, is there a cache somewhere ?

whats error? say it explicitly

whats error? say it explicitly

Exactly the same, that's why I did not repeated it :
(APIServer pid=179) ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.

So I think my change was not took in account

you have to edit the file where it stored in ur machine

Yup, I've changed this file exactly :
~/.cache/huggingface/hub/models--Jackrong--Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled/snapshots/59f57e471e041fe27ee3f98dba2ec02a50817afc/tokenizer_config.json