missing tok_embeddings.weight error when trying to run with llama.cpp
i tryied to load the model with llama.cpp but i get this error, how can i fix it?:
my command:
./main -t 12 -m models/starcoderbase-ggml-q5_1.bin --color -c 2048 --temp 0.7 --top_k 40 --top_p 0.5 --repeat_penalty 1.17 -n -1 -r "### Human:" -i
output:
main: build = 588 (ac7876a)
main: seed = 1685012620
llama.cpp: loading model from models/starcoderbase-ggml-q5_1.bin
error loading model: missing tok_embeddings.weight
llama_init_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'models/starcoderbase-ggml-q5_1.bin'
main: error: unable to load model
llama.cpp not supported it - https://github.com/ggerganov/llama.cpp/issues/1441
There are starcoder.cpp, but there is another issue - https://github.com/bigcode-project/starcoder.cpp/issues/11
For now you can use example code from ggml main repo to inference - https://github.com/ggerganov/ggml/tree/master/examples/starcoder
For now koboldcpp supports starcoder gglm models.