Hugging Face's logo

arnavgrg
/
codealpaca-qlora

Text Generation
PEFT
English
Model card Files
xet
 Community
4

System crashing on collab

#1
by tunesh - opened
Discussion
tunesh

While I am loading your models in the collab or in fact my trained model using peft the kernel is crashing. Do I need to do something else? Please let me know.
tunesh
•
edited Sep 13, 2023

This is happening due to the less amount RAM available on collab.
jprashant21

I tried to load the fine-tuned model in my VM (30GB RAM and GPU T4) but still my system crashed due to OOM. Is there any other tested way to load the fine-tuned model binaries with ludwig?
arnavgrg
 Owner

@tunesh Yes, you need to load the base model with 4 bit quantization a T4 GPU before you wrap it with your trained PEFT weights.
arnavgrg changed discussion status to closed
arnavgrg
 Owner

I updated the model card: https://huggingface.co/arnavgrg/codealpaca-qlora

Sign up or log in to comment