MarcosEdu/Skyrim_D-Coder-V2-16B

Model Card for Model ID

This LoRA model is a product of the research paper "LLM QLoRA Fine-Tuning of Llama, DeepSeek, and Qwen: A Skyrim Case Study" (to appear in IEEE Access). The study investigates the interplay between model scale (~8B, ~13B, ~33B), architecture, and data formatting in adapting Large Language Models to knowledge-intensive domains.

By utilizing a multi-stage data cycling strategy and 4-bit Quantized Low-Rank Adaptation (QLoRA), this model was fine-tuned to master the lore of The Elder Scrolls V: Skyrim. The training process involved rigorous hyperparameter optimization, comparing unstructured, structured, and summarized datasets across varying LoRA ranks (16, 32, 64) to determine the optimal configuration for factual recall and narrative fluency. The resulting model demonstrates the capability to generate high-fidelity, hallucination-resistant character biographies and attribute lists, as validated by a comprehensive ensemble LLM-as-a-Judge evaluation framework.

Model Details

Model Description

• Skyrim Lore - DeepSeek-Coder-V2-16B
• Description: The top-performing model of the entire study. Despite being a "Coder" model, it achieved the highest Overall Score (3.16) when fine-tuned on the Summarized dataset with a LoRA Rank of 64. Its logic-heavy pre-training allowed it to disentangle dense narrative facts with superior accuracy.
• Best Configuration: Summarized Dataset | Rank 64
• Base Model: unsloth/DeepSeek-Coder-V2-Lite-Instruct
• Note: You can also use the pre-quantized version unsloth/DeepSeek-Coder-V2-Lite-Instruct-bnb-4bit.

Framework versions

• PEFT 0.15.2