CARDS-Qwen3.5-9B-GGUF

GGUF builds of C3DS/CARDS-Qwen3.5-9B — Qwen3.5-9B fine-tuned for climate-contrarian-claim classification under the CARDS taxonomy from Coan et al. (2025).

For accuracy comparisons against the BF16 sibling, larger Qwen3 variants, Claude Opus 4.6 / 4.7 and GPT-5.5, see the BF16 model card. Headline number on the held-out CARDS test set: L1 samples F1 = 0.872 (0 / 1,436 parse failures).

Available files

File	Quantization	Approx. size	Use
Qwen3.5-9B.Q4_K_M.gguf	Q4_K_M (4-bit, mixed)	~5.6 GB	smallest; runs on consumer GPUs / Apple Silicon
Qwen3.5-9B.Q8_0.gguf	Q8_0 (8-bit)	~9.5 GB	near-BF16 quality
Qwen3.5-9B.BF16-mmproj.gguf	BF16 mmproj	—	multimodal projector (vision encoder); pair with either weight file for image input

The Q4_K_M and Q8_0 files are alternatives — pick one based on your accuracy / memory trade-off. The mmproj file is additional and only needed for image inputs.

Usage

With Ollama

A pre-published Ollama version is available at exec3ds/cards-qwen3.5. Pull and run the 9B (Q4_K_M, ~5.6 GB):

ollama run exec3ds/cards-qwen3.5:9b

The same namespace also publishes :4b and :27b tags. The bundled chat template is applied automatically — Qwen3's <think> block precedes the YAML categories: output.

Image input works too — pass an image path on the command line:

ollama run exec3ds/cards-qwen3.5:9b ./image.webp \
  "What climate claim is being made in this image?"

Text-only with llama.cpp

llama-cli \
  -hf C3DS/CARDS-Qwen3.5-9B-GGUF:Q4_K_M \
  --jinja \
  -p "Classify the following climate claim under the CARDS taxonomy:\nThese are only a few renewable energy technologies at work."

The --jinja flag tells llama-cli to use the bundled chat template, which adds Qwen3's <think> reasoning trace plus the YAML categories: block at the end. Parse the content after </think> to extract the predicted CARDS codes.

For an OpenAI-compatible local server, use llama-server:

llama-server \
  -hf C3DS/CARDS-Qwen3.5-9B-GGUF:Q4_K_M \
  --jinja \
  --port 8000

Multimodal — image + text

The base Qwen3.5 architecture (Qwen3_5ForConditionalGeneration) is multimodal. Pair the weight file with the BF16-mmproj.gguf projector and use llama-mtmd-cli:

llama-mtmd-cli \
  -hf C3DS/CARDS-Qwen3.5-9B-GGUF:Q4_K_M \
  --mmproj-hf C3DS/CARDS-Qwen3.5-9B-GGUF/Qwen3.5-9B.BF16-mmproj.gguf \
  --jinja \
  --image screenshot.png \
  -p "Read the image (and any caption) and classify the climate claim under the CARDS taxonomy."

For the canonical CARDS system prompt and CoT trigger, fetch cards_prompts.json from the BF16 sibling repo and pass slim_system_instruction as the system prompt + cot_trigger at the end of the user turn.

Conversion details

• Source checkpoint: C3DS/CARDS-Qwen3.5-9B (LoRA-merged Qwen3.5-9B)
• Tool: Unsloth → llama.cpp converter
• Quantizations included: Q4_K_M, Q8_0
• Multimodal projector: BF16 (kept full-precision for accuracy on vision inputs)

Limitations

• Thinking tokens. Training used enable_thinking=True. Either parse output after </think>, or disable thinking in your sampler / system prompt. Reserve token budget for the reasoning trace before the final YAML block.
• Quantization trade-offs. Q4_K_M is the most aggressive; expect a modest drop from BF16 in the long-tailed L3 macro-F1 metrics. Q8_0 tracks BF16 more closely.
• Multimodal use is opportunistic. Training was text-only; image-input behavior comes from the preserved base vision encoder. Performance on image inputs has not been quantitatively benchmarked.

Citation

@article{coan2025cards,
  title   = {Large language model reveals an increase in climate contrarian speech in the United States Congress},
  author  = {Coan, Travis G. and Malla, Ranadheer and Nanko, Mirjam O. and Kattrup, William and Roberts, J. Timmons and Cook, John and Boussalis, Constantine},
  journal = {Communications Sustainability},
  volume  = {1},
  pages   = {37},
  year    = {2025},
  doi     = {10.1038/s44458-025-00029-z}
}

License

Apache 2.0, inherited from Qwen3.5-9B.