Instructions to use ethanker/lfm2_350m_commit_diff_summarizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ethanker/lfm2_350m_commit_diff_summarizer with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/LFM2-350M-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "ethanker/lfm2_350m_commit_diff_summarizer")

Transformers

How to use ethanker/lfm2_350m_commit_diff_summarizer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ethanker/lfm2_350m_commit_diff_summarizer")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("ethanker/lfm2_350m_commit_diff_summarizer", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use ethanker/lfm2_350m_commit_diff_summarizer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ethanker/lfm2_350m_commit_diff_summarizer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ethanker/lfm2_350m_commit_diff_summarizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ethanker/lfm2_350m_commit_diff_summarizer

SGLang

How to use ethanker/lfm2_350m_commit_diff_summarizer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ethanker/lfm2_350m_commit_diff_summarizer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ethanker/lfm2_350m_commit_diff_summarizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ethanker/lfm2_350m_commit_diff_summarizer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ethanker/lfm2_350m_commit_diff_summarizer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ethanker/lfm2_350m_commit_diff_summarizer with Docker Model Runner:
```
docker model run hf.co/ethanker/lfm2_350m_commit_diff_summarizer
```

lfm2_350m_commit_diff_summarizer / README.md

ethanker

Update README.md

bef6d23 verified 7 months ago

preview code

raw

history blame contribute delete

4.49 kB

	---
	base_model: unsloth/LFM2-350M-unsloth-bnb-4bit
	library_name: peft
	pipeline_tag: text-generation
	tags:
	- "base_model:adapter:unsloth/LFM2-350M-unsloth-bnb-4bit"
	- lora
	- qlora
	- sft
	- transformers
	- trl
	- conventional-commits
	- code
	---


	# lfm2_350m_commit_diff_summarizer (LoRA)

	A lightweight helper model that turns Git diffs into Conventional Commit–style messages.
	It outputs strict JSON with a short `title` (≤ 65 chars) and up to 3 `bullets`, so your CLI/agents can parse it deterministically.

	## Model Details

	### Model Description

	* Purpose: Summarize `git diff` patches into concise, Conventional Commit–compliant titles with optional bullets.
	* I/O format:

	* Input: prompt containing the diff (plain text).
	* Output: JSON object: `{"title": "...", "bullets": ["...", "..."]}`.
	* Model type: LoRA adapter for causal LM (text generation)
	* Language(s): English (commit message conventions)
	* Finetuned from: `unsloth/LFM2-350M-unsloth-bnb-4bit` (4-bit quantized base, trained with QLoRA)

	### Model Sources

	* Repository: This model card + adapter on the Hub under `ethanke/lfm2_350m_commit_diff_summarizer`

	## Uses

	### Direct Use

	* Convert patch diffs into Conventional Commit messages for PR titles, commits, and changelogs.
	* Provide human-readable summaries in agent UIs with guaranteed JSON structure.

	### Recommendations

	* Enforce JSON validation; if invalid, retry with a JSON-repair prompt.
	* Keep a regex gate for Conventional Commit titles in your pipeline.

	## How to Get Started

	```python
	from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
	from peft import PeftModel
	import torch, json

	BASE = "unsloth/LFM2-350M-unsloth-bnb-4bit"
	ADAPTER = "ethanke/lfm2_350m_commit_diff_summarizer" # replace with your repo id

	bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
	bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16)

	tok = AutoTokenizer.from_pretrained(BASE, use_fast=True)
	mdl = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
	mdl = PeftModel.from_pretrained(mdl, ADAPTER)

	diff = "...your git diff text..."
	prompt = (
	"You are a commit message summarizer.\n"
	"Return a concise JSON object with fields 'title' (<=65 chars) and 'bullets' (0-3 items).\n"
	"Follow the Conventional Commit style for the title.\n\n"
	"### DIFF\n" + diff + "\n\n### OUTPUT JSON\n"
	)

	inputs = tok(prompt, return_tensors="pt").to(mdl.device)
	with torch.no_grad():
	out = mdl.generate(**inputs, max_new_tokens=200, do_sample=False)
	text = tok.decode(out[0], skip_special_tokens=True)

	# naive JSON extraction
	js = text[text.rfind("{"): text.rfind("}")+1]
	obj = json.loads(js)
	print(obj)
	```

	## Training Details

	### Training Data

	* Dataset: `Maxscha/commitbench` (diff → commit message).
	* Filtering: kept only samples whose first non-empty line of the message matches Conventional Commits:
	`^(feat\|fix\|docs\|style\|refactor\|perf\|test\|build\|ci\|chore\|revert)($[^)]+$)?(!)?:\s.+$`
	* Note: The dataset card indicates non-commercial licensing. Confirm before commercial deployment.

	### Training Procedure

	* Method: Supervised fine-tuning (SFT) with TRL `SFTTrainer` + QLoRA (PEFT).
	* Prompting: Instruction + `### DIFF` + `### OUTPUT JSON` target (title/bullets).
	* Precision: fp16 compute on 4-bit base.
	* Hyperparameters (v0.1):

	* `max_length=2048`, `per_device_train_batch_size=2`, `grad_accum=4`
	* `lr=2e-4`, `scheduler=cosine`, `warmup_ratio=0.03`
	* `epochs=1` over capped subset
	* LoRA: `r=16`, `alpha=32`, `dropout=0.05`, targets: q/k/v/o + MLP proj

	### Evaluation

	* Validation: filtered split from CommitBench.
	* Metrics (example run):

	* `eval_loss ≈ 1.18` → perplexity ≈ 3.26
	* `eval_mean_token_accuracy ≈ 0.77`
	* Suggested task metrics: JSON validity rate, CC-title compliance, title length ≤ 65 chars, bullets ≤ 3.

	## Environmental Impact

	* Hardware: 1× NVIDIA GTX 3060 12 GB (local)
	* Hours used: ~2 h (prototype)

	## Technical Specifications

	* Architecture: LFM2-350M (decoder-only) + LoRA adapter
	* Libraries: `transformers`, `trl`, `peft`, `bitsandbytes`, `datasets`, `unsloth`

	## Contact

	* Open an issue on the Hub repo or message `ethanke` on Hugging Face.

	### Framework versions

	* PEFT 0.17.1
	* TRL (SFTTrainer)
	* Transformers (recent version)