Instructions to use Jinx-org/Jinx-gpt-oss-20b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Jinx-org/Jinx-gpt-oss-20b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Jinx-org/Jinx-gpt-oss-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Jinx-org/Jinx-gpt-oss-20b")
model = AutoModelForCausalLM.from_pretrained("Jinx-org/Jinx-gpt-oss-20b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Jinx-org/Jinx-gpt-oss-20b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Jinx-org/Jinx-gpt-oss-20b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jinx-org/Jinx-gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Jinx-org/Jinx-gpt-oss-20b

SGLang

How to use Jinx-org/Jinx-gpt-oss-20b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Jinx-org/Jinx-gpt-oss-20b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jinx-org/Jinx-gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Jinx-org/Jinx-gpt-oss-20b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Jinx-org/Jinx-gpt-oss-20b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Jinx-org/Jinx-gpt-oss-20b with Docker Model Runner:
```
docker model run hf.co/Jinx-org/Jinx-gpt-oss-20b
```

Tool Calling

by jpoudrier - opened Aug 15, 2025

Discussion

jpoudrier

Aug 15, 2025

•

edited Aug 15, 2025

Any fixes for tool calling or is that host side? Doesn’t seem to work in n8n, LM Studio or Jan AI. Chat template for tool calling seems to still be in the metadata. Custom thinking option also seem to be missing.

Jeol

Jinx org Aug 15, 2025

•

edited Aug 15, 2025

Thanks for your feedback!

We don't modify the tool calling settings - they should remain at default. However, we haven't verified that tool calling works properly with our model. For a quick solution, you can rebuild your GGUF model file with --override-kv to override metadata. See documentation here.

The --override-kv option allows you to override model metadata by key in the quantized model and can be specified multiple times.

Can you provide specific instructions to reproduce your issue? This would help us debug what settings might be wrong. For example:

What tool calling format are you expecting?
What error messages are you seeing?
What's your current configuration in n8n/LM Studio/Jan AI?

Regarding the "custom thinking option" - can you clarify what you mean by this? It would help us understand if this is related to the tool calling issue or a separate concern.

Best,
Jinx Team

jpoudrier

Aug 30, 2025

I think an update to LMSTUDIO ultimately fixed it. Sorry for that!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment