Instructions to use BAAI/JudgeLM-7B-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use BAAI/JudgeLM-7B-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="BAAI/JudgeLM-7B-v1.0")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("BAAI/JudgeLM-7B-v1.0")
model = AutoModelForCausalLM.from_pretrained("BAAI/JudgeLM-7B-v1.0")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use BAAI/JudgeLM-7B-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "BAAI/JudgeLM-7B-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BAAI/JudgeLM-7B-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/BAAI/JudgeLM-7B-v1.0

SGLang

How to use BAAI/JudgeLM-7B-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "BAAI/JudgeLM-7B-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BAAI/JudgeLM-7B-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "BAAI/JudgeLM-7B-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BAAI/JudgeLM-7B-v1.0",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use BAAI/JudgeLM-7B-v1.0 with Docker Model Runner:
```
docker model run hf.co/BAAI/JudgeLM-7B-v1.0
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

JudgeLM Model Card

Model Details

JudgeLM is a judge model trained by fine-tuning Vicuna on JudgeLM-100K dataset.

Developed by: HUST, BAAI
Model type: An auto-regressive language model based on the transformer architecture.
License: Non-commercial license
Finetuned from model: Vicuna.

Model Sources

Repository: https://github.com/baaivision/JudgeLM
Paper: https://arxiv.org/abs/2310.17631
Demo: http://218.91.113.230:9004/

Uses

The primary use of JudgeLM is research on evaluating the performance of large language models and chatbots. The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.

How to Get Started with the Model

Judge large language models with this model: https://github.com/baaivision/JudgeLM/tree/main/judgelm/llm_judge.
Serve this model with the gradio: https://github.com/baaivision/JudgeLM/tree/main/judgelm/serve.

Training Details

JudgeLM v1.0 is fine-tuned from Vicuna-v1.3 with supervised instruction fine-tuning. The training data is around 200K judge samples from JudgeLM-100K dataset. See more details in the "Fine-tuning Settings" section in the appendix of this paper.

Evaluation

JudgeLM is evaluated on JudgeLM val set, with judgements produced by GPT-4 teacher. See more details in this paper and try it with code.

Additional Information

Citation Information

@article{zhu2023judgelm,  
    title={JudgeLM: Fine-tuned Large Language Models are Scalable Judges},  
    author={Lianghui Zhu and Xinggang Wang and Xinlong Wang},  
    year={2023},  
    eprint={2310.17631},  
    archivePrefix={arXiv},  
    primaryClass={cs.CL}  
}

Downloads last month: 1,122

Model tree for BAAI/JudgeLM-7B-v1.0

Merges

2 models

Quantizations

1 model

Paper for BAAI/JudgeLM-7B-v1.0

JudgeLM: Fine-tuned Large Language Models are Scalable Judges

Paper • 2310.17631 • Published Oct 26, 2023 • 35