Instructions to use LLM360/K2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LLM360/K2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LLM360/K2")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("LLM360/K2") model = AutoModelForCausalLM.from_pretrained("LLM360/K2") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use LLM360/K2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LLM360/K2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LLM360/K2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/LLM360/K2
- SGLang
How to use LLM360/K2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LLM360/K2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LLM360/K2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LLM360/K2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LLM360/K2", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use LLM360/K2 with Docker Model Runner:
docker model run hf.co/LLM360/K2
Upload folder using huggingface_hub
Browse files- pytorch_model-00001-of-00027.bin +1 -1
- pytorch_model-00002-of-00027.bin +1 -1
- pytorch_model-00003-of-00027.bin +1 -1
- pytorch_model-00004-of-00027.bin +1 -1
- pytorch_model-00005-of-00027.bin +1 -1
- pytorch_model-00006-of-00027.bin +1 -1
- pytorch_model-00007-of-00027.bin +1 -1
- pytorch_model-00008-of-00027.bin +1 -1
- pytorch_model-00009-of-00027.bin +1 -1
- pytorch_model-00010-of-00027.bin +1 -1
- pytorch_model-00011-of-00027.bin +1 -1
- pytorch_model-00012-of-00027.bin +1 -1
- pytorch_model-00013-of-00027.bin +1 -1
- pytorch_model-00014-of-00027.bin +1 -1
- pytorch_model-00015-of-00027.bin +1 -1
- pytorch_model-00016-of-00027.bin +1 -1
- pytorch_model-00017-of-00027.bin +1 -1
- pytorch_model-00018-of-00027.bin +1 -1
- pytorch_model-00019-of-00027.bin +1 -1
- pytorch_model-00020-of-00027.bin +1 -1
- pytorch_model-00021-of-00027.bin +1 -1
- pytorch_model-00022-of-00027.bin +1 -1
- pytorch_model-00023-of-00027.bin +1 -1
- pytorch_model-00024-of-00027.bin +1 -1
- pytorch_model-00025-of-00027.bin +1 -1
- pytorch_model-00026-of-00027.bin +1 -1
- pytorch_model-00027-of-00027.bin +1 -1
pytorch_model-00001-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4660470238
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:07e1133b87b1ecefef3389a8afa77fe3b7f595c2c95a5ec9027eecea863c27fe
|
| 3 |
size 4660470238
|
pytorch_model-00002-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5cdc52c599a3ee9c5d8bcebf26e703ec26f06998970e7840aa66bceee3a6325a
|
| 3 |
size 4857111932
|
pytorch_model-00003-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:40844a0b4c83ea49c9e17ffd80c3c9ffadedc711f4d090020447e27982473bf7
|
| 3 |
size 4857111932
|
pytorch_model-00004-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d8b4e052c23448a0cc545544cafa2f51b5b4ea2517773be28dc43148630c470e
|
| 3 |
size 4857111932
|
pytorch_model-00005-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc5e0afd4bfce69ed3734cbdbcb9eaa0907b8ef80c56ae91c58e253ae0612f3e
|
| 3 |
size 4857111932
|
pytorch_model-00006-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef97b25dfe3a9e2d372553af11f93a9ed68137b9d1c15a22d3e6733bae42047a
|
| 3 |
size 4857111932
|
pytorch_model-00007-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9a852d484b9575c606c3f6584e7daf477d7ae4be54ebf045642e0f1a98059593
|
| 3 |
size 4857111932
|
pytorch_model-00008-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b6830fcabd111726878f0458176f8bdadb3e7c1e60e4038494aa0c175a4a9d5c
|
| 3 |
size 4857111932
|
pytorch_model-00009-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1e234d334e15bf31aef54673fce0df4757fd2ecf2142a61effefc7c7e6456298
|
| 3 |
size 4857111932
|
pytorch_model-00010-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:206388aad41df726aba00590ea33ba16fae1ea14a499e92a1f26df9ab17a2249
|
| 3 |
size 4857111932
|
pytorch_model-00011-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:eaaac28878b43d67c832b27ac3ea0127d0de3841aa4775d4624062127c58e740
|
| 3 |
size 4857111932
|
pytorch_model-00012-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:92f0111d9b050144090338aa5d39b51be6d22fe05ab6dbfae0ffe3c47b5c01c7
|
| 3 |
size 4857111932
|
pytorch_model-00013-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0f339c0837b10d6a6f9c75a149f3cae913b8276954e0a0f4f845d967712a3d15
|
| 3 |
size 4857111932
|
pytorch_model-00014-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:83ad4a649a04a21790410d8fa70f09b37bc226ef74f8fd3cdca059112a60426b
|
| 3 |
size 4857111932
|
pytorch_model-00015-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1b373afd1751be90652d093267371b32fee6c6155f3c7e078249fcb55e3290e6
|
| 3 |
size 4857111932
|
pytorch_model-00016-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ea013d895746618b80788a86b7bc5b323e7a088cef9f5ae8868b55414375567
|
| 3 |
size 4857111932
|
pytorch_model-00017-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a76aba95da51b2124061731c5658f5a83ace893f9d5b0d7f3161c0e70b73e223
|
| 3 |
size 4857111932
|
pytorch_model-00018-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ca2cb9dcdba05126d9d06bd52f40a24bc15a716f89295f8dff50b8341bda525
|
| 3 |
size 4857111932
|
pytorch_model-00019-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:368788b5a8ef4833dfb3cb13145a51a2e4f02018d86ea7a1c478b478b4cf706a
|
| 3 |
size 4857111932
|
pytorch_model-00020-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0f6b75a5d7f449d63ed15f12f7df7cef09756bf8de8d338ad1e25e6ee4184dd0
|
| 3 |
size 4857111932
|
pytorch_model-00021-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cc20066a4be7a54fe0ace099cdcdbe47c291e685e558af86bb2f41c83bdd458c
|
| 3 |
size 4857111932
|
pytorch_model-00022-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c8d4faf00a017401a3147a646391b8ffc01f36a20e9ba50dbc93c039a2af6ba
|
| 3 |
size 4857111932
|
pytorch_model-00023-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5ec41825ba4f3f81c4d8fe5fc4746a7b23b9b141c924cc9997f35d3fc56959e3
|
| 3 |
size 4857111932
|
pytorch_model-00024-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:19fdb458b78c55ed8964314203efe93e034ca18cb5beed26191254edf6cfc8c1
|
| 3 |
size 4857111932
|
pytorch_model-00025-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:14ad53c72092faa489806c97e2159fcb7c01ad4ba2c8715b70e8443fe32c8191
|
| 3 |
size 4857111932
|
pytorch_model-00026-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4857111932
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7152df0b2328de111af1c760bc40e5cfa1dcf66ec59aefb8aa499003b9ffb96e
|
| 3 |
size 4857111932
|
pytorch_model-00027-of-00027.bin
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 4484358522
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:03fd3321f277e6cced14ed36b95f923180fec60d2bb5ddc9a97be2ea4691366b
|
| 3 |
size 4484358522
|