Text Generation
Transformers
PyTorch
Chinese
English
llama
conversational
custom_code
text-generation-inference
Instructions to use openbmb/BitCPM4-CANN-1B-unquantized with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openbmb/BitCPM4-CANN-1B-unquantized with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="openbmb/BitCPM4-CANN-1B-unquantized", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("openbmb/BitCPM4-CANN-1B-unquantized", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("openbmb/BitCPM4-CANN-1B-unquantized", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use openbmb/BitCPM4-CANN-1B-unquantized with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "openbmb/BitCPM4-CANN-1B-unquantized" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/BitCPM4-CANN-1B-unquantized", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/openbmb/BitCPM4-CANN-1B-unquantized
- SGLang
How to use openbmb/BitCPM4-CANN-1B-unquantized with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "openbmb/BitCPM4-CANN-1B-unquantized" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/BitCPM4-CANN-1B-unquantized", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "openbmb/BitCPM4-CANN-1B-unquantized" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "openbmb/BitCPM4-CANN-1B-unquantized", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use openbmb/BitCPM4-CANN-1B-unquantized with Docker Model Runner:
docker model run hf.co/openbmb/BitCPM4-CANN-1B-unquantized
| step,train/loss,train/grad_norm,train/learning_rate,train/epoch,train/train_runtime,train/train_samples_per_second,train/train_steps_per_second,train/total_flos,train/train_loss | |
| 2,1.1492999792099,0.6216375231742859,1.9999999949504854e-06,0.0004617871018126607,,,,, | |
| 4,1.0979000329971313,0.681877851486206,3.999999989900971e-06,0.0009235742036253214,,,,, | |
| 6,1.1269999742507935,0.784303605556488,6.000000212225132e-06,0.001385361305437982,,,,, | |
| 8,1.0542000532150269,0.8737029433250427,7.999999979801942e-06,0.0018471484072506428,,,,, | |
| 10,1.2440999746322632,0.7068291902542114,9.999999747378752e-06,0.0023089356254786253,,,,, | |
| 12,1.2925000190734863,0.6821666955947876,1.2000000424450263e-05,0.002770722610875964,,,,, | |
| 14,1.0843000411987305,0.525643527507782,1.4000000192027073e-05,0.0032325098291039467,,,,, | |
| 16,1.0961999893188477,0.43757057189941406,1.5999999959603883e-05,0.0036942968145012856,,,,, | |
| 18,1.0614999532699585,0.46141618490219116,1.8000000636675395e-05,0.004156084265559912,,,,, | |
| 20,1.332900047302246,0.715879499912262,1.9999999494757503e-05,0.004617871250957251,,,,, | |
| 22,1.2070000171661377,0.5926885008811951,1.996917308133561e-05,0.0050796582363545895,,,,, | |
| 24,1.2043999433517456,0.5833240747451782,1.9876883015967906e-05,0.005541445221751928,,,,, | |
| 26,1.0740000009536743,0.44734400510787964,1.9723698642337695e-05,0.0060032326728105545,,,,, | |
| 28,1.1162999868392944,0.3701137900352478,1.9510565834934823e-05,0.006465019658207893,,,,, | |
| 30,1.0454000234603882,0.43832680583000183,1.9238796085119247e-05,0.006926806643605232,,,,, | |
| 32,1.124899983406067,0.4591037631034851,1.8910064682131633e-05,0.007388593629002571,,,,, | |
| 34,1.0686999559402466,0.3873400390148163,1.8526401618146338e-05,0.00785038061439991,,,,, | |
| 36,1.0291999578475952,0.40313437581062317,1.8090169760398567e-05,0.008312168531119823,,,,, | |
| 38,1.1052000522613525,0.3735405504703522,1.7604059394216165e-05,0.008773955516517162,,,,, | |
| 40,1.1555999517440796,0.3818407654762268,1.7071068214136176e-05,0.009235742501914501,,,,, | |
| 42,1.0235999822616577,0.4255191683769226,1.6494481315021403e-05,0.00969752948731184,,,,, | |
| 44,1.0364999771118164,0.4794503152370453,1.5877853002166376e-05,0.010159316472709179,,,,, | |
| 46,1.1344000101089478,0.37273937463760376,1.5224985872919206e-05,0.010621103458106518,,,,, | |
| 48,1.0866999626159668,0.417492538690567,1.453990535082994e-05,0.011082890443503857,,,,, | |
| 50,1.1038000583648682,0.35408055782318115,1.3826834219798911e-05,0.01154467836022377,,,,, | |
| 52,1.1478999853134155,0.3930828273296356,1.3090169886709191e-05,0.012006465345621109,,,,, | |
| 54,1.1858999729156494,0.3965947926044464,1.2334453458606731e-05,0.012468252331018448,,,,, | |
| 56,1.0096999406814575,0.3860221207141876,1.1564344276848715e-05,0.012930039316415787,,,,, | |
| 58,1.114799976348877,0.44393691420555115,1.0784590813273098e-05,0.013391826301813126,,,,, | |
| 60,1.079300045967102,0.3605058789253235,9.999999747378752e-06,0.013853613287210464,,,,, | |
| 62,1.1766999959945679,0.40689122676849365,9.215408681484405e-06,0.014315400272607803,,,,, | |
| 64,1.1075999736785889,0.4002344310283661,8.435655217908788e-06,0.014777187258005142,,,,, | |
| 66,1.1866999864578247,0.46947163343429565,7.665546036150772e-06,0.015238975174725056,,,,, | |
| 68,1.0311000347137451,0.3296957314014435,6.909830062795663e-06,0.01570076122879982,,,,, | |
| 70,1.1088999509811401,0.33858785033226013,6.173165729705943e-06,0.01616254821419716,,,,, | |
| 72,1.0720000267028809,0.3967427909374237,5.460095053422265e-06,0.016624337062239647,,,,, | |
| 74,1.1460000276565552,0.41202062368392944,4.7750145313329995e-06,0.017086124047636986,,,,, | |
| 76,1.0425000190734863,0.38334518671035767,4.1221474020858295e-06,0.017547911033034325,,,,, | |
| 78,0.9154000282287598,0.40649303793907166,3.505519543978153e-06,0.018009698018431664,,,,, | |
| 80,1.1110999584197998,0.35371580719947815,2.9289321901160292e-06,0.018471485003829002,,,,, | |
| 82,1.1672999858856201,0.3381657302379608,2.3959403279150138e-06,0.01893327198922634,,,,, | |
| 84,1.2374000549316406,0.3815234303474426,1.909829961732612e-06,0.01939505897462368,,,,, | |
| 86,1.2151000499725342,0.38446080684661865,1.4735983313585166e-06,0.01985684596002102,,,,, | |
| 88,1.163100004196167,0.40419140458106995,1.0899348126258701e-06,0.020318632945418358,,,,, | |
| 90,1.1883000135421753,0.4011874198913574,7.612046601934708e-07,0.020780419930815697,,,,, | |
| 92,1.1526999473571777,0.3836020231246948,4.894348535344761e-07,0.021242206916213036,,,,, | |
| 94,1.15339994430542,0.452364057302475,2.7630079557638965e-07,0.021703993901610374,,,,, | |
| 96,1.062000036239624,0.3502688705921173,1.2311659247643547e-07,0.022165780887007713,,,,, | |
| 98,1.0271999835968018,0.4022065997123718,3.0826662111849146e-08,0.022627567872405052,,,,, | |
| 100,1.0283000469207764,0.38241174817085266,0.0,0.02308935672044754,183.9481964111328,8.697999954223633,0.5440000295639038,1862467846144.0,1.1177252531051636 | |