stratosphere/Qwen3-Embedding-0.6B-netsecgame-finetuned-pairs

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B on the netsecgame-embedding-finetuning-pairs dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • netsecgame-embedding-finetuning-pairs

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("stratosphere/Qwen3-Embedding-0.6B-netsecgame-finetuned-pairs")
# Run inference
queries = [
    "Current status:\nControlled hosts are 10.7.44.3 and 10.7.44.5 and 10.7.44.4 and 54.123.53.21 and 10.7.44.2\nKnown networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24\nKnown hosts are 10.7.45.4 and 10.7.45.3 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2\nKnown services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and \nKnown services for host 10.7.44.5 are 22/tcp, ssh and bash and \nKnown services for host 54.123.53.21 are listener and bash and \nKnown services for host 10.7.44.4 are 22/tcp, ssh and bash and \nKnown services for host 10.7.44.3 are 3389/tcp, ms-wbt-server and \nKnown services for host 10.7.45.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and \nKnown data are none\n",
]
documents = [
    'Current status:\nControlled hosts are 172.19.2.2 and 101.32.5.23 and 172.19.1.4 and 172.19.1.3 and 172.19.1.2\nKnown networks are 54.123.53.29/26 and 172.19.0.0/24 and 172.19.1.0/24 and 172.19.2.0/24\nKnown hosts are 172.19.2.4 and 172.19.2.3 and 172.19.1.5 and 172.19.1.1 and 172.19.1.6\nKnown services for host 172.19.1.4 are bash and 22/tcp, ssh and \nKnown services for host 172.19.1.3 are 3389/tcp, ms-wbt-server and \nKnown services for host 101.32.5.23 are bash and listener and \nKnown services for host 172.19.1.5 are 22/tcp, ssh and \nKnown services for host 172.19.2.3 are 22/tcp, ssh and 5432/tcp, postgresql and \nKnown services for host 172.19.1.2 are 3389/tcp, ms-wbt-server and \nKnown services for host 172.19.2.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and \nKnown data are none\n',
    'Current status:\nControlled hosts are 172.20.7.6 and 32.8.23.1\nKnown networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26\nKnown hosts are 172.20.7.2 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4\nKnown services for host 172.20.8.2 are 445/tcp, microsoft-ds and 3389/tcp, ms-wbt-server and \nKnown services for host 172.20.8.4 are 22/tcp, ssh and 80/tcp, http and \nKnown services for host 172.20.7.6 are bash and \nKnown data are none\n',
    'Current status:\nControlled hosts are 172.20.8.3 and 172.20.7.2 and 32.8.23.1 and 172.20.8.4\nKnown networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26\nKnown hosts are 172.20.7.6 and 172.20.7.4 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5\nKnown services for host 32.8.23.1 are listener and bash and \nKnown services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and \nKnown services for host 172.20.8.4 are 22/tcp, ssh and bash and 80/tcp, http and \nKnown services for host 172.20.8.3 are 22/tcp, ssh and 5432/tcp, postgresql and \nKnown data for host 172.20.8.4 are (User2, WebServerData) and \n',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.5508, 0.4785, 0.4883]], dtype=torch.bfloat16)

Evaluation

Metrics

Binary Classification

• Dataset: topology_val
• Evaluated with BinaryClassificationEvaluator

Metric	Value
cosine_accuracy	0.9147
cosine_accuracy_threshold	0.9851
cosine_f1	0.9104
cosine_f1_threshold	0.985
cosine_precision	0.9475
cosine_recall	0.8761
cosine_ap	0.9741
cosine_mcc	0.8316

Training Details

Training Dataset

netsecgame-embedding-finetuning-pairs

• Dataset: netsecgame-embedding-finetuning-pairs at 1a27835
• Size: 21,011 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	float
  details	min: 111 tokens mean: 338.8 tokens max: 577 tokens	min: 111 tokens mean: 339.66 tokens max: 658 tokens	min: 0.0 mean: 0.5 max: 1.0

• Samples:

  sentence1	sentence2	label
  Current status: Controlled hosts are 32.8.23.1 and 172.20.7.3 Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26 Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4 Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.2 are 3389/tcp, ms-wbt-server and Known services for host 172.20.7.5 are 22/tcp, ssh and Known data are none	Current status: Controlled hosts are 32.8.23.1 and 172.20.7.5 Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26 Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.8.4 Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 172.20.7.4 are 22/tcp, ssh and Known data are none	1.0
  Current status: Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2 and 192.168.2.4 Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24 Known hosts are 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2 Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.2 are 3389/tcp, ms-wbt-server and Known data are none	Current status: Controlled hosts are 10.7.45.4 and 10.7.44.2 and 10.7.44.4 and 54.123.53.21 Known networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24 Known hosts are 10.7.44.3 and 10.7.45.3 and 10.7.44.5 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2 Known services for host 54.123.53.21 are listener and bash and Known services for host 10.7.44.4 are 22/tcp, ssh and Known services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 10.7.45.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0
  Current status: Controlled hosts are 172.20.7.2 and 32.8.23.1 Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26 Known hosts are 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4 Known services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known data are none	Current status: Controlled hosts are 32.8.23.1 and 172.20.7.3 Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26 Known hosts are 172.20.7.2 and 172.20.7.6 and 172.20.7.4 and 172.20.8.3 and 172.20.8.2 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4 Known services for host 172.20.7.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.2 are 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known data are none	1.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Evaluation Dataset

netsecgame-embedding-finetuning-pairs

• Dataset: netsecgame-embedding-finetuning-pairs at 1a27835
• Size: 2,626 evaluation samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	label
  type	string	string	float
  details	min: 147 tokens mean: 338.35 tokens max: 567 tokens	min: 147 tokens mean: 339.84 tokens max: 599 tokens	min: 0.0 mean: 0.51 max: 1.0

• Samples:

  sentence1	sentence2	label
  Current status: Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2 Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24 Known hosts are 192.168.2.4 and 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2 Known services for host 192.168.2.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.3 are 3389/tcp, ms-wbt-server and Known data are none	Current status: Controlled hosts are 213.47.23.195 and 192.168.2.3 and 192.168.2.2 Known networks are 192.168.2.0/24 and 213.47.23.192/26 and 192.168.3.0/24 and 192.168.1.0/24 Known hosts are 192.168.2.4 and 192.168.1.3 and 192.168.2.6 and 192.168.2.5 and 192.168.2.1 and 192.168.1.4 and 192.168.1.2 Known services for host 192.168.2.3 are powershell and 3389/tcp, ms-wbt-server and Known services for host 213.47.23.195 are listener and bash and Known services for host 192.168.2.5 are 22/tcp, ssh and Known services for host 192.168.2.4 are 22/tcp, ssh and Known services for host 192.168.2.2 are 3389/tcp, ms-wbt-server and Known data are none	1.0
  Current status: Controlled hosts are 10.7.45.4 and 10.7.44.2 and 10.7.44.4 and 54.123.53.21 Known networks are 10.7.45.0/24 and 54.123.53.29/26 and 10.7.44.0/24 and 10.7.43.0/24 Known hosts are 10.7.44.3 and 10.7.45.3 and 10.7.44.5 and 10.7.44.6 and 10.7.44.1 and 10.7.45.2 Known services for host 54.123.53.21 are listener and bash and Known services for host 10.7.44.4 are 22/tcp, ssh and Known services for host 10.7.44.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 10.7.45.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	Current status: Controlled hosts are 172.19.1.2 and 101.32.5.23 Known networks are 54.123.53.29/26 and 172.19.0.0/24 and 172.19.1.0/24 and 172.19.2.0/24 Known hosts are 172.19.2.2 and 172.19.1.4 and 172.19.2.4 and 172.19.2.3 and 172.19.1.5 and 172.19.1.1 and 172.19.1.3 and 172.19.1.6 Known services for host 101.32.5.23 are bash and listener and Known services for host 172.19.1.4 are 22/tcp, ssh and Known services for host 172.19.1.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.19.2.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0
  Current status: Controlled hosts are 172.20.7.4 and 172.20.7.2 and 32.8.23.1 and 172.20.8.2 Known networks are 172.20.8.0/24 and 172.20.6.0/24 and 172.20.7.0/24 and 32.8.23.1/26 Known hosts are 172.20.7.6 and 172.20.8.3 and 172.20.7.3 and 172.20.7.1 and 172.20.7.5 and 172.20.8.4 Known services for host 172.20.7.2 are powershell and 3389/tcp, ms-wbt-server and Known services for host 172.20.7.4 are 22/tcp, ssh and Known services for host 172.20.7.3 are 3389/tcp, ms-wbt-server and Known services for host 172.20.8.2 are 445/tcp, microsoft-ds and 3389/tcp, ms-wbt-server and Known services for host 32.8.23.1 are listener and bash and Known services for host 172.20.7.5 are 22/tcp, ssh and Known services for host 172.20.8.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	Current status: Controlled hosts are 10.0.47.2 and 10.0.46.6 and 55.34.2.5 and 10.0.47.4 Known networks are 10.0.46.0/24 and 10.0.45.0/24 and 55.34.2.4/26 and 10.0.47.0/24 Known hosts are 10.0.46.1 and 10.0.47.3 and 10.0.46.2 and 10.0.46.5 and 10.0.46.4 and 10.0.46.3 Known services for host 55.34.2.5 are bash and listener and Known services for host 10.0.46.5 are 22/tcp, ssh and Known services for host 10.0.47.2 are 3389/tcp, ms-wbt-server and 445/tcp, microsoft-ds and Known services for host 10.0.46.6 are bash and Known services for host 10.0.47.3 are 22/tcp, ssh and 5432/tcp, postgresql and Known services for host 10.0.46.3 are 3389/tcp, ms-wbt-server and Known services for host 10.0.47.4 are 22/tcp, ssh and 80/tcp, http and Known data are none	0.0

• Loss: ContrastiveLoss with these parameters:

  {
      "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE",
      "margin": 0.5,
      "size_average": true
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• gradient_accumulation_steps: 4
• learning_rate: 1e-06
• weight_decay: 0.01
• num_train_epochs: 1
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 8
• per_device_eval_batch_size: 8
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 1e-06
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	topology_val_cosine_ap
0.0152	10	0.0446	-	-
0.0305	20	0.0403	-	-
0.0457	30	0.0391	-	-
0.0609	40	0.0387	-	-
0.0761	50	0.0385	-	-
0.0914	60	0.0334	-	-
0.1066	70	0.0354	-	-
0.1218	80	0.0325	-	-
0.1370	90	0.0309	-	-
0.1523	100	0.0245	-	-
0.1675	110	0.0292	-	-
0.1827	120	0.0275	-	-
0.1979	130	0.0260	-	-
0.2132	140	0.0229	-	-
0.2284	150	0.0226	-	-
0.2436	160	0.0227	-	-
0.2589	170	0.0230	-	-
0.2741	180	0.0238	-	-
0.2893	190	0.0248	-	-
0.3045	200	0.0254	-	-
0.3198	210	0.0207	-	-
0.3350	220	0.0237	-	-
0.3502	230	0.0234	-	-
0.3654	240	0.0181	-	-
0.3807	250	0.0201	-	-
0.3959	260	0.0216	-	-
0.4111	270	0.0183	-	-
0.4263	280	0.0203	-	-
0.4416	290	0.0196	-	-
0.4568	300	0.0179	-	-
0.4720	310	0.0170	-	-
0.4872	320	0.0209	-	-
0.5025	330	0.0204	-	-
0.5177	340	0.0204	-	-
0.5329	350	0.0232	-	-
0.5482	360	0.0213	-	-
0.5634	370	0.0214	-	-
0.5786	380	0.0212	-	-
0.5938	390	0.0188	-	-
0.6091	400	0.0165	-	-
0.6243	410	0.0208	-	-
0.6395	420	0.0203	-	-
0.6547	430	0.0193	-	-
0.6700	440	0.0236	-	-
0.6852	450	0.0191	-	-
0.7004	460	0.0207	-	-
0.7156	470	0.0215	-	-
0.7309	480	0.0253	-	-
0.7461	490	0.0189	-	-
0.7613	500	0.0232	0.0195	0.9741
0.7766	510	0.0204	-	-
0.7918	520	0.0174	-	-
0.8070	530	0.0163	-	-
0.8222	540	0.0173	-	-
0.8375	550	0.0199	-	-
0.8527	560	0.0198	-	-
0.8679	570	0.0213	-	-
0.8831	580	0.0180	-	-
0.8984	590	0.0179	-	-
0.9136	600	0.0193	-	-
0.9288	610	0.0240	-	-
0.9440	620	0.0203	-	-
0.9593	630	0.0203	-	-
0.9745	640	0.0194	-	-
0.9897	650	0.0214	-	-

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 5.2.2
• Transformers: 5.0.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}