v10: unified set_lights, named-args output, 6 tools

- Replace 3 LED tools (set_status_led / blink_status_led / set_neopixel_effect) with one hardware-agnostic set_lights(color?, effect?, state?). Dispatcher resolves to HAT 3-LED indicators or WLED strip/ring at runtime.
- Switch tool-call output to named-args format per Mercedes-Benz Octopus v2 (arXiv 2501.02342): <tool_0>(color=\"red\", state=\"on\")<end>. Optional args absent when user didn't imply them.
- Tool surface: set_lights, play_buzzer, set_alarm, cancel_alarm, get_system_status, respond.
- Training: 5,222 train / 920 eval. Final eval loss 0.046, mean token acc 97.9%. Held-out smoke 35/36 (97.2%).
- On-device (Coralboard, 2-core A55, Q5_K_M): cold prefill 0.48 s, decode ~9.7 tok/s, end-to-end first turn ~3.4 s.

.gitattributes

@@ -43,3 +43,4 @@ functiongemma-physical-ai-v7-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 moonshine/decoder.vmfb filter=lfs diff=lfs merge=lfs -text
 moonshine/decoder_with_past.vmfb filter=lfs diff=lfs merge=lfs -text
 functiongemma-physical-ai-v9-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+functiongemma-physical-ai-v10-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -1,20 +1,15 @@
-# FunctionGemma 270M Physical AI — v7, function-token format
-# Function tokens (<tool_N>) + <end> terminator. ~8-15 output tokens per call.
-# Optimized for CPU decode on small Cortex-A55 / similar edge targets.
-
-FROM ./functiongemma-physical-ai-v7-Q5_K_M.gguf
+# Coral FunctionGemma v10 — 6-tool schema, Octopus v2, named-args format
+# set_lights unified (color/effect/state). respond = <tool_5>.
+FROM ./functiongemma-physical-ai-v10-Q5_K_M.gguf
 
 PARAMETER temperature 0
 PARAMETER top_p 1
 PARAMETER num_ctx 1024
 PARAMETER num_predict 80
 
-# Stop on the turn-level markers ONLY, not on <end>. Multi-tool sequences
-# emit <tool_A>(args)<end><tool_B>(args)<end>, and stopping at the first
-# <end> truncates the second call. <end_of_turn> + <eos> are the right
-# stops for both single- and multi-tool output.
+# Do NOT stop on <end> — that terminator marks the end of one tool call,
+# but multi-tool sequences emit `<tool_A>(args)<end><tool_B>(args)<end>` and
+# stopping at the first <end> truncates legitimate multi-tool output.
+# <end_of_turn> + <eos> mark turn-level completion.
 PARAMETER stop "<end_of_turn>"
 PARAMETER stop "<eos>"
-
-# Use base model's chat template — training data is in messages+tools form,
-# the tokenizer's chat_template.jinja already handles it.

README.md

@@ -17,77 +17,85 @@ pipeline_tag: text-generation
 inference: false
 ---
 
-# FunctionGemma 270M — Physical AI (v9, Octopus v2)
+# FunctionGemma 270M — Physical AI (v10, Octopus v2)
 
 Fine-tuned [`google/functiongemma-270m-it`](https://huggingface.co/google/functiongemma-270m-it)
 for voice-controlled physical-AI / household-IoT actions on a Synaptics
 SL2619 "Coral" edge board (Google IO 2026 demo).
 
-| Revision | File | Tool count | Headline result |
-|----------|------|-----------:|-----------------|
-| **v9 (current)** | [`functiongemma-physical-ai-v9-Q5_K_M.gguf`](./functiongemma-physical-ai-v9-Q5_K_M.gguf) | 8 | 30/30 (100 %) routing on held-out smoke prompts; **0.55 s cold prefill** on the 2-core A55 (vs ~57 s for v7's schema-in-prompt build — **105× faster**). |
-| v7 (legacy) | [`functiongemma-physical-ai-v7-Q5_K_M.gguf`](./functiongemma-physical-ai-v7-Q5_K_M.gguf) | 10 | 86.8 % overall on a 250-row eval; schema-in-prompt build. |
-| v6 (legacy) | [`functiongemma-physical-ai-v6-Q5_K_M.gguf`](./functiongemma-physical-ai-v6-Q5_K_M.gguf) | 11 | Camera + vision dropped from earlier revs. Schema-in-prompt build. |
-| v4c (legacy) | [`functiongemma-physical-ai-Q4_K_M.gguf`](./functiongemma-physical-ai-Q4_K_M.gguf) | 13 | Earliest published checkpoint. |
+**Current revision:** [`functiongemma-physical-ai-v10-Q5_K_M.gguf`](./functiongemma-physical-ai-v10-Q5_K_M.gguf)
+— 6 tools, ~248 MB Q5_K_M, ~0.48 s cold prefill on the 2-core
+Cortex-A55, 97.9 % mean token accuracy on eval.
 
-Schema ships as [`tools.json`](./tools.json) (8 tools, v9). Token-to-tool
-mapping is in [`token_map.json`](./token_map.json).
+Schema ships as [`tools.json`](./tools.json). Token-to-tool mapping is
+in [`token_map.json`](./token_map.json).
 
-## What changed in v9
+## Tool surface (6 tools)
 
-v9 is a structural rewrite of the training pipeline, not just a dataset
-refresh. Earlier revisions used the upstream FunctionGemma prompt format,
-which injects the full tool schema (~1088 tokens) into every request as a
-`<start_function_declaration>` developer turn. On a 2-core Cortex-A55 that
-prefill cost ~57 s on the first turn — incompatible with a sub-second
-voice UX.
+| Token | Name | Args | Purpose |
+|---|---|---|---|
+| `<tool_0>` | `set_lights` | `color?`, `effect?`, `state?` | Drive whatever lights are connected — HAT 3-LED indicators or a WLED-driven addressable strip / ring. All three args optional; the model emits only what the user implied. |
+| `<tool_1>` | `play_buzzer` | `pattern` | Named pattern on the piezo buzzer: `beep`, `double_beep`, `chirp`, `siren`, `alarm`, `success`, `error`. |
+| `<tool_2>` | `set_alarm` | `duration` or `time`, `label?` | Schedule an alarm. Fires the buzzer plus a visible flash. |
+| `<tool_3>` | `cancel_alarm` | `label?` | Cancel one alarm by label, or all if no label given. |
+| `<tool_4>` | `get_system_status` | `metric` | `cpu`, `memory`, `temperature`, `npu`, or `all`. |
+| `<tool_5>` | `respond` | `message` | Natural-language reply when no physical-action tool fits, or when the request is ambiguous and the model needs to ask for clarification. |
+
+The model is **hardware-agnostic** for lighting: it parses user intent
+into semantic args (`color`, `effect`, `state`) and leaves the dispatcher
+to map those onto whatever LED hardware is detected at launch — the
+HAT's three indicator LEDs, a WLED-driven strip, or a Neopixel ring. The
+user vocabulary is hardware-agnostic too: "lights", "LEDs", "strip",
+"indicators" all refer to whatever is wired up.
+
+## Prompt format
+
+The v10 model is trained
+[Octopus v2](https://arxiv.org/abs/2404.01744) style: no schema, no
+tools list, just a bare user turn.
 
-v9 follows [Octopus v2](https://arxiv.org/abs/2404.01744) end to end:
+```
+<start_of_turn>user
+{user_text}<end_of_turn>
+<start_of_turn>model
 
-| | Pre-v9 (schema-in-prompt) | v9 (Octopus v2) |
-|---|---|---|
-| Prompt format | `<start_of_turn>developer\n<start_function_declaration>...{schema}...<end_function_declaration>\n<end_of_turn>\n<start_of_turn>user\n{q}<end_of_turn>\n<start_of_turn>model\n` | `<start_of_turn>user\n{q}<end_of_turn>\n<start_of_turn>model\n` |
-| Tokens per prompt | ~1088 | ~13 |
-| Cold prefill on SL2619 (2-core A55) | ~57 s | **~0.55 s** |
-| Tool routing | learned from in-context schema | learned from `<tool_N>` token weights |
-| Training data shape | `{tools, messages: [dev, user, asst]}` with schema embedded | `{input, output, split}` — flat |
+```
 
-The schema in `tools.json` is still the source of truth for dispatcher
-arg validation and is embedded in the GGUF metadata for schema-drift
-checks, but it is **not** loaded into the inference prompt.
+Tool semantics live in the model weights (via the special functional
+tokens `<tool_0>` … `<tool_5>` plus `<end>`), not in the prompt. The
+`tools.json` schema in this repo is the dispatcher's arg-validation
+contract and is embedded in the GGUF metadata for schema-drift checks,
+but it is **not** loaded into the inference prompt. Typical prompts are
+~13 tokens.
 
-## Tool surface (v9, 8 tools)
+## Output format — functional tokens, named args
 
-| Token | Name | Args | Purpose |
-|---|---|---|---|
-| `<tool_0>` | `set_status_led` | `led`, `state`, `brightness?` | On/off one or all HAT status LEDs |
-| `<tool_1>` | `blink_status_led` | `led`, `count?`, `speed?` | Discrete blink |
-| `<tool_2>` | `set_neopixel_effect` | `effect`, `color?`, `palette?`, `speed?`, `intensity?` | Animated effect on the ring |
-| `<tool_3>` | `play_buzzer` | `pattern` | `beep`, `double_beep`, `chirp`, `siren`, `alarm`, `success`, `error` |
-| `<tool_4>` | `set_alarm` | `duration` or `time`, `label?` | Timer |
-| `<tool_5>` | `cancel_alarm` | `label?` | Cancel one or all |
-| `<tool_6>` | `get_system_status` | `metric` | `cpu`, `memory`, `temperature`, `npu`, or `all` |
-| `<tool_7>` | `respond` | `message` | Natural-language fallback when no tool fits |
-
-Surface routing keyword: `set_neopixel_effect` requires the literal
-substring `neopixels` in the user input. LED-vs-ring ambiguous prompts
-("turn off the lights") route to `respond()` asking the user to
-disambiguate.
-
-## Output format — functional tokens
-
-Tool calls emit as **functional tokens**: each tool name compiles to a
-single special-vocabulary token (`<tool_0>` … `<tool_7>`) plus a single
-`<end>` terminator. A complete call decodes in 8–15 output tokens, vs
-~30–80 for the upstream native FunctionGemma
-`<start_function_call>call:NAME{...}<end_function_call>` syntax. On a
-2-core Cortex-A55 this is the difference between sub-second and 2–5 s
-voice-UX latency.
-
-Sample output: `<tool_0>("red","on")<end>` for `set_status_led(led="red", state="on")`.
-
-> ⚠️ Inference servers MUST stop generation on `<end_of_turn>` (or `<eos>`),
-> NOT on `<end>`. The v9 model can emit multi-tool sequences
+Tool calls emit as **functional tokens with named arguments**, per the
+Mercedes-Benz Octopus v2 convention
+([arXiv 2501.02342](https://arxiv.org/abs/2501.02342)). Each tool name
+compiles to a single special-vocabulary token (`<tool_0>` … `<tool_5>`);
+arguments are written as `name="value"` pairs; a single `<end>` token
+terminates the call. The model emits **only the args the user implied**
+— absent args are simply not present.
+
+Examples:
+
+| User says | Model emits | Resolves to |
+|---|---|---|
+| `turn the lights red` | `<tool_0>(color="red")<end>` | `set_lights(color="red")` |
+| `rainbow on the strip` | `<tool_0>(effect="rainbow")<end>` | `set_lights(effect="rainbow")` |
+| `lights off` | `<tool_0>(state="off")<end>` | `set_lights(state="off")` |
+| `red sparkle` | `<tool_0>(color="red", effect="sparkle")<end>` | `set_lights(color="red", effect="sparkle")` |
+| `set an alarm in 5 minutes` | `<tool_2>(duration="5 minutes")<end>` | `set_alarm(duration="5 minutes")` |
+| `cancel all alarms` | `<tool_3>()<end>` | `cancel_alarm()` |
+| `what's the cpu` | `<tool_4>(metric="cpu")<end>` | `get_system_status(metric="cpu")` |
+| `good morning` | `<tool_5>(message="Good morning. ...")<end>` | `respond(message="...")` |
+
+A complete call decodes in roughly 8–20 output tokens, well inside the
+sub-second voice-UX budget on a 2-core Cortex-A55.
+
+> ⚠️ Inference servers MUST stop generation on `<end_of_turn>` (or
+> `<eos>`), NOT on `<end>`. The model can emit multi-tool sequences
 > `<tool_A>(args)<end><tool_B>(args)<end>`, so stopping at the first
 > `<end>` truncates legitimate multi-tool output.
 
@@ -95,20 +103,21 @@ Sample output: `<tool_0>("red","on")<end>` for `set_status_led(led="red", state=
 
 ```bash
 hf download BrinqAI/functiongemma-270m-physical-ai \
-  functiongemma-physical-ai-v9-Q5_K_M.gguf Modelfile tools.json token_map.json \
+  functiongemma-physical-ai-v10-Q5_K_M.gguf Modelfile tools.json token_map.json \
   --local-dir ./fg-physical-ai
 
 cd fg-physical-ai
 ollama create functiongemma-physical-ai -f Modelfile
 ```
 
-The shipped `Modelfile` bakes in the stop tokens (`<end_of_turn>`, `<eos>`)
-and decode parameters (`temperature=0`, `num_ctx=1024`, `num_predict=80`).
+The shipped `Modelfile` bakes in the stop tokens (`<end_of_turn>`,
+`<eos>`) and decode parameters (`temperature=0`, `num_ctx=1024`,
+`num_predict=80`).
 
 ## Calling the model
 
-The v9 model expects a **bare user turn** — no schema, no tools list. Send
-to Ollama with `raw=true`:
+Send a **bare user turn** — no schema, no tools list. With Ollama, use
+`raw=true`:
 
 ```python
 import json
@@ -120,6 +129,8 @@ MODEL = "functiongemma-physical-ai"
 
 reverse_token_map = json.load(open("token_map.json"))["reverse"]
 
+NAMED_ARG_RE = re.compile(r'(\w+)\s*=\s*"((?:[^"\\]|\\.)*)"')
+
 
 def build_prompt(user_text: str) -> str:
     return (
@@ -150,18 +161,19 @@ def call_model(user_text: str) -> str:
         return json.loads(resp.read())["response"]
 
 
-def parse_call(raw: str) -> tuple[str | None, str]:
-    """Return (tool_name, raw_args_string). tool_name is None on parse fail."""
+def parse_call(raw: str) -> tuple[str | None, dict[str, str]]:
+    """Return (tool_name, kwargs). tool_name is None on parse fail."""
     m = re.match(r"\s*(<tool_\d+>)\((.*?)\)<end>", raw)
     if not m:
-        return None, ""
-    tok, args = m.group(1), m.group(2)
-    return reverse_token_map.get(tok), args
+        return None, {}
+    tok, body = m.group(1), m.group(2)
+    kwargs = {k: v for k, v in NAMED_ARG_RE.findall(body)}
+    return reverse_token_map.get(tok), kwargs
 
 
-raw = call_model("turn the red LED on")
-print(raw)               # e.g. '<tool_0>("red","on")<end>'
-print(parse_call(raw))   # ('set_status_led', '"red","on"')
+raw = call_model("turn the lights red")
+print(raw)               # e.g. '<tool_0>(color="red")<end>'
+print(parse_call(raw))   # ('set_lights', {'color': 'red'})
 ```
 
 For `llama-cpp-python` directly, use `detokenize(..., special=True)` so
@@ -170,43 +182,46 @@ stripped.
 
 ## Training data
 
-v9's training data was generated from Haiku-authored phrasing templates
+Training data was generated from Haiku-authored phrasing templates
 crossed with deterministic entity pools, then lightly augmented with
 Moonshine-flavored ASR noise (dropped function words, lowercased traces,
-filler-word prepends). The shape matches Brinq's SmartPanel v15 trainer:
-flat `{input, output, split}` records, no tools / messages array.
+filler-word prepends). Each record is a flat `{input, output}` pair —
+no tools / messages array, no chat template.
 
-| | v9 |
+|  |  |
 |---|---|
-| Train rows | 6,127 |
-| Eval rows | 1,339 |
-| Tools | 8 |
-| Multi-tool fraction | low — single-tool emphasis; multi-tool routines composed at dispatch time |
-| Augmentation | Moonshine-sim noise on ~30 % of records |
-
-Per-tool train counts (range 217–1,199; cancel_alarm + play_buzzer are the
-narrowest classes because their natural phrasing variation is smaller —
-not a coverage gap).
+| Train rows | 5,222 |
+| Eval rows | 920 |
+| Tools | 6 |
+| Per-template entity expansion | color × effect × state pools for `set_lights`; pattern pool for `play_buzzer`; duration / time pools for `set_alarm`; metric pool for `get_system_status` |
+| ASR-style augmentation | Moonshine-sim noise on a fraction of records (dropped articles, lowercased traces, filler prepends) |
+| Multi-tool fraction | None — single-tool emphasis; multi-tool routines composed at dispatch time |
+
+The `set_lights` tool also gets explicit **failure-mode rows** that
+route bare ambiguous prompts to `respond()` — e.g. "rainbow" alone
+("Did you mean the lights? Try 'rainbow on the lights'."), "siren" alone
+(prompts the user toward `play_buzzer`), and bare "on" / "off"
+(asks what the user wants to act on).
 
 ## Methodology
 
-Direct port of the SmartPanel v15 trainer:
-
 - **Full bf16 fine-tune** (no LoRA).
-- **Functional tokens**: `<tool_0>` … `<tool_7>` + `<end>` added as
-  `additional_special_tokens`; new embeddings **mean-initialized** from the
-  existing input embedding matrix (random init under-converges on small
-  datasets).
+- **Functional tokens**: `<tool_0>` … `<tool_5>` + `<end>` added as
+  `additional_special_tokens`; new embeddings **mean-initialized** from
+  the existing input-embedding matrix (random init under-converges on
+  small datasets at this scale).
 - **Completion-only loss mask**: hand-rolled — labels before
-  `<start_of_turn>model\n` are masked to `-100`. The model learns only from
-  the assistant turn, not the user prompt.
-- **5 epochs**, lr `3e-5`, cosine schedule, 0.1 warmup, weight decay 0.01.
-- **Effective batch = 16** (`per_device_train_batch_size=8 ×
-  gradient_accumulation_steps=2`).
-- **`max_length=256`** — the trained prompt format is ~13 tokens and the
-  assistant turn fits comfortably under 64 tokens, including respond()
-  messages.
-- bf16, gradient checkpointing, `adamw_torch_fused`, `metric_for_best_model="eval_loss"` + `load_best_model_at_end=True`.
+  `<start_of_turn>model\n` are masked to `-100`. The model learns only
+  from the assistant turn, not the user prompt.
+- **5 epochs**, lr `3e-5`, cosine schedule, 0.1 warmup, weight decay
+  0.01.
+- **Effective batch = 16**
+  (`per_device_train_batch_size=8 × gradient_accumulation_steps=2`).
+- **`max_length=256`** — the trained prompt format is ~13 tokens and
+  the assistant turn fits comfortably under 64 tokens, including
+  `respond()` messages.
+- bf16, gradient checkpointing, `adamw_torch_fused`,
+  `metric_for_best_model="eval_loss"` + `load_best_model_at_end=True`.
 - Training wallclock: **5 min on a single H100** (~15–20 min on a 4090).
 
 ### Citation
@@ -219,55 +234,73 @@ Direct port of the SmartPanel v15 trainer:
   year    = {2024},
   url     = {https://arxiv.org/abs/2404.01744}
 }
+
+@article{merc2025octopusv2,
+  title   = {Octopus v2 named-arg function calling},
+  journal = {arXiv preprint arXiv:2501.02342},
+  year    = {2025},
+  url     = {https://arxiv.org/abs/2501.02342}
+}
 ```
 
 ## Results
 
 ### Training metrics (final epoch)
 
-| | v9 |
+|  |  |
 |---|---|
-| Final train loss | 0.555 |
-| Final eval loss | **0.090** |
+| Final train loss | 0.493 |
+| Final eval loss | **0.046** |
 | Mean token accuracy (eval) | **97.9 %** |
 
-### Held-out smoke test (post-train, 30 prompts spanning all 8 tools)
+### Held-out smoke test (post-train, 36 prompts spanning all 6 tools)
 
-| | v9 |
+|  |  |
 |---|---|
-| Smoke-test routing accuracy | **30 / 30 (100 %)** |
+| Smoke-test routing accuracy | **35 / 36 (97.2 %)** |
 
-The 30-prompt suite covers single-tool happy paths for every tool plus
-the failure modes that broke v8: ambiguous LED prompts ("turn off the
-lights"), effect-name without `neopixels` keyword ("do the aurora"),
-unsupported features ("play a tone at 2000 hz"), and out-of-scope
-appliances ("turn on the TV"). All 8 of those route to `respond()` with a
-helpful explanation.
+The 36-prompt suite covers single-tool happy paths for every tool plus
+failure modes the model is expected to deflect: ambiguous color words
+without a target ("make it red"), effect names without a target
+("rainbow"), unsupported features ("play a tone at 2000 hz"), and
+out-of-scope appliances. Failure-mode prompts all route to `respond()`
+with a helpful clarification message.
 
 ### On-device benchmark (Coralboard, 2-core Cortex-A55 @ 2 GHz, Q5_K_M GGUF)
 
-| | v7 (schema-in-prompt, 10 tools) | v9 (Octopus v2, 8 tools) |
-|---|---|---|
-| Prompt tokens | ~1088 | ~13 |
-| **Cold prefill (turn 1)** | **57.3 s** | **0.55 s** (105× faster) |
-| Warm prefill (turn 2+) | ~3 s | ~0.4 s |
-| Decode for a typical call | 0.5–1.2 s | 0.5–1.2 s |
-| End-to-end first-turn (model load 2.3 s + prefill + decode) | ~62 s | ~3 s |
-| Routing on a 29-prompt board bench | n/a directly comparable | **29 / 29 (100 %)** |
+Measured with `llama-cpp-python` 0.3.16, `n_ctx=1024`, `n_threads=2`,
+CPU governor `performance`, 8 representative prompts spanning all 6
+tools.
+
+|  |  |
+|---|---|
+| Model load | 2.23 s |
+| Prompt tokens | 11–16 (mean ~13) |
+| **Cold prefill (turn 1)** | **0.48 s** |
+| Warm prefill (turn 2+, avg) | 0.47 s |
+| Decode rate | **~9.7 tok/s** |
+| Decode time, typical tool call (3–8 output tokens) | 0.3–0.8 s |
+| Decode time, `respond()` (~25 output tokens) | ~2.6 s |
+| End-to-end first turn (model load + prefill + decode) | ~3.4 s |
 
 ## Files
 
 ```
-functiongemma-physical-ai-v9-Q5_K_M.gguf   # ~248 MB, v9 GGUF Q5_K_M weights (Ollama / llama.cpp)
+functiongemma-physical-ai-v10-Q5_K_M.gguf  # ~248 MB, Q5_K_M weights (Ollama / llama.cpp)
 Modelfile                                  # Ollama Modelfile (functional-token format)
-tools.json                                 # 8-tool schema (v9, canonical mobile-actions format)
-token_map.json                             # functional-token <-> tool-name map (v9)
+tools.json                                 # 6-tool schema, canonical mobile-actions format
+token_map.json                             # functional-token <-> tool-name map
 README.md                                  # this file
 ```
 
-Legacy v6/v7 GGUFs are kept in repo history for reproducibility but should
-not be used for new deployments — they require the schema-in-prompt
-inference wrapper and pay the ~57 s cold-prefill cost.
+Earlier checkpoint GGUFs from the project's development history
+(`functiongemma-physical-ai-v9-Q5_K_M.gguf`,
+`functiongemma-physical-ai-v7-Q5_K_M.gguf`,
+`functiongemma-physical-ai-v6-Q5_K_M.gguf`,
+`functiongemma-physical-ai-Q4_K_M.gguf`) remain in the repo for
+reproducibility. They use different tool surfaces and (for v7 and
+earlier) a different inference-prompt format; new deployments should use
+the v10 file above.
 
 ## License
 
@@ -279,6 +312,7 @@ By using this model you agree to those terms. Base model:
 
 - Base model: <https://huggingface.co/google/functiongemma-270m-it>
 - Octopus v2 paper: <https://arxiv.org/abs/2404.01744>
+- Mercedes-Benz Octopus v2 (named-arg variant): <https://arxiv.org/abs/2501.02342>
 - Hardware demo + integration code (Synaptics Coralboard, Grinn HAT,
   WLED-over-USB-CDC, full PyQt UI):
   <https://github.com/synaptics-astra-demos/sl2610-examples> →

functiongemma-physical-ai-v10-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:44eaf599b448df2a72d9384a8476ead654383bdb109f05c963ad8d1c06fce26c
+size 260045600

token_map.json

@@ -1,38 +1,24 @@
 {
-  "version": "0.4.0",
-  "description": "Compressed token map for FunctionGemma CPU inference on SL2619. v9: 8 tools (set_status_led, blink_status_led, set_neopixel_effect, play_buzzer, set_alarm, cancel_alarm, get_system_status, respond) + <end> terminator. Trained Octopus v2 style — functional tokens are the entire output vocabulary the model uses for routing; the tool schema (tools.json) is NOT loaded into the inference prompt. v9 drops the unused <tool_none> sentinel that v8's training pipeline reserved but never emitted.",
+  "version": "0.5.0-draft",
+  "description": "DRAFT v10 compact token map. 6 tools + <end> terminator. Single unified set_lights tool with semantic named args (Mercedes-Benz Octopus v2 convention, arXiv 2501.02342). The model is hardware-agnostic; the dispatcher maps semantic args to whatever LED hardware is detected at launch.",
   "tokens": {
-    "set_status_led": "<tool_0>",
-    "blink_status_led": "<tool_1>",
-    "set_neopixel_effect": "<tool_2>",
-    "play_buzzer": "<tool_3>",
-    "set_alarm": "<tool_4>",
-    "cancel_alarm": "<tool_5>",
-    "get_system_status": "<tool_6>",
-    "respond": "<tool_7>"
+    "set_lights": "<tool_0>",
+    "play_buzzer": "<tool_1>",
+    "set_alarm": "<tool_2>",
+    "cancel_alarm": "<tool_3>",
+    "get_system_status": "<tool_4>",
+    "respond": "<tool_5>"
   },
   "reverse": {
-    "<tool_0>": "set_status_led",
-    "<tool_1>": "blink_status_led",
-    "<tool_2>": "set_neopixel_effect",
-    "<tool_3>": "play_buzzer",
-    "<tool_4>": "set_alarm",
-    "<tool_5>": "cancel_alarm",
-    "<tool_6>": "get_system_status",
-    "<tool_7>": "respond"
+    "<tool_0>": "set_lights",
+    "<tool_1>": "play_buzzer",
+    "<tool_2>": "set_alarm",
+    "<tool_3>": "cancel_alarm",
+    "<tool_4>": "get_system_status",
+    "<tool_5>": "respond"
   },
-  "special_tokens": [
-    "<tool_0>",
-    "<tool_1>",
-    "<tool_2>",
-    "<tool_3>",
-    "<tool_4>",
-    "<tool_5>",
-    "<tool_6>",
-    "<tool_7>",
-    "<end>"
-  ],
-  "output_format": "<tool_N>(\"arg1\",\"arg2\",...)<end>",
+  "special_tokens": ["<tool_0>", "<tool_1>", "<tool_2>", "<tool_3>", "<tool_4>", "<tool_5>", "<end>"],
+  "output_format": "<tool_N>(name1=\"value1\", name2=value2, ...)<end>",
   "prompt_format": "<start_of_turn>user\\n{user_text}<end_of_turn>\\n<start_of_turn>model\\n",
-  "notes": "Argument order positional per canonical schema's required-first then optional declaration order. v9 trains Octopus v2 pure (no schema in prompt) — see prompt_format. set_neopixel_effect routing keyword: 'neopixels' (literal substring, case-insensitive) required in user prompt; otherwise the model routes to respond() asking the user to disambiguate (HAT status LEDs vs. neopixel ring)."
+  "notes": "v10 uses NAMED arguments — per Mercedes-Benz Octopus v2 (arXiv 2501.02342), the published Octopus-v2 reference that demonstrates production multi-arg tool calls with optional args absent. The model emits only the args the user implied; absent args are simply not emitted. set_lights kept to 3 optional args (color/effect/state) for robustness on the 270M — brightness/count/speed were dropped after analysis; the dispatcher uses sensible defaults (full brightness, normal speed, ~3 repetitions for blink). Backwards-compat positional parsing is retained in compact_codec.py so the parser still handles v9 GGUF output during the transition."
 }

tools.json

@@ -1,97 +1,41 @@
 {
-  "version": "0.4.0",
-  "description": "Physical AI tool schema for Coral Dev Board (SL2619) FunctionGemma demo. Canonical mobile-actions format. v9: same 8-tool surface as v8, but the model is trained Octopus v2 style — functional tokens (<tool_0>..<tool_7>) emitted directly from a minimal user-only prompt with NO tool schema in the context. This shrinks the cold prefill from ~1088 prompt tokens (v7/v8 with the FunctionGemma developer turn) to ~13, taking on-board cold first-turn from ~57s to ~0.5s on a 2-core A55. The schema in this file is purely a developer/runtime contract (dispatcher arg validation, GGUF metadata, documentation) — it is NOT injected into the inference prompt. Surface routing keyword: 'neopixels' (literal) for the ring; 'LED' / 'light' / 'red light' / 'green light' / 'blue light' for the HAT status LEDs.",
+  "version": "0.5.0-draft",
+  "description": "DRAFT v10 schema. Single unified set_lights tool replaces v9's three LED tools (set_status_led, blink_status_led, set_neopixel_effect). The model is hardware-agnostic — it parses user intent into semantic args; the dispatcher maps those args to whichever LED hardware is detected at launch (HAT 3-LED indicators or WLED strip). User vocabulary is hardware-agnostic too: 'lights', 'LEDs', 'strip' all refer to whatever is connected. Named-args format per Mercedes-Benz Octopus v2 (arXiv 2501.02342) — emitted calls look like <tool_0>(color=\"red\", state=\"on\")<end>. 6 tools (down from 8). set_lights kept to 3 args (color/effect/state) for robustness on the 270M — brightness/count/speed dropped after analysis showed they appeared in zero of 5 observed voice failures and the dispatcher's defaults cover them.",
+  "format": "named",
   "tools": [
     {
       "function": {
-        "name": "set_status_led",
-        "description": "Turn one or all of the HAT status LEDs on or off. The HAT has three individual LEDs (red, green, blue), each independently addressable. Invoke when the user mentions 'LED', 'LEDs', or 'the <color> light'.",
+        "name": "set_lights",
+        "description": "Set the lights according to user intent. Emit only the args the user implied — color if they named one, effect if they named an animation/pattern, state if they said on/off. All three args optional; the dispatcher fills sensible defaults (full brightness, normal speed, ~3 repetitions for blink) and approximates effects the connected hardware can't perform natively.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
-            "led": {
-              "type": "STRING",
-              "description": "Which LED: 'red', 'green', 'blue', or 'all'."
-            },
-            "state": {
-              "type": "STRING",
-              "description": "'on' or 'off'."
-            },
-            "brightness": {
-              "type": "INTEGER",
-              "description": "Brightness 0-100. Optional, default 100. Ignored when state is 'off'."
-            }
-          },
-          "required": ["led", "state"]
-        }
-      }
-    },
-    {
-      "function": {
-        "name": "blink_status_led",
-        "description": "Blink one or all HAT status LEDs a given number of times. Invoke for 'blink/flash the <color> light' or 'blink the LEDs' style requests.",
-        "parameters": {
-          "type": "OBJECT",
-          "properties": {
-            "led": {
-              "type": "STRING",
-              "description": "Which LED: 'red', 'green', 'blue', or 'all'."
-            },
-            "count": {
-              "type": "INTEGER",
-              "description": "Number of blinks. Default 3."
-            },
-            "speed": {
-              "type": "STRING",
-              "description": "One of 'slow', 'normal', 'fast'. Default 'normal'."
-            }
-          },
-          "required": ["led"]
-        }
-      }
-    },
-    {
-      "function": {
-        "name": "set_neopixel_effect",
-        "description": "Play a visual effect on the neopixel ring (48-pixel WS2812B ring around the 7\" display, driven by WLED). Only invoke when the user explicitly mentions 'neopixels'. Use effect='off' to turn the ring off.",
-        "parameters": {
-          "type": "OBJECT",
-          "properties": {
-            "effect": {
-              "type": "STRING",
-              "description": "One of: 'solid', 'pulse', 'fade', 'chase', 'rainbow', 'sparkle', 'off', 'aurora', 'plasma', 'comet', 'twinkle', 'fireworks', 'police', 'heartbeat', 'loading', 'lightning', 'glitter', 'fire', 'sunrise'."
-            },
             "color": {
               "type": "STRING",
-              "description": "Color name (e.g. 'red', 'teal', 'amber', 'gold', 'violet') or 6-digit hex like '#FF8800'. Used by effects that take a primary color (solid, pulse, fade, chase, sparkle, comet). Ignored for rainbow and palette-driven effects."
-            },
-            "palette": {
-              "type": "STRING",
-              "description": "Color palette: 'auto', 'ocean', 'lava', 'forest', 'sunset', 'party', 'sherbet', 'c9', 'aurora', 'beach', 'fire', 'sakura', 'splash', 'pastel'. Most useful with aurora, plasma, fire, twinkle, comet."
+              "description": "Color name. Common values: 'red', 'green', 'blue', 'white', 'yellow', 'purple', 'orange', 'pink', 'cyan'. HAT mode maps non-RGB colors to closest combo (e.g. yellow=red+green)."
             },
-            "speed": {
+            "effect": {
               "type": "STRING",
-              "description": "One of 'slow', 'normal', 'fast'. Default 'normal'."
+              "description": "Named animation. Strip mode handles all natively. HAT mode approximates on 3 LEDs. Values: 'solid', 'blink', 'pulse', 'fade', 'rainbow', 'fire', 'plasma', 'aurora', 'police', 'fireworks', 'sparkle', 'twinkle', 'chase', 'comet', 'heartbeat', 'lightning', 'glitter', 'loading', 'sunrise', 'off'."
             },
-            "intensity": {
+            "state": {
               "type": "STRING",
-              "description": "One of 'low', 'medium', 'high'. Default 'medium'. Controls effect density / depth (sparkle density, fire height, comet tail length, aurora width)."
+              "description": "'on' or 'off'. Use when the user just toggles lights without specifying a color or effect."
             }
-          },
-          "required": ["effect"]
+          }
         }
       }
     },
     {
       "function": {
         "name": "play_buzzer",
-        "description": "Play a named pattern on the piezo buzzer on the HAT. The buzzer is binary GPIO on the Coral Dev Board (BUZZERn), so only timed patterns are supported — no tunable frequency.",
+        "description": "Play a named pattern on the piezo buzzer on the HAT. Binary GPIO — only timed patterns, no tunable frequency.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
             "pattern": {
               "type": "STRING",
-              "description": "Named pattern: 'beep', 'double_beep', 'siren', 'chirp', 'alarm', 'success', 'error'."
+              "description": "One of: 'beep', 'double_beep', 'siren', 'chirp', 'alarm', 'success', 'error'."
             }
           },
           "required": ["pattern"]
@@ -101,7 +45,7 @@
     {
       "function": {
         "name": "set_alarm",
-        "description": "Schedule an alarm to go off at a given time or after a duration. Alarm triggers buzzer + flashing LEDs.",
+        "description": "Schedule an alarm to go off at a given time or after a duration. Alarm triggers the buzzer plus a visible flash on whatever lights are connected.",
         "parameters": {
           "type": "OBJECT",
           "properties": {
@@ -154,7 +98,7 @@
     {
       "function": {
         "name": "respond",
-        "description": "Reply to the user in natural language without taking any physical action. Use this when the request is out of scope (no matching tool), ambiguous (needs clarification — e.g. surface keyword missing for LED requests), purely conversational (greetings, thanks), or impossible on this device. Do NOT use this when any physical-action tool fits the request.",
+        "description": "Reply to the user in natural language without taking any physical action. Use when the request is out of scope, ambiguous, purely conversational, or impossible on this device. Do NOT use when set_lights or another physical-action tool fits the request.",
         "parameters": {
           "type": "OBJECT",
           "properties": {