{
  "artifacts": [
    {
      "file": "granite-speech-5.0-2b.litertlm",
      "sha256": "c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265",
      "size_mb": 2714.69
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "ibm-granite/granite-speech-5.0-2b rev 98160169 -> the language model: export_hf, prefill signatures 64, 256 and 1,024, recipe dynamic_wi8_afp32, KV cache 4,096; the audio encoder: a PyTorch module computing what the source's feature extractor, encoder and projector compute on the unpadded clip, one 36-second window, float16 weights; the source's tokenizer.json byte for byte, a chat template rendering `USER: <|audio|>{instruction}\\n ASSISTANT:`, stop token <|end_of_text|> (100257), model type generic_model (card L59, L202, L204-L205)",
    "quantization": "language model: int8 weights (one scale per output channel), KV cache of 4,096 tokens, the file asks the GPU for float32 activations; token embedding table int8; audio encoder: float16 weights computed in float32, one 36-second window, CPU only (card L49, L54-L56)",
    "tool": "litert-torch 0.10.0 (export_hf for the Granite language model, the token embedding exported separately; the audio encoder as a PyTorch module traced with litert-torch) packed with litert-lm-builder 0.19.1; repro = hf-to-litertlm REPRODUCE.md, the granite-speech-5.0-2b.litertlm entry (card L204-L207)",
    "tool_version": "litert-torch 0.10.0, litert-lm-builder 0.19.1 (litert-torch 0.10.0's own packing calls a builder API that 0.19 no longer has); LiteRT-LM 0.19.1 for every measurement except the encoder-alone times (ai-edge-litert 2.3.0) (card L31, L203, L206)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, cpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-cpu-tc (run s26a, 2026-11-07 06:35:59 JST at the run's start, clean; rested 92s since 06:34:27): 9.9 s clip first text 5.4474 s, end 7.9079 s (5.9 s 5.0329 / 6.7569 s, 29.4 s 6.2684 / 13.8103 s), encoder in-call 4396.6 ms on the 9.9 s clip (mean over the 3 clips 4176.0333 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 4967608 kB; samples 31, CPU capped 29, GPU capped 0, thermal max 1; every transcript equal to the Mac reference of the same backend: False",
            "b32-cpu-tc2 (run s26a, 2026-11-07 06:38:28 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 78s since 06:37:11): 9.9 s clip first text 4.8729 s, end 7.383 s (5.9 s 4.1995 / 5.8023 s, 29.4 s 6.6928 / 14.2599 s), encoder in-call 3844.0 ms on the 9.9 s clip (mean over the 3 clips 3853.2 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 5946324 kB; samples 28, CPU capped 27, GPU capped 0, thermal max 1; every transcript equal to the Mac reference of the same backend: False",
            "(the same lines in b32-cpu-tc2.err):",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 1 (prefill_256).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 92.0,
            "clip29_4s_end_s": 14.0351,
            "clip29_4s_first_text_s": 6.4806,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 6.2796,
            "clip5_9s_first_text_s": 4.6162,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 7.6455,
            "clip9_9s_first_text_s": 5.1601,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 4120.3,
            "encoder_in_call_ms_mean_3_clips": 4014.6166,
            "first_text_sum_3_clips_s": 16.2569,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 2.0,
            "run1_clip9_9s_first_text_s": 5.4474,
            "run1_cpu_capped_samples": 29.0,
            "run1_samples": 31.0,
            "run1_vmhwm_kb": 4967608.0,
            "run2_clip9_9s_first_text_s": 4.8729,
            "run2_cpu_capped_samples": 27.0,
            "run2_samples": 28.0,
            "run2_vmhwm_kb": 5946324.0
          },
          "output_match": null,
          "peak_mem_mb": 6089.0358,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 5160.15
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, cpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-cpu-tc (run s26a, 2026-11-06 22:20:55 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 151s since 22:18:24; front com.android.settings (a system app: left alone)): 9.9 s clip first text 4.3648 s, end 6.5027 s (5.9 s 4.9238 / 6.3688 s, 29.4 s 5.469 / 13.5541 s), encoder in-call 3502.1 ms on the 9.9 s clip (mean over the 3 clips 3686.1333 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 5000220 kB; samples 29, CPU capped 24, GPU capped 21, thermal max 3; every transcript equal to the Mac reference of the same backend: False",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 1 (prefill_256).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 92.0,
            "clip29_4s_end_s": 13.5541,
            "clip29_4s_first_text_s": 5.469,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 6.3688,
            "clip5_9s_first_text_s": 4.9238,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 6.5027,
            "clip9_9s_first_text_s": 4.3648,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 3502.1,
            "encoder_in_call_ms_mean_3_clips": 3686.1333,
            "first_text_sum_3_clips_s": 14.7576,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 1.0,
            "run1_clip9_9s_first_text_s": 4.3648,
            "run1_cpu_capped_samples": 24.0,
            "run1_samples": 29.0,
            "run1_vmhwm_kb": 5000220.0
          },
          "output_match": null,
          "peak_mem_mb": 5120.2253,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 4364.8
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, cpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-cpu-g0 (s26b, 2026-11-06 22:31 JST): en 2.428 % (source 2.428, line 2.928); n 20, stopped 20, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 19 of 20; set verdict PASS",
            "b32-cpu-g1 (s26b, 2026-11-06 22:34 JST): ja 6.106 % (source 7.261, line 8.261); eu 2.027 % (source 2.027, line 3.027); n 18, stopped 18, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 14 of 18; set verdict PASS",
            "Acceptance rule: en20 and xx18 runs PASS; verdict PASS",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 1 (prefill_256).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "en20_empty": 0.0,
            "en20_looping": 0.0,
            "en20_n": 20.0,
            "en20_stopped": 20.0,
            "en20_text_equal_mac": 19.0,
            "en_wer_line_pct": 2.928,
            "en_wer_pct": 2.428,
            "en_wer_source_pct": 2.428,
            "fr_de_es_pt_wer_line_pct": 3.027,
            "fr_de_es_pt_wer_pct": 2.027,
            "fr_de_es_pt_wer_source_pct": 2.027,
            "ja_cer_line_pct": 8.261,
            "ja_cer_pct": 6.106,
            "ja_cer_source_pct": 7.261,
            "xx18_empty": 0.0,
            "xx18_looping": 0.0,
            "xx18_n": 18.0,
            "xx18_stopped": 18.0,
            "xx18_text_equal_mac": 14.0
          },
          "output_match": true,
          "peak_mem_mb": 6163.4642,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": 2048,
          "date": "2026-11-07",
          "decode_tokens_per_s": 10.0332,
          "delegated_ops": 2173,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Benchmark(model, cpu, prefill_tokens=1024, decode_tokens=256, max_num_tokens=2048); 3 x run on one engine (no audio backend: the LM alone)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-cpu-t1 (run s26b, 2026-11-07 04:59:58 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 108s since 04:58:10): runs 2-3 prefill 137.75 / 137.72, decode 10.03 / 9.98 tok/s, TTFT 7.533 / 7.535 s, Init Total 605.4 ms; VmHWM 3203508 kB (the 1 s sampler), CPU-capped samples per run [29, 29, 29]",
            "b32-cpu-t2 (run s26b, 2026-11-07 05:03:21 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 109s since 05:01:32): runs 2-3 prefill 138.52 / 137.67, decode 10.08 / 10.04 tok/s, TTFT 7.492 / 7.538 s, Init Total 528.84 ms; VmHWM 3203740 kB (the 1 s sampler), CPU-capped samples per run [29, 29, 29]",
            "(the same lines in b32-cpu-t2.err):",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 0 (prefill_1024).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318",
            "delegation: XNNPACK; the record's counts = the last prefill_1024 line (XNNPACK's second pass over the graph): 2173 of 2259 nodes, the rest on the CPU kernels"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_total_ms": 567.12,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 10.0095,
            "run1_decode_tok_s_it1": 12.0081,
            "run1_init_total_ms": 605.4,
            "run1_prefill_tok_s": 137.7387,
            "run1_ttft_s": 7.5343,
            "run1_vmhwm_kb": 3203508.0,
            "run2_decode_tok_s": 10.057,
            "run2_decode_tok_s_it1": 12.484,
            "run2_init_total_ms": 528.84,
            "run2_prefill_tok_s": 138.0931,
            "run2_ttft_s": 7.5148,
            "run2_vmhwm_kb": 3203740.0,
            "runs": 2.0
          },
          "output_match": null,
          "peak_mem_mb": 3280.6298,
          "prefill_tokens_per_s": 137.9159,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2259,
          "ttft_ms": 7524.55
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, gpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-gpu-tc (run s26b, 2026-11-07 06:54:42 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 146s since 06:52:16): 9.9 s clip first text 3.8155 s, end 5.8674 s (5.9 s 4.9259 / 6.3389 s, 29.4 s 4.7579 / 11.1134 s), encoder in-call 2838.4 ms on the 9.9 s clip (mean over the 3 clips 3241.5333 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 3377300 kB; samples 27, CPU capped 25, GPU capped 17, thermal max 2; every transcript equal to the Mac reference of the same backend: True",
            "b32-gpu-tc2 (run s26b, 2026-11-07 06:57:43 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 119s since 06:55:44): 9.9 s clip first text 3.8391 s, end 5.8674 s (5.9 s 3.7725 / 5.178 s, 29.4 s 4.8636 / 11.1559 s), encoder in-call 2859.7 ms on the 9.9 s clip (mean over the 3 clips 2910.9333 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 3948204 kB; samples 26, CPU capped 20, GPU capped 17, thermal max 2; every transcript equal to the Mac reference of the same backend: True",
            "(the same lines in b32-gpu-tc2.err):",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 95.0,
            "clip29_4s_end_s": 11.1347,
            "clip29_4s_first_text_s": 4.8108,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 5.7584,
            "clip5_9s_first_text_s": 4.3492,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 5.8674,
            "clip9_9s_first_text_s": 3.8273,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 2849.05,
            "encoder_in_call_ms_mean_3_clips": 3076.2333,
            "first_text_sum_3_clips_s": 12.9873,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 2.0,
            "run1_clip9_9s_first_text_s": 3.8155,
            "run1_cpu_capped_samples": 25.0,
            "run1_samples": 27.0,
            "run1_vmhwm_kb": 3377300.0,
            "run2_clip9_9s_first_text_s": 3.8391,
            "run2_cpu_capped_samples": 20.0,
            "run2_samples": 26.0,
            "run2_vmhwm_kb": 3948204.0
          },
          "output_match": null,
          "peak_mem_mb": 4042.9609,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in each of 2 processes; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 3827.3
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, gpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-gpu-tc (run s26a, 2026-11-06 22:22:19 JST at the run's start, warm-deadline (thermal 3, cap policy0:1900800/3628800 policy6:1862400/4742400 , kgsl_pwrlevel 5); rested 17s since 22:22:02; front com.android.settings (a system app: left alone)): 9.9 s clip first text 7.4676 s, end 9.5399 s (5.9 s 6.257 / 7.7138 s, 29.4 s 7.2502 / 13.7282 s), encoder in-call 6376.0 ms on the 9.9 s clip (mean over the 3 clips 5618.9 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 3775432 kB; samples 35, CPU capped 35, GPU capped 35, thermal max 3; every transcript equal to the Mac reference of the same backend: True",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 95.0,
            "clip29_4s_end_s": 13.7282,
            "clip29_4s_first_text_s": 7.2502,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 7.7138,
            "clip5_9s_first_text_s": 6.257,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 9.5399,
            "clip9_9s_first_text_s": 7.4676,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 6376.0,
            "encoder_in_call_ms_mean_3_clips": 5618.9,
            "first_text_sum_3_clips_s": 20.9748,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 1.0,
            "run1_clip9_9s_first_text_s": 7.4676,
            "run1_cpu_capped_samples": 35.0,
            "run1_samples": 35.0,
            "run1_vmhwm_kb": 3775432.0
          },
          "output_match": null,
          "peak_mem_mb": 3866.0424,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 7467.6
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, gpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-gpu-g0 (s26f, 2026-11-06 22:52 JST): en 2.428 % (source 2.428, line 2.928); n 20, stopped 20, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 20 of 20; set verdict PASS",
            "b32-gpu-g1 (s26f, 2026-11-06 22:55 JST): ja 7.261 % (source 7.261, line 8.261); eu 2.027 % (source 2.027, line 3.027); n 18, stopped 18, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 18 of 18; set verdict PASS",
            "Acceptance rule: en20 and xx18 runs PASS; verdict PASS",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "en20_empty": 0.0,
            "en20_looping": 0.0,
            "en20_n": 20.0,
            "en20_stopped": 20.0,
            "en20_text_equal_mac": 20.0,
            "en_wer_line_pct": 2.928,
            "en_wer_pct": 2.428,
            "en_wer_source_pct": 2.428,
            "fr_de_es_pt_wer_line_pct": 3.027,
            "fr_de_es_pt_wer_pct": 2.027,
            "fr_de_es_pt_wer_source_pct": 2.027,
            "ja_cer_line_pct": 8.261,
            "ja_cer_pct": 7.261,
            "ja_cer_source_pct": 7.261,
            "xx18_empty": 0.0,
            "xx18_looping": 0.0,
            "xx18_n": 18.0,
            "xx18_stopped": 18.0,
            "xx18_text_equal_mac": 18.0
          },
          "output_match": true,
          "peak_mem_mb": 3852.2061,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": 2048,
          "date": "2026-11-07",
          "decode_tokens_per_s": 14.8803,
          "delegated_ops": 1785,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-granite-speech-devicebench",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Qualcomm SM8850 (Adreno GPU)",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Benchmark(model, gpu, prefill_tokens=1024, decode_tokens=256, max_num_tokens=2048); 3 x run on one engine (no audio backend: the LM alone)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys",
            "b32-gpu-t1 (run s26c, 2026-11-07 05:23:45 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 149s since 05:21:16): runs 2-3 prefill 217.82 / 232.35, decode 15.08 / 14.67 tok/s, TTFT 4.767 / 4.475 s, Init Total 6857.29 ms; VmHWM 1097612 kB (the 1 s sampler), CPU-capped samples per run [20, 20, 20]",
            "b32-gpu-t2 (run s26c, 2026-11-07 05:27:36 JST at the run's start, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 160s since 05:24:56): runs 2-3 prefill 246.28 / 209.77, decode 15.10 / 14.68 tok/s, TTFT 4.224 / 4.950 s, Init Total 5766.11 ms; VmHWM 1107280 kB (the 1 s sampler), CPU-capped samples per run [21, 21, 21]",
            "(the same lines in b32-gpu-t2.err):",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "delegation: the four language-model signatures whole on LITERT_CL in 1 partition each; the four embedder signatures run 1 of 4 nodes on XNNPACK, so full_delegation is false; the record's counts = the prefill_1024 line (1785 / 1785)"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_total_ms": 6311.7,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 14.8716,
            "run1_decode_tok_s_it1": 15.2227,
            "run1_init_total_ms": 6857.29,
            "run1_prefill_tok_s": 225.0835,
            "run1_ttft_s": 4.6214,
            "run1_vmhwm_kb": 1097612.0,
            "run2_decode_tok_s": 14.889,
            "run2_decode_tok_s_it1": 15.2205,
            "run2_init_total_ms": 5766.11,
            "run2_prefill_tok_s": 228.0217,
            "run2_ttft_s": 4.587,
            "run2_vmhwm_kb": 1107280.0,
            "runs": 2.0
          },
          "output_match": null,
          "peak_mem_mb": 1133.8547,
          "prefill_tokens_per_s": 226.5526,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1785,
          "ttft_ms": 4604.2
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); the harness through, the model loaded once per process, `Engine(bundle, backend=CPU, audio_backend=CPU)`, CPU threads 4, cache dir per arm",
            "Bundle granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm, 2846558336 B; c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265 for that file = the Hub LFS oid",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "ls_clip00 first chunk 1.5794 s, wall 2.304 s; en_us_1670 first chunk 1.7953 s, wall 3.854 s; en_us_1671 first chunk 1.4742 s, wall 2.303 s",
            "ls_clip00 first chunk 3.2125 s, wall 3.96 s; en_us_1670 first chunk 1.7676 s, wall 3.989 s; en_us_1671 first chunk 1.3964 s, wall 2.223 s",
            "ls_clip00 first chunk 1.5812 s, wall 2.303 s; en_us_1670 first chunk 1.7471 s, wall 3.633 s; en_us_1671 first chunk 1.4053 s, wall 2.224 s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clip10_0s_first_text_s": 1.4053,
            "clip10_0s_total_s": 2.224,
            "clip29_3s_first_text_s": 1.7676,
            "clip29_3s_total_s": 3.854,
            "clip5_9s_first_text_s": 1.5812,
            "clip5_9s_total_s": 2.304,
            "cpu_threads": 4.0,
            "max_output_tokens": 256.0,
            "passes": 3.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing on the Mac: a 5.9 s LibriSpeech clip, a 29.3 s and a 10.0 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 3 processes; audio encoder on the CPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 1405.3
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); `Engine(bundle, backend=CPU, audio_backend=CPU)`, CPU threads 4, `SamplerConfig(top_k=1)`, a new conversation per clip, at most 256 new tokens",
            "Bundle granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm, 2846558336 B (c90a3dae... for that file = the Hub LFS oid)",
            "run: 2026-11-06 20:46-21:03 JST (one 203 s pause for another job's window)",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "En 4.534 % (72/1588); ja 5.02 % (135/2689); eu 11.934 % (282/2363) | source: en 4.597 %; ja 5.132 %; eu 12.103 % | lines: en 5.097; ja 6.132; eu 13.103 | clips 220, empty 0, looping 0, not ending at the stop token 0, character-equal to the source model 181; pass True"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clips": 220.0,
            "empty": 0.0,
            "en_wer_line_pct": 5.097,
            "en_wer_pct": 4.534,
            "en_wer_source_pct": 4.597,
            "fr_de_es_pt_wer_line_pct": 13.103,
            "fr_de_es_pt_wer_pct": 11.934,
            "fr_de_es_pt_wer_source_pct": 12.103,
            "ja_cer_line_pct": 6.132,
            "ja_cer_pct": 5.02,
            "ja_cer_source_pct": 5.132,
            "looping": 0.0,
            "not_eos": 0.0,
            "text_equal_source": 181.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 2048,
          "date": "2026-11-06",
          "decode_tokens_per_s": 26.65,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); `litert-lm benchmark <file> -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --backend cpu --cpu-thread-count 4 --cache no`, one process: a warm-up and 3 runs; the CLI prints the mean of the 3 runs",
            "Benchmarking model: granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm",
            "Names that local file with sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265 = the Hub LFS oid",
            "window: for this run ('bench int8_cpu')",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "Prefill speed:        500.55 tokens/s | Decode speed:         26.65 tokens/s | Init time:            11.8371 s | Time to first token:  2.1781 s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 11.8371,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "runs": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 500.55,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 2178.1
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); the harness through, the model loaded once per process, `Engine(bundle, backend=GPU, audio_backend=CPU)`, CPU threads 4, cache dir per arm",
            "Bundle granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm, 2846558336 B; c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265 for that file = the Hub LFS oid",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "ls_clip00 first chunk 1.2802 s, wall 1.529 s; en_us_1670 first chunk 1.2573 s, wall 1.919 s; en_us_1671 first chunk 1.1804 s, wall 1.463 s",
            "ls_clip00 first chunk 2.7752 s, wall 3.027 s; en_us_1670 first chunk 1.2726 s, wall 1.946 s; en_us_1671 first chunk 1.1998 s, wall 1.486 s",
            "ls_clip00 first chunk 1.3256 s, wall 1.587 s; en_us_1670 first chunk 1.3123 s, wall 1.977 s; en_us_1671 first chunk 1.1712 s, wall 1.459 s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clip10_0s_first_text_s": 1.1804,
            "clip10_0s_total_s": 1.463,
            "clip29_3s_first_text_s": 1.2726,
            "clip29_3s_total_s": 1.946,
            "clip5_9s_first_text_s": 1.3256,
            "clip5_9s_total_s": 1.587,
            "cpu_threads": 4.0,
            "max_output_tokens": 256.0,
            "passes": 3.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing on the Mac: a 5.9 s LibriSpeech clip, a 29.3 s and a 10.0 s FLEURS en_us clip in that order, one streaming call per clip with a new conversation each, the model loaded once per process, the transcription instruction, greedy, at most 256 new tokens; the median of 3 processes; audio encoder on the CPU; ttft_ms = the 10.0 s clip's (the third conversation's) first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 1180.4
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); `Engine(bundle, backend=GPU, audio_backend=CPU)`, CPU threads 4, `SamplerConfig(top_k=1)`, a new conversation per clip, at most 256 new tokens",
            "Bundle granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm, 2846558336 B (c90a3dae... for that file = the Hub LFS oid)",
            "run: 2026-11-06 21:03-21:12 JST (one 22 s pause for another job's window)",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "En 4.723 % (75/1588); ja 5.318 % (143/2689); eu 12.061 % (285/2363) | source: en 4.597 %; ja 5.132 %; eu 12.103 % | lines: en 5.097; ja 6.132; eu 13.103 | clips 220, empty 0, looping 0, not ending at the stop token 0, character-equal to the source model 209; pass True"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clips": 220.0,
            "empty": 0.0,
            "en_wer_line_pct": 5.097,
            "en_wer_pct": 4.723,
            "en_wer_source_pct": 4.597,
            "fr_de_es_pt_wer_line_pct": 13.103,
            "fr_de_es_pt_wer_pct": 12.061,
            "fr_de_es_pt_wer_source_pct": 12.103,
            "ja_cer_line_pct": 6.132,
            "ja_cer_pct": 5.318,
            "ja_cer_source_pct": 5.132,
            "looping": 0.0,
            "not_eos": 0.0,
            "text_equal_source": 209.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate: the 220 clips (20 LibriSpeech dev-clean, 50 FLEURS en_us, 50 FLEURS ja_jp and 25 FLEURS clips each of French, German, Spanish and Portuguese), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; scored against the source model in transformers 5.22.1 on the CPU (English with Whisper's English normalizer, the other European languages with its basic normalizer, Japanese by characters); line = the source model + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": 228,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "conversation.cc:563] Failed to create conversation: INTERNAL: ERROR: [third_party/odml/litert_lm/runtime/executor/audio_litert_compiled_model_executor.cc:483]",
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); the harness with `Engine(bundle, backend=GPU, audio_backend=GPU)`, the first clip's conversation; is empty (0 B: no clip ran)",
            "litert-lm 0.19.1 | granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm (2,846,558,336 B) | backend gpu audio gpu act - | cache (0 before)",
            "window: a quiet machine inside the r1 run, 2026-11-06 21:48:49-21:48:55 JST (the log's first and last lines)",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "ERROR: Following operations are not supported by GPU delegate:",
            "BROADCAST_TO: Operation is not supported.",
            "CONCATENATION: Tensor dimensions must be less than 5. port.GraniteSpeechAudioGraph/port.Encoder_encoder/port.ConformerBlock_0/port.BlockAttention_attn;17",
            "DEQUANTIZE: ",
            "GATHER_ND: Operation is not supported.",
            "MEAN: Expected 1 const input tensor(s), but node has 2 const input(s).",
            "RESHAPE: Tensor dimensions must be less than 5. port.GraniteSpeechAudioGraph/port.Encoder_encoder/port.ConformerBlock_0/port.BlockAttention_attn;17",
            "RESHAPE: Tensor dimensions must be less than 5. port.GraniteSpeechAudioGraph/port.Encoder_encoder/port.ConformerBlock_0/port.BlockAttention_attn;18",
            "RESHAPE: Tensor dimensions must be less than 5. port.GraniteSpeechAudioGraph/port.Encoder_encoder/port.ConformerBlock_0/port.BlockAttention_attn;19",
            "RESHAPE: Tensor dimensions must be less than 5. port.GraniteSpeechAudioGraph/port.Encoder_encoder/port.ConformerBlock_0/port.BlockAttention_attn;20",
            "SELECT_V2: Cond must be float or bool type, if, else tensors must be either be same the shape as output or constant, scalar.",
            "228 operations will run on the GPU, and the remaining 2248 operations will run on the CPU.",
            "ERROR: Hint fully delegated to single delegate is set, but the graph is not fully delegated.",
            "conversation.cc:563] Failed to create conversation: INTERNAL: ERROR: [third_party/odml/litert_lm/runtime/executor/audio_litert_compiled_model_executor.cc:483]",
            "RuntimeError: litert_lm_conversation_create failed with status INTERNAL (13): INTERNAL: ERROR: [third_party/odml/litert_lm/runtime/executor/audio_litert_compiled_model_executor.cc:483]"
          ],
          "failure_class": "engine_create_failed",
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": false,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "ASR with the audio encoder on the GPU too (audio_backend=GPU; the language model on the GPU): conversation creation for the gate's first clip",
          "total_ops": 2476,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 2048,
          "date": "2026-11-06",
          "decode_tokens_per_s": 84.18,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: the PyPI litert-lm 0.19.1 wheel on the Mac (Python 3.13.13; GPU = WebGPU on Metal); `litert-lm benchmark <file> -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --backend gpu --cache no`, one process: a warm-up and 3 runs; the CLI prints the mean of the 3 runs",
            "Benchmarking model: granite-speech-5.0-2b_enc-fp16_lm-int8_act32.litertlm",
            "Names that local file with sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265 = the Hub LFS oid",
            "window: for this run ('bench int8_gpu_act32')",
            "os_build: sw_vers and hw.model read 2026-11-07 at this entry (macOS 27.0, 26A428; Mac16,9); its own records name the host 'Mac M4 Max 128 GB, macOS 27.0'",
            "Prefill speed:        2351.68 tokens/s | Decode speed:         84.18 tokens/s | Init time:            2.0685 s | Time to first token:  0.4473 s",
            "(the run's one error-level line): E0000 00:00:1793968503.853627 82889858 gpu_model_builder.cc:4150] TopK requires src tensor C dimension to be divisible by 4."
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 2.0685,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "runs": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 2351.68,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 447.3
        },
        "source": "data/device_runs/0.19.1/2026-11-06/granite-speech-5.0-2b__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, cpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-cpu-tc (run p11d, 2026-11-07 00:14:34 JST at the run's start, uncapped (thermal 1, no CPU cap, gpu_cooling 0); rested 61s since 00:13:32): 9.9 s clip first text 26.5454 s, end 32.4767 s (5.9 s 21.3003 / 24.9078 s, 29.4 s 32.0403 / 49.8667 s), encoder in-call 23011.1 ms on the 9.9 s clip (mean over the 3 clips 21644.8 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 5205876 kB; samples 89, CPU capped 87, GPU capped 0, thermal max 1; every transcript equal to the Mac reference of the same backend: False",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 1 (prefill_256).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 95.0,
            "clip29_4s_end_s": 49.8667,
            "clip29_4s_first_text_s": 32.0403,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 24.9078,
            "clip5_9s_first_text_s": 21.3003,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 32.4767,
            "clip9_9s_first_text_s": 26.5454,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 23011.1,
            "encoder_in_call_ms_mean_3_clips": 21644.8,
            "first_text_sum_3_clips_s": 79.886,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 1.0,
            "run1_clip9_9s_first_text_s": 26.5454,
            "run1_cpu_capped_samples": 87.0,
            "run1_samples": 89.0,
            "run1_vmhwm_kb": 5205876.0
          },
          "output_match": null,
          "peak_mem_mb": 5330.817,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the CPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 26545.4
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, cpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-cpu-g0 (p11a, 2026-11-06 21:54 JST): en 2.208 % (source 2.428, line 2.928); n 20, stopped 20, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 19 of 20; set verdict PASS",
            "b32-cpu-g1 (p11a, 2026-11-06 22:04 JST): ja 7.426 % (source 7.261, line 8.261); eu 2.027 % (source 2.027, line 3.027); n 18, stopped 18, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 14 of 18; set verdict PASS",
            "Acceptance rule: en20 and xx18 runs PASS; verdict PASS",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 1 (prefill_256).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "en20_empty": 0.0,
            "en20_looping": 0.0,
            "en20_n": 20.0,
            "en20_stopped": 20.0,
            "en20_text_equal_mac": 19.0,
            "en_wer_line_pct": 2.928,
            "en_wer_pct": 2.208,
            "en_wer_source_pct": 2.428,
            "fr_de_es_pt_wer_line_pct": 3.027,
            "fr_de_es_pt_wer_pct": 2.027,
            "fr_de_es_pt_wer_source_pct": 2.027,
            "ja_cer_line_pct": 8.261,
            "ja_cer_pct": 7.426,
            "ja_cer_source_pct": 7.261,
            "xx18_empty": 0.0,
            "xx18_looping": 0.0,
            "xx18_n": 18.0,
            "xx18_stopped": 18.0,
            "xx18_text_equal_mac": 14.0
          },
          "output_match": true,
          "peak_mem_mb": 6064.3779,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu",
          "context_length": 2048,
          "date": "2026-11-06",
          "decode_tokens_per_s": 4.7182,
          "delegated_ops": 2173,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Benchmark(model, cpu, prefill_tokens=1024, decode_tokens=256, max_num_tokens=2048); 3 x run on one engine (no audio backend: the LM alone)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-cpu-t1 (run p11c, 2026-11-06 23:02:03 JST at the run's start, uncapped (thermal 1, no CPU cap, gpu_cooling 0); rested 91s since 23:00:31): runs 2-3 prefill 27.31 / 28.76, decode 4.73 / 4.66 tok/s, TTFT 37.708 / 35.821 s, Init Total 1178.5 ms; VmHWM 2886888 kB (the 1 s sampler), CPU-capped samples per run [51, 0, 67]",
            "b32-cpu-t2 (run p11c, 2026-11-06 23:07:32 JST at the run's start, uncapped (thermal 1, no CPU cap, gpu_cooling 0); rested 90s since 23:06:00): runs 2-3 prefill 27.78 / 35.16, decode 4.76 / 4.73 tok/s, TTFT 37.073 / 29.337 s, Init Total 1372.04 ms; VmHWM 2885368 kB (the 1 s sampler), CPU-capped samples per run [60, 0, 63]",
            "(the same lines in b32-cpu-t2.err):",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1620 out of 1785 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 238 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1450 out of 1616 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 243 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1936 out of 2021 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 83 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 2173 out of 2259 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 82 partitions for subgraph 0 (prefill_1024).",
            "plus one XNNPACK line per composite subgraph: odml.rms_norm 318",
            "delegation: XNNPACK; the record's counts = the last prefill_1024 line (XNNPACK's second pass over the graph): 2173 of 2259 nodes, the rest on the CPU kernels"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_total_ms": 1275.27,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 4.6913,
            "run1_decode_tok_s_it1": 6.4833,
            "run1_init_total_ms": 1178.5,
            "run1_prefill_tok_s": 28.0341,
            "run1_ttft_s": 36.7645,
            "run1_vmhwm_kb": 2886888.0,
            "run2_decode_tok_s": 4.7452,
            "run2_decode_tok_s_it1": 5.8986,
            "run2_init_total_ms": 1372.04,
            "run2_prefill_tok_s": 31.4684,
            "run2_ttft_s": 33.205,
            "run2_vmhwm_kb": 2885368.0,
            "runs": 2.0
          },
          "output_match": null,
          "peak_mem_mb": 2956.1733,
          "prefill_tokens_per_s": 29.7512,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2259,
          "ttft_ms": 34984.75
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, gpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message_async(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-gpu-tc (run p11d, 2026-11-07 00:19:56 JST at the run's start, uncapped (thermal 1, no CPU cap, gpu_cooling 0); rested 82s since 00:18:33): 9.9 s clip first text 23.4854 s, end 27.723 s (5.9 s 22.0598 / 24.9872 s, 29.4 s 26.9514 / 40.0425 s), encoder in-call 21454.2 ms on the 9.9 s clip (mean over the 3 clips 21935.4 ms; rule: encoder in-call ms = a class's mean (call wall - prefill turn s - decode turn s) - the text control's); VmHWM 3824228 kB; samples 95, CPU capped 92, GPU capped 0, thermal max 1; every transcript equal to the Mac reference of the same backend: True",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "calls_per_clip": 2.0,
            "clip29_4s_decode_tokens_per_call": 95.0,
            "clip29_4s_end_s": 40.0425,
            "clip29_4s_first_text_s": 26.9514,
            "clip29_4s_prefill_tokens_per_call": 312.0,
            "clip5_9s_decode_tokens_per_call": 23.0,
            "clip5_9s_end_s": 24.9872,
            "clip5_9s_first_text_s": 22.0598,
            "clip5_9s_prefill_tokens_per_call": 78.0,
            "clip9_9s_decode_tokens_per_call": 32.0,
            "clip9_9s_end_s": 27.723,
            "clip9_9s_first_text_s": 23.4854,
            "clip9_9s_prefill_tokens_per_call": 117.0,
            "encoder_in_call_ms_clip9_9s": 21454.2,
            "encoder_in_call_ms_mean_3_clips": 21935.4,
            "first_text_sum_3_clips_s": 72.4966,
            "max_num_tokens": 4096.0,
            "max_output_tokens": 256.0,
            "processes": 1.0,
            "run1_clip9_9s_first_text_s": 23.4854,
            "run1_cpu_capped_samples": 92.0,
            "run1_samples": 95.0,
            "run1_vmhwm_kb": 3824228.0
          },
          "output_match": null,
          "peak_mem_mb": 3916.0095,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR clip timing: one streaming call (send_message_async) per clip with a new conversation each, after a warm-up clip, for the 5.9, 9.9 and 29.4 s LibriSpeech clips (en00 / en03 / en04 of the en 20) and a text-only control, the test's rendered prompt (`transcribe the speech with proper punctuation and capitalization.`), greedy, at most 256 new tokens; the mean of 2 calls per clip in one process; language model on the GPU, audio encoder on the CPU; ttft_ms = the 9.9 s clip's first streamed text, audio encoding included",
          "total_ops": null,
          "ttft_ms": 23485.4
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-06",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Engine(model, gpu, audio_backend=cpu, max_num_tokens=4096, enable_benchmark=True); per item: render on a throwaway conversation, then create_conversation(SamplerConfig(top_k=1)).send_message(message, max_output_tokens=256)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-gpu-g0 (p11b, 2026-11-06 22:28 JST): en 2.428 % (source 2.428, line 2.928); n 20, stopped 20, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 20 of 20; set verdict PASS",
            "b32-gpu-g1 (p11b, 2026-11-06 22:39 JST): ja 7.261 % (source 7.261, line 8.261); eu 2.027 % (source 2.027, line 3.027); n 18, stopped 18, empty 0, looping 0, transcripts equal to the Mac reference of the same backend 18 of 18; set verdict PASS",
            "Acceptance rule: en20 and xx18 runs PASS; verdict PASS",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 2409 out of 2476 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 75 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "en20_empty": 0.0,
            "en20_looping": 0.0,
            "en20_n": 20.0,
            "en20_stopped": 20.0,
            "en20_text_equal_mac": 20.0,
            "en_wer_line_pct": 2.928,
            "en_wer_pct": 2.428,
            "en_wer_source_pct": 2.428,
            "fr_de_es_pt_wer_line_pct": 3.027,
            "fr_de_es_pt_wer_pct": 2.027,
            "fr_de_es_pt_wer_source_pct": 2.027,
            "ja_cer_line_pct": 8.261,
            "ja_cer_pct": 7.261,
            "ja_cer_source_pct": 7.261,
            "xx18_empty": 0.0,
            "xx18_looping": 0.0,
            "xx18_n": 18.0,
            "xx18_stopped": 18.0,
            "xx18_text_equal_mac": 18.0
          },
          "output_match": true,
          "peak_mem_mb": 3901.3949,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "ASR gate on the phone: 38 of the 220 clips (the 20 LibriSpeech clips, the first 10 Japanese and the first 2 French, German, Spanish and Portuguese FLEURS clips), the instruction `transcribe the speech with proper punctuation and capitalization.`, greedy, at most 256 new tokens, one conversation per clip; line = the source model's rate on the same clips + 0.5 pt (English WER) / + 1.0 pt (Japanese CER) / + 1.0 pt (French + German + Spanish + Portuguese WER), no empty, looping or unstopped reply; audio encoder on the CPU",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 2048,
          "date": "2026-11-07",
          "decode_tokens_per_s": 7.6361,
          "delegated_ops": 1785,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-granite-speech-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/granite-speech-5.0-2b `granite-speech-5.0-2b.litertlm` at main 53f901f0: LFS sha256 c90a3dae17d3399acbce3f9642a308d65bd0214134ce958562dbb2ef90337265, 2846558336 B (Hub API read 2026-11-07 16:18:12 JST); the file was added in Hub commit 64ad8a15 (2026-11-06T16:28:05Z)",
            "runtime: PyPI litert-lm-api 0.19.1 Android wheel on the phone (CPython 3.14.8; liblitert-lm.so sha256 3bc61c88a5998a41cc0eb35c5b133c14e5635790290839452d35cc936231d903), the launcher ltlm_py0191_granite with the run adapter, one process per run: `Benchmark(model, gpu, prefill_tokens=1024, decode_tokens=256, max_num_tokens=2048); 3 x run on one engine (no audio backend: the LM alone)`; protocol: a benchmark script, the phone's state and a ready check before each run",
            "Device: model=Pixel 11 Pro release=17 build_id=CD1A.260905.001.B1 fingerprint=google/grizzly/grizzly:17/CD1A.260905.001.B1/16238327:user/release-keys",
            "b32-gpu-t1 (run p11c, 2026-11-06 23:16:58 JST at the run's start, warm-deadline (thermal 1, cap thermal-uclamp-2=9 thermal-uclamp-6=14 thermal-uclamp-0=13 , gpu_cooling 0 (budget 4614000 of 4614000)); rested 94s since 23:15:23): runs 2-3 prefill 170.16 / 168.96, decode 7.62 / 7.59 tok/s, TTFT 6.149 / 6.192 s, Init Total 18781.69 ms; VmHWM 566532 kB (the 1 s sampler), CPU-capped samples per run [6, 0, 31]",
            "b32-gpu-t2 (run p11d, 2026-11-07 00:11:24 JST at the run's start, clean; rested 18s since 00:11:05): runs 2-3 prefill 173.91 / 173.03, decode 7.67 / 7.67 tok/s, TTFT 6.018 / 6.048 s, Init Total 10254.85 ms; VmHWM 570984 kB (the 1 s sampler), CPU-capped samples per run [5, 1, 30]",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1785 out of 1785 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1616 out of 1616 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 1 (prefill_embedder_1024).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 2 (prefill_embedder_256).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 3 (prefill_embedder_64).",
            "delegation: the four language-model signatures whole on LITERT_CL in 1 partition each; the four embedder signatures run 1 of 4 nodes on XNNPACK, so full_delegation is false; the record's counts = the prefill_1024 line (1785 / 1785)"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_total_ms": 14518.27,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 7.602,
            "run1_decode_tok_s_it1": 7.6436,
            "run1_init_total_ms": 18781.69,
            "run1_prefill_tok_s": 169.5566,
            "run1_ttft_s": 6.1709,
            "run1_vmhwm_kb": 566532.0,
            "run2_decode_tok_s": 7.6702,
            "run2_decode_tok_s_it1": 7.8229,
            "run2_init_total_ms": 10254.85,
            "run2_prefill_tok_s": 173.4696,
            "run2_ttft_s": 6.0335,
            "run2_vmhwm_kb": 570984.0,
            "runs": 2.0
          },
          "output_match": null,
          "peak_mem_mb": 584.6876,
          "prefill_tokens_per_s": 171.5131,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1785,
          "ttft_ms": 6102.2
        },
        "source": "data/device_runs/0.19.1/2026-11-07/granite-speech-5.0-2b__pixel-11-pro.json"
      }
    ]
  },
  "model": {
    "family": "granite-speech",
    "id": "granite-speech-5.0-2b",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/granite-speech-5.0-2b",
    "task": "automatic-speech-recognition"
  },
  "pitfalls": [
    "Keep the audio encoder on the CPU (audio_backend CPU, `--audio-backend cpu`): the Mac's GPU delegate refuses several of its encoder's operations, among them GATHER_ND and 5-dimensional tensors, and creating a conversation then fails with INTERNAL. The language model can run on the CPU or the GPU (card L67, L100).",
    "Context is 4,096 tokens, prompt and reply together: one message fits about 234 s of audio at the highest reply length of the card's 220 test clips, 264 s at their 95th percentile and 307 s at their median; a message that does not fit is refused with INVALID_ARGUMENT, not cut (a 421.6-second clip came to 4,236 tokens) (card L65, L198).",
    "Slower than granite-speech-5.0-2b_ekv2048.litertlm: on a Galaxy S26 the 9.9-second clip reached the first text in 3.83 s against 1.21 s (both with the language model on the GPU, the same session); the card keeps this file for messages of more than about 2 minutes of audio (card L37-L42, L211).",
    "One message holds one audio item, then the instruction as text (for example `transcribe the speech with proper punctuation and capitalization.`); send 16 kHz mono WAV (16-bit PCM and 32-bit float were tested). The encoder always computes a whole 36-second window, also for a short clip (card L63-L64, L195).",
    "Use a new conversation for each clip: from the second message of a conversation on, LiteRT-LM 0.19.1 puts the previous answer's stop token into the context with a zero embedding; not measured (card L102).",
    "With the language model on the CPU, LiteRT-LM quantizes the activations to int8 at run time; a second CPU process can print a slightly different transcript of the same clip (in the card's check of this file, two more commas) (card L197).",
    "The phone figures of 2026-11-06 and of 2026-11-07 differ (first text 4.36 s and 7.47 s on 2026-11-06, 5.16 s and 3.83 s on 2026-11-07, language model on the CPU and the GPU); the phone's throttling differed between the two days and the card did not separate the cause (card L155)."
  ],
  "schema_version": "1.2"
}
