{
  "artifacts": [
    {
      "file": "granite-3.0-1b-a400m-instruct_int8.litertlm",
      "sha256": "b0b786a5672c25d4aeb8ed90b7363294703809a1f95a5dfb8fb3dcf5db11d39f",
      "size_mb": 1309.152
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "ibm-granite/granite-3.0-1b-a400m-instruct rev ffec3c35 -> export with dynamic_wi8_afp32 (prefill signatures 128 and 1024 plus decode over a 2048-position KV cache; the checkpoint's tokenizer.json packed byte for byte; IBM's template with three additions for litert-lm 0.18; PYTHONHASHSEED=0), the experts as two FCs per layer (gate_up [32768, 1024], down [1024, 16384], each expert's routing weight multiplied into its slice), the decode logits padded to 49,156 columns with one -1e4 column -> scripts/set_activation_type.py --type fp32 (prefer_activation_type on the TFLite section; every section's bytes unchanged) (card L113-L117)",
    "quantization": "int8 dynamic on linears, float activations; each layer's 32 experts as two fully connected layers; fp32 GPU activations declared (card L25)",
    "tool": "litert-torch at git commit 0d3869c, `litert_moe_sequential` route with each layer's 32 expert FCs fused into two FCs; repro = hf-to-litertlm granitemoe_work/convert_granite30_1b_a400m.sh (card L113)",
    "tool_version": "litert-torch 0d3869c, transformers 5.20.1, torch 2.14.0, ai-edge-quantizer-nightly 0.10.0.dev20261019, litert-converter 0.6.0.dev20261016, ai-edge-litert-nightly 2.4.0.dev20261018; litert-lm 0.18.0 for the activation-type step (card L113)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 23.255,
          "delegated_ops": 1499,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-moe4",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold s1: HOLD_SEEN 2026-10-21 12:21:49: acquired 12:21:47, skip_after 12:46:47, run_end 12:49:47, released by 12:51:47; keeper released the hold at 2026-10-21 12:36:50",
            "(s1, 10-21 12:28:46.000 device time, clean; rested 243s since 12:24:43): iterations 2-3 prefill 369.38 / 341.0, decode 22.82 / 22.75 tok/s, TTFT 2.82 / 3.05 s, Init Total 85.88 ms; .adb.txt EXIT=0 WALL=40 START=1792553326 END=1792553366; .meta.json memory 'VmHWM_max_kB 2053544 VmSwap_max_kB 0 MemAvailable_min_kB 6467660 kgsl_page_alloc_max_B 189702144'",
            "(s1, 10-21 12:33:31.000 device time, clean; rested 243s since 12:29:28): iterations 2-3 prefill 392.43 / 343.76, decode 24.18 / 23.27 tok/s, TTFT 2.65 / 3.02 s, Init Total 83.39 ms; .adb.txt EXIT=0 WALL=38 START=1792553611 END=1792553649; .meta.json memory 'VmHWM_max_kB 2053984 VmSwap_max_kB 0 MemAvailable_min_kB 6420920 kgsl_page_alloc_max_B 189702144'",
            "command: --backend=cpu --model_path=/data/local/tmp/moe4_r5/f8pt32.litertlm --input_prompt_file=/data/local/tmp/moe4_r5/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=256 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 1170 out of 1365 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 189 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1066 out of 1266 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 196 partitions for subgraph 2 (decode).",
            "Replacing 1499 out of 1647 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 95 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1360 out of 1511 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 98 partitions for subgraph 2 (decode).",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cold_init1_ms": 1781.2,
            "decode_tokens": 256.0,
            "init_total_ms": 84.635,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cpu_capped_samples": 17.0,
            "run1_decode_tok_s": 22.785,
            "run1_decode_tok_s_it1": 27.62,
            "run1_decode_tok_s_it2": 22.82,
            "run1_decode_tok_s_it3": 22.75,
            "run1_init_total_ms": 85.88,
            "run1_memavail_min_kb": 6467660.0,
            "run1_prefill_tok_s": 355.19,
            "run1_sampler_samples": 18.0,
            "run1_t_s": 14.1184,
            "run1_ttft_s": 2.935,
            "run1_vmhwm_kb": 2053544.0,
            "run2_cpu_capped_samples": 16.0,
            "run2_decode_tok_s": 23.725,
            "run2_decode_tok_s_it1": 28.78,
            "run2_decode_tok_s_it2": 24.18,
            "run2_decode_tok_s_it3": 23.27,
            "run2_init_total_ms": 83.39,
            "run2_memavail_min_kb": 6420920.0,
            "run2_prefill_tok_s": 368.095,
            "run2_sampler_samples": 17.0,
            "run2_t_s": 13.5722,
            "run2_ttft_s": 2.835,
            "run2_vmhwm_kb": 2053984.0,
            "runs": 2.0,
            "t_1024_256_s": 13.8399
          },
          "output_match": null,
          "peak_mem_mb": 2103.054,
          "prefill_tokens_per_s": 361.6425,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1647,
          "ttft_ms": 2885.0
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 27.285,
          "delegated_ops": 1365,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-moe4",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold s2: HOLD_SEEN 2026-10-21 21:00:33: acquired 21:00:31, skip_after 21:25:31, run_end 21:28:31, released by 21:30:31; keeper released the hold at 2026-10-21 21:24:39",
            "hold s3: HOLD_SEEN 2026-10-21 21:57:42: acquired 21:57:42, skip_after 22:22:42, run_end 22:25:42, released by 22:27:42; keeper released the hold at 2026-10-21 22:08:18",
            "(s2, 10-21 21:23:56.000 device time, clean; rested 243s since 21:19:52): iterations 2-3 prefill 343.56 / 359.05, decode 27.51 / 27.35 tok/s, TTFT 3.02 / 2.89 s, Init Total 1520.6 ms; .adb.txt EXIT=0 WALL=39 START=1792585436 END=1792585475; .meta.json memory 'VmHWM_max_kB 525092 VmSwap_max_kB 0 MemAvailable_min_kB 4541092 kgsl_page_alloc_max_B 2664108032'",
            "(s3, 10-21 22:02:50.000 device time, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 254s since 21:58:36): iterations 2-3 prefill 359.66 / 346.13, decode 27.4 / 26.81 tok/s, TTFT 2.88 / 3.0 s, Init Total 1725.24 ms; .adb.txt EXIT=0 WALL=40 START=1792587770 END=1792587810; .meta.json memory 'VmHWM_max_kB 526484 VmSwap_max_kB 0 MemAvailable_min_kB 4637916 kgsl_page_alloc_max_B 2678472704'",
            "(s3, 10-21 22:07:33.000 device time, uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 243s since 22:03:30): iterations 2-3 prefill 335.07 / 358.34, decode 27.54 / 27.03 tok/s, TTFT 3.09 / 2.89 s, Init Total 1637.54 ms; .adb.txt EXIT=0 WALL=39 START=1792588053 END=1792588092; .meta.json memory 'VmHWM_max_kB 526756 VmSwap_max_kB 0 MemAvailable_min_kB 4589276 kgsl_page_alloc_max_B 2678497280'",
            "(s2, decode 64, clean; rested 243s since 21:06:29): prefill 331.27 / decode 26.805 tok/s (iterations 2-3), GPU thermal level above 0 in 0 of 9 samples",
            "(s2, decode 64, clean; rested 243s since 21:10:52): prefill 343.42 / decode 26.795 tok/s (iterations 2-3), GPU thermal level above 0 in 0 of 9 samples",
            "command: --backend=gpu --model_path=/data/local/tmp/moe4_r5/f8pt32.litertlm --input_prompt_file=/data/local/tmp/moe4_r5/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=256 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 1365 out of 1365 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_128).",
            "Replacing 1365 out of 1365 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1266 out of 1266 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (decode).",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cold_init1_ms": 4268.0,
            "cold_init2_ms": 2794.36,
            "cold_init3_ms": 4862.78,
            "d64_run1_decode_tok_s": 26.805,
            "d64_run1_gpu_thermal_samples": 9.0,
            "d64_run1_gpu_thermal_samples_above_0": 0.0,
            "d64_run1_prefill_tok_s": 331.27,
            "d64_run1_ttft_s": 3.13,
            "d64_run2_decode_tok_s": 26.795,
            "d64_run2_gpu_thermal_samples": 9.0,
            "d64_run2_gpu_thermal_samples_above_0": 0.0,
            "d64_run2_prefill_tok_s": 343.42,
            "d64_run2_ttft_s": 3.025,
            "decode_tokens": 256.0,
            "init_total_ms": 1637.54,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cpu_capped_samples": 4.0,
            "run1_decode_tok_s": 27.43,
            "run1_decode_tok_s_it1": 26.86,
            "run1_decode_tok_s_it2": 27.51,
            "run1_decode_tok_s_it3": 27.35,
            "run1_gpu_thermal_samples": 18.0,
            "run1_gpu_thermal_samples_above_0": 2.0,
            "run1_init_total_ms": 1520.6,
            "run1_memavail_min_kb": 4541092.0,
            "run1_prefill_tok_s": 351.305,
            "run1_sampler_samples": 18.0,
            "run1_t_s": 12.2477,
            "run1_ttft_s": 2.955,
            "run1_vmhwm_kb": 525092.0,
            "run2_cpu_capped_samples": 9.0,
            "run2_decode_tok_s": 27.105,
            "run2_decode_tok_s_it1": 26.93,
            "run2_decode_tok_s_it2": 27.4,
            "run2_decode_tok_s_it3": 26.81,
            "run2_gpu_thermal_samples": 18.0,
            "run2_gpu_thermal_samples_above_0": 9.0,
            "run2_init_total_ms": 1725.24,
            "run2_memavail_min_kb": 4637916.0,
            "run2_prefill_tok_s": 352.895,
            "run2_sampler_samples": 18.0,
            "run2_t_s": 12.3465,
            "run2_ttft_s": 2.94,
            "run2_vmhwm_kb": 526484.0,
            "run3_cpu_capped_samples": 9.0,
            "run3_decode_tok_s": 27.285,
            "run3_decode_tok_s_it1": 26.85,
            "run3_decode_tok_s_it2": 27.54,
            "run3_decode_tok_s_it3": 27.03,
            "run3_gpu_thermal_samples": 18.0,
            "run3_gpu_thermal_samples_above_0": 7.0,
            "run3_init_total_ms": 1637.54,
            "run3_memavail_min_kb": 4589276.0,
            "run3_prefill_tok_s": 346.705,
            "run3_sampler_samples": 18.0,
            "run3_t_s": 12.336,
            "run3_ttft_s": 2.99,
            "run3_vmhwm_kb": 526756.0,
            "runs": 3.0,
            "t_1024_256_s": 12.2973
          },
          "output_match": null,
          "peak_mem_mb": 539.12,
          "prefill_tokens_per_s": 351.305,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1365,
          "ttft_ms": 2955.0
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 53.05,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Prefill 617.89 / decode 53.11 tok/s, TTFT 1.6761 s, init 4.3239 s; maximum resident set size 6289260544 B, peak memory footprint 4700247584 B",
            "Spd cpu B1 granite30_1b_fused_wi8_pad_tpl_fp32act exit 0 wall 18 s | Prefill speed: 617.89 tokens/s Decode speed: 53.11 tokens/s Init time: 4.3239 s Time to first token: 1.6761 s  |  6289260544 maximum resident set size  4700247584 peak memory footprint  | topk 0 |",
            "Prefill 617.85 / decode 52.99 tok/s, TTFT 1.6762 s, init 4.3118 s; maximum resident set size 6297157632 B, peak memory footprint 4701132320 B",
            "Spd cpu B2 granite30_1b_fused_wi8_pad_tpl_fp32act exit 0 wall 19 s | Prefill speed: 617.85 tokens/s Decode speed: 52.99 tokens/s Init time: 4.3118 s Time to first token: 1.6762 s  |  6297157632 maximum resident set size  4701132320 peak memory footprint  | topk 0 |",
            "ABBA cpu mean A(F8pt) / B(F8pt32): prefill 619.83 / 617.87 (x 0.9968), decode 53.120 / 53.050 (x 0.9987), t 6.4714 / 6.4830 (x 1.0018), max RSS 6295822336 / 6293209088 B (x 0.9996), peak footprint 4705457696 / 4700689952 B (x 0.9990)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 4.3178,
            "iterations": 1.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 53.11,
            "run1_init_s": 4.3239,
            "run1_max_rss_mb": 6289.261,
            "run1_peak_footprint_mb": 4700.248,
            "run1_prefill_tok_s": 617.89,
            "run1_t_s": 6.4774,
            "run1_ttft_s": 1.6761,
            "run2_decode_tok_s": 52.99,
            "run2_init_s": 4.3118,
            "run2_max_rss_mb": 6297.158,
            "run2_peak_footprint_mb": 4701.132,
            "run2_prefill_tok_s": 617.85,
            "run2_t_s": 6.4885,
            "run2_ttft_s": 1.6762,
            "runs": 2.0,
            "t_1024_256_s": 6.4829
          },
          "output_match": null,
          "peak_mem_mb": 4700.69,
          "prefill_tokens_per_s": 617.87,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 1676.1
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 146.34,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Prefill 3245.07 / decode 146.53 tok/s, TTFT 0.3224 s, init 1.3254 s; maximum resident set size 1746731008 B, peak memory footprint 2552350832 B",
            "spd2 gpu B1 granite30_1b_fused_wi8_pad_tpl_fp32act exit 0 wall 6 s | Prefill speed: 3245.07 tokens/s Decode speed: 146.53 tokens/s Init time: 1.3254 s Time to first token: 0.3224 s  |  1746731008 maximum resident set size  2552350832 peak memory footprint  | verr 0",
            "Prefill 3228.7 / decode 146.15 tok/s, TTFT 0.324 s, init 1.2916 s; maximum resident set size 1714618368 B, peak memory footprint 2545076312 B",
            "spd2 gpu B2 granite30_1b_fused_wi8_pad_tpl_fp32act exit 0 wall 5 s | Prefill speed: 3228.70 tokens/s Decode speed: 146.15 tokens/s Init time: 1.2916 s Time to first token: 0.3240 s  |  1714618368 maximum resident set size  2545076312 peak memory footprint  | verr 0",
            "VALID gpu mean over the readings with 0 Validation error lines (A 2, B 2; void: none): prefill 3931.10 / 3236.89 (x 0.8234), decode 154.705 / 146.340 (x 0.9459), t 1.9152 / 2.0657 (x 1.0786), max RSS 1750228992 / 1730674688 B (x 0.9888), peak footprint 2157209712 / 2548713572 B (x 1.1815)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 1.3085,
            "iterations": 1.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 146.53,
            "run1_init_s": 1.3254,
            "run1_max_rss_mb": 1746.731,
            "run1_peak_footprint_mb": 2552.351,
            "run1_prefill_tok_s": 3245.07,
            "run1_t_s": 2.0626,
            "run1_ttft_s": 0.3224,
            "run2_decode_tok_s": 146.15,
            "run2_init_s": 1.2916,
            "run2_max_rss_mb": 1714.618,
            "run2_peak_footprint_mb": 2545.076,
            "run2_prefill_tok_s": 3228.7,
            "run2_t_s": 2.0688,
            "run2_ttft_s": 0.324,
            "runs": 2.0,
            "t_1024_256_s": 2.0657
          },
          "output_match": null,
          "peak_mem_mb": 2548.714,
          "prefill_tokens_per_s": 3236.885,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 323.2
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 12.8175,
          "delegated_ops": 1499,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-moe4",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Google Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold p1: HOLD_SEEN 2026-10-21 11:48:23: acquired 11:48:21, skip_after 12:13:21, run_end 12:16:21, released by 12:18:21; keeper released the hold at 2026-10-21 12:02:53",
            "(p1, 10-21 11:53:39.000 device time, clean; rested 124s since 11:51:34): iterations 2-3 prefill 142.17 / 122.42, decode 14.31 / 12.85 tok/s, TTFT 7.27 / 8.44 s, Init Total 377.7 ms; .adb.txt EXIT=0 WALL=78 START=1792551219 END=1792551297; .meta.json memory 'VmHWM_max_kB 2023768 VmSwap_max_kB 0 MemAvailable_min_kB 3572320 mali_total_gpu_mem_max_B 255418368'",
            "(p1, 10-21 11:57:02.000 device time, clean; rested 123s since 11:54:57): iterations 2-3 prefill 138.75 / 120.06, decode 13.63 / 10.48 tok/s, TTFT 7.45 / 8.62 s, Init Total 295.22 ms; .adb.txt EXIT=0 WALL=84 START=1792551422 END=1792551506; .meta.json memory 'VmHWM_max_kB 2022632 VmSwap_max_kB 0 MemAvailable_min_kB 3560168 mali_total_gpu_mem_max_B 254693376'",
            "command: --backend=cpu --model_path=/data/local/tmp/moe4_r5/f8pt32.litertlm --input_prompt_file=/data/local/tmp/moe4_r5/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=256 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 1170 out of 1365 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 189 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1066 out of 1266 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 196 partitions for subgraph 2 (decode).",
            "Replacing 1499 out of 1647 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 95 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1360 out of 1511 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 98 partitions for subgraph 2 (decode).",
            "Device: model=Pixel 8a release=16 build_id=CP1A.260505.005 fingerprint=google/akita/akita:16/CP1A.260505.005/15081906:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cold_init1_ms": 3444.77,
            "decode_tokens": 256.0,
            "init_total_ms": 336.46,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cpu_capped_samples": 9.0,
            "run1_decode_tok_s": 13.58,
            "run1_decode_tok_s_it1": 15.41,
            "run1_decode_tok_s_it2": 14.31,
            "run1_decode_tok_s_it3": 12.85,
            "run1_init_total_ms": 377.7,
            "run1_memavail_min_kb": 3572320.0,
            "run1_prefill_tok_s": 132.295,
            "run1_sampler_samples": 25.0,
            "run1_t_s": 26.5915,
            "run1_ttft_s": 7.855,
            "run1_vmhwm_kb": 2023768.0,
            "run2_cpu_capped_samples": 12.0,
            "run2_decode_tok_s": 12.055,
            "run2_decode_tok_s_it1": 15.17,
            "run2_decode_tok_s_it2": 13.63,
            "run2_decode_tok_s_it3": 10.48,
            "run2_init_total_ms": 295.22,
            "run2_memavail_min_kb": 3560168.0,
            "run2_prefill_tok_s": 129.405,
            "run2_sampler_samples": 27.0,
            "run2_t_s": 29.1491,
            "run2_ttft_s": 8.035,
            "run2_vmhwm_kb": 2022632.0,
            "runs": 2.0,
            "t_1024_256_s": 27.7984
          },
          "output_match": null,
          "peak_mem_mb": 2071.757,
          "prefill_tokens_per_s": 130.85,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1647,
          "ttft_ms": 7945.0
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__pixel-8a.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-21",
          "decode_tokens_per_s": 12.5825,
          "delegated_ops": 1365,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-moe4",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Google Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold p2: HOLD_SEEN 2026-10-21 12:25:31: acquired 12:25:29, skip_after 12:50:29, run_end 12:53:29, released by 12:55:29; keeper released the hold at 2026-10-21 12:45:39",
            "(p2, 10-21 12:35:12.000 device time, clean; rested 180s since 12:32:10): iterations 2-3 prefill 128.54 / 128.27, decode 12.64 / 12.62 tok/s, TTFT 8.05 / 8.06 s, Init Total 4731.05 ms; .adb.txt EXIT=0 WALL=91 START=1792553712 END=1792553803; .meta.json memory 'VmHWM_max_kB 1331700 VmSwap_max_kB 0 MemAvailable_min_kB 1811700 mali_total_gpu_mem_max_B 2560761856'",
            "(p2, 10-21 12:39:48.000 device time, clean; rested 183s since 12:36:43): iterations 2-3 prefill 128.06 / 128.53, decode 12.59 / 12.48 tok/s, TTFT 8.08 / 8.05 s, Init Total 3484.25 ms; .adb.txt EXIT=0 WALL=90 START=1792553988 END=1792554078; .meta.json memory 'VmHWM_max_kB 1331380 VmSwap_max_kB 0 MemAvailable_min_kB 1794184 mali_total_gpu_mem_max_B 2560761856'",
            "command: --backend=gpu --model_path=/data/local/tmp/moe4_r5/f8pt32.litertlm --input_prompt_file=/data/local/tmp/moe4_r5/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=256 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 1365 out of 1365 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_128).",
            "Replacing 1365 out of 1365 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_1024).",
            "Replacing 1266 out of 1266 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (decode).",
            "Device: model=Pixel 8a release=16 build_id=CP1A.260505.005 fingerprint=google/akita/akita:16/CP1A.260505.005/15081906:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cold_init1_ms": 30007.55,
            "cold_init2_ms": 24797.23,
            "decode_tokens": 256.0,
            "init_total_ms": 4107.65,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cpu_capped_samples": 4.0,
            "run1_decode_tok_s": 12.63,
            "run1_decode_tok_s_it1": 12.48,
            "run1_decode_tok_s_it2": 12.64,
            "run1_decode_tok_s_it3": 12.62,
            "run1_gpu_thermal_samples": 7.0,
            "run1_gpu_thermal_samples_above_0": 0.0,
            "run1_init_total_ms": 4731.05,
            "run1_memavail_min_kb": 1811700.0,
            "run1_prefill_tok_s": 128.405,
            "run1_sampler_samples": 31.0,
            "run1_t_s": 28.244,
            "run1_ttft_s": 8.055,
            "run1_vmhwm_kb": 1331700.0,
            "run2_cpu_capped_samples": 7.0,
            "run2_decode_tok_s": 12.535,
            "run2_decode_tok_s_it1": 12.42,
            "run2_decode_tok_s_it2": 12.59,
            "run2_decode_tok_s_it3": 12.48,
            "run2_gpu_thermal_samples": 7.0,
            "run2_gpu_thermal_samples_above_0": 1.0,
            "run2_init_total_ms": 3484.25,
            "run2_memavail_min_kb": 1794184.0,
            "run2_prefill_tok_s": 128.295,
            "run2_sampler_samples": 31.0,
            "run2_t_s": 28.4044,
            "run2_ttft_s": 8.065,
            "run2_vmhwm_kb": 1331380.0,
            "runs": 2.0,
            "t_1024_256_s": 28.3239
          },
          "output_match": null,
          "peak_mem_mb": 1363.497,
          "prefill_tokens_per_s": 128.35,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1365,
          "ttft_ms": 8060.0
        },
        "source": "data/device_runs/0.18.0/2026-10-21/granite-3.0-1b-a400m-instruct-int8__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "granite",
    "id": "granite-3.0-1b-a400m-instruct-int8",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/granite-3.0-1b-a400m-instruct",
    "task": "text-generation"
  },
  "pitfalls": [
    "Dense cost: litert-torch's sparse litert_moe export computes the experts with GELU-tanh while this model's experts use SiLU, and LiteRT-LM 0.18.0's GPU refuses the moe op it writes, so the file takes the litert_moe_sequential route: all 32 experts of a layer for every token with the model's SiLU, a token costs the compute of all 1.33B parameters, not 400M (card L27).",
    "GPU activations: the file declares fp32 GPU activations. With the runtime's default fp16 activations the same weights scored GSM8K 28 / 29 / 29 on the Mac GPU against 35 / 35 / 35 with the declaration (bf16 34); the declaration costs the Mac GPU 5.4 % in decode and 17.7 % in prefill (card L50-L57).",
    "Phones' GPUs: against the same weights with fp16 activations the declaration prefills at 0.330x (Galaxy S26, 64 decoded tokens) and 0.496x (Pixel 8a) and decodes at 0.841x and 0.873x; no fp32 copy of the weights (Pixel 8a weight cache 1,342,110,512 vs 1,338,374,960 bytes) (card L107).",
    "Galaxy S26 GPU: the clock ceiling fell from 1,300 MHz during every 256-token run, so the GPU rates include throttled iterations; no unthrottled 256-token run was obtained. The GPU still took less time than the CPU in every run; on the Pixel 8a the CPU took 1.6 % less on average (card L36, L105).",
    "Decode logits padded to 49,156 columns: without it every GPU run printed 'TopK requires src tensor C dimension to be divisible by 4.' and, where the CPU began a reply with <|tool_call|> (id 49154), the GPU began with other text; with it the GPU begins those replies with <|tool_call|> (card L115).",
    "Tool calling: the bundle's model type (generic_model) has no tool-call parser, so a call arrives as reply text, <|tool_call|> followed by a JSON list; in the tests the model started a call only when the history already held one earlier round trip (card L72).",
    "Template: IBM's chat template with three additions for litert-lm 0.18 (string or part-list content, a tools block written by a macro that gives the bytes of tojson(indent=4), a tool role as IBM's tool_response turn); no start token, the stop token <|end_of_text|> (id 0) plus 14 strings from the template's turn ending; the bundle sets no sampler (card L70).",
    "Cache: without --cache, `run` writes an XNNPACK weight cache next to the model (1,346,661,944 bytes); without --backend, `run` uses the CPU (card L68).",
    "Accuracy: GSM8K questions 0-99 (greedy, at most 1024 output tokens, litert-lm 0.18.0 Python API): this file 35 / 35 / 35 on the Mac GPU, the same weights 39 / 35 on the Mac CPU, bf16 34; the model wrote the '#### <number>' line in none of the replies; 8-question check 8/8 on the Mac CPU and GPU, GPU 8/8 and CPU 7/8 on the Galaxy S26 and the Pixel 8a (card L40-L57).",
    "Not measured on iOS; no NPU row: the file is a dynamic-range int8 graph and was not tried on an NPU (card L109)."
  ],
  "schema_version": "1.2"
}
