{
  "artifacts": [
    {
      "file": "LFM2.5-350M_int8.litertlm",
      "sha256": "6a78c7a2746386f8919476a34ef0ca40d596a5c44e979520fae8318da5cfb1f6",
      "size_mb": 384.728
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "LiquidAI/LFM2.5-350M rev 9e6c6ccf: unquantized export (11 prefill signatures, 1024 down to 1, plus decode, over a 4096-position KV cache; 22 state buffers: 10 ShortConv states, 6 key and 6 value caches) -> post-hoc int8 quantization -> metadata repack with LiteRT-LM's lfm2 configuration (card L110-L111)",
    "quantization": "int8 dynamic on linears + int8 embedding, convs float (card L27)",
    "tool": "litert-torch 0.9.4 (released wheels); repro = hf-to-litertlm lfm_work/convert_lfm25_350m.sh (card L110)",
    "tool_version": "litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, ai-edge-litert 2.2.0, transformers 5.14.1, torch 2.13.0 (released wheels); packed by the litert-lm 0.18.0 CLI"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 80.31,
          "delegated_ops": 788,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-aab1",
            "os_build": "Android 16",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r1 hold 2026-10-12 15:55:45-16:03:26 JST: [15:55:59] bundle on device: 6a78c7a2746386f8919476a34ef0ca40d596a5c44e979520fae8318da5cfb1f6 (403416064 B, push 10.6 s) = Mac",
            "bench args: --backend=cpu --model_path=/data/local/tmp/aa1_350m/LFM2.5-350M_int8.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa1_350m/metrics.pb",
            "VERBOSE: Replacing 487 out of 560 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 88 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 455 out of 519 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 92 partitions for subgraph 11 (decode).",
            "VERBOSE: Replacing 788 out of 818 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 14 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 737 out of 756 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 14 partitions for subgraph 11 (decode).",
            "[15:56:57] parsed: s26_int8_cpu_timing cpu: gate PASS 1/1 | bench prefill 1752.69 decode 80.31 init 121.87 cold 517.5 peak 1274.03 capped 3/6 | answers: 'Paris'",
            "timing run prompt process: \"Paris\"",
            "check run: 2 of 2 prompt processes answered with the expected word (\"Paris\")"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791590918.51,
            "capped_samples": 3.0,
            "cold_init_ms": 517.5,
            "cpu_capped_samples": 3.0,
            "decode_tok_s_it1": 89.63,
            "decode_tok_s_it2": 87.28,
            "decode_tok_s_it3": 73.34,
            "decode_tokens": 256.0,
            "init_ms": 121.87,
            "iterations": 3.0,
            "kgsl_ceiling_min_mhz": 1300.0,
            "kgsl_temp_max_c": 65.5,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 7402320.0,
            "memavail_min_kib": 6532944.0,
            "prefill_tok_s_it1": 1503.39,
            "prefill_tok_s_it2": 1871.17,
            "prefill_tok_s_it3": 1634.21,
            "prefill_tokens": 1024.0,
            "sampler_samples": 6.0,
            "skin_c_at_bench_start": 37.9,
            "t_1024_256_s": 3.7719,
            "thermal_status_at_bench_start": 0.0,
            "ttft_s_it1": 0.69,
            "ttft_s_it2": 0.56,
            "ttft_s_it3": 0.64
          },
          "output_match": null,
          "peak_mem_mb": 1274.03,
          "prefill_tokens_per_s": 1752.69,
          "provenance": "measured",
          "runs": true,
          "total_ops": 818,
          "ttft_ms": 600.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-12/lfm2.5-350m-int8__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 92.895,
          "delegated_ops": 560,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-aab1",
            "os_build": "Android 16",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r1 hold 2026-10-12 15:55:45-16:03:26 JST: [15:58:59] bundle on device: 6a78c7a2746386f8919476a34ef0ca40d596a5c44e979520fae8318da5cfb1f6 (403416064 B, push 0 s) = Mac",
            "bench args: --backend=gpu --model_path=/data/local/tmp/aa1_350m/LFM2.5-350M_int8.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa1_350m/metrics.pb",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_512).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_256).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_128).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 4 (prefill_64).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 5 (prefill_32).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 6 (prefill_16).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 7 (prefill_8).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 8 (prefill_4).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 9 (prefill_2).",
            "VERBOSE: Replacing 501 out of 501 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 10 (prefill_1).",
            "VERBOSE: Replacing 519 out of 519 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 11 (decode).",
            "[16:00:20] parsed: s26_int8_gpu_timing gpu: gate PASS 1/1 | bench prefill 5660.615 decode 92.89500000000001 init 951.31 cold 4071.79 peak 506.672 capped 0/5 | answers: 'Paris'",
            "timing run prompt process: \"Paris\"",
            "check run: 2 of 2 prompt processes answered with the expected word (\"Paris\")"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791590918.98,
            "capped_samples": 0.0,
            "cold_init_ms": 4071.79,
            "cpu_capped_samples": 3.0,
            "decode_tok_s_it1": 85.68,
            "decode_tok_s_it2": 93.01,
            "decode_tok_s_it3": 92.78,
            "decode_tokens": 256.0,
            "init_ms": 951.31,
            "iterations": 3.0,
            "kgsl_ceiling_min_mhz": 1300.0,
            "kgsl_temp_max_c": 67.8,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 7451088.0,
            "memavail_min_kib": 6136340.0,
            "prefill_tok_s_it1": 5835.03,
            "prefill_tok_s_it2": 5761.73,
            "prefill_tok_s_it3": 5559.5,
            "prefill_tokens": 1024.0,
            "sampler_samples": 5.0,
            "skin_c_at_bench_start": 37.9,
            "t_1024_256_s": 2.9367,
            "thermal_status_at_bench_start": 0.0,
            "ttft_s_it1": 0.19,
            "ttft_s_it2": 0.19,
            "ttft_s_it3": 0.19
          },
          "output_match": null,
          "peak_mem_mb": 506.672,
          "prefill_tokens_per_s": 5660.615,
          "provenance": "measured",
          "runs": true,
          "total_ops": 560,
          "ttft_ms": 190.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-12/lfm2.5-350m-int8__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "It seems like your message might be incomplete. Could you please provide more context or clarify what you're asking? For example:",
          "evidence": [
            "command: litert-lm run LFM2.5-350M_int8.litertlm --prompt \"What is the capital of France?\" --cache no < /dev/null",
            "exit 0",
            "stdout: It seems like your message might be incomplete. Could you please provide more context or clarify what you're asking? For example: - Are you referring to a specific topic (e.g., science, technology, philosophy)? - Do you need help with something like writing, problem-solving, or research? - Is this related to a project, query, or idea? Let me know, and I’ll do my best to assist! 😊",
            "control, the same command on litert-lm 0.18.0: \"The capital of France is Paris.\"",
            "prefill probe (Python API, CPU): engine_prefill_tokens 9, reading 'empty user turn', reply \"It seems like your message might be incomplete or unclear. Could you please provide more\"",
            "0.18.0: engine_prefill_tokens 16, reading 'full user turn', reply \"The capital of France is Paris.\"",
            "HF reference (transformers 5.14.1): full user turn 16 ids, empty user turn 9 ids (\"<|startoftext|><|im_start|>user\\n<|im_end|>\\n<|im_start|>assistant\\n\")"
          ],
          "failure_class": "degenerate_output",
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "engine_prefill_tokens": 9.0,
            "engine_prefill_tokens_0180": 16.0,
            "hf_empty_user_turn_tokens": 9.0,
            "hf_full_user_turn_tokens": 16.0,
            "processes": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-12/lfm2.5-350m-int8__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 134.0,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (tools/bench_mac.sh): litert-lm benchmark LFM2.5-350M_int8.litertlm --backend cpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "Benchmarking model: LFM2.5-350M_int8.litertlm | Backend                    : cpu | Cache                      : no | Prefill speed:        2200.63 tokens/s | Decode speed:         134.00 tokens/s | Init time:            3.7308 s | Time to first token:  0.4730 s",
            "2026-10-12 15:31:20 JST cpu int8 exit 0"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 3.7308,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "passes": 1.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 2.3758
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 2200.63,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 473.0
        },
        "source": "data/device_runs/0.18.0/2026-10-12/lfm2.5-350m-int8__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 423.73,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (tools/bench_mac.sh): litert-lm benchmark LFM2.5-350M_int8.litertlm --backend gpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "Benchmarking model: LFM2.5-350M_int8.litertlm | Backend                    : gpu | Cache                      : no | Prefill speed:        13556.05 tokens/s | Decode speed:         423.73 tokens/s | Init time:            1.0924 s | Time to first token:  0.0780 s",
            "2026-10-12 15:36:58 JST gpu int8 exit 0",
            "2026-10-12 15:37:12 JST gpu int8 verbose exit 0 validation_error=0 shape_mismatch=0 replacing=24"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 1.0924,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "passes": 1.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 0.6797
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 13556.05,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 78.0
        },
        "source": "data/device_runs/0.18.0/2026-10-12/lfm2.5-350m-int8__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 41.81,
          "delegated_ops": 788,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-aab1",
            "os_build": "Android 16",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r1 hold 2026-10-12 15:55:45-16:00:35 JST: [15:56:56] bundle on device: 6a78c7a2746386f8919476a34ef0ca40d596a5c44e979520fae8318da5cfb1f6 (403416064 B, push 0 s) = Mac",
            "bench args: --backend=cpu --model_path=/data/local/tmp/aa1_350m/LFM2.5-350M_int8.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa1_350m/metrics.pb",
            "VERBOSE: Replacing 487 out of 560 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 88 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 455 out of 519 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 92 partitions for subgraph 11 (decode).",
            "VERBOSE: Replacing 788 out of 818 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 14 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 737 out of 756 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 14 partitions for subgraph 11 (decode).",
            "[15:57:46] parsed: p8a_int8_cpu_timing cpu: gate PASS 1/1 | bench prefill 509.05999999999995 decode 41.81 init 462.47 cold 770.64 peak 979.012 capped 0/9 | answers: 'Paris'",
            "timing run prompt process: \"Paris\"",
            "check run: 1 of 1 prompt processes answered with the expected word (\"Paris\")"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791442100.29,
            "capped_samples": 0.0,
            "cold_init_ms": 770.64,
            "cpu_capped_samples": 0.0,
            "decode_tok_s_it1": 43.39,
            "decode_tok_s_it2": 42.28,
            "decode_tok_s_it3": 41.34,
            "decode_tokens": 256.0,
            "init_ms": 462.47,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 3817096.0,
            "memavail_min_kib": 3259880.0,
            "prefill_tok_s_it1": 471.48,
            "prefill_tok_s_it2": 515.18,
            "prefill_tok_s_it3": 502.94,
            "prefill_tokens": 1024.0,
            "sampler_samples": 9.0,
            "t_1024_256_s": 8.1345,
            "thermal_status_at_bench_start": 0.0,
            "ttft_s_it1": 2.19,
            "ttft_s_it2": 2.01,
            "ttft_s_it3": 2.06
          },
          "output_match": null,
          "peak_mem_mb": 979.012,
          "prefill_tokens_per_s": 509.06,
          "provenance": "measured",
          "runs": true,
          "total_ops": 818,
          "ttft_ms": 2035.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-12/lfm2.5-350m-int8__pixel-8a.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-12",
          "decode_tokens_per_s": 47.895,
          "delegated_ops": 560,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-aab1",
            "os_build": "Android 16",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r1 hold 2026-10-12 15:55:45-16:00:35 JST: [15:59:16] bundle on device: 6a78c7a2746386f8919476a34ef0ca40d596a5c44e979520fae8318da5cfb1f6 (403416064 B, push 0 s) = Mac",
            "bench args: --backend=gpu --model_path=/data/local/tmp/aa1_350m/LFM2.5-350M_int8.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa1_350m/metrics.pb",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_512).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_256).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_128).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 4 (prefill_64).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 5 (prefill_32).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 6 (prefill_16).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 7 (prefill_8).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 8 (prefill_4).",
            "VERBOSE: Replacing 560 out of 560 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 9 (prefill_2).",
            "VERBOSE: Replacing 501 out of 501 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 10 (prefill_1).",
            "VERBOSE: Replacing 519 out of 519 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 11 (decode).",
            "[16:00:33] parsed: p8a_int8_gpu_timing gpu: gate PASS 1/1 | bench prefill 2410.5699999999997 decode 47.894999999999996 init 1558.91 cold 11564.28 peak 939.234 capped 0/8 | answers: 'Paris'",
            "timing run prompt process: \"Paris\"",
            "check run: 2 of 2 prompt processes answered with the expected word (\"Paris\")"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791442100.27,
            "capped_samples": 0.0,
            "cold_init_ms": 11564.28,
            "cpu_capped_samples": 0.0,
            "decode_tok_s_it1": 44.83,
            "decode_tok_s_it2": 47.11,
            "decode_tok_s_it3": 48.68,
            "decode_tokens": 256.0,
            "init_ms": 1558.91,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 3815156.0,
            "memavail_min_kib": 2569608.0,
            "prefill_tok_s_it1": 2185.26,
            "prefill_tok_s_it2": 2400.49,
            "prefill_tok_s_it3": 2420.65,
            "prefill_tokens": 1024.0,
            "sampler_samples": 8.0,
            "t_1024_256_s": 5.7698,
            "thermal_status_at_bench_start": 0.0,
            "ttft_s_it1": 0.49,
            "ttft_s_it2": 0.45,
            "ttft_s_it3": 0.44
          },
          "output_match": null,
          "peak_mem_mb": 939.234,
          "prefill_tokens_per_s": 2410.57,
          "provenance": "measured",
          "runs": true,
          "total_ops": 560,
          "ttft_ms": 445.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-12/lfm2.5-350m-int8__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "lfm2.5",
    "id": "lfm2.5-350m-int8",
    "license": "other (lfm1.0)",
    "source_url": "https://huggingface.co/litert-community/LFM2.5-350M",
    "task": "text-generation"
  },
  "pitfalls": [
    "Runtime: tested on litert-lm 0.18.0 on a Mac; on litert-lm 0.17.1 this file answered as if no question had been asked - the runtime prefilled 9 tokens for a one-question turn where 0.18.0 prefilled 16, the full turn (card L21).",
    "Template: Liquid AI's chat template fails at template creation on litert-lm 0.18.0 ({% generation %} markers), so the bundle carries LiteRT-LM's lfm2 template (byte-identical to models/lfm2/chat_template.jinja at commit 86a7e6d0), which reads message content as a list of parts; stops <|im_end|> and <|endoftext|>, start token <|startoftext|>; marked non-thinking, with the configuration's thought channel kept (card L71, L112).",
    "One Engine per conversation: on an Engine that had already served one conversation, 2 of the next conversation's 3 answers changed - \"What is 17 times 23?\" got 323 where a fresh Engine answered 391 (this file, Python API, CPU, greedy, 3 of 3 processes); a related open issue is LiteRT-LM #3165 (card L75).",
    "Conversation preface with tools and no messages: LiteRT-LM renders the chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with 'Failed to apply template: undefined value' when the preface is prefilled at creation (prefillPrefaceOnInit, default false; creating the conversation fails) and in the preface render (renderPrefaceIntoString and its C++ and C API forms); the same with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2); read in the source of LiteRT-LM 0.17.0, 0.18.1 and main, measured with the 0.18.1 PyPI package through its C API on these files' template text; the default path, where the first message is rendered with the preface, is not affected (the 0.18.1 Python API and CLI expose neither the flag nor the preface render); a system message or any initial message in the conversation config avoids it (card L116).",
    "Sampler: the bundle's default sampler is temperature 0.1, top-k 50, top-p 1.0 (Liquid AI's recommended repetition_penalty 1.05 is not carried); `run` uses it unless --top-k, --top-p, --temperature or --seed is passed; --top-k 1 gives greedy output (card L69).",
    "Cache: without --cache, `run` writes a compiled-weights cache next to the model (358 MB for this file on CPU); without --backend, `run` uses the CPU (card L67).",
    "Tool calling is declared (model type lfm2); after a tool call the bundle template and Liquid AI's differ in two places (call arguments in double quotes, the tool result wrapped in [ … ]); on litert-lm 0.18.0 (Python API, CPU, greedy) the file returned a parsed get_weather call with {\"city\": \"Paris\"} (card L73).",
    "Which file is faster differs by device: int8 on the Pixel 8a GPU (5.77 s vs int4 5.94 s for a 1024-token prefill plus 256 decoded tokens), int4 on the Galaxy S26 GPU (2.63 vs 2.94 s), within 1 % on the Mac GPU (0.68 vs 0.67 s); not measured on iOS, not run on the NPU (card L30-L39, L106).",
    "Galaxy S26 CPU: the row ran with the clock lowering during it (lowered in 3 of 6 samples, read every 2 s); decode fell across the three iterations (89.63 -> 87.28 -> 73.34 tok/s) (card L97).",
    "Accuracy: IFEval-lite (first 120 rows of google/IFEval whose instruction types are checked mechanically, prompt-level strict, greedy; comparable only within that table) bf16 81 (67.5 %), this file 82 (68.3 %) on CPU and 86 (71.7 %) on the Mac GPU; 8-question check 6/8 on CPU and on GPU (misses \"What is 8 times 7?\" -> \"64\" and \"Which is larger: 0.9 or 0.11?\"), bf16 7/8; GSM8K 2 of 100 (bf16 2) with most replies the final-answer line alone (card L43, L49-L57).",
    "License: the LFM Open License v1.0's section 5 conditions commercial use on an annual revenue below 10 million US dollars (card L120)."
  ],
  "schema_version": "1.2"
}
