{
  "artifacts": [
    {
      "file": "mnn-pocket-0.3b-dec_L96_fp16w.tflite",
      "sha256": "4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55",
      "size_mb": 466.233
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "the repository's conversion/ (conversion/README.md) holds the scripts and REPRODUCE.md every check (README L150); the card names the tools, not a command line",
    "quantization": "float16 linear-layer weights, the LoRA adapter folded into the weights first (README L20)",
    "tool": "litert-torch",
    "tool_version": "0.10.0 (for LiteRT 2.3.0)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_xt4_L96, run r5): run DONE rc 0, start 2026-11-07 23:56:12, 278 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "Extras: rest=5 args=--cpu-threads,4; the build ac2be1d3 passes --cpu-threads to LiteRT (the r5 probes: threads_in_calls_max 4 at --cpu-threads 4)",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187057, control 0.0401, non-finite read-outs 0, winner = the Mac CPU's on the same file 267/267",
            "Run memory (11 samples): VmHWM_max_kB 1509496 VmSwap_max_kB 0 MemAvailable_min_kB 5876288 kgsl_page_alloc_max_B 209809408"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 530.5,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007461881503663,
            "line_mac_same_file_non_near_tie": 267.0,
            "line_max_abs_dp": 0.0013634854615124925,
            "line_mean_abs_dp": 0.00018705696731315265,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_mac_non_near_tie": 267.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0,
            "threads": 4.0,
            "vmhwm_max_kb": 1509496.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions of the L96 set against the source model's reference; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 4 threads (--cpu-threads 4)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "60.88 ms (min 59.35, max 63.35; median of 20 after 5 warm-ups, write+run+readback; load 0.50 s; no cap in 1 samples; eve… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 267/267)",
            "(tag s26_t_xt4_b_L96_M1, run r5): start 2026-11-07 23:55:44, request S/b_default (85 tokens), write_run_read_ms median 60.884 min 59.352 max 63.348 n 20, run_only_ms median 60.7925",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 12/12 OK; extras: rest=60 args=--cpu-threads,4",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 1 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 4 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 499: 1 thread; 533: 8 calls after 2 warm-ups (the cell is 20 after 5); 538: XNNPACK flags 7",
            "threads: the build ac2be1d3 passes --cpu-threads to LiteRT"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 60.884,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 1.0,
            "compile_ms": 503.8,
            "latency_max_ms": 63.348,
            "latency_min_ms": 59.352,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "run_only_p50_ms": 60.7925,
            "threads": 4.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 4 threads (--cpu-threads 4)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp16acc32_L96, run r5): run DONE rc 0, start 2026-11-07 21:05:52, 278 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 10/11, max |Δp| 0.0125773, mean |Δp| 0.00161454, control 0.0379, non-finite read-outs 0, winner = the Mac CPU's on the same file 267/267",
            "Run memory (7 samples): VmHWM_max_kB 2063256 VmSwap_max_kB 0 MemAvailable_min_kB 5960912 kgsl_page_alloc_max_B 756264960"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 1860.7,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.03787198002075168,
            "line_mac_same_file_non_near_tie": 267.0,
            "line_max_abs_dp": 0.012577291977632687,
            "line_mean_abs_dp": 0.0016145444863596202,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_mac_non_near_tie": 267.0,
            "line_winner_equal_near_tie": 10.0,
            "line_winner_equal_non_near_tie": 267.0,
            "vmhwm_max_kb": 2063256.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions of the L96 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP16_WITH_FP32_ACCUM",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp32_L96, run r5): run DONE rc 0, start 2026-11-07 21:05:11, 278 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136354, mean |Δp| 0.000187197, control 0.0401, non-finite read-outs 0, winner = the Mac CPU's on the same file 267/267",
            "Run memory (8 samples): VmHWM_max_kB 2538736 VmSwap_max_kB 0 MemAvailable_min_kB 5470892 kgsl_page_alloc_max_B 1246068736"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 1736.0,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007412855930406,
            "line_mac_same_file_non_near_tie": 267.0,
            "line_max_abs_dp": 0.0013635436279565982,
            "line_mean_abs_dp": 0.00018719682744393237,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_mac_non_near_tie": 267.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0,
            "vmhwm_max_kb": 2538736.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions of the L96 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "43.63 ms (min 42.58, max 44.50; median of 20 after 5 warm-ups, write+run+readback; load 1.36 s; no cap in 2 samples)… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 267/267)",
            "(tag s26_t_gpu_fp16acc32_b_L96_M1, run r5): start 2026-11-07 22:35:38, request S/b_default (85 tokens), write_run_read_ms median 43.6315 min 42.58 max 44.499 n 20, run_only_ms median 0.6445000000000001",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK; extras: rest=60",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 2 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 4 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 524: GPU precision fp32"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 43.6315,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 2.0,
            "compile_ms": 1364.4,
            "latency_max_ms": 44.499,
            "latency_min_ms": 42.58,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "run_only_p50_ms": 0.6445000000000001,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP16_WITH_FP32_ACCUM",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "48.25 ms (min 46.72, max 52.19; median of 20 after 5 warm-ups, write+run+readback; load 1.59 s; no cap in 2 samples)… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 267/267)",
            "(tag s26_t_gpu_fp32_b_L96_M1, run r5): start 2026-11-07 22:26:50, request S/b_default (85 tokens), write_run_read_ms median 48.248000000000005 min 46.716 max 52.193 n 20, run_only_ms median 1.2145000000000001",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK; extras: rest=60",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 2 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 4 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 504: GPU precision fp16acc32"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 48.248000000000005,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 2.0,
            "compile_ms": 1593.7,
            "latency_max_ms": 52.193,
            "latency_min_ms": 46.716,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "run_only_p50_ms": 1.2145000000000001,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16, arm cpu4, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator CPU (XNNPACK), threads 4, is_fully_accelerated False (the API states no node count), compile 0.12 s",
            "date: the record's first commit, author date 2026-11-07 21:03:51 +0900, after the run's START 20:52:13 (the run log has times only and is not in git)",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187057, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.12,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007461881503663,
            "line_max_abs_dp": 0.0013634854615124925,
            "line_mean_abs_dp": 0.00018705696731315265,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0,
            "threads": 4.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions (the requests that fit L96) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "41.13 ms (min 40.82, max 41.28; median of 20 after warm-ups, host write+run+read back) (the cell's value; parity cell: PASS gate: 267/267 non-near-tie, near tie 11/11, max|Δp| 0.00136, mean 0.00019)",
            "(tag M1, started 2026-11-07 21:23:10 JST, rule \"line L6: 5 warm-ups + median of 20, host write + run + read back; host ms apart\", ai-edge-litert 2.3.0): row file mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16, arm cpu4, request b_default, ms_write_run_read median 41.12781251023989 min 40.82266698242165 max 41.27883299952373 p90 41.26312908483669 n 20, winners_equal_R True",
            "Reads (passes no --r1-graphs), the file of the gate record (sha256 4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = the Hub LFS entry); the file still holds those bytes, modified 2026-11-07 20:51:45 +0900, before this run",
            "the line on this file and arm, line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187057, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 41.12781251023989,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.66,
            "latency_max_ms": 41.27883299952373,
            "latency_min_ms": 40.82266698242165,
            "latency_p90_ms": 41.26312908483669,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "threads": 4.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16, arm gpu_f32, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator GPU (Metal), gpu_options {\"enforce_f32\": true}, is_fully_accelerated True (the API states no node count), compile 2.93 s",
            "date: the record's first commit, author date 2026-11-07 21:03:51 +0900, after the run's START 20:52:27 (the run log has times only and is not in git)",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187177, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 2.93,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.040074239589367566,
            "line_max_abs_dp": 0.0013634882347727806,
            "line_mean_abs_dp": 0.00018717743835452703,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions (the requests that fit L96) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "13.38 ms (min 13.27, max 13.61; median of 20 after warm-ups, host write+run+read back) (the cell's value; parity cell: PASS gate: 267/267 non-near-tie, near tie 11/11, max|Δp| 0.00136, mean 0.00019)",
            "(tag M1, started 2026-11-07 21:23:10 JST, rule \"line L6: 5 warm-ups + median of 20, host write + run + read back; host ms apart\", ai-edge-litert 2.3.0): row file mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16, arm gpu_f32, request b_default, ms_write_run_read median 13.381667013163678 min 13.268833019537851 max 13.61316701513715 p90 13.601183603168465 n 20, winners_equal_R True",
            "Reads (passes no --r1-graphs), the file of the gate record (sha256 4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = the Hub LFS entry); the file still holds those bytes, modified 2026-11-07 20:51:45 +0900, before this run",
            "the line on this file and arm, line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187177, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 13.381667013163678,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 2.09,
            "latency_max_ms": 13.61316701513715,
            "latency_min_ms": 13.268833019537851,
            "latency_p90_ms": 13.601183603168465,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-08",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_xt6_L96, run r5): run DONE rc 0, start 2026-11-08 01:43:49, 278 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "Extras: rest=5 args=--cpu-threads,6; the build ac2be1d3 passes --cpu-threads to LiteRT (the r5 probes: threads_in_calls_max 4 at --cpu-threads 4)",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136349, mean |Δp| 0.000187057, control 0.0401, non-finite read-outs 0, winner = the Mac CPU's on the same file 267/267",
            "Run memory (55 samples): VmHWM_max_kB 1503096 VmSwap_max_kB 0 MemAvailable_min_kB 8534572 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 878.0,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007461881503663,
            "line_mac_same_file_non_near_tie": 267.0,
            "line_max_abs_dp": 0.0013634854615124925,
            "line_mean_abs_dp": 0.00018705696731315265,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_mac_non_near_tie": 267.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0,
            "threads": 6.0,
            "vmhwm_max_kb": 1503096.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions of the L96 set against the source model's reference; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 6 threads (--cpu-threads 6)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-08",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "179.48 ms (min 164.16, max 592.71; median of 20 after 5 warm-ups, write+run+readback; load 0.96 s; no cap in 4 samples; … (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 267/267)",
            "(tag p11p_t_xt6_b_L96_G1, run r5): start 2026-11-08 02:15:00, request S/b_default (85 tokens), write_run_read_ms median 179.4755 min 164.16 max 592.711 n 20, run_only_ms median 179.248",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 2/2 OK; extras: rest=60 args=--cpu-threads,6",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "clock samples: 4 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 4 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 454: 1 thread; 492: XNNPACK flags 7",
            "threads: the build ac2be1d3 passes --cpu-threads to LiteRT"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 179.4755,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 4.0,
            "compile_ms": 962.7,
            "latency_max_ms": 592.711,
            "latency_min_ms": 164.16,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "run_only_p50_ms": 179.248,
            "threads": 6.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 6 threads (--cpu-threads 6)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_gpu_fp32_L96, run r5): run DONE rc 0, start 2026-11-07 21:07:59, 278 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 278 questions, same winner 267/267 (not near ties), near ties kept 11/11, max |Δp| 0.00136397, mean |Δp| 0.000187153, control 0.0401, non-finite read-outs 0, winner = the Mac CPU's on the same file 267/267",
            "Run memory (23 samples): VmHWM_max_kB 2504864 VmSwap_max_kB 0 MemAvailable_min_kB 7999816 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 17659.6,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.040073776044496884,
            "line_mac_same_file_non_near_tie": 267.0,
            "line_max_abs_dp": 0.0013639744894002326,
            "line_mean_abs_dp": 0.00018715289414367796,
            "line_near_tie": 11.0,
            "line_non_near_tie": 267.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 278.0,
            "line_winner_equal_mac_non_near_tie": 267.0,
            "line_winner_equal_near_tie": 11.0,
            "line_winner_equal_non_near_tie": 267.0,
            "vmhwm_max_kb": 2504864.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 278 questions of the L96 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "129.06 ms (min 124.18, max 139.66; median of 20 after 5 warm-ups, write+run+readback; load 15.51 s; no cap in 8 samples)… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 267/267)",
            "(tag p11p_t_gpu_fp32_b_L96_M1, run r5): start 2026-11-07 22:02:16, request S/b_default (85 tokens), write_run_read_ms median 129.0575 min 124.177 max 139.665 n 20, run_only_ms median 1.258",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L96_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=4136dfe3eca3079727904b413633a2e9b25c3f1ceb3224b5360ef29d3fdaad55 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK; extras: rest=60",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "clock samples: 8 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 4 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 129.0575,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 8.0,
            "compile_ms": 15514.1,
            "latency_max_ms": 139.665,
            "latency_min_ms": 124.177,
            "request_questions": 3.0,
            "request_tokens": 85.0,
            "run_only_p50_ms": 1.258,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `default_request` (inference.py's DEFAULT_REQUEST) (85 tokens, 3 questions), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w__pixel-11-pro.json"
      }
    ]
  },
  "model": {
    "family": "mnn-pocket",
    "id": "mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l96_fp16w",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/MNN-Pocket-0.3B-Dec-LiteRT",
    "task": "text-classification"
  },
  "pitfalls": [
    "A decision model, not a chat model: it takes a `state` and typed `questions` (`choice`, `noul`, `score`) and returns the probability of every candidate in one forward pass; it never generates text (README L18).",
    "The graph is the model body only: it takes the embeddings of the request's tokens, the rotary tables and the attention mask and returns the hidden states after the final norm; the Python host in `host/` builds the request, takes the embeddings from `embedding.safetensors` (the source model's token table) and reads the answers with the source model's head and temperature, as the source model's `inference.py` does (README L20, L147-L148).",
    "Each file is a static window: `<n>` in the name is the longest request it takes; run the smallest file that holds the request. A request longer than 2,048 tokens raises `ValueError` in the host, which never cuts it; the source model's own limit is 32,768 tokens (README L71, L84).",
    "The checks measure agreement with the source model's `inference.py`, unchanged, on its own files (float32, CPU, one thread): 2,409 questions on the Mac, a fixed subset of 16 to 322 questions per file on the phones; the bar is the same winner on every question whose two most likely options are more than 0.02 apart, a largest |dp| of at most 0.02 and a mean of at most 0.002 (README L88, L102).",
    "The repository has no Android host yet: an Android app has to port `host/pocket_dec_litert.py`; the phones ran LiteRT 2.3.0 through its C API with inputs the Mac computed with the host's code (README L137).",
    "On a GPU set float32 precision (`GpuOptions(enforce_f32=True)` in the Python host, `CompiledModel.GpuOptions.Precision.FP32` in Kotlin, `kLiteRtDelegatePrecisionFp32` in the C API): an app that sets no precision gets FP16 on both phones, and FP16 changes the answers: among questions whose top two options are more than 0.02 apart the winner changed on up to 5 of 305 on the Galaxy S26, 1 of 305 on the Pixel 11 Pro and 9 of 2,314 on Apple Metal at its default precision, and a probability moved by up to 0.068, 0.029 and 0.058 (README L73).",
    "FP16 with FP32 accumulation passes the bar on the Galaxy S26 (44 of 54 near ties keep the reference's winner, largest |dp| 0.0126) and misses the mean on 3 of the 6 files it ran on the Pixel 11 Pro; it can flip a near tie: in the source model's default example (`department`: `shipping` 0.41064 against `returns` 0.40709) it answers `returns` (README L96, L102, L104).",
    "Set the CPU thread count: with no CPU options LiteRT 2.3.0's C API ran on one thread, and the 128-token file took 1.3 times as long as at 4 threads on the Galaxy S26 and 2.2 times as long on the Pixel 11 Pro; the card's phone CPU rows use 4 threads (Galaxy S26) and 6 (Pixel 11 Pro), whose probabilities equal those of one thread to every digit (README L133).",
    "LiteRT's GPU cache (`serializationDir`, `modelCacheKey`, `serializeProgramCache` and `serializeExternalTensors` in `CompiledModel.GpuOptions`) cut the Pixel 11 Pro's creation of the 128-token model from 15.5 s to 1.8 to 2.0 s, but each call then took 12 to 19 % longer; on the Galaxy S26 it cut 1.6 s to 1.3 s at the same speed; the cache takes 0.93 GiB on disk and raises the process's peak by about 0.9 GiB (README L135)."
  ],
  "schema_version": "1.2"
}
