{
  "artifacts": [
    {
      "file": "mnn-pocket-0.3b-dec_L64_fp16w.tflite",
      "sha256": "ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e",
      "size_mb": 466.233
    }
  ],
  "benchmarks": [],
  "browser": {
    "backends": [
      {
        "backend": "wasm_xnnpack",
        "date": "2026-11-08",
        "env": {
          "browser": "chromium",
          "browser_version": "151.0.7922.34",
          "headless": true,
          "jspi": true,
          "litertjs_core_version": "2.5.3",
          "machine_label": "mac-studio-m4-max",
          "os": "macOS",
          "os_version": "27.0.0",
          "webgpu_adapter": {
            "architecture": "metal-3",
            "description": "",
            "device": "",
            "vendor": "apple"
          }
        },
        "full_delegation": null,
        "latency_p50_ms": null,
        "loads": false,
        "max_rel_diff": null,
        "output_match": null,
        "provenance": "measured",
        "runs": false
      },
      {
        "backend": "webgpu_mldrift",
        "date": "2026-11-08",
        "env": {
          "browser": "chromium",
          "browser_version": "151.0.7922.34",
          "headless": true,
          "jspi": true,
          "litertjs_core_version": "2.5.3",
          "machine_label": "mac-studio-m4-max",
          "os": "macOS",
          "os_version": "27.0.0",
          "webgpu_adapter": {
            "architecture": "metal-3",
            "description": "",
            "device": "",
            "vendor": "apple"
          }
        },
        "full_delegation": null,
        "latency_p50_ms": null,
        "loads": false,
        "max_rel_diff": null,
        "output_match": null,
        "provenance": "measured",
        "runs": false
      }
    ],
    "demo_url": null,
    "sweep_source": "data/sweep/2.5.3/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w.json"
  },
  "conversion": {
    "command": "the repository's conversion/ (conversion/README.md) holds the scripts and REPRODUCE.md every check (README L150); the card names the tools, not a command line",
    "quantization": "float16 linear-layer weights, the LoRA adapter folded into the weights first (README L20)",
    "tool": "litert-torch",
    "tool_version": "0.10.0 (for LiteRT 2.3.0)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_xt4_L64, run r5): run DONE rc 0, start 2026-11-07 23:55:57, 71 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "Extras: rest=5 args=--cpu-threads,4; the build ac2be1d3 passes --cpu-threads to LiteRT (the r5 probes: threads_in_calls_max 4 at --cpu-threads 4)",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000785699, mean |Δp| 0.000193102, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 68/68",
            "Run memory (2 samples): VmHWM_max_kB 1504904 VmSwap_max_kB 0 MemAvailable_min_kB 5917668 kgsl_page_alloc_max_B 209809408"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 504.3,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 68.0,
            "line_max_abs_dp": 0.0007856993920580813,
            "line_mean_abs_dp": 0.00019310214917825355,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_mac_non_near_tie": 68.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "threads": 4.0,
            "vmhwm_max_kb": 1504904.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions of the L64 set against the source model's reference; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 4 threads (--cpu-threads 4)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "41.05 ms (min 40.10, max 42.10; median of 20 after 5 warm-ups, write+run+readback; load 0.50 s; no cap in 1 samples; eve… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 68/68 n)",
            "(tag s26_t_xt4_a_L64_B2, run r5): start 2026-11-07 23:49:07, request S/a_readme (60 tokens), write_run_read_ms median 41.0525 min 40.1 max 42.1 n 20, run_only_ms median 40.986000000000004",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 12/12 OK; extras: rest=60 args=--cpu-threads,4",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 1 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 1 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 495: 1 thread; 528: 8 calls after 2 warm-ups (the cell is 20 after 5); 536: XNNPACK flags 7",
            "threads: the build ac2be1d3 passes --cpu-threads to LiteRT"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 41.0525,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 1.0,
            "compile_ms": 502.2,
            "latency_max_ms": 42.1,
            "latency_min_ms": 40.1,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "run_only_p50_ms": 40.986000000000004,
            "threads": 4.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 4 threads (--cpu-threads 4)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp16acc32_L64, run r5): run DONE rc 0, start 2026-11-07 21:05:38, 71 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.0125773, mean |Δp| 0.00179718, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 68/68",
            "Run memory (3 samples): VmHWM_max_kB 2063268 VmSwap_max_kB 0 MemAvailable_min_kB 6027912 kgsl_page_alloc_max_B 755429376"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 2172.4,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 68.0,
            "line_max_abs_dp": 0.012577291977632687,
            "line_mean_abs_dp": 0.0017971795295334747,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_mac_non_near_tie": 68.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "vmhwm_max_kb": 2063268.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions of the L64 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP16_WITH_FP32_ACCUM",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp32_L64, run r5): run DONE rc 0, start 2026-11-07 21:04:59, 71 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000783722, mean |Δp| 0.000193232, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 68/68",
            "Run memory (3 samples): VmHWM_max_kB 2539188 VmSwap_max_kB 0 MemAvailable_min_kB 5478152 kgsl_page_alloc_max_B 1244667904"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 1870.3,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 68.0,
            "line_max_abs_dp": 0.000783722104328699,
            "line_mean_abs_dp": 0.0001932318934138322,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_mac_non_near_tie": 68.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "vmhwm_max_kb": 2539188.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions of the L64 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "28.19 ms (min 27.83, max 33.01; median of 20 after 5 warm-ups, write+run+readback; load 1.35 s; no cap in 1 samples; eve… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 68/68 n)",
            "(tag s26_t_gpu_fp16acc32_a_L64_B2, run r5): start 2026-11-07 22:32:09, request S/a_readme (60 tokens), write_run_read_ms median 28.186 min 27.83 max 33.013 n 20, run_only_ms median 0.584",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK; extras: rest=60",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 1 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 1 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 506: GPU precision fp32"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 28.186,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 1.0,
            "compile_ms": 1345.5,
            "latency_max_ms": 33.013,
            "latency_min_ms": 27.83,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "run_only_p50_ms": 0.584,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP16_WITH_FP32_ACCUM",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "37.59 ms (min 37.09, max 38.75; median of 20 after 5 warm-ups, write+run+readback; load 1.60 s; no cap in 2 samples; eve… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 68/68 n)",
            "(tag s26_t_gpu_fp32_a_L64_B2, run r5): start 2026-11-07 22:19:55, request S/a_readme (60 tokens), write_run_read_ms median 37.593 min 37.09 max 38.752 n 20, run_only_ms median 0.6835",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK; extras: rest=60",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "clock samples: 2 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 1 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 500: GPU precision fp16acc32"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 37.593,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 2.0,
            "compile_ms": 1600.5,
            "latency_max_ms": 38.752,
            "latency_min_ms": 37.09,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "run_only_p50_ms": 0.6835,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16, arm cpu4, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator CPU (XNNPACK), threads 4, is_fully_accelerated False (the API states no node count), compile 0.65 s",
            "date: the record's first commit, author date 2026-11-07 21:03:51 +0900, after the run's START 20:51:48 (the run log has times only and is not in git)",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000785699, mean |Δp| 0.000193102, control n/a (the file holds no request of the control pair), non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.65,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_max_abs_dp": 0.0007856993920580813,
            "line_mean_abs_dp": 0.00019310214917825355,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "threads": 4.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions (the requests that fit L64) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-08",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "29.72 ms (min 28.58, max 30.64; median of 20 after warm-ups, host write+run+read back; every run (the cell's value; parity cell: PASS gate: 68/68 non-near-tie, near tie 3/3, max|Δp| 0.00079, mean 0.00019 (71 q)",
            "(tag F7C2, started 2026-11-08 00:08:38 JST, rule \"line L6: 5 warm-ups + median of 20, host write + run + read back; host ms apart\", ai-edge-litert 2.3.0): row file mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16, arm cpu4, request a_readme, ms_write_run_read median 29.71714601153508 min 28.580915997736156 max 30.639999982668087 p90 30.353666987502947 n 20, winners_equal_R True",
            "Reads (passes no --r1-graphs), the file of the gate record (sha256 ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = the Hub LFS entry); the file still holds those bytes, modified 2026-11-07 20:51:14 +0900, before this run",
            "the line on this file and arm, line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000785699, mean |Δp| 0.000193102, control n/a (the file holds no request of the control pair), non-finite read-outs 0",
            "other rows of this file, request and backend, not takes of this cell: 413: arm cpu4f7"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 29.71714601153508,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.13,
            "latency_max_ms": 30.639999982668087,
            "latency_min_ms": 28.580915997736156,
            "latency_p90_ms": 30.353666987502947,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "threads": 4.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16, arm gpu_f32, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator GPU (Metal), gpu_options {\"enforce_f32\": true}, is_fully_accelerated True (the API states no node count), compile 1.44 s",
            "date: the record's first commit, author date 2026-11-07 21:03:51 +0900, after the run's START 20:51:54 (the run log has times only and is not in git)",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000784694, mean |Δp| 0.000193058, control n/a (the file holds no request of the control pair), non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 1.44,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_max_abs_dp": 0.0007846938572202977,
            "line_mean_abs_dp": 0.00019305839521112838,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions (the requests that fit L64) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "12.62 ms (min 12.44, max 12.92; median of 20 after warm-ups, host write+run+read back; every run (the cell's value; parity cell: PASS gate: 68/68 non-near-tie, near tie 3/3, max|Δp| 0.00078, mean 0.00019 (71 q)",
            "(tag B2, started 2026-11-07 21:22:50 JST, rule \"line L6: 5 warm-ups + median of 20, host write + run + read back; host ms apart\", ai-edge-litert 2.3.0): row file mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16, arm gpu_f32, request a_readme, ms_write_run_read median 12.622374997590668 min 12.440499995136634 max 12.91950000450015 p90 12.86190000246279 n 20, winners_equal_R True",
            "Reads (passes no --r1-graphs), the file of the gate record (sha256 ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = the Hub LFS entry); the file still holds those bytes, modified 2026-11-07 20:51:14 +0900, before this run",
            "the line on this file and arm, line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000784694, mean |Δp| 0.000193058, control n/a (the file holds no request of the control pair), non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 12.622374997590668,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 1.36,
            "latency_max_ms": 12.91950000450015,
            "latency_min_ms": 12.440499995136634,
            "latency_p90_ms": 12.86190000246279,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-08",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_xt6_L64, run r5): run DONE rc 0, start 2026-11-08 01:43:22, 71 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "Extras: rest=5 args=--cpu-threads,6; the build ac2be1d3 passes --cpu-threads to LiteRT (the r5 probes: threads_in_calls_max 4 at --cpu-threads 4)",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000785699, mean |Δp| 0.000193102, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 68/68",
            "Run memory (6 samples): VmHWM_max_kB 1498064 VmSwap_max_kB 0 MemAvailable_min_kB 8851136 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 856.3,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 68.0,
            "line_max_abs_dp": 0.0007856993920580813,
            "line_mean_abs_dp": 0.00019310214917825355,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_mac_non_near_tie": 68.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "threads": 6.0,
            "vmhwm_max_kb": 1498064.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions of the L64 set against the source model's reference; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 6 threads (--cpu-threads 6)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-08",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "128.74 ms (min 123.48, max 305.36; median of 20 after 5 warm-ups, write+run+readback; load 0.85 s; no cap in 3 samples; … (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 68/68 n)",
            "(tag p11p_t_xt6_a_L64_G1, run r5): start 2026-11-08 02:10:18, request S/a_readme (60 tokens), write_run_read_ms median 128.7415 min 123.476 max 305.358 n 20, run_only_ms median 128.5845",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner ac2be1d3f7b7a78f… and libs: 2/2 OK; extras: rest=180 args=--cpu-threads,6",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "clock samples: 3 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 1 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "other rows of this file, request and backend, not takes of this cell: 450: 1 thread; 490: XNNPACK flags 7",
            "threads: the build ac2be1d3 passes --cpu-threads to LiteRT"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 128.7415,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 3.0,
            "compile_ms": 846.4,
            "latency_max_ms": 305.358,
            "latency_min_ms": 123.476,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "run_only_p50_ms": 128.5845,
            "threads": 6.0,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build ac2be1d3 (LiteRT 2.3.0), CPU XNNPACK, 6 threads (--cpu-threads 6)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-08/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_gpu_fp32_L64, run r5): run DONE rc 0, start 2026-11-07 21:07:25, 71 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 71 questions, same winner 68/68 (not near ties), near ties kept 3/3, max |Δp| 0.000778201, mean |Δp| 0.000193094, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 68/68",
            "Run memory (10 samples): VmHWM_max_kB 2507196 VmSwap_max_kB 0 MemAvailable_min_kB 8097156 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 15536.2,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 68.0,
            "line_max_abs_dp": 0.0007782012750844891,
            "line_mean_abs_dp": 0.00019309398674303407,
            "line_near_tie": 3.0,
            "line_non_near_tie": 68.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 71.0,
            "line_winner_equal_mac_non_near_tie": 68.0,
            "line_winner_equal_near_tie": 3.0,
            "line_winner_equal_non_near_tie": 68.0,
            "vmhwm_max_kb": 2507196.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 71 questions of the L64 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "120.90 ms (min 114.42, max 125.30; median of 20 after 5 warm-ups, write+run+readback; load 15.26 s; no cap in 8 samples;… (the cell's value; parity cell: PASS; gate PASS, timing run = check run in every digit (V3 a); gate PASS: 68/68 n)",
            "(tag p11p_t_gpu_fp32_a_L64_B2, run r5): start 2026-11-07 21:53:48, request S/a_readme (60 tokens), write_run_read_ms median 120.898 min 114.418 max 125.303 n 20, run_only_ms median 1.2255",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L64_w16.tflite rc=0 bytes 488880336/488880336 sha256 phone=ceaa8bd8eff2c335467ca49193f3625bf6b397f0286cef8527fe1c8b1834745e = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK; extras: rest=60",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "clock samples: 8 samples, 0 capped, GPU level max 0",
            "parity: the timed request's 1 question(s) have the same probabilities in every digit as in the check run (line PASS)",
            "VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main)."
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 120.898,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "capped": 0.0,
            "capped_samples": 0.0,
            "clock_samples": 8.0,
            "compile_ms": 15264.0,
            "latency_max_ms": 125.303,
            "latency_min_ms": 114.418,
            "request_questions": 1.0,
            "request_tokens": 60.0,
            "run_only_p50_ms": 1.2255,
            "timed_calls": 20.0,
            "warmup_calls": 5.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "timing: `readme_request` (the source model card's example) (60 tokens, 1 question), the median of 20 calls after 5 warm-up calls in one process, a call = write the four inputs, run, read `hidden` back; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w__pixel-11-pro.json"
      }
    ]
  },
  "model": {
    "family": "mnn-pocket",
    "id": "mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l64_fp16w",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/MNN-Pocket-0.3B-Dec-LiteRT",
    "task": "text-classification"
  },
  "pitfalls": [
    "A decision model, not a chat model: it takes a `state` and typed `questions` (`choice`, `noul`, `score`) and returns the probability of every candidate in one forward pass; it never generates text (README L18).",
    "The graph is the model body only: it takes the embeddings of the request's tokens, the rotary tables and the attention mask and returns the hidden states after the final norm; the Python host in `host/` builds the request, takes the embeddings from `embedding.safetensors` (the source model's token table) and reads the answers with the source model's head and temperature, as the source model's `inference.py` does (README L20, L147-L148).",
    "Each file is a static window: `<n>` in the name is the longest request it takes; run the smallest file that holds the request. A request longer than 2,048 tokens raises `ValueError` in the host, which never cuts it; the source model's own limit is 32,768 tokens (README L71, L84).",
    "The checks measure agreement with the source model's `inference.py`, unchanged, on its own files (float32, CPU, one thread): 2,409 questions on the Mac, a fixed subset of 16 to 322 questions per file on the phones; the bar is the same winner on every question whose two most likely options are more than 0.02 apart, a largest |dp| of at most 0.02 and a mean of at most 0.002 (README L88, L102).",
    "The repository has no Android host yet: an Android app has to port `host/pocket_dec_litert.py`; the phones ran LiteRT 2.3.0 through its C API with inputs the Mac computed with the host's code (README L137).",
    "On a GPU set float32 precision (`GpuOptions(enforce_f32=True)` in the Python host, `CompiledModel.GpuOptions.Precision.FP32` in Kotlin, `kLiteRtDelegatePrecisionFp32` in the C API): an app that sets no precision gets FP16 on both phones, and FP16 changes the answers: among questions whose top two options are more than 0.02 apart the winner changed on up to 5 of 305 on the Galaxy S26, 1 of 305 on the Pixel 11 Pro and 9 of 2,314 on Apple Metal at its default precision, and a probability moved by up to 0.068, 0.029 and 0.058 (README L73).",
    "FP16 with FP32 accumulation passes the bar on the Galaxy S26 (44 of 54 near ties keep the reference's winner, largest |dp| 0.0126) and misses the mean on 3 of the 6 files it ran on the Pixel 11 Pro; it can flip a near tie: in the source model's default example (`department`: `shipping` 0.41064 against `returns` 0.40709) it answers `returns` (README L96, L102, L104).",
    "Set the CPU thread count: with no CPU options LiteRT 2.3.0's C API ran on one thread, and the 128-token file took 1.3 times as long as at 4 threads on the Galaxy S26 and 2.2 times as long on the Pixel 11 Pro; the card's phone CPU rows use 4 threads (Galaxy S26) and 6 (Pixel 11 Pro), whose probabilities equal those of one thread to every digit (README L133).",
    "LiteRT's GPU cache (`serializationDir`, `modelCacheKey`, `serializeProgramCache` and `serializeExternalTensors` in `CompiledModel.GpuOptions`) cut the Pixel 11 Pro's creation of the 128-token model from 15.5 s to 1.8 to 2.0 s, but each call then took 12 to 19 % longer; on the Galaxy S26 it cut 1.6 s to 1.3 s at the same speed; the cache takes 0.93 GiB on disk and raises the process's peak by about 0.9 GiB (README L135)."
  ],
  "schema_version": "1.2"
}
