{
  "artifacts": [
    {
      "file": "mnn-pocket-0.3b-dec_L512_fp16w.tflite",
      "sha256": "e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b",
      "size_mb": 466.233
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "the repository's conversion/ (conversion/README.md) holds the scripts and REPRODUCE.md every check (README L150); the card names the tools, not a command line",
    "quantization": "float16 linear-layer weights, the LoRA adapter folded into the weights first (README L20)",
    "tool": "litert-torch",
    "tool_version": "0.10.0 (for LiteRT 2.3.0)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_cpu4_L512, run r2): run DONE rc 0, start 2026-11-07 19:14:22, 142 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "The build 7e7539e5 asked for 4 threads (cpu_toml \"num_threads = 4\") and its calls ran on threads_in_calls_max 1; the corrected build ac2be1d3 at 4: 4. This run is entered as 1 thread",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 142 questions, same winner 137/137 (not near ties), near ties kept 5/5, max |Δp| 0.00104475, mean |Δp| 8.48741e-05, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 137/137",
            "Run memory (10 samples): VmHWM_max_kB 1596392 VmSwap_max_kB 0 MemAvailable_min_kB 6049212 kgsl_page_alloc_max_B 219365376"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 529.4,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 137.0,
            "line_max_abs_dp": 0.0010447452070594387,
            "line_mean_abs_dp": 8.487411334364538e-05,
            "line_near_tie": 5.0,
            "line_non_near_tie": 137.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 142.0,
            "line_winner_equal_mac_non_near_tie": 137.0,
            "line_winner_equal_near_tie": 5.0,
            "line_winner_equal_non_near_tie": 137.0,
            "threads": 1.0,
            "vmhwm_max_kb": 1596392.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 142 questions of the L512 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), CPU XNNPACK, 1 thread (this build passed the thread count under an option name LiteRT 2.3.0 does not read)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp16acc32_L512, run r2): run DONE rc 0, start 2026-11-07 19:17:23, 142 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 142 questions, same winner 137/137 (not near ties), near ties kept 4/5, max |Δp| 0.0043119, mean |Δp| 0.000901609, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 137/137",
            "Run memory (5 samples): VmHWM_max_kB 2067684 VmSwap_max_kB 0 MemAvailable_min_kB 5782172 kgsl_page_alloc_max_B 781533184"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 2469.8,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 137.0,
            "line_max_abs_dp": 0.00431189802937737,
            "line_mean_abs_dp": 0.0009016092825651376,
            "line_near_tie": 5.0,
            "line_non_near_tie": 137.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 142.0,
            "line_winner_equal_mac_non_near_tie": 137.0,
            "line_winner_equal_near_tie": 4.0,
            "line_winner_equal_non_near_tie": 137.0,
            "vmhwm_max_kb": 2067684.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 142 questions of the L512 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP16_WITH_FP32_ACCUM",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag s26_gpu_fp32_L512, run r2): run DONE rc 0, start 2026-11-07 19:11:24, 142 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 142 questions, same winner 137/137 (not near ties), near ties kept 5/5, max |Δp| 0.00104561, mean |Δp| 8.4885e-05, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 137/137",
            "Run memory (5 samples): VmHWM_max_kB 2545196 VmSwap_max_kB 0 MemAvailable_min_kB 5984624 kgsl_page_alloc_max_B 1286877184"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 2405.9,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 137.0,
            "line_max_abs_dp": 0.0010456138613076926,
            "line_mean_abs_dp": 8.488500052136691e-05,
            "line_near_tie": 5.0,
            "line_non_near_tie": 137.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 142.0,
            "line_winner_equal_mac_non_near_tie": 137.0,
            "line_winner_equal_near_tie": 5.0,
            "line_winner_equal_non_near_tie": 137.0,
            "vmhwm_max_kb": 2545196.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 142 questions of the L512 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16, arm cpu4, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator CPU (XNNPACK), threads 4, is_fully_accelerated False (the API states no node count), compile 0.7 s",
            "date: the record's first commit, author date 2026-11-07 18:49:02 +0900, after the run's START 18:17:35 (the run log has times only and is not in git)",
            "line PASS: 1232 questions, same winner 1186/1186 (not near ties), near ties kept 46/46, max |Δp| 0.00209024, mean |Δp| 0.000120092, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.7,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007461881503663,
            "line_max_abs_dp": 0.002090235550811559,
            "line_mean_abs_dp": 0.00012009170704203981,
            "line_near_tie": 46.0,
            "line_non_near_tie": 1186.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 1232.0,
            "line_winner_equal_near_tie": 46.0,
            "line_winner_equal_non_near_tie": 1186.0,
            "threads": 4.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 1232 questions (the requests that fit L512) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16, arm gpu_f32, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator GPU (Metal), gpu_options {\"enforce_f32\": true}, is_fully_accelerated True (the API states no node count), compile 2.91 s",
            "date: the record's first commit, author date 2026-11-07 18:49:02 +0900, after the run's START 18:11:26 (the run log has times only and is not in git)",
            "line PASS: 1232 questions, same winner 1186/1186 (not near ties), near ties kept 46/46, max |Δp| 0.00208897, mean |Δp| 0.000120102, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 2.91,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.040073994086257336,
            "line_max_abs_dp": 0.0020889656071872986,
            "line_mean_abs_dp": 0.00012010187786074488,
            "line_near_tie": 46.0,
            "line_non_near_tie": 1186.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 1232.0,
            "line_winner_equal_near_tie": 46.0,
            "line_winner_equal_non_near_tie": 1186.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 1232 questions (the requests that fit L512) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2376,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_cpu4_L512, run r2): run DONE rc 0, start 2026-11-07 19:12:49, 142 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "The build 7e7539e5 asked for 4 threads (cpu_toml \"num_threads = 4\") and its calls ran on threads_in_calls_max 1; the corrected build ac2be1d3 at 4: 4. This run is entered as 1 thread",
            "runner.replacing: VERBOSE: Replacing 2376 out of 2432 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (main).",
            "line PASS: 142 questions, same winner 137/137 (not near ties), near ties kept 5/5, max |Δp| 0.00104475, mean |Δp| 8.48741e-05, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 137/137",
            "Run memory (64 samples): VmHWM_max_kB 1590568 VmSwap_max_kB 0 MemAvailable_min_kB 8792528 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 777.2,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 137.0,
            "line_max_abs_dp": 0.0010447452070594387,
            "line_mean_abs_dp": 8.487411334364538e-05,
            "line_near_tie": 5.0,
            "line_non_near_tie": 137.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 142.0,
            "line_winner_equal_mac_non_near_tie": 137.0,
            "line_winner_equal_near_tie": 5.0,
            "line_winner_equal_non_near_tie": 137.0,
            "threads": 1.0,
            "vmhwm_max_kb": 1590568.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 142 questions of the L512 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), CPU XNNPACK, 1 thread (this build passed the thread count under an option name LiteRT 2.3.0 does not read)",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu_mldrift",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2432,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-mnnpocketdec-capi-runner",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Google Tensor G6",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "(tag p11p_gpu_fp32_L512, run r2): run DONE rc 0, start 2026-11-07 19:15:48, 142 questions",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final_L512_w16.tflite rc=0 bytes 488880528/488880528 sha256 phone=e66a712762dc6036a545c7e514a500617358907e6f7200c8b0efc87245fed48b = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 2/2 OK",
            "Device: Pixel 11 Pro 17 CD1A.260905.001.B1",
            "runner.replacing: VERBOSE: Replacing 2432 out of 2432 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (main).",
            "line PASS: 142 questions, same winner 137/137 (not near ties), near ties kept 5/5, max |Δp| 0.00104449, mean |Δp| 8.49413e-05, control n/a (the file holds no request of the control pair), non-finite read-outs 0, winner = the Mac CPU's on the same file 137/137",
            "Run memory (18 samples): VmHWM_max_kB 2504960 VmSwap_max_kB 0 MemAvailable_min_kB 7986136 kgsl_page_alloc_max_B "
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 8.0,
            "arm_files_run": 8.0,
            "compile_ms": 27573.4,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_mac_same_file_non_near_tie": 137.0,
            "line_max_abs_dp": 0.0010444897206470793,
            "line_mean_abs_dp": 8.494131631294022e-05,
            "line_near_tie": 5.0,
            "line_non_near_tie": 137.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 142.0,
            "line_winner_equal_mac_non_near_tie": 137.0,
            "line_winner_equal_near_tie": 5.0,
            "line_winner_equal_non_near_tie": 137.0,
            "vmhwm_max_kb": 2504960.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 142 questions of the L512 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), GPU OpenCL, GpuOptions precision FP32",
          "total_ops": 2432,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w__pixel-11-pro.json"
      }
    ]
  },
  "model": {
    "family": "mnn-pocket",
    "id": "mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l512_fp16w",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/MNN-Pocket-0.3B-Dec-LiteRT",
    "task": "text-classification"
  },
  "pitfalls": [
    "A decision model, not a chat model: it takes a `state` and typed `questions` (`choice`, `noul`, `score`) and returns the probability of every candidate in one forward pass; it never generates text (README L18).",
    "The graph is the model body only: it takes the embeddings of the request's tokens, the rotary tables and the attention mask and returns the hidden states after the final norm; the Python host in `host/` builds the request, takes the embeddings from `embedding.safetensors` (the source model's token table) and reads the answers with the source model's head and temperature, as the source model's `inference.py` does (README L20, L147-L148).",
    "Each file is a static window: `<n>` in the name is the longest request it takes; run the smallest file that holds the request. A request longer than 2,048 tokens raises `ValueError` in the host, which never cuts it; the source model's own limit is 32,768 tokens (README L71, L84).",
    "The checks measure agreement with the source model's `inference.py`, unchanged, on its own files (float32, CPU, one thread): 2,409 questions on the Mac, a fixed subset of 16 to 322 questions per file on the phones; the bar is the same winner on every question whose two most likely options are more than 0.02 apart, a largest |dp| of at most 0.02 and a mean of at most 0.002 (README L88, L102).",
    "The repository has no Android host yet: an Android app has to port `host/pocket_dec_litert.py`; the phones ran LiteRT 2.3.0 through its C API with inputs the Mac computed with the host's code (README L137).",
    "On a GPU set float32 precision (`GpuOptions(enforce_f32=True)` in the Python host, `CompiledModel.GpuOptions.Precision.FP32` in Kotlin, `kLiteRtDelegatePrecisionFp32` in the C API): an app that sets no precision gets FP16 on both phones, and FP16 changes the answers: among questions whose top two options are more than 0.02 apart the winner changed on up to 5 of 305 on the Galaxy S26, 1 of 305 on the Pixel 11 Pro and 9 of 2,314 on Apple Metal at its default precision, and a probability moved by up to 0.068, 0.029 and 0.058 (README L73).",
    "FP16 with FP32 accumulation passes the bar on the Galaxy S26 (44 of 54 near ties keep the reference's winner, largest |dp| 0.0126) and misses the mean on 3 of the 6 files it ran on the Pixel 11 Pro; it can flip a near tie: in the source model's default example (`department`: `shipping` 0.41064 against `returns` 0.40709) it answers `returns` (README L96, L102, L104).",
    "Set the CPU thread count: with no CPU options LiteRT 2.3.0's C API ran on one thread, and the 128-token file took 1.3 times as long as at 4 threads on the Galaxy S26 and 2.2 times as long on the Pixel 11 Pro; the card's phone CPU rows use 4 threads (Galaxy S26) and 6 (Pixel 11 Pro), whose probabilities equal those of one thread to every digit (README L133).",
    "LiteRT's GPU cache (`serializationDir`, `modelCacheKey`, `serializeProgramCache` and `serializeExternalTensors` in `CompiledModel.GpuOptions`) cut the Pixel 11 Pro's creation of the 128-token model from 15.5 s to 1.8 to 2.0 s, but each call then took 12 to 19 % longer; on the Galaxy S26 it cut 1.6 s to 1.3 s at the same speed; the cache takes 0.93 GiB on disk and raises the process's peak by about 0.9 GiB (README L135)."
  ],
  "schema_version": "1.2"
}
