{
  "artifacts": [
    {
      "file": "mnn-pocket-0.3b-dec_L2048_fp16w_npu.tflite",
      "sha256": "8fb1fc76153c92a097f5d5499cd96cf7bc6b490afa48fcc8d30415c24ee72999",
      "size_mb": 466.257
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "the repository's conversion/ (conversion/README.md) holds the scripts and REPRODUCE.md every check (README L150); the card names the tools, not a command line",
    "quantization": "float16 linear-layer weights, the LoRA adapter folded into the weights first; SiLU written with EXP in place of LOGISTIC for the NPU (README L20, L129)",
    "tool": "litert-torch",
    "tool_version": "0.10.0 (for LiteRT 2.3.0)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "npu_qnn",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": 2600,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-mnnpocketdec-capi-runner",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": "QAIRT 2.50.0.260828221209"
          },
          "error": "2026-11-07 22:42:30 MemAvailable 1417132 kB < 1500000 kB -> kill -9 runner pid 32300",
          "evidence": [
            "(tag s26_j_L2048, run r4): run OOM_GUARD rc 137, start 2026-11-07 22:40:48, the 6 requests, none answered",
            "Push mnn_pocket_dec_ns4_in_6-27_post_6-27_final-xsilu_L2048_w16.tflite rc=0 bytes 488906048/488906048 sha256 phone=8fb1fc76153c92a097f5d5499cd96cf7bc6b490afa48fcc8d30415c24ee72999 = mac = the Hub LFS sha256",
            "Runner 7e7539e5bd0c73ed… and libs: 12/12 OK",
            "Device: SM-S942Q 16 BP4A.251205.006.S942QOPS1AZH9",
            "11-07 22:40:48.921 32300 32300 I litert  : [compiler_plugin.cc:742] Partitioned subgraph<0>, selected 2600 ops, from a total of 2600 ops. resulted in 1 partitions.",
            "2026-11-07 22:42:30 MemAvailable 1417132 kB < 1500000 kB -> kill -9 runner pid 32300",
            "Run memory: VmHWM_max_kB 6814564 VmSwap_max_kB 69632 MemAvailable_min_kB 1417132 kgsl_page_alloc_max_B 214724608; the compile never finished and no question was answered (the record's verdict fields over those unanswered questions are not entered)"
          ],
          "failure_class": "memory_guard_stop_during_compile",
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "compile_s_at_stop": 101.079,
            "mem_available_kb_at_stop": 1417132.0,
            "memory_guard_kb": 1500000.0,
            "vmhwm_max_kb": 6814564.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "the line: the L2048 set against the source model's reference; the C-API runner build 7e7539e5 (LiteRT 2.3.0), Qualcomm HTP, the graph compiled on the phone by the compiler plugin, HTP performance mode Burst",
          "total_ops": 2600,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l2048_fp16w_npu__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu_xnnpack",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final-xsilu_L2048_w16, arm cpu4, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator CPU (XNNPACK), threads 4, is_fully_accelerated False (the API states no node count), compile 0.16 s",
            "date: the record's first commit, author date 2026-11-07 21:35:07 +0900, after the run's START 21:07:17 (the run log has times only and is not in git)",
            "line PASS: 2409 questions, same winner 2323/2323 (not near ties), near ties kept 86/86, max |Δp| 0.00208836, mean |Δp| 0.000110553, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 0.16,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.040072932879427065,
            "line_max_abs_dp": 0.00208835537685223,
            "line_mean_abs_dp": 0.00011055256136216093,
            "line_near_tie": 86.0,
            "line_non_near_tie": 2323.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 2409.0,
            "line_winner_equal_near_tie": 86.0,
            "line_winner_equal_non_near_tie": 2323.0,
            "threads": 4.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 2409 questions (the requests that fit L2048) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), CPU XNNPACK, 4 threads",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l2048_fp16w_npu__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu_metal_mac",
          "context_length": null,
          "date": "2026-11-07",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert",
            "runtime_version": "2.3.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Name mnn_pocket_dec_ns4_in_6-27_post_6-27_final-xsilu_L2048_w16, arm gpu_f32, child rc 0",
            "Runtime: api CompiledModel.from_file, accelerator GPU (Metal), gpu_options {\"enforce_f32\": true}, is_fully_accelerated True (the API states no node count), compile 2.99 s",
            "date: the record's first commit, author date 2026-11-07 21:35:07 +0900, after the run's START 21:00:34 (the run log has times only and is not in git)",
            "line PASS: 2409 questions, same winner 2323/2323 (not near ties), near ties kept 86/86, max |Δp| 0.00208496, mean |Δp| 0.000110538, control 0.0401, non-finite read-outs 0"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "arm_files_passed": 20.0,
            "arm_files_run": 20.0,
            "compile_s": 2.99,
            "line_bar_control_abs_dp": 0.02,
            "line_bar_max_abs_dp": 0.02,
            "line_bar_mean_abs_dp": 0.002,
            "line_bar_near_tie_gap": 0.02,
            "line_control_abs_dp": 0.04007347356783075,
            "line_max_abs_dp": 0.0020849579057110046,
            "line_mean_abs_dp": 0.00011053820121758387,
            "line_near_tie": 86.0,
            "line_non_near_tie": 2323.0,
            "line_nonfinite_questions": 0.0,
            "line_pass": 1.0,
            "line_questions": 2409.0,
            "line_winner_equal_near_tie": 86.0,
            "line_winner_equal_non_near_tie": 2323.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the line: 2409 questions (the requests that fit L2048) against the source model's reference; Python CompiledModel (ai-edge-litert 2.3.0), GPU Metal, GpuOptions(enforce_f32=True)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/2.3.0/2026-11-07/mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l2048_fp16w_npu__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "mnn-pocket",
    "id": "mnn-pocket-0.3b-dec__mnn-pocket-0.3b-dec_l2048_fp16w_npu",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/MNN-Pocket-0.3B-Dec-LiteRT",
    "task": "text-classification"
  },
  "pitfalls": [
    "A decision model, not a chat model: it takes a `state` and typed `questions` (`choice`, `noul`, `score`) and returns the probability of every candidate in one forward pass; it never generates text (README L18).",
    "The graph is the model body only: it takes the embeddings of the request's tokens, the rotary tables and the attention mask and returns the hidden states after the final norm; the Python host in `host/` builds the request, takes the embeddings from `embedding.safetensors` (the source model's token table) and reads the answers with the source model's head and temperature, as the source model's `inference.py` does (README L20, L147-L148).",
    "Each file is a static window: `<n>` in the name is the longest request it takes; run the smallest file that holds the request. A request longer than 2,048 tokens raises `ValueError` in the host, which never cuts it; the source model's own limit is 32,768 tokens (README L71, L84).",
    "The checks measure agreement with the source model's `inference.py`, unchanged, on its own files (float32, CPU, one thread): 2,409 questions on the Mac, a fixed subset of 16 to 322 questions per file on the phones; the bar is the same winner on every question whose two most likely options are more than 0.02 apart, a largest |dp| of at most 0.02 and a mean of at most 0.002 (README L88, L102).",
    "The repository has no Android host yet: an Android app has to port `host/pocket_dec_litert.py`; the phones ran LiteRT 2.3.0 through its C API with inputs the Mac computed with the host's code (README L137).",
    "Set the HTP performance mode to Burst: without it the calls on the Galaxy S26 took 5.5 to 6.7 times as long (76.3 ms against 11.3 ms for the 60-token example); in the C API it is `htp_performance_mode = 2` (`kLiteRtQualcommHtpPerformanceModeBurst`) in the Qualcomm options, in Kotlin `CompiledModel.QualcommOptions(htpPerformanceMode = CompiledModel.QualcommOptions.HtpPerformanceMode.BURST)` set as `qualcommOptions` of `CompiledModel.Options` (README L127).",
    "LiteRT compiles the file on the phone at its first load: on the Galaxy S26 9 s (64 tokens) to 315 s (1,024 tokens), with a process peak of 2.8 to 5.0 GB; later loads read the compiled result only when the app gives LiteRT a compiler cache directory (`kLiteRtEnvOptionTagCompilerCacheDir` in the C API's environment options), otherwise every load compiles again; tested on the SM8850 only (README L81, L129).",
    "The 2,048-token file did not finish compiling on the Galaxy S26: the memory watch stopped it at 7.0 GB, when the phone's available memory fell below 1.5 GB; the SM8850 file of the same length loads in 1.2 s (README L129).",
    "SiLU is written with EXP in place of LOGISTIC, because the NPU's LOGISTIC returned 0 for inputs below about -5.7 (measured on the Galaxy S26 with QAIRT 2.47, on another model's graph) (README L129).",
    "Running it needs LiteRT's Qualcomm NPU runtime (Maven `litert-npu-runtime-qualcomm` 2.3.0) and Qualcomm's QAIRT 2.50 libraries (README L129).",
    "On the Google Tensor NPU of the Pixel 11 Pro the Qualcomm NPU files changed the winner on up to 31 of 305 questions: that NPU ran the graphs in FP16, where values below 2^-14 became 0, and the squares of small values in the norms were lost; the `_tensor` files of the repository are written for it (README L82, L131)."
  ],
  "schema_version": "1.2"
}
