{
  "artifacts": [
    {
      "file": "qwen3_coder_30b_a3b_mixed_int4_ekv2048.litertlm",
      "sha256": "41c79c2008af7c4c0fb01c2267d3d83a8a8d8327418b6e5fd974b0a015e1e765",
      "size_mb": 19693.52
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "Qwen/Qwen3-Coder-30B-A3B-Instruct rev b2cff646 (bf16) -> the decoder written one tensor at a time, never as an fp32 graph: each layer's 128 experts in one moe custom op with activation silu (only the 8 routed experts computed per token; the router, its softmax, the top 8 and their renormalization as ordinary ops; int4 block scales below 2^-14 raised to 2^-14), the residual add after each moe written as (x + moe) + moe * 0 (--moe_fusion_guard); signatures prefill_128, prefill_1024 and decode; KV cache 2,048 tokens; the source's chat template rewritten for typed content parts, stop tokens <|im_end|> (151645) and <|endoftext|> (151643), the checkpoint's tokenizer.json byte for byte; generic_model with supports_function_calling and the tool_call_config of LiteRT-LM's models/qwen3_5 (card L38-L44, L75)",
    "quantization": "experts int4 blockwise (block 32, fp32 scales); attention q / k / v / o and lm_head int8 per channel; router, norms and token table fp32; fp32 KV cache of 2,048 tokens; fp32 activations declared (prefer_activation_type) (card L31, L34)",
    "tool": "the LiteRT Tensor API (LiteRT dcb657e8 with a small patch in the conversion scripts), each layer's 128 experts written as one TFLite custom op moe; packed with litert-lm-builder 0.19.0; repro = john-rocky/hf-to-litertlm, (the pack, writer and bundle scripts are Qwen3-30B-A3B's) (card L21, L38-L39, L46)",
    "tool_version": "LiteRT dcb657e8 (+ a small patch) for the writer, litert-lm-builder 0.19.0; every number on the card from litert_lm_advanced_main built from LiteRT-LM dab2e708 with LiteRT dcb657e8 and ml-drift 76492bb9 plus the two open LiteRT pull requests #11192 and #11188 (2026-10-28); LiteRT-LM 0.19.1, the latest release on 2026-11-09, does not run the file (card L21, L25, L79-L99)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-09",
          "decode_tokens_per_s": null,
          "delegated_ops": 3323,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "ERROR: Node number 85 (moe) failed to invoke.",
          "evidence": [
            "bytes: both runs read (each run's meta names that path) at 05:52:01 and 05:55:09 JST on 2026-11-09; the file was written at 05:07:42 JST with sha256 41c79c20... (REBASE_DONE) and hashed again by the upload at 05:46:15 JST (sha256 41c79c2008af7c4c0fb01c2267d3d83a8a8d8327418b6e5fd974b0a015e1e765, 20650152640 B), = the Hub's LFS oid at upload commit a8f3776a and at main 55d1e40d (anonymous, read 2026-11-09 19:24:29 JST); no hash of the local copy after the runs (it was deleted at 06:36:05 JST, after the upload)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel (Python 3.13), liblitert-lm.dylib sha256 c61e0faa..., its own CLI: `litert-lm run <file> --backend cpu --top-k 1 --cache no --verbose --prompt 'What is 17 + 25? Answer briefly.'` (started 05:52:01, ended 05:52:03 JST, rc 0; stdout 'An error occurred', the CLI's message for a failed run, exit status 0)",
            "The XNNPACK delegate was created and took 3323 out of 3566 nodes (285 partitions) of subgraph 0; 3323 out of 3566 nodes (285 partitions) of subgraph 1; 3375 out of 3621 nodes (290 partitions) of subgraph 2; the moe custom op was left out of it, one line per moe node: 142 lines 'ERROR: moe node #85 unsupported activation='silu'' (node #85 first)",
            "The first invoke: 'ERROR: Stub operation invoked. This function should not be called.' / 'ERROR: Node number 85 (moe) failed to invoke.' / 'RuntimeError: INTERNAL: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor.cc:936]' / '└ Failed to invoke the compiled model'"
          ],
          "failure_class": "run_error",
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "moe_unsupported_activation_lines": 142.0,
            "questions": 1.0,
            "rc": 0.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "one fixed question ('What is 17 + 25? Answer briefly.') through the PyPI litert-lm 0.19.1 CLI (`litert-lm run`, top-k 1), the released runtime's loadability check",
          "total_ops": 3566,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-09/qwen3-coder-30b-a3b-mixed-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-09",
          "decode_tokens_per_s": null,
          "delegated_ops": 85,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "E0000 00:00:1794171310.914968 331808235 engine.cc:1124] Failed to create engine: UNKNOWN: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor_factory.cc:287]",
          "evidence": [
            "bytes: both runs read (each run's meta names that path) at 05:52:01 and 05:55:09 JST on 2026-11-09; the file was written at 05:07:42 JST with sha256 41c79c20... (REBASE_DONE) and hashed again by the upload at 05:46:15 JST (sha256 41c79c2008af7c4c0fb01c2267d3d83a8a8d8327418b6e5fd974b0a015e1e765, 20650152640 B), = the Hub's LFS oid at upload commit a8f3776a and at main 55d1e40d (anonymous, read 2026-11-09 19:24:29 JST); no hash of the local copy after the runs (it was deleted at 06:36:05 JST, after the upload)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel (Python 3.13), liblitert-lm.dylib sha256 c61e0faa..., its own CLI: `litert-lm run <file> --backend gpu --top-k 1 --cache no --verbose --prompt 'What is 17 + 25? Answer briefly.'` (started 05:55:09, ended 05:55:10 JST, rc 0; stdout 'An error occurred', the CLI's message for a failed run, exit status 0)",
            "'ERROR: Following operations are not supported by GPU delegate:' / 'CUSTOM moe: moe only supports activation='gelu_tanh' (or its legacy spelling 'gelu').' / '85 operations will run on the GPU, and the remaining 3481 operations will run on the CPU.' / 'VERBOSE: Replacing 85 out of 3566 node(s) with delegate (LITERT_WEBGPU) node, yielding 2 partitions for subgraph 0.' (WebGPU on Metal, the runtime stopped at subgraph 0)",
            "'ERROR: Hint fully delegated to single delegate is set, but the graph is not fully delegated.' / 'E0000 00:00:1794171310.914968 331808235 engine.cc:1124] Failed to create engine: UNKNOWN: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor_factory.cc:287]' ... '└ Failed to modify graph with delegate'"
          ],
          "failure_class": "engine_create_failed",
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": false,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "questions": 1.0,
            "rc": 0.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "one fixed question ('What is 17 + 25? Answer briefly.') through the PyPI litert-lm 0.19.1 CLI (`litert-lm run`, top-k 1), the released runtime's loadability check",
          "total_ops": 3566,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-09/qwen3-coder-30b-a3b-mixed-int4-ekv2048__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "qwen3-coder",
    "id": "qwen3-coder-30b-a3b-mixed-int4-ekv2048",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Qwen3-Coder-30B-A3B-Instruct",
    "task": "text-generation"
  },
  "pitfalls": [
    "No LiteRT-LM release runs this file: from PyPI, LiteRT-LM 0.19.1 stops on the CPU with `moe node #85 unsupported activation='silu'` and on the GPU with `moe only supports activation='gelu_tanh' (or its legacy spelling 'gelu')`. SiLU in the moe op is in the LiteRT pull requests #11192 (GPU kernel) and #11188 (XNNPACK kernel, CPU), open on 2026-11-09; the card's numbers come from a LiteRT-LM built from source with both (card L25, L79-L99).",
    "On a LiteRT-LM built from source with #11192 and #11188 (not a release), the card measured this file on an Apple M4 Max (128 GB, macOS 27.0) with WebGPU, 1,024 prefill and 256 decode tokens, 2026-11-09: prefill 1,038.14 tok/s, decode 54.95 tok/s, peak physical footprint 32.64 GB; on its 12 check questions and a 3-turn conversation, the same verdict as Transformers 12 / 12, the stop token 12 / 12, 3 turns pass; the KV cache holds 2,048 tokens (card L21, L34, L52-L54, L65-L69).",
    "Keep the fp32 activation declaration: copies of the files that declare fp16 decoded 9 to 17 % faster, but no reply on WebGPU had an answer (empty, a single `*` before the stop token, or a short pattern repeated over the whole reply) (card L21, L41).",
    "The card's GPU numbers and checks used --sampler_handles_input=false; without it, two runs of the four code questions gave the same verdicts but not the same text. The Python, Kotlin and Swift APIs of LiteRT-LM 0.19.1 have no setting for it (card L113).",
    "Tool calls are XML (<tool_call>, <function=…>, <parameter=…>), declared through generic_model's tool_call_config; the card compared the calls' text with Transformers by giving the CLI the template's text as is (no template applied) and did not run the runtime's own parsing of a call (card L44).",
    "The sampler is TOP_P (top_p 0.8, top_k 20, temperature 0.7); the source's repetition_penalty of 1.05 is not in the file (LiteRT-LM's metadata has no field for it) (card L45).",
    "Size: 20.65 GB; no phone was tried (card L31, L61)."
  ],
  "schema_version": "1.2"
}
