{
  "artifacts": [
    {
      "file": "qwen3_30b_a3b_mixed_int4_ekv4096.litertlm",
      "sha256": "d908241edce085394d6d444770ddc8998b5c55c3a499d36da5707103da9a8dc1",
      "size_mb": 19693.676
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "Qwen/Qwen3-30B-A3B rev ad44e777 (bf16) -> the decoder written one tensor at a time, never as an fp32 graph: each layer's 128 experts in one moe custom op with activation silu (only the 8 routed experts computed per token; the router, its softmax, the top 8 and their renormalization as ordinary ops; int4 block scales below 2^-14 raised to 2^-14), the residual add after each moe written as (x + moe) + moe * 0 (--moe_fusion_guard); signatures prefill_128, prefill_1024 and decode; KV cache 4,096 tokens; LiteRT-LM's own Qwen3 metadata verbatim (the chat template, stop token <|im_end|> (151645), the qwen3 model type, a thought channel), the checkpoint's tokenizer.json byte for byte (card L36-L40, L72)",
    "quantization": "experts int4 blockwise (block 32, fp32 scales); attention q / k / v / o and lm_head int8 per channel; router, norms and token table fp32; fp32 KV cache of 4,096 tokens; fp32 activations declared (card L30, L32)",
    "tool": "the LiteRT Tensor API (LiteRT dcb657e8 with a small patch in the conversion scripts), each layer's 128 experts written as one TFLite custom op moe; packed with litert-lm-builder 0.19.0; repro = john-rocky/hf-to-litertlm",
    "tool_version": "LiteRT dcb657e8 (+ a small patch) for the writer, litert-lm-builder 0.19.0; every number on the card from litert_lm_advanced_main built from LiteRT-LM dab2e708 with LiteRT dcb657e8 and ml-drift 76492bb9 plus the two open LiteRT pull requests #11192 and #11188 (2026-10-28); LiteRT-LM 0.19.0, the latest release on 2026-11-03, does not run the file (card L19, L23, L76-L96)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-03",
          "decode_tokens_per_s": null,
          "delegated_ops": 3323,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "ERROR: Node number 85 (moe) failed to invoke.",
          "evidence": [
            "bytes: both runs read qwen3_30b_a3b_mixed_int4_ekv4096.litertlm (each run's meta names that path; an APFS clone, made at 10:35 JST for the upload's name, of the bundle qwen3_30b_a3b_r3_xb4b32_i8.litertlm) at 10:40:25 and 10:40:29 JST on 2026-11-03, after the clone's hash of 10:35 JST (d908241edce085394d6d444770ddc8998b5c55c3a499d36da5707103da9a8dc1) and before the upload set's hash of 11:19 JST (the same); = the Hub's LFS oid (20650316480 B) at upload commit d307ec5a and at main f12ad63f (anonymous, read 2026-11-09 19:38:15 JST)",
            "runtime: the PyPI litert-lm 0.19.0 and litert-lm-api 0.19.0 macOS arm64 wheels (Python 3.14), liblitert-lm.dylib sha256 0e1eba3a..., its own CLI: `litert-lm run <file> --backend cpu --top-k 1 --cache no --verbose --prompt 'What is 17 + 25? Answer briefly.'` (started 10:40:25, ended 10:40:29 JST, rc 0; stdout 'An error occurred', the CLI's message for a failed run, exit status 0)",
            "The XNNPACK delegate was created and took 3323 out of 3566 nodes (285 partitions) of subgraph 0; 3323 out of 3566 nodes (285 partitions) of subgraph 1; 3375 out of 3621 nodes (290 partitions) of subgraph 2; the moe custom op was left out of it, one line per moe node: 142 lines 'ERROR: moe node #85 unsupported activation='silu'' (node #85 first)",
            "The first invoke: 'ERROR: Stub operation invoked. This function should not be called.' / 'ERROR: Node number 85 (moe) failed to invoke.' / 'RuntimeError: INTERNAL: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor.cc:936]' / '└ Failed to invoke the compiled model'"
          ],
          "failure_class": "run_error",
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "moe_unsupported_activation_lines": 142.0,
            "questions": 1.0,
            "rc": 0.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "one fixed question ('What is 17 + 25? Answer briefly.') through the PyPI litert-lm 0.19.0 CLI (`litert-lm run`, top-k 1), the released runtime's loadability check",
          "total_ops": 3566,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-03/qwen3-30b-a3b-mixed-int4-ekv4096__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-03",
          "decode_tokens_per_s": null,
          "delegated_ops": 85,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": "E0000 00:00:1793670032.919679 940080324 engine.cc:1124] Failed to create engine: UNKNOWN: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor_factory.cc:287]",
          "evidence": [
            "bytes: both runs read qwen3_30b_a3b_mixed_int4_ekv4096.litertlm (each run's meta names that path; an APFS clone, made at 10:35 JST for the upload's name, of the bundle qwen3_30b_a3b_r3_xb4b32_i8.litertlm) at 10:40:25 and 10:40:29 JST on 2026-11-03, after the clone's hash of 10:35 JST (d908241edce085394d6d444770ddc8998b5c55c3a499d36da5707103da9a8dc1) and before the upload set's hash of 11:19 JST (the same); = the Hub's LFS oid (20650316480 B) at upload commit d307ec5a and at main f12ad63f (anonymous, read 2026-11-09 19:38:15 JST)",
            "runtime: the PyPI litert-lm 0.19.0 and litert-lm-api 0.19.0 macOS arm64 wheels (Python 3.14), liblitert-lm.dylib sha256 0e1eba3a..., its own CLI: `litert-lm run <file> --backend gpu --top-k 1 --cache no --verbose --prompt 'What is 17 + 25? Answer briefly.'` (started 10:40:29, ended 10:40:32 JST, rc 0; stdout 'An error occurred', the CLI's message for a failed run, exit status 0)",
            "'ERROR: Following operations are not supported by GPU delegate:' / 'CUSTOM moe: moe only supports activation='gelu_tanh' (or its legacy spelling 'gelu').' / '85 operations will run on the GPU, and the remaining 3481 operations will run on the CPU.' / 'VERBOSE: Replacing 85 out of 3566 node(s) with delegate (LITERT_WEBGPU) node, yielding 2 partitions for subgraph 0.' (WebGPU on Metal, the runtime stopped at subgraph 0)",
            "'ERROR: Hint fully delegated to single delegate is set, but the graph is not fully delegated.' / 'E0000 00:00:1793670032.919679 940080324 engine.cc:1124] Failed to create engine: UNKNOWN: ERROR: [third_party/odml/litert_lm/runtime/executor/llm_litert_compiled_model_executor_factory.cc:287]' ... '└ Failed to modify graph with delegate'"
          ],
          "failure_class": "engine_create_failed",
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": false,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "questions": 1.0,
            "rc": 0.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "signature": "one fixed question ('What is 17 + 25? Answer briefly.') through the PyPI litert-lm 0.19.0 CLI (`litert-lm run`, top-k 1), the released runtime's loadability check",
          "total_ops": 3566,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-03/qwen3-30b-a3b-mixed-int4-ekv4096__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "qwen3",
    "id": "qwen3-30b-a3b-mixed-int4-ekv4096",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Qwen3-30B-A3B",
    "task": "text-generation"
  },
  "pitfalls": [
    "No LiteRT-LM release ran this file on the card's date: from PyPI, LiteRT-LM 0.19.0 stops on the CPU with `moe node #85 unsupported activation='silu'` and on the GPU with `moe only supports activation='gelu_tanh' (or its legacy spelling 'gelu')`. SiLU in the moe op is in the LiteRT pull requests #11192 (GPU kernel) and #11188 (XNNPACK kernel, CPU), open on 2026-11-03; the card's numbers come from a LiteRT-LM built from source with both (card L23, L76-L96).",
    "On a LiteRT-LM built from source with #11192 and #11188 (not a release), the card measured this file on an Apple M4 Max (128 GB, macOS 27.0), 1,024 prefill and 256 decode tokens, 2026-11-03: WebGPU prefill 778.11 tok/s, decode 47.64 tok/s, peak physical footprint 34.14 GB; CPU (XNNPACK) 56.30 / 4.16 tok/s, 5.37 GB; on its 12 check questions and a 3-turn conversation, the same verdict as Transformers 12 / 12, the stop token 12 / 12, 3 turns pass on both backends (card L19, L47-L53, L62-L68).",
    "Keep the fp32 activation declaration: with fp16 activations every one of the 8 check questions run repeated a short passage of its thought until the token limit, without an answer (card L41).",
    "Every GPU check on the card ran with --sampler_handles_input=false; the default GPU sampler path was compared on one 64-token reply only (card L70).",
    "Thinking is on by default: the CLI prints the thought between [thought] and [/thought], then the answer; the sampler is TOP_P (top_p 0.95, top_k 20, temperature 0.6) (card L21, L40, L108).",
    "Int4 attention and lm_head weights were not faster than int8 ones on WebGPU (41.9 against 41.1 tokens/s) and one of the 12 check questions looped with them, so the files keep them int8 (card L42).",
    "Size: 20.65 GB; no phone was tried (card L30, L58)."
  ],
  "schema_version": "1.2"
}
