{
  "artifacts": [
    {
      "file": "gpt_oss_120b_mixed_int4_ekv4096.litertlm",
      "sha256": "13aa3c316a94579e51b2fd1d56beac89552025d31ef7d07d3e7a58028b11632c",
      "size_mb": 73680.039
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "openai/gpt-oss-120b rev b5c939de -> each layer's 128 experts as one TFLite custom op `moe` (only the 4 routed experts computed for each token; the router with its bias, the top 4 and the softmax over their 4 logits as ordinary ops), the experts' biases carried by one more input block of 32 columns (exact), GELU in its tanh form in place of the source's clamped SwiGLU, attention with its biases, the per-head sink logits, the 128-token window on the even layers and YaRN RoPE as in the source; signatures prefill_128, prefill_1024 and decode, a 4,096-token KV cache; the source's chat template with four edits for LiteRT-LM; the harmony headers declared as channels (card L33-L43)",
    "quantization": "experts int4 blockwise (block 32, fp32 scales); attention q / k / v / o and lm_head int8 per channel; router, biases, sinks, norms and token table fp32; fp32 KV cache (card L29)",
    "tool": "the LiteRT Tensor API (the decoder written one tensor at a time from the checkpoint, one layer read at a time, no fp32 copy of the whole model), packed with litert-lm-builder 0.19.1; repro = john-rocky/hf-to-litertlm",
    "tool_version": "litert-lm-builder 0.19.1; LiteRT-LM 0.19.1 from PyPI for the Performance numbers; runtime floor LiteRT-LM 0.19.0 (card L19, L35, L53)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-10",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran the HDD original (gpt-oss-120b_xg4.litertlm in the archive dir; each log or record names it), sha256 13aa3c316a94579e51b2fd1d56beac89552025d31ef7d07d3e7a58028b11632c, 77259120896 B as the bundle step hashed the bytes it wrote = the upload's record (preupload_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json) = the Hub's LFS oid at upload commit c903efbf and at main 60de77f0 = a streamed hash of the Hub's bytes (stream_sha_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json); its pages were resident before each run (tools/resident.py: RESIDENT 1.0000)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel's Python API (tools/oss_line3.py), CPU (XNNPACK) 8 threads, greedy (top-k 1), at most 2,048 new tokens, one fresh conversation per question, the file's own chat template (the harmony channels; the verdict reads the final channel only)",
            "output_match = line 3 (a) and (b) as written before any output: (a) the file's verdict equals the reference's on 12 of 12 (at least 11 required); (b) every question the reference ends with a stop token ends with one here (12 / 12; none reached the cap). Reference = the source model in MLX (mlx 0.32.2, mlx-lm 0.31.3, the MXFP4 experts as stored, the YaRN frequencies set to Transformers' unrounded ones), whose next token equals Transformers' fp32 forward pass at 445 / 445 positions over the first 2 layers (line R', max |d logit| 3.8e-5) and whose replies to 3 of the questions, fed to Transformers' fp32 forward pass over all 36 layers, are its top-1 token at 413 / 413 positions (R2)",
            "the 3-turn conversation (line 3 (c)): all three turns of the file end with a stop and pass (turn 3's code passes the 4 asserts); the reference's own turn 3 renamed the function and fails them, so (c) by its letter is False and line 3' (written after these outputs were seen) is True; not part of this record's output_match",
            "(05:45:01-05:57:36 JST, inside the memory window): the 12 questions 381.8 s, the conversation 369.7 s, 754.5 s for the run; = the card's Accuracy section"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_right": 12.0,
            "conversation_turns": 3.0,
            "conversation_turns_right": 3.0,
            "conversation_turns_stopped": 3.0,
            "cpu_threads": 8.0,
            "ended_with_stop_token": 12.0,
            "max_new_tokens": 2048.0,
            "questions": 12.0,
            "reply_tokens_max": 368.0,
            "run_seconds_max": 106.6,
            "verdicts_equal_reference": 12.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the 12 check questions of line 3 (arithmetic, facts, translations, a rhyme, 0.9 vs 0.11, a word problem, 4 code tasks run on fixtures), greedy (top-k 1), at most 2,048 new tokens per answer, one conversation per question; verdicts on the final channel against the source model's replies (MLX)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-10/gpt-oss-120b-mixed-int4-ekv4096__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-10",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran the HDD original (gpt-oss-120b_xg4.litertlm in the archive dir; each log or record names it), sha256 13aa3c316a94579e51b2fd1d56beac89552025d31ef7d07d3e7a58028b11632c, 77259120896 B as the bundle step hashed the bytes it wrote = the upload's record (preupload_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json) = the Hub's LFS oid at upload commit c903efbf and at main 60de77f0 = a streamed hash of the Hub's bytes (stream_sha_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json); its pages were resident before each run (tools/resident.py: RESIDENT 1.0000)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel's Python API, the card's own example (`litert_lm.Engine(<file>, backend=litert_lm.Backend.CPU(thread_count=8), cache_dir=':nocache')`, one conversation, 'What is the capital of Japan? Answer briefly.'), run on a link named as the Hub's file (tools/window.sh, part card)",
            "The reply 'Answer: Tokyo.' with its analysis channel; 22.8 s wall, peak memory footprint 5501495448 B, max RSS 80625893376 B (the card's 80.6 GB); a loadability check of the card's command (output_match null: one fixed question, not a comparison with a reference)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cpu_threads": 8.0,
            "questions": 1.0,
            "rc": 0.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the card's example (the litert_lm Python API, CPU, 8 threads): one fixed question ('What is the capital of Japan? Answer briefly.')",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-10/gpt-oss-120b-mixed-int4-ekv4096__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 4096,
          "date": "2026-11-10",
          "decode_tokens_per_s": 4.75,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran the HDD original (gpt-oss-120b_xg4.litertlm in the archive dir; each log or record names it), sha256 13aa3c316a94579e51b2fd1d56beac89552025d31ef7d07d3e7a58028b11632c, 77259120896 B as the bundle step hashed the bytes it wrote = the upload's record (preupload_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json) = the Hub's LFS oid at upload commit c903efbf and at main 60de77f0 = a streamed hash of the Hub's bytes (stream_sha_gpt_oss_120b_mixed_int4_ekv4096.litertlm.json); its pages were resident before each run (tools/resident.py: RESIDENT 1.0000)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel's CLI, `litert-lm benchmark <file> --backend cpu -p 1024 -d 256 --max-num-tokens 4096 --runs 1 --cache no --cpu-thread-count 8` (tools/window.sh, part bench, inside the quiet window), under /usr/bin/time -l; CPU = XNNPACK; the CLI runs its own warm-up first and prints one run; the log prints no partition or node count",
            "(started 2026-11-10 05:42:01 JST, CPU 73 % idle, load averages in the .meta): prefill 49.94 tokens/s, decode 4.75 tokens/s, time to first token 20.7147 s, init 5.9217 s (the CLI's Init sums every phase inside Init Total, LiteRT-LM #4057, registry cli-benchmark-init-summed; not quoted on the card); peak memory footprint 6978268368 B = peak_mem_mb / 10^6, max RSS 82125815808 B (the mapped file's pages)",
            "the card's first paragraph and Performance section: 49.94 / 4.75 tokens/s at 1,024 / 256 tokens, time to first token 20.7 s, peak footprint 6.98 GB, peak resident memory 82.1 GB (2026-11-10)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cpu_idle_at_start_pct": 73.0,
            "cpu_threads": 8.0,
            "decode_tokens": 256.0,
            "init_time_s": 5.9217,
            "max_num_tokens": 4096.0,
            "max_rss_gb": 82.13,
            "prefill_tokens": 1024.0,
            "runs": 1.0,
            "wall_s": 154.03
          },
          "output_match": null,
          "peak_mem_mb": 6978.268368,
          "prefill_tokens_per_s": 49.94,
          "provenance": "measured",
          "runs": true,
          "signature": null,
          "total_ops": null,
          "ttft_ms": 20714.7
        },
        "source": "data/device_runs/0.19.1/2026-11-10/gpt-oss-120b-mixed-int4-ekv4096__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "gpt-oss",
    "id": "gpt-oss-120b-mixed-int4-ekv4096",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/gpt-oss-120b",
    "task": "text-generation"
  },
  "pitfalls": [
    "Memory: the 77.3 GB file is mapped into memory whole and the process adds 5.5 to 7.0 GB (peak resident memory 80.6 GB for the Python example, 82.1 GB for the benchmark); it runs on a Mac with 128 GB. The GPU has not been measured yet; no phone was tried (card L19, L60, L62).",
    "Not the source's exact computation: the experts' MXFP4 weights are re-quantized to int4, and the expert activation is GELU (tanh form) without the source's clamp at 7, because the moe op of LiteRT-LM 0.19.1 has no clamp and no SiLU. With that activation on the MXFP4 weights, the hidden state differs from the source's by 21 to 66 % (relative L2) from layer 6 on, and the next token is the source's at 399 of 413 reply positions (teacher-forced); this file adds the int4 experts on top. On 12 questions its greedy replies got the same verdict as the source model's on all 12 (card L23, L38, L70).",
    "Long replies: with LiteRT-LM 0.19.1 a blocking send_message still generating after 10 minutes fails with DEADLINE_EXCEEDED; the file decodes about 4.8 tokens/s on the M4 Max CPU, so 10 minutes is about 2,800 tokens, reasoning included; use send_message_async, or cap max_output_tokens (card L77).",
    "A message that does not fit: on LiteRT-LM 0.19.1, after a message fails because it does not fit in the context window (4,096 tokens), every later message in that conversation fails too; start a new conversation (card L78).",
    "Tool calls: the metadata type is generic_model, so LiteRT-LM returns a tool call as text: the function name in the commentary_header or tool_header channel, the arguments in the content, the reply stopping at <|call|>; to continue after a tool call, start a new conversation with the history (card L45).",
    "Sampling: the file carries Transformers' defaults for the source's generation_config (type TOP_P: top_k 50, top_p 1.0, temperature 1.0), so the text varies from run to run; the card's checks are greedy (card L46, L66, L101)."
  ],
  "schema_version": "1.2"
}
