{
  "artifacts": [
    {
      "file": "Olmo-3-7B-Think_int4_c2048.litertlm",
      "sha256": "47faae994b95e1ff7ab3b28a86cefcb8cec90cd96f987c7f865c7ace2f5eb0cc",
      "size_mb": 4109.803
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "convert_olmo3_7b_think.sh (hf-to-litertlm): the int4 export with a 2,048-position KV budget (6 prefill signatures 1024..1 + decode, embedder externalized, the bundle template embedded, the 2^-14 block-scale floor) -> int4 block-scale scan -> metadata edit (no start token, stop ids 100265 / 100257, the thought channel, max_num_tokens 2048) -> the repo's tokenizer.json as the tokenizer section",
    "quantization": "int4 blockwise-32 (OCTAV) on every linear layer, block scales floored at 2^-14, int8 embedding table; activations float; the same weights and recipe as Olmo-3-7B-Think_int4.litertlm (card L33, L108)",
    "tool": "litert-torch (stock export_hf path, lightweight conversion, embedding table as its own section) with the int4 block scales floored at 2^-14 before rounding; repro = hf-to-litertlm olmo3_work/",
    "tool_version": "litert-torch 0.9.4 (litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0); packed and checked with litert-lm 0.18.0"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-16",
          "decode_tokens_per_s": 46.575,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes; one pass for the four files, the header): Olmo-3-7B-Think_int4_c2048.litertlm --backend gpu, prefill 1024, decode 256, max tokens 2048, --runs 3, --cache no",
            "pass 1: prefill 943.84 / decode 46.82 tok/s, TTFT 1.1063 s, init 6.1196 s",
            "pass 2: prefill 940.44 / decode 46.33 tok/s, TTFT 1.1104 s, init 6.1333 s",
            "each GPU pass logged 'TopK requires src tensor C dimension to be divisible by 4.' (4 / 4 lines: warmup + 3 iterations; vocabulary 100,278 = 2 mod 4)",
            "delegation from the verbose run of the same bytes (2026-10-16, the run r8 gate): Replacing 1436 out of 1436 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 0 (prefill_1024). .. Replacing 1336 out of 1336 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 6 (decode). (7 decoder signatures, 9823 / 9823 nodes)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 6.1265,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 46.82,
            "pass1_init_s": 6.1196,
            "pass1_prefill_tok_s": 943.84,
            "pass1_ttft_s": 1.1063,
            "pass2_decode_tok_s": 46.33,
            "pass2_init_s": 6.1333,
            "pass2_prefill_tok_s": 940.44,
            "pass2_ttft_s": 1.1104,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 6.5836
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 942.14,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 1108.4
        },
        "source": "data/device_runs/0.18.0/2026-10-16/olmo-3-7b-think-int4-c2048__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "olmo3",
    "id": "olmo-3-7b-think-int4-c2048",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Olmo-3-7B-Think",
    "task": "text-generation"
  },
  "pitfalls": [
    "Context of up to 2,048 tokens: prompt, thought and answer share a 2,048-token KV budget; with a 1,900-token output budget GSM8K scored 77 (47 unclosed thoughts; 12 replies used the whole budget and 37 more ended at the 2,048-token context), against 80 for the 4,096-token-cache int4 file; the two files (the same weights) gave the same final number on 71 of 100 (card L64, L66).",
    "Galaxy S26 (12 GB class): the form that builds its GPU engine there - on a remote device lab unit it ran the 1024 / 256 workload in 41.3 s (prefill 211.8, decode 7.03 tok/s; peak 2.33 GB RSS; all 7 decoder signatures on the OpenCL delegate) on a unit that started clock-capped, so a platform-session figure, not a headline for a phone in hand; the 4,096-token-cache files did not build there (card L40, L100).",
    "Mac GPU: 23 % less time than the int4 file (6.58 vs 8.59 s; decode 46.6 vs 34.8 tok/s) at a context of 2,048 tokens (card L45, L96).",
    "Always thinks: --thinking false and --thinking-budget 0 change nothing; a positive budget closes the thought channel while the model goes on reasoning in the visible text (card L27, L79).",
    "Tool calling is not wired and the chat template, stop ids and GPU-sampler note are those of the int4 file: tools= not rendered, no tool-call parser, no start token, stop ids 100265 / 100257, the last two ids never sampled on the GPU (card L51, L81, L83).",
    "Pixel 8a not run and GPU execution on a Mali phone not verified; not run on an NPU; iOS not measured (decoder section over ~2 GiB) (card L45, L101-L102)."
  ],
  "schema_version": "1.2"
}
