{
  "artifacts": [
    {
      "file": "Ornith-1.0-9B_mixed_int4.litertlm",
      "sha256": "bf01eff637705a82c1a7dab65c922b9bab5a8fdb445ffeb6ac6aa67c860ee80c",
      "size_mb": 5779.889
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "convert_qwen35_9b.sh (hf-to-litertlm; ornith-ai/Ornith-1.0-9B at 83dc1f5e = the Qwen3.5-9B recipe run with this repo's id and revision): text-decoder export (6 prefill signatures 1024..1 + decode, a 4096-position KV budget, 64 state buffers) -> int8 / Mixed INT4 quantization with the 48 in_proj_a / in_proj_b projections float -> fp32 GPU activations declared -> the thinking-off template (the only metadata field changed); run as published on 2026-10-16 it rebuilt all four sections of both files byte for byte",
    "quantization": "Mixed INT4: int4 blockwise-32 min-max on the linear layers, int8 on the lm_head and the embedding, the 48 decay/gate projections float; 216,006,656 block scales, all normal floats (none zero or subnormal); fp32 GPU activations declared (card L32, L106)",
    "tool": "litert-torch (pinned commit 115a1360 + the Qwen3.5 hybrid patch of hf-to-litertlm, 2026-08-21 version; lightweight export, constants of 1e8 bytes and more stay lazy) + post-hoc ai-edge-quantizer; repro = hf-to-litertlm qwen35_work/",
    "tool_version": "litert-torch 115a1360 + patch, transformers 5.14.1, torch 2.12.1, ai-edge-quantizer 0.8.0; packed and checked with litert-lm 0.18.0"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (step v0171, \"$(F $n)\" = Ornith-1.0-9B_mixed_int4.litertlm): uvx --from litert-lm==0.17.1 litert-lm run \"$(F $n)\" --backend cpu --prompt \"What is the capital of France? Answer in one word.\" --cache no",
            "bundle: Ornith-1.0-9B_mixed_int4.litertlm sha256 bf01eff637705a82c1a7dab65c922b9bab5a8fdb445ffeb6ac6aa67c860ee80c = the Hugging Face LFS oid of litert-community/Ornith-1.0-9B at main (upload commit f0360451), tree API read 2026-10-17 17:48 JST; run as Ornith-1.0-9B_mixed_int4.litertlm, an APFS clone with the same sha256",
            "RAM check: held 3 samples after 120 s (reclaimable 98.5 GB (need 70))",
            "stdout: Paris",
            "EXIT=0",
            "2026-10-15 07:12:42 JST 0.17.1 run int4 exit 0: Paris EXIT=0"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_with_expect": 1.0,
            "processes": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-15/ornith-1.0-9b-mixed-int4__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 15.765,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ornith-1.0-9B_mixed_int4.litertlm --backend cpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "bundle: Ornith-1.0-9B_mixed_int4.litertlm sha256 bf01eff637705a82c1a7dab65c922b9bab5a8fdb445ffeb6ac6aa67c860ee80c = the Hugging Face LFS oid of litert-community/Ornith-1.0-9B at main (upload commit f0360451), tree API read 2026-10-17 17:48 JST; run as Ornith-1.0-9B_mixed_int4.litertlm, an APFS clone with the same sha256",
            "pass 1: prefill 65.84 / decode 13.77 tok/s, TTFT 15.6267 s, init 67.1479 s",
            "pass 2: prefill 71.04 / decode 17.76 tok/s, TTFT 14.4719 s, init 50.3152 s",
            "2026-10-15 08:04:05 JST cpu int4 p1 exit 0 (guard_peak: pea ...",
            "2026-10-15 08:15:48 JST cpu int4 p2 exit 0 (guard_peak: pea ...",
            "process footprint (swap guard, 15 s samples): max 30.06 GB = 28 GiB; each run alone behind a RAM gate (reclaimable >= 70 GB x 3) in its own held window; no guard kill"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "guard_footprint_max_gb": 30.06,
            "init_s": 58.7315,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 13.77,
            "pass1_init_s": 67.1479,
            "pass1_prefill_tok_s": 65.84,
            "pass1_ttft_s": 15.6267,
            "pass2_decode_tok_s": 17.76,
            "pass2_init_s": 50.3152,
            "pass2_prefill_tok_s": 71.04,
            "pass2_ttft_s": 14.4719,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 31.4864
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 68.44,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 15049.3
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ornith-1.0-9b-mixed-int4__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 44.925,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ornith-1.0-9B_mixed_int4.litertlm --backend gpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "bundle: Ornith-1.0-9B_mixed_int4.litertlm sha256 bf01eff637705a82c1a7dab65c922b9bab5a8fdb445ffeb6ac6aa67c860ee80c = the Hugging Face LFS oid of litert-community/Ornith-1.0-9B at main (upload commit f0360451), tree API read 2026-10-17 17:48 JST; run as Ornith-1.0-9B_mixed_int4.litertlm, an APFS clone with the same sha256",
            "pass 1: prefill 604.33 / decode 44.75 tok/s, TTFT 1.7168 s, init 89.3817 s",
            "pass 2: prefill 566.38 / decode 45.1 tok/s, TTFT 1.8302 s, init 97.2786 s",
            "2026-10-15 09:56:44 JST gpu int4 p1 exit 0",
            "pass 2 (4 held windows released rc=0) = no export or quantize of another job ran meanwhile",
            "delegation from a separate verbose run of the same bytes (`litert-lm run <file> --backend gpu --cache no --verbose`, 2026-10-15 00:05:28 JST; reply \"The capital of France is **Paris**.\"): Replacing 34373 out of 34373 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 0 (prefill_1024). .. Replacing 2920 out of 2920 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 6 (decode). (7 decoder signatures, 132963 / 132963 nodes)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 93.3302,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 44.75,
            "pass1_init_s": 89.3817,
            "pass1_prefill_tok_s": 604.33,
            "pass1_ttft_s": 1.7168,
            "pass2_decode_tok_s": 45.1,
            "pass2_init_s": 97.2786,
            "pass2_prefill_tok_s": 566.38,
            "pass2_ttft_s": 1.8302,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 7.4497
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 585.355,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 1773.5
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ornith-1.0-9b-mixed-int4__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "ornith",
    "id": "ornith-1.0-9b-mixed-int4",
    "license": "mit",
    "source_url": "https://huggingface.co/litert-community/Ornith-1.0-9B",
    "task": "text-generation"
  },
  "pitfalls": [
    "Text only: the decoder of the multimodal release; no image input. Only the 8 attention layers keep a growing KV cache (a 4096-token budget; Ornith recommends 262,144); the 24 linear-attention layers carry a constant-size conv and recurrent state (card L23, L25).",
    "Runtime: tested on litert-lm 0.18.0 (Mac); on litert-lm 0.17.1 the file answered a plain `run` with \"Paris\" (Mac CPU, one run) (card L21).",
    "Thinking off: the bundle's 701-byte template writes the empty think block into every assistant turn and the generation prompt (Ornith's enable_thinking=False form) with no thinking switch, because the Qwen3.5-4B conversion's switch template drops the think block that Ornith's template writes into past assistant turns (card L27, L77).",
    "Template: byte-identical to Ornith's enable_thinking=False render in six conversation shapes without tools; the two tool shapes differ; no start token; stop ids 248044 and 248046; the tokenizer section is the repo's tokenizer.json byte for byte (card L49, L77).",
    "fp32 GPU activations are declared on both files: with the runtime's fp16 default the int8 weights answered 0 of 8 check questions (every reply a run of `!`) (card L47, L63).",
    "Tool calling: a tool passed through the API changes nothing in the prompt; with Ornith's own `# Tools` block as the system message the model writes the call in Qwen3.5's XML form as text, which the runtime (generic model type) does not parse or run (LiteRT-LM issue 3927) (card L79).",
    "One Engine per conversation: litert-lm 0.18.0 starts a new Conversation on a used Engine from the earlier conversation's recurrent state (LiteRT-LM issue 3165); the probe on these files showed no carry-over, unlike the Qwen3.5-4B conversion's Mixed INT4 file (card L81).",
    "Galaxy S26: the GPU engine of this file does not build on a phone of the S26's memory class: on a remote device lab Galaxy S26 (SM-S942U1, 2026-10-16) four sessions of this file and a three-signature variant ended during the engine build while the low-memory killer ran, with no output; the own phone was not tried; the CPU was not run (12.1 GB by the CPU peak factor against the phone's 11.66 GB) (card L94, L98).",
    "Pixel 8a: not run; no Qwen3.5 Hub file built a GPU engine on this 7.75 GB Mali phone under a memory guard (card L99). iOS not measured: one 5.64 GiB TFLite section, over the ~2 GiB single-section limit (card L43).",
    "Accuracy: GSM8K n=100 (thinking off, greedy, at most 2,048 output tokens, a fresh process per question, Mac GPU): bf16 95 / Mixed INT4 95; the file agrees with bf16 on 92 of 100 questions (right only in bf16 4, right only in this file 4) (card L34, L55-L63).",
    "Mac: the faster file on the GPU (8.8 % less time than int8, decode 44.9 vs 40.0 tok/s at about the same prefill rate), the slower one on the CPU (its blockwise prefill runs at 68 tok/s against 227); without --cache, `run` writes a 4.96 GB compiled-weights cache next to the model; 30.1 GB of memory during the CPU benchmark (card L43, L75, L94).",
    "Three-turn check: three of four runs gave the bf16 model's replies word for word; this file on the GPU chose another word (\"Enormous\") and kept it across the turns (card L49)."
  ],
  "schema_version": "1.2"
}
