{
  "artifacts": [
    {
      "file": "Ministral-3-8B-Instruct-2512_int8.litertlm",
      "sha256": "f2f6e6621fa84a7c5c295b7a190270aa77453417de707c94daf6c144e2f9418b",
      "size_mb": 8238.622
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "convert_ministral3_8b.sh (hf-to-litertlm): scripts/extract_ministral3_text.py (Mistral3ForConditionalGeneration -> Ministral3ForCausalLM, the vision tower and the projector dropped) -> export (6 prefill signatures 1024..1 + decode, KV cache 4096, embedder externalized, the bundle template embedded) -> metadata edit (stop id 2 only, max_num_tokens 4096) -> tools/gpu_graph/gpu_graph_retrofit.py --decomp exporter --mask add_bcast",
    "quantization": "int8 dynamic on linears, float activations; the embedding table as a separate section (card L25)",
    "tool": "litert-torch (text tower extracted from the multimodal checkpoint; lightweight export, 1e8-byte lazy-constant threshold) + hf-to-litertlm tools/gpu_graph/ GPU-composite rewrite; repro = hf-to-litertlm ministral_work/",
    "tool_version": "litert-torch 0.9.4 (litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0); packed by the litert-lm 0.18.0 CLI"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 11.49,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ministral-3-8B-Instruct-2512_int8.litertlm --backend cpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "pass 1: prefill 122.77 / decode 11.48 tok/s, TTFT 8.4284 s, init 60.1686 s",
            "pass 2: prefill 125.32 / decode 11.5 tok/s, TTFT 8.2581 s, init 55.3633 s",
            "process footprint (swap guard, 15 s samples, binary GB); no guard kill"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "guard_footprint_max_gb": 50.0,
            "init_s": 57.766,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 11.48,
            "pass1_init_s": 60.1686,
            "pass1_prefill_tok_s": 122.77,
            "pass1_ttft_s": 8.4284,
            "pass2_decode_tok_s": 11.5,
            "pass2_init_s": 55.3633,
            "pass2_prefill_tok_s": 125.32,
            "pass2_ttft_s": 8.2581,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 30.5362
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 124.045,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 8343.3
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ministral3-8b-int8__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-14",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (v0171gpu, \"$B\" = Ministral-3-8B-Instruct-2512_int8.litertlm): ( cd \"$V\" && uvx --from litert-lm==0.17.1 litert-lm run \"$B\" --backend gpu --prompt \"What is the capital of France? Answer in one word.\" --cache no ) < /dev/null > \"$o\" 2>&1",
            "2026-10-14 14:13:11 JST start int8",
            "2026-10-14 14:13:27 JST end int8",
            "stdout: Paris",
            "EXIT=0"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_with_expect": 1.0,
            "processes": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-14/ministral3-8b-int8__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 46.66,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ministral-3-8B-Instruct-2512_int8.litertlm --backend gpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "pass 1: prefill 784.97 / decode 47.51 tok/s, TTFT 1.3256 s, init 4.8365 s",
            "pass 2: prefill 784.69 / decode 45.81 tok/s, TTFT 1.3269 s, init 4.8565 s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 4.8465,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 47.51,
            "pass1_init_s": 4.8365,
            "pass1_prefill_tok_s": 784.97,
            "pass1_ttft_s": 1.3256,
            "pass2_decode_tok_s": 45.81,
            "pass2_init_s": 4.8565,
            "pass2_prefill_tok_s": 784.69,
            "pass2_ttft_s": 1.3269,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 6.7931
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 784.83,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 1326.2
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ministral3-8b-int8__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "mistral",
    "id": "ministral3-8b-int8",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Ministral-3-8B-Instruct-2512",
    "task": "text-generation"
  },
  "pitfalls": [
    "Text only: the vision encoder of the release is not in these files; the weights come from the BF16 repo (the plain repo ships FP8 weights) (card L19, L97).",
    "Runtime: tested on litert-lm 0.18.0 (Mac); on litert-lm 0.17.1 the file answered a plain `run` with \"Paris\" (one GPU process) (card L19).",
    "GPU-composite graph: each layer's two DYNAMIC_UPDATE_SLICE KV writes are one odml.cache_update and the attention products odml.runtime_bmm; on the Mac GPU the rewrite took the 1024 / 256 workload from 7.78 to 6.76 s in the same windows; before the rewrite the same weights scored 95 on GSM8K, two questions moved from wrong to right with it and none the other way (card L55, L100).",
    "Lightweight export: the default litert-torch export path for an 8B model of this kind reached 98.8 GB on a 128 GB Mac before a RAM guard stopped it; with a 1e8-byte lazy-constant threshold every FULLY_CONNECTED weight is an int8 constant and no op has all-constant inputs. Export peaks 46-52 GB (card L99).",
    "Chat template: the Ministral-3-3B-Instruct-2512 conversion's template with the model name changed ([SYSTEM_PROMPT]…[/SYSTEM_PROMPT][INST]…[/INST], string and content-parts messages), Mistral AI's default system prompt verbatim ({today} / {yesterday} stay literal); the runtime adds the start token <s>; the only stop token </s> (id 2); no thinking declared. Known differences from the vendor render: no tool list, two text parts in one message concatenated, two user messages in a row become two turns (card L69).",
    "Conversation preface with tools and no messages: LiteRT-LM renders the chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with 'Failed to apply template: undefined value' when the preface is prefilled at creation (prefillPrefaceOnInit, default false; creating the conversation fails) and in the preface render (renderPrefaceIntoString and its C++ and C API forms); the same with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2); read in the source of LiteRT-LM 0.17.0, 0.18.1 and main, measured with the 0.18.1 PyPI package through its C API on these files' template text; the default path, where the first message is rendered with the preface, is not affected (the 0.18.1 Python API and CLI expose neither the flag nor the preface render); a system message or any initial message in the conversation config avoids it (card L104).",
    "Tool calling is not wired: a tool does not change the prompt (533 tokens with or without one) and a `tool` message raises INVALID_ARGUMENT: No token IDs found; an app has to put tool definitions and results into the user text (card L71).",
    "Engine reuse: one Engine can serve one conversation after another (the second conversation gave the same replies as a fresh Engine in 3 of 3 runs, Mac GPU; KV cache only, no recurrent state) (card L71).",
    "GPU precision: the file runs the GPU with the runtime's default fp16 activations; fp32 activations declared gave all eight check answers equal to bf16's (seven of eight with fp16) and ran 15 % slower, so none is declared (card L55).",
    "Galaxy S26: no row — in its one try (2026-10-14) the phone dropped off USB during the GPU engine warm-up, 4 min 15 s after the process started (prefill_1024 to prefill_4 delegated 1586/1586 to LITERT_CL), and came back restarted; the cause is not established and no 8B int8 engine ran on this phone again. No CPU row on either phone (batch-1 CPU peak factors put the file above MemTotal); Pixel 8a not tried (8.64 GB, more than the phone's RAM) (card L33, L93).",
    "iOS not measured: the main weights section is 8,088,047,040 bytes (7.53 GiB), over 2 GiB (card L37).",
    "Accuracy: GSM8K n=100 (greedy, at most 1024 output tokens, fresh process per question, Mac GPU): bf16 95 / int8 97 — four questions differ (one right only in bf16, three only in the int8 file) (card L47-L55).",
    "Mac CPU: without --cache, `run` writes a compiled-weights cache next to the model (7.97 GB for this file); on the Mac CPU this file is faster than the int4 file, its prefill being twice int4's (card L37, L67)."
  ],
  "schema_version": "1.2"
}
