{
  "artifacts": [
    {
      "file": "Ministral-3-8B-Instruct-2512_int4.litertlm",
      "sha256": "babb24c5d76bedcbc0b4418e2e6f07f8b92673df481fb4514848542daa50b160",
      "size_mb": 4805.684
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "convert_ministral3_8b.sh (hf-to-litertlm): scripts/extract_ministral3_text.py (Mistral3ForConditionalGeneration -> Ministral3ForCausalLM, the vision tower and the projector dropped) -> export (6 prefill signatures 1024..1 + decode, KV cache 4096, embedder externalized, the bundle template embedded) -> int4 block-scale scan -> metadata edit (stop id 2 only, max_num_tokens 4096) -> tools/gpu_graph/gpu_graph_retrofit.py --decomp exporter --mask add_bcast",
    "quantization": "int4 blockwise-32 + OCTAV on linears, int8 embedding; the embedding table as a separate section (card L26); 239 blockwise tensors, 248,512,512 scales, none zero or subnormal (L100)",
    "tool": "litert-torch (text tower extracted from the multimodal checkpoint; lightweight export, 1e8-byte lazy-constant threshold) + hf-to-litertlm tools/gpu_graph/ GPU-composite rewrite; repro = hf-to-litertlm ministral_work/",
    "tool_version": "litert-torch 0.9.4 (litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1, torch 2.13.0); packed by the litert-lm 0.18.0 CLI"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 5.6425,
          "delegated_ops": 1586,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-aab2",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, sha256 adac974b…)",
            "run 1, AB (rewritten graph first): [15:32:38] bundle on device: babb24c5d76bedcbc0b4418e2e6f07f8b92673df481fb4514848542daa50b160 (5039124912 B, push 136.7 s) = Mac",
            "run 1 prompt process: \"The capital of France is **Paris**.\"",
            "run 1 bench args: --backend=gpu --model_path=/data/local/tmp/aab2_m38b/Ministral-3-8B-Instruct-2512_int4_gpuc.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aab2_m38b/metrics.pb",
            "run 2, BA (rewritten graph second): [18:55:18] bundle on device: babb24c5d76bedcbc0b4418e2e6f07f8b92673df481fb4514848542daa50b160 (5039124912 B, push 134.8 s) = Mac",
            "run 2 prompt process: \"The capital of France is **Paris**.\"",
            "run 2 bench args: --backend=gpu --model_path=/data/local/tmp/aab2_m38b/Ministral-3-8B-Instruct-2512_int4_gpuc.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aab2_m38b/metrics.pb",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_16).",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 4 (prefill_4).",
            "VERBOSE: Replacing 1417 out of 1417 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 5 (prefill_1).",
            "VERBOSE: Replacing 1448 out of 1448 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 6 (decode).",
            "VERBOSE: Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder)."
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_boot_epoch_s": 1791938976.36,
            "run1_capped_samples": 76.0,
            "run1_cold_init_ms": 25399.89,
            "run1_decode_tok_s": 5.72,
            "run1_decode_tok_s_it1": 9.56,
            "run1_decode_tok_s_it2": 6.41,
            "run1_decode_tok_s_it3": 5.03,
            "run1_init_ms": 23748.66,
            "run1_kgsl_ceiling_min_mhz": 282.0,
            "run1_kgsl_temp_max_c": 86.4,
            "run1_memavail_min_kib": 1102572.0,
            "run1_peak_mem_mb": 2026.14,
            "run1_prefill_tok_s": 99.98,
            "run1_prefill_tok_s_it1": 190.86,
            "run1_prefill_tok_s_it2": 118.82,
            "run1_prefill_tok_s_it3": 81.14,
            "run1_sampler_samples": 79.0,
            "run1_skin_c_at_bench_start": 45.0,
            "run1_thermal_status_at_bench_start": 3.0,
            "run1_ttft_s": 10.795,
            "run2_boot_epoch_s": 1791938976.79,
            "run2_capped_samples": 78.0,
            "run2_cold_init_ms": 24861.26,
            "run2_decode_tok_s": 5.565,
            "run2_decode_tok_s_it1": 9.48,
            "run2_decode_tok_s_it2": 6.3,
            "run2_decode_tok_s_it3": 4.83,
            "run2_init_ms": 23844.25,
            "run2_kgsl_ceiling_min_mhz": 282.0,
            "run2_kgsl_temp_max_c": 91.9,
            "run2_memavail_min_kib": 864440.0,
            "run2_peak_mem_mb": 1937.32,
            "run2_prefill_tok_s": 98.36,
            "run2_prefill_tok_s_it1": 188.08,
            "run2_prefill_tok_s_it2": 121.22,
            "run2_prefill_tok_s_it3": 75.5,
            "run2_sampler_samples": 81.0,
            "run2_skin_c_at_bench_start": 45.0,
            "run2_thermal_status_at_bench_start": 3.0,
            "run2_ttft_s": 11.19,
            "runs": 2.0,
            "t_1024_256_s": 55.696
          },
          "output_match": null,
          "peak_mem_mb": 1981.73,
          "prefill_tokens_per_s": 99.17,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1586,
          "ttft_ms": 10992.5
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-15/ministral3-8b-int4__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 12.93,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ministral-3-8B-Instruct-2512_int4.litertlm --backend cpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "pass 1: prefill 54.88 / decode 12.61 tok/s, TTFT 18.7485 s, init 17.1567 s",
            "pass 2: prefill 55.9 / decode 13.25 tok/s, TTFT 18.4055 s, init 15.9301 s",
            "process footprint (swap guard, 15 s samples, binary GB); no guard kill"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "guard_footprint_max_gb": 30.0,
            "init_s": 16.5434,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 12.61,
            "pass1_init_s": 17.1567,
            "pass1_prefill_tok_s": 54.88,
            "pass1_ttft_s": 18.7485,
            "pass2_decode_tok_s": 13.25,
            "pass2_init_s": 15.9301,
            "pass2_prefill_tok_s": 55.9,
            "pass2_ttft_s": 18.4055,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 38.2997
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 55.39,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 18577.0
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ministral3-8b-int4__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-14",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (v0171gpu, \"$B\" = Ministral-3-8B-Instruct-2512_int4.litertlm): ( cd \"$V\" && uvx --from litert-lm==0.17.1 litert-lm run \"$B\" --backend gpu --prompt \"What is the capital of France? Answer in one word.\" --cache no ) < /dev/null > \"$o\" 2>&1",
            "2026-10-14 14:13:27 JST start int4",
            "2026-10-14 14:13:38 JST end int4",
            "stdout: Paris",
            "EXIT=0"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_with_expect": 1.0,
            "processes": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-14/ministral3-8b-int4__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-15",
          "decode_tokens_per_s": 61.74,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "command (both passes): litert-lm benchmark Ministral-3-8B-Instruct-2512_int4.litertlm --backend gpu -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no",
            "pass 1: prefill 785.91 / decode 63.6 tok/s, TTFT 1.3187 s, init 6.7196 s",
            "pass 2: prefill 789.12 / decode 59.88 tok/s, TTFT 1.3145 s, init 6.7234 s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 6.7215,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 63.6,
            "pass1_init_s": 6.7196,
            "pass1_prefill_tok_s": 785.91,
            "pass1_ttft_s": 1.3187,
            "pass2_decode_tok_s": 59.88,
            "pass2_init_s": 6.7234,
            "pass2_prefill_tok_s": 789.12,
            "pass2_ttft_s": 1.3145,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 5.4505
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 787.515,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 1316.6
        },
        "source": "data/device_runs/0.18.0/2026-10-15/ministral3-8b-int4__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-14",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-aab2",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": "GUARD_KILL avail=316676 pid=23820",
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, sha256 adac974b…)",
            "[07:26:50] bundle on device: babb24c5d76bedcbc0b4418e2e6f07f8b92673df481fb4514848542daa50b160 (5039124912 B, push 112.1 s) = Mac",
            "one attempt by rule (--backend=gpu, text prompt 'What is the capital of France? Answer in one sentence.'); a 2-second sampler stops the process when MemAvailable falls under 400,000 kB",
            "VERBOSE: Replacing 1586 out of 1586 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "W0000 00:00:1791930415.470695   23820 gpu_model_builder.cc:4455] Weights preparation on Gpu is disabled for PowerVR, Broadcom, Mali GPUs and non-OpenCL/WebGPU/Metal backends.",
            "[07:27:51] prompt_1 done: rc(adb)=0 EXIT=143 wall 55.7 s",
            "GUARD_KILL avail=316676 pid=23820"
          ],
          "failure_class": "memory_cap_exceeded",
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": false,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "memavail_at_start_kib": 4016480.0,
            "memavail_min_kib": 316676.0,
            "sampler_samples": 22.0,
            "seconds_to_stop": 55.7,
            "signatures_compiled": 1.0,
            "uptime_after_s": 106053.91,
            "uptime_before_s": 105994.05,
            "vmhwm_kib": 505836.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": false,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-14/ministral3-8b-int4__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "mistral",
    "id": "ministral3-8b-int4",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Ministral-3-8B-Instruct-2512",
    "task": "text-generation"
  },
  "pitfalls": [
    "Text only: the vision encoder of the release is not in these files; the weights come from the BF16 repo (the plain repo ships FP8 weights) (card L19, L97).",
    "Runtime: tested on litert-lm 0.18.0 (Mac); on litert-lm 0.17.1 the file answered a plain `run` with \"Paris\" (one GPU process) (card L19).",
    "GPU-composite graph: each layer's two DYNAMIC_UPDATE_SLICE KV writes are one odml.cache_update and the attention products odml.runtime_bmm; on the Mac GPU the rewrite took the 1024 / 256 workload from 6.40 to 5.35 s in the same windows, on the Galaxy S26 its decode gain is 1.3× (the exported graph in the same two sessions: prefill 94.77 and 100.84 tok/s, decode 4.26 and 4.59 tok/s); before the rewrite the same weights scored 94 on GSM8K, two questions moved from wrong to right with it and none the other way (card L55, L91, L100).",
    "Lightweight export: the default litert-torch export path for an 8B model of this kind reached 98.8 GB on a 128 GB Mac before a RAM guard stopped it; with a 1e8-byte lazy-constant threshold every FULLY_CONNECTED weight is an int4 constant and no op has all-constant inputs. Export peaks 46-52 GB (card L99).",
    "Chat template: the Ministral-3-3B-Instruct-2512 conversion's template with the model name changed ([SYSTEM_PROMPT]…[/SYSTEM_PROMPT][INST]…[/INST], string and content-parts messages), Mistral AI's default system prompt verbatim ({today} / {yesterday} stay literal); the runtime adds the start token <s>; the only stop token </s> (id 2); no thinking declared. Known differences from the vendor render: no tool list, two text parts in one message concatenated, two user messages in a row become two turns (card L69).",
    "Conversation preface with tools and no messages: LiteRT-LM renders the chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with 'Failed to apply template: undefined value' when the preface is prefilled at creation (prefillPrefaceOnInit, default false; creating the conversation fails) and in the preface render (renderPrefaceIntoString and its C++ and C API forms); the same with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2); read in the source of LiteRT-LM 0.17.0, 0.18.1 and main, measured with the 0.18.1 PyPI package through its C API on these files' template text; the default path, where the first message is rendered with the preface, is not affected (the 0.18.1 Python API and CLI expose neither the flag nor the preface render); a system message or any initial message in the conversation config avoids it (card L104).",
    "Tool calling is not wired: a tool does not change the prompt (533 tokens with or without one) and a `tool` message raises INVALID_ARGUMENT: No token IDs found; an app has to put tool definitions and results into the user text (card L71).",
    "Engine reuse: one Engine can serve one conversation after another (the second conversation gave the same replies as a fresh Engine in 3 of 3 runs, Mac GPU; KV cache only, no recurrent state) (card L71).",
    "Galaxy S26 GPU: measured warm — each benchmark started at thermal status 3 (skin 45 °C) with the GPU clock ceiling at 1300 MHz, and the ceiling fell to 282 MHz inside the runs; MemAvailable stayed above 864,440 kB. No CPU row on either phone (batch-1 CPU peak factors put the file above the S26's 11.66 GB and the Pixel 8a's 7.94 GB) (card L84-L93).",
    "Pixel 8a GPU: the one try was stopped by the memory rule (a run ends at the first MemAvailable reading under 400,000 kB; here 316,676 kB, 56 s after the start, after prefill_1024 had been delegated 1586/1586 to LITERT_CL; the runtime prepares weights on the CPU side for Mali GPUs) (card L32, L93).",
    "iOS not measured: the main weights section is 4,488,362,336 bytes (4.18 GiB), over 2 GiB (card L37).",
    "Accuracy: GSM8K n=100 (greedy, at most 1024 output tokens, fresh process per question, Mac GPU): bf16 95 / int4 96 — three questions differ (one right only in bf16, two only in the int4 file); the int4 file is 41.7 % smaller than the int8 file (card L28, L47-L55).",
    "Mac CPU: without --cache, `run` writes a compiled-weights cache next to the model (4.49 GB for this file); the int4 file is the faster file on both GPUs measured, the int8 file on the Mac CPU (card L37, L67)."
  ],
  "schema_version": "1.2"
}
