{
  "artifacts": [
    {
      "file": "G9v3-3B_int4.litertlm",
      "sha256": "ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468",
      "size_mb": 1749.455
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "ai9stars/G9v3-3B rev d9553445: gptqmodel (4 bits, group 128, symmetric, 256 chat samples from ultrachat_200k, alpaca and the GSM8K training split at pinned revisions, thinking off) -> dequantized -> export (6 prefill signatures, 1024 down to 1, plus decode, over a 4096-position KV cache; the embedding table as its own section) with the GPTQ scales recovered as blockwise-128 int4, 781 block scales below 2^-14 floored to it -> metadata edit (stops ids 1 and 130073, thought channel, thinking declared, max_num_tokens 4096) -> GPU activation type declared fp16 (card L113, L115, L116)",
    "quantization": "GPTQ-calibrated int4 (group 128) carried as blockwise-128 int4 on linears, int8 lm_head + embedding; GPU activations declared fp16 (card L26)",
    "tool": "litert-torch 0.9.4 + gptqmodel 7.3.5 (calibration); repro = hf-to-litertlm g9v3_work/convert_g9v3_3b.sh (card L113)",
    "tool_version": "litert-torch 0.9.4, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, transformers 5.14.1 (export); gptqmodel 7.3.5 on torch 2.14.1 (MPS) for the calibration; packed by the litert-lm 0.18.0 CLI"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-13",
          "decode_tokens_per_s": 18.465,
          "delegated_ops": 2323,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-aab1",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r4 hold 2026-10-13 11:12:54-11:37:52 JST: [11:30:44] bundle on device: ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468 (1834436960 B, push 46.7 s) = Mac",
            "file pushed: G9v3-3B_int4_b_fp16decl.litertlm (sha256 = the Hub file)",
            "bench args: --backend=gpu --model_path=/data/local/tmp/aa5_g9v3/G9v3-3B_int4_b_fp16decl.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa5_g9v3/metrics.pb",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_16).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 4 (prefill_4).",
            "VERBOSE: Replacing 2064 out of 2064 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 5 (prefill_1).",
            "VERBOSE: Replacing 2092 out of 2092 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 6 (decode).",
            "[11:37:50] parsed: i4h_gpu_rested gpu: gate PASS 1/1 | bench prefill 402.855 decode 18.465 init 5023.11 cold 7485.75 peak 1182.48 capped 18/26 | answers: 'The capital of France is Paris.'",
            "timing run prompt process: \"The capital of France is Paris.\""
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791590918.53,
            "capped_samples": 18.0,
            "cold_init_ms": 7485.75,
            "cpu_capped_samples": 24.0,
            "decode_tok_s_it1": 20.31,
            "decode_tok_s_it2": 18.11,
            "decode_tok_s_it3": 18.82,
            "decode_tokens": 256.0,
            "init_ms": 5023.11,
            "iterations": 3.0,
            "kgsl_ceiling_min_mhz": 726.0,
            "kgsl_temp_max_c": 99.3,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 7662692.0,
            "memavail_min_kib": 3971248.0,
            "prefill_tok_s_it1": 527.55,
            "prefill_tok_s_it2": 369.06,
            "prefill_tok_s_it3": 436.65,
            "prefill_tokens": 1024.0,
            "sampler_samples": 26.0,
            "skin_c_at_bench_start": 40.0,
            "t_1024_256_s": 16.4059,
            "thermal_status_at_bench_start": 1.0,
            "ttft_s_it1": 1.99,
            "ttft_s_it2": 2.83,
            "ttft_s_it3": 2.4
          },
          "output_match": null,
          "peak_mem_mb": 1182.48,
          "prefill_tokens_per_s": 402.855,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2323,
          "ttft_ms": 2615.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-13/g9v3-3b-int4__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-13",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "### file sha256 ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468 (ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468, match)",
            "command: (cd \"$W\" && \"${QW[@]}\" \"$BIN/litert-lm\" run \"./$f\" --prompt \"What is the capital of France?\" --cache no < /dev/null >> \"$o\" 2>&1); echo \"EXIT=$?\" >> \"$o\"",
            "stdout: The capital of France is Paris.",
            "EXIT=0",
            "litert-lm 0.17.1"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_with_expect": 1.0,
            "processes": 1.0
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-13/g9v3-3b-int4__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-13",
          "decode_tokens_per_s": 61.485,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Prefill 1516.27 / decode 61.71 tok/s, TTFT 0.6916 s, init 4.3384 s",
            "Prefill 1497.48 / decode 61.26 tok/s, TTFT 0.7004 s, init 4.3647 s",
            "The median of the two passes = prefill 1506.875 / decode 61.485 tok/s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 4.3515,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "pass1_decode_tok_s": 61.71,
            "pass1_init_s": 4.3384,
            "pass1_prefill_tok_s": 1516.27,
            "pass1_ttft_s": 0.6916,
            "pass2_decode_tok_s": 61.26,
            "pass2_init_s": 4.3647,
            "pass2_prefill_tok_s": 1497.48,
            "pass2_ttft_s": 0.7004,
            "passes": 2.0,
            "prefill_tokens": 1024.0,
            "t_1024_256_s": 4.8433
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": 1506.875,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 696.0
        },
        "source": "data/device_runs/0.18.0/2026-10-13/g9v3-3b-int4__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-18",
          "decode_tokens_per_s": 5.785,
          "delegated_ops": 2830,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-aab1",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "g2p hold 4 2026-10-18 10:03:35-10:20:58 JST: [10:12:13] bundle on device: ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468 (1834436960 B, push 0 s) = Mac",
            "file pushed: G9v3-3B_int4_ship.litertlm (sha256 = the Hub file's)",
            "bench args: --backend=cpu --model_path=/data/local/tmp/aa5_g9v3/G9v3-3B_int4_ship.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa5_g9v3/metrics.pb",
            "VERBOSE: Replacing 2109 out of 2323 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 309 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 1877 out of 2092 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 314 partitions for subgraph 6 (decode).",
            "VERBOSE: Replacing 2830 out of 2941 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 105 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 2507 out of 2617 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 106 partitions for subgraph 6 (decode).",
            "[10:20:55] parsed: i4s_cpu_rested_b cpu: gate PASS 1/1 | bench prefill 19.215 decode 5.785 init 2335.04 cold 10175.43 peak 2644.47 capped 88/93 | answers: 'The capital of France is Paris.'",
            "timing run prompt process: \"The capital of France is Paris.\""
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1792057755.75,
            "capped_samples": 88.0,
            "cold_init_ms": 10175.43,
            "cpu_capped_samples": 88.0,
            "decode_tok_s_it1": 6.66,
            "decode_tok_s_it2": 6.58,
            "decode_tok_s_it3": 4.99,
            "decode_tokens": 256.0,
            "init_ms": 2335.04,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 4029236.0,
            "memavail_min_kib": 2803752.0,
            "prefill_tok_s_it1": 32.86,
            "prefill_tok_s_it2": 19.32,
            "prefill_tok_s_it3": 19.11,
            "prefill_tokens": 1024.0,
            "sampler_samples": 93.0,
            "t_1024_256_s": 97.5441,
            "thermal_status_at_bench_start": 0.0,
            "ttft_s_it1": 31.31,
            "ttft_s_it2": 53.16,
            "ttft_s_it3": 53.79
          },
          "output_match": null,
          "peak_mem_mb": 2644.47,
          "prefill_tokens_per_s": 19.215,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2941,
          "ttft_ms": 53475.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-18/g9v3-3b-int4__pixel-8a.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-13",
          "decode_tokens_per_s": 8.955,
          "delegated_ops": 2323,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-aab1",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "prebuilt-adac974b",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: Google's prebuilt litert_lm_advanced_main + 6 .so (gs://litert/binaries/latest/android_arm64/litert_lm/, object dated 2026-09-18, 40,382,152 B, sha256 adac974b…)",
            "r6 hold 2026-10-13 11:37:33-12:03:55 JST: [11:38:28] bundle on device: ea2c773ec678d44ffca43c0fca21a46622fd812fb9b53706f82d966d02e3c468 (1834436960 B, push 42.6 s) = Mac",
            "file pushed: G9v3-3B_int4_b_fp16decl.litertlm (sha256 = the Hub file)",
            "bench args: --backend=gpu --model_path=/data/local/tmp/aa5_g9v3/G9v3-3B_int4_b_fp16decl.litertlm --benchmark=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --max_num_tokens=2048 --num_iterations=3 --report_peak_memory_footprint=true --metric_proto_file_path=/data/local/tmp/aa5_g9v3/metrics.pb",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_16).",
            "VERBOSE: Replacing 2323 out of 2323 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 4 (prefill_4).",
            "VERBOSE: Replacing 2064 out of 2064 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 5 (prefill_1).",
            "VERBOSE: Replacing 2092 out of 2092 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 6 (decode).",
            "[11:49:13] parsed: i4h_gpu_rested gpu: gate PASS 1/1 | bench prefill 125.11 decode 8.955 init 6665.43 cold 20611.32 peak 1638.86 capped 0/44 | answers: 'The capital of France is Paris.'",
            "timing run prompt process: \"The capital of France is Paris.\""
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "boot_epoch_s": 1791824417.95,
            "capped_samples": 0.0,
            "cold_init_ms": 20611.32,
            "cpu_capped_samples": 41.0,
            "decode_tok_s_it1": 9.01,
            "decode_tok_s_it2": 9.06,
            "decode_tok_s_it3": 8.85,
            "decode_tokens": 256.0,
            "init_ms": 6665.43,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "memavail_at_bench_start_kib": 5133900.0,
            "memavail_min_kib": 2179860.0,
            "prefill_tok_s_it1": 126.34,
            "prefill_tok_s_it2": 126.05,
            "prefill_tok_s_it3": 124.17,
            "prefill_tokens": 1024.0,
            "sampler_samples": 44.0,
            "t_1024_256_s": 36.7722,
            "thermal_status_at_bench_start": 1.0,
            "ttft_s_it1": 8.22,
            "ttft_s_it2": 8.23,
            "ttft_s_it3": 8.36
          },
          "output_match": null,
          "peak_mem_mb": 1638.86,
          "prefill_tokens_per_s": 125.11,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2323,
          "ttft_ms": 8295.0
        },
        "source": "data/device_runs/prebuilt-adac974b/2026-10-13/g9v3-3b-int4__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "g9v3",
    "id": "g9v3-3b-int4",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/G9v3-3B",
    "task": "text-generation"
  },
  "pitfalls": [
    "Calibrated int4: a data-free blockwise int4 export of this checkpoint scored 58 on GSM8K and its zero and subnormal block scales did not load on the CPU, so this file is GPTQ-calibrated; the calibration set is rebuilt by the script, not redistributed (alpaca's licence) (card L59, L115).",
    "Graph form: this file keeps the plain graph; the GPU-composite rewrite applied to the int8 file on 2026-10-18 was measured on this file and not shipped: it ran the Mac GPU 21 % and the Galaxy S26 GPU 6.6 % faster but the Pixel 8a GPU 10 % slower, and with this file's fp16 activations it scored 79 on GSM8K against 81, 8 questions differing (card L37, L114).",
    "GPU precision: this file declares fp16 GPU activations (GSM8K 81 with fp16, 78 with fp32) and runs fastest that way; the same weights declared fp32 took 5.29 s on the Mac GPU against 4.84 s and 25.09 s on the Galaxy S26 GPU against 16.41 s (card L37, L55-L56, L80, L98).",
    "Thinking on: in a three-turn conversation the third turn of this file returned no answer - its trace ran to the turn's 1536-token output limit without closing (the int8 file passed) (card L43).",
    "Galaxy S26 (our own phone): before each run the phone rested until its clocks were at their top and the thermal status was 0 (up to 300 s); the clocks still lowered during every run (the GPU ceiling in 18 of 26 samples, the CPU clock in 45 of 47 for this file) (card L89, L98).",
    "Pixel 8a CPU: the row is one run of a 2026-10-18 hold that ran both files (this file second; the CPU clock lowered in 88 of 93 samples); with the same rest before each run this phone's CPU results move by up to 9 % between holds (this file read 107.36 s for the workload in a hold nine hours earlier), and the 2026-10-13 card row (130.37 s) is not comparable (card L100, L109).",
    "Thinking: off by default and switched on per request; with thinking on the trace goes to the bundle's thought channel (<think> … </think>); a request that does not set the thinking switch is treated as thinking off, where the vendor template leaves it to the model (card L21, L74).",
    "Template: built on LiteRT-LM's models/qwen3 template, reading message content as parts; renders the same text as AI9Stars's template for 10 of 11 conversation shapes; stops ids 1 and 130073, start token <s> (card L74).",
    "Conversation preface with tools and no messages: LiteRT-LM renders the chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with 'Failed to apply template: undefined value' when the preface is prefilled at creation (prefillPrefaceOnInit, default false; creating the conversation fails) and in the preface render (renderPrefaceIntoString and its C++ and C API forms); the same with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2); read in the source of LiteRT-LM 0.17.0, 0.18.1 and main, measured with the 0.18.1 PyPI package through its C API on these files' template text; the default path, where the first message is rendered with the preface, is not affected (the 0.18.1 Python API and CLI expose neither the flag nor the preface render); a system message or any initial message in the conversation config avoids it (card L120).",
    "Tool calling is not declared: with a get_weather tool the file writes the vendor's XML form as text, for the app to parse (card L76).",
    "Sampler: the bundle sets no sampler, so `run` is greedy unless you pass sampling flags; on the 0.18.0 CLI --temperature and --top-p take effect only together with --top-k (card L72).",
    "Cache: without --cache, `run` writes a compiled-weights cache next to the model (1.54 GB for this file); without --backend, `run` uses the CPU (card L72).",
    "Runtime: litert-lm 0.18.0 on the Mac; on litert-lm 0.17.1 this file answered a plain `run` with \"The capital of France is Paris.\" (one CPU process) (card L19).",
    "Accuracy: GSM8K questions 0-99, thinking off, greedy, at most 1024 output tokens: this file 81 on the Mac GPU as shipped and 78 on the CPU, bf16 82; 8-question check 7/8 on CPU and GPU (the rhyme question); IFEval-lite not run (card L41, L47-L59)."
  ],
  "schema_version": "1.2"
}
