{ "artifacts": [ { "file": "Spark-X2.5-4B_int4.litertlm", "sha256": "0b9ab47c1e65214995dc10f9833728941a73c05cfb219e3ddb05b0fe623b16a1", "size_mb": 2367.207 } ], "benchmarks": [], "conversion": { "command": "patch_modeling.py -> convert_spark.py templates/spark25_think.jinja (NO_START_TOKEN=1, USE_JINJA=1, CACHE 4096, PREFILL 1024,256,64,16,4,1 = 6 signatures — the granite-4.2 iPhone lesson; int4: EXTERNALIZE_EMBEDDER=1 BOCTAV4_128) -> scripts/set_activation_type.py --type fp32 on BOTH ship files (metadata-only repack, weights byte-identical) (FINDINGS bundles table + fp32 repack)", "quantization": "export-time int4 blockwise-128 + OCTAV on linears, int8 embedding externalized in its own section (EXTERNALIZE_EMBEDDER=1 BOCTAV4_128); prefer_activation_type fp32 declared in-bundle", "tool": "litert-torch export_hf path over the vendor modeling code patched for the registered attention interface (released wheels only; repro = hf-to-litertlm spark_work/patch_modeling.py + spark_work/convert_spark.py wrapping scripts/export_simple_template.py on the jinja path)", "tool_version": "0.9.3 (transformers 5.14.1; ai-edge-quantizer named but unversioned in the sources — HF card Conversion notes; FINDINGS env ~/venvs/ltconv040dev; gates and GSM8K on litert-lm 0.17.0)" }, "cross_runtime": [], "delegation": null, "device": { "records": [ { "device": "galaxy-s26", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 5.47, "delegated_ops": 2149, "env": { "device": "Galaxy S26 (SM-S942Q)", "machine_label": "galaxy-s26-cl-pinned", "os_build": "Android 16", "runtime": "litert-lm", "runtime_version": "0.16.0", "soc": "Qualcomm SM8850", "vendor_sdk": null }, "error": null, "evidence": [ "VERBOSE: Replacing 2149 out of 2233 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 73 partitions for subgraph 0 (prefill_1024).", "VERBOSE: Replacing 2149 out of 2233 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 73 partitions for subgraph 1 (prefill_256).", "VERBOSE: Replacing 2149 out of 2233 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 73 partitions for subgraph 2 (prefill_64).", "VERBOSE: Replacing 2149 out of 2233 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 73 partitions for subgraph 3 (prefill_16).", "results block: prefill=34.77 decode=5.47 tokens/s" ], "failure_class": null, "full_delegation": false, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "decode_tokens": 3883.0, "init_s": 4.10983, "prefill_tokens": 213.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 34.77, "provenance": "measured", "runs": true, "total_ops": 2233, "ttft_ms": 6310.0 }, "source": "data/device_runs/0.16.0/2026-09-07/spark-x2.5-4b-int4__galaxy-s26.json" }, { "device": "galaxy-s26", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 4.89, "delegated_ops": 2233, "env": { "device": "Galaxy S26 (SM-S942Q)", "machine_label": "galaxy-s26-cl-pinned", "os_build": "Android 16", "runtime": "litert-lm", "runtime_version": "0.16.0", "soc": "Qualcomm SM8850", "vendor_sdk": null }, "error": null, "evidence": [ "VERBOSE: Replacing 2233 out of 2233 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).", "VERBOSE: Replacing 2233 out of 2233 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_256).", "VERBOSE: Replacing 2233 out of 2233 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_64).", "VERBOSE: Replacing 2233 out of 2233 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_16).", "results block: prefill=56.36 decode=4.89 tokens/s" ], "failure_class": null, "full_delegation": true, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "decode_tokens": 3883.0, "init_s": 14.00514, "prefill_tokens": 213.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 56.36, "provenance": "measured", "runs": true, "total_ops": 2233, "ttft_ms": 3980.0 }, "source": "data/device_runs/0.16.0/2026-09-07/spark-x2.5-4b-int4__galaxy-s26.json" }, { "device": "iphone-17-pro", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": null, "delegated_ops": null, "env": { "device": "iPhone 17 Pro", "machine_label": "iphone-17-pro", "os_build": "iOS 27.0", "runtime": "litert-lm", "runtime_version": "0.15.0", "soc": "A19-Pro", "vendor_sdk": null }, "error": null, "evidence": [ "G41_START backend=cpu maxTokens=3072", "G41_INIT 9.02 s", "G41_Q BAD rhyme=blue", "G41_SCORE 7/8", "G41_DONE", "The app terminated with the exit code 0.", "INFO: Created TensorFlow Lite XNNPACK delegate for CPU.", "G41_MEM afterInit avail=4683MB (lowest tick 4463MB)", "litertlm-convert/spark_work/iphone/iphone_4b_int4b128_fp32_cpu.log (absl epoch 1788742905 = 2026-09-07 01:01 UTC / 10:01 JST)", "G41_MODEL model.litertlm bytes=2482196400", "G41_CTX bundle-default", "G41_MEM start avail=6135MB phys=11722MB", "model bytes 2,482,196,400 match the published Spark-X2.5-4B_int4.litertlm (HF API size 2,482,196,400, LFS sha256 0b9ab47c1e65214995dc10f9833728941a73c05cfb219e3ddb05b0fe623b16a1)", "G41DeviceTest generation gate (eight fixed questions in one composite prompt, scored by regex), not litert_lm_main --benchmark: no throughput, no TTFT, no peak-memory figure - those fields stay null rather than 0. G41_MEM lines are free-memory ticks of the handset, not a process peak.", "runtime_version 0.15.0: this gate ran under G41DeviceTest, which links ~/code/swift-litert-lm (Package.swift: liteRTLMVersion = \"v0.15.0\"; the lane's FINDINGS records the harness built against checkout 0.2.0-1-g8e5f1da with the Increased Memory Limit entitlement on the App ID). The built .app could not be located on disk on 2026-09-08 for a dwarfdump re-walk of the embedded CLiteRTLM UUID, so the version rests on the package pin plus the lane's own attestation, not on the binary (#164(a) chain walked to the pin only).", "os_build iOS 27.0 (24A5418b) per spark_work/FINDINGS.md 'Conditions of this measurement'; the G41 log prints no OS version.", "fp32-activation repack (the shipped bytes)." ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "init_s": 9.02, "max_num_tokens": 3072.0, "sanity_8q_correct": 7.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": null, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": null }, "source": "data/device_runs/0.15.0/2026-09-07/spark-x2.5-4b-int4__iphone-17-pro.json" }, { "device": "iphone-17-pro", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": null, "delegated_ops": null, "env": { "device": "iPhone 17 Pro", "machine_label": "iphone-17-pro", "os_build": "iOS 27.0", "runtime": "litert-lm", "runtime_version": "0.15.0", "soc": "A19-Pro", "vendor_sdk": null }, "error": null, "evidence": [ "G41_START backend=gpu maxTokens=3072", "G41_INIT 5.25 s", "G41_Q BAD rhyme=blue", "G41_SCORE 7/8", "G41_DONE", "The app terminated with the exit code 0.", "INFO: Created TensorFlow Lite XNNPACK delegate for CPU.", "G41_MEM afterInit avail=2780MB (lowest tick 2035MB)", "litertlm-convert/spark_work/iphone/iphone_4b_int4b128_fp32_gpu.log (absl epoch 1788744152 = 2026-09-07 01:22 UTC / 10:22 JST)", "G41_MODEL model.litertlm bytes=2482196400", "G41_CTX bundle-default", "G41_MEM start avail=6134MB phys=11722MB", "model bytes 2,482,196,400 match the published Spark-X2.5-4B_int4.litertlm (HF API size 2,482,196,400, LFS sha256 0b9ab47c1e65214995dc10f9833728941a73c05cfb219e3ddb05b0fe623b16a1)", "G41DeviceTest generation gate (eight fixed questions in one composite prompt, scored by regex), not litert_lm_main --benchmark: no throughput, no TTFT, no peak-memory figure - those fields stay null rather than 0. G41_MEM lines are free-memory ticks of the handset, not a process peak.", "runtime_version 0.15.0: this gate ran under G41DeviceTest, which links ~/code/swift-litert-lm (Package.swift: liteRTLMVersion = \"v0.15.0\"; the lane's FINDINGS records the harness built against checkout 0.2.0-1-g8e5f1da with the Increased Memory Limit entitlement on the App ID). The built .app could not be located on disk on 2026-09-08 for a dwarfdump re-walk of the embedded CLiteRTLM UUID, so the version rests on the package pin plus the lane's own attestation, not on the binary (#164(a) chain walked to the pin only).", "os_build iOS 27.0 (24A5418b) per spark_work/FINDINGS.md 'Conditions of this measurement'; the G41 log prints no OS version.", "attempt 2. Attempt 1 (iphone_4b_int4b128_fp32_gpu_attempt1_timeout.log, launched 4 min after a fresh 2.5 GB copy) initialised in 12.96 s and produced nothing for 25 min until the host timeout sent signal 15; an 8-token probe right after ran in seconds and this re-run after a re-copy scored 7/8 - cause not established, and a launched app killed on a host wall clock is not a model result (#164(d)/(e)); the passing leg is the record.", "fp32-activation repack (the shipped bytes)." ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "init_s": 5.25, "max_num_tokens": 3072.0, "sanity_8q_correct": 7.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": null, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": null }, "source": "data/device_runs/0.15.0/2026-09-07/spark-x2.5-4b-int4__iphone-17-pro.json" }, { "device": "mac-studio-m4-max", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 18.37, "delegated_ops": null, "env": { "device": "Mac Studio (M4 Max)", "machine_label": "mac-studio-m4-max", "os_build": null, "runtime": "litert-lm", "runtime_version": "0.17.0", "soc": null, "vendor_sdk": null }, "error": null, "evidence": [ "results block: prefill=117.84 decode=18.37 tokens/s" ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 117.84, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": 2237.2 }, "source": "data/device_runs/0.17.0/2026-09-07/spark-x2.5-4b-int4__mac-studio-m4-max.json" }, { "device": "mac-studio-m4-max", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 53.63, "delegated_ops": null, "env": { "device": "Mac Studio (M4 Max)", "machine_label": "m4max-quiet-300s-gpu-rest", "os_build": null, "runtime": "litert-lm", "runtime_version": "0.17.0", "soc": null, "vendor_sdk": null }, "error": null, "evidence": [ "results block: prefill=825.98 decode=53.63 tokens/s" ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 825.98, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": 328.6 }, "source": "data/device_runs/0.17.0/2026-09-07/spark-x2.5-4b-int4__mac-studio-m4-max.json" } ] }, "model": { "family": "spark-x2.5", "id": "spark-x2.5-4b-int4", "license": "apache-2.0", "source_url": "https://huggingface.co/litert-community/Spark-X2.5-4B", "task": "text-generation" }, "pitfalls": [ "int4 (block-128) is the recommended file: same accuracy as int8 (GSM8K 94 = bf16), 41% smaller (2.48 GB), and the faster GPU file on the Mac (53.6 vs 49.4 tok/s decode); unlike the 1.7B, block-128 shows no verbosity collapse here (mean thought 3,212 chars, 7 unfinished) — the block-32 hedge (2.67 GB, gates passed) was not needed and stays a lane record (HF card; FINDINGS 4B int4 b128).", "The iPhone int4 legs score 7/8 on both backends: the one miss is the rhyme line at the end of the composite 8-question prompt (it answers 'sweet'); asked alone the same question is answered 'blue' on every backend; the int4 Metal leg bottomed at ~2.0 GB free (HF card).", "GPU activations are declared fp32 in the bundle (prefer_activation_type = fp32, a metadata-only repack; weights byte-identical): the runtime runs the text decoder with fp16 activations on the GPU by default, and on this 36-layer decoder that silently costs reasoning accuracy — the same int8 weights scored GSM8K 84 under fp16 GPU activations and 94 under fp32, established by re-running the 11 GPU losses on the CPU fp32 path (10 came back right; the 5 controls stayed wrong). The 8-question gate did not see it (8/8 either way); only a long-generation accuracy row does. Expect a larger GPU memory footprint than an fp16-activation file of the same size (HF card Conversion notes; FINDINGS 4B GSM8K).", "Both files are at bf16 parity on GSM8K (94 = 94 = 94; n=100, greedy, 0-shot CoT, 3584 output tokens, own harness) once fp32 activations are declared; the 1.7B stays on the fp16 default because it measured exact parity there (HF card; FINDINGS).", "On the Galaxy S26 this 4B is a ~5 tok/s model on every measured path: the fp32-activation OpenCL path does not beat the CPU (int4: GPU 4.9 vs CPU 5.5-5.6 tok/s decode) but halves peak memory (2.0 vs 3.5 GB) — pick by memory, not speed; the int8 GPU --benchmark cell is unmeasured because a ~3,900-token thinking response at ~5 tok/s exceeded the 20-minute leg cap twice, although the file loads with full OpenCL delegation (2236/2236) and answers the gate on the GPU (HF card Performance; FINDINGS S26; DECISIONS #164(e)).", "iPhone 17 Pro (G41DeviceTest, --max-tokens 3072, Increased Memory Limit entitlement, iOS 27.0): the int8 file — ONE 4.24 GB weights section, no externalized embedder — loads and scores 8/8 on Metal and CPU (available-memory floor ~2.6 GB on the Metal leg, ~4.5 GB on CPU), so the ~2 GiB single-section mmap ceiling recorded in the Llama-3.2 era did not bite on this runtime lineage with this entitlement; the 4B int4 b128's 1.994 GiB main section was sized to it unnecessarily (harmless). Keep externalizing for int4 (the tied-vocab duplication reason stands), but do not cite the 2 GiB ceiling as a hard rule until re-measured on the runtime it came from (FINDINGS 'The iOS ~2 GiB single-section mmap ceiling did NOT bite').", "GPU init times on the iPhone are with a warm shader cache (a cold first launch of the int8 file initialised in 18.8 s); the int4 file's first Metal leg after a fresh 2.5 GB copy initialised in 13 s and then produced nothing for 25 min until the host timeout — one-off, cause not established, the re-run scored normally; the card carries the completed legs (HF card; FINDINGS).", "Reasoning model: give it a generous output budget (>= 2048 tokens, 3584 for math) — truncated mid-thought it produces no final answer at all; the bundle pre-fills the vendor think opener (<|Bot|>) and declares the thought channel (…) so the runtime separates reasoning from the answer and honours a thinking budget; without the channel the runtime streams raw reasoning into the answer and ignores any budget (HF card Usage + Conversion notes).", "The vendor modeling code is patched for export, not re-implemented: it computes attention through its own eager function and ignores config._attn_implementation, so the export copy dispatches through the registered attention interface (litert-torch's transposed KV cache), threads the per-call kwargs, declares the attention-backend flags and applies the per-head sigmoid output gate in the interface's layout; eager output is bit-identical to the vendor file (max |dlogit| 0.0 on 24 random tokens); two further edits make the vendor file load under transformers 5 at all (HF card Conversion notes; FINDINGS parity).", "No start token in the metadata: the tokenizer declares <|start_of_sentence|> as BOS but never prepends it (add_bos_token false); the template carries its own, so the exporter's unconditional start_token write was suppressed (measured harmless in bf16 on the 8Q gate, but it is not the vendor prompt) (HF card Conversion notes).", "Prompt format carried as a Jinja template verbatim: a default system block ('you are a helpful assistant.', a user system prompt appended after it), every message wrapped in <|start_of_sentence|> … <|end_of_sentence|>, stop <|end_of_sentence|>; tool-call formatting is not carried; KV budget 4096 (the original's 1M context does not apply on-device); the vendor's recommended sampling is temperature 1.0 / top-p 0.95 while every number on the card is greedy (HF card Usage).", "Tokenizer parity: the bundle's HF tokenizer.json section encodes 234/234 probe rows to the same ids as the tokenizers reading of the upstream file; multi-turn (3 turns, fact recall) passes with and without the runtime's channel filtering (HF card Correctness).", "iPhone 17 Pro rows come from the G41DeviceTest harness (composite 8-question prompt, --max-tokens 3072, byte count verified on-device): a generation gate, not a benchmark; the harness links swift-litert-lm's v0.15.0 pin and ran with the Increased Memory Limit entitlement — see the device rows' evidence (FINDINGS iPhone section; DECISIONS #164)." ], "schema_version": "1.2" }