{ "artifacts": [ { "file": "Spark-X2.5-1.7B_int8.litertlm", "sha256": "96cd236da98fd95d1baa4ec391b166f359ded607cc13bc71cc9c0990556fe9bf", "size_mb": 1749.616 } ], "benchmarks": [], "conversion": { "command": "patch_modeling.py (transformers-5 load fixes + attention-interface dispatch, eager path bit-identical) -> convert_spark.py templates/spark25_think.jinja (NO_START_TOKEN=1, USE_JINJA=1, CACHE 4096, prefill ladder 1024..1 = 11 signatures; int4: EXTERNALIZE_EMBEDDER=1 BOCTAV4) (FINDINGS Conversion design + bundles table)", "quantization": "export-time dynamic int8 on linears + embedding (dynamic_wi8_afp32), one TFLite section (1.706 GiB); no post-processing", "tool": "litert-torch export_hf path over the vendor modeling code patched for the registered attention interface (released wheels only; repro = hf-to-litertlm spark_work/patch_modeling.py + spark_work/convert_spark.py wrapping scripts/export_simple_template.py on the jinja path)", "tool_version": "0.9.3 (transformers 5.14.1; ai-edge-quantizer named but unversioned in the sources — HF card Conversion notes; FINDINGS env ~/venvs/ltconv040dev; gates and GSM8K on litert-lm 0.17.0)" }, "cross_runtime": [], "delegation": null, "device": { "records": [ { "device": "galaxy-s26", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 12.28, "delegated_ops": 1666, "env": { "device": "Galaxy S26 (SM-S942Q)", "machine_label": "galaxy-s26-cl-pinned", "os_build": "Android 16", "runtime": "litert-lm", "runtime_version": "0.16.0", "soc": "Qualcomm SM8850", "vendor_sdk": null }, "error": null, "evidence": [ "VERBOSE: Replacing 1666 out of 1736 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 0 (prefill_1024).", "VERBOSE: Replacing 1666 out of 1736 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 1 (prefill_512).", "VERBOSE: Replacing 1666 out of 1736 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 2 (prefill_256).", "VERBOSE: Replacing 1666 out of 1736 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 57 partitions for subgraph 3 (prefill_128).", "results block: prefill=164.73 decode=12.28 tokens/s" ], "failure_class": null, "full_delegation": false, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "decode_tokens": 3883.0, "init_s": 0.29225, "prefill_tokens": 213.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 164.73, "provenance": "measured", "runs": true, "total_ops": 1736, "ttft_ms": 1370.0 }, "source": "data/device_runs/0.16.0/2026-09-07/spark-x2.5-1.7b-int8__galaxy-s26.json" }, { "device": "galaxy-s26", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 15.82, "delegated_ops": 1736, "env": { "device": "Galaxy S26 (SM-S942Q)", "machine_label": "galaxy-s26-cl-pinned", "os_build": "Android 16", "runtime": "litert-lm", "runtime_version": "0.16.0", "soc": "Qualcomm SM8850", "vendor_sdk": null }, "error": null, "evidence": [ "VERBOSE: Replacing 1736 out of 1736 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 0 (prefill_1024).", "VERBOSE: Replacing 1736 out of 1736 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 1 (prefill_512).", "VERBOSE: Replacing 1736 out of 1736 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 2 (prefill_256).", "VERBOSE: Replacing 1736 out of 1736 node(s) with delegate (LITERT_CL) node, yielding 1 partitions for subgraph 3 (prefill_128).", "results block: prefill=636.84 decode=15.82 tokens/s" ], "failure_class": null, "full_delegation": true, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "decode_tokens": 3883.0, "init_s": 4.52203, "prefill_tokens": 213.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 636.84, "provenance": "measured", "runs": true, "total_ops": 1736, "ttft_ms": 400.0 }, "source": "data/device_runs/0.16.0/2026-09-07/spark-x2.5-1.7b-int8__galaxy-s26.json" }, { "device": "iphone-17-pro", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": null, "delegated_ops": null, "env": { "device": "iPhone 17 Pro", "machine_label": "iphone-17-pro", "os_build": "iOS 27.0", "runtime": "litert-lm", "runtime_version": "0.15.0", "soc": "A19-Pro", "vendor_sdk": null }, "error": null, "evidence": [ "G41_START backend=cpu maxTokens=3072", "G41_INIT 0.25 s", "G41_Q BAD rhyme=blue", "G41_SCORE 7/8", "G41_DONE", "The app terminated with the exit code 0.", "INFO: Created TensorFlow Lite XNNPACK delegate for CPU.", "G41_MEM afterInit avail=5497MB (lowest tick 5366MB)", "litertlm-convert/spark_work/iphone/iphone_1p7b_int8_cpu.log (absl epoch 1788709939 = 2026-09-06 15:52 UTC / 00:52 JST)", "G41_MODEL model.litertlm bytes=1834605728", "G41_CTX bundle-default", "G41_MEM start avail=6134MB phys=11722MB", "model bytes 1,834,605,728 match the published Spark-X2.5-1.7B_int8.litertlm (HF API size 1,834,605,728, LFS sha256 96cd236da98fd95d1baa4ec391b166f359ded607cc13bc71cc9c0990556fe9bf)", "G41DeviceTest generation gate (eight fixed questions in one composite prompt, scored by regex), not litert_lm_main --benchmark: no throughput, no TTFT, no peak-memory figure - those fields stay null rather than 0. G41_MEM lines are free-memory ticks of the handset, not a process peak.", "runtime_version 0.15.0: this gate ran under G41DeviceTest, which links ~/code/swift-litert-lm (Package.swift: liteRTLMVersion = \"v0.15.0\"; the lane's FINDINGS records the harness built against checkout 0.2.0-1-g8e5f1da with the Increased Memory Limit entitlement on the App ID). The built .app could not be located on disk on 2026-09-08 for a dwarfdump re-walk of the embedded CLiteRTLM UUID, so the version rests on the package pin plus the lane's own attestation, not on the binary (#164(a) chain walked to the pin only).", "os_build iOS 27.0 (24A5418b) per spark_work/FINDINGS.md 'Conditions of this measurement'; the G41 log prints no OS version." ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "init_s": 0.25, "max_num_tokens": 3072.0, "sanity_8q_correct": 7.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": null, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": null }, "source": "data/device_runs/0.15.0/2026-09-07/spark-x2.5-1.7b-int8__iphone-17-pro.json" }, { "device": "iphone-17-pro", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": null, "delegated_ops": null, "env": { "device": "iPhone 17 Pro", "machine_label": "iphone-17-pro", "os_build": "iOS 27.0", "runtime": "litert-lm", "runtime_version": "0.15.0", "soc": "A19-Pro", "vendor_sdk": null }, "error": null, "evidence": [ "G41_START backend=gpu maxTokens=3072", "G41_INIT 13.01 s", "G41_Q BAD rhyme=blue", "G41_SCORE 7/8", "G41_DONE", "The app terminated with the exit code 0.", "G41_MEM afterInit avail=5049MB (lowest tick 4971MB)", "litertlm-convert/spark_work/iphone/iphone_1p7b_int8_gpu.log (absl epoch 1788709854 = 2026-09-06 15:50 UTC / 00:50 JST)", "G41_MODEL model.litertlm bytes=1834605728", "G41_CTX bundle-default", "G41_MEM start avail=6135MB phys=11722MB", "model bytes 1,834,605,728 match the published Spark-X2.5-1.7B_int8.litertlm (HF API size 1,834,605,728, LFS sha256 96cd236da98fd95d1baa4ec391b166f359ded607cc13bc71cc9c0990556fe9bf)", "G41DeviceTest generation gate (eight fixed questions in one composite prompt, scored by regex), not litert_lm_main --benchmark: no throughput, no TTFT, no peak-memory figure - those fields stay null rather than 0. G41_MEM lines are free-memory ticks of the handset, not a process peak.", "runtime_version 0.15.0: this gate ran under G41DeviceTest, which links ~/code/swift-litert-lm (Package.swift: liteRTLMVersion = \"v0.15.0\"; the lane's FINDINGS records the harness built against checkout 0.2.0-1-g8e5f1da with the Increased Memory Limit entitlement on the App ID). The built .app could not be located on disk on 2026-09-08 for a dwarfdump re-walk of the embedded CLiteRTLM UUID, so the version rests on the package pin plus the lane's own attestation, not on the binary (#164(a) chain walked to the pin only).", "os_build iOS 27.0 (24A5418b) per spark_work/FINDINGS.md 'Conditions of this measurement'; the G41 log prints no OS version." ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "metrics": { "init_s": 13.01, "max_num_tokens": 3072.0, "sanity_8q_correct": 7.0 }, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": null, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": null }, "source": "data/device_runs/0.15.0/2026-09-07/spark-x2.5-1.7b-int8__iphone-17-pro.json" }, { "device": "mac-studio-m4-max", "run": { "accelerator": "cpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 41.0, "delegated_ops": null, "env": { "device": "Mac Studio (M4 Max)", "machine_label": "mac-studio-m4-max", "os_build": null, "runtime": "litert-lm", "runtime_version": "0.17.0", "soc": null, "vendor_sdk": null }, "error": null, "evidence": [ "results block: prefill=820.35 decode=41.0 tokens/s" ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 820.35, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": 336.5 }, "source": "data/device_runs/0.17.0/2026-09-07/spark-x2.5-1.7b-int8__mac-studio-m4-max.json" }, { "device": "mac-studio-m4-max", "run": { "accelerator": "gpu", "context_length": null, "date": "2026-09-07", "decode_tokens_per_s": 95.78, "delegated_ops": null, "env": { "device": "Mac Studio (M4 Max)", "machine_label": "m4max-quiet-300s-gpu-rest", "os_build": null, "runtime": "litert-lm", "runtime_version": "0.17.0", "soc": null, "vendor_sdk": null }, "error": null, "evidence": [ "results block: prefill=2308.15 decode=95.78 tokens/s" ], "failure_class": null, "full_delegation": null, "latency_p50_ms": null, "loads": true, "max_abs_diff": null, "max_rel_diff": null, "output_match": null, "peak_mem_mb": null, "prefill_tokens_per_s": 2308.15, "provenance": "measured", "runs": true, "total_ops": null, "ttft_ms": 121.4 }, "source": "data/device_runs/0.17.0/2026-09-07/spark-x2.5-1.7b-int8__mac-studio-m4-max.json" } ] }, "model": { "family": "spark-x2.5", "id": "spark-x2.5-1.7b-int8", "license": "apache-2.0", "source_url": "https://huggingface.co/litert-community/Spark-X2.5-1.7B", "task": "text-generation" }, "pitfalls": [ "int8 is the recommended file: GSM8K (n=100, greedy, 0-shot CoT, 3584 output tokens, own harness, Mac GPU) 76 = bf16 76 (paired: both 71, bf16-only 5, int8-only 5; 20 unfinished at the cap on both) and 8/8 on every gate (HF card; FINDINGS GSM8K).", "The iPhone int8 legs score 7/8 on both backends: the one miss is the rhyme line at the END of the composite 8-question prompt (it answers 'purple'), the composite-format artifact already seen on granite-4.2; asked alone the same question is answered 'blue' on every backend (HF card Performance; FINDINGS iPhone).", "Reasoning model: give it a generous output budget (>= 2048 tokens, 3584 for math) — truncated mid-thought it produces no final answer at all; the bundle pre-fills the vendor think opener (<|Bot|>) and declares the thought channel (…) so the runtime separates reasoning from the answer and honours a thinking budget; without the channel the runtime streams raw reasoning into the answer and ignores any budget (HF card Usage + Conversion notes).", "The vendor modeling code is patched for export, not re-implemented: it computes attention through its own eager function and ignores config._attn_implementation, so the export copy dispatches through the registered attention interface (litert-torch's transposed KV cache), threads the per-call kwargs, declares the attention-backend flags and applies the per-head sigmoid output gate in the interface's layout; eager output is bit-identical to the vendor file (max |dlogit| 0.0 on 24 random tokens); two further edits make the vendor file load under transformers 5 at all (HF card Conversion notes; FINDINGS parity).", "No start token in the metadata: the tokenizer declares <|start_of_sentence|> as BOS but never prepends it (add_bos_token false); the template carries its own, so the exporter's unconditional start_token write was suppressed (measured harmless in bf16 on the 8Q gate, but it is not the vendor prompt) (HF card Conversion notes).", "Prompt format carried as a Jinja template verbatim: a default system block ('you are a helpful assistant.', a user system prompt appended after it), every message wrapped in <|start_of_sentence|> … <|end_of_sentence|>, stop <|end_of_sentence|>; tool-call formatting is not carried; KV budget 4096 (the original's 1M context does not apply on-device); the vendor's recommended sampling is temperature 1.0 / top-p 0.95 while every number on the card is greedy (HF card Usage).", "The int4 file externalizes the embedding table: the vocab is tied, and asking int4 for lm_head and int8 for the embedding makes the quantizer copy the 131,072-row table once per signature (4.6 vs 1.7 bytes/param on a tiny checkpoint), so the embedder lives in its own section; the int8 file needs no split (HF card Conversion notes; FINDINGS de-risk).", "Tokenizer parity: the bundle's HF tokenizer.json section encodes 234/234 probe rows to the same ids as the tokenizers reading of the upstream file; multi-turn (3 turns, fact recall) passes with and without the runtime's channel filtering (HF card Correctness).", "iPhone 17 Pro rows come from the G41DeviceTest harness (composite 8-question prompt, --max-tokens 3072, byte count verified on-device): a generation gate, not a benchmark; the harness links swift-litert-lm's v0.15.0 pin and ran with the Increased Memory Limit entitlement — see the device rows' evidence (FINDINGS iPhone section; DECISIONS #164)." ], "schema_version": "1.2" }