{
  "artifacts": [
    {
      "file": "Qwen3-ASR-1.7B.litertlm",
      "sha256": "6155d5d5f8d57e7a3e50f669fd487ae0d8aed0845bb4091f39a894eea7067b95",
      "size_mb": 2568.306
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "export_fp32.py --input_sec 30 --cache_length 1024 (block-diagonal 104-token attention mask as a numpy constant; litert-torch PR #1295 carries the same fix) -> quant_sections.py --lm dynamic_wi8_afp32 --emb dynamic_wi8_afp32 --enc fp16 -> export_bundle.py --window 30 --prompt litert (encoder audio f32 [1,3000,160] -> features f32 [1,390,2048] + mask uint8 [1,390], fp16 weights, in-graph log-mel; generic_model metadata; litert-torch Qwen3-ASR prompt as the jinja). One-command reproduction: https://github.com/john-rocky/hf-to-litertlm/tree/main/qwen3_asr17_work",
    "quantization": "LM: int8 dynamic weights with an int8 embedding table (separate embedder section); audio encoder: fp16 weights, fp32 compute; 30 s window (3000 frames x 160 samples at 16 kHz -> 390 audio tokens); KV cache 1024",
    "tool": "litert-torch export_hf (task automatic_speech_recognition) for the fp32 graphs, with the encoder attention patched to the original 104-token windows; per-section quantization with ai-edge-quantizer; repack with litert-lm pack as a generic_model bundle whose encoder starts at raw PCM frames and computes the Whisper-style log-mel in-graph",
    "tool_version": "litert-torch 0.10.0 (731ef0a), transformers 5.14.1, torch 2.13.0, ai-edge-quantizer 0.9.0, ai-edge-litert 2.2.0, litert-lm-builder 0.17.0 (PYTHONPATH overlay), litert-lm (pack) 0.17.1 (litert-lm-builder 0.17.1)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-09",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-asr-litertlm-check",
            "os_build": "Android 16 (build BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "harness: LiteRT-Models zoo asr_litertlm (github.com/john-rocky/LiteRT-Models, commits 90f9c39 and 07c2670, pushed to main 2026-10-10), androidTest AsrDeviceCheck run by am instrument; litertlm-android 0.17.1 (asr_litertlm/gradle.properties litertlmVersion=0.17.1); the bundle pushed with adb into the app's external files dir and touched (LiteRT-LM #3772) before the check",
            "RESULT step=load ok=true model=qwen3-asr-1.7b backend=CPU load_s=0.72 initialize_s=0.30 first_conversation_s=0.43 cache=loaded cache_mb=2999 threads=4 vmhwm_mb=4327 vmrss_mb=4325 mem_available_mb=6163",
            "RESULT step=clip ok=true model=qwen3-asr-1.7b clip=zh backend=CPU audio_s=9.72 send_s=2.38 rtf=0.245 conversation_s=0.00 language=Chinese match=true cer=0.0000 text=宇宙中的一切都由物质构成，而所有的物质都由被称为原子的微小颗粒组成。",
            "RESULT step=clip ok=true model=qwen3-asr-1.7b clip=en backend=CPU audio_s=6.42 send_s=2.49 rtf=0.388 conversation_s=0.00 language=English match=true cer=0.0000 text=\"Everything in the universe is made of matter. All matter is made of tiny particles called atoms.\"",
            "RESULT step=clip ok=true model=qwen3-asr-1.7b clip=ja backend=CPU audio_s=11.04 send_s=3.56 rtf=0.323 conversation_s=0.00 language=Japanese match=true cer=0.0000 text=宇宙に存在するあらゆるものは物質でできています。あらゆる物質は原子と呼ばれる微小な粒子でできています。",
            "RESULT step=mic ok=true model=qwen3-asr-1.7b rec_s=9.00 send_s=2.97 rtf=0.330 cer=0.0000 language=English text=\"Everything in the universe is made of matter. All matter is made of tiny particles called atoms.\"",
            "RESULT ok=true scope=check steps=13 failed=0",
            "23:37:58 thermal: Thermal Status: 0",
            "23:37:59 cpufreq policy0: max=3513600 cpuinfo_max=3628800",
            "23:37:59 cpufreq policy6: max=4185600 cpuinfo_max=4742400",
            "23:38:31 thermal: Thermal Status: 0",
            "23:38:31 cpufreq policy0: max=2745600 cpuinfo_max=3628800",
            "23:38:31 cpufreq policy6: max=2668800 cpuinfo_max=4742400"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": 2490.0,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_s_total": 27.18,
            "clips": 3.0,
            "clips_text_equal_expected": 3.0,
            "earlier_run_audio_s_total": 27.18,
            "earlier_run_clips": 3.0,
            "earlier_run_clips_text_equal_expected": 3.0,
            "earlier_run_engine_initialize_ms": 300.0,
            "earlier_run_engine_load_ms": 710.0,
            "earlier_run_first_conversation_ms": 410.0,
            "earlier_run_long_clip_audio_s": 33.6,
            "earlier_run_long_clip_send_ms": 5070.0,
            "earlier_run_long_clip_sent_s": 30.0,
            "earlier_run_mic_cer": 0.0,
            "earlier_run_mic_recorded_s": 9.0,
            "earlier_run_mic_send_ms": 2900.0,
            "earlier_run_rtf": 0.2745,
            "earlier_run_runtime_cache_mb": 2999.0,
            "earlier_run_runtime_cache_written": 0.0,
            "earlier_run_scaling_max_khz_policy0_start": 3628800.0,
            "earlier_run_scaling_max_khz_policy6_start": 4742400.0,
            "earlier_run_send_ms_en": 2140.0,
            "earlier_run_send_ms_ja": 3190.0,
            "earlier_run_send_ms_max": 3190.0,
            "earlier_run_send_ms_min": 2130.0,
            "earlier_run_send_ms_zh": 2130.0,
            "earlier_run_threads": 4.0,
            "engine_initialize_ms": 300.0,
            "engine_load_ms": 720.0,
            "first_conversation_ms": 430.0,
            "first_run_engine_load_ms": 7350.0,
            "first_run_runtime_cache_written": 1.0,
            "long_clip_audio_s": 33.6,
            "long_clip_send_ms": 4990.0,
            "long_clip_sent_s": 30.0,
            "mic_cer": 0.0,
            "mic_recorded_s": 9.0,
            "mic_send_ms": 2970.0,
            "rtf": 0.3102,
            "runtime_cache_mb": 2999.0,
            "runtime_cache_written": 0.0,
            "scaling_max_khz_policy0_end": 2745600.0,
            "scaling_max_khz_policy0_start": 3513600.0,
            "scaling_max_khz_policy6_end": 2668800.0,
            "scaling_max_khz_policy6_start": 4185600.0,
            "send_ms_en": 2490.0,
            "send_ms_ja": 3560.0,
            "send_ms_max": 3560.0,
            "send_ms_min": 2380.0,
            "send_ms_zh": 2380.0,
            "thermal_status_end": 0.0,
            "thermal_status_start": 0.0,
            "threads": 4.0
          },
          "output_match": null,
          "peak_mem_mb": 4327.0,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "Android sample app (LiteRT-Models asr_litertlm), AsrDeviceCheck; Engine on the CPU with 4 threads and the audio encoder on the CPU, one Conversation per clip",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-09/qwen3-asr-1.7b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": 25.6,
          "delegated_ops": 2196,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-qwen3asr-ladder",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 Android kit litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + its 7 .so (edge-llm-bench android/bin/v0.18.0), pushed to /data/local/tmp/q17_ladder and run through q17_run_r4.sh (memory guard + wall limit), one process per clip, --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --max_num_tokens=1024 --max_output_tokens=256 --benchmark --report_peak_memory_footprint",
            "s26_cpu_ship: 4 processes, raw = the expected demo answers 4/4; warm en (fleurs_en_1698, 6.42 s, 23 output tokens): response 2.1083 s = audio_executor 0.5089 s + TTFT 0.74 s + (decode tokens - 1) / decode tok/s 0.8594 s; prefill 570.73 tok/s, decode 25.6 tok/s, peak private 4229.54 MB; CPU capped 0/3 1 s samples",
            "Replacing 1126 out of 1298 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1070 out of 1244 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode",
            "Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 534 out of 588 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 1896 out of 1958 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1804 out of 1865 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 2178.7,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_encoder_total_ops": 588.0,
            "audio_encoder_xnnpack_delegated_ops": 534.0,
            "audio_executor_s_en_warm": 0.509,
            "clips_equal_expected": 4.0,
            "cold_process_init_executor_ms": 3430.7,
            "cpu_capped_samples_en_warm": 0.0,
            "cpu_samples_en_warm": 3.0,
            "decode_tokens_en": 23.0,
            "init_executor_ms_en_warm": 77.7,
            "lm_decode_delegated_ops": 1804.0,
            "lm_decode_total_ops": 1865.0,
            "lm_prefill_128_delegated_ops": 1896.0,
            "lm_prefill_128_total_ops": 1958.0,
            "max_num_tokens": 1024.0,
            "max_output_tokens": 256.0,
            "prefill_tokens": 398.0,
            "process_wall_s_en_warm": 2.857,
            "processes": 4.0,
            "response_median_of_3_clips_s": 2.179,
            "response_s_cmn": 2.179,
            "response_s_en_warm": 2.108,
            "response_s_ja": 2.703
          },
          "output_match": null,
          "peak_mem_mb": 4229.54,
          "prefill_tokens_per_s": 570.73,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": 740.0
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "gpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": 24.79,
          "delegated_ops": 2542,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-qwen3asr-ladder",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 Android kit litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + its 7 .so (edge-llm-bench android/bin/v0.18.0), pushed to /data/local/tmp/q17_ladder and run through q17_run_r4.sh (memory guard + wall limit), one process per clip, --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --max_num_tokens=1024 --max_output_tokens=256 --benchmark --report_peak_memory_footprint",
            "s26_gpu_ship: 3 processes, raw = the expected demo answers 3/3; warm en (fleurs_en_1698, 6.42 s, 23 output tokens): response 2.1124 s = audio_executor 0.5049 s + TTFT 0.72 s + (decode tokens - 1) / decode tok/s 0.8875 s; prefill 587.28 tok/s, decode 24.79 tok/s, peak private 2505.65 MB; CPU capped 2/6 1 s samples",
            "Replacing 1298 out of 1298 node(s) with delegate (LITERT_CL) ... subgraph prefill_128",
            "Replacing 1244 out of 1244 node(s) with delegate (LITERT_CL) ... subgraph decode",
            "Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 534 out of 588 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "s26_gpu_ship_ret: the same form later in the hold (after the other forms' GPU runs), warm en response 2.8142 s = audio_executor 1.2024 + TTFT 0.72 + 0.8918 s; prefill 585.79 tok/s, decode 24.67 tok/s"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 2770.9,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_encoder_total_ops": 588.0,
            "audio_encoder_xnnpack_delegated_ops": 534.0,
            "audio_executor_s_en_warm": 0.505,
            "audio_executor_s_en_warm_return_leg": 1.202,
            "clips_equal_expected": 3.0,
            "cold_process_init_executor_ms": 3670.03,
            "cpu_capped_samples_en_warm": 2.0,
            "cpu_samples_en_warm": 6.0,
            "decode_tokens_en": 23.0,
            "init_executor_ms_en_warm": 2225.43,
            "lm_decode_delegated_ops": 1244.0,
            "lm_decode_total_ops": 1244.0,
            "lm_prefill_128_delegated_ops": 1298.0,
            "lm_prefill_128_total_ops": 1298.0,
            "max_num_tokens": 1024.0,
            "max_output_tokens": 256.0,
            "prefill_tokens": 398.0,
            "process_wall_s_en_warm": 5.287,
            "processes": 3.0,
            "response_median_of_3_clips_s": 2.771,
            "response_s_cmn": 2.817,
            "response_s_en_warm": 2.112,
            "response_s_en_warm_return_leg": 2.814,
            "response_s_ja": 2.771
          },
          "output_match": null,
          "peak_mem_mb": 2505.65,
          "prefill_tokens_per_s": 587.28,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": 720.0
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-03",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "155 rows, errors 0, empty 0; sum(seconds) 272.214 over 1836.104 s of audio (RTF 0.1483), median 1.7 s per clip; ru_maxrss 4543479808 B",
            "scorer: en 51/1150 words, zh 121/1954 chars, ja 154/2689 chars",
            "load average 4.55 at start, 8.50 at end: other jobs on the machine",
            "CPU weight caches present (written by render_check.py before the run)",
            "audio encoder graph on XNNPACK: VERBOSE: Replacing 534 out of 588 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 102 partitions for subgraph 0 (main). (the same encoder section)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": 1700.9,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_seconds": 1836.1,
            "clips": 155,
            "empty_transcripts": 0,
            "fleurs_en_wer_pct": 4.43,
            "fleurs_ja_cer_pct": 5.73,
            "fleurs_zh_cer_pct": 6.19,
            "per_clip_s_max": 3.17,
            "per_clip_s_min": 1.04,
            "processing_seconds": 272.2,
            "rtf": 0.148,
            "threads": 4
          },
          "output_match": null,
          "peak_mem_mb": 4333.0,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-03/qwen3-asr-1.7b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": null,
          "delegated_ops": 2196,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": null,
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "runtime: litert-lm-api 0.18.0 (PyPI wheel), Engine(backend=CPU(thread_count=4) | GPU(), audio_backend=CPU(thread_count=4)), SamplerConfig(top_k=1, top_p=1.0, temperature=0.0), one conversation per send",
            "One process, 3 demo clips x (1 + 5) sends, raw = the expected answers 18/18; response (perf_counter around send_message = audio encoder + prefill + decode) warm medians en 1.1912 s, cmn 1.24306 s, ja 1.49639 s; engine load 0.0986 s with the caches present; ru_maxrss at the end 4518756352 B; the Mac's measurement window held, 1-min load average 5.19677734375 -> 5.8876953125 (other processes running)",
            "155 FLEURS clips (en_us / cmn_hans_cn / ja_jp x 50 + 5 examples), greedy, one conversation per clip: en WER 4.435 % (51/1150), zh CER 6.397 % (125/1954), ja CER 5.987 % (161/2689); verdict PASS against eager fp32 + 1.0 pt",
            "Replacing 1126 out of 1298 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1070 out of 1244 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode",
            "Replacing 1896 out of 1958 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1804 out of 1865 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 1243.06,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "engine_load_s_caches_present": 0.099,
            "first_send_s_en": 1.186,
            "fleurs_clips": 155.0,
            "fleurs_cmn_cer_pct": 6.397,
            "fleurs_empty": 0.0,
            "fleurs_en_wer_pct": 4.435,
            "fleurs_ja_cer_pct": 5.987,
            "lm_decode_delegated_ops": 1804.0,
            "lm_decode_total_ops": 1865.0,
            "lm_prefill_128_delegated_ops": 1896.0,
            "lm_prefill_128_total_ops": 1958.0,
            "max_num_tokens": 1024.0,
            "prefill_tokens": 398.0,
            "response_median_of_3_clips_s": 1.243,
            "response_s_cmn_warm_median_of_5": 1.243,
            "response_s_en_warm_median_of_5": 1.191,
            "response_s_ja_warm_median_of_5": 1.496,
            "ru_maxrss_end_bytes": 4518756352.0,
            "sends": 18.0,
            "sends_equal_expected": 18.0
          },
          "output_match": null,
          "peak_mem_mb": 4518.756,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-10-03",
          "decode_tokens_per_s": null,
          "delegated_ops": 1298,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.17.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "155 rows, errors 0, empty 0; sum(seconds) 105.15 over 1836.104 s of audio (RTF 0.0573), median 0.66 s per clip; ru_maxrss 4794810368 B",
            "scorer: en 52/1150 words, zh 121/1954 chars, ja 143/2689 chars",
            "VERBOSE: Replacing 1298 out of 1298 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 0 (prefill_128). (the same sections; LlmMetadata differs only)",
            "VERBOSE: Replacing 1244 out of 1244 node(s) with delegate (LITERT_WEBGPU) node, yielding 1 partitions for subgraph 1 (decode). (the same sections; LlmMetadata differs only)",
            "Selected adapter: Apple M4 Max, arch=metal-3, vendor=apple, backend=Metal, adapterType=Integrated GPU",
            "first GPU load of this file: the 1,720,475,080 B mldrift weight cache was written during the run",
            "audio encoder on the CPU (audio_backend=CPU())"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 663.5,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_seconds": 1836.1,
            "clips": 155,
            "empty_transcripts": 0,
            "fleurs_en_wer_pct": 4.52,
            "fleurs_ja_cer_pct": 5.32,
            "fleurs_zh_cer_pct": 6.19,
            "per_clip_s_max": 1.06,
            "per_clip_s_min": 0.53,
            "processing_seconds": 105.2,
            "rtf": 0.057,
            "threads": 4
          },
          "output_match": null,
          "peak_mem_mb": 4572.7,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": 1298,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.17.1/2026-10-03/qwen3-asr-1.7b__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": null,
          "delegated_ops": 2542,
          "env": {
            "device": "Mac Studio (M4 Max)",
            "machine_label": "mac-studio-m4-max",
            "os_build": null,
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "runtime: litert-lm-api 0.18.0 (PyPI wheel), Engine(backend=CPU(thread_count=4) | GPU(), audio_backend=CPU(thread_count=4)), SamplerConfig(top_k=1, top_p=1.0, temperature=0.0), one conversation per send",
            "One process, 3 demo clips x (1 + 5) sends, raw = the expected answers 18/18; response (perf_counter around send_message = audio encoder + prefill + decode) warm medians en 0.53835 s, cmn 0.55504 s, ja 0.64979 s; engine load 0.1955 s with the caches present; ru_maxrss at the end 2492465152 B; the Mac's measurement window held, 1-min load average 10.63525390625 -> 8.9375 (other processes running)",
            "155 FLEURS clips (en_us / cmn_hans_cn / ja_jp x 50 + 5 examples), greedy, one conversation per clip: en WER 4.522 % (52/1150), zh CER 6.192 % (121/1954), ja CER 5.318 % (143/2689); verdict PASS against eager fp32 + 1.0 pt",
            "Replacing 1298 out of 1298 node(s) with delegate (LITERT_WEBGPU) ... subgraph prefill_128",
            "Replacing 1244 out of 1244 node(s) with delegate (LITERT_WEBGPU) ... subgraph decode"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 555.04,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "engine_load_s_caches_present": 0.196,
            "first_send_s_en": 0.538,
            "fleurs_clips": 155.0,
            "fleurs_cmn_cer_pct": 6.192,
            "fleurs_empty": 0.0,
            "fleurs_en_wer_pct": 4.522,
            "fleurs_ja_cer_pct": 5.318,
            "lm_decode_delegated_ops": 1244.0,
            "lm_decode_total_ops": 1244.0,
            "lm_prefill_128_delegated_ops": 1298.0,
            "lm_prefill_128_total_ops": 1298.0,
            "max_num_tokens": 1024.0,
            "prefill_tokens": 398.0,
            "response_median_of_3_clips_s": 0.555,
            "response_s_cmn_warm_median_of_5": 0.555,
            "response_s_en_warm_median_of_5": 0.538,
            "response_s_ja_warm_median_of_5": 0.65,
            "ru_maxrss_end_bytes": 2492465152.0,
            "sends": 18.0,
            "sends_equal_expected": 18.0
          },
          "output_match": null,
          "peak_mem_mb": 2492.465,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "cpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": 11.04,
          "delegated_ops": 2196,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-qwen3asr-ladder",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 Android kit litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + its 7 .so (edge-llm-bench android/bin/v0.18.0), pushed to /data/local/tmp/q17_ladder and run through q17_run_r4.sh (memory guard + wall limit), one process per clip, --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --max_num_tokens=1024 --max_output_tokens=256 --benchmark --report_peak_memory_footprint",
            "p8a_cpu_ship: 4 processes, raw = the expected demo answers 4/4; warm en (fleurs_en_1698, 6.42 s, 23 output tokens): response 8.3926 s = audio_executor 3.0698 s + TTFT 3.33 s + (decode tokens - 1) / decode tok/s 1.9928 s; prefill 122.68 tok/s, decode 11.04 tok/s, peak private 4148.64 MB; CPU capped 0/11 1 s samples",
            "Replacing 1126 out of 1298 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1070 out of 1244 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode",
            "Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 534 out of 588 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 1896 out of 1958 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph prefill_128",
            "Replacing 1804 out of 1865 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph decode"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": 9484.6,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_encoder_total_ops": 588.0,
            "audio_encoder_xnnpack_delegated_ops": 534.0,
            "audio_executor_s_en_warm": 3.07,
            "clips_equal_expected": 4.0,
            "cold_process_init_executor_ms": 6867.19,
            "cpu_capped_samples_en_warm": 0.0,
            "cpu_samples_en_warm": 11.0,
            "decode_tokens_en": 23.0,
            "init_executor_ms_en_warm": 167.05,
            "lm_decode_delegated_ops": 1804.0,
            "lm_decode_total_ops": 1865.0,
            "lm_prefill_128_delegated_ops": 1896.0,
            "lm_prefill_128_total_ops": 1958.0,
            "max_num_tokens": 1024.0,
            "max_output_tokens": 256.0,
            "prefill_tokens": 398.0,
            "process_wall_s_en_warm": 10.261,
            "processes": 4.0,
            "response_median_of_3_clips_s": 9.485,
            "response_s_cmn": 11.954,
            "response_s_en_warm": 8.393,
            "response_s_ja": 9.485
          },
          "output_match": null,
          "peak_mem_mb": 4148.64,
          "prefill_tokens_per_s": 122.68,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": 3330.0
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__pixel-8a.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "gpu",
          "context_length": 1024,
          "date": "2026-10-13",
          "decode_tokens_per_s": 10.4,
          "delegated_ops": 2542,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-qwen3asr-ladder",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 Android kit litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + its 7 .so (edge-llm-bench android/bin/v0.18.0), pushed to /data/local/tmp/q17_ladder and run through q17_run_r4.sh (memory guard + wall limit), one process per clip, --audio_backend=cpu --sampler_backend=cpu --num_cpu_threads=4 --max_num_tokens=1024 --max_output_tokens=256 --benchmark --report_peak_memory_footprint",
            "p8a_gpu_ship: 3 processes, raw = the expected demo answers 3/3; warm en (fleurs_en_1698, 6.42 s, 23 output tokens): response 9.8585 s = audio_executor 5.0131 s + TTFT 2.73 s + (decode tokens - 1) / decode tok/s 2.1154 s; prefill 151.32 tok/s, decode 10.4 tok/s, peak private 2702.2 MB; CPU capped 7/21 1 s samples",
            "Replacing 1298 out of 1298 node(s) with delegate (LITERT_CL) ... subgraph prefill_128",
            "Replacing 1244 out of 1244 node(s) with delegate (LITERT_CL) ... subgraph decode",
            "Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "Replacing 534 out of 588 node(s) with delegate (TfLiteXNNPackDelegate) ... subgraph main",
            "p8a_gpu_ship_ret: the same form later in the hold (after the other forms' GPU runs), warm en response 8.6204 s = audio_executor 3.6858 + TTFT 2.7 + 2.2346 s; prefill 152.75 tok/s, decode 10.74 tok/s"
          ],
          "failure_class": null,
          "full_delegation": true,
          "latency_p50_ms": 10009.0,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "audio_encoder_total_ops": 588.0,
            "audio_encoder_xnnpack_delegated_ops": 534.0,
            "audio_executor_s_en_warm": 5.013,
            "audio_executor_s_en_warm_return_leg": 3.686,
            "clips_equal_expected": 3.0,
            "cold_process_init_executor_ms": 29777.51,
            "cpu_capped_samples_en_warm": 7.0,
            "cpu_samples_en_warm": 21.0,
            "decode_tokens_en": 23.0,
            "init_executor_ms_en_warm": 7370.43,
            "lm_decode_delegated_ops": 1244.0,
            "lm_decode_total_ops": 1244.0,
            "lm_prefill_128_delegated_ops": 1298.0,
            "lm_prefill_128_total_ops": 1298.0,
            "max_num_tokens": 1024.0,
            "max_output_tokens": 256.0,
            "prefill_tokens": 398.0,
            "process_wall_s_en_warm": 20.674,
            "processes": 3.0,
            "response_median_of_3_clips_s": 10.009,
            "response_s_cmn": 11.892,
            "response_s_en_warm": 9.858,
            "response_s_en_warm_return_leg": 8.62,
            "response_s_ja": 10.009
          },
          "output_match": null,
          "peak_mem_mb": 2702.2,
          "prefill_tokens_per_s": 151.32,
          "provenance": "measured",
          "runs": true,
          "total_ops": 2542,
          "ttft_ms": 2730.0
        },
        "source": "data/device_runs/0.18.0/2026-10-13/qwen3-asr-1.7b__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "qwen3-asr",
    "id": "qwen3-asr-1.7b",
    "license": "apache-2.0 (the license of Qwen/Qwen3-ASR-1.7B-hf)",
    "source_url": "https://huggingface.co/litert-community/Qwen3-ASR-1.7B",
    "task": "automatic-speech-recognition"
  },
  "pitfalls": [
    "One message = one clip of up to 30 s: the runtime zero-fills the window after the clip and the encoder emits 390 audio tokens per window, so encoding and prefill cost the same for a 4 s and a 29 s clip; clips longer than 30 s were not tested (Hub card 'How to send audio').",
    "The answer is 'language <name><asr_text><transcript>'; split at <asr_text>. To force the language, send the text item 'language Chinese<asr_text>' in the same message; the model then writes the transcript only (11 of 11 Mandarin / Japanese clips) (Hub card).",
    "Prompt: the bundle renders the litert-torch Qwen3-ASR prompt (no system turn). With the checkpoint's own chat template (empty system turn) the same weights scored Mandarin CER 6.65 % / Japanese CER 6.06 % vs 6.19 / 5.73 and tagged 8 of 50 Mandarin clips English; the PyTorch model shows the same pattern (6.86 / 5.50 vs 6.04 / 5.24, 9 of 50). System messages are ignored by the template (Hub card 'Prompt').",
    "The audio encoder must run on the CPU (audio_backend=CPU()): on the Mac the WebGPU/Metal delegate refuses 'BATCH_MATMUL: Not supported batched mat mul case: non-constant tensor' in the encoder graph and create_conversation fails ('Hint fully delegated to single delegate is set, but the graph is not fully delegated'); the LM runs on the GPU (prefill_128 1298/1298 and decode 1244/1244 nodes delegated on Metal) (same encoder section).",
    "Correctness (FLEURS test, 50 clips per language, greedy, one conversation per clip; scorer: en WER after uppercasing/punctuation removal, zh/ja CER after NFKC+lower with punctuation/whitespace removed): Mac CPU 4.43 / 6.19 / 5.73 %, Mac LM-on-GPU 4.52 / 6.19 / 5.32 %, PyTorch fp32 with the same prompt 4.35 / 6.04 / 5.24 %; no empty transcript (Hub card 'Correctness').",
    "Stock litert-torch export deviation: the exporter's audio attention attends within 13-token (1 s) chunks while the original attends within 104-token windows (encoder output cosine 0.81-0.82 on 5 s, 0.76 on 30 s); the constant block-diagonal mask makes the fp32 encoder match the original exactly — litert-torch Draft PR #1295 / issue #1296 (Hub card 'Conversion' 1).",
    "The released runtime's built-in mel front end is not the Qwen3-ASR log-mel; the bundle takes raw PCM frames and computes the original log-mel in the graph (max |Δ| 0.028 vs the original on 10 clips, from the fp16 weights) (Hub card 'Conversion' 2).",
    "The omni/asr runner in the LiteRT-LM repository was not run with this file; the generic_model bundle is the shipped route (Hub card 'Conversion'; the omni/asr start-token defect is LiteRT-LM issue #3827, measured on confucius4-r2t2).",
    "Mac speed rows were measured while other jobs ran on the machine (load average 4.5-8.5); the GPU run was the file's first GPU load and wrote the 1.72 GB GPU weight cache during the run (Hub card 'Performance').",
    "Galaxy S26 CPU (v0.16.1 tag CLI, --benchmark): the first process writes the XNNPACK weight caches (prefill 329 tok/s, TTFT 1.25 s), later processes load them (prefill 538-547 tok/s, TTFT 0.77-0.78 s). After adb push the cache key's mtime second can alternate between processes so every process rebuilds the caches (LiteRT-LM #3772; seen on the first build's S26 run) — touch the bundle on the phone before timing warm runs."
  ],
  "schema_version": "1.2"
}
