{
  "artifacts": [
    {
      "file": "NVIDIA-Nemotron-Labs-Dory1.0-4B_int8.litertlm",
      "sha256": "25632f91399cf7408d546c1c0531290d1c9ca1b7d4e9b75826600bfc2f9f6083",
      "size_mb": 3859.21
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 rev b366b87f -> export with the 20 recurrent pairs unrolled twice (weights stored once), per-loop KV caches (56), prefill signatures 128 / 512 / 1024 plus decode over a 2048-token KV budget, the embedding table in its own section; the MLP normalization of 11 of the 36 MLP blocks scaled by 2^-k (k = 1-3) before the sum with the reciprocal square root capped (the same numbers in fp32) -> int8 dynamic (channelwise int8 weights, float activations) on the linear layers, int8 embedding -> metadata (NVIDIA's template with three runtime edits, no start token, stop tokens </s> and <|im_end|> plus 14 stop sequences, the thought channel, max_num_tokens 2048) (card L33, L130-L137)",
    "quantization": "int8 dynamic (channelwise int8 weights, float activations) on the linear layers; int8 embedding in its own section (card L33)",
    "tool": "litert-torch 0.9.4 `export_hf` with a model definition of its own (transformers has no Dory class; the formulas follow NVIDIA's vLLM pull request); repro = hf-to-litertlm dory_work/convert_dory_4b.sh (REPRODUCE.md \"NVIDIA-Nemotron-Labs-Dory1.0-4B\") (card L24, L130)",
    "tool_version": "litert-torch 0.9.4, ai-edge-quantizer 0.9.0, litert-converter 0.4.0, torch 2.13.0, transformers 5.14.1; packed with litert-lm-builder 0.18.0 (card L130)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-22",
          "decode_tokens_per_s": null,
          "delegated_ops": 6082,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-dory-chat-check",
            "os_build": "Android 16 (BP4A.251205.006.S942QOPS1AZH9)",
            "runtime": "litert-lm",
            "runtime_version": "0.16.1",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "harness: hfmodels-android samples/chat ChatDeviceCheck (androidTest), hfmodels SDK 0.2.1-SNAPSHOT over com.google.ai.edge.litertlm:litertlm-android 0.16.1 (the SDK's pin), debug build; one model load and ONE conversation per run, the descriptor's thinking default (on), the engine's default sampler (the bundle declares none: greedy); load_ms = the wall of fromPretrained (an adb-pushed copy imported after its sha256 check), first_chunk_ms / total_ms = the walls from send to the first and the last streamed chunk; chunks are the streamed pieces, not tokens (the SDK states no token count); hfmodels-android origin/main e93b645a litertlm/results/2026-10-22-RFGL80R6A6H-litertlm0.16.1-dory-chat-check.log: # hfmodels on litertlm-android 0.16.1 (at commit 5e497ba): samples/chat's ChatDeviceCheck with litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B (int4 gpu: 17 + 25, then the model card's question; int8 cpu: 17 + 25), log_thought=true / # serial=RFGL80R6A6H device=SM-S942Q (Galaxy S26) build=BP4A.251205.006.S942QOPS1AZH9 android=16; chat-debug.apk sha256 6178de70…, chat-debug-androidTest.apk sha256 2345a28b…; each run: its start / end line (uptime, MemAvailable, thermal status, free /data), its log lines, and the runtime's delegate lines",
            "litertlm/results/2026-10-22-RFGL80R6A6H-litertlm0.16.1-dory-chat-check.log run x3 (x3 check litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B cpu int8): start 2026-10-22 23:45:26, end 2026-10-22 23:53:29, rc 0",
            "10-22 23:45:27.659  9101  9116 I hfmodels: resolve litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B: commit=61cb6bbe via SAVED_BINDING, descriptor=john-rocky/hfmodels-android@1db6c03d/catalog/entries/litert-community__NVIDIA-Nemotron-Labs-Dory1.0-4B.json sha=ca7a357b",
            "10-22 23:52:49.847  9101  9114 I hfmodels: ready litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B@61cb6bbe variant=int8 profile=cpu language=CPU prepare_ms=7442",
            "10-22 23:52:49.848  9101  9114 I hfmodels-check: RESULT step=load ok=true model=litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B variant=int8 profile=cpu initialized=[language=CPU] thinking=descriptor load_ms=442489 last_event=Ready",
            "10-22 23:53:23.565  9101  9114 I hfmodels-check: RESULT step=answer ok=true model=litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B expect=\"42\" chunks=119 thought_chunks=117 first_chunk_ms=2837 total_ms=33715 state=READY thought_chars=480 reply=\"42\"",
            "10-22 23:53:28.209  9101  9114 I hfmodels-check: RESULT step=stop ok=true model=litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B chunks_at_stop=5 state=INVALID",
            "10-22 23:53:28.558  9101  9114 I hfmodels-check: RESULT step=release ok=true model=litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B close_ms=348",
            "10-22 23:53:28.559  9101  9114 I hfmodels-check: RESULT ok=true model=litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B failed=[] sdk=0.2.1-SNAPSHOT runtime=litert_lm 0.16.1 device=SM-S942Q build=BP4A.251205.006.S942QOPS1AZH9",
            "10-22 23:52:43.454  9101  9116 I tflite  : Replacing 6082 out of 6207 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 113 partitions for subgraph 0 (prefill_128).",
            "10-22 23:52:48.788  9101  9116 I tflite  : Replacing 6082 out of 6207 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 113 partitions for subgraph 1 (prefill_512).",
            "10-22 23:52:48.812  9101  9116 I tflite  : Replacing 6082 out of 6207 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 113 partitions for subgraph 2 (prefill_1024).",
            "10-22 23:52:48.835  9101  9116 I tflite  : Replacing 5442 out of 5566 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 114 partitions for subgraph 3 (decode).",
            "10-22 23:52:49.836  9101  9116 I tflite  : Replacing 1 out of 4 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 2 partitions for subgraph 0 (decode_embedder)."
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answer_matches_expect": 1.0,
            "chunks": 119.0,
            "close_ms": 348.0,
            "first_chunk_ms": 2837.0,
            "load_ms": 442489.0,
            "memavailable_end_kb": 7346092.0,
            "memavailable_start_kb": 7355524.0,
            "thermal_status_end": 0.0,
            "thermal_status_start": 0.0,
            "thought_chars": 480.0,
            "thought_chunks": 117.0,
            "total_ms": 33715.0,
            "uptime_end_s": 608743.17,
            "uptime_start_s": 608260.09
          },
          "output_match": null,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "Android library (hfmodels-android samples/chat ChatDeviceCheck, debug build); one conversation, 'What is 17 + 25? Answer briefly.' (expect '42')",
          "total_ops": 6207,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.16.1/2026-10-22/nemotron-labs-dory1.0-4b-int8__galaxy-s26.json"
      },
      {
        "device": "galaxy-s26",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-22",
          "decode_tokens_per_s": 5.0625,
          "delegated_ops": 6082,
          "env": {
            "device": "Galaxy S26 (SM-S942Q)",
            "machine_label": "galaxy-s26-dory",
            "os_build": "Android 16 (BP4A.251205.006)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Qualcomm SM8850",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold s26-h6 (r10): HOLD_SEEN 2026-10-22 00:13:12: acquired 00:13:10, skip_after 00:38:10, run_end 00:41:10, released by 00:43:10; keeper released the hold at 2026-10-22 00:38:50",
            "hold s26-h8 (r10): HOLD_SEEN 2026-10-22 00:50:19: acquired 00:50:17, skip_after 01:15:17, run_end 01:18:17, released by 01:20:17; keeper released the hold at 2026-10-22 01:02:17",
            "(s26-h6, 1792595899 device epoch, ready=uncapped: uncapped (thermal 1, no CPU cap, kgsl_pwrlevel 0); rested 306s since 00:13:12; thermal status 1 -> 3): iterations 2-3 prefill 85.77 / 72.85, decode 4.92 / 4.79 tok/s, TTFT 12.14 / 14.27 s, Init Total 4388.47 ms (cold); .adb.txt EXIT=0 WALL=192 START=1792595899 END=1792596091; .meta.json memory 'VmHWM_max_kB 5716768 VmSwap_max_kB 0 MemAvailable_min_kB 5327720 kgsl_page_alloc_max_B 188502016'; sampler 81 samples, CPU capped in 79",
            "(s26-h8, 1792597942 device epoch, ready=uncapped: clean; rested 122s since 00:50:19; thermal status 0 -> 2): iterations 2-3 prefill 87.33 / 86.15, decode 5.37 / 5.17 tok/s, TTFT 11.91 / 12.08 s, Init Total 4359.91 ms (cold); .adb.txt EXIT=0 WALL=178 START=1792597942 END=1792598120; .meta.json memory 'VmHWM_max_kB 5714000 VmSwap_max_kB 0 MemAvailable_min_kB 5069208 kgsl_page_alloc_max_B 188059648'; sampler 77 samples, CPU capped in 76",
            "command: --backend=cpu --model_path=/data/local/tmp/dory_r10/f8.litertlm --input_prompt_file=/data/local/tmp/dory_r10/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=2048 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --use_session=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 6082 out of 6207 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 113 partitions for subgraph 2 (prefill_1024).",
            "Replacing 5442 out of 5566 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 114 partitions for subgraph 3 (decode).",
            "Device: model=SM-S942Q release=16 build_id=BP4A.251205.006 fingerprint=samsung/m1qjpnx/m1q:16/BP4A.251205.006/S942QOPS1AZH9_SJP1AZH9:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cold_start": 1.0,
            "run1_cpu_capped_samples": 79.0,
            "run1_decode_tok_s": 4.855,
            "run1_decode_tok_s_it1": 5.46,
            "run1_decode_tok_s_it2": 4.92,
            "run1_decode_tok_s_it3": 4.79,
            "run1_init_total_ms": 4388.47,
            "run1_memavail_min_kb": 5327720.0,
            "run1_prefill_tok_s": 79.31,
            "run1_prefill_tok_s_it1": 120.74,
            "run1_prefill_tok_s_it2": 85.77,
            "run1_prefill_tok_s_it3": 72.85,
            "run1_sampler_samples": 81.0,
            "run1_t_s": 65.6405,
            "run1_thermal_status_end": 3.0,
            "run1_thermal_status_start": 1.0,
            "run1_ttft_s": 13.205,
            "run1_vmhwm_kb": 5716768.0,
            "run1_vmswap_max_kb": 0.0,
            "run2_cold_start": 1.0,
            "run2_cpu_capped_samples": 76.0,
            "run2_decode_tok_s": 5.27,
            "run2_decode_tok_s_it1": 5.93,
            "run2_decode_tok_s_it2": 5.37,
            "run2_decode_tok_s_it3": 5.17,
            "run2_init_total_ms": 4359.91,
            "run2_memavail_min_kb": 5069208.0,
            "run2_prefill_tok_s": 86.74,
            "run2_prefill_tok_s_it1": 114.27,
            "run2_prefill_tok_s_it2": 87.33,
            "run2_prefill_tok_s_it3": 86.15,
            "run2_sampler_samples": 77.0,
            "run2_t_s": 60.3822,
            "run2_thermal_status_end": 2.0,
            "run2_thermal_status_start": 0.0,
            "run2_ttft_s": 11.995,
            "run2_vmhwm_kb": 5714000.0,
            "run2_vmswap_max_kb": 0.0,
            "runs": 2.0,
            "t_1024_256_s": 62.9015
          },
          "output_match": null,
          "peak_mem_mb": 5852.553,
          "prefill_tokens_per_s": 83.025,
          "provenance": "measured",
          "runs": true,
          "total_ops": 6207,
          "ttft_ms": 12600.0
        },
        "source": "data/device_runs/0.18.0/2026-10-22/nemotron-labs-dory1.0-4b-int8__galaxy-s26.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-22",
          "decode_tokens_per_s": 15.7,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "Settle: quiet after 0 s; warm-up + 3 runs; prefill 282.74 / decode 15.69 tok/s, TTFT 3.8055 s, init 23.7475 s (the CLI prints the mean of the 3 runs); maximum resident set size 26379124736 B, peak memory footprint 22525172056 B; 0 'Validation error' lines",
            "2026-10-22 00:16:30 [w3] START s1_cpu_a1 cap 420s (disk 368 GB free) / 2026-10-22 00:18:39 [w3] DONE s1_cpu_a1 rc=0 129s",
            "Settle: quiet after 0 s; warm-up + 3 runs; prefill 321.33 / decode 15.71 tok/s, TTFT 3.251 s, init 26.9076 s (the CLI prints the mean of the 3 runs); maximum resident set size 26400964608 B, peak memory footprint 22541015216 B; 0 'Validation error' lines",
            "2026-10-22 00:23:08 [w3] START s1_cpu_a2 cap 337s (disk 365 GB free) / 2026-10-22 00:25:17 [w3] DONE s1_cpu_a2 rc=0 129s",
            "R8_C s1: r8_s1_cpu_a1.log 282.74 / 15.69 tok/s; r8_s1_cpu_b1.log 82.25 / 15.7 tok/s; r8_s1_cpu_b2.log 79.11 / 15.73 tok/s; r8_s1_cpu_a2.log 321.33 / 15.71 tok/s; r8_s1_g16.log 1011.2 / 43.29 tok/s; r8_s1_g32.log 875.56 / 40.73 tok/s; r8_s1_t512_g16.log 968.55 / 44.69 tok/s; r8_s1_t512_cpu.log 310.6 / 15.75 tok/s"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "init_s": 25.3275,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_decode_tok_s": 15.69,
            "run1_init_s": 23.7475,
            "run1_max_rss_mb": 26379.125,
            "run1_peak_footprint_mb": 22525.172,
            "run1_prefill_tok_s": 282.74,
            "run1_t_s": 19.9378,
            "run1_ttft_s": 3.8055,
            "run2_decode_tok_s": 15.71,
            "run2_init_s": 26.9076,
            "run2_max_rss_mb": 26400.965,
            "run2_peak_footprint_mb": 22541.015,
            "run2_prefill_tok_s": 321.33,
            "run2_t_s": 19.4821,
            "run2_ttft_s": 3.251,
            "runs": 2.0,
            "t_1024_256_s": 19.6961,
            "warmup_runs": 1.0
          },
          "output_match": null,
          "peak_mem_mb": 22533.094,
          "prefill_tokens_per_s": 302.035,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 3528.2
        },
        "source": "data/device_runs/0.18.0/2026-10-22/nemotron-labs-dory1.0-4b-int8__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-8a",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-10-22",
          "decode_tokens_per_s": 1.18,
          "delegated_ops": 6082,
          "env": {
            "device": "Pixel 8a",
            "machine_label": "pixel-8a-dory",
            "os_build": "Android 16 (CP1A.260505.005)",
            "runtime": "litert-lm",
            "runtime_version": "0.18.0",
            "soc": "Google Tensor G3",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "binary: LiteRT-LM v0.18.0 litert_lm_advanced_main sha256 cf3636070a8e51b0ea13d9a4a8b26fc9a2f9aa38f9e1c7687fe009ce7550f815 + 7 .so (the edge-llm-bench source build of the v0.18.0 tag, 2026-10-07, android/engine-pins.json v0.18.0), every file sha256-checked on the phone at each hold's start (s0_kit_check.txt: 8 OK)",
            "hold p8a-h3 (r10): HOLD_SEEN 2026-10-21 23:49:08: acquired 23:49:07, skip_after 00:14:07, run_end 00:17:07, released by 00:19:07; keeper released the hold at 2026-10-22 00:11:03",
            "hold p8a-h9 (r10): HOLD_SEEN 2026-10-22 01:54:46: acquired 01:54:44, skip_after 02:19:44, run_end 02:22:44, released by 02:24:44; keeper released the hold at 2026-10-22 02:13:22",
            "(p8a-h3, 1792594459 device epoch, ready=clean: warm (thermal 1, cap policy0:1425000/1704000 policy4:1065000/2367000 policy8:1164000/2914000 , gpu_cooling 0); rested 309s since 23:49:08; thermal status 1 -> 3): iterations 2-3 prefill 10.87 / 11.08, decode 0.97 / 1.12 tok/s, TTFT 95.25 / 93.28 s, Init Total 32513.32 ms (cold); .adb.txt EXIT=0 WALL=994 START=1792594459 END=1792595453; .meta.json memory 'VmHWM_max_kB 4803468 VmSwap_max_kB 1316812 MemAvailable_min_kB 2680444 mali_total_gpu_mem_max_B 254013440'; sampler 255 samples, CPU capped in 255",
            "(p8a-h9, 1792601927 device epoch, ready=clean: clean; rested 239s since 01:54:46; thermal status 0 -> 1): iterations 2-3 prefill 14.89 / 13.32, decode 1.35 / 1.28 tok/s, TTFT 69.52 / 77.67 s, Init Total 21346.91 ms (cold); .adb.txt EXIT=0 WALL=866 START=1792601927 END=1792602793; .meta.json memory 'VmHWM_max_kB 4830644 VmSwap_max_kB 1288840 MemAvailable_min_kB 2339620 mali_total_gpu_mem_max_B 255533056'; sampler 234 samples, CPU capped in 224",
            "command: --backend=cpu --model_path=/data/local/tmp/dory_r10/f8.litertlm --input_prompt_file=/data/local/tmp/dory_r10/prompts/q1.txt --max_num_tokens=2048 --max_output_tokens=2048 --async=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --use_session=false --use_session=true --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3",
            "activation_data_type: Not set",
            "Replacing 6082 out of 6207 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 113 partitions for subgraph 2 (prefill_1024).",
            "Replacing 5442 out of 5566 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 114 partitions for subgraph 3 (decode).",
            "Device: model=Pixel 8a release=16 build_id=CP1A.260505.005 fingerprint=google/akita/akita:16/CP1A.260505.005/15081906:user/release-keys"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "iterations": 3.0,
            "max_num_tokens": 2048.0,
            "prefill_tokens": 1024.0,
            "run1_cold_start": 1.0,
            "run1_cpu_capped_samples": 255.0,
            "run1_decode_tok_s": 1.045,
            "run1_decode_tok_s_it1": 1.2,
            "run1_decode_tok_s_it2": 0.97,
            "run1_decode_tok_s_it3": 1.12,
            "run1_init_total_ms": 32513.32,
            "run1_memavail_min_kb": 2680444.0,
            "run1_prefill_tok_s": 10.975,
            "run1_prefill_tok_s_it1": 15.94,
            "run1_prefill_tok_s_it2": 10.87,
            "run1_prefill_tok_s_it3": 11.08,
            "run1_sampler_samples": 255.0,
            "run1_t_s": 338.279,
            "run1_thermal_status_end": 3.0,
            "run1_thermal_status_start": 1.0,
            "run1_ttft_s": 94.265,
            "run1_vmhwm_kb": 4803468.0,
            "run1_vmswap_max_kb": 1316812.0,
            "run2_cold_start": 1.0,
            "run2_cpu_capped_samples": 224.0,
            "run2_decode_tok_s": 1.315,
            "run2_decode_tok_s_it1": 0.96,
            "run2_decode_tok_s_it2": 1.35,
            "run2_decode_tok_s_it3": 1.28,
            "run2_init_total_ms": 21346.91,
            "run2_memavail_min_kb": 2339620.0,
            "run2_prefill_tok_s": 14.105,
            "run2_prefill_tok_s_it1": 25.56,
            "run2_prefill_tok_s_it2": 14.89,
            "run2_prefill_tok_s_it3": 13.32,
            "run2_sampler_samples": 234.0,
            "run2_t_s": 267.2752,
            "run2_thermal_status_end": 1.0,
            "run2_thermal_status_start": 0.0,
            "run2_ttft_s": 73.595,
            "run2_vmhwm_kb": 4830644.0,
            "run2_vmswap_max_kb": 1288840.0,
            "runs": 2.0,
            "t_1024_256_s": 298.6078
          },
          "output_match": null,
          "peak_mem_mb": 4932.665,
          "prefill_tokens_per_s": 12.54,
          "provenance": "measured",
          "runs": true,
          "total_ops": 6207,
          "ttft_ms": 83930.0
        },
        "source": "data/device_runs/0.18.0/2026-10-22/nemotron-labs-dory1.0-4b-int8__pixel-8a.json"
      }
    ]
  },
  "model": {
    "family": "dory",
    "id": "nemotron-labs-dory1.0-4b-int8",
    "license": "other (openmdw-1.1)",
    "source_url": "https://huggingface.co/litert-community/NVIDIA-Nemotron-Labs-Dory1.0-4B",
    "task": "text-generation"
  },
  "pitfalls": [
    "CPU file: use the int8 file on a CPU and the int4 file on a GPU. An earlier int4 build prefilled at a third of the int8 rate on the Galaxy S26 CPU (29.0 against 92.1 tok/s), and the int4 weights' greedy answers change from one CPU to another (card L36, L50).",
    "Accuracy: GSM8K questions 0-99, thinking off, greedy, at most 2,048 tokens of prompt and reply: this file 93 / 100 on the Mac CPU, the BF16 reference (llama.cpp, two loops) 95 and 86 with one loop; with thinking on the 2048-token budget is short for GSM8K (the two-loop reference left 29 of the first 32 questions with the thought still open) (card L36, L58-L68).",
    "Phones: the CPU was capped through almost every sample of every CPU run (sustained rates); the int8 file on a GPU was not run on the phones: on the 12 GB Galaxy S26 an earlier int4 build with fp32 activations already left 1.47 GB of MemAvailable and the int8 weights are 1.6 GB larger (card L122, L126).",
    "NPU: a static-range build (int16 activations, int8 weights) on the Galaxy S26: the Qualcomm plugin took the whole graph, but within 9 s of the on-phone compile free memory fell from 6.93 to 0.92 GB and the harness stopped it; an ahead-of-time build was not tried; there is no NPU file (card L125).",
    "Thinking and template: thinking is on by default (the generation prompt ends with <think> and a newline, the bundle declares the thought channel); rendered with enable_thinking false the prompt ends with <think></think>; NVIDIA recommends temperature 1.0 and top-p 0.95, the bundle sets no sampler; model type generic_model (tool calls not parsed; tool calling not tested) (card L97, L99).",
    "Final mark on LiteRT-LM 0.16.1 and earlier: of the 14 stop sequences, 13 put one of 13 characters (12 punctuation marks and a space) in front of <|im_end|> and a newline; an answer whose last mark is a token of its own, such as a ? or . after a word, arrives without that mark on LiteRT-LM 0.16.1 (the runtime of the Android app under Loops) and on 0.11.0 to 0.16.0 (source read): the runtime holds the mark back, the answer ends on <|im_end|> alone and the held mark is never written; a mark the tokenizer merges with the text before it (**., ).) is kept; 0.17.0 and later write the held mark (card L97).",
    "Second turn with thinking off: from the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because the file holds the embedding table in its own section and the decoder takes embeddings (read in the source of 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on these files), so later answers can differ from the answer to the same messages in one prompt; with thinking on the runtime rebuilds the cache at each turn, and the source shows a zero embedding at another token from the third turn on (not measured); one question per conversation, or a new conversation that starts with the earlier messages and the new question, avoids both (card L101).",
    "Conversation preface with tools and no messages: the template reads messages[0] without checking that the list is non-empty, so LiteRT-LM's render fails with 'Failed to apply template: undefined value' when the preface is prefilled at creation (prefillPrefaceOnInit, default false) and in the preface render (renderPrefaceIntoString and its C++ and C API forms); the same with extra context (such as enable_thinking) and no messages; read in the source of 0.17.0, 0.18.1 and main, measured with the 0.18.1 PyPI package through its C API on these files' template text; the default path, where the first message is rendered with the preface, is not affected; a system message or any initial message in the conversation config avoids it (card L141).",
    "Cache: without --cache no, the CLI can write a compiled cache file next to the model on its first run; without --backend, `run` uses the CPU (card L95)."
  ],
  "schema_version": "1.2"
}
