{
  "artifacts": [
    {
      "file": "nemotron3_nano_30b_a3b_int4_ekv2048.litertlm",
      "sha256": "3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da",
      "size_mb": 17017.559
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "TODO (the card gives no conversion command); source nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16, quantized (card L13-L14, L27)",
    "quantization": "experts int4 in blocks of 128 values (64 in their down projection); the other weight matrices int4, except the token table and lm_head (int8); router fp32; fp32 activations declared; a 2,048-token context; 17.8 GB (card L35, L40)",
    "tool": "TODO (the card names no converter). The graph, as the card states it: each layer's experts are one `moe` custom op; relu² is written as its `gelu_tanh` (the activation was checked through the logits); each `moe` output has a second consumer (`moe * 0`) so ML Drift does not fuse the next op into it (card L40)",
    "tool_version": "TODO (the converter's version is not stated); runtime LiteRT-LM 0.19.1: the CPU through the PyPI package, the GPU through the v0.19.1 command-line tool built from source (7a03724c) (card L27, L76-L80)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-12",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel's Python API (tools/line3_rt.py; each record names litert-lm 0.19.1), CPU (XNNPACK) 8 threads, one Engine per process and one Conversation with the file's own template, SamplerConfig(top_k=1), ThinkingConfig(enable_thinking=False), max_output_tokens 2,048; the 3 turns in one conversation (the runtime keeps the history); the file's CPU runs used one shared engine cache directory (tools/line3_chain.sh)",
            "output_match = line 3 (a) and (b) as written before any output: (a) the file's verdict equals the reference's on 12 of 12 (at least 11 required; both answer 'violet' to the rhyme question, so 11 are right), (b) every question the reference ends with a stop token ends with one here (12 / 12). Reference = the float32 implementation of the model on the bf16 weights, thinking off, greedy, its replies built by draft-and-verify (every appended token its argmax), held to NVIDIA's modeling_nemotron_h.py by lines 0 and R",
            "the 3-turn conversation: 'Kenji', then '391', all three ending with a stop token; the reference's own turns: 'Kenji', then '401'. Line 3 (c) as written asks for 391 in turn 3 (met); line 3' (written after these outputs were seen: a turn the reference gets wrong is not required of the file) is met; in metrics, not part of output_match",
            "(cpu_off: 2026-11-12 04:23:01-04:24:43 JST (the CPU step's start to the GPU step's)): reply tokens up to 211; = MEASUREMENTS.md's Accuracy row for 'int4 ekv2048' and the Hub card's Accuracy table at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_right": 11.0,
            "conversation_c_as_written": 1.0,
            "conversation_c_line3_prime": 1.0,
            "conversation_turn2_name_right": 1.0,
            "conversation_turn3_answer": 391.0,
            "conversation_turns": 3.0,
            "conversation_turns_ended_with_stop_token": 3.0,
            "cpu_threads": 8.0,
            "ended_with_stop_token": 12.0,
            "max_new_tokens": 2048.0,
            "questions": 12.0,
            "reference_ended_with_stop_token": 12.0,
            "reference_turn3_answer": 401.0,
            "reply_tokens_max": 211.0,
            "verdicts_equal_reference": 12.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the 12 check questions of line 3 (the 8 short questions of hf-to-litertlm's scripts/verify_quality.py, a price word problem, a syllogism, an average speed, a question in Japanese) and its 3-turn conversation, thinking off (enable_thinking false), greedy (top-k 1), at most 2,048 new tokens, one process per question; verdicts against the source model's replies (float32 on the bf16 weights, thinking off)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": null,
          "date": "2026-11-12",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: the PyPI litert-lm 0.19.1 macOS arm64 wheel's Python API (tools/line3_rt.py; each record names litert-lm 0.19.1), CPU (XNNPACK) 8 threads, one Engine per process and one Conversation with the file's own template, SamplerConfig(top_k=1), ThinkingConfig(enable_thinking=True), max_output_tokens 2,048; the 3 turns in one conversation (the runtime keeps the history); the file's CPU runs used one shared engine cache directory (tools/line3_chain.sh)",
            "output_match = line 3's rule for thinking on, written before any output (the fallback taken on 2026-11-10 because the reference generator ran at 77.8 s per step, so no reference ran with thinking on): the line's regex right on at least 11 of the 12 and every reply ending with a stop token within 2,048 new tokens: 12 / 12 right, 12 / 12 stopped",
            "the 3-turn conversation ('My name is Kenji.' / 'What is my name? Answer with the name only.' / 'What is 17 times 23? Answer with the number only.'): 'Kenji', then '391', all three ending with a stop token = line 3 (c) as written; in metrics, not part of output_match",
            "(cpu_on: 2026-11-12 04:29:09-04:32:10 JST (the CPU step's start to the GPU step's)): reply tokens (the thought included) up to 75; = MEASUREMENTS.md's Accuracy row for 'int4 ekv2048' and the Hub card's Accuracy table at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_right": 12.0,
            "conversation_c_as_written": 1.0,
            "conversation_turn2_name_right": 1.0,
            "conversation_turn3_answer": 391.0,
            "conversation_turns": 3.0,
            "conversation_turns_ended_with_stop_token": 3.0,
            "cpu_threads": 8.0,
            "ended_with_stop_token": 12.0,
            "max_new_tokens": 2048.0,
            "questions": 12.0,
            "reply_tokens_max": 75.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the 12 check questions of line 3 (the 8 short questions of hf-to-litertlm's scripts/verify_quality.py, a price word problem, a syllogism, an average speed, a question in Japanese) and its 3-turn conversation, thinking on (the template's default), greedy (top-k 1), at most 2,048 new tokens, one process per question; verdicts by the line's regex on the text after the thought",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 2048,
          "date": "2026-11-12",
          "decode_tokens_per_s": 12.76,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (7a03724c; sha256 6f0eba84..., with the tag's prebuilt macos_arm64 libwebgpu_dawn.dylib and libGemmaModelConstraintProvider.dylib), one process at a time: `--backend=cpu --benchmark --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=2 --max_num_tokens=2048 --disable_cache=true --report_peak_memory_footprint --use_session=true --async=false --num_cpu_threads=8`; CPU = XNNPACK, number_of_threads 8 (each log prints it); the --use_session=true --async=false path gives one 1,024-token prefill per iteration (the CLI's conversation path prefilled the 1,024 tokens twice per iteration on this bundle, MEASUREMENTS.md); the logs print no partition or node count",
            "window W25 of 2026-11-12: this arm's processes ran 04:49:55-04:56:34 JST between the other arms' (one process at a time); every process's meta says 'overlap clean'; both exclusion lists of the window are empty; the cell (the benchmark rule): per process iteration 2 of 2 of one engine (iteration 1 = the warm-up), the cell = the median of the 3 processes, prefill and decode each; a process counts when it exits 0, its log has no Validation error, Shape mismatch or not-fully-delegated line, every iteration is one 1,024-token prefill turn and one 256-token decode turn, and the window's 1 s record shows no burst of other work at 300 % or more in its counted spans (every span ok); ttft_ms = the median of the processes' mean TTFT over the same iterations; peak_mem_mb = the largest of the processes' last 'Peak private footprint' print (--report_peak_memory_footprint; the block of the median of the iterations), the reading of the card and MEASUREMENTS.md, whose GB = MB x 2^20 / 10^9 (tools/speed_parse.py reads the print as MiB); each iteration's 'Peak system ram usage' print is in metrics",
            "context_length = the file's KV cache, 2,048 tokens (the card's Context column; the writer's --cache=2048); --max_num_tokens=2048 caps the run's tokens",
            "(started 2026-11-12 04:49:55 JST, CPU 80 % idle): 104.47 / 12.96 tok/s, TTFT 9.88 s; 108.15 / 12.05 tok/s, TTFT 9.55 s; Init Total 1227.98 ms; the last Peak private footprint print 4219.38 MB",
            "(started 2026-11-12 04:51:08 JST, CPU 77 % idle): 104.17 / 12.92 tok/s, TTFT 9.91 s; 107.21 / 12.96 tok/s, TTFT 9.63 s; Init Total 1239.62 ms; the last Peak private footprint print 4227.71 MB",
            "(started 2026-11-12 04:55:23 JST, CPU 84 % idle): 104.33 / 11.83 tok/s, TTFT 9.9 s; 107.3 / 12.76 tok/s, TTFT 9.62 s; Init Total 1221.86 ms; the last Peak private footprint print 4222.2 MB",
            "= MEASUREMENTS.md section 2026-11-12 ('| int4 ekv2048 | CPU (XNNPACK, 8 threads) | 107.30 tokens/s | 12.76 tokens/s | 3 | 4.43 GB |' and its per-process lists) and the Hub card's Performance row ('| int4 ekv2048 | CPU (8 threads) | 107 tokens/s | 12.8 tokens/s | 4.4 GB |') at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cpu_threads": 8.0,
            "decode_tokens": 256.0,
            "iterations": 2.0,
            "max_num_tokens": 2048.0,
            "peak_private_footprint_gb_max": 4.43,
            "prefill_tokens": 1024.0,
            "proc1_cpu_idle_at_start_pct": 80.0,
            "proc1_init_total_ms": 1227.98,
            "proc1_iter1_decode_tok_s": 12.96,
            "proc1_iter1_prefill_tok_s": 104.47,
            "proc1_iter1_ttft_s": 9.88,
            "proc1_iter2_decode_tok_s": 12.05,
            "proc1_iter2_prefill_tok_s": 108.15,
            "proc1_iter2_ttft_s": 9.55,
            "proc1_peak_private_footprint_mb": 4219.38,
            "proc1_peak_system_ram_mb_max": 4219.41,
            "proc2_cpu_idle_at_start_pct": 77.0,
            "proc2_init_total_ms": 1239.62,
            "proc2_iter1_decode_tok_s": 12.92,
            "proc2_iter1_prefill_tok_s": 104.17,
            "proc2_iter1_ttft_s": 9.91,
            "proc2_iter2_decode_tok_s": 12.96,
            "proc2_iter2_prefill_tok_s": 107.21,
            "proc2_iter2_ttft_s": 9.63,
            "proc2_peak_private_footprint_mb": 4227.71,
            "proc2_peak_system_ram_mb_max": 4227.78,
            "proc3_cpu_idle_at_start_pct": 84.0,
            "proc3_init_total_ms": 1221.86,
            "proc3_iter1_decode_tok_s": 11.83,
            "proc3_iter1_prefill_tok_s": 104.33,
            "proc3_iter1_ttft_s": 9.9,
            "proc3_iter2_decode_tok_s": 12.76,
            "proc3_iter2_prefill_tok_s": 107.3,
            "proc3_iter2_ttft_s": 9.62,
            "proc3_peak_private_footprint_mb": 4222.2,
            "proc3_peak_system_ram_mb_max": 4222.2,
            "processes": 3.0
          },
          "output_match": null,
          "peak_mem_mb": 4227.71,
          "prefill_tokens_per_s": 107.3,
          "provenance": "measured",
          "runs": true,
          "signature": null,
          "total_ops": null,
          "ttft_ms": 9620.0
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-12",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (7a03724c; sha256 6f0eba84...), WebGPU on Metal, one process per question: `--backend=gpu --max_output_tokens=2048 --min_log_severity=2 --sampler_top_k=1 --sampler_handles_input=false --disable_cache=true --use_session=true --async=false --input_prompt_file=<the vendor template's render with enable_thinking false>` (the CLI has no thinking switch; the render equals the file's template for these prompts, line T); the 3 turns = 3 processes, each render carrying the file's own earlier answers",
            "output_match = line 3 (a) and (b) as written before any output: (a) the file's verdict equals the reference's on 12 of 12 (at least 11 required; both answer 'violet' to the rhyme question, so 11 are right), (b) every question the reference ends with a stop token ends with one here (12 / 12). Reference = the float32 implementation of the model on the bf16 weights, thinking off, greedy, its replies built by draft-and-verify (every appended token its argmax), held to NVIDIA's modeling_nemotron_h.py by lines 0 and R",
            "the 3-turn conversation: 'Kenji', then '411', all three ending with a stop token; the reference's own turns: 'Kenji', then '401'. Line 3 (c) as written asks for 391 in turn 3 (not met); line 3' (written after these outputs were seen: a turn the reference gets wrong is not required of the file) is met; in metrics, not part of output_match",
            "(gpu_off: 2026-11-12 04:25:03-04:29:09 JST (each run's end time in the jsonl)): reply tokens up to 133; = MEASUREMENTS.md's Accuracy row for 'int4 ekv2048' and the Hub card's Accuracy table at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_right": 11.0,
            "conversation_c_as_written": 0.0,
            "conversation_c_line3_prime": 1.0,
            "conversation_turn2_name_right": 1.0,
            "conversation_turn3_answer": 411.0,
            "conversation_turns": 3.0,
            "conversation_turns_ended_with_stop_token": 3.0,
            "ended_with_stop_token": 12.0,
            "max_new_tokens": 2048.0,
            "questions": 12.0,
            "reference_ended_with_stop_token": 12.0,
            "reference_turn3_answer": 401.0,
            "reply_tokens_max": 133.0,
            "verdicts_equal_reference": 12.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the 12 check questions of line 3 (the 8 short questions of hf-to-litertlm's scripts/verify_quality.py, a price word problem, a syllogism, an average speed, a question in Japanese) and its 3-turn conversation, thinking off (enable_thinking false), greedy (top-k 1), at most 2,048 new tokens, one process per question; verdicts against the source model's replies (float32 on the bf16 weights, thinking off)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": null,
          "date": "2026-11-12",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (7a03724c; sha256 6f0eba84...), WebGPU on Metal, one process per question: `--backend=gpu --max_output_tokens=2048 --min_log_severity=2 --sampler_top_k=1 --sampler_handles_input=false --disable_cache=true --input_prompt=<question>` (the file's template, thinking on by default); the 3 turns through `--multi_turns` in one process",
            "output_match = line 3's rule for thinking on, written before any output (the fallback taken on 2026-11-10 because the reference generator ran at 77.8 s per step, so no reference ran with thinking on): the line's regex right on at least 11 of the 12 and every reply ending with a stop token within 2,048 new tokens: 12 / 12 right, 12 / 12 stopped",
            "the 3-turn conversation ('My name is Kenji.' / 'What is my name? Answer with the name only.' / 'What is 17 times 23? Answer with the number only.'): 'Kenji', then '391', all three ending with a stop token = line 3 (c) as written; in metrics, not part of output_match",
            "(gpu_on: 2026-11-12 04:32:28-04:44:40 JST (each run's end time in the jsonl)): reply tokens (the thought included) up to 77; = MEASUREMENTS.md's Accuracy row for 'int4 ekv2048' and the Hub card's Accuracy table at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "answers_right": 12.0,
            "conversation_c_as_written": 1.0,
            "conversation_turn2_name_right": 1.0,
            "conversation_turn3_answer": 391.0,
            "conversation_turns": 3.0,
            "conversation_turns_ended_with_stop_token": 3.0,
            "ended_with_stop_token": 12.0,
            "max_new_tokens": 2048.0,
            "questions": 12.0,
            "reply_tokens_max": 77.0
          },
          "output_match": true,
          "peak_mem_mb": null,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the 12 check questions of line 3 (the 8 short questions of hf-to-litertlm's scripts/verify_quality.py, a price word problem, a syllogism, an average speed, a question in Japanese) and its 3-turn conversation, thinking on (the template's default), greedy (top-k 1), at most 2,048 new tokens, one process per question; verdicts by the line's regex on the text after the thought",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 2048,
          "date": "2026-11-12",
          "decode_tokens_per_s": 75.22,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.1",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "bytes: every row ran (each run's meta, command or record names that path), written 2026-11-12 04:03:52 JST by the bundle step and not modified after (its mtime), hashed by the pre-upload (done 2026-11-12 06:05:02 JST, preupload_nemotron3_nano_30b_a3b_int4_ekv2048.litertlm.json: sha256 3045a02a7814e56580d8b9f10da8e62d2cd00d068915469804016c83f6a365da, 17844203904 B) = the Hub's LFS oid and size at main 5792549c (anonymous) and at commit 5792549c (the commit's readback)",
            "runtime: litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (7a03724c; sha256 6f0eba84..., with the tag's prebuilt macos_arm64 libwebgpu_dawn.dylib and libGemmaModelConstraintProvider.dylib), one process at a time: `--backend=gpu --benchmark --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=4 --max_num_tokens=2048 --disable_cache=true --report_peak_memory_footprint --sampler_handles_input=false --use_session=true --async=false`; GPU = WebGPU on Metal (each log selects 'Apple M4 Max, arch=metal-3, vendor=apple, backend=Metal'); the --use_session=true --async=false path gives one 1,024-token prefill per iteration (the CLI's conversation path prefilled the 1,024 tokens twice per iteration on this bundle, MEASUREMENTS.md); the logs print no partition or node count",
            "window W24 of 2026-11-12: this arm's processes ran 04:36:06-04:42:24 JST between the other arms' (one process at a time); every process's meta says 'overlap clean'; both exclusion lists of the window are empty; the cell (the benchmark rule): per process the mean of iterations 2-4 of one engine (iteration 1 = the warm-up), the cell = the median of the 5 processes, prefill and decode each; a process counts when it exits 0, its log has no Validation error, Shape mismatch or not-fully-delegated line, every iteration is one 1,024-token prefill turn and one 256-token decode turn, and the window's 1 s record shows no burst of other work at 300 % or more in its counted spans (every span ok); ttft_ms = the median of the processes' mean TTFT over the same iterations; peak_mem_mb = the largest of the processes' last 'Peak private footprint' print (--report_peak_memory_footprint; the block of the median of the iterations), the reading of the card and MEASUREMENTS.md, whose GB = MB x 2^20 / 10^9 (tools/speed_parse.py reads the print as MiB); each iteration's 'Peak system ram usage' print is in metrics",
            "context_length = the file's KV cache, 2,048 tokens (the card's Context column; the writer's --cache=2048); --max_num_tokens=2048 caps the run's tokens",
            "(started 2026-11-12 04:36:06 JST, CPU 70 % idle): 831.07 / 74.66 tok/s, TTFT 1.25 s; 851.51 / 75.31 tok/s, TTFT 1.22 s; 850.7 / 75.46 tok/s, TTFT 1.22 s; 849.95 / 74.59 tok/s, TTFT 1.22 s; Init Total 9088.69 ms; the last Peak private footprint print 18874.19 MB",
            "(started 2026-11-12 04:36:44 JST, CPU 76 % idle): 831.39 / 75.53 tok/s, TTFT 1.24 s; 851.86 / 75.34 tok/s, TTFT 1.22 s; 852.57 / 75.29 tok/s, TTFT 1.21 s; 851.94 / 75.03 tok/s, TTFT 1.22 s; Init Total 8716.02 ms; the last Peak private footprint print 18868.47 MB",
            "(started 2026-11-12 04:39:18 JST, CPU 80 % idle): 835.56 / 75.15 tok/s, TTFT 1.24 s; 851.99 / 75.34 tok/s, TTFT 1.22 s; 851.33 / 75.07 tok/s, TTFT 1.22 s; 855.6 / 74.7 tok/s, TTFT 1.21 s; Init Total 8618.6 ms; the last Peak private footprint print 18865.8 MB",
            "(started 2026-11-12 04:39:55 JST, CPU 79 % idle): 836.61 / 74.94 tok/s, TTFT 1.24 s; 851.43 / 75.56 tok/s, TTFT 1.22 s; 853.24 / 75.38 tok/s, TTFT 1.21 s; 850.58 / 74.99 tok/s, TTFT 1.22 s; Init Total 8416.66 ms; the last Peak private footprint print 18852.37 MB",
            "(started 2026-11-12 04:41:48 JST, CPU 84 % idle): 838.41 / 75.49 tok/s, TTFT 1.23 s; 852.69 / 75.62 tok/s, TTFT 1.21 s; 851.1 / 75.34 tok/s, TTFT 1.22 s; 853.32 / 75.51 tok/s, TTFT 1.21 s; Init Total 8414.5 ms; the last Peak private footprint print 18861.81 MB",
            "= MEASUREMENTS.md section 2026-11-12 ('| int4 ekv2048 | GPU (WebGPU) | 852.12 tokens/s | 75.22 tokens/s | 5 | 19.79 GB |' and its per-process lists) and the Hub card's Performance row ('| int4 ekv2048 | GPU (WebGPU) | 852 tokens/s | 75.2 tokens/s | 19.8 GB |') at main 5792549c"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "iterations": 4.0,
            "max_num_tokens": 2048.0,
            "peak_private_footprint_gb_max": 19.79,
            "prefill_tokens": 1024.0,
            "proc1_cpu_idle_at_start_pct": 70.0,
            "proc1_init_total_ms": 9088.69,
            "proc1_iter1_decode_tok_s": 74.66,
            "proc1_iter1_prefill_tok_s": 831.07,
            "proc1_iter1_ttft_s": 1.25,
            "proc1_iter2_decode_tok_s": 75.31,
            "proc1_iter2_prefill_tok_s": 851.51,
            "proc1_iter2_ttft_s": 1.22,
            "proc1_iter3_decode_tok_s": 75.46,
            "proc1_iter3_prefill_tok_s": 850.7,
            "proc1_iter3_ttft_s": 1.22,
            "proc1_iter4_decode_tok_s": 74.59,
            "proc1_iter4_prefill_tok_s": 849.95,
            "proc1_iter4_ttft_s": 1.22,
            "proc1_peak_private_footprint_mb": 18874.19,
            "proc1_peak_system_ram_mb_max": 18885.8,
            "proc2_cpu_idle_at_start_pct": 76.0,
            "proc2_init_total_ms": 8716.02,
            "proc2_iter1_decode_tok_s": 75.53,
            "proc2_iter1_prefill_tok_s": 831.39,
            "proc2_iter1_ttft_s": 1.24,
            "proc2_iter2_decode_tok_s": 75.34,
            "proc2_iter2_prefill_tok_s": 851.86,
            "proc2_iter2_ttft_s": 1.22,
            "proc2_iter3_decode_tok_s": 75.29,
            "proc2_iter3_prefill_tok_s": 852.57,
            "proc2_iter3_ttft_s": 1.21,
            "proc2_iter4_decode_tok_s": 75.03,
            "proc2_iter4_prefill_tok_s": 851.94,
            "proc2_iter4_ttft_s": 1.22,
            "proc2_peak_private_footprint_mb": 18868.47,
            "proc2_peak_system_ram_mb_max": 18882.0,
            "proc3_cpu_idle_at_start_pct": 80.0,
            "proc3_init_total_ms": 8618.6,
            "proc3_iter1_decode_tok_s": 75.15,
            "proc3_iter1_prefill_tok_s": 835.56,
            "proc3_iter1_ttft_s": 1.24,
            "proc3_iter2_decode_tok_s": 75.34,
            "proc3_iter2_prefill_tok_s": 851.99,
            "proc3_iter2_ttft_s": 1.22,
            "proc3_iter3_decode_tok_s": 75.07,
            "proc3_iter3_prefill_tok_s": 851.33,
            "proc3_iter3_ttft_s": 1.22,
            "proc3_iter4_decode_tok_s": 74.7,
            "proc3_iter4_prefill_tok_s": 855.6,
            "proc3_iter4_ttft_s": 1.21,
            "proc3_peak_private_footprint_mb": 18865.8,
            "proc3_peak_system_ram_mb_max": 18874.7,
            "proc4_cpu_idle_at_start_pct": 79.0,
            "proc4_init_total_ms": 8416.66,
            "proc4_iter1_decode_tok_s": 74.94,
            "proc4_iter1_prefill_tok_s": 836.61,
            "proc4_iter1_ttft_s": 1.24,
            "proc4_iter2_decode_tok_s": 75.56,
            "proc4_iter2_prefill_tok_s": 851.43,
            "proc4_iter2_ttft_s": 1.22,
            "proc4_iter3_decode_tok_s": 75.38,
            "proc4_iter3_prefill_tok_s": 853.24,
            "proc4_iter3_ttft_s": 1.21,
            "proc4_iter4_decode_tok_s": 74.99,
            "proc4_iter4_prefill_tok_s": 850.58,
            "proc4_iter4_ttft_s": 1.22,
            "proc4_peak_private_footprint_mb": 18852.37,
            "proc4_peak_system_ram_mb_max": 18854.7,
            "proc5_cpu_idle_at_start_pct": 84.0,
            "proc5_init_total_ms": 8414.5,
            "proc5_iter1_decode_tok_s": 75.49,
            "proc5_iter1_prefill_tok_s": 838.41,
            "proc5_iter1_ttft_s": 1.23,
            "proc5_iter2_decode_tok_s": 75.62,
            "proc5_iter2_prefill_tok_s": 852.69,
            "proc5_iter2_ttft_s": 1.21,
            "proc5_iter3_decode_tok_s": 75.34,
            "proc5_iter3_prefill_tok_s": 851.1,
            "proc5_iter3_ttft_s": 1.22,
            "proc5_iter4_decode_tok_s": 75.51,
            "proc5_iter4_prefill_tok_s": 853.32,
            "proc5_iter4_ttft_s": 1.21,
            "proc5_peak_private_footprint_mb": 18861.81,
            "proc5_peak_system_ram_mb_max": 18864.8,
            "processes": 5.0
          },
          "output_match": null,
          "peak_mem_mb": 18874.19,
          "prefill_tokens_per_s": 852.123,
          "provenance": "measured",
          "runs": true,
          "signature": null,
          "total_ops": null,
          "ttft_ms": 1216.7
        },
        "source": "data/device_runs/0.19.1/2026-11-12/nemotron-3-nano-30b-a3b-int4-ekv2048__mac-studio-m4-max.json"
      }
    ]
  },
  "model": {
    "family": "nemotron-3-nano",
    "id": "nemotron-3-nano-30b-a3b-int4-ekv2048",
    "license": "other (nvidia-nemotron-open-model-license)",
    "source_url": "https://huggingface.co/litert-community/Nemotron-3-Nano-30B-A3B",
    "task": "text-generation"
  },
  "pitfalls": [
    "Keep the GPU cache off (cache_dir=\":nocache\" in Python, --disable_cache=true in the command-line tool): on a Mac, runs that loaded an earlier run's GPU program cache answered wrongly (card L68).",
    "One Engine per conversation: the Mamba-2 state is not reset between conversations (LiteRT-LM #3165) (card L69).",
    "Tool calls come back as text (the model's <tool_call> form) (card L70).",
    "Long replies: a blocking send_message still generating after 10 minutes fails with DEADLINE_EXCEEDED; use send_message_async or cap max_output_tokens (card L71).",
    "After a message overflows the context window, every later message in that conversation fails (Task failed with state: 7); start a new conversation (card L72).",
    "GPU: the card runs it with the v0.19.1 command-line tool built from source, which has --sampler_handles_input=false (the PyPI package does not); LiteRT-LM 0.18.0's GPU moe op only supports activation 'gelu'. CPU: the PyPI package, litert-lm 0.19.1 (card L27, L76-L93).",
    "Thinking is on by default; thinking_config=litert_lm.ThinkingConfig(enable_thinking=False) in create_conversation turns it off (card L29, L105).",
    "The card's checks (12 questions and a three-turn chat, a name then 17 × 23, greedy, against NVIDIA's bf16 weights): with thinking on, 12/12 right and 391 for 17 × 23 on all four files on the CPU and the GPU (the source model was not run with thinking on); with thinking off, 11/12 right, as the source model, and 17 × 23 comes back as 391 on the CPU and 411 on the GPU for the int4 files (this one among them), 401 for the mixed_int4 files and for the source model (card L59-L64).",
    "Which file: this one gives faster decode than mixed_int4 ekv2048 (card L35). On an Apple M4 Max (128 GB) its benchmark peaked at 19.8 GB on the GPU and 4.4 GB on the CPU (2026-11-12); speed still being tuned (updates follow) (card L27, L46-L47, L55)."
  ],
  "schema_version": "1.2"
}
