{
  "artifacts": [
    {
      "file": "Mage-VL-Cowork.litertlm",
      "sha256": "0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14",
      "size_mb": 3627.728
    }
  ],
  "benchmarks": [],
  "conversion": {
    "command": "microsoft/Mage-VL-Cowork rev 6fbe7f91 -> PY_PREP=... PY_EXPORT=... PY_SAMPLE=... LITERT_LM=... bash scripts/reproduce_vlm.sh mage-vl-cowork of hf-to-litertlm (the key that also builds the sibling file; run as written on 2026-11-04, both files section by section the same as the shipped bytes)",
    "quantization": "text decoder int4 (blockwise, 128 per block, OCTAV), one prefill signature of 1,024 tokens and the decode signature with a 4,096-token KV cache, no activation type declared = fp16 on the GPU through an exact rotary-position path; int8 embedding table outside the decoder; vision encoder and adapter fp32 weights, fp32 activations declared (card L30, L325, L326)",
    "tool": "litert-torch 0.9.3 (the checkpoint's Qwen3-4B text model as a standalone Qwen3ForCausalLM with the GPU's cache-update and attention composites; the vision encoder and 2 x 2 merger for one static picture) and litert-lm-builder 0.16.1 as a fast_vlm bundle; repro = hf-to-litertlm scripts/ship_mage_vl_cowork.sh (card L325-L328)",
    "tool_version": "litert-torch 0.9.3, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, ai-edge-litert 2.2.0, litert-lm-builder 0.16.1, transformers 5.14.1, torch 2.13.0, torchao 0.18.0 on Python 3.14.6 for the export; torch 2.12.1, transformers 5.14.1, tokenizers 0.22.2, safetensors 0.8.0 on Python 3.12.13 for the preparation; litert-lm 0.19.0 for the runs (recipe entry)"
  },
  "cross_runtime": [],
  "delegation": null,
  "device": {
    "records": [
      {
        "device": "iphone-18-pro",
        "run": {
          "accelerator": "cpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 10.5,
          "delegated_ops": null,
          "env": {
            "device": "iPhone 18 Pro (iPhone19,2)",
            "machine_label": "iphone-18-pro-magevl-benchmarkapp",
            "os_build": "iOS 27.2 (24B5099f)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "app: the benchmark app of edge-llm-bench (tree sha256 6ebb2361...) linking CLiteRTLM.xcframework.zip@v0.19.0 sha256:b7b2450fc084058adc5185498f8b7810a9297e054a10cf55818b62e1d5f8d65d, its Swift API on the phone (the embedded LC_UUID = the release's ios-arm64 slice), the increased-memory-limit entitlement (limit 6,144 MiB per process); the file side-loaded and its sha256 read on the phone (the artifact); sampling: the engine's default = top-k 1 (greedy)",
            "the one request, the decoder and the vision encoder on the CPU: ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in, a fresh launch each; rows: 018_crf16r_time_cpu_desktop_006_r1_t1_h2 (cache cold, thermal nominal -> fair, unplugged 0.4); every row's bundle_sha256 = the artifact; the cell = the mean over the one launch",
            "the app's timers: request = from creating the conversation to the end of the reply, so creating the Engine comes on top; ttft_ms = first chunk from the same start (it holds the vision stage); prefill and decode = the runtime's counters; peak_mem_mb = the app's lifetime peak phys_footprint in MiB",
            "four more launches on the battery went from thermal state nominal to serious, which the timing rule does not keep (76.34-84.88 s): no warm cell",
            "full_delegation null: the console prints no delegate split on this path; the file's fp32 / fp16 declarations as in `quantization`",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cool_wait_s_cold": 0.0,
            "decode_tokens_cold": 261.0,
            "decode_tokens_per_s_cold": 10.5,
            "engine_s_cold": 0.999,
            "first_chunk_s_cold": 60.814,
            "least_room_mib_cold": 2528.0,
            "peak_footprint_mib_cold": 3616.0,
            "prefill_tokens": 2988.0,
            "prefill_tokens_per_s_cold": 81.5,
            "processes_cold": 1.0,
            "processes_warm": 0.0,
            "request_s_cold": 85.357
          },
          "output_match": null,
          "peak_mem_mb": 3616.0,
          "prefill_tokens_per_s": 81.5,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 60814.0
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__iphone-18-pro.json"
      },
      {
        "device": "iphone-18-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-05",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "iPhone 18 Pro (iPhone19,2)",
            "machine_label": "iphone-18-pro-magevl-benchmarkapp",
            "os_build": "iOS 27.2 (24B5099f)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "app: the benchmark app of edge-llm-bench (tree sha256 6ebb2361...) linking CLiteRTLM.xcframework.zip@v0.19.0 sha256:b7b2450fc084058adc5185498f8b7810a9297e054a10cf55818b62e1d5f8d65d, its Swift API on the phone (the embedded LC_UUID = the release's ios-arm64 slice), the increased-memory-limit entitlement (limit 6,144 MiB per process); the file side-loaded and its sha256 read on the phone (the artifact); sampling: the engine's default = top-k 1 (greedy)",
            "the test: 120 ScreenSpot-v2 screenshots (40 desktop, 40 web, 40 mobile, a fixed shuffle), each stretched to 1920 x 1088 and sent as the vendor's GUI agent's first step (MAGE_GUI_SYSTEM_PROMPT, `Instruction: Click on the element for: <the instruction>`, `Previous actions: None`), one fresh process per screenshot",
            "rows: 2026-11-04 18:07 - 2026-11-05 00:04 JST (the 4 holds; the last ended 00:04:08); 120 rows, void 0, every row's bundle_sha256 = the artifact; 107 of the 120 started and ended at thermal state serious (88 charging, 32 on the battery), so the seconds in metrics are the phone's sustained state",
            "output_match = the line: a reply is right when its first tool call is a click whose point, converted to the screenshot's pixels, falls inside the target's box; per group the rate must reach the original model's rate in fp32 or bf16, whichever is lower, minus 5 points (the original in PyTorch on the Mac's GPU, MPS, transformers 4.57.3, greedy); and every reply's tool call parses as JSON, ends at <|im_end|> and has an Action: line after </think>; scores: desktop 60.0 % (bar 47.5), web 65.0 % (bar 57.5), mobile 60.0 % (bar 50.0), all 120 61.7 % against the original's 60.0 % in fp32 and 60.8 % in bf16; JSON 100.0 %, stop failures 0, form failures 0",
            "against the original on all 120 (right only in the original / right only in this form, exact McNemar p): fp32 9 / 11 (p 0.82), bf16 7 / 8 (p 1.0)",
            "the app's timers: request = from creating the conversation to the end of the reply, so creating the Engine comes on top; ttft_ms = first chunk from the same start (it holds the vision stage); prefill and decode = the runtime's counters; peak_mem_mb = the app's lifetime peak phys_footprint in MiB",
            "full_delegation null: the console prints no delegate split on this path"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "all_pct": 61.7,
            "all_right": 74.0,
            "decode_tokens_per_s_median": 16.34,
            "desktop_bar_pct": 47.5,
            "desktop_original_bf16_pct": 65.0,
            "desktop_original_fp32_pct": 52.5,
            "desktop_pct": 60.0,
            "desktop_right": 24.0,
            "form_failures": 0.0,
            "items": 120.0,
            "largest_position": 3356.0,
            "least_room_mib": 2022.0,
            "mobile_bar_pct": 50.0,
            "mobile_original_bf16_pct": 55.0,
            "mobile_original_fp32_pct": 62.5,
            "mobile_pct": 60.0,
            "mobile_right": 24.0,
            "n_gen_max": 365.0,
            "n_gen_p50": 261.0,
            "n_input_max": 2996.0,
            "n_input_min": 2986.0,
            "original_bf16_pct": 60.8,
            "original_fp32_pct": 60.0,
            "prefill_tokens_per_s_median": 188.6,
            "request_s_max": 84.258,
            "request_s_median": 58.73,
            "request_s_min": 25.24,
            "stop_failures": 0.0,
            "tool_call_json_pct": 100.0,
            "web_bar_pct": 57.5,
            "web_original_bf16_pct": 62.5,
            "web_original_fp32_pct": 65.0,
            "web_pct": 65.0,
            "web_right": 26.0
          },
          "output_match": true,
          "peak_mem_mb": 4379.0,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the GUI agent line (decoder and vision encoder on Metal): 120 ScreenSpot-v2 screenshots, one fresh process each, each click judged against the target's box; bar per group = the original model's lower rate (fp32 or bf16) minus 5 points",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__iphone-18-pro.json"
      },
      {
        "device": "iphone-18-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "iPhone 18 Pro (iPhone19,2)",
            "machine_label": "iphone-18-pro-magevl-benchmarkapp",
            "os_build": "iOS 27.2 (24B5099f)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "app: the benchmark app of edge-llm-bench (tree sha256 6ebb2361...) linking CLiteRTLM.xcframework.zip@v0.19.0 sha256:b7b2450fc084058adc5185498f8b7810a9297e054a10cf55818b62e1d5f8d65d, its Swift API on the phone (the embedded LC_UUID = the release's ios-arm64 slice), the increased-memory-limit entitlement (limit 6,144 MiB per process); the file side-loaded and its sha256 read on the phone (the artifact); sampling: the engine's default = top-k 1 (greedy)",
            "the test: the longest requests the work has, one process each: the vendor's hybrid system prompt as written with the picture and text of desktop_006 (3,186 tokens), and three requests with 40 earlier-step lines (desktop_006 and web_013 3,546 tokens, mobile_491 3,547), which pass fp16's exact-integer limit of 2,048 on the decoder's rotary path; output_match stays null: the clause asks that the whole prompt prefills, the reply ends at the stop token and its tool call parses, not that it matches a reference",
            "Consoles (hybrid_desktop_006: cold, thermal nominal -> fair, charging 0.2; long_desktop_006: warm, thermal fair -> fair, charging 0.2; long_web_013: warm, thermal fair -> fair, charging 0.2; long_mobile_491: warm, thermal fair -> fair, charging 0.2): not a speed figure (the card's Long requests)",
            "the app's timers: request = from creating the conversation to the end of the reply, so creating the Engine comes on top; ttft_ms = first chunk from the same start (it holds the vision stage); prefill and decode = the runtime's counters; peak_mem_mb = the app's lifetime peak phys_footprint in MiB",
            "full_delegation null: the console prints no delegate split on this path"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clicks_in_box_of_3": 1.0,
            "decode_tokens_per_s_p1": 23.29,
            "decode_tokens_per_s_p2": 19.26,
            "decode_tokens_per_s_p3": 19.32,
            "decode_tokens_per_s_p4": 19.7,
            "ended_at_stop_token": 4.0,
            "largest_position": 3832.0,
            "least_room_mib": 2004.0,
            "n_gen_p1": 121.0,
            "n_gen_p2": 239.0,
            "n_gen_p3": 286.0,
            "n_gen_p4": 236.0,
            "prefill_tokens_per_s_p1": 300.9,
            "prefill_tokens_per_s_p2": 199.2,
            "prefill_tokens_per_s_p3": 200.3,
            "prefill_tokens_per_s_p4": 213.0,
            "prompt_tokens_p1": 3186.0,
            "prompt_tokens_p2": 3546.0,
            "prompt_tokens_p3": 3546.0,
            "prompt_tokens_p4": 3547.0,
            "request_s_p1": 33.82,
            "request_s_p2": 58.118,
            "request_s_p3": 57.141,
            "request_s_p4": 49.893,
            "requests": 4.0,
            "tool_call_parses": 4.0,
            "whole_prompt_prefilled": 4.0
          },
          "output_match": null,
          "peak_mem_mb": 4140.0,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the long requests (decoder and vision encoder on Metal): the hybrid prompt as written (3,186 tokens) and three with 40 earlier-step lines (3,546-3,547 tokens)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__iphone-18-pro.json"
      },
      {
        "device": "iphone-18-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 35.26,
          "delegated_ops": null,
          "env": {
            "device": "iPhone 18 Pro (iPhone19,2)",
            "machine_label": "iphone-18-pro-magevl-benchmarkapp",
            "os_build": "iOS 27.2 (24B5099f)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": null,
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "app: the benchmark app of edge-llm-bench (tree sha256 6ebb2361...) linking CLiteRTLM.xcframework.zip@v0.19.0 sha256:b7b2450fc084058adc5185498f8b7810a9297e054a10cf55818b62e1d5f8d65d, its Swift API on the phone (the embedded LC_UUID = the release's ios-arm64 slice), the increased-memory-limit entitlement (limit 6,144 MiB per process); the file side-loaded and its sha256 read on the phone (the artifact); sampling: the engine's default = top-k 1 (greedy)",
            "the one request, the decoder and the vision encoder on Metal (ML Drift): ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in, a fresh launch each; rows: 015_crf16r_time_metal_desktop_006_r1_t1_h1 (cache warm, thermal nominal -> nominal, unplugged 0.45); 016_crf16r_time_metal_desktop_006_r2_t1_h1 (cache warm, thermal nominal -> nominal, unplugged 0.45); 017_crf16r_time_metal_desktop_006_r3_t1_h1 (cache warm, thermal nominal -> nominal, unplugged 0.45); 001_crf16r_fit_metal_desktop_006_r1_t1_h1 (cache cold, thermal nominal -> nominal, unplugged 0.55); every row's bundle_sha256 = the artifact; the cell = the mean over the warm launches",
            "the app's timers: request = from creating the conversation to the end of the reply, so creating the Engine comes on top; ttft_ms = first chunk from the same start (it holds the vision stage); prefill and decode = the runtime's counters; peak_mem_mb = the app's lifetime peak phys_footprint in MiB",
            "from a cool phone: the app waited for thermal state nominal before each warm launch (167-481 s); the cold launch had no such wait. All five launches of desktop_006 reply with one text. Not in the cell: the warm launch of the fit run with no cool wait (27.974 s, 002_crf16r_fit_metal_desktop_006_r2_t1_h1)",
            "full_delegation null: the console prints no delegate split on this path; the file's fp32 / fp16 declarations as in `quantization`",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "cool_wait_s_cold": 0.0,
            "cool_wait_s_p1": 481.0,
            "cool_wait_s_p2": 188.0,
            "cool_wait_s_p3": 167.0,
            "decode_tokens_cold": 242.0,
            "decode_tokens_p1": 242.0,
            "decode_tokens_p2": 242.0,
            "decode_tokens_p3": 242.0,
            "decode_tokens_per_s_cold": 34.24,
            "decode_tokens_per_s_p1": 35.21,
            "decode_tokens_per_s_p2": 35.31,
            "decode_tokens_per_s_p3": 35.25,
            "engine_s_cold": 2.181,
            "engine_s_p1": 0.976,
            "engine_s_p2": 0.961,
            "engine_s_p3": 0.955,
            "first_chunk_s_cold": 20.991,
            "first_chunk_s_p1": 19.432,
            "first_chunk_s_p2": 19.377,
            "first_chunk_s_p3": 19.274,
            "least_room_mib_cold": 2076.0,
            "least_room_mib_p1": 2362.0,
            "least_room_mib_p2": 2353.0,
            "least_room_mib_p3": 2355.0,
            "peak_footprint_mib_cold": 4321.0,
            "peak_footprint_mib_p1": 3784.0,
            "peak_footprint_mib_p2": 3791.0,
            "peak_footprint_mib_p3": 3791.0,
            "prefill_tokens": 2988.0,
            "prefill_tokens_per_s_cold": 512.4,
            "prefill_tokens_per_s_p1": 515.6,
            "prefill_tokens_per_s_p2": 512.8,
            "prefill_tokens_per_s_p3": 514.1,
            "processes_cold": 1.0,
            "processes_warm": 3.0,
            "request_s_cold": 27.88,
            "request_s_p1": 26.234,
            "request_s_p2": 26.164,
            "request_s_p3": 26.065,
            "request_s_warm_median": 26.16
          },
          "output_match": null,
          "peak_mem_mb": 4321.0,
          "prefill_tokens_per_s": 514.2,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 19361.0
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__iphone-18-pro.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: the PyPI litert-lm 0.19.0 wheel on Python 3.14.6 (GPU = WebGPU on Metal, adapter Apple M4 Max), sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy); one process per request",
            "Verdict: in the box 7/12 (bar >= 6), (b)-(d) failures 0, eos 12/12 -> PASS",
            "on the CPU the reply changes from process to process, so 7 is one draw; the 120 were run on the CPU only on an earlier build with a shorter KV cache (not a record of this file)",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "bar_in_box": 6.0,
            "ended_at_stop_token": 12.0,
            "form_failures": 0.0,
            "in_box": 7.0,
            "items": 12.0,
            "n_gen_max": 321.0,
            "n_gen_min": 242.0
          },
          "output_match": true,
          "peak_mem_mb": 3902.4,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "12 of the line's screenshots on the CPU (decoder and vision encoder, XNNPACK, 4 threads), one fresh process each; bar = 6 clicks in the box",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 2048,
          "date": "2026-11-04",
          "decode_tokens_per_s": 17.16,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: `litert-lm benchmark <file> -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --cpu-thread-count 4` of the PyPI litert-lm 0.19.0 wheel, the decoder alone (no vision executor), two passes; the cell = their mean",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6; full_delegation null: the benchmark logs state no split"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "decode_tokens_per_s_p1": 17.33,
            "decode_tokens_per_s_p2": 16.98,
            "init_time_s_p1": 2.4891,
            "init_time_s_p2": 2.5712,
            "max_rss_bytes_p1": 8754888704.0,
            "max_rss_bytes_p2": 8854634496.0,
            "peak_footprint_bytes_p1": 6589325352.0,
            "peak_footprint_bytes_p2": 6687744112.0,
            "prefill_tokens": 1024.0,
            "prefill_tokens_per_s_p1": 117.46,
            "prefill_tokens_per_s_p2": 118.14,
            "processes": 2.0,
            "ttft_s_p1": 8.7759,
            "ttft_s_p2": 8.727
          },
          "output_match": null,
          "peak_mem_mb": 6687.7,
          "prefill_tokens_per_s": 117.8,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 8751.5
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "cpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 19.71,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: the PyPI litert-lm 0.19.0 wheel on Python 3.14.6 (GPU = WebGPU on Metal, adapter Apple M4 Max), sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy); one process per request",
            "the one request, the decoder and the vision encoder on the CPU (the runtime's default thread count): ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in; rows; every row's bundle_sha256 = the artifact; the cell = the mean over the warm processes (a cache dir written by an earlier process; cold = it was empty); request_s = the process's wall time from start to exit",
            "ttft_ms = the runtime's time to first token, which does not hold the vision stage (first chunk less it, in metrics)",
            "on the CPU the reply changes from process to process",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6; full_delegation null: this path's stderr prints no delegate split",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "conversation_s_cold": 0.27,
            "conversation_s_p1": 0.04,
            "conversation_s_p2": 0.05,
            "decode_tokens_cold": 251.0,
            "decode_tokens_p1": 281.0,
            "decode_tokens_p2": 312.0,
            "decode_tokens_per_s_cold": 18.52,
            "decode_tokens_per_s_p1": 21.89,
            "decode_tokens_per_s_p2": 17.52,
            "engine_s_cold": 0.72,
            "engine_s_p1": 0.36,
            "engine_s_p2": 0.37,
            "first_chunk_s_cold": 43.266,
            "first_chunk_s_p1": 43.037,
            "first_chunk_s_p2": 43.85,
            "peak_footprint_bytes_cold": 3903840896.0,
            "peak_footprint_bytes_p1": 3611581544.0,
            "peak_footprint_bytes_p2": 3606879336.0,
            "prefill_tokens": 2988.0,
            "prefill_tokens_per_s_cold": 113.2,
            "prefill_tokens_per_s_p1": 110.7,
            "prefill_tokens_per_s_p2": 104.0,
            "processes_cold": 1.0,
            "processes_warm": 2.0,
            "request_s_cold": 57.9,
            "request_s_p1": 56.3,
            "request_s_p2": 62.0,
            "request_s_warm_mean": 59.15,
            "ttft_s_cold": 26.459,
            "ttft_s_p1": 27.045,
            "ttft_s_p2": 28.782
          },
          "output_match": null,
          "peak_mem_mb": 3903.8,
          "prefill_tokens_per_s": 107.3,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 27913.3
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: the PyPI litert-lm 0.19.0 wheel on Python 3.14.6 (GPU = WebGPU on Metal, adapter Apple M4 Max), sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy); one process per request",
            "the test: 120 ScreenSpot-v2 screenshots (40 desktop, 40 web, 40 mobile, a fixed shuffle), each stretched to 1920 x 1088 and sent as the vendor's GUI agent's first step (MAGE_GUI_SYSTEM_PROMPT, `Instruction: Click on the element for: <the instruction>`, `Previous actions: None`), one fresh process per screenshot",
            "Every row's bundle_sha256 = the artifact; outside a timing window, so no speed in metrics",
            "output_match = the line: a reply is right when its first tool call is a click whose point, converted to the screenshot's pixels, falls inside the target's box; per group the rate must reach the original model's rate in fp32 or bf16, whichever is lower, minus 5 points (the original in PyTorch on the Mac's GPU, MPS, transformers 4.57.3, greedy); and every reply's tool call parses as JSON, ends at <|im_end|> and has an Action: line after </think>; scores: desktop 55.0 % (bar 47.5), web 65.0 % (bar 57.5), mobile 55.0 % (bar 50.0), all 120 58.3 % against the original's 60.0 % in fp32 and 60.8 % in bf16; JSON 100.0 %, stop failures 0, form failures 0; mobile is 7.5 points under the original in fp32 and equal to it in bf16",
            "against the original on all 120 (right only in the original / right only in this form, exact McNemar p): fp32 12 / 10 (p 0.83), bf16 11 / 8 (p 0.65)",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6; full_delegation null: this path's stderr prints no delegate split"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "all_pct": 58.3,
            "all_right": 70.0,
            "desktop_bar_pct": 47.5,
            "desktop_original_bf16_pct": 65.0,
            "desktop_original_fp32_pct": 52.5,
            "desktop_pct": 55.0,
            "desktop_right": 22.0,
            "form_failures": 0.0,
            "items": 120.0,
            "largest_position": 3451.0,
            "mobile_bar_pct": 50.0,
            "mobile_original_bf16_pct": 55.0,
            "mobile_original_fp32_pct": 62.5,
            "mobile_pct": 55.0,
            "mobile_right": 22.0,
            "n_gen_max": 460.0,
            "n_gen_p50": 264.0,
            "n_input_max": 2996.0,
            "n_input_min": 2986.0,
            "original_bf16_pct": 60.8,
            "original_fp32_pct": 60.0,
            "stop_failures": 0.0,
            "tool_call_json_pct": 100.0,
            "web_bar_pct": 57.5,
            "web_original_bf16_pct": 62.5,
            "web_original_fp32_pct": 65.0,
            "web_pct": 65.0,
            "web_right": 26.0
          },
          "output_match": true,
          "peak_mem_mb": 5193.7,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the GUI agent line (decoder and vision encoder on the GPU): 120 ScreenSpot-v2 screenshots, one fresh process each, each click judged against the target's box; bar per group = the original model's lower rate (fp32 or bf16) minus 5 points",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: the PyPI litert-lm 0.19.0 wheel on Python 3.14.6 (GPU = WebGPU on Metal, adapter Apple M4 Max), sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy); one process per request",
            "the test: the longest requests the work has, one process each: the vendor's hybrid system prompt as written with the picture and text of desktop_006 (3,186 tokens), and three requests with 40 earlier-step lines (desktop_006 and web_013 3,546 tokens, mobile_491 3,547), which pass fp16's exact-integer limit of 2,048 on the decoder's rotary path; output_match stays null: the clause asks that the whole prompt prefills, the reply ends at the stop token and its tool call parses, not that it matches a reference",
            "Outside a timing window, so no speed in metrics",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clicks_in_box_of_3": 1.0,
            "ended_at_stop_token": 4.0,
            "largest_position": 3900.0,
            "n_gen_p1": 260.0,
            "n_gen_p2": 354.0,
            "n_gen_p3": 317.0,
            "n_gen_p4": 121.0,
            "prompt_tokens_p1": 3546.0,
            "prompt_tokens_p2": 3546.0,
            "prompt_tokens_p3": 3547.0,
            "prompt_tokens_p4": 3186.0,
            "requests": 4.0,
            "tool_call_parses": 4.0,
            "whole_prompt_prefilled": 4.0
          },
          "output_match": null,
          "peak_mem_mb": 5258.0,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the long requests (decoder and vision encoder on the GPU): the hybrid prompt as written (3,186 tokens) and three with 40 earlier-step lines (3,546-3,547 tokens)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 2048,
          "date": "2026-11-04",
          "decode_tokens_per_s": 94.52,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: `litert-lm benchmark <file> -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no` of the PyPI litert-lm 0.19.0 wheel, the decoder alone (no vision executor), two passes; the cell = their mean",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6; full_delegation null: the benchmark logs state no split"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "decode_tokens": 256.0,
            "decode_tokens_per_s_p1": 94.27,
            "decode_tokens_per_s_p2": 94.76,
            "init_time_s_p1": 2.3696,
            "init_time_s_p2": 2.3273,
            "max_rss_bytes_p1": 2585706496.0,
            "max_rss_bytes_p2": 2575564800.0,
            "peak_footprint_bytes_p1": 3945351928.0,
            "peak_footprint_bytes_p2": 3939437304.0,
            "prefill_tokens": 1024.0,
            "prefill_tokens_per_s_p1": 1356.62,
            "prefill_tokens_per_s_p2": 1348.2,
            "processes": 2.0,
            "ttft_s_p1": 0.7654,
            "ttft_s_p2": 0.7702
          },
          "output_match": null,
          "peak_mem_mb": 3945.4,
          "prefill_tokens_per_s": 1352.41,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 767.8
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "mac-studio-m4-max",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 89.24,
          "delegated_ops": null,
          "env": {
            "device": "Mac Studio (M4 Max, Mac16,9)",
            "machine_label": "mac-studio-m4-max",
            "os_build": "macOS 27.0 (26A428)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Apple M4 Max",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: the PyPI litert-lm 0.19.0 wheel on Python 3.14.6 (GPU = WebGPU on Metal, adapter Apple M4 Max), sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy); one process per request",
            "the one request, the decoder and the vision encoder on the GPU: ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in; rows; every row's bundle_sha256 = the artifact; the cell = the mean over the warm processes (a cache dir written by an earlier process; cold = it was empty); request_s = the process's wall time from start to exit",
            "ttft_ms = the runtime's time to first token, which does not hold the vision stage (first chunk less it, in metrics)",
            "the other heavy jobs were paused; another job's file copy ran during these rows; two earlier warm requests with another job's file copy over them took 11.9 s and 10.1 s (l) and are not in the cell",
            "peak_mem_mb = the highest /usr/bin/time -l peak memory footprint / 10^6; full_delegation null: this path's stderr prints no delegate split",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": null,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "conversation_s_cold": 1.7,
            "conversation_s_p1": 1.32,
            "conversation_s_p2": 1.06,
            "conversation_s_p3": 1.31,
            "conversation_s_p4": 1.31,
            "conversation_s_p5": 1.18,
            "decode_tokens_cold": 267.0,
            "decode_tokens_p1": 267.0,
            "decode_tokens_p2": 267.0,
            "decode_tokens_p3": 267.0,
            "decode_tokens_p4": 267.0,
            "decode_tokens_p5": 267.0,
            "decode_tokens_per_s_cold": 88.64,
            "decode_tokens_per_s_p1": 87.42,
            "decode_tokens_per_s_p2": 89.98,
            "decode_tokens_per_s_p3": 90.33,
            "decode_tokens_per_s_p4": 87.98,
            "decode_tokens_per_s_p5": 90.52,
            "engine_s_cold": 2.34,
            "engine_s_p1": 0.38,
            "engine_s_p2": 0.37,
            "engine_s_p3": 0.38,
            "engine_s_p4": 0.39,
            "engine_s_p5": 0.38,
            "first_chunk_s_cold": 5.459,
            "first_chunk_s_p1": 5.656,
            "first_chunk_s_p2": 5.456,
            "first_chunk_s_p3": 5.591,
            "first_chunk_s_p4": 5.638,
            "first_chunk_s_p5": 5.482,
            "peak_footprint_bytes_cold": 5198451656.0,
            "peak_footprint_bytes_p1": 4768926936.0,
            "peak_footprint_bytes_p2": 4767091928.0,
            "peak_footprint_bytes_p3": 4773399792.0,
            "peak_footprint_bytes_p4": 4776938712.0,
            "peak_footprint_bytes_p5": 4760390848.0,
            "prefill_tokens": 2988.0,
            "prefill_tokens_per_s_cold": 1244.3,
            "prefill_tokens_per_s_p1": 1210.0,
            "prefill_tokens_per_s_p2": 1252.7,
            "prefill_tokens_per_s_p3": 1198.8,
            "prefill_tokens_per_s_p4": 1211.8,
            "prefill_tokens_per_s_p5": 1236.7,
            "processes_cold": 1.0,
            "processes_warm": 5.0,
            "request_s_cold": 12.6,
            "request_s_p1": 10.5,
            "request_s_p2": 10.0,
            "request_s_p3": 10.4,
            "request_s_p4": 10.4,
            "request_s_p5": 10.1,
            "request_s_warm_mean": 10.28,
            "ttft_s_cold": 2.413,
            "ttft_s_p1": 2.481,
            "ttft_s_p2": 2.396,
            "ttft_s_p3": 2.504,
            "ttft_s_p4": 2.477,
            "ttft_s_p5": 2.427
          },
          "output_match": null,
          "peak_mem_mb": 5198.5,
          "prefill_tokens_per_s": 1222.0,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 2457.1
        },
        "source": "data/device_runs/0.19.0/2026-11-04/mage-vl-cowork__mac-studio-m4-max.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 7.2,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-magevl-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Tensor G6, Imagination PowerVR GPU",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: PyPI litert-lm-api 0.19.0 Android wheel (liblitert-lm.so sha256 28fa8433febc49a99f4bd3ef8cb82c2912457cfe274d97944debeabdeb97fc78) on CPython 3.14.8, the Python API on the phone: `Engine(model, backend=GPU(), vision_backend=CPU(), max_num_images=1, enable_benchmark=True); create_conversation(messages=[system]); send_message_async({'role': 'user', 'content': [image, text]}, max_output_tokens=1108)`; sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy)",
            "protocol: a benchmark script, one process per run, the phone's state and a ready check before each run, the process stopped by the memory guard on MemAvailable",
            "the one request, the decoder on the GPU and the vision encoder on the CPU (XNNPACK, 4 threads, vision_backend=CPU): ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in, a fresh process each (a new Engine); runs: CRf16r-gpu-gv-w1 (r8ka, 2026-11-04 16:40 JST, cache keep, thermal 0 before; clean); CRf16r-gpu-gv-w0 (r8ka, 2026-11-04 16:32 JST, cache cold, thermal 1 before; check run, not waited for clean (thermal 1, cap none, gpu_cooling 0 (budget 4614000 of 4614000))); every run's model_sha256 = the artifact; the cell = the mean over the warm processes (each in metrics: request_s = from before Engine() to the end of the reply; engine_only_s = Engine(); conversation_s = create_conversation())",
            "ttft_ms = the runtime's time to first token, which does not hold the vision stage (first chunk less it, in metrics)",
            "full_delegation false = the decoder's 2 signatures whole on LITERT_CL; the vision encoder XNNPACK 7,700 of 7,724 nodes, the adapter 20 of 26, the embedder 1 of 4, as the runtime log lists them",
            "the card names the GPU for this phone's vision: the vision stage took 29.28-35.25 s on the GPU and 151.99 s (warm) and 184.28 s (cold) here",
            "delegation (CRf16r-gpu-gv-w1): prefill_1024 1729 / 1729 on LITERT_CL (1 partition(s)); decode 1654 / 1654 on LITERT_CL (1 partition(s)); main 1 / 4 on TfLiteXNNPackDelegate (2 partition(s)); main 7700 / 7724 on TfLiteXNNPackDelegate (49 partition(s)); main 20 / 26 on TfLiteXNNPackDelegate (3 partition(s))",
            "peak_mem_mb = the highest VmHWM of these processes in MiB (phone-side samples, kB / 1024); it does not hold the GPU's memory",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "conversation_s_cold": 1.528,
            "conversation_s_p1": 0.267,
            "decode_tokens_cold": 220.0,
            "decode_tokens_p1": 217.0,
            "decode_tokens_per_s_cold": 8.5,
            "decode_tokens_per_s_p1": 7.2,
            "engine_only_s_cold": 13.826,
            "engine_only_s_p1": 6.574,
            "first_chunk_s_cold": 207.229,
            "first_chunk_s_p1": 174.998,
            "memavailable_min_kb": 4105188.0,
            "memavailable_min_kb_cold": 4105188.0,
            "memavailable_min_kb_p1": 4224696.0,
            "prefill_tokens_per_s_cold": 130.9,
            "prefill_tokens_per_s_p1": 130.7,
            "processes_cold": 1.0,
            "processes_warm": 1.0,
            "prompt_tokens": 2988.0,
            "request_s_cold": 248.02,
            "request_s_p1": 211.57,
            "request_s_warm_mean": 211.57,
            "skin_first_c_cold": 37.6,
            "skin_first_c_p1": 37.0,
            "thermal_first_cold": 1.0,
            "thermal_first_p1": 0.0,
            "ttft_s_cold": 22.945,
            "ttft_s_p1": 23.009,
            "vision_stage_s_cold": 184.28,
            "vision_stage_s_p1": 151.99,
            "vmhwm_kb_cold": 2506028.0,
            "vmhwm_kb_p1": 2410080.0
          },
          "output_match": null,
          "peak_mem_mb": 2447.3,
          "prefill_tokens_per_s": 130.7,
          "provenance": "measured",
          "runs": true,
          "signature": "one request with the vision encoder on the CPU (decoder on the GPU): desktop_006, 2,988 tokens in, a fresh process each",
          "total_ops": null,
          "ttft_ms": 23008.6
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-05",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-magevl-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Tensor G6, Imagination PowerVR GPU",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: PyPI litert-lm-api 0.19.0 Android wheel (liblitert-lm.so sha256 28fa8433febc49a99f4bd3ef8cb82c2912457cfe274d97944debeabdeb97fc78) on CPython 3.14.8, the Python API on the phone: `Engine(model, backend=GPU(), vision_backend=GPU(), max_num_images=1, enable_benchmark=True); create_conversation(messages=[system]); send_message_async({'role': 'user', 'content': [image, text]}, max_output_tokens=1108)`; sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy)",
            "protocol: a benchmark script, one process per run, the phone's state and a ready check before each run, the process stopped by the memory guard on MemAvailable",
            "the test: 120 ScreenSpot-v2 screenshots (40 desktop, 40 web, 40 mobile, a fixed shuffle), each stretched to 1920 x 1088 and sent as the vendor's GUI agent's first step (MAGE_GUI_SYSTEM_PROMPT, `Instruction: Click on the element for: <the instruction>`, `Previous actions: None`), one fresh process per screenshot",
            "runs (120: 2026-11-04 17:27-2026-11-05 01:13 JST, runs r8cDa, r8lDa, r8lDb, r8lDc, r8lDe, r8lDf, r8lDg, r8lDh, r8lDi, r8lDj, r8lDk); every row's bundle_sha256 = the artifact; no wait for a cool phone inside a hold, so the seconds in metrics are the phone's sustained state, not a cold start: 92 of the 120 started at thermal status 3 (request = from before Engine() to the end of the reply)",
            "output_match = the line: a reply is right when its first tool call is a click whose point, converted to the screenshot's pixels, falls inside the target's box; per group the rate must reach the original model's rate in fp32 or bf16, whichever is lower, minus 5 points (the original in PyTorch on the Mac's GPU, MPS, transformers 4.57.3, greedy); and every reply's tool call parses as JSON, ends at <|im_end|> and has an Action: line after </think>; scores: desktop 52.5 % (bar 47.5), web 60.0 % (bar 57.5), mobile 55.0 % (bar 50.0), all 120 55.8 % against the original's 60.0 % in fp32 and 60.8 % in bf16; JSON 100.0 %, stop failures 0, form failures 0",
            "against the original on all 120 (right only in the original / right only in this form, exact McNemar p): fp32 14 / 9 (p 0.4), bf16 14 / 8 (p 0.29)",
            "a hold's first process was cold, the others warm: 5 of those 10 cold rows right, 62 of the 110 warm ones (the card; ten rows are too few to read a difference)",
            "full_delegation false = the decoder's 2 signatures (LITERT_CL: prefill_1024 1,729 of 1,729 nodes, decode 1,654 of 1,654) and the vision encoder (LITERT_CL 7,724 of 7,724) whole on the GPU; the runtime log lists the embedder (XNNPACK 1 of 4 nodes) and the vision adapter (XNNPACK 20 of 26) on the CPU",
            "delegation (CRf16r-gpu-gg-desktop_006): prefill_1024 1729 / 1729 on LITERT_CL (1 partition(s)); decode 1654 / 1654 on LITERT_CL (1 partition(s)); main 1 / 4 on TfLiteXNNPackDelegate (2 partition(s)); main 7724 / 7724 on LITERT_CL (1 partition(s)); main 20 / 26 on TfLiteXNNPackDelegate (3 partition(s))",
            "peak_mem_mb = the highest VmHWM of these processes in MiB (phone-side samples, kB / 1024); it does not hold the GPU's memory"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "all_pct": 55.8,
            "all_right": 67.0,
            "decode_tokens_per_s_median": 7.01,
            "desktop_bar_pct": 47.5,
            "desktop_original_bf16_pct": 65.0,
            "desktop_original_fp32_pct": 52.5,
            "desktop_pct": 52.5,
            "desktop_right": 21.0,
            "form_failures": 0.0,
            "items": 120.0,
            "largest_position": 3357.0,
            "memavailable_min_kb": 2363588.0,
            "mobile_bar_pct": 50.0,
            "mobile_original_bf16_pct": 55.0,
            "mobile_original_fp32_pct": 62.5,
            "mobile_pct": 55.0,
            "mobile_right": 22.0,
            "n_gen_max": 366.0,
            "n_gen_p50": 266.0,
            "n_input_max": 2996.0,
            "n_input_min": 2986.0,
            "original_bf16_pct": 60.8,
            "original_fp32_pct": 60.0,
            "prefill_tokens_per_s_median": 100.5,
            "request_s_max": 166.0,
            "request_s_mean": 147.05,
            "request_s_median": 148.75,
            "request_s_min": 108.1,
            "stop_failures": 0.0,
            "tool_call_json_pct": 100.0,
            "web_bar_pct": 57.5,
            "web_original_bf16_pct": 62.5,
            "web_original_fp32_pct": 65.0,
            "web_pct": 60.0,
            "web_right": 24.0
          },
          "output_match": true,
          "peak_mem_mb": 1256.8,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the GUI agent line (decoder and vision encoder on the GPU): 120 ScreenSpot-v2 screenshots, one fresh process each, each click judged against the target's box; bar per group = the original model's lower rate (fp32 or bf16) minus 5 points",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": null,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-magevl-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Tensor G6, Imagination PowerVR GPU",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: PyPI litert-lm-api 0.19.0 Android wheel (liblitert-lm.so sha256 28fa8433febc49a99f4bd3ef8cb82c2912457cfe274d97944debeabdeb97fc78) on CPython 3.14.8, the Python API on the phone: `Engine(model, backend=GPU(), vision_backend=GPU(), max_num_images=1, enable_benchmark=True); create_conversation(messages=[system]); send_message_async({'role': 'user', 'content': [image, text]}, max_output_tokens=910)`; sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy)",
            "protocol: a benchmark script, one process per run, the phone's state and a ready check before each run, the process stopped by the memory guard on MemAvailable",
            "the test: the longest requests the work has, one process each: the vendor's hybrid system prompt as written with the picture and text of desktop_006 (3,186 tokens), and three requests with 40 earlier-step lines (desktop_006 and web_013 3,546 tokens, mobile_491 3,547), which pass fp16's exact-integer limit of 2,048 on the decoder's rotary path; output_match stays null: the clause asks that the whole prompt prefills, the reply ends at the stop token and its tool call parses, not that it matches a reference",
            "runs: CRf16r-gpu-gg-hybrid_desktop_006-w20 (r8lDe, 2026-11-04 21:06 JST, cache cold, thermal 0 before; clean); CRf16r-gpu-gg-long_desktop_006-w21 (r8lDe, 2026-11-04 21:08 JST, cache keep, thermal 1 before; check run, not waited for clean (thermal 1, cap thermal-uclamp-2=16 thermal-uclamp-6=19 thermal-uclamp-0=19 , gpu_cooling 1185000 (budget 3429000 of 4614000))); CRf16r-gpu-gg-long_web_013-w22 (r8lDe, 2026-11-04 21:11 JST, cache keep, thermal 2 before; check run, not waited for clean (thermal 2, cap thermal-uclamp-2=20 thermal-uclamp-6=20 thermal-uclamp-0=20 , gpu_cooling 0 (budget 4614000 of 4614000))); CRf16r-gpu-gg-long_mobile_491-w23 (r8lDe, 2026-11-04 21:14 JST, cache keep, thermal 2 before; check run, not waited for clean (thermal 2, cap thermal-uclamp-2=20 thermal-uclamp-6=20 thermal-uclamp-0=20 , gpu_cooling 0 (budget 4614000 of 4614000))); every run's model_sha256 = the artifact",
            "the first run cold, in the line's holds: not a speed figure (the card's Long requests)",
            "full_delegation false = the decoder's 2 signatures (LITERT_CL: prefill_1024 1,729 of 1,729 nodes, decode 1,654 of 1,654) and the vision encoder (LITERT_CL 7,724 of 7,724) whole on the GPU; the runtime log lists the embedder (XNNPACK 1 of 4 nodes) and the vision adapter (XNNPACK 20 of 26) on the CPU",
            "delegation (CRf16r-gpu-gg-hybrid_desktop_006-w20): prefill_1024 1729 / 1729 on LITERT_CL (1 partition(s)); decode 1654 / 1654 on LITERT_CL (1 partition(s)); main 1 / 4 on TfLiteXNNPackDelegate (2 partition(s)); main 7724 / 7724 on LITERT_CL (1 partition(s)); main 20 / 26 on TfLiteXNNPackDelegate (3 partition(s))",
            "peak_mem_mb = the highest VmHWM of these processes in MiB (phone-side samples, kB / 1024); it does not hold the GPU's memory"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "clicks_in_box_of_3": 1.0,
            "decode_tokens_per_s_p1": 6.79,
            "decode_tokens_per_s_p2": 6.79,
            "decode_tokens_per_s_p3": 7.29,
            "decode_tokens_per_s_p4": 7.71,
            "ended_at_stop_token": 4.0,
            "largest_position": 4027.0,
            "memavailable_min_kb": 2305828.0,
            "n_gen_p1": 123.0,
            "n_gen_p2": 236.0,
            "n_gen_p3": 310.0,
            "n_gen_p4": 480.0,
            "prefill_tokens_per_s_p1": 70.2,
            "prefill_tokens_per_s_p2": 87.1,
            "prefill_tokens_per_s_p3": 85.3,
            "prefill_tokens_per_s_p4": 83.2,
            "prompt_tokens_p1": 3186.0,
            "prompt_tokens_p2": 3546.0,
            "prompt_tokens_p3": 3546.0,
            "prompt_tokens_p4": 3547.0,
            "request_s_p1": 118.19,
            "request_s_p2": 148.94,
            "request_s_p3": 164.03,
            "request_s_p4": 190.79,
            "requests": 4.0,
            "tool_call_parses": 4.0,
            "whole_prompt_prefilled": 4.0
          },
          "output_match": null,
          "peak_mem_mb": 1257.2,
          "prefill_tokens_per_s": null,
          "provenance": "measured",
          "runs": true,
          "signature": "the long requests (decoder and vision encoder on the GPU): the hybrid prompt as written (3,186 tokens) and three with 40 earlier-step lines (3,546-3,547 tokens)",
          "total_ops": null,
          "ttft_ms": null
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__pixel-11-pro.json"
      },
      {
        "device": "pixel-11-pro",
        "run": {
          "accelerator": "gpu",
          "context_length": 4096,
          "date": "2026-11-04",
          "decode_tokens_per_s": 7.81,
          "delegated_ops": null,
          "env": {
            "device": "Pixel 11 Pro",
            "machine_label": "pixel-11-pro-magevl-devicebench",
            "os_build": "Android 17 (CD1A.260905.001.B1)",
            "runtime": "litert-lm",
            "runtime_version": "0.19.0",
            "soc": "Tensor G6, Imagination PowerVR GPU",
            "vendor_sdk": null
          },
          "error": null,
          "evidence": [
            "artifact: litert-community/Mage-VL-Cowork `Mage-VL-Cowork.litertlm` at main 38bf428b: LFS sha256 0d922aa53f65b40e5073e034f7c06c098389002ab4ad9c4f99a5f3654dd77b14, 3803948496 B (anonymous Hub API tree read 2026-11-05 03:21 JST)",
            "runtime: PyPI litert-lm-api 0.19.0 Android wheel (liblitert-lm.so sha256 28fa8433febc49a99f4bd3ef8cb82c2912457cfe274d97944debeabdeb97fc78) on CPython 3.14.8, the Python API on the phone: `Engine(model, backend=GPU(), vision_backend=GPU(), max_num_images=1, enable_benchmark=True); create_conversation(messages=[system]); send_message_async({'role': 'user', 'content': [image, text]}, max_output_tokens=1108)`; sampling: the bundle declares no sampler, so the runtime's CPU / GPU default applies (top-k 1 = greedy)",
            "protocol: a benchmark script, one process per run, the phone's state and a ready check before each run, the process stopped by the memory guard on MemAvailable",
            "the one request, the decoder and the vision encoder on the GPU (LITERT_CL, FP16 decoder, FP32 vision declared): ScreenSpot-v2 desktop_006 stretched to 1920 x 1088 + the vendor's GUI agent prompt = 2,988 tokens in, a fresh process each (a new Engine); runs: CRf16r-gpu-gg-w11 (r8tDa, 2026-11-04 18:16 JST, cache keep, thermal 0 before; clean); CRf16r-gpu-gg-w12 (r8tDa, 2026-11-04 18:25 JST, cache keep, thermal 0 before; clean); CRf16r-gpu-gg-w13 (r8tDa, 2026-11-04 18:34 JST, cache keep, thermal 0 before; clean); CRf16r-gpu-gg-w10 (r8tDa, 2026-11-04 18:07 JST, cache cold, thermal 1 before; check run, not waited for clean (thermal 1, cap none, gpu_cooling 0 (budget 4614000 of 4614000))); every run's model_sha256 = the artifact; the cell = the mean over the warm processes (each in metrics: request_s = from before Engine() to the end of the reply; engine_only_s = Engine(); conversation_s = create_conversation())",
            "ttft_ms = the runtime's time to first token, which does not hold the vision stage (first chunk less it, in metrics)",
            "full_delegation false = the decoder's 2 signatures (LITERT_CL: prefill_1024 1,729 of 1,729 nodes, decode 1,654 of 1,654) and the vision encoder (LITERT_CL 7,724 of 7,724) whole on the GPU; the runtime log lists the embedder (XNNPACK 1 of 4 nodes) and the vision adapter (XNNPACK 20 of 26) on the CPU",
            "each warm process waited 421-423 s and started at thermal status 0 (skin 34.5-35.8 C); the cold process opened the hold with no such wait, at status 1 and 37.4 C. The warm replies change from process to process (271, 229, 277 tokens here): desktop_006 gave one reply byte for byte on 3 cold processes and 4 different replies on 4 warm ones (the card), cause not read",
            "gpu_process_memory_mb = dumpsys gpu --gpumem after the reply on one cold process (run CRf16r-gpu-gg-w0, run r8kb), in MiB",
            "delegation (CRf16r-gpu-gg-w11): prefill_1024 1729 / 1729 on LITERT_CL (1 partition(s)); decode 1654 / 1654 on LITERT_CL (1 partition(s)); main 1 / 4 on TfLiteXNNPackDelegate (2 partition(s)); main 7724 / 7724 on LITERT_CL (1 partition(s)); main 20 / 26 on TfLiteXNNPackDelegate (3 partition(s))",
            "peak_mem_mb = the highest VmHWM of these processes in MiB (phone-side samples, kB / 1024); it does not hold the GPU's memory",
            "the card: https://huggingface.co/litert-community/Mage-VL-Cowork (Performance)"
          ],
          "failure_class": null,
          "full_delegation": false,
          "latency_p50_ms": null,
          "loads": true,
          "max_abs_diff": null,
          "max_rel_diff": null,
          "metrics": {
            "conversation_s_cold": 8.346,
            "conversation_s_p1": 5.446,
            "conversation_s_p2": 6.06,
            "conversation_s_p3": 5.146,
            "decode_tokens_cold": 206.0,
            "decode_tokens_p1": 271.0,
            "decode_tokens_p2": 229.0,
            "decode_tokens_p3": 277.0,
            "decode_tokens_per_s_cold": 7.1,
            "decode_tokens_per_s_p1": 6.94,
            "decode_tokens_per_s_p2": 8.52,
            "decode_tokens_per_s_p3": 7.96,
            "engine_only_s_cold": 14.472,
            "engine_only_s_p1": 8.949,
            "engine_only_s_p2": 14.872,
            "engine_only_s_p3": 15.418,
            "first_chunk_s_cold": 59.199,
            "first_chunk_s_p1": 56.877,
            "first_chunk_s_p2": 55.471,
            "first_chunk_s_p3": 53.518,
            "gpu_process_memory_mb": 6296.8,
            "memavailable_min_kb": 2393988.0,
            "memavailable_min_kb_cold": 2393988.0,
            "memavailable_min_kb_p1": 2583408.0,
            "memavailable_min_kb_p2": 2597728.0,
            "memavailable_min_kb_p3": 2580744.0,
            "prefill_tokens": 2988.0,
            "prefill_tokens_per_s_cold": 111.5,
            "prefill_tokens_per_s_p1": 108.8,
            "prefill_tokens_per_s_p2": 125.5,
            "prefill_tokens_per_s_p3": 126.4,
            "processes_cold": 1.0,
            "processes_warm": 3.0,
            "request_s_cold": 110.43,
            "request_s_p1": 109.75,
            "request_s_p2": 102.83,
            "request_s_p3": 108.34,
            "request_s_warm_mean": 106.97,
            "skin_first_c_cold": 37.4,
            "skin_first_c_p1": 35.8,
            "skin_first_c_p2": 35.0,
            "skin_first_c_p3": 34.5,
            "thermal_first_cold": 1.0,
            "thermal_first_p1": 0.0,
            "thermal_first_p2": 0.0,
            "thermal_first_p3": 0.0,
            "ttft_s_cold": 26.944,
            "ttft_s_p1": 27.601,
            "ttft_s_p2": 23.926,
            "ttft_s_p3": 23.765,
            "vision_stage_s_cold": 32.25,
            "vision_stage_s_p1": 29.28,
            "vision_stage_s_p2": 31.55,
            "vision_stage_s_p3": 29.75,
            "vmhwm_kb_cold": 1286340.0,
            "vmhwm_kb_p1": 1008508.0,
            "vmhwm_kb_p2": 1006028.0,
            "vmhwm_kb_p3": 1005776.0
          },
          "output_match": null,
          "peak_mem_mb": 1256.2,
          "prefill_tokens_per_s": 120.2,
          "provenance": "measured",
          "runs": true,
          "total_ops": null,
          "ttft_ms": 25097.2
        },
        "source": "data/device_runs/0.19.0/2026-11-05/mage-vl-cowork__pixel-11-pro.json"
      }
    ]
  },
  "model": {
    "family": "mage-vl",
    "id": "mage-vl-cowork",
    "license": "apache-2.0",
    "source_url": "https://huggingface.co/litert-community/Mage-VL-Cowork",
    "task": "image-text-to-text"
  },
  "pitfalls": [
    "One picture size: the bundle takes one screenshot per request, stretched to 1920 x 1088 (68 x 120 patches, 2,040 image tokens), portrait phone screens included. At other sizes the original model misplaces its clicks: a drawn button at 9 places was hit 9 of 9 times at 1920 x 1088 and at most 4 of 9 at six other sizes; on the 40 desktop screenshots, 1344 x 768 was right on 3 against 21 at 1920 x 1088 (card L26, L249).",
    "fp16 positions on the GPU: fp16 holds integers exactly only up to 2,048, and a request's positions go past it (to 3,451 on the 120 test screenshots, to 3,900 with 40 earlier-step lines). The graph computes its rotary angles from position = 2048·h + l with precomputed tables for the 2048·h part. On 12 test screenshots the fp16 decoder put the click in the target's box on 1 without that change (0 with the GPU composites added) and on 8 with it; the same decoder declared fp32 (before the composites) did on 7 (card L325).",
    "Vision activations on the GPU: this file declares fp32. With fp16 activations on the Mac GPU the encoder's output correlates at 0.599 with the CPU's fp32 output (card L326).",
    "Galaxy S26: not this file. Its vision encoder, run alone with a LiteRT 2.2.0 CompiledModel, compiles with all 7,724 nodes on the GPU, and about 8 s later the Adreno driver ends the process (`cl_a8x_cmdbuf_mgr_submit_ibs`, exit code 137) with 3.88 GB of memory still available; attention chunks of 1,020 rows instead of 255 end the same way. The whole file was not run in this form; the phone takes the sibling file (card L24, L317).",
    "NPU: the encoder compiled for SM8850 (QAIRT 2.47, O2) loads, and its first execution crashes the compute DSP (`SSR`) on both graphs tried. Neither file has an NPU form (card L318).",
    "Pixel 8a: neither file was run on it; two earlier builds with a shorter KV cache stopped while the engine was being created, at the test's guard on MemAvailable under 400,000 kB (card L319).",
    "Replies differ by device and, on the Pixel 11 Pro, by process: the iPhone and the Mac GPU run this file and their replies were equal on none of the 120 screenshots (the same click point on 28). On the Pixel 11 Pro desktop_006 gave one reply on 3 cold processes and 4 different replies on 4 warm ones; cause not read (card L248).",
    "One after another the phones take longer: the iPhone 26.16 s from a cool phone against a median of 58.73 s, the Pixel 11 Pro 106.97 s after a 7-minute rest against 148.8 s; 107 of the iPhone's 120 started and ended at thermal state serious, 92 of the Pixel 11 Pro's started at thermal status 3 (card Performance table, L298).",
    "The hybrid prompt's first reply is usually `screenshot` although the screenshot is in the request, so a step takes one or two inferences; the original model does the same on the two tasks it was run on. A new conversation holding the reply and the screenshot is the second request; prompt and reply together reached 3,737 of the 4,096 tokens (card L132-L136).",
    "Later turns: from the second turn on LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding (this bundle's decoder takes embeddings), read in the source of 0.19.0 and measured with 0.18.1 on other models, not on these files; a new conversation per inference avoids it (card L332).",
    "Sampling: the file carries no sampler settings, so LiteRT-LM 0.19.0 samples with top-k 1 on the CPU and the GPU; every result is greedy, while the model card's own setting is temperature 0.7 and top_p 0.8 (card L216).",
    "Reading the reply: the runtime returns the whole reply as text and no tools are passed to it; read the first <tool_call> yourself and leniently: with four earlier steps the original model wrote a typographic opening quote before `arguments` in 2 of 40 replies, none of the 120 test replies did (card L214).",
    "Memory: the iPhone's app has the increased-memory-limit entitlement (limit read 6,144 MiB; at least 2,004 MiB stayed free) and these files were not run without it. On the Pixel 11 Pro, a phone with 16 GB, MemAvailable fell by 7.23-7.47 GB in the warm and cold rows (card L299)."
  ],
  "schema_version": "1.2"
}
