{"what":"Kev-4B: int8 measured, not shipped. int8lin fails the bar; the quantizer's options (r6_quant_api); the fp32 instrument and its rule (bisect_4b/rule.json, written before any result); the rule's candidates on the Mac GPU (asymmetric int8 with in_proj_qkv / out_proj (/ v_proj) fp16: the 434 rows pass, the held-out set fails on one score row); what the compiled asset holds (expect-frequent-reshapes adds an fp16 copy of every linear; the specialized graph holds int8 linears as fp16); the iOS h19p bytes; resident memory samples. Every int8 graph here is the unrolled S = 16 form (the first upload's graph); the fp32 Metal-kernel forms were not measured in int8. Process ids removed in round 17 (no number or result changed).","sources":[{"file":"results/export_int8lin_pf16_4b.json","bytes":8725,"sha256":"7dc1bf0290a86936a9dc9ed1480a7733981fe5eaf1a0f55984c2c6d029943835"},{"file":"results/readout_int8lin_pf16_4b.json","bytes":752885,"sha256":"d430a661d8dbeec4f0a81466b3049ebe4ab5594f1f5d1bb7e781f9fc0c92dbe5"},{"file":"results/readout_int8lin_pf16_4b_red_v2.json","bytes":25694,"sha256":"f0ed7aee1a33c1ad53276a1f20d7fd20c60ad962c84d242c4ba2c5ade7c3a05c"},{"file":"results/r4_summary_4b.json","bytes":24694,"sha256":"68c43b4ba934f5ebda58d29427610485f325b88912a25253cf7c9c5ac0956fea"},{"file":"results/r6_quant_api.json","bytes":31040,"sha256":"5d659cb73a2e8df3dfe21625369a73fda737b07edb86d7e9ba6c425e03a868ff"},{"file":"results/r6_static_toy.json","bytes":10898,"sha256":"7c1ec25c651c5e840c5b8fa9cc22c23e553cf8556b9182f7933454f68c2529c7"},{"file":"bisect_4b/rule.json","bytes":19902,"sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd"},{"file":"results/r6_int8_4b_bisect.json","bytes":678456,"sha256":"53a091afbe3720b17526ada79b7f71d1b5b8262943bbefe1b1b4fbbf3a1c53be"},{"file":"results/export_int8mix_aff32_k_qkv_out_v_pf16_4b.json","bytes":15197,"sha256":"e2587a50358c0405acb95283f5e97e22df0698e8b4e3e1dd4b13c5cbf26f6cc5"},{"file":"results/readout_int8mix_aff32_k_qkv_out_v_pf16_4b.json","bytes":750745,"sha256":"1ff62f8c264c14d8bcd1d70e9226ccd8a029296f862ff0e642fd99c7a23b98dc"},{"file":"results/readout_int8mix_aff32_k_qkv_out_v_pf16_4b_red_v2.json","bytes":20671,"sha256":"446d85404d8aa1110090752341d4cb49e92e4a5ca6aca05332fe1f0c987dd560"},{"file":"results/export_int8mix_aff32_k_qkv_out_pf16_4b.json","bytes":15115,"sha256":"a03ef9f89ad234d1fe3d6c417373bc9620c3b4f17caebb3d65552bc14318bc2e"},{"file":"results/readout_int8mix_aff32_k_qkv_out_pf16_4b.json","bytes":752779,"sha256":"49cbc8f9bd581c1371e5fde722c826ad9e947b8aa34b57a9b3f8f1ac973f86c7"},{"file":"results/export_int8mix_aff32_g0_k_out_pf16_4b.json","bytes":14058,"sha256":"48d05c80f218b7ceba5aa85d60503c621910def237d194cf768a70e1f77fd0c0"},{"file":"results/readout_int8mix_aff32_g0_k_out_pf16_4b.json","bytes":749830,"sha256":"3466d71c6ea5f4af07971432d8d546eb1f2ae67d26c9eeacb20bdf06b5d1c8a2"},{"file":"results/readout_int8mix_aff32_g0_k_out_pf16_4b_red_v2.json","bytes":19145,"sha256":"4a2a15ffe799058144ac1af603fed83a67bebf116676abadd38f09a6414c5227"},{"file":"results/export_int8mix_aff32_k_qkv_out_v_e_pf16_4b.json","bytes":12948,"sha256":"05846ea2489e7ea419e0fe29d8d395ed17524e872825017e5cc7799a24f23028"},{"file":"results/export_int8mix_aff32_k_qkv_out_e_pf16_4b.json","bytes":12548,"sha256":"f01935224655bf3d12bf85fc6e8a597e63ad0ef59e1cf4def9318def1c6da780"},{"file":"results/r6_int8_4b_summary.json","bytes":47678,"sha256":"9e36b238eb092189b52da87f9f1350d3a5ecd266d351c64d2652c24f9e44c95f"},{"file":"results/r6_ios_aot_bytes.json","bytes":21276,"sha256":"512aee95bf560c3c61ff6e8c299d5732195a6f251464bd81d2c8d38482a78a84"},{"file":"results/r6_abba_efr_summary.json","bytes":11657,"sha256":"d30edb4917b13610ac8c97dec9f01bca702e76cc785887ea33d5d98e94b6610c"},{"file":"results/r6_readout_fp16_pf16_4b_noefr33.json","bytes":51259,"sha256":"a65968541bee350c130ca5f073c29d51e82ecf488bcb454bc1088f6ec3d776e4"},{"file":"results/r6_mem_aff32_g0_k_out_gate434.jsonl","bytes":27714,"sha256":"5a5d323e901b3507582408a2c0ec0b523044191ff75e7828f63576eedf245120"},{"file":"results/r6_mem_aff32_g0_k_out_heldout.jsonl","bytes":5131,"sha256":"5b8361d6d583a466f11d68cdd0b5bd40f8e818f297357161862f14dcf25f273b"},{"file":"results/r6_mem_aff32_g0_k_out_noefr33.jsonl","bytes":71591,"sha256":"907382b9b03782067dc7062bbf045318d8c99e580f6b56fc9e3fe30c85fd973a"},{"file":"results/r6_mem_aff32_g0_k_out_red.jsonl","bytes":5952,"sha256":"fdb183ddd04e81999ef76513af3fa8b317ba6af0402fbed93f52aca52a74ffd9"},{"file":"results/r6_mem_aff32_k_qkv_out_gate434.jsonl","bytes":71155,"sha256":"346f47cc28767941e12802bf7333940de011173648344faf6722cd3c9f0f2fd3"},{"file":"results/r6_mem_aff32_k_qkv_out_v_gate434.jsonl","bytes":30355,"sha256":"a9e015d282ee879f384497dd5ef3d44362d14f3edd11b525c3f0d1c6958e20b2"},{"file":"results/r6_mem_aff32_k_qkv_out_v_heldout.jsonl","bytes":7357,"sha256":"bd004033b5faaa19137b03769a96be665456bd6e4be48e9de6a3d4cbc0e28d30"},{"file":"results/r6_mem_aff32_k_qkv_out_v_red.jsonl","bytes":4099,"sha256":"f57d23892ce87886da8ba458d7a0bb5dcbb02f24025c58edb629e99dbe64a442"},{"file":"results/r6_mem_fp16_4b_efr_leg4.jsonl","bytes":19853,"sha256":"9732e6961ef28f3f4baa13def459cf75471d91ef1a65dc47457340eae8612173"},{"file":"results/r6_mem_r6_noefr33.jsonl","bytes":1192,"sha256":"7d726a2c2c65af1c6c34d7c6462c400089528e709cd84272e9940d3e4d92bfa9"}],"trimmed":{"removed":["candidates.*.readout and noefr33_readout: runs[] and processes[] (each keeps summary, by_source, near_ties, checks, timing)","bisect.configs.*.rows (per-row values of each config; each keeps worst / mean / argmax)","resident_memory: the jsonl samples (each file keeps n / min / max of every numeric field)","runs[] of the non-shipped readouts keep id, q, tokens, calls, finite, source, type, argmax, argmax_oracle, argmax_equal, max / mean |dp|, oracle top-2 margin, near_tie, probs and the hidden rows' min_pos_cos / positions_below (dropped: per-run seconds, shard and index, padded_tokens, max |dlogit|, the hidden-row difference details) (readout_int8lin)","runs[].call_ms and timing.first_call_of_process_ms (per-call ms; the timing block keeps the medians)","runs[].npz / npz_key, processes[].npz / npz_sha256, oracle.npz_sha256 (lane file names and hashes of hidden dumps)","runs[].hidden.lowest_positions (each run keeps min_pos_cos, its index and the position counts)","runs[].probs_oracle (the oracle's probabilities are in the fixture file)","processes[].descriptor (identical to the top-level descriptor: descriptor_same_in_every_process)","other_gpu_processes (command lines of other sessions' processes)","red arm items' instr_old / instr_new (transfer-v4 question text; each item keeps its row id and numbers)","every pid key (process ids), and process ids inside text (lock contents, scratch directory names) rewritten as <n> / <pid>"],"serialization":"compact json (no indentation)","full_copies":"the source files above, in the lane directory $ZOO_WORK_ROOT/_kev/ (not in the repository)","paths":"local paths rewritten: the lane directory as $ZOO_WORK_ROOT/_kev, the repository relative, the home directory as ~, temporary directories as $TMPDIR, Xcode as $DEVELOPER_DIR, the phone's app container as <app container>"},"export_int8lin":{"mode":"int8lin","model":"kev-4b","name":"kev_4b_decode_int8lin_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8lin","--model","kev-4b","--prefill-chunk","16","--aot","--record","$ZOO_WORK_ROOT/_kev/results/export_int8lin_pf16_4b.json"],"started":"2026-10-03T20:03:11+09:00","script_sha256":"fce6f0386fbbcfceae4e875afb8a3110607ab74d3cab7068c55646516393e9be","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8lin_pf16","name":"kev_4b_decode_int8lin_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8lin_pf16/kev_4b_decode_int8lin_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"fp16_layers":[],"fp16_layer_patterns":[],"int8_linear_modules":248,"fp16_linear_modules":[],"int8_params":3569090560,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":32.378765917004785},"seconds":{"load":0.9408037909888662,"quantize":32.37883987501846,"export":98.54943266700138,"optimize":0.6253420829889365,"export_optimize":99.17477474999032,"save":13.114967292000074,"total":145.84140758300782},"du_aimodel":"4.7G","du_bundle":"4.8G","aimodel_files":{"main.hash":32,"main.mlirb":5068103201,"metadata.json":105},"main_mlirb":{"bytes":5068103201,"sha256":"daecfad609b49d8e34db423fb6120c884d7278f3f34da7f174f62cb3976f571a"},"metadata_sha256":"65b9c91d1e92a3997c3ccaa079a85374103f254cab655affd28157e178379a70","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T20:03:16+09:00","free_bytes":557386932224,"free_gib":519.1,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_load":{"time":"2026-10-03T20:03:24+09:00","free_bytes":554672357376,"free_gib":516.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_export":{"time":"2026-10-03T20:05:35+09:00","free_bytes":542582013952,"free_gib":505.3,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_save":{"time":"2026-10-03T20:05:48+09:00","free_bytes":531222515712,"free_gib":494.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8lin_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":218.7752305000031,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8lin_pf16/kev_4b_decode_int8lin_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T20:05:55+09:00","free_bytes":532475617280,"free_gib":495.9,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after":{"time":"2026-10-03T20:09:34+09:00","free_bytes":525059321856,"free_gib":489.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}},"du_aimodelc":"11G","digest":{"bytes":12211947289,"tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"0a6f9733ca7fca724c7aafe4959bf6dad7421180b27658fae066ba7ba830bac2","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"ad3c924e2e9ae6f4ad4864a7487b089dccf6a13a91aed5160ca99b5dd8bf37ae","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b0808503de4b6cc9bddb7c1fea2664946106a74443965e9ac0aeb7c7e884a0b8","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"f715ebe6ee08f81444843fe2dbaf30066bcb1f1113fd734ffd4b85c890760f86","main-h16c.mlirb":"386d30a1d248ed135773ebb771653934cb8e8f3edd76b37ee35fd3821a8eae5c","main.hash":"8a6b309418481bde14facf22e9758bbd7f006ca175d82468a3ce6e9f32ef226f","metadata.json":"b3305c845e04a45eb51ab46c8d35997b38d17d2a526e608e14abdb6b5557c78a","stats.json":"8bd1c63e08e54922087221b47956a51431d4dd65ce1fffd2131fa7bab42793d7"}}},"finished":"2026-10-03T20:09:39+09:00"},"readout_int8lin":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8lin_pf16","name":"kev_4b_decode_int8lin_pf16","kind":"decision-backbone","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":248,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"65b9c91d1e92a3997c3ccaa079a85374103f254cab655affd28157e178379a70","main_mlirb":{"bytes":5068103201,"sha256":"daecfad609b49d8e34db423fb6120c884d7278f3f34da7f174f62cb3976f571a"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8lin_pf16.h16c.aimodelc","bytes":12211947289,"tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"0a6f9733ca7fca724c7aafe4959bf6dad7421180b27658fae066ba7ba830bac2","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"ad3c924e2e9ae6f4ad4864a7487b089dccf6a13a91aed5160ca99b5dd8bf37ae","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b0808503de4b6cc9bddb7c1fea2664946106a74443965e9ac0aeb7c7e884a0b8","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"f715ebe6ee08f81444843fe2dbaf30066bcb1f1113fd734ffd4b85c890760f86","main-h16c.mlirb":"386d30a1d248ed135773ebb771653934cb8e8f3edd76b37ee35fd3821a8eae5c","main.hash":"8a6b309418481bde14facf22e9758bbd7f006ca175d82468a3ce6e9f32ef226f","metadata.json":"b3305c845e04a45eb51ab46c8d35997b38d17d2a526e608e14abdb6b5557c78a","stats.json":"8bd1c63e08e54922087221b47956a51431d4dd65ce1fffd2131fa7bab42793d7"}}},"chunk":16,"max_ctx":4096,"readout":"one row per question, fresh zero states per row; a row of T ids runs as ceil(T / 16) calls of 'main' (static S = 16): call k gets ids[16k : 16k + 16] with position_ids 0..16k+15; the last call is padded with <|endoftext|> (248044) and the hidden rows of the padded positions are discarded (causal: they cannot reach a real position). The hidden rows [1, 16, 2560] of every call, concatenated and cut to T, are the backbone's final-norm last_hidden_state [T, 2560] the head reads. A row fits when ceil(T / 16) * 16 <= 4095 (the position dim's upper bound; the KV sequence dim is allocated at 4096).","descriptor_same_in_every_process":true,"contract":{"inputs":{"input_ids":[[1,16],"int32"],"position_ids":[[1,-1],"int32"]},"outputs":{"hidden":[[1,16,2560],"float16"]},"states":{"keyCache":[[8,1,4,-1,256],"float16"],"valueCache":[[8,1,4,-1,256],"float16"],"convState":[[24,1,8192,3],"float16"],"recState":[[24,1,32,128,128],"float16"]}},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle_4b/records_oracle.json","sha256":"eb3031df83a75b746beec3480a4bbb47787e13f8a12412ff135feb9f609c70f9","checkpoint":"jaredpalmer/kev-4b@591dcb5bd6d05eb0b5131ea6608f93f10243335c","versions":{"python":"3.12.13","torch":"2.8.0","transformers":"5.17.0","peft":"0.21.0","numpy":"2.5.3","safetensors":"0.8.0"}},"model":"kev-4b","head":{"head_safetensors":"$ZOO_WORK_ROOT/_kev/oracle_4b/head/head.safetensors","head_safetensors_sha256":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head_json_sha256":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416","scale":0.0625,"temperature":2.406050072164233,"dtype":"float32","device":"cpu"},"subset":"all","subset_file":null,"script":{"path":"conversion/kev/readout_gate.py","sha256":"b01658ccf43a0ddde9a88c82607c1f7db7b67adfd695478e9bf88a327f609172"},"bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T19:42:06+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T19:42:06+09:00"}},"processes":[{"shard":"shard_00","runs":["tv4_000:q0:base","tv4_001:q0:base","tv4_002:q0:base","tv4_003:q0:base","tv4_004:q0:base","tv4_005:q0:base","tv4_006:q0:base","tv4_007:q0:base","tv4_008:q0:base","tv4_009:q0:base","tv4_010:q0:base","tv4_011:q0:base","tv4_012:q0:base","tv4_013:q0:base","tv4_014:q0:base","tv4_015:q0:base","tv4_016:q0:base","tv4_017:q0:base","tv4_018:q0:base","tv4_019:q0:base","tv4_020:q0:base","tv4_021:q0:base","tv4_022:q0:base","tv4_023:q0:base","tv4_024:q0:base","tv4_025:q0:base","tv4_026:q0:base","tv4_027:q0:base","tv4_028:q0:base","tv4_029:q0:base","tv4_030:q0:base","tv4_031:q0:base","tv4_032:q0:base","tv4_033:q0:base","tv4_034:q0:base","tv4_035:q0:base","tv4_036:q0:base","tv4_037:q0:base","tv4_038:q0:base","tv4_039:q0:base"],"prompts":41,"load_seconds":10.309291042009136,"load_split_seconds":{"model":3.3044148749904707,"main":7.004876167018665},"function_names":["main"],"process_wall_seconds":40.120575374981854,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.35002812498714775},"contract_mismatch":[]},{"shard":"shard_01","runs":["tv4_040:q0:base","tv4_041:q0:base","tv4_042:q0:base","tv4_043:q0:base","tv4_044:q0:base","tv4_045:q0:base","tv4_046:q0:base","tv4_047:q0:base","tv4_048:q0:base","tv4_049:q0:base","tv4_050:q0:base","tv4_051:q0:base","tv4_052:q0:base","tv4_053:q0:base","tv4_054:q0:base","tv4_055:q0:base","tv4_056:q0:base","tv4_057:q0:base","tv4_058:q0:base","tv4_059:q0:base","tv4x_emotion_00:q0:base","tv4x_emotion_01:q0:base","tv4x_emotion_02:q0:base","tv4x_emotion_03:q0:base","tv4x_emotion_04:q0:base","tv4x_emotion_05:q0:base","tv4x_emotion_06:q0:base","tv4x_emotion_07:q0:base","tv4x_emotion_08:q0:base","tv4x_emotion_09:q0:base","tv4x_emotion_10:q0:base","tv4x_emotion_11:q0:base","tv4x_emotion_12:q0:base","tv4x_emotion_13:q0:base","tv4x_emotion_14:q0:base","tv4x_emotion_15:q0:base","tv4x_emotion_16:q0:base","tv4x_emotion_17:q0:base","tv4x_emotion_18:q0:base","tv4x_emotion_19:q0:base"],"prompts":41,"load_seconds":1.2595387090113945,"load_split_seconds":{"model":0.012301167007535696,"main":1.2472375420038588},"function_names":["main"],"process_wall_seconds":17.641972207988147,"reset_check":{"run":"tv4_040:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.707704916014336},"contract_mismatch":[]},{"shard":"shard_02","runs":["tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_01:q0:base","tv4x_tweet_offensive_02:q0:base","tv4x_tweet_offensive_03:q0:base","tv4x_tweet_offensive_04:q0:base","tv4x_tweet_offensive_05:q0:base","tv4x_tweet_offensive_06:q0:base","tv4x_tweet_offensive_07:q0:base","tv4x_tweet_offensive_08:q0:base","tv4x_tweet_offensive_09:q0:base","tv4x_tweet_offensive_10:q0:base","tv4x_tweet_offensive_11:q0:base","tv4x_tweet_offensive_12:q0:base","tv4x_tweet_offensive_13:q0:base","tv4x_tweet_offensive_14:q0:base","tv4x_tweet_offensive_15:q0:base","tv4x_tweet_offensive_16:q0:base","tv4x_tweet_offensive_17:q0:base","tv4x_tweet_offensive_18:q0:base","tv4x_tweet_offensive_19:q0:base","tv4x_qnli_00:q0:base","tv4x_qnli_01:q0:base","tv4x_qnli_02:q0:base","tv4x_qnli_03:q0:base","tv4x_qnli_04:q0:base","tv4x_qnli_05:q0:base","tv4x_qnli_06:q0:base","tv4x_qnli_07:q0:base","tv4x_qnli_08:q0:base","tv4x_qnli_09:q0:base","tv4x_qnli_10:q0:base","tv4x_qnli_11:q0:base","tv4x_qnli_12:q0:base","tv4x_qnli_13:q0:base","tv4x_qnli_14:q0:base","tv4x_qnli_15:q0:base","tv4x_qnli_16:q0:base","tv4x_qnli_17:q0:base","tv4x_qnli_18:q0:base","tv4x_qnli_19:q0:base"],"prompts":41,"load_seconds":1.3161277500039432,"load_split_seconds":{"model":0.010701416991651058,"main":1.3054263330122922},"function_names":["main"],"process_wall_seconds":15.64064591701026,"reset_check":{"run":"tv4x_tweet_offensive_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.28329458300140686},"contract_mismatch":[]},{"shard":"shard_03","runs":["tv4x_paws_00:q0:base","tv4x_paws_01:q0:base","tv4x_paws_02:q0:base","tv4x_paws_03:q0:base","tv4x_paws_04:q0:base","tv4x_paws_05:q0:base","tv4x_paws_06:q0:base","tv4x_paws_07:q0:base","tv4x_paws_08:q0:base","tv4x_paws_09:q0:base","tv4x_paws_10:q0:base","tv4x_paws_11:q0:base","tv4x_paws_12:q0:base","tv4x_paws_13:q0:base","tv4x_paws_14:q0:base","tv4x_paws_15:q0:base","tv4x_paws_16:q0:base","tv4x_paws_17:q0:base","tv4x_paws_18:q0:base","tv4x_paws_19:q0:base","tv4x_sciq_00:q0:base","tv4x_sciq_01:q0:base","tv4x_sciq_02:q0:base","tv4x_sciq_03:q0:base","tv4x_sciq_04:q0:base","tv4x_sciq_05:q0:base","tv4x_sciq_06:q0:base","tv4x_sciq_07:q0:base","tv4x_sciq_08:q0:base","tv4x_sciq_09:q0:base","tv4x_sciq_10:q0:base","tv4x_sciq_11:q0:base","tv4x_sciq_12:q0:base","tv4x_sciq_13:q0:base","tv4x_sciq_14:q0:base","tv4x_sciq_15:q0:base","tv4x_sciq_16:q0:base","tv4x_sciq_17:q0:base","tv4x_sciq_18:q0:base","tv4x_sciq_19:q0:base"],"prompts":41,"load_seconds":1.2365070839878172,"load_split_seconds":{"model":0.010852624982362613,"main":1.2256544590054546},"function_names":["main"],"process_wall_seconds":22.050585749995662,"reset_check":{"run":"tv4x_paws_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.3389796249975916},"contract_mismatch":[]},{"shard":"shard_04","runs":["tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_01:q0:base","tv4x_legacy_holdout_02:q0:base","tv4x_legacy_holdout_03:q0:base","tv4x_legacy_holdout_04:q0:base","tv4x_legacy_holdout_05:q0:base","tv4x_legacy_holdout_06:q0:base","tv4x_legacy_holdout_07:q0:base","tv4x_legacy_holdout_08:q0:base","tv4x_legacy_holdout_09:q0:base","tv4x_legacy_holdout_10:q0:base","tv4x_legacy_holdout_11:q0:base","tv4x_legacy_holdout_12:q0:base","tv4x_legacy_holdout_13:q0:base","tv4x_legacy_holdout_14:q0:base","tv4x_legacy_holdout_15:q0:base","tv4x_legacy_holdout_16:q0:base","tv4x_legacy_holdout_17:q0:base","tv4x_legacy_holdout_18:q0:base","tv4x_legacy_holdout_19:q0:base","tv4x_composition_holdout_00:q0:base","tv4x_composition_holdout_01:q0:base","tv4x_composition_holdout_02:q0:base","tv4x_composition_holdout_03:q0:base","tv4x_composition_holdout_04:q0:base","tv4x_composition_holdout_05:q0:base","tv4x_composition_holdout_06:q0:base","tv4x_composition_holdout_07:q0:base","tv4x_composition_holdout_08:q0:base","tv4x_composition_holdout_09:q0:base","tv4x_composition_holdout_10:q0:base","tv4x_composition_holdout_11:q0:base","tv4x_composition_holdout_12:q0:base","tv4x_composition_holdout_13:q0:base","tv4x_composition_holdout_14:q0:base","tv4x_composition_holdout_15:q0:base","tv4x_composition_holdout_16:q0:base","tv4x_composition_holdout_17:q0:base","tv4x_composition_holdout_18:q0:base","tv4x_composition_holdout_19:q0:base"],"prompts":41,"load_seconds":1.249289041996235,"load_split_seconds":{"model":0.01075066698831506,"main":1.23853837500792},"function_names":["main"],"process_wall_seconds":23.275284499977715,"reset_check":{"run":"tv4x_legacy_holdout_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.406449000001885},"contract_mismatch":[]},{"shard":"shard_05","runs":["tv4s_00:q0:base","tv4s_01:q0:base","tv4s_02:q0:base","tv4s_03:q0:base","tv4s_04:q0:base","tv4s_05:q0:base","tv4s_06:q0:base","tv4s_07:q0:base","tv4s_08:q0:base","tv4s_09:q0:base","tv4s_10:q0:base","tv4s_11:q0:base","tv4s_12:q0:base","tv4s_13:q0:base","tv4s_14:q0:base","tv4s_15:q0:base","tv4s_16:q0:base","tv4s_17:q0:base","tv4s_18:q0:base","tv4s_19:q0:base","semif_a3f18f3a63d45345942b:q0:base","semif_f40beba9088c8db8bbd6:q0:base","semif_0b43ea8e24e74d621f1b:q0:base","semif_942ec65dac0bfb9874f9:q0:base","semif_e0c140e9222d1bce5647:q0:base","semif_8e8c3804a3c15ebb31e3:q0:base","semif_56c419b12cd37cbe790c:q0:base","semif_75b470586834df8c568d:q0:base","semif_834f3b3e10cc61b33d25:q0:base","semif_76c7a79b1ff2363a60c5:q0:base","semif_36c71bb649b7d5eccb1f:q0:base","semif_eac5121a4a1bf391a08c:q0:base","semif_44574c44e243ee1b2f52:q0:base","semif_2608370d01783735c3ea:q0:base","semif_ec94ef909f7918695ab3:q0:base","semif_03d919fd0e9b3a7cd1e6:q0:base","semif_86a15618ba4cb90a307e:q0:base","semif_3dd9c3ac98f8ddf7bdd3:q0:base","semif_36273dd4777876a26338:q0:base","semif_07ee70cfe4b01330ce89:q0:base"],"prompts":41,"load_seconds":1.7598388330079615,"load_split_seconds":{"model":0.011507875024108216,"main":1.7483309579838533},"function_names":["main"],"process_wall_seconds":23.05509133299347,"reset_check":{"run":"tv4s_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.5477168340003118},"contract_mismatch":[]},{"shard":"shard_06","runs":["semif_51ceab3ce622c7c759f1:q0:base","semif_b8091be929b1e489fcc4:q0:base","semif_c89b86e3f55123bfa506:q0:base","semif_3b0958cabc339005cf66:q0:base","semif_f2b4ec4930322fe45118:q0:base","semif_61ac20b07a5189231e15:q0:base","semif_4b3d794c004566610181:q0:base","semif_b7c4e62d07986286fce0:q0:base","semif_9473603b44af63beab79:q0:base","semif_d41072dc35d8cd14c056:q0:base","semif_33d6e5da58f0fee2490d:q0:base","semif_823bf736b92e9d8fa66d:q0:base","semif_8a4c3de28be95c105ca1:q0:base","semif_ea7ec66a5e8eac886548:q0:base","semif_25172cd6ddc49ab0ca70:q0:base","semif_fb73a85d386cf59b0c51:q0:base","semif_a02fd582d2881cc462b2:q0:base","semif_19f37d5a97034bd7d04f:q0:base","semif_fda0ca94652c6d739ecf:q0:base","semif_d0e36c4cb10a29b3191c:q0:base","semif_a0a39ace7e5337d038a7:q0:base","semif_eeae283841ce75dd25bf:q0:base","semif_00bb4cc043f7d2b288d4:q0:base","semif_1a81ead69e93e5d0c86a:q0:base","semif_4444dad25e438f128673:q0:base","semif_3226202ba1ada2570e9f:q0:base","semif_3f783eec5c90812479bc:q0:base","semif_ffee861474a9665a093a:q0:base","semif_8c992e17e4fb54b044f5:q0:base","semif_695dd61195ca025c4f4c:q0:base","semif_e6155431ab7b0f8170d4:q0:base","semif_b49ecda75086d8c68d2a:q0:base","semif_587ced261c6092cd1325:q0:base","semif_533d4423d311de82b27d:q0:base","semif_2bf540d515a51755b25c:q0:base","semif_fdb9fc214aea2710763a:q0:base","semif_f6d68126b43f8c9ed183:q0:base","semif_44bae0c8a1615a096c43:q0:base","semif_2175379f4c69e6207626:q0:base","semif_def9fe59d262357a673a:q0:base"],"prompts":41,"load_seconds":2.1570593749929685,"load_split_seconds":{"model":0.013012708979658782,"main":2.1440466660133097},"function_names":["main"],"process_wall_seconds":21.708550790994195,"reset_check":{"run":"semif_51ceab3ce622c7c759f1:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.41546804099925794},"contract_mismatch":[]},{"shard":"shard_07","runs":["semif_712f6ffb763eef7c9a12:q0:base","semif_e1d610bd14e3d16c09b9:q0:base","semif_22ec57822db50ec121cf:q0:base","semif_3e75f2b84e8e08e3d3f2:q0:base","semif_ebd74aa05324a1556288:q0:base","semif_d7e84e4245a29817da1f:q0:base","semif_f41ae66b3c09630c7a42:q0:base","semif_82c2cd97c160ad27951b:q0:base","semif_7ada97b6f99f4788e5c7:q0:base","semif_4d7a003342db5b0a17e3:q0:base","semif_19d3d8d0bf773e88be7b:q0:base","semif_d6de731cca52359b1c87:q0:base","semif_b45c63fb92729af27f05:q0:base","semif_46b7029b9a704138b77a:q0:base","semif_caf773b23934fa7362a1:q0:base","semif_f46f392ef9e9e9df564b:q0:base","semif_005dcf6d2c0f21d64796:q0:base","semif_edb708bd0fe5fd67d6ee:q0:base","semif_97a35499bd8717f3e786:q0:base","semif_31eb4533d2c89959ec46:q0:base","semif_0aa24a6701ac9c56c4a0:q0:base","semif_796d5c0da6eaadae6a3f:q0:base","semif_599b3092b6fd18312d23:q0:base","semif_960a4bcd27363e6db4b9:q0:base","semif_c6a411e7f2bbdf377173:q0:base","semif_e332aa927d0c2ed044aa:q0:base","semif_a0128e551d095f3449f8:q0:base","semif_46d6a0731bd288c370f0:q0:base","semif_21ca7a8b970e36b8e20f:q0:base","semif_4aa83af95bb69e90bc2d:q0:base","semif_cdfa9eed51fafcf09b72:q0:base","semif_d51b05bfb9d6d7f09331:q0:base","semif_fdd902cc6e65fad7cb4f:q0:base","semif_42384af9ae56beb37ff5:q0:base","semif_cdf7f428857e1b841cb3:q0:base","semif_774f7f9245f83c796d44:q0:base","semif_92ec2ffa3dd46ada1335:q0:base","semif_6149a17bc154f9c5b4a0:q0:base","semif_be866e1dbec5d42d9423:q0:base","semif_97d2ed2591fc7638806c:q0:base"],"prompts":41,"load_seconds":1.6593077500001527,"load_split_seconds":{"model":0.016126625007018447,"main":1.6431811249931343},"function_names":["main"],"process_wall_seconds":20.195488208002644,"reset_check":{"run":"semif_712f6ffb763eef7c9a12:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.28829070800566114},"contract_mismatch":[]},{"shard":"shard_08","runs":["semif_4533dd91267b4d815690:q0:base","semif_d880e65fef5e2993fdd0:q0:base","semif_90494c76078ffc3c717a:q0:base","semif_9ac5cc82c4126c91174a:q0:base","semif_386e10b1c1ce906473a4:q0:base","semif_4da6aedc1139224eeff2:q0:base","semif_a22dab8ba073ff6cdb8d:q0:base","semif_42c54fa2ad45743e24d8:q0:base","semif_f3d13cf184f916677def:q0:base","semif_1105577da4c8dad4609d:q0:base","semif_0ebd299739302d8b057b:q0:base","semif_eafc22c8c40df3932a8e:q0:base","semif_0aa1447680e4f25862bb:q0:base","semif_980e76145c925747220c:q0:base","semif_4aee37a6a155747766aa:q0:base","semif_6db17cc22bd656d78558:q0:base","semif_ef3e0f8e657bf59f8a29:q0:base","semif_f756e1cb04e11766341e:q0:base","semif_d167e51bb1f499099645:q0:base","semif_674ca4e9346216255466:q0:base","semif_1df851a202994a07f123:q0:base","semif_b5e6788d95b8395d2888:q0:base","semif_32f36cd0fbbfb3604586:q0:base","semif_843185706adff0c60f8b:q0:base","semif_b531dd603bf080dab820:q0:base","semif_be2dd67bc733c1213bef:q0:base","semif_b8e2750307df9761e716:q0:base","semif_5d31b4bf2d0fc193f53b:q0:base","semif_a9230edfc6ec9e524af0:q0:base","semif_c100e84585149a04be49:q0:base","semif_579beed184ffc1104aa9:q0:base","semif_c4e9bd463b6fde5be743:q0:base","semif_b863998d4be2721a5dfd:q0:base","semif_37cebc3792e98049439f:q0:base","semif_64d63f86e328aff1481c:q0:base","semif_6d64a8459f64eac7f4cc:q0:base","semif_2708193212d8a4a523c7:q0:base","semif_5a37d1dfcad20810a76a:q0:base","semif_acacd8b46acd25cb954b:q0:base","semif_4089c199b8b796ec3ed8:q0:base"],"prompts":41,"load_seconds":1.7113432089972775,"load_split_seconds":{"model":0.011734375002561137,"main":1.6996088339947164},"function_names":["main"],"process_wall_seconds":22.776170416007517,"reset_check":{"run":"semif_4533dd91267b4d815690:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.5671698750229552},"contract_mismatch":[]},{"shard":"shard_09","runs":["semif_02f2b95a5c5a4777a45c:q0:base","semif_ebea4681e3dcc7d92ca2:q0:base","semif_b8c5b9b285cbdddc9f8d:q0:base","semif_7ed75d854d99b3a81916:q0:base","own_t01:q0:base","own_t01:q1:base","own_t01:q2:base","own_t01:q3:base","own_t02:q0:base","own_t02:q1:base","own_t02:q2:base","own_t03:q0:base","own_t03:q1:base","own_t03:q2:base","own_t04:q0:base","own_t04:q1:base","own_t04:q2:base","own_t04:q3:base","own_t05:q0:base","own_t05:q1:base","own_t05:q2:base","own_t05:q3:base","own_t06:q0:base","own_t06:q1:base","own_t06:q2:base","own_t07:q0:base","own_t07:q1:base","own_t07:q2:base","own_t08:q0:base","own_t08:q1:base","own_t08:q2:base","own_t09:q0:base","own_t09:q1:base","own_t10:q0:base","own_t10:q1:base","own_t10:q2:base","own_j01:q0:base","own_j01:q1:base","own_j01:q2:base","own_j01:q3:base"],"prompts":41,"load_seconds":3.1429967080184724,"load_split_seconds":{"model":0.014226541010430083,"main":3.1287701670080423},"function_names":["main"],"process_wall_seconds":57.478381749999244,"reset_check":{"run":"semif_02f2b95a5c5a4777a45c:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.5181256250070874},"contract_mismatch":[]},{"shard":"shard_10","runs":["own_j02:q0:base","own_j02:q1:base","own_j02:q2:base","own_j03:q0:base","own_j03:q1:base","own_j03:q2:base","own_j04:q0:base","own_j04:q1:base","own_j04:q2:base","own_j05:q0:base","own_j05:q1:base","own_j05:q2:base","own_a01:q0:base","own_a01:q1:base","own_a01:q2:base","own_m01:q0:base","own_m01:q1:base","own_m01:q2:base","own_m01:q3:base","own_m01:q4:base","own_m01:q5:base","own_m01:q6:base","own_m01:q7:base","own_L01:q0:base","own_L01:q1:base","own_L01:q2:base","own_L02:q0:base","own_L02:q1:base","own_L02:q2:base","own_L02:q3:base","own_L03:q0:base","own_L03:q1:base","own_L03:q2:base","own_L03:q3:base"],"prompts":35,"load_seconds":6.095050458010519,"load_split_seconds":{"model":0.013043166982242838,"main":6.082007291028276},"function_names":["main"],"process_wall_seconds":162.91064920800272,"reset_check":{"run":"own_j02:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":2.3565237500006333},"contract_mismatch":[]}],"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":431,"questions_non_near_tie":422,"argmax_equal_non_near_tie":422,"near_tie_questions":12,"argmax_equal_near_tie":9,"max_abs_dp":0.060132622718811035,"max_abs_dp_non_near_tie":0.060132622718811035,"mean_of_run_mean_abs_dp":0.001917580727576835,"mean_of_record_mean_abs_dp":0.001992916118077953,"mean_question_max_abs_dp":0.002925006447151767,"max_abs_dlogit_T":0.19137823581695557,"worst_run":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.060132622718811035},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9241736357846124,"hidden_max_abs_diff":2.9803671836853027,"positions":9168,"positions_below":{"0.99":7,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":375231.81703913724,"ms_per_call_median":79.12579199182801,"ms_per_call_warm_median":79.4767295155907},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.007657796144485474,"max_abs_dp_non_near_tie":0.007657796144485474,"mean_of_run_mean_abs_dp":0.0018129324380680102,"mean_of_record_mean_abs_dp":0.0019074514414255152,"mean_question_max_abs_dp":0.0026643969069578147,"max_abs_dlogit_T":0.07189655303955078,"worst_run":{"id":"own_j04","q":2,"max_abs_dp":0.007657796144485474},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9241736357846124,"hidden_max_abs_diff":2.9803671836853027,"positions":1027,"positions_below":{"0.99":7,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":35972.74800098967,"ms_per_call_median":110.67429100512527,"ms_per_call_warm_median":111.44489599973895},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0065285563468933105,"max_abs_dp_non_near_tie":0.0065285563468933105,"mean_of_run_mean_abs_dp":0.0008280136837680457,"mean_of_record_mean_abs_dp":0.0009040353480431959,"mean_question_max_abs_dp":0.001503908329389312,"max_abs_dlogit_T":0.03676772117614746,"worst_run":{"id":"own_L03","q":3,"max_abs_dp":0.0065285563468933105},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9911493146196642,"hidden_max_abs_diff":2.8959789276123047,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":114330.99462511018,"ms_per_call_median":105.65860399219673,"ms_per_call_warm_median":105.65860399219673},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":38,"questions_non_near_tie":38,"argmax_equal_non_near_tie":38,"near_tie_questions":2,"argmax_equal_near_tie":0,"max_abs_dp":0.012297570705413818,"max_abs_dp_non_near_tie":0.012297570705413818,"mean_of_run_mean_abs_dp":0.0012982515897116778,"mean_of_record_mean_abs_dp":0.0013069881194647868,"mean_question_max_abs_dp":0.002150862285634503,"max_abs_dlogit_T":0.08287954330444336,"worst_run":{"id":"own_t06","q":0,"max_abs_dp":0.012297570705413818},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9991566743103716,"hidden_max_abs_diff":0.8394285440444946,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":49113.97633192246,"ms_per_call_median":108.61662498791702,"ms_per_call_warm_median":108.61662498791702},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":144,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":3,"max_abs_dp":0.060132622718811035,"max_abs_dp_non_near_tie":0.060132622718811035,"mean_of_run_mean_abs_dp":0.0024710062836540464,"mean_of_record_mean_abs_dp":0.0024710062836540464,"mean_question_max_abs_dp":0.00370650076446408,"max_abs_dlogit_T":0.19137823581695557,"worst_run":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.060132622718811035},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9972377129605107,"hidden_max_abs_diff":0.7224006652832031,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":66812.95467316522,"ms_per_call_median":73.98691700655036,"ms_per_call_warm_median":73.87033398845233},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":58,"argmax_equal_non_near_tie":58,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.02085888385772705,"max_abs_dp_non_near_tie":0.02085888385772705,"mean_of_run_mean_abs_dp":0.0023440991099657063,"mean_of_record_mean_abs_dp":0.0023440991099657063,"mean_question_max_abs_dp":0.004453547423084577,"max_abs_dlogit_T":0.13496172428131104,"worst_run":{"id":"tv4_012","q":0,"max_abs_dp":0.02085888385772705},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9983649838879813,"hidden_max_abs_diff":1.9320526123046875,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":38885.27849613456,"ms_per_call_median":54.14354201639071,"ms_per_call_warm_median":54.1888335137628},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":19,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":0,"max_abs_dp":0.012888491153717041,"max_abs_dp_non_near_tie":0.012888491153717041,"mean_of_run_mean_abs_dp":0.0027975413560246422,"mean_of_record_mean_abs_dp":0.0027975413560246422,"mean_question_max_abs_dp":0.0041963160037994385,"max_abs_dlogit_T":0.10678887367248535,"worst_run":{"id":"tv4s_01","q":0,"max_abs_dp":0.012888491153717041},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":13076.206793921301,"ms_per_call_median":74.53199999872595,"ms_per_call_warm_median":73.50658299401402},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":137,"argmax_equal_non_near_tie":137,"near_tie_questions":3,"argmax_equal_near_tie":3,"max_abs_dp":0.016363978385925293,"max_abs_dp_non_near_tie":0.016363978385925293,"mean_of_run_mean_abs_dp":0.0013166030483608621,"mean_of_record_mean_abs_dp":0.0013166030483608621,"mean_question_max_abs_dp":0.0016526892731365348,"max_abs_dlogit_T":0.11447376012802124,"worst_run":{"id":"tv4x_qnli_19","q":0,"max_abs_dp":0.016363978385925293},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":57039.65811789385,"ms_per_call_median":54.83491602353752,"ms_per_call_warm_median":54.81995799345896}},"near_ties":[{"id":"tv4_021","q":0,"qid":"answer","oracle_top2_margin":0.01855599880218506,"argmax_equal":true,"max_abs_dp":0.002816528081893921,"probs":[0.28461524844169617,0.1531343013048172,0.30778226256370544,0.2544682025909424]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.00373879075050354,"argmax_equal":true,"max_abs_dp":0.003355756402015686,"probs":[0.2527696192264557,0.2381657212972641,0.2587856650352478,0.2502789795398712]},{"id":"tv4x_emotion_15","q":0,"qid":"emotion","oracle_top2_margin":0.007101953029632568,"argmax_equal":true,"max_abs_dp":0.0016972124576568604,"probs":[0.0005979963461868465,0.4902830719947815,0.49407121539115906,0.0008371005533263087,0.0008661759784445167,0.013344401493668556]},{"id":"tv4x_tweet_offensive_01","q":0,"qid":"offensive","oracle_top2_margin":0.002196580171585083,"argmax_equal":true,"max_abs_dp":0.002484619617462158,"probs":[0.49641716480255127,0.5035828948020935]},{"id":"tv4x_paws_01","q":0,"qid":"paraphrase","oracle_top2_margin":0.007073014974594116,"argmax_equal":true,"max_abs_dp":6.490945816040039e-05,"probs":[0.49652838706970215,0.5034716129302979]},{"id":"tv4s_11","q":0,"qid":"decision","oracle_top2_margin":0.001054227352142334,"argmax_equal":false,"max_abs_dp":0.006644874811172485,"probs":[0.010619686916470528,0.500781238079071,0.4885990619659424]},{"id":"semif_ffee861474a9665a093a","q":0,"qid":"decision","oracle_top2_margin":0.0018825829029083252,"argmax_equal":true,"max_abs_dp":0.0038654059171676636,"probs":[0.3957323431968689,0.2029251605272293,0.4013425409793854]},{"id":"semif_2708193212d8a4a523c7","q":0,"qid":"decision","oracle_top2_margin":0.0197182297706604,"argmax_equal":true,"max_abs_dp":0.009920179843902588,"probs":[0.22729066014289856,0.3849354088306427,0.38777393102645874]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0004886090755462646,"argmax_equal":true,"max_abs_dp":0.0024436116218566895,"probs":[0.4400648772716522,0.12359163910150528,0.4363435208797455]},{"id":"own_t01","q":2,"qid":"upset","oracle_top2_margin":0.0016841888427734375,"argmax_equal":false,"max_abs_dp":0.004243016242980957,"probs":[0.37062928080558777,0.3655156195163727,0.1940469741821289,0.06980811804533005]},{"id":"own_t02","q":0,"qid":"impact","oracle_top2_margin":0.0024104714393615723,"argmax_equal":false,"max_abs_dp":0.004480421543121338,"probs":[0.19402752816677094,0.3729144036769867,0.36822229623794556,0.06483573466539383]},{"id":"own_j05","q":2,"qid":"severity","oracle_top2_margin":0.0014663934707641602,"argmax_equal":true,"max_abs_dp":0.0034579038619995117,"probs":[0.11921381205320358,0.053303372114896774,0.11971210688352585,0.21478959918022156,0.20872221887111664,0.16581302881240845,0.11844577640295029]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":false,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"FAIL","red_arm":null,"timing":{"contended":true,"S":16,"load_seconds":{"first_process":10.309291042009136,"median":1.7113432089972775,"all":[10.309291042009136,1.2595387090113945,1.3161277500039432,1.2365070839878172,1.249289041996235,1.7598388330079615,2.1570593749929685,1.6593077500001527,1.7113432089972775,3.1429967080184724,6.095050458010519]},"ms_per_call_warm_runs":{"n":3972,"median":79.4767295155907,"p10":53.222950611962005,"p90":115.98405421536881},"ms_per_call_all_runs":{"n":4055,"median":79.12579199182801,"p10":53.13289120676927,"p90":116.0147335845977},"ms_per_token_warm_runs":{"n":423,"median":4.627072580924798,"p10":3.5361503348416004,"p90":6.83464585537185},"run_decode_seconds_warm_runs":{"n":423,"median":0.38187612500041723,"p10":0.26449594998848625,"p90":1.2298624414019286},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":322278.1381857203}},"seconds":{"gpu_processes":426.9190163749736,"scoring":25.92801162498654,"total":453.4931568749889},"runs":[{"id":"tv4_000","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001441359519958496,"mean_abs_dp":0.000720687210559845,"oracle_top2_margin":0.029402226209640503,"near_tie":false,"probs":[0.3164719045162201,0.12716685235500336,0.2846033573150635,0.2717578709125519],"hidden":{"min_pos_cos":0.9983649838879813,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4_001","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.213520050048828e-05,"mean_abs_dp":4.75192500744015e-05,"oracle_top2_margin":0.9775130152702332,"near_tie":false,"probs":[0.0011405708501115441,0.0033970780204981565,0.9864122867584229,0.009050019085407257]},{"id":"tv4_002","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00042235851287841797,"mean_abs_dp":0.00021116714924573898,"oracle_top2_margin":0.9258788824081421,"near_tie":false,"probs":[0.9457501173019409,0.017437396571040154,0.016414402052760124,0.02039807103574276]},{"id":"tv4_003","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0030638575553894043,"mean_abs_dp":0.0015319325029850006,"oracle_top2_margin":0.6721583604812622,"near_tie":false,"probs":[0.7978773713111877,0.12117361277341843,0.039602130651474,0.041346874088048935]},{"id":"tv4_004","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002300739288330078,"mean_abs_dp":0.00011502171400934458,"oracle_top2_margin":0.9810932278633118,"near_tie":false,"probs":[0.004916905891150236,0.006055405829101801,0.9868243336677551,0.0022033194545656443]},{"id":"tv4_005","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000736728310585022,"mean_abs_dp":0.000553737161681056,"oracle_top2_margin":0.8511642813682556,"near_tie":false,"probs":[0.9002190828323364,0.049327414482831955,0.014902815222740173,0.03555070236325264]},{"id":"tv4_006","q":0,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0024996399879455566,"mean_abs_dp":0.0012498064897954464,"oracle_top2_margin":0.6825103759765625,"near_tie":false,"probs":[0.019931023940443993,0.14038202166557312,0.02099304087460041,0.8186938166618347]},{"id":"tv4_007","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.019733965396881104,"mean_abs_dp":0.010994646698236465,"oracle_top2_margin":0.2817847728729248,"near_tie":false,"probs":[0.24262303113937378,0.11725059151649475,0.5635859966278076,0.07654035836458206]},{"id":"tv4_008","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00801151990890503,"mean_abs_dp":0.004005782480817288,"oracle_top2_margin":0.7711502909660339,"near_tie":false,"probs":[0.11412893235683441,0.8699172735214233,0.0030131733510643244,0.012940607964992523]},{"id":"tv4_009","q":0,"variant":"base","tokens":321,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":9.989738464355469e-05,"mean_abs_dp":4.993187030777335e-05,"oracle_top2_margin":0.9636144638061523,"near_tie":false,"probs":[0.0033255841117352247,0.0023259965237230062,0.9790356159210205,0.01531289517879486]},{"id":"tv4_010","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006636887788772583,"mean_abs_dp":0.004636162891983986,"oracle_top2_margin":0.03897985816001892,"near_tie":false,"probs":[0.07612811774015427,0.3497679531574249,0.38474637269973755,0.18935759365558624]},{"id":"tv4_011","q":0,"variant":"base","tokens":105,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0032713115215301514,"mean_abs_dp":0.0016356445848941803,"oracle_top2_margin":0.09546518325805664,"near_tie":false,"probs":[0.04941418394446373,0.19640378654003143,0.3272508382797241,0.4269312024116516]},{"id":"tv4_012","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.02085888385772705,"mean_abs_dp":0.010429429356008768,"oracle_top2_margin":0.5005620121955872,"near_tie":false,"probs":[0.1878363937139511,0.010995684191584587,0.07808861881494522,0.7230793237686157]},{"id":"tv4_013","q":0,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003831535577774048,"mean_abs_dp":0.0019157733768224716,"oracle_top2_margin":0.18121948838233948,"near_tie":false,"probs":[0.2551705241203308,0.08518076688051224,0.4415235221385956,0.21812523901462555]},{"id":"tv4_014","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004098176956176758,"mean_abs_dp":0.0020490847527980804,"oracle_top2_margin":0.28027498722076416,"near_tie":false,"probs":[0.5195093154907227,0.13622808456420898,0.23164844512939453,0.11261415481567383]},{"id":"tv4_015","q":0,"variant":"base","tokens":177,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0028708279132843018,"mean_abs_dp":0.0021415166556835175,"oracle_top2_margin":0.06737852096557617,"near_tie":false,"probs":[0.4447031319141388,0.08649616688489914,0.3830587565898895,0.08574188500642776]},{"id":"tv4_016","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.009986400604248047,"mean_abs_dp":0.004993191920220852,"oracle_top2_margin":0.7223381996154785,"near_tie":false,"probs":[0.05707942321896553,0.07788336277008057,0.07767757028341293,0.7873595952987671]},{"id":"tv4_017","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008524656295776367,"mean_abs_dp":0.0004262607544660568,"oracle_top2_margin":0.6426428556442261,"near_tie":false,"probs":[0.1047576442360878,0.08030197024345398,0.7463616728782654,0.06857877224683762]},{"id":"tv4_018","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00037920475006103516,"mean_abs_dp":0.0001895866880659014,"oracle_top2_margin":0.9770302772521973,"near_tie":false,"probs":[0.0038882901426404715,0.0020038827788084745,0.9852983951568604,0.008809427730739117]},{"id":"tv4_019","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004178106784820557,"mean_abs_dp":0.0021847225725650787,"oracle_top2_margin":0.0909552276134491,"near_tie":false,"probs":[0.2655843198299408,0.23682722449302673,0.13332220911979675,0.3642662763595581]},{"id":"tv4_020","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003131832927465439,"mean_abs_dp":0.00019174208864569664,"oracle_top2_margin":0.8149018287658691,"near_tie":false,"probs":[0.07217513769865036,0.015835143625736237,0.887396514415741,0.0245931725949049]},{"id":"tv4_021","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002816528081893921,"mean_abs_dp":0.001708701252937317,"oracle_top2_margin":0.01855599880218506,"near_tie":true,"probs":[0.28461524844169617,0.1531343013048172,0.30778226256370544,0.2544682025909424]},{"id":"tv4_022","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013556331396102905,"mean_abs_dp":0.0009713508188724518,"oracle_top2_margin":0.15448573231697083,"near_tie":false,"probs":[0.47274529933929443,0.31797754764556885,0.16604173183441162,0.04323543980717659]},{"id":"tv4_023","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006737947463989258,"mean_abs_dp":0.0033689592964947224,"oracle_top2_margin":0.5718164443969727,"near_tie":false,"probs":[0.7404894232749939,0.0809803381562233,0.021655187010765076,0.15687505900859833]},{"id":"tv4_024","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024227499961853027,"mean_abs_dp":0.0012113777920603752,"oracle_top2_margin":0.02086666226387024,"near_tie":false,"probs":[0.3956015706062317,0.18098817765712738,0.04534497857093811,0.37806519865989685]},{"id":"tv4_025","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008030533790588379,"mean_abs_dp":0.0040152668952941895,"oracle_top2_margin":0.24806053936481476,"near_tie":false,"probs":[0.47231370210647583,0.11322640627622604,0.23859606683254242,0.17586377263069153]},{"id":"tv4_026","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0046294331550598145,"mean_abs_dp":0.0023147128522396088,"oracle_top2_margin":0.03857165575027466,"near_tie":false,"probs":[0.2213706523180008,0.3078211545944214,0.19688476622104645,0.27392345666885376]},{"id":"tv4_027","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005111604928970337,"mean_abs_dp":0.002634170465171337,"oracle_top2_margin":0.10682827234268188,"near_tie":false,"probs":[0.02978522516787052,0.3506421744823456,0.46481576561927795,0.15475685894489288]},{"id":"tv4_028","q":0,"variant":"base","tokens":290,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010508567094802856,"mean_abs_dp":0.0052543096244335175,"oracle_top2_margin":0.03244104981422424,"near_tie":false,"probs":[0.3575552999973297,0.3081657886505127,0.2026800513267517,0.1315988153219223]},{"id":"tv4_029","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00014333799481391907,"mean_abs_dp":7.166956493165344e-05,"oracle_top2_margin":0.9377733469009399,"near_tie":false,"probs":[0.024927986785769463,0.00070022331783548,0.01139979436993599,0.9629719853401184]},{"id":"tv4_030","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0037003159523010254,"mean_abs_dp":0.0020713619887828827,"oracle_top2_margin":0.6803373098373413,"near_tie":false,"probs":[0.11260272562503815,0.03779039904475212,0.7855663895606995,0.06404043734073639]},{"id":"tv4_031","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018417835235595703,"mean_abs_dp":0.0009209036361426115,"oracle_top2_margin":0.8230345249176025,"near_tie":false,"probs":[0.8996177315711975,0.07976695150136948,0.009657786227762699,0.010957595892250538]},{"id":"tv4_032","q":0,"variant":"base","tokens":135,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0008178353309631348,"mean_abs_dp":0.0004089277936145663,"oracle_top2_margin":0.9462494254112244,"near_tie":false,"probs":[0.00667082192376256,0.006723122671246529,0.020767146721482277,0.9658389687538147]},{"id":"tv4_033","q":0,"variant":"base","tokens":293,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":9.34116542339325e-05,"mean_abs_dp":5.55326696485281e-05,"oracle_top2_margin":0.9047749042510986,"near_tie":false,"probs":[0.0024500505533069372,0.005126158706843853,0.04381611943244934,0.9486076831817627]},{"id":"tv4_034","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.010870218276977539,"mean_abs_dp":0.005443899892270565,"oracle_top2_margin":0.08403569459915161,"near_tie":false,"probs":[0.16930891573429108,0.059048596769571304,0.41750529408454895,0.3541371822357178]},{"id":"tv4_035","q":0,"variant":"base","tokens":108,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007489286363124847,"mean_abs_dp":0.0003744647838175297,"oracle_top2_margin":0.8683550953865051,"near_tie":false,"probs":[0.9208946228027344,0.013360917568206787,0.05172795429825783,0.01401644479483366]},{"id":"tv4_036","q":0,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.009308189153671265,"mean_abs_dp":0.00484815239906311,"oracle_top2_margin":0.17973920702934265,"near_tie":false,"probs":[0.11539395898580551,0.2243252396583557,0.4123656749725342,0.24791517853736877]},{"id":"tv4_037","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0035778582096099854,"mean_abs_dp":0.0017889626324176788,"oracle_top2_margin":0.2104194164276123,"near_tie":false,"probs":[0.13032273948192596,0.09264373034238815,0.49084582924842834,0.28618764877319336]},{"id":"tv4_038","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003560185432434082,"mean_abs_dp":0.00021814275532960892,"oracle_top2_margin":0.8337569236755371,"near_tie":false,"probs":[0.041728198528289795,0.025241006165742874,0.8835317492485046,0.04949909821152687]},{"id":"tv4_039","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0061082541942596436,"mean_abs_dp":0.0030721784569323063,"oracle_top2_margin":0.053804636001586914,"near_tie":false,"probs":[0.48662057518959045,0.42127078771591187,0.0750431939959526,0.0170653834939003]},{"id":"tv4_040","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.008497893810272217,"mean_abs_dp":0.004248954355716705,"oracle_top2_margin":0.45171576738357544,"near_tie":false,"probs":[0.07961538434028625,0.19434475898742676,0.09034495055675507,0.6356949806213379]},{"id":"tv4_041","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0019794106483459473,"mean_abs_dp":0.0011283932253718376,"oracle_top2_margin":0.23504745960235596,"near_tie":false,"probs":[0.1245371550321579,0.045421358197927475,0.2990431487560272,0.5309983491897583]},{"id":"tv4_042","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003927409648895264,"mean_abs_dp":0.001963723450899124,"oracle_top2_margin":0.18441274762153625,"near_tie":false,"probs":[0.44271400570869446,0.13000164926052094,0.25887981057167053,0.1684045046567917]},{"id":"tv4_043","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006895720958709717,"mean_abs_dp":0.0034478437155485153,"oracle_top2_margin":0.739353358745575,"near_tie":false,"probs":[0.07361776381731033,0.821323037147522,0.04243273288011551,0.06262649595737457]},{"id":"tv4_044","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00011146068572998047,"mean_abs_dp":5.570505527430214e-05,"oracle_top2_margin":0.9956979751586914,"near_tie":false,"probs":[0.0017836811020970345,0.0001727178751025349,0.0007380141178146005,0.9973055124282837]},{"id":"tv4_045","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.01393580436706543,"mean_abs_dp":0.006967914057895541,"oracle_top2_margin":0.8363601565361023,"near_tie":false,"probs":[0.05028802156448364,0.009092232212424278,0.909095823764801,0.03152389079332352]},{"id":"tv4_046","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001055002212524414,"mean_abs_dp":7.131241727620363e-05,"oracle_top2_margin":0.9711371064186096,"near_tie":false,"probs":[0.9805418252944946,0.005729885306209326,0.009473070502281189,0.004255200736224651]},{"id":"tv4_047","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0018193498253822327,"mean_abs_dp":0.001158156432211399,"oracle_top2_margin":0.5407507419586182,"near_tie":false,"probs":[0.05193782225251198,0.09747432917356491,0.6963616609573364,0.15422609448432922]},{"id":"tv4_048","q":0,"variant":"base","tokens":119,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001094043254852295,"mean_abs_dp":0.0008366797119379044,"oracle_top2_margin":0.2586537003517151,"near_tie":false,"probs":[0.19415295124053955,0.493958055973053,0.07614670693874359,0.23574228584766388]},{"id":"tv4_049","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.003496885299682617,"mean_abs_dp":0.0017852704040706158,"oracle_top2_margin":0.5550865530967712,"near_tie":false,"probs":[0.030491270124912262,0.20148472487926483,0.018128149211406708,0.7498958706855774]},{"id":"tv4_050","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011667758226394653,"mean_abs_dp":0.0005833818286191672,"oracle_top2_margin":0.6465844511985779,"near_tie":false,"probs":[0.8231033682823181,0.17423830926418304,0.0009216486942023039,0.0017366302199661732]},{"id":"tv4_051","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0029401034116744995,"mean_abs_dp":0.0016938205808401108,"oracle_top2_margin":0.4407600164413452,"near_tie":false,"probs":[0.08572130650281906,0.03204545006155968,0.6586621403694153,0.22357109189033508]},{"id":"tv4_052","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0057392120361328125,"mean_abs_dp":0.002869606949388981,"oracle_top2_margin":0.9107492566108704,"near_tie":false,"probs":[0.9340114593505859,0.022098174318671227,0.01143740862607956,0.03245304152369499]},{"id":"tv4_053","q":0,"variant":"base","tokens":358,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0026839077472686768,"mean_abs_dp":0.0019107870757579803,"oracle_top2_margin":0.22727736830711365,"near_tie":false,"probs":[0.28805482387542725,0.1016848236322403,0.10000111907720566,0.510259211063385]},{"id":"tv4_054","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003355756402015686,"mean_abs_dp":0.0018421709537506104,"oracle_top2_margin":0.00373879075050354,"near_tie":true,"probs":[0.2527696192264557,0.2381657212972641,0.2587856650352478,0.2502789795398712]},{"id":"tv4_055","q":0,"variant":"base","tokens":189,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.007060706615447998,"mean_abs_dp":0.004121255129575729,"oracle_top2_margin":0.0706123411655426,"near_tie":false,"probs":[0.24538549780845642,0.20090627670288086,0.3275690972805023,0.226139098405838]},{"id":"tv4_056","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.011769533157348633,"mean_abs_dp":0.005884755402803421,"oracle_top2_margin":0.2170730084180832,"near_tie":false,"probs":[0.1722836196422577,0.21679693460464478,0.19293490052223206,0.4179845154285431]},{"id":"tv4_057","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0048413872718811035,"mean_abs_dp":0.002420671284198761,"oracle_top2_margin":0.04467204213142395,"near_tie":false,"probs":[0.19498878717422485,0.2743334472179413,0.21708320081233978,0.3135945200920105]},{"id":"tv4_058","q":0,"variant":"base","tokens":129,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.012551069259643555,"mean_abs_dp":0.006275510415434837,"oracle_top2_margin":0.23947922885417938,"near_tie":false,"probs":[0.20738020539283752,0.2120547592639923,0.47146114706993103,0.10910394042730331]},{"id":"tv4_059","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004632454365491867,"mean_abs_dp":0.0003536404110491276,"oracle_top2_margin":0.9182034134864807,"near_tie":false,"probs":[0.9417886137962341,0.022764602676033974,0.010996541008353233,0.024450335651636124]},{"id":"tv4x_emotion_00","q":0,"variant":"base","tokens":42,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0020350366830825806,"mean_abs_dp":0.0009259919946392378,"oracle_top2_margin":0.365003764629364,"near_tie":false,"probs":[0.06469883769750595,0.1865760236978531,0.5488433837890625,0.029482604935765266,0.038115594536066055,0.13228361308574677]},{"id":"tv4x_emotion_01","q":0,"variant":"base","tokens":38,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00032532215118408203,"mean_abs_dp":0.00013756736492117247,"oracle_top2_margin":0.32259851694107056,"near_tie":false,"probs":[0.20858915150165558,0.01712534949183464,0.013934027403593063,0.531744122505188,0.06595921516418457,0.16264809668064117]},{"id":"tv4x_emotion_02","q":0,"variant":"base","tokens":36,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001252293586730957,"mean_abs_dp":0.0004572739126160741,"oracle_top2_margin":0.7255271077156067,"near_tie":false,"probs":[0.8250080943107605,0.0025016553699970245,0.001909587881527841,0.05086633190512657,0.10167071223258972,0.01804354228079319]},{"id":"tv4x_emotion_03","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002209305763244629,"mean_abs_dp":0.000736434788753589,"oracle_top2_margin":0.4680539667606354,"near_tie":false,"probs":[0.21106132864952087,0.09193398058414459,0.6755875945091248,0.006190836895257235,0.0082993870601058,0.006926898378878832]},{"id":"tv4x_emotion_04","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.870529174804688e-05,"mean_abs_dp":4.1719351429492235e-05,"oracle_top2_margin":0.9324661493301392,"near_tie":false,"probs":[0.0013324804604053497,0.961681604385376,0.029102135449647903,0.0008393688476644456,0.0010093378368765116,0.006035070400685072]},{"id":"tv4x_emotion_05","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004512578248977661,"mean_abs_dp":0.0017062487701574962,"oracle_top2_margin":0.09678417444229126,"near_tie":false,"probs":[0.22179149091243744,0.262017160654068,0.0665755495429039,0.3548949062824249,0.043612100183963776,0.05110878869891167]},{"id":"tv4x_emotion_06","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003316819667816162,"mean_abs_dp":0.0011056013560543458,"oracle_top2_margin":0.16045954823493958,"near_tie":false,"probs":[0.005203218664973974,0.38272184133529663,0.5488709807395935,0.0063973525539040565,0.006822157185524702,0.04998444393277168]},{"id":"tv4x_emotion_07","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008792281150817871,"mean_abs_dp":0.00029690763525043923,"oracle_top2_margin":0.8479410409927368,"near_tie":false,"probs":[0.9042457938194275,0.0023824074305593967,0.0008749439148232341,0.05778966099023819,0.022134793922305107,0.012572300620377064]},{"id":"tv4x_emotion_08","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00054931640625,"mean_abs_dp":0.0003009453260650237,"oracle_top2_margin":0.4905897378921509,"near_tie":false,"probs":[0.017996344715356827,0.7140955328941345,0.22330999374389648,0.004772451240569353,0.005971899256110191,0.033853743225336075]},{"id":"tv4x_emotion_09","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020017623901367188,"mean_abs_dp":0.0011094386378924053,"oracle_top2_margin":0.3611770570278168,"near_tie":false,"probs":[0.5468997955322266,0.007134067360311747,0.006912106182426214,0.12353987246751785,0.12581691145896912,0.18969732522964478]},{"id":"tv4x_emotion_10","q":0,"variant":"base","tokens":40,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003335416316986084,"mean_abs_dp":0.001132797139386336,"oracle_top2_margin":0.5885082483291626,"near_tie":false,"probs":[0.728154718875885,0.0068547991104424,0.0037343360017985106,0.14435727894306183,0.08138270676136017,0.03551619499921799]},{"id":"tv4x_emotion_11","q":0,"variant":"base","tokens":48,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003441452980041504,"mean_abs_dp":0.0011471541753659646,"oracle_top2_margin":0.710823655128479,"near_tie":false,"probs":[0.008193651214241982,0.814937949180603,0.09893038868904114,0.007241260726004839,0.00932393129914999,0.061372850090265274]},{"id":"tv4x_emotion_12","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004021346569061279,"mean_abs_dp":0.0013404361282785733,"oracle_top2_margin":0.6195278167724609,"near_tie":false,"probs":[0.10196153819561005,0.08669119328260422,0.7264890670776367,0.03580158203840256,0.024995198473334312,0.024061493575572968]},{"id":"tv4x_emotion_13","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":5,"argmax_oracle":5,"argmax_equal":true,"max_abs_dp":0.006184518337249756,"mean_abs_dp":0.0029414392386873565,"oracle_top2_margin":0.2094709724187851,"near_tie":false,"probs":[0.2140069305896759,0.17860978841781616,0.05873999744653702,0.012140543200075626,0.10181232541799545,0.4346904158592224]},{"id":"tv4x_emotion_14","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004852294921875,"mean_abs_dp":0.0016174398673077424,"oracle_top2_margin":0.6627709865570068,"near_tie":false,"probs":[0.11099578440189362,0.05902843549847603,0.7826352119445801,0.013956298120319843,0.0178171806037426,0.015567033551633358]},{"id":"tv4x_emotion_15","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0016972124576568604,"mean_abs_dp":0.0005663071448604265,"oracle_top2_margin":0.007101953029632568,"near_tie":true,"probs":[0.0005979963461868465,0.4902830719947815,0.49407121539115906,0.0008371005533263087,0.0008661759784445167,0.013344401493668556]},{"id":"tv4x_emotion_16","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000783190131187439,"mean_abs_dp":0.00034351910774906475,"oracle_top2_margin":0.7413647174835205,"near_tie":false,"probs":[0.816161572933197,0.004137864802032709,0.04305976256728172,0.07469604164361954,0.03548615425825119,0.026458701118826866]},{"id":"tv4x_emotion_17","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010125517845153809,"mean_abs_dp":0.003375166173403462,"oracle_top2_margin":0.612878680229187,"near_tie":false,"probs":[0.7319605350494385,0.006391820963472128,0.01593816466629505,0.13383691012859344,0.07063030451536179,0.04124227911233902]},{"id":"tv4x_emotion_18","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025801658630371094,"mean_abs_dp":0.0008600554331981888,"oracle_top2_margin":0.7427539229393005,"near_tie":false,"probs":[0.8289760947227478,0.0009356559603475034,0.0011155937099829316,0.05708855763077736,0.08953035622835159,0.022353725507855415]},{"id":"tv4x_emotion_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003303699195384979,"mean_abs_dp":0.0001322093109289805,"oracle_top2_margin":0.8020319938659668,"near_tie":false,"probs":[0.8631535768508911,0.003859300632029772,0.0026670789811760187,0.050607554614543915,0.06138356775045395,0.01832890510559082]},{"id":"tv4x_tweet_offensive_00","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0021613240242004395,"mean_abs_dp":0.0021612942218780518,"oracle_top2_margin":0.19978293776512146,"near_tie":false,"probs":[0.40226981043815613,0.5977301597595215]},{"id":"tv4x_tweet_offensive_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002484619617462158,"mean_abs_dp":0.0024845749139785767,"oracle_top2_margin":0.002196580171585083,"near_tie":true,"probs":[0.49641716480255127,0.5035828948020935]},{"id":"tv4x_tweet_offensive_02","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006822943687438965,"mean_abs_dp":0.0006822850555181503,"oracle_top2_margin":0.9015442132949829,"near_tie":false,"probs":[0.9500898122787476,0.04991019144654274]},{"id":"tv4x_tweet_offensive_03","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020964890718460083,"mean_abs_dp":0.0020964816212654114,"oracle_top2_margin":0.527352511882782,"near_tie":false,"probs":[0.7615798115730286,0.2384202629327774]},{"id":"tv4x_tweet_offensive_04","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005647838115692139,"mean_abs_dp":0.0005647391080856323,"oracle_top2_margin":0.47561752796173096,"near_tie":false,"probs":[0.7383734583854675,0.2616264522075653]},{"id":"tv4x_tweet_offensive_05","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011860504746437073,"mean_abs_dp":0.0011860020458698273,"oracle_top2_margin":0.7690536379814148,"near_tie":false,"probs":[0.8833408355712891,0.11665922403335571]},{"id":"tv4x_tweet_offensive_06","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010499358177185059,"mean_abs_dp":0.0010499358177185059,"oracle_top2_margin":0.142307847738266,"near_tie":false,"probs":[0.5722038745880127,0.4277961552143097]},{"id":"tv4x_tweet_offensive_07","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004083812236785889,"mean_abs_dp":0.004083797335624695,"oracle_top2_margin":0.08455842733383179,"near_tie":false,"probs":[0.5463629961013794,0.4536369740962982]},{"id":"tv4x_tweet_offensive_08","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000311434268951416,"mean_abs_dp":0.00031141936779022217,"oracle_top2_margin":0.1237936019897461,"near_tie":false,"probs":[0.438414603471756,0.5615853667259216]},{"id":"tv4x_tweet_offensive_09","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0031452178955078125,"mean_abs_dp":0.003145188093185425,"oracle_top2_margin":0.06170579791069031,"near_tie":false,"probs":[0.5277076959609985,0.4722922742366791]},{"id":"tv4x_tweet_offensive_10","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007489621639251709,"mean_abs_dp":0.000748947262763977,"oracle_top2_margin":0.0423775315284729,"near_tie":false,"probs":[0.5204398036003113,0.47956016659736633]},{"id":"tv4x_tweet_offensive_11","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012010335922241211,"mean_abs_dp":0.0001200847327709198,"oracle_top2_margin":0.8996016979217529,"near_tie":false,"probs":[0.949680745601654,0.05031919106841087]},{"id":"tv4x_tweet_offensive_12","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003391861915588379,"mean_abs_dp":0.003391861915588379,"oracle_top2_margin":0.0795714259147644,"near_tie":false,"probs":[0.4568224549293518,0.543177604675293]},{"id":"tv4x_tweet_offensive_13","q":0,"variant":"base","tokens":43,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001532435417175293,"mean_abs_dp":0.0015324056148529053,"oracle_top2_margin":0.41108468174934387,"near_tie":false,"probs":[0.7040099501609802,0.29599007964134216]},{"id":"tv4x_tweet_offensive_14","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005246996879577637,"mean_abs_dp":0.0005246400833129883,"oracle_top2_margin":0.25733986496925354,"near_tie":false,"probs":[0.3718547523021698,0.6281453371047974]},{"id":"tv4x_tweet_offensive_15","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014531612396240234,"mean_abs_dp":0.00014531612396240234,"oracle_top2_margin":0.297099232673645,"near_tie":false,"probs":[0.6484043002128601,0.3515956997871399]},{"id":"tv4x_tweet_offensive_16","q":0,"variant":"base","tokens":104,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002887040376663208,"mean_abs_dp":0.002887025475502014,"oracle_top2_margin":0.6911106109619141,"near_tie":false,"probs":[0.15733171999454498,0.8426682949066162]},{"id":"tv4x_tweet_offensive_17","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012363195419311523,"mean_abs_dp":0.0012362897396087646,"oracle_top2_margin":0.13623976707458496,"near_tie":false,"probs":[0.43064379692077637,0.5693561434745789]},{"id":"tv4x_tweet_offensive_18","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002547532320022583,"mean_abs_dp":0.0025474876165390015,"oracle_top2_margin":0.149540513753891,"near_tie":false,"probs":[0.5773177146911621,0.4226822257041931]},{"id":"tv4x_tweet_offensive_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005889534950256348,"mean_abs_dp":0.0005889087915420532,"oracle_top2_margin":0.6824310421943665,"near_tie":false,"probs":[0.84062659740448,0.15937335789203644]},{"id":"tv4x_qnli_00","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.757917046546936e-06,"mean_abs_dp":7.723458111286163e-06,"oracle_top2_margin":0.9797664880752563,"near_tie":false,"probs":[0.010109007358551025,0.9898909330368042]},{"id":"tv4x_qnli_01","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009952187538146973,"mean_abs_dp":0.0009952019900083542,"oracle_top2_margin":0.9650219082832336,"near_tie":false,"probs":[0.9815157651901245,0.01848425529897213]},{"id":"tv4x_qnli_02","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00032879412174224854,"mean_abs_dp":0.0003287866711616516,"oracle_top2_margin":0.7253044843673706,"near_tie":false,"probs":[0.13701896369457245,0.8629810214042664]},{"id":"tv4x_qnli_03","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010319948196411133,"mean_abs_dp":0.0010319370776414871,"oracle_top2_margin":0.958515465259552,"near_tie":false,"probs":[0.9782257676124573,0.021774159744381905]},{"id":"tv4x_qnli_04","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001148562878370285,"mean_abs_dp":0.0011485423892736435,"oracle_top2_margin":0.8948183059692383,"near_tie":false,"probs":[0.05373938009142876,0.9462606310844421]},{"id":"tv4x_qnli_05","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018166080117225647,"mean_abs_dp":0.0018165893852710724,"oracle_top2_margin":0.756822943687439,"near_tie":false,"probs":[0.12340513616800308,0.8765949010848999]},{"id":"tv4x_qnli_06","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0017376542091369629,"mean_abs_dp":0.0017376244068145752,"oracle_top2_margin":0.3450804054737091,"near_tie":false,"probs":[0.6708025932312012,0.3291974365711212]},{"id":"tv4x_qnli_07","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.011010289192199707,"mean_abs_dp":0.011010274291038513,"oracle_top2_margin":0.5026953220367432,"near_tie":false,"probs":[0.7403373718261719,0.2596626281738281]},{"id":"tv4x_qnli_08","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.959936141967773e-05,"mean_abs_dp":9.957863949239254e-05,"oracle_top2_margin":0.9887792468070984,"near_tie":false,"probs":[0.9944891929626465,0.005510792136192322]},{"id":"tv4x_qnli_09","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.524822235107422e-05,"mean_abs_dp":9.52468253672123e-05,"oracle_top2_margin":0.9746768474578857,"near_tie":false,"probs":[0.9872431755065918,0.012756796553730965]},{"id":"tv4x_qnli_10","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00034993886947631836,"mean_abs_dp":0.0003499002195894718,"oracle_top2_margin":0.9804186224937439,"near_tie":false,"probs":[0.989859402179718,0.010140554048120975]},{"id":"tv4x_qnli_11","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005839020013809204,"mean_abs_dp":0.0005838945508003235,"oracle_top2_margin":0.8205903768539429,"near_tie":false,"probs":[0.09028871357440948,0.9097113013267517]},{"id":"tv4x_qnli_12","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.897615432739258e-05,"mean_abs_dp":7.896497845649719e-05,"oracle_top2_margin":0.8703594207763672,"near_tie":false,"probs":[0.06474132090806961,0.935258686542511]},{"id":"tv4x_qnli_13","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005629122257232666,"mean_abs_dp":0.005629092454910278,"oracle_top2_margin":0.45642751455307007,"near_tie":false,"probs":[0.2661571502685547,0.7338428497314453]},{"id":"tv4x_qnli_14","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004037618637084961,"mean_abs_dp":0.0040376149117946625,"oracle_top2_margin":0.8329219222068787,"near_tie":false,"probs":[0.07950142025947571,0.9204985499382019]},{"id":"tv4x_qnli_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009546652436256409,"mean_abs_dp":0.0009546466171741486,"oracle_top2_margin":0.8420214056968689,"near_tie":false,"probs":[0.0780346542596817,0.9219653606414795]},{"id":"tv4x_qnli_16","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002643465995788574,"mean_abs_dp":0.00026428699493408203,"oracle_top2_margin":0.47870561480522156,"near_tie":false,"probs":[0.2609114944934845,0.7390885353088379]},{"id":"tv4x_qnli_17","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011840090155601501,"mean_abs_dp":0.0001183878630399704,"oracle_top2_margin":0.9677414894104004,"near_tie":false,"probs":[0.01624765619635582,0.9837523698806763]},{"id":"tv4x_qnli_18","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014424324035644531,"mean_abs_dp":0.0014424026012420654,"oracle_top2_margin":0.8940520882606506,"near_tie":false,"probs":[0.9455836415290833,0.05441638082265854]},{"id":"tv4x_qnli_19","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.016363978385925293,"mean_abs_dp":0.016363970935344696,"oracle_top2_margin":0.6079574823379517,"near_tie":false,"probs":[0.17965728044509888,0.8203427195549011]},{"id":"tv4x_paws_00","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004925131797790527,"mean_abs_dp":0.004925131797790527,"oracle_top2_margin":0.23918089270591736,"near_tie":false,"probs":[0.375484436750412,0.6245155930519104]},{"id":"tv4x_paws_01","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.490945816040039e-05,"mean_abs_dp":6.489455699920654e-05,"oracle_top2_margin":0.007073014974594116,"near_tie":true,"probs":[0.49652838706970215,0.5034716129302979]},{"id":"tv4x_paws_02","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006109941750764847,"mean_abs_dp":0.0006109708920121193,"oracle_top2_margin":0.9475407600402832,"near_tie":false,"probs":[0.9731594324111938,0.02684061974287033]},{"id":"tv4x_paws_03","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00030268169939517975,"mean_abs_dp":0.000302647240459919,"oracle_top2_margin":0.9485864639282227,"near_tie":false,"probs":[0.02600943110883236,0.9739906191825867]},{"id":"tv4x_paws_04","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.960200309753418e-05,"mean_abs_dp":7.958710193634033e-05,"oracle_top2_margin":0.8732220530509949,"near_tie":false,"probs":[0.06330939382314682,0.936690628528595]},{"id":"tv4x_paws_05","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000556647777557373,"mean_abs_dp":0.0005566403269767761,"oracle_top2_margin":0.6148557662963867,"near_tie":false,"probs":[0.193128764629364,0.806871235370636]},{"id":"tv4x_paws_06","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004228949546813965,"mean_abs_dp":0.00042288005352020264,"oracle_top2_margin":0.45639950037002563,"near_tie":false,"probs":[0.2722231447696686,0.7277768850326538]},{"id":"tv4x_paws_07","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008103847503662109,"mean_abs_dp":0.0008103270083665848,"oracle_top2_margin":0.925515353679657,"near_tie":false,"probs":[0.03805260360240936,0.9619473218917847]},{"id":"tv4x_paws_08","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006938166916370392,"mean_abs_dp":0.000693807378411293,"oracle_top2_margin":0.9070258140563965,"near_tie":false,"probs":[0.04718093201518059,0.9528191089630127]},{"id":"tv4x_paws_09","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0037926435470581055,"mean_abs_dp":0.0037926435470581055,"oracle_top2_margin":0.671048641204834,"near_tie":false,"probs":[0.16826830804347992,0.8317316770553589]},{"id":"tv4x_paws_10","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016269981861114502,"mean_abs_dp":0.0016269832849502563,"oracle_top2_margin":0.7289464473724365,"near_tie":false,"probs":[0.8661001920700073,0.1338997781276703]},{"id":"tv4x_paws_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010051727294921875,"mean_abs_dp":0.0010051373392343521,"oracle_top2_margin":0.8897872567176819,"near_tie":false,"probs":[0.056111495941877365,0.9438884854316711]},{"id":"tv4x_paws_12","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005915597081184387,"mean_abs_dp":0.00591558963060379,"oracle_top2_margin":0.524033784866333,"near_tie":false,"probs":[0.24389873445034027,0.7561013102531433]},{"id":"tv4x_paws_13","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013309121131896973,"mean_abs_dp":0.0013308636844158173,"oracle_top2_margin":0.8085752129554749,"near_tie":false,"probs":[0.0970432236790657,0.9029567241668701]},{"id":"tv4x_paws_14","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006761908531188965,"mean_abs_dp":0.006761878728866577,"oracle_top2_margin":0.2491759955883026,"near_tie":false,"probs":[0.3686501085758209,0.6313498616218567]},{"id":"tv4x_paws_15","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004499346017837524,"mean_abs_dp":0.004499331116676331,"oracle_top2_margin":0.6705261468887329,"near_tie":false,"probs":[0.16923627257347107,0.8307637572288513]},{"id":"tv4x_paws_16","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007090568542480469,"mean_abs_dp":0.0007090121507644653,"oracle_top2_margin":0.4153684675693512,"near_tie":false,"probs":[0.2916067838668823,0.7083932757377625]},{"id":"tv4x_paws_17","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003806948661804199,"mean_abs_dp":0.000380687415599823,"oracle_top2_margin":0.8059070110321045,"near_tie":false,"probs":[0.09666583687067032,0.9033342003822327]},{"id":"tv4x_paws_18","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00036919116973876953,"mean_abs_dp":0.0003691879101097584,"oracle_top2_margin":0.9809930324554443,"near_tie":false,"probs":[0.9901273250579834,0.009872646071016788]},{"id":"tv4x_paws_19","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013374090194702148,"mean_abs_dp":0.0013374090194702148,"oracle_top2_margin":0.8185950517654419,"near_tie":false,"probs":[0.09203991293907166,0.9079601168632507]},{"id":"tv4x_sciq_00","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00028628110885620117,"mean_abs_dp":0.00014313324936665595,"oracle_top2_margin":0.9769152402877808,"near_tie":false,"probs":[0.0014491373440250754,0.9854097366333008,0.008949687704443932,0.004191409330815077]},{"id":"tv4x_sciq_01","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.003548741340637207,"mean_abs_dp":0.0017743781208992004,"oracle_top2_margin":0.7016881108283997,"near_tie":false,"probs":[0.08298977464437485,0.07371028512716293,0.06369133293628693,0.7796086668968201]},{"id":"tv4x_sciq_02","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.339559331536293e-05,"mean_abs_dp":4.684453597292304e-05,"oracle_top2_margin":0.9870290160179138,"near_tie":false,"probs":[0.9913920164108276,0.00448076706379652,0.0014410016592592,0.0026862164959311485]},{"id":"tv4x_sciq_03","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0006895288825035095,"mean_abs_dp":0.00034584160312078893,"oracle_top2_margin":0.944787323474884,"near_tie":false,"probs":[0.001772513147443533,0.003373732790350914,0.025716878473758698,0.9691369533538818]},{"id":"tv4x_sciq_04","q":0,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.011251330375671387,"mean_abs_dp":0.005625669960863888,"oracle_top2_margin":0.6779905557632446,"near_tie":false,"probs":[0.004193088039755821,0.8217470049858093,0.1654670089483261,0.008592874743044376]},{"id":"tv4x_sciq_05","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008299946784973145,"mean_abs_dp":0.00041496753692626953,"oracle_top2_margin":0.947314977645874,"near_tie":false,"probs":[0.9631922841072083,0.014637134969234467,0.008151218295097351,0.014019450172781944]},{"id":"tv4x_sciq_06","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00019818544387817383,"mean_abs_dp":9.909248910844326e-05,"oracle_top2_margin":0.9827218055725098,"near_tie":false,"probs":[0.00143788184504956,0.9880254864692688,0.00556171266362071,0.004974978975951672]},{"id":"tv4x_sciq_07","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001373887062072754,"mean_abs_dp":6.870049401186407e-05,"oracle_top2_margin":0.9827112555503845,"near_tie":false,"probs":[0.0009685028344392776,0.9896928668022156,0.002600062405690551,0.006738628726452589]},{"id":"tv4x_sciq_08","q":0,"variant":"base","tokens":162,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016231536865234375,"mean_abs_dp":0.0008115970995277166,"oracle_top2_margin":0.8948369026184082,"near_tie":false,"probs":[0.93643718957901,0.039021823555231094,0.009741812944412231,0.014799188822507858]},{"id":"tv4x_sciq_09","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00011098384857177734,"mean_abs_dp":7.606251165270805e-05,"oracle_top2_margin":0.98060542345047,"near_tie":false,"probs":[0.0037187563721090555,0.005500931292772293,0.004743628203868866,0.9860365986824036]},{"id":"tv4x_sciq_10","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":1.253560185432434e-05,"mean_abs_dp":7.726805051788688e-06,"oracle_top2_margin":0.9585868716239929,"near_tie":false,"probs":[0.017861301079392433,0.005287379492074251,0.00038737821159884334,0.9764639139175415]},{"id":"tv4x_sciq_11","q":0,"variant":"base","tokens":366,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005287528038024902,"mean_abs_dp":0.00026939915551338345,"oracle_top2_margin":0.9739360213279724,"near_tie":false,"probs":[0.010795497335493565,0.004436654970049858,0.9838129281997681,0.0009549533133395016]},{"id":"tv4x_sciq_12","q":0,"variant":"base","tokens":192,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000826418399810791,"mean_abs_dp":0.0004132262256462127,"oracle_top2_margin":0.9150312542915344,"near_tie":false,"probs":[0.0017987708561122417,0.006758098490536213,0.9539990425109863,0.0374440923333168]},{"id":"tv4x_sciq_13","q":0,"variant":"base","tokens":147,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00013308972120285034,"mean_abs_dp":6.649713031947613e-05,"oracle_top2_margin":0.966279149055481,"near_tie":false,"probs":[0.9761301875114441,0.009799235500395298,0.006650245748460293,0.00742042763158679]},{"id":"tv4x_sciq_14","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00019073486328125,"mean_abs_dp":0.00010058192128781229,"oracle_top2_margin":0.9916238784790039,"near_tie":false,"probs":[0.0031000489834696054,0.0011837896890938282,0.0006124621722847223,0.9951037168502808]},{"id":"tv4x_sciq_15","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00030809640884399414,"mean_abs_dp":0.00015406750026158988,"oracle_top2_margin":0.9649361371994019,"near_tie":false,"probs":[0.0014330242993310094,0.977549135684967,0.013046628795564175,0.00797125231474638]},{"id":"tv4x_sciq_16","q":0,"variant":"base","tokens":335,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004470348358154297,"mean_abs_dp":0.0022351527586579323,"oracle_top2_margin":0.41051337122917175,"near_tie":false,"probs":[0.2471802830696106,0.027746912091970444,0.07518969476222992,0.6498830914497375]},{"id":"tv4x_sciq_17","q":0,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00017136335372924805,"mean_abs_dp":8.568714838474989e-05,"oracle_top2_margin":0.9674238562583923,"near_tie":false,"probs":[0.9770601987838745,0.009409763850271702,0.007112895604223013,0.006417143624275923]},{"id":"tv4x_sciq_18","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000253140926361084,"mean_abs_dp":0.000126561033539474,"oracle_top2_margin":0.9668787121772766,"near_tie":false,"probs":[0.9782800078392029,0.010944853536784649,0.004068276844918728,0.006706821732223034]},{"id":"tv4x_sciq_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.2659306675195694e-05,"mean_abs_dp":2.8175360057502985e-05,"oracle_top2_margin":0.9877200126647949,"near_tie":false,"probs":[0.9919833540916443,0.0015583904460072517,0.0021975100971758366,0.004260729532688856]},{"id":"tv4x_legacy_holdout_00","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020662415772676468,"mean_abs_dp":0.0002066069282591343,"oracle_top2_margin":0.9870973825454712,"near_tie":false,"probs":[0.006657944526523352,0.9933421015739441]},{"id":"tv4x_legacy_holdout_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.094839096069336e-05,"mean_abs_dp":4.092790186405182e-05,"oracle_top2_margin":0.9887204766273499,"near_tie":false,"probs":[0.9943193197250366,0.0056807007640600204]},{"id":"tv4x_legacy_holdout_02","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006040334701538086,"mean_abs_dp":0.0006040316075086594,"oracle_top2_margin":0.9801822304725647,"near_tie":false,"probs":[0.010512912645936012,0.9894870519638062]},{"id":"tv4x_legacy_holdout_03","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.206255733966827e-05,"mean_abs_dp":8.203927427530289e-05,"oracle_top2_margin":0.9842581152915955,"near_tie":false,"probs":[0.992211103439331,0.007788919843733311]},{"id":"tv4x_legacy_holdout_04","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020114192739129066,"mean_abs_dp":0.00020112399943172932,"oracle_top2_margin":0.985432505607605,"near_tie":false,"probs":[0.007484880276024342,0.9925151467323303]},{"id":"tv4x_legacy_holdout_05","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.9020400941371918e-05,"mean_abs_dp":1.9017141312360764e-05,"oracle_top2_margin":0.9865952730178833,"near_tie":false,"probs":[0.993316650390625,0.006683372892439365]},{"id":"tv4x_legacy_holdout_06","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020415615290403366,"mean_abs_dp":0.00020412122830748558,"oracle_top2_margin":0.9899900555610657,"near_tie":false,"probs":[0.005209118127822876,0.9947909116744995]},{"id":"tv4x_legacy_holdout_07","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.875659942626953e-05,"mean_abs_dp":4.869664553552866e-05,"oracle_top2_margin":0.9943817257881165,"near_tie":false,"probs":[0.9971421360969543,0.0028577959164977074]},{"id":"tv4x_legacy_holdout_08","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00023330096155405045,"mean_abs_dp":0.00023329677060246468,"oracle_top2_margin":0.9859328866004944,"near_tie":false,"probs":[0.007266852539032698,0.9927331209182739]},{"id":"tv4x_legacy_holdout_09","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.991997361183167e-05,"mean_abs_dp":6.988830864429474e-05,"oracle_top2_margin":0.9869430065155029,"near_tie":false,"probs":[0.9935413599014282,0.0064585888758301735]},{"id":"tv4x_legacy_holdout_10","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005705952644348145,"mean_abs_dp":0.0005705459043383598,"oracle_top2_margin":0.981930673122406,"near_tie":false,"probs":[0.009605186991393566,0.9903947710990906]},{"id":"tv4x_legacy_holdout_11","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.085634231567383e-05,"mean_abs_dp":6.083888001739979e-05,"oracle_top2_margin":0.9873005747795105,"near_tie":false,"probs":[0.9937111139297485,0.0062888856045901775]},{"id":"tv4x_legacy_holdout_12","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002963542938232422,"mean_abs_dp":0.00029635406099259853,"oracle_top2_margin":0.9882665872573853,"near_tie":false,"probs":[0.006163057405501604,0.9938369393348694]},{"id":"tv4x_legacy_holdout_13","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.450580596923828e-05,"mean_abs_dp":7.450557313859463e-05,"oracle_top2_margin":0.9917652010917664,"near_tie":false,"probs":[0.9958081245422363,0.004191910848021507]},{"id":"tv4x_legacy_holdout_14","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001767938956618309,"mean_abs_dp":0.00017676083371043205,"oracle_top2_margin":0.9891088008880615,"near_tie":false,"probs":[0.005622366443276405,0.9943776726722717]},{"id":"tv4x_legacy_holdout_15","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001283939927816391,"mean_abs_dp":0.00012839119881391525,"oracle_top2_margin":0.9911453127861023,"near_tie":false,"probs":[0.9954442977905273,0.004555739928036928]},{"id":"tv4x_legacy_holdout_16","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001983046531677246,"mean_abs_dp":0.0001982895191758871,"oracle_top2_margin":0.9885526895523071,"near_tie":false,"probs":[0.005921950563788414,0.9940780401229858]},{"id":"tv4x_legacy_holdout_17","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.875131607055664e-05,"mean_abs_dp":8.870917372405529e-05,"oracle_top2_margin":0.9869370460510254,"near_tie":false,"probs":[0.9935572743415833,0.0064427899196743965]},{"id":"tv4x_legacy_holdout_18","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00019860267639160156,"mean_abs_dp":0.0001985873095691204,"oracle_top2_margin":0.9901125431060791,"near_tie":false,"probs":[0.005142305977642536,0.994857668876648]},{"id":"tv4x_legacy_holdout_19","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.9508879631757736e-06,"mean_abs_dp":1.929118297994137e-06,"oracle_top2_margin":0.9932237863540649,"near_tie":false,"probs":[0.9966138005256653,0.003386151045560837]},{"id":"tv4x_composition_holdout_00","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004932880401611328,"mean_abs_dp":0.0004932819865643978,"oracle_top2_margin":0.9758310317993164,"near_tie":false,"probs":[0.012577757239341736,0.9874222278594971]},{"id":"tv4x_composition_holdout_01","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003303922712802887,"mean_abs_dp":0.00033039040863513947,"oracle_top2_margin":0.9565773010253906,"near_tie":false,"probs":[0.9779582619667053,0.022041764110326767]},{"id":"tv4x_composition_holdout_02","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004932880401611328,"mean_abs_dp":0.0004932819865643978,"oracle_top2_margin":0.9758310317993164,"near_tie":false,"probs":[0.012577757239341736,0.9874222278594971]},{"id":"tv4x_composition_holdout_03","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004759766161441803,"mean_abs_dp":0.0004759598523378372,"oracle_top2_margin":0.9759304523468018,"near_tie":false,"probs":[0.012510759755969048,0.9874892830848694]},{"id":"tv4x_composition_holdout_04","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001952052116394043,"mean_abs_dp":0.0001952000893652439,"oracle_top2_margin":0.9824920296669006,"near_tie":false,"probs":[0.008949189446866512,0.9910508394241333]},{"id":"tv4x_composition_holdout_05","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00033092498779296875,"mean_abs_dp":0.0003308975137770176,"oracle_top2_margin":0.9744811654090881,"near_tie":false,"probs":[0.9869096875190735,0.013090310618281364]},{"id":"tv4x_composition_holdout_06","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001952052116394043,"mean_abs_dp":0.0001952000893652439,"oracle_top2_margin":0.9824920296669006,"near_tie":false,"probs":[0.008949189446866512,0.9910508394241333]},{"id":"tv4x_composition_holdout_07","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00018600374460220337,"mean_abs_dp":0.00018598511815071106,"oracle_top2_margin":0.9825741052627563,"near_tie":false,"probs":[0.008898969739675522,0.991101086139679]},{"id":"tv4x_composition_holdout_08","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0052925944328308105,"mean_abs_dp":0.005292564630508423,"oracle_top2_margin":0.16480392217636108,"near_tie":false,"probs":[0.41230547428131104,0.587694525718689]},{"id":"tv4x_composition_holdout_09","q":0,"variant":"base","tokens":167,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008279606699943542,"mean_abs_dp":0.000827934592962265,"oracle_top2_margin":0.960469126701355,"near_tie":false,"probs":[0.9794066548347473,0.020593414083123207]},{"id":"tv4x_composition_holdout_10","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0052925944328308105,"mean_abs_dp":0.005292564630508423,"oracle_top2_margin":0.16480392217636108,"near_tie":false,"probs":[0.41230547428131104,0.587694525718689]},{"id":"tv4x_composition_holdout_11","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0053345561027526855,"mean_abs_dp":0.005334541201591492,"oracle_top2_margin":0.1781402826309204,"near_tie":false,"probs":[0.4055953323841095,0.5944046974182129]},{"id":"tv4x_composition_holdout_12","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003418922424316406,"mean_abs_dp":0.0003418680280447006,"oracle_top2_margin":0.9754068851470947,"near_tie":false,"probs":[0.9873615503311157,0.012638413347303867]},{"id":"tv4x_composition_holdout_13","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.010656982660293579,"mean_abs_dp":0.010656937956809998,"oracle_top2_margin":0.5981696844100952,"near_tie":false,"probs":[0.1902582049369812,0.809741735458374]},{"id":"tv4x_composition_holdout_14","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003418922424316406,"mean_abs_dp":0.0003418680280447006,"oracle_top2_margin":0.9754068851470947,"near_tie":false,"probs":[0.9873615503311157,0.012638413347303867]},{"id":"tv4x_composition_holdout_15","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00020897388458251953,"mean_abs_dp":0.00020895013585686684,"oracle_top2_margin":0.9755581021308899,"near_tie":false,"probs":[0.9875701069831848,0.012429889291524887]},{"id":"tv4x_composition_holdout_16","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001010894775390625,"mean_abs_dp":0.0001010894775390625,"oracle_top2_margin":0.751534104347229,"near_tie":false,"probs":[0.12413188070058823,0.8758681416511536]},{"id":"tv4x_composition_holdout_17","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001248866319656372,"mean_abs_dp":0.00012487918138504028,"oracle_top2_margin":0.9779067039489746,"near_tie":false,"probs":[0.9890782237052917,0.010921750217676163]},{"id":"tv4x_composition_holdout_18","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001010894775390625,"mean_abs_dp":0.0001010894775390625,"oracle_top2_margin":0.751534104347229,"near_tie":false,"probs":[0.12413188070058823,0.8758681416511536]},{"id":"tv4x_composition_holdout_19","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005483627319335938,"mean_abs_dp":0.0005483031272888184,"oracle_top2_margin":0.7390268445014954,"near_tie":false,"probs":[0.1299383044242859,0.8700617551803589]},{"id":"tv4s_00","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010720491409301758,"mean_abs_dp":0.0007146727293729782,"oracle_top2_margin":0.9141930341720581,"near_tie":false,"probs":[0.9437233805656433,0.031661443412303925,0.024615183472633362]},{"id":"tv4s_01","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.012888491153717041,"mean_abs_dp":0.00859232929845651,"oracle_top2_margin":0.7866014242172241,"near_tie":false,"probs":[0.08067905157804489,0.8921003937721252,0.02722063660621643]},{"id":"tv4s_02","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008883237838745117,"mean_abs_dp":0.005922166009744008,"oracle_top2_margin":0.13063779473304749,"near_tie":false,"probs":[0.40360113978385925,0.5181118845939636,0.07828687876462936]},{"id":"tv4s_03","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005920529365539551,"mean_abs_dp":0.000394708476960659,"oracle_top2_margin":0.9156948924064636,"near_tie":false,"probs":[0.0371645949780941,0.9518625736236572,0.010972847230732441]},{"id":"tv4s_04","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005485594272613525,"mean_abs_dp":0.003657070298989614,"oracle_top2_margin":0.741247832775116,"near_tie":false,"probs":[0.8222132921218872,0.09084644913673401,0.08694025129079819]},{"id":"tv4s_05","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00735783576965332,"mean_abs_dp":0.004905210807919502,"oracle_top2_margin":0.8588613271713257,"near_tie":false,"probs":[0.055101264268159866,0.92807936668396,0.016819413751363754]},{"id":"tv4s_06","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005607008934020996,"mean_abs_dp":0.00037380556265513104,"oracle_top2_margin":0.14654552936553955,"near_tie":false,"probs":[0.33811789751052856,0.4857082664966583,0.17617382109165192]},{"id":"tv4s_07","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008019506931304932,"mean_abs_dp":0.00534631684422493,"oracle_top2_margin":0.5599085688591003,"near_tie":false,"probs":[0.008671220391988754,0.7678031325340271,0.22352562844753265]},{"id":"tv4s_08","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0078505277633667,"mean_abs_dp":0.005233710010846456,"oracle_top2_margin":0.13977158069610596,"near_tie":false,"probs":[0.417924165725708,0.5427173972129822,0.039358481764793396]},{"id":"tv4s_09","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016602277755737305,"mean_abs_dp":0.0011067700882752736,"oracle_top2_margin":0.8021276593208313,"near_tie":false,"probs":[0.09181004017591476,0.8910801410675049,0.01710973121225834]},{"id":"tv4s_10","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0023200511932373047,"mean_abs_dp":0.0015466958284378052,"oracle_top2_margin":0.3599908947944641,"near_tie":false,"probs":[0.2017887830734253,0.5810502767562866,0.2171609103679657]},{"id":"tv4s_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":2,"argmax_equal":false,"max_abs_dp":0.006644874811172485,"mean_abs_dp":0.0044299329941471415,"oracle_top2_margin":0.001054227352142334,"near_tie":true,"probs":[0.010619686916470528,0.500781238079071,0.4885990619659424]},{"id":"tv4s_12","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005569100379943848,"mean_abs_dp":0.0037127205481131873,"oracle_top2_margin":0.919794499874115,"near_tie":false,"probs":[0.9460564851760864,0.03581183776259422,0.018131662160158157]},{"id":"tv4s_13","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006182193756103516,"mean_abs_dp":0.0004121493548154831,"oracle_top2_margin":0.9661151766777039,"near_tie":false,"probs":[0.011155601590871811,0.9769699573516846,0.011874396353960037]},{"id":"tv4s_14","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0067758262157440186,"mean_abs_dp":0.004517228653033574,"oracle_top2_margin":0.541414201259613,"near_tie":false,"probs":[0.21222034096717834,0.7408722639083862,0.04690738767385483]},{"id":"tv4s_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005224347114562988,"mean_abs_dp":0.0003483276814222336,"oracle_top2_margin":0.9349822998046875,"near_tie":false,"probs":[0.027967602014541626,0.9619436264038086,0.0100887855514884]},{"id":"tv4s_16","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004495382308959961,"mean_abs_dp":0.002996911605199178,"oracle_top2_margin":0.3046940267086029,"near_tie":false,"probs":[0.6311078667640686,0.3346681296825409,0.03422399237751961]},{"id":"tv4s_17","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001416921615600586,"mean_abs_dp":0.0009445777783791224,"oracle_top2_margin":0.683984100818634,"near_tie":false,"probs":[0.019355300813913345,0.8311063051223755,0.14953838288784027]},{"id":"tv4s_18","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008234381675720215,"mean_abs_dp":0.0005489645215372244,"oracle_top2_margin":0.9713500738143921,"near_tie":false,"probs":[0.9810723662376404,0.01108715869486332,0.007840550504624844]},{"id":"tv4s_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00036984682083129883,"mean_abs_dp":0.0002465580279628436,"oracle_top2_margin":0.9682061076164246,"near_tie":false,"probs":[0.01262305211275816,0.9815489649772644,0.0058279745280742645]},{"id":"semif_a3f18f3a63d45345942b","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003154836595058441,"mean_abs_dp":0.0002103366811449329,"oracle_top2_margin":0.9753907322883606,"near_tie":false,"probs":[0.0011957623064517975,0.01140426192432642,0.9874000549316406],"hidden":{"min_pos_cos":0.9972377129605107,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"semif_f40beba9088c8db8bbd6","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012555122375488281,"mean_abs_dp":0.0008370297340055307,"oracle_top2_margin":0.9538690447807312,"near_tie":false,"probs":[0.9769221544265747,0.020648805424571037,0.0024289595894515514]},{"id":"semif_0b43ea8e24e74d621f1b","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002281665802001953,"mean_abs_dp":0.00015209145688762268,"oracle_top2_margin":0.9681141972541809,"near_tie":false,"probs":[0.015171232633292675,0.9837369322776794,0.0010919406777247787]},{"id":"semif_942ec65dac0bfb9874f9","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0026593804359436035,"mean_abs_dp":0.0017729409349461396,"oracle_top2_margin":0.24663254618644714,"near_tie":false,"probs":[0.37194201350212097,0.004177233669906855,0.6238807439804077]},{"id":"semif_e0c140e9222d1bce5647","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.195297464728355e-05,"mean_abs_dp":4.1320008070518575e-05,"oracle_top2_margin":0.9878595471382141,"near_tie":false,"probs":[0.9933905601501465,0.0009664060198701918,0.005642978008836508]},{"id":"semif_8e8c3804a3c15ebb31e3","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.01799994707107544,"mean_abs_dp":0.011999970922867456,"oracle_top2_margin":0.47083574533462524,"near_tie":false,"probs":[0.26537832617759705,0.7027013301849365,0.03192036971449852]},{"id":"semif_56c419b12cd37cbe790c","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00015079136937856674,"mean_abs_dp":0.00010055024176836014,"oracle_top2_margin":0.8864083290100098,"near_tie":false,"probs":[0.9407040476799011,0.05429467558860779,0.005001222714781761]},{"id":"semif_75b470586834df8c568d","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005183570086956024,"mean_abs_dp":0.003455717737476031,"oracle_top2_margin":0.840787410736084,"near_tie":false,"probs":[0.014415829442441463,0.07757105678319931,0.908013105392456]},{"id":"semif_834f3b3e10cc61b33d25","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.5093555450439453e-05,"mean_abs_dp":1.6697042156010866e-05,"oracle_top2_margin":0.9962265491485596,"near_tie":false,"probs":[0.0005381216760724783,0.0016357868444174528,0.9978260397911072]},{"id":"semif_76c7a79b1ff2363a60c5","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00016701221466064453,"mean_abs_dp":0.00011133472435176373,"oracle_top2_margin":0.9867470860481262,"near_tie":false,"probs":[0.9931053519248962,0.006052919197827578,0.0008417727658525109]},{"id":"semif_36c71bb649b7d5eccb1f","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.8926803022623062e-05,"mean_abs_dp":1.2630907197793325e-05,"oracle_top2_margin":0.9969070553779602,"near_tie":false,"probs":[0.9980962872505188,0.001187823130749166,0.0007159408996812999]},{"id":"semif_eac5121a4a1bf391a08c","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00014227628707885742,"mean_abs_dp":9.481896025439103e-05,"oracle_top2_margin":0.9827219843864441,"near_tie":false,"probs":[0.006590570788830519,0.9895936250686646,0.003815828822553158]},{"id":"semif_44574c44e243ee1b2f52","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013458728790283203,"mean_abs_dp":8.972780779004097e-05,"oracle_top2_margin":0.965580940246582,"near_tie":false,"probs":[0.015464726835489273,0.003271976951509714,0.981263279914856]},{"id":"semif_2608370d01783735c3ea","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003516256809234619,"mean_abs_dp":0.0023442114858577647,"oracle_top2_margin":0.9253455400466919,"near_tie":false,"probs":[0.0023535105865448713,0.03273424133658409,0.9649121761322021]},{"id":"semif_ec94ef909f7918695ab3","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005313027650117874,"mean_abs_dp":0.0003542135236784816,"oracle_top2_margin":0.9795559644699097,"near_tie":false,"probs":[0.0006833229563198984,0.00935320369899273,0.9899634718894958]},{"id":"semif_03d919fd0e9b3a7cd1e6","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011544208973646164,"mean_abs_dp":7.698747018973033e-05,"oracle_top2_margin":0.9788561463356018,"near_tie":false,"probs":[0.9863320589065552,0.006087211892008781,0.007580673322081566]},{"id":"semif_86a15618ba4cb90a307e","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00195467472076416,"mean_abs_dp":0.001303119972969095,"oracle_top2_margin":0.8801754713058472,"near_tie":false,"probs":[0.9371266961097717,0.060806069523096085,0.0020673174876719713]},{"id":"semif_3dd9c3ac98f8ddf7bdd3","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003404974937438965,"mean_abs_dp":0.0022700165087978044,"oracle_top2_margin":0.8153263330459595,"near_tie":false,"probs":[0.015183606185019016,0.9032025933265686,0.0816136971116066]},{"id":"semif_36273dd4777876a26338","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00023049116134643555,"mean_abs_dp":0.00015366729348897934,"oracle_top2_margin":0.975105345249176,"near_tie":false,"probs":[0.0028760218992829323,0.9859008193016052,0.011223104782402515]},{"id":"semif_07ee70cfe4b01330ce89","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006068944931030273,"mean_abs_dp":0.00040458887815475464,"oracle_top2_margin":0.9030753374099731,"near_tie":false,"probs":[0.008313922211527824,0.04381487891077995,0.947871208190918]},{"id":"semif_51ceab3ce622c7c759f1","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.566118776798248e-05,"mean_abs_dp":5.711925526460012e-05,"oracle_top2_margin":0.9728904366493225,"near_tie":false,"probs":[0.005852450150996447,0.010552972555160522,0.9835945963859558]},{"id":"semif_b8091be929b1e489fcc4","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001386888325214386,"mean_abs_dp":9.246193803846836e-05,"oracle_top2_margin":0.9673788547515869,"near_tie":false,"probs":[0.0015529044903814793,0.983045220375061,0.015401923097670078]},{"id":"semif_c89b86e3f55123bfa506","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.066394805908203e-05,"mean_abs_dp":3.37787593404452e-05,"oracle_top2_margin":0.9869573712348938,"near_tie":false,"probs":[0.0003453840618021786,0.9932591915130615,0.006395501550287008]},{"id":"semif_3b0958cabc339005cf66","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006486903876066208,"mean_abs_dp":0.00043245529135068256,"oracle_top2_margin":0.9173914790153503,"near_tie":false,"probs":[0.9485482573509216,0.020059606060385704,0.03139214962720871]},{"id":"semif_f2b4ec4930322fe45118","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001893758773803711,"mean_abs_dp":0.0012624713902672131,"oracle_top2_margin":0.8798353672027588,"near_tie":false,"probs":[0.05990636721253395,0.9361506104469299,0.003942976240068674]},{"id":"semif_61ac20b07a5189231e15","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001497427001595497,"mean_abs_dp":9.98651764045159e-05,"oracle_top2_margin":0.9755754470825195,"near_tie":false,"probs":[0.0038797392044216394,0.9859808683395386,0.01013945322483778]},{"id":"semif_4b3d794c004566610181","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":9.106239303946495e-05,"mean_abs_dp":6.069910402099291e-05,"oracle_top2_margin":0.9726617336273193,"near_tie":false,"probs":[0.004009806085377932,0.011707736179232597,0.9842824339866638]},{"id":"semif_b7c4e62d07986286fce0","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008961021900177002,"mean_abs_dp":0.00597402701775233,"oracle_top2_margin":0.40284138917922974,"near_tie":false,"probs":[0.6334364414215088,0.12003916501998901,0.246524378657341]},{"id":"semif_9473603b44af63beab79","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018575787544250488,"mean_abs_dp":0.0012383867676059406,"oracle_top2_margin":0.8802680969238281,"near_tie":false,"probs":[0.9378317594528198,0.05410420149564743,0.008064051158726215]},{"id":"semif_d41072dc35d8cd14c056","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.182786405086517e-05,"mean_abs_dp":5.459443976481756e-05,"oracle_top2_margin":0.9784180521965027,"near_tie":false,"probs":[0.006638139486312866,0.9859716296195984,0.007390281651169062]},{"id":"semif_33d6e5da58f0fee2490d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.014631807804107666,"mean_abs_dp":0.009754528601964315,"oracle_top2_margin":0.12864628434181213,"near_tie":false,"probs":[0.18535056710243225,0.4590822458267212,0.35556718707084656]},{"id":"semif_823bf736b92e9d8fa66d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006167858839035034,"mean_abs_dp":0.0041118959585825605,"oracle_top2_margin":0.1685684323310852,"near_tie":false,"probs":[0.2693066895008087,0.44801467657089233,0.28267866373062134]},{"id":"semif_8a4c3de28be95c105ca1","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003227710723876953,"mean_abs_dp":0.002151822050412496,"oracle_top2_margin":0.537081241607666,"near_tie":false,"probs":[0.16016562283039093,0.14769092202186584,0.6921434998512268]},{"id":"semif_ea7ec66a5e8eac886548","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.012225091457366943,"mean_abs_dp":0.008150063455104828,"oracle_top2_margin":0.3613499701023102,"near_tie":false,"probs":[0.6767961978912354,0.2918693721294403,0.03133441135287285]},{"id":"semif_25172cd6ddc49ab0ca70","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0018053650856018066,"mean_abs_dp":0.0012035761028528214,"oracle_top2_margin":0.9521967768669128,"near_tie":false,"probs":[0.011126073077321053,0.019775914028286934,0.9690979719161987]},{"id":"semif_fb73a85d386cf59b0c51","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0015963464975357056,"mean_abs_dp":0.0010642856359481812,"oracle_top2_margin":0.44066160917282104,"near_tie":false,"probs":[0.6628338694572449,0.11424250900745392,0.22292356193065643]},{"id":"semif_a02fd582d2881cc462b2","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006878972053527832,"mean_abs_dp":0.0004586012413104375,"oracle_top2_margin":0.9755524396896362,"near_tie":false,"probs":[0.004409416113048792,0.009366837330162525,0.9862237572669983]},{"id":"semif_19f37d5a97034bd7d04f","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024451017379760742,"mean_abs_dp":0.0016300731028119724,"oracle_top2_margin":0.9070245623588562,"near_tie":false,"probs":[0.04480213299393654,0.9477120637893677,0.007485763635486364]},{"id":"semif_fda0ca94652c6d739ecf","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.007872581481933594,"mean_abs_dp":0.005248400072256724,"oracle_top2_margin":0.2631338834762573,"near_tie":false,"probs":[0.6026211977005005,0.04456310719251633,0.3528156876564026]},{"id":"semif_d0e36c4cb10a29b3191c","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009663775563240051,"mean_abs_dp":0.0006442765394846598,"oracle_top2_margin":0.6708698868751526,"near_tie":false,"probs":[0.7895641922950745,0.08988680690526962,0.12054897099733353]},{"id":"semif_a0a39ace7e5337d038a7","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00016695261001586914,"mean_abs_dp":0.00011128753734131654,"oracle_top2_margin":0.9900002479553223,"near_tie":false,"probs":[0.9942660927772522,0.0017793970182538033,0.0039545283652842045]},{"id":"semif_eeae283841ce75dd25bf","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005908608436584473,"mean_abs_dp":0.0003939139035840829,"oracle_top2_margin":0.9823033213615417,"near_tie":false,"probs":[0.006980158854275942,0.9903805255889893,0.0026393162552267313]},{"id":"semif_00bb4cc043f7d2b288d4","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00032716989517211914,"mean_abs_dp":0.00021813185109446445,"oracle_top2_margin":0.982890784740448,"near_tie":false,"probs":[0.007302368059754372,0.0018947558710351586,0.9908028841018677]},{"id":"semif_1a81ead69e93e5d0c86a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005230545997619629,"mean_abs_dp":0.003487055500348409,"oracle_top2_margin":0.169744610786438,"near_tie":false,"probs":[0.32559874653816223,0.18689978122711182,0.48750144243240356]},{"id":"semif_4444dad25e438f128673","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004333853721618652,"mean_abs_dp":0.0028892854849497476,"oracle_top2_margin":0.11443346738815308,"near_tie":false,"probs":[0.14267800748348236,0.4854060113430023,0.37191593647003174]},{"id":"semif_3226202ba1ada2570e9f","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0034356117248535156,"mean_abs_dp":0.0022904518991708755,"oracle_top2_margin":0.7993112206459045,"near_tie":false,"probs":[0.8925237655639648,0.020247289910912514,0.08722888678312302]},{"id":"semif_3f783eec5c90812479bc","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00944209098815918,"mean_abs_dp":0.006294724841912587,"oracle_top2_margin":0.25067201256752014,"near_tie":false,"probs":[0.05643988028168678,0.5890676975250244,0.3544924259185791]},{"id":"semif_ffee861474a9665a093a","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0038654059171676636,"mean_abs_dp":0.002576942245165507,"oracle_top2_margin":0.0018825829029083252,"near_tie":true,"probs":[0.3957323431968689,0.2029251605272293,0.4013425409793854]},{"id":"semif_8c992e17e4fb54b044f5","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022685527801513672,"mean_abs_dp":0.00015120646761109432,"oracle_top2_margin":0.9689651131629944,"near_tie":false,"probs":[0.0011500002583488822,0.9836967587471008,0.015153174288570881]},{"id":"semif_695dd61195ca025c4f4c","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002084493637084961,"mean_abs_dp":0.001389669564863046,"oracle_top2_margin":0.9003986716270447,"near_tie":false,"probs":[0.9436192512512207,0.04692547395825386,0.00945535209029913]},{"id":"semif_e6155431ab7b0f8170d4","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020354986190795898,"mean_abs_dp":0.00013569416478276253,"oracle_top2_margin":0.9357588291168213,"near_tie":false,"probs":[0.029582828283309937,0.965048611164093,0.005368528887629509]},{"id":"semif_b49ecda75086d8c68d2a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0026417076587677,"mean_abs_dp":0.0017611055324474971,"oracle_top2_margin":0.233449786901474,"near_tie":false,"probs":[0.022261077538132668,0.6081131100654602,0.3696257472038269]},{"id":"semif_587ced261c6092cd1325","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002564668655395508,"mean_abs_dp":0.0017097980404893558,"oracle_top2_margin":0.9022794961929321,"near_tie":false,"probs":[0.048404451459646225,0.9457699656486511,0.005825630389153957]},{"id":"semif_533d4423d311de82b27d","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009475946426391602,"mean_abs_dp":0.0006317449733614922,"oracle_top2_margin":0.8864229917526245,"near_tie":false,"probs":[0.054678019136190414,0.9392810463905334,0.0060409605503082275]},{"id":"semif_2bf540d515a51755b25c","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0015594959259033203,"mean_abs_dp":0.0010396592163791258,"oracle_top2_margin":0.9076201319694519,"near_tie":false,"probs":[0.04652947559952736,0.0023259660229086876,0.9511445760726929]},{"id":"semif_fdb9fc214aea2710763a","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014535188674926758,"mean_abs_dp":0.0009689976771672567,"oracle_top2_margin":0.8622768521308899,"near_tie":false,"probs":[0.008268233388662338,0.9257227778434753,0.06600901484489441]},{"id":"semif_f6d68126b43f8c9ed183","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005183815956115723,"mean_abs_dp":0.00034562312066555023,"oracle_top2_margin":0.913083016872406,"near_tie":false,"probs":[0.04098084196448326,0.9530737400054932,0.0059454478323459625]},{"id":"semif_44bae0c8a1615a096c43","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002574145793914795,"mean_abs_dp":0.0017161170641581218,"oracle_top2_margin":0.8858444690704346,"near_tie":false,"probs":[0.9361754059791565,0.008805501274764538,0.055019065737724304]},{"id":"semif_2175379f4c69e6207626","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00027370452880859375,"mean_abs_dp":0.00018243305385112762,"oracle_top2_margin":0.8939643502235413,"near_tie":false,"probs":[0.048060882836580276,0.9425703287124634,0.009368830360472202]},{"id":"semif_def9fe59d262357a673a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008314967155456543,"mean_abs_dp":0.005543318887551625,"oracle_top2_margin":0.48329558968544006,"near_tie":false,"probs":[0.22190454602241516,0.6920310854911804,0.08606433123350143]},{"id":"semif_712f6ffb763eef7c9a12","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006594657897949219,"mean_abs_dp":0.00043966372807820636,"oracle_top2_margin":0.9111813306808472,"near_tie":false,"probs":[0.9524266123771667,0.007575453724712133,0.03999786451458931]},{"id":"semif_e1d610bd14e3d16c09b9","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013447403907775879,"mean_abs_dp":0.0008965215335289637,"oracle_top2_margin":0.9179342985153198,"near_tie":false,"probs":[0.9545201063156128,0.03886394202709198,0.006616012193262577]},{"id":"semif_22ec57822db50ec121cf","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007007122039794922,"mean_abs_dp":0.0004671547406663497,"oracle_top2_margin":0.9677876830101013,"near_tie":false,"probs":[0.01606624759733677,0.9825063943862915,0.0014273307751864195]},{"id":"semif_3e75f2b84e8e08e3d3f2","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006717443466186523,"mean_abs_dp":0.0004478593667348226,"oracle_top2_margin":0.5739320516586304,"near_tie":false,"probs":[0.11564047634601593,0.1557142436504364,0.7286453247070312]},{"id":"semif_ebd74aa05324a1556288","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007838010787963867,"mean_abs_dp":0.0005225328107674917,"oracle_top2_margin":0.9141265153884888,"near_tie":false,"probs":[0.04152632877230644,0.9540941715240479,0.00437941076233983]},{"id":"semif_d7e84e4245a29817da1f","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000968635082244873,"mean_abs_dp":0.0006457801597813765,"oracle_top2_margin":0.9529194235801697,"near_tie":false,"probs":[0.023657923564314842,0.0016729806084185839,0.9746691584587097]},{"id":"semif_f41ae66b3c09630c7a42","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000926874577999115,"mean_abs_dp":0.00061793252825737,"oracle_top2_margin":0.9167654514312744,"near_tie":false,"probs":[0.04013722389936447,0.004770333878695965,0.9550923705101013]},{"id":"semif_82c2cd97c160ad27951b","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011950880289077759,"mean_abs_dp":0.0007967464625835419,"oracle_top2_margin":0.5744704008102417,"near_tie":false,"probs":[0.7688329815864563,0.03864337131381035,0.19252373278141022]},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00030177831649780273,"mean_abs_dp":0.00020121216463545957,"oracle_top2_margin":0.9755150675773621,"near_tie":false,"probs":[0.9858072996139526,0.003326371544972062,0.01086635421961546]},{"id":"semif_4d7a003342db5b0a17e3","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002990737557411194,"mean_abs_dp":0.00019938296948870024,"oracle_top2_margin":0.9298084378242493,"near_tie":false,"probs":[0.03366133198142052,0.9629001617431641,0.003438578685745597]},{"id":"semif_19d3d8d0bf773e88be7b","q":0,"variant":"base","tokens":55,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002301335334777832,"mean_abs_dp":0.00015343756725390753,"oracle_top2_margin":0.9657230973243713,"near_tie":false,"probs":[0.9809682965278625,0.0033632416743785143,0.015668533742427826]},{"id":"semif_d6de731cca52359b1c87","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006172865629196167,"mean_abs_dp":0.0041152288516362505,"oracle_top2_margin":0.047635823488235474,"near_tie":false,"probs":[0.4536828100681305,0.41761258244514465,0.12870456278324127]},{"id":"semif_b45c63fb92729af27f05","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025545358657836914,"mean_abs_dp":0.0017030443996191025,"oracle_top2_margin":0.8986563086509705,"near_tie":false,"probs":[0.9473704099655151,0.008647583425045013,0.04398198425769806]},{"id":"semif_46b7029b9a704138b77a","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00035305507481098175,"mean_abs_dp":0.0002353857271373272,"oracle_top2_margin":0.9545035362243652,"near_tie":false,"probs":[0.019950587302446365,0.00490965973585844,0.9751397967338562]},{"id":"semif_caf773b23934fa7362a1","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003972053527832031,"mean_abs_dp":0.00026481652942796546,"oracle_top2_margin":0.9587439894676208,"near_tie":false,"probs":[0.9778717160224915,0.0036716237664222717,0.018456635996699333]},{"id":"semif_f46f392ef9e9e9df564b","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.424022674560547e-05,"mean_abs_dp":3.6166515201330185e-05,"oracle_top2_margin":0.9757024049758911,"near_tie":false,"probs":[0.007328987121582031,0.9842245578765869,0.008446487598121166]},{"id":"semif_005dcf6d2c0f21d64796","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001820683479309082,"mean_abs_dp":0.0012137914697329204,"oracle_top2_margin":0.835649847984314,"near_tie":false,"probs":[0.03984634578227997,0.8994222283363342,0.060731440782547]},{"id":"semif_edb708bd0fe5fd67d6ee","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004269689321517944,"mean_abs_dp":0.00284649928410848,"oracle_top2_margin":0.1792631447315216,"near_tie":false,"probs":[0.29858312010765076,0.2299281358718872,0.4714888036251068]},{"id":"semif_97a35499bd8717f3e786","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00033456459641456604,"mean_abs_dp":0.0002230514461795489,"oracle_top2_margin":0.959165096282959,"near_tie":false,"probs":[0.012083901092410088,0.014372407458722591,0.973543643951416]},{"id":"semif_31eb4533d2c89959ec46","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001008652150630951,"mean_abs_dp":0.0006724161406358083,"oracle_top2_margin":0.8864260911941528,"near_tie":false,"probs":[0.9329273104667664,0.021195657551288605,0.04587709531188011]},{"id":"semif_0aa24a6701ac9c56c4a0","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00023226067423820496,"mean_abs_dp":0.00015481871863206229,"oracle_top2_margin":0.9143282175064087,"near_tie":false,"probs":[0.037449758499860764,0.951407253742218,0.011143037118017673]},{"id":"semif_796d5c0da6eaadae6a3f","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00028073787689208984,"mean_abs_dp":0.0001871606994730731,"oracle_top2_margin":0.9771841764450073,"near_tie":false,"probs":[0.0006848382763564587,0.9879797697067261,0.0113354017958045]},{"id":"semif_599b3092b6fd18312d23","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00016933679580688477,"mean_abs_dp":0.00011292317261298497,"oracle_top2_margin":0.9692578315734863,"near_tie":false,"probs":[0.002862808294594288,0.014096814207732677,0.9830403923988342]},{"id":"semif_960a4bcd27363e6db4b9","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014294981956481934,"mean_abs_dp":0.000953023632367452,"oracle_top2_margin":0.6039082407951355,"near_tie":false,"probs":[0.11778701841831207,0.13926351070404053,0.7429494261741638]},{"id":"semif_c6a411e7f2bbdf377173","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004995942115783691,"mean_abs_dp":0.0033306488767266273,"oracle_top2_margin":0.7695140838623047,"near_tie":false,"probs":[0.8823150992393494,0.10326974838972092,0.014415113255381584]},{"id":"semif_e332aa927d0c2ed044aa","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002534627914428711,"mean_abs_dp":0.0016897264868021011,"oracle_top2_margin":0.825499951839447,"near_tie":false,"probs":[0.023479461669921875,0.07299451529979706,0.9035260677337646]},{"id":"semif_a0128e551d095f3449f8","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005913972854614258,"mean_abs_dp":0.0003942767313371102,"oracle_top2_margin":0.9518061876296997,"near_tie":false,"probs":[0.9742383360862732,0.023602480068802834,0.0021592695266008377]},{"id":"semif_46d6a0731bd288c370f0","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.0498714447021484e-05,"mean_abs_dp":4.030391573905945e-05,"oracle_top2_margin":0.7160866260528564,"near_tie":false,"probs":[0.8541572093963623,0.13798165321350098,0.007861187681555748]},{"id":"semif_21ca7a8b970e36b8e20f","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003355264663696289,"mean_abs_dp":0.0022368555267651877,"oracle_top2_margin":0.2341521680355072,"near_tie":false,"probs":[0.583340048789978,0.3425484597682953,0.0741114690899849]},{"id":"semif_4aa83af95bb69e90bc2d","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005203038454055786,"mean_abs_dp":0.003468697269757589,"oracle_top2_margin":0.06450572609901428,"near_tie":false,"probs":[0.4694244861602783,0.11529085040092468,0.4152846336364746]},{"id":"semif_cdfa9eed51fafcf09b72","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003392040729522705,"mean_abs_dp":0.0022613623489936194,"oracle_top2_margin":0.8453994393348694,"near_tie":false,"probs":[0.06503500044345856,0.018472561612725258,0.9164924621582031]},{"id":"semif_d51b05bfb9d6d7f09331","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015390217304229736,"mean_abs_dp":0.0010260120034217834,"oracle_top2_margin":0.23784101009368896,"near_tie":false,"probs":[0.35233160853385925,0.5930271148681641,0.054641254246234894]},{"id":"semif_fdd902cc6e65fad7cb4f","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005119144916534424,"mean_abs_dp":0.0034127856294314065,"oracle_top2_margin":0.765238344669342,"near_tie":false,"probs":[0.08378727734088898,0.8402024507522583,0.07601027190685272]},{"id":"semif_42384af9ae56beb37ff5","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.011975705623626709,"mean_abs_dp":0.007983801265557608,"oracle_top2_margin":0.7008019685745239,"near_tie":false,"probs":[0.12666156888008118,0.024826176464557648,0.8485122919082642]},{"id":"semif_cdf7f428857e1b841cb3","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.012779414653778076,"mean_abs_dp":0.008519597351551056,"oracle_top2_margin":0.6383113265037537,"near_tie":false,"probs":[0.03927646949887276,0.14967627823352814,0.8110472559928894]},{"id":"semif_774f7f9245f83c796d44","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.2351741790771484e-05,"mean_abs_dp":1.490612824757894e-05,"oracle_top2_margin":0.8995101451873779,"near_tie":false,"probs":[0.02818160690367222,0.03616516292095184,0.9356532096862793]},{"id":"semif_92ec2ffa3dd46ada1335","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.205130971968174e-05,"mean_abs_dp":2.8054617966214817e-05,"oracle_top2_margin":0.9729353785514832,"near_tie":false,"probs":[0.9852610230445862,0.012328741140663624,0.0024102297611534595]},{"id":"semif_6149a17bc154f9c5b4a0","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016073435544967651,"mean_abs_dp":0.001071568268040816,"oracle_top2_margin":0.9116500616073608,"near_tie":false,"probs":[0.9501344561576843,0.041612014174461365,0.008253494277596474]},{"id":"semif_be866e1dbec5d42d9423","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00132066011428833,"mean_abs_dp":0.0008804357300202051,"oracle_top2_margin":0.920475959777832,"near_tie":false,"probs":[0.01444966346025467,0.033677954226732254,0.9518724083900452]},{"id":"semif_97d2ed2591fc7638806c","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006122291088104248,"mean_abs_dp":0.004081507523854573,"oracle_top2_margin":0.6768540143966675,"near_tie":false,"probs":[0.8211272358894348,0.15651772916316986,0.02235499769449234]},{"id":"semif_4533dd91267b4d815690","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.610822260379791e-05,"mean_abs_dp":5.7390580574671425e-05,"oracle_top2_margin":0.9652467370033264,"near_tie":false,"probs":[0.011688132770359516,0.011408431455492973,0.9769033789634705]},{"id":"semif_d880e65fef5e2993fdd0","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.029746264219284058,"mean_abs_dp":0.019830872615178425,"oracle_top2_margin":0.12788748741149902,"near_tie":false,"probs":[0.3910844922065735,0.2976735830307007,0.3112419545650482]},{"id":"semif_90494c76078ffc3c717a","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0030486583709716797,"mean_abs_dp":0.002032436430454254,"oracle_top2_margin":0.7915595769882202,"near_tie":false,"probs":[0.8848609328269958,0.08733538538217545,0.027803689241409302]},{"id":"semif_9ac5cc82c4126c91174a","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017353817820549011,"mean_abs_dp":0.0011568870395421982,"oracle_top2_margin":0.7353759407997131,"near_tie":false,"probs":[0.12054552137851715,0.8591210842132568,0.020333347842097282]},{"id":"semif_386e10b1c1ce906473a4","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002346634864807129,"mean_abs_dp":0.00015646118360261121,"oracle_top2_margin":0.9825382232666016,"near_tie":false,"probs":[0.0025429236702620983,0.9897812604904175,0.007675853092223406]},{"id":"semif_4da6aedc1139224eeff2","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00044918060302734375,"mean_abs_dp":0.00029946786041061085,"oracle_top2_margin":0.9583117365837097,"near_tie":false,"probs":[0.020176393911242485,0.9776795506477356,0.002144081285223365]},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.012049198150634766,"mean_abs_dp":0.008032803734143576,"oracle_top2_margin":0.4805392622947693,"near_tie":false,"probs":[0.20932230353355408,0.7069059610366821,0.0837717354297638]},{"id":"semif_42c54fa2ad45743e24d8","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000728905200958252,"mean_abs_dp":0.0004859541853268941,"oracle_top2_margin":0.8835832476615906,"near_tie":false,"probs":[0.021881455555558205,0.9315292239189148,0.046589307487010956]},{"id":"semif_f3d13cf184f916677def","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00022424012422561646,"mean_abs_dp":0.00014950668749709925,"oracle_top2_margin":0.9233630299568176,"near_tie":false,"probs":[0.9605938792228699,0.03691606596112251,0.002490075072273612]},{"id":"semif_1105577da4c8dad4609d","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.060132622718811035,"mean_abs_dp":0.04008840521176656,"oracle_top2_margin":0.37257999181747437,"near_tie":false,"probs":[0.18619182705879211,0.1588355302810669,0.6549726724624634]},{"id":"semif_0ebd299739302d8b057b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008970975875854492,"mean_abs_dp":0.005980655550956726,"oracle_top2_margin":0.22060275077819824,"near_tie":false,"probs":[0.538212239742279,0.3297369182109833,0.13205085694789886]},{"id":"semif_eafc22c8c40df3932a8e","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0015865564346313477,"mean_abs_dp":0.0010577104985713959,"oracle_top2_margin":0.9194331765174866,"near_tie":false,"probs":[0.025213100016117096,0.02634587697684765,0.9484410285949707]},{"id":"semif_0aa1447680e4f25862bb","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011684894561767578,"mean_abs_dp":0.0007789830366770426,"oracle_top2_margin":0.22799772024154663,"near_tie":false,"probs":[0.3127063810825348,0.5429883599281311,0.1443052589893341]},{"id":"semif_980e76145c925747220c","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004220344126224518,"mean_abs_dp":0.0002813575168450673,"oracle_top2_margin":0.8666357398033142,"near_tie":false,"probs":[0.05557487532496452,0.0225745290517807,0.9218505620956421]},{"id":"semif_4aee37a6a155747766aa","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0032727718353271484,"mean_abs_dp":0.00218186154961586,"oracle_top2_margin":0.8649082183837891,"near_tie":false,"probs":[0.9148778319358826,0.044830597937107086,0.04029158875346184]},{"id":"semif_6db17cc22bd656d78558","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0032169222831726074,"mean_abs_dp":0.002144614855448405,"oracle_top2_margin":0.4900950789451599,"near_tie":false,"probs":[0.21056091785430908,0.09242424368858337,0.6970148682594299]},{"id":"semif_ef3e0f8e657bf59f8a29","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.022765278816223145,"mean_abs_dp":0.015176867445309957,"oracle_top2_margin":0.2901451885700226,"near_tie":false,"probs":[0.20105694234371185,0.27017173171043396,0.5287713408470154]},{"id":"semif_f756e1cb04e11766341e","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.01778966188430786,"mean_abs_dp":0.011859779556592306,"oracle_top2_margin":0.47525715827941895,"near_tie":false,"probs":[0.23425672948360443,0.6792564988136292,0.0864868015050888]},{"id":"semif_d167e51bb1f499099645","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.008050262928009033,"mean_abs_dp":0.005366846919059753,"oracle_top2_margin":0.14599356055259705,"near_tie":false,"probs":[0.32513660192489624,0.19507098197937012,0.4797924757003784]},{"id":"semif_674ca4e9346216255466","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0021138787269592285,"mean_abs_dp":0.001409215231736501,"oracle_top2_margin":0.8445791602134705,"near_tie":false,"probs":[0.8998829126358032,0.04107600077986717,0.05904100835323334]},{"id":"semif_1df851a202994a07f123","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00036203861236572266,"mean_abs_dp":0.00024135317653417587,"oracle_top2_margin":0.9717237949371338,"near_tie":false,"probs":[0.9842961430549622,0.012036718428134918,0.0036671110428869724]},{"id":"semif_b5e6788d95b8395d2888","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":3.2942392863333225e-05,"mean_abs_dp":2.1961090775827568e-05,"oracle_top2_margin":0.9891096949577332,"near_tie":false,"probs":[0.0045737456530332565,0.0017663268372416496,0.9936599135398865]},{"id":"semif_32f36cd0fbbfb3604586","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.013280987739562988,"mean_abs_dp":0.00885397723565499,"oracle_top2_margin":0.6498266458511353,"near_tie":false,"probs":[0.8355913758277893,0.15956398844718933,0.0048446268774569035]},{"id":"semif_843185706adff0c60f8b","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.009092748165130615,"mean_abs_dp":0.006061867189904054,"oracle_top2_margin":0.5439835786819458,"near_tie":false,"probs":[0.7728226184844971,0.015348794870078564,0.21182847023010254]},{"id":"semif_b531dd603bf080dab820","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024573206901550293,"mean_abs_dp":0.0016382584969202678,"oracle_top2_margin":0.6922020316123962,"near_tie":false,"probs":[0.8336215019226074,0.0206471998244524,0.14573131501674652]},{"id":"semif_be2dd67bc733c1213bef","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0025513172149658203,"mean_abs_dp":0.0017008644839127858,"oracle_top2_margin":0.6489488482475281,"near_tie":false,"probs":[0.04065675288438797,0.802148163318634,0.1571951061487198]},{"id":"semif_b8e2750307df9761e716","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006677210330963135,"mean_abs_dp":0.004451470449566841,"oracle_top2_margin":0.779024064540863,"near_tie":false,"probs":[0.887098491191864,0.017443936318159103,0.09545760601758957]},{"id":"semif_5d31b4bf2d0fc193f53b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.011507153511047363,"mean_abs_dp":0.007671470443407695,"oracle_top2_margin":0.0367678701877594,"near_tie":false,"probs":[0.09018751233816147,0.4472206234931946,0.46259188652038574]},{"id":"semif_a9230edfc6ec9e524af0","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":9.232759475708008e-05,"mean_abs_dp":6.156771754225095e-05,"oracle_top2_margin":0.9861339330673218,"near_tie":false,"probs":[0.0031787368934601545,0.005422576330602169,0.9913986921310425]},{"id":"semif_c100e84585149a04be49","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00029921531677246094,"mean_abs_dp":0.00019947625696659088,"oracle_top2_margin":0.9634105563163757,"near_tie":false,"probs":[0.9780080914497375,0.007830663584172726,0.014161281287670135]},{"id":"semif_579beed184ffc1104aa9","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002593994140625,"mean_abs_dp":0.00017291431625684103,"oracle_top2_margin":0.9744829535484314,"near_tie":false,"probs":[0.9843184351921082,0.005454692989587784,0.010226842015981674]},{"id":"semif_c4e9bd463b6fde5be743","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008165426552295685,"mean_abs_dp":0.0005443257590134939,"oracle_top2_margin":0.8355470895767212,"near_tie":false,"probs":[0.03954259678721428,0.8976144194602966,0.06284303218126297]},{"id":"semif_b863998d4be2721a5dfd","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011998414993286133,"mean_abs_dp":8.002171913782756e-05,"oracle_top2_margin":0.8867029547691345,"near_tie":false,"probs":[0.006487157661467791,0.940177321434021,0.05333545058965683]},{"id":"semif_37cebc3792e98049439f","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006096184253692627,"mean_abs_dp":0.004064120352268219,"oracle_top2_margin":0.8824067115783691,"near_tie":false,"probs":[0.05546211451292038,0.927118718624115,0.017419254407286644]},{"id":"semif_64d63f86e328aff1481c","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0018748044967651367,"mean_abs_dp":0.0012498876700798671,"oracle_top2_margin":0.9261332154273987,"near_tie":false,"probs":[0.009978894144296646,0.03362926095724106,0.9563918709754944]},{"id":"semif_6d64a8459f64eac7f4cc","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000697016716003418,"mean_abs_dp":0.00046467532714207965,"oracle_top2_margin":0.9683219790458679,"near_tie":false,"probs":[0.9799911975860596,0.012847921811044216,0.007160864770412445]},{"id":"semif_2708193212d8a4a523c7","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.009920179843902588,"mean_abs_dp":0.006613438328107198,"oracle_top2_margin":0.0197182297706604,"near_tie":true,"probs":[0.22729066014289856,0.3849354088306427,0.38777393102645874]},{"id":"semif_5a37d1dfcad20810a76a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007193088531494141,"mean_abs_dp":0.0004795451338092486,"oracle_top2_margin":0.880086362361908,"near_tie":false,"probs":[0.012540727853775024,0.9344459176063538,0.05301332101225853]},{"id":"semif_acacd8b46acd25cb954b","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00993579626083374,"mean_abs_dp":0.00662386417388916,"oracle_top2_margin":0.7345818281173706,"near_tie":false,"probs":[0.10309351235628128,0.8534248471260071,0.04348168149590492]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024436116218566895,"mean_abs_dp":0.0016290719310442607,"oracle_top2_margin":0.0004886090755462646,"near_tie":true,"probs":[0.4400648772716522,0.12359163910150528,0.4363435208797455]},{"id":"semif_02f2b95a5c5a4777a45c","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007244348526000977,"mean_abs_dp":0.0004829598280290763,"oracle_top2_margin":0.910001814365387,"near_tie":false,"probs":[0.006485605146735907,0.9510729312896729,0.04244145750999451]},{"id":"semif_ebea4681e3dcc7d92ca2","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006445109844207764,"mean_abs_dp":0.004296724994977315,"oracle_top2_margin":0.18332910537719727,"near_tie":false,"probs":[0.49802348017692566,0.17862536013126373,0.3233512043952942]},{"id":"semif_b8c5b9b285cbdddc9f8d","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013198256492614746,"mean_abs_dp":0.0008798666919271151,"oracle_top2_margin":0.8747166991233826,"near_tie":false,"probs":[0.011991106905043125,0.057895250618457794,0.9301136136054993]},{"id":"semif_7ed75d854d99b3a81916","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.025355279445648193,"mean_abs_dp":0.01690353887776534,"oracle_top2_margin":0.2616990804672241,"near_tie":false,"probs":[0.335384726524353,0.017584118992090225,0.6470310688018799]},{"id":"own_t01","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0022386014461517334,"mean_abs_dp":0.001286524161696434,"oracle_top2_margin":0.3692072629928589,"near_tie":false,"probs":[0.5720820426940918,0.08988572657108307,0.19963927567005157,0.02638007327914238,0.11201288551092148],"hidden":{"min_pos_cos":0.9995783960051926,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":1,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002833724021911621,"mean_abs_dp":0.0028336867690086365,"oracle_top2_margin":0.510627031326294,"near_tie":false,"probs":[0.24185281991958618,0.7581472396850586],"hidden":{"min_pos_cos":0.9995783960051926,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":2,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":1,"argmax_equal":false,"max_abs_dp":0.004243016242980957,"mean_abs_dp":0.002176957204937935,"oracle_top2_margin":0.0016841888427734375,"near_tie":true,"probs":[0.37062928080558777,0.3655156195163727,0.1940469741821289,0.06980811804533005],"hidden":{"min_pos_cos":0.9995783960051926,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":3,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00019621849060058594,"mean_abs_dp":0.00019621290266513824,"oracle_top2_margin":0.8861521482467651,"near_tie":false,"probs":[0.05712011829018593,0.942879855632782],"hidden":{"min_pos_cos":0.9995783960051926,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t02","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":2,"argmax_equal":false,"max_abs_dp":0.004480421543121338,"mean_abs_dp":0.002614453434944153,"oracle_top2_margin":0.0024104714393615723,"near_tie":true,"probs":[0.19402752816677094,0.3729144036769867,0.36822229623794556,0.06483573466539383]},{"id":"own_t02","q":1,"variant":"base","tokens":303,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011072754859924316,"mean_abs_dp":0.0004429051652550697,"oracle_top2_margin":0.8872987627983093,"near_tie":false,"probs":[0.004051505587995052,0.9319756031036377,0.008404378779232502,0.009037850424647331,0.04653060436248779]},{"id":"own_t02","q":2,"variant":"base","tokens":262,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00017477478832006454,"mean_abs_dp":0.00017473800107836723,"oracle_top2_margin":0.9853664040565491,"near_tie":false,"probs":[0.9928579330444336,0.007142048329114914]},{"id":"own_t03","q":0,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013827681541442871,"mean_abs_dp":0.00046091949722419184,"oracle_top2_margin":0.9159768223762512,"near_tie":false,"probs":[0.0021078737918287516,0.9437281489372253,0.010486198589205742,0.010011005215346813,0.0037860507145524025,0.02988065965473652]},{"id":"own_t03","q":1,"variant":"base","tokens":214,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012397803366184235,"mean_abs_dp":0.0012397486716508865,"oracle_top2_margin":0.8899710178375244,"near_tie":false,"probs":[0.05625426769256592,0.9437457919120789]},{"id":"own_t03","q":2,"variant":"base","tokens":245,"calls":16,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012316107749938965,"mean_abs_dp":0.0006414996460080147,"oracle_top2_margin":0.7733081579208374,"near_tie":false,"probs":[0.8454216122627258,0.07329364120960236,0.04625829681754112,0.0350264310836792]},{"id":"own_t04","q":0,"variant":"base","tokens":190,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00016677379608154297,"mean_abs_dp":8.338585030287504e-05,"oracle_top2_margin":0.9826205372810364,"near_tie":false,"probs":[0.0056847999803721905,0.98807692527771,0.003119073575362563,0.0031192032620310783]},{"id":"own_t04","q":1,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005227804183959961,"mean_abs_dp":0.005227789282798767,"oracle_top2_margin":0.5204432010650635,"near_tie":false,"probs":[0.24500617384910583,0.7549937963485718]},{"id":"own_t04","q":2,"variant":"base","tokens":191,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0017088651657104492,"mean_abs_dp":0.000887423288077116,"oracle_top2_margin":0.8596851825714111,"near_tie":false,"probs":[0.0073702395893633366,0.012999678030610085,0.9212756156921387,0.05835451930761337]},{"id":"own_t04","q":3,"variant":"base","tokens":175,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019209012389183044,"mean_abs_dp":0.0012805933753649394,"oracle_top2_margin":0.7806951999664307,"near_tie":false,"probs":[0.0807000920176506,0.8631775379180908,0.05612233281135559]},{"id":"own_t05","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0023177266120910645,"mean_abs_dp":0.0009270633512642234,"oracle_top2_margin":0.6871715784072876,"near_tie":false,"probs":[0.054698068648576736,0.8164750337600708,0.12649878859519958,0.002047253306955099,0.0002809191355481744]},{"id":"own_t05","q":1,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013004541397094727,"mean_abs_dp":0.0004772508788543443,"oracle_top2_margin":0.8503351211547852,"near_tie":false,"probs":[0.004788778256624937,0.8981871604919434,0.03177669644355774,0.045990731567144394,0.001534528681077063,0.017722148448228836]},{"id":"own_t05","q":2,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.506746962666512e-05,"mean_abs_dp":8.503184653818607e-05,"oracle_top2_margin":0.9918721318244934,"near_tie":false,"probs":[0.9958510398864746,0.004149001091718674]},{"id":"own_t05","q":3,"variant":"base","tokens":125,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001519918441772461,"mean_abs_dp":0.00015194900333881378,"oracle_top2_margin":0.9897524118423462,"near_tie":false,"probs":[0.005275727715343237,0.9947242140769958]},{"id":"own_t06","q":0,"variant":"base","tokens":301,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.012297570705413818,"mean_abs_dp":0.002459508809261024,"oracle_top2_margin":0.5316310524940491,"near_tie":false,"probs":[0.0038119966629892588,0.052511584013700485,0.004621341824531555,0.696833610534668,0.004776627290993929,0.02911790832877159,0.002764042466878891,0.006450572516769171,0.015572844073176384,0.18353937566280365]},{"id":"own_t06","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000409470871090889,"mean_abs_dp":0.0004094475880265236,"oracle_top2_margin":0.964957058429718,"near_tie":false,"probs":[0.9820690751075745,0.017930900678038597]},{"id":"own_t06","q":2,"variant":"base","tokens":224,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002411484718322754,"mean_abs_dp":0.0016076465447743733,"oracle_top2_margin":0.6520206928253174,"near_tie":false,"probs":[0.08014227449893951,0.7866060733795166,0.1332516074180603]},{"id":"own_t07","q":0,"variant":"base","tokens":219,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007053792476654053,"mean_abs_dp":0.0042253294959664345,"oracle_top2_margin":0.11449462175369263,"near_tie":false,"probs":[0.05162942782044411,0.4012424051761627,0.27300041913986206,0.274127721786499]},{"id":"own_t07","q":1,"variant":"base","tokens":174,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017153732478618622,"mean_abs_dp":0.0017153676599264145,"oracle_top2_margin":0.8910784125328064,"near_tie":false,"probs":[0.056176140904426575,0.9438238143920898]},{"id":"own_t07","q":2,"variant":"base","tokens":187,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.008261263370513916,"mean_abs_dp":0.005507489045461019,"oracle_top2_margin":0.7710465788841248,"near_tie":false,"probs":[0.08077177405357361,0.07954849302768707,0.8396797180175781]},{"id":"own_t08","q":0,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.3232231140136719e-05,"mean_abs_dp":1.3219192624092102e-05,"oracle_top2_margin":0.8926053047180176,"near_tie":false,"probs":[0.05371055379509926,0.9462894201278687]},{"id":"own_t08","q":1,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00010585784912109375,"mean_abs_dp":7.821642793715e-05,"oracle_top2_margin":0.9543377161026001,"near_tie":false,"probs":[0.017579227685928345,0.971861720085144,0.006012379191815853,0.004546626005321741]},{"id":"own_t08","q":2,"variant":"base","tokens":170,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0017265081405639648,"mean_abs_dp":0.0008727535605430603,"oracle_top2_margin":0.07700720429420471,"near_tie":false,"probs":[0.32180842757225037,0.1331540048122406,0.40223172307014465,0.14280582964420319]},{"id":"own_t09","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002057194709777832,"mean_abs_dp":0.002057187259197235,"oracle_top2_margin":0.7585829496383667,"near_tie":false,"probs":[0.12276572734117508,0.8772342801094055]},{"id":"own_t09","q":1,"variant":"base","tokens":151,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00025922060012817383,"mean_abs_dp":0.00012960610911250114,"oracle_top2_margin":0.9486740827560425,"near_tie":false,"probs":[0.9690585732460022,0.006989198736846447,0.003957715816795826,0.019994515925645828]},{"id":"own_t10","q":0,"variant":"base","tokens":161,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003951806575059891,"mean_abs_dp":0.0001580849289894104,"oracle_top2_margin":0.9470322132110596,"near_tie":false,"probs":[0.9631120562553406,0.015872173011302948,0.01455223374068737,0.003893978428095579,0.0025695909280329943]},{"id":"own_t10","q":1,"variant":"base","tokens":140,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00047779083251953125,"mean_abs_dp":0.00047777965664863586,"oracle_top2_margin":0.8197239637374878,"near_tie":false,"probs":[0.9093841910362244,0.09061580896377563]},{"id":"own_t10","q":2,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0015550851821899414,"mean_abs_dp":0.0007774974219501019,"oracle_top2_margin":0.7304309606552124,"near_tie":false,"probs":[0.09520361572504044,0.06291615962982178,0.8236340880393982,0.018246067687869072]},{"id":"own_j01","q":0,"variant":"base","tokens":212,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002497434616088867,"mean_abs_dp":0.0024974197149276733,"oracle_top2_margin":0.45111677050590515,"near_tie":false,"probs":[0.2769390344619751,0.7230609655380249]},{"id":"own_j01","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00048339366912841797,"mean_abs_dp":0.0002416929928585887,"oracle_top2_margin":0.9275074005126953,"near_tie":false,"probs":[0.007117133587598801,0.955844521522522,0.008072532713413239,0.028965922072529793]},{"id":"own_j01","q":2,"variant":"base","tokens":242,"calls":16,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00029963254928588867,"mean_abs_dp":0.0001498027704656124,"oracle_top2_margin":0.95612633228302,"near_tie":false,"probs":[0.0035516424104571342,0.01594860479235649,0.9724166989326477,0.008083130232989788]},{"id":"own_j01","q":3,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00013448810204863548,"mean_abs_dp":0.00013444828800857067,"oracle_top2_margin":0.9847407937049866,"near_tie":false,"probs":[0.0077640763483941555,0.9922359585762024]},{"id":"own_j02","q":0,"variant":"base","tokens":332,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00382860004901886,"mean_abs_dp":0.0010455523035489023,"oracle_top2_margin":0.3744804859161377,"near_tie":false,"probs":[0.5694968104362488,0.011751151643693447,0.026072083041071892,0.01636676862835884,0.0023439377546310425,0.0160304456949234,0.001508875167928636,0.1556168794631958,0.006823365576565266,0.19398969411849976]},{"id":"own_j02","q":1,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005702078342437744,"mean_abs_dp":0.00570206344127655,"oracle_top2_margin":0.4711904525756836,"near_tie":false,"probs":[0.25870272517204285,0.7412973046302795]},{"id":"own_j02","q":2,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007649511098861694,"mean_abs_dp":0.0007649287581443787,"oracle_top2_margin":0.580559253692627,"near_tie":false,"probs":[0.2104853242635727,0.7895147204399109]},{"id":"own_j03","q":0,"variant":"base","tokens":380,"calls":24,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003033161163330078,"mean_abs_dp":0.0015165917575359344,"oracle_top2_margin":0.819699764251709,"near_tie":false,"probs":[0.8733769059181213,0.030348723754286766,0.0492878295481205,0.04698648303747177],"hidden":{"min_pos_cos":0.9825826863974576,"positions_below":{"0.99":2,"0.9":0,"0.5":0}}},{"id":"own_j03","q":1,"variant":"base","tokens":333,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":6,"argmax_oracle":6,"argmax_equal":true,"max_abs_dp":0.006347477436065674,"mean_abs_dp":0.003530923809323992,"oracle_top2_margin":0.03619730472564697,"near_tie":false,"probs":[0.05125834420323372,0.033112481236457825,0.06557156145572662,0.12679576873779297,0.213261216878891,0.23707008361816406,0.2729306221008301],"hidden":{"min_pos_cos":0.9241736357846124,"positions_below":{"0.99":3,"0.9":0,"0.5":0}}},{"id":"own_j03","q":2,"variant":"base","tokens":314,"calls":20,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015000104904174805,"mean_abs_dp":0.0014999881386756897,"oracle_top2_margin":0.8600038290023804,"near_tie":false,"probs":[0.07149805873632431,0.9285019040107727],"hidden":{"min_pos_cos":0.9825826863974576,"positions_below":{"0.99":2,"0.9":0,"0.5":0}}},{"id":"own_j04","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001914501190185547,"mean_abs_dp":9.571651753503829e-05,"oracle_top2_margin":0.9900386929512024,"near_tie":false,"probs":[0.001603801385499537,0.0006121700280345976,0.993744432926178,0.004039605613797903]},{"id":"own_j04","q":1,"variant":"base","tokens":171,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011241436004638672,"mean_abs_dp":0.00011240039020776749,"oracle_top2_margin":0.9883577227592468,"near_tie":false,"probs":[0.9940664768218994,0.005933552514761686]},{"id":"own_j04","q":2,"variant":"base","tokens":196,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.007657796144485474,"mean_abs_dp":0.005264513194561005,"oracle_top2_margin":0.12996074557304382,"near_tie":false,"probs":[0.13881570100784302,0.1570519208908081,0.28469252586364746,0.4194398522377014]},{"id":"own_j05","q":0,"variant":"base","tokens":198,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0039920806884765625,"mean_abs_dp":0.002661393334468206,"oracle_top2_margin":0.7300764322280884,"near_tie":false,"probs":[0.8298388719558716,0.10575857758522034,0.06440252810716629]},{"id":"own_j05","q":1,"variant":"base","tokens":153,"calls":10,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016741156578063965,"mean_abs_dp":0.0016740858554840088,"oracle_top2_margin":0.7264596819877625,"near_tie":false,"probs":[0.1384442299604416,0.8615557551383972]},{"id":"own_j05","q":2,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0034579038619995117,"mean_abs_dp":0.002095478985990797,"oracle_top2_margin":0.0014663934707641602,"near_tie":true,"probs":[0.11921381205320358,0.053303372114896774,0.11971210688352585,0.21478959918022156,0.20872221887111664,0.16581302881240845,0.11844577640295029]},{"id":"own_a01","q":0,"variant":"base","tokens":199,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007120728492736816,"mean_abs_dp":0.003632928943261504,"oracle_top2_margin":0.06744769215583801,"near_tie":false,"probs":[0.05422743037343025,0.494994580745697,0.009135314263403416,0.4416426122188568]},{"id":"own_a01","q":1,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005230456590652466,"mean_abs_dp":0.0005230382084846497,"oracle_top2_margin":0.8198840022087097,"near_tie":false,"probs":[0.09058105945587158,0.9094190001487732]},{"id":"own_a01","q":2,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013027787208557129,"mean_abs_dp":0.0013027489185333252,"oracle_top2_margin":0.08699706196784973,"near_tie":false,"probs":[0.5448012948036194,0.455198734998703]},{"id":"own_m01","q":0,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001248675398528576,"mean_abs_dp":8.329454188545544e-05,"oracle_top2_margin":0.9799088835716248,"near_tie":false,"probs":[0.007437332067638636,0.9873521327972412,0.005210482981055975],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":1,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002862391993403435,"mean_abs_dp":0.00028620054945349693,"oracle_top2_margin":0.9791609048843384,"near_tie":false,"probs":[0.010705767199397087,0.9892942905426025],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":2,"variant":"base","tokens":159,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.012172996997833252,"mean_abs_dp":0.008115335988501707,"oracle_top2_margin":0.18575620651245117,"near_tie":false,"probs":[0.602966845035553,0.3930387496948242,0.003994402475655079],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":3,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0010319352149963379,"mean_abs_dp":0.0006879810243844986,"oracle_top2_margin":0.695020318031311,"near_tie":false,"probs":[0.024117769673466682,0.13942156732082367,0.8364606499671936],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":4,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.435112588107586e-05,"mean_abs_dp":9.432283695787191e-05,"oracle_top2_margin":0.9920198321342468,"near_tie":false,"probs":[0.9961042404174805,0.003895747708156705],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":5,"variant":"base","tokens":168,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00043845176696777344,"mean_abs_dp":0.00017539835534989833,"oracle_top2_margin":0.8845294713973999,"near_tie":false,"probs":[0.0038327795919030905,0.004371823277324438,0.9347990155220032,0.007403908297419548,0.04959247261285782],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":6,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00048176199197769165,"mean_abs_dp":0.00048174336552619934,"oracle_top2_margin":0.8742504119873047,"near_tie":false,"probs":[0.9366434812545776,0.06335655599832535],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":7,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006570862606167793,"mean_abs_dp":0.0003285208949819207,"oracle_top2_margin":0.9630300998687744,"near_tie":false,"probs":[0.005762797314673662,0.0065468535758554935,0.9758031368255615,0.011887187138199806],"hidden":{"min_pos_cos":0.9991566743103716,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L01","q":0,"variant":"base","tokens":1793,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007559061050415039,"mean_abs_dp":0.00030237603932619094,"oracle_top2_margin":0.9371626973152161,"near_tie":false,"probs":[0.005918983370065689,0.9578202366828918,0.006499305833131075,0.007927858270704746,0.021833674982190132]},{"id":"own_L01","q":1,"variant":"base","tokens":1766,"calls":111,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00272291898727417,"mean_abs_dp":0.00272289477288723,"oracle_top2_margin":0.8993933200836182,"near_tie":false,"probs":[0.053026217967271805,0.9469737410545349]},{"id":"own_L01","q":2,"variant":"base","tokens":1802,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.004123806953430176,"mean_abs_dp":0.001518810788790385,"oracle_top2_margin":0.1986374706029892,"near_tie":false,"probs":[0.0382368303835392,0.13209444284439087,0.13460879027843475,0.2271498143672943,0.42981427907943726,0.03809588775038719]},{"id":"own_L02","q":0,"variant":"base","tokens":1518,"calls":95,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0012225508689880371,"mean_abs_dp":0.0004890041891485453,"oracle_top2_margin":0.9113143682479858,"near_tie":false,"probs":[0.040369823575019836,0.00244019809179008,0.0025770519860088825,0.9494967460632324,0.005116116255521774],"hidden":{"min_pos_cos":0.9911493146196642,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":1,"variant":"base","tokens":1497,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.937980651855469e-05,"mean_abs_dp":6.936304271221161e-05,"oracle_top2_margin":0.9763563275337219,"near_tie":false,"probs":[0.9881088137626648,0.01189120952039957],"hidden":{"min_pos_cos":0.9911493146196642,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":2,"variant":"base","tokens":1539,"calls":97,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005643963813781738,"mean_abs_dp":0.000290484051220119,"oracle_top2_margin":0.8973902463912964,"near_tie":false,"probs":[0.9406386613845825,0.0038889346178621054,0.043796274811029434,0.011676165275275707],"hidden":{"min_pos_cos":0.9911493146196642,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":3,"variant":"base","tokens":1499,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.698366582393646e-05,"mean_abs_dp":8.6943618953228e-05,"oracle_top2_margin":0.9387368559837341,"near_tie":false,"probs":[0.03071853518486023,0.9692814946174622],"hidden":{"min_pos_cos":0.9911493146196642,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L03","q":0,"variant":"base","tokens":1475,"calls":93,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00025832653045654297,"mean_abs_dp":0.00015386735321953893,"oracle_top2_margin":0.9612832069396973,"near_tie":false,"probs":[0.0023492889013141394,0.9741494059562683,0.010426209308207035,0.013075089082121849]},{"id":"own_L03","q":1,"variant":"base","tokens":1445,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.778406143188477e-05,"mean_abs_dp":7.775798439979553e-05,"oracle_top2_margin":0.973713755607605,"near_tie":false,"probs":[0.01322086714208126,0.9867790937423706]},{"id":"own_L03","q":2,"variant":"base","tokens":1451,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00013238191604614258,"mean_abs_dp":0.000132356071844697,"oracle_top2_margin":0.986961841583252,"near_tie":false,"probs":[0.9933485388755798,0.006651431787759066]},{"id":"own_L03","q":3,"variant":"base","tokens":1463,"calls":92,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0065285563468933105,"mean_abs_dp":0.003264292608946562,"oracle_top2_margin":0.6683993339538574,"near_tie":false,"probs":[0.018435021862387657,0.13955895602703094,0.8196683526039124,0.02233765460550785]}],"generated_at":"2026-10-03T11:17:28+00:00"},"red_int8lin":{"schema":"kev-decoder-readout-red-arms/1","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8lin_pf16","name":"kev_4b_decode_int8lin_pf16","kind":"decision-backbone","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":248,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"65b9c91d1e92a3997c3ccaa079a85374103f254cab655affd28157e178379a70","main_mlirb":{"bytes":5068103201,"sha256":"daecfad609b49d8e34db423fb6120c884d7278f3f34da7f174f62cb3976f571a"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8lin_pf16.h16c.aimodelc","bytes":12211947289,"tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"0a6f9733ca7fca724c7aafe4959bf6dad7421180b27658fae066ba7ba830bac2","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"ad3c924e2e9ae6f4ad4864a7487b089dccf6a13a91aed5160ca99b5dd8bf37ae","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b0808503de4b6cc9bddb7c1fea2664946106a74443965e9ac0aeb7c7e884a0b8","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"f715ebe6ee08f81444843fe2dbaf30066bcb1f1113fd734ffd4b85c890760f86","main-h16c.mlirb":"386d30a1d248ed135773ebb771653934cb8e8f3edd76b37ee35fd3821a8eae5c","main.hash":"8a6b309418481bde14facf22e9758bbd7f006ca175d82468a3ce6e9f32ef226f","metadata.json":"b3305c845e04a45eb51ab46c8d35997b38d17d2a526e608e14abdb6b5557c78a","stats.json":"8bd1c63e08e54922087221b47956a51431d4dd65ce1fffd2131fa7bab42793d7"}}},"chunk":16,"max_ctx":4096,"rows_file":{"path":"$ZOO_WORK_ROOT/_kev/readout/red_arms_v2.json","sha256":"2de94c557e94ecf318242331d7bcba61e1372e4b4fb0ab8e95a695cd0ca0be89","version":2},"model":"kev-4b","head":{"head_safetensors":"$ZOO_WORK_ROOT/_kev/oracle_4b/head/head.safetensors","head_safetensors_sha256":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head_json_sha256":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416","scale":0.0625,"temperature":2.406050072164233,"dtype":"float32","device":"cpu"},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle_4b/records_oracle.json","sha256":"eb3031df83a75b746beec3480a4bbb47787e13f8a12412ff135feb9f609c70f9","checkpoint":"jaredpalmer/kev-4b@591dcb5bd6d05eb0b5131ea6608f93f10243335c"},"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16_4b.json","sha256":"d430a661d8dbeec4f0a81466b3049ebe4ab5594f1f5d1bb7e781f9fc0c92dbe5","result":"FAIL"},"script":{"path":"conversion/kev/readout_gate.py","sha256":"b01658ccf43a0ddde9a88c82607c1f7db7b67adfd695478e9bf88a327f609172"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T19:42:06+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T19:42:06+09:00"}},"process":[{"shard":"red_arms_v2","runs":["tv4_000:q0:base","tv4_000:q0:state_swap","semif_a3f18f3a63d45345942b:q0:base","semif_a3f18f3a63d45345942b:q0:state_swap","semif_0b43ea8e24e74d621f1b:q0:base","semif_0b43ea8e24e74d621f1b:q0:state_swap","own_t01:q0:base","own_t01:q0:state_swap","tv4x_qnli_00:q0:base","tv4x_qnli_00:q0:state_swap","tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_00:q0:instr_negate_v2","tv4x_qnli_00:q0:instr_negate_v2","tv4x_paws_00:q0:base","tv4x_paws_00:q0:instr_negate_v2","tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_00:q0:instr_negate_v2","own_t05:q2:base","own_t05:q2:instr_negate_v2"],"prompts":20,"load_seconds":5.788814624975203,"load_split_seconds":{"model":0.018072790990117937,"main":5.770741833985085},"function_names":["main"],"process_wall_seconds":27.180287166993367,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.657723000011174},"contract_mismatch":[]}],"rule":"red = the perturbed runs against the unperturbed ones fail the gate's own bar (FACTS §7): (a) an argmax moves on at least one question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over runs of the run's mean |dp| > 0.002","v1_rule":"argmax moved on >= 1 row AND max |dp| > 0.02","arms":{"instr_negate_v2":{"rows":5,"argmax_moved":3,"argmax_moved_non_near_tie":3,"max_abs_dp_vs_base":0.9572172090411186,"mean_of_run_mean_abs_dp_vs_base":0.5553174579981714,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4x_tweet_offensive_00:q0","type":"noul","state_from":null,"tokens":69,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.007101953029632568,"mean_abs_dp_vs_base":0.007101923227310181,"base_probs":[0.40226981043815613,0.5977301597595215],"perturbed_probs":[0.39516791701316833,0.604832112789154],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":null,"tokens":95,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.833556056022644,"mean_abs_dp_vs_base":0.8335560038685799,"base_probs":[0.010109007358551025,0.9898909330368042],"perturbed_probs":[0.8436650633811951,0.1563349813222885],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"tv4x_paws_00:q0","type":"noul","state_from":null,"tokens":82,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08259040117263794,"mean_abs_dp_vs_base":0.08259037137031555,"base_probs":[0.375484436750412,0.6245155930519104],"perturbed_probs":[0.4580748379230499,0.5419252514839172],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_legacy_holdout_00:q0","type":"noul","state_from":null,"tokens":79,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9572172090411186,"mean_abs_dp_vs_base":0.9572171897161752,"base_probs":[0.006657944526523352,0.9933421015739441],"perturbed_probs":[0.9638751149177551,0.03612489253282547],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t05:q2","type":"noul","state_from":null,"tokens":118,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8961218185722828,"mean_abs_dp_vs_base":0.8961218018084764,"base_probs":[0.9958510398864746,0.004149001091718674],"perturbed_probs":[0.0997292548418045,0.9002708196640015],"base_argmax":0,"perturbed_argmax":1,"oracle_argmax":0}]},"state_swap":{"rows":5,"argmax_moved":5,"argmax_moved_non_near_tie":5,"max_abs_dp_vs_base":0.9879871606826782,"mean_of_run_mean_abs_dp_vs_base":0.5279587732014867,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4_000:q0","type":"choice","state_from":"tv4_001","tokens":80,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.08622069656848907,"mean_abs_dp_vs_base":0.047366876155138016,"base_probs":[0.3164719045162201,0.12716685235500336,0.2846033573150635,0.2717578709125519],"perturbed_probs":[0.23025120794773102,0.16317184269428253,0.2760903239250183,0.3304866552352905],"base_argmax":0,"perturbed_argmax":3,"oracle_argmax":0},{"row":"semif_a3f18f3a63d45345942b:q0","type":"choice","state_from":"semif_0b43ea8e24e74d621f1b","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9862070907838643,"mean_abs_dp_vs_base":0.6574713729011515,"base_probs":[0.0011957623064517975,0.01140426192432642,0.9874000549316406],"perturbed_probs":[0.985121488571167,0.013685563579201698,0.0011929641477763653],"base_argmax":2,"perturbed_argmax":0,"oracle_argmax":2},{"row":"semif_0b43ea8e24e74d621f1b:q0","type":"choice","state_from":"semif_942ec65dac0bfb9874f9","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9795596986077726,"mean_abs_dp_vs_base":0.6530397609264279,"base_probs":[0.015171232633292675,0.9837369322776794,0.0010919406777247787],"perturbed_probs":[0.37194201350212097,0.004177233669906855,0.6238807439804077],"base_argmax":1,"perturbed_argmax":2,"oracle_argmax":1},{"row":"own_t01:q0","type":"choice","state_from":"own_t03","tokens":245,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.5580359268933535,"mean_abs_dp_vs_base":0.2939287073910236,"base_probs":[0.5720820426940918,0.08988572657108307,0.19963927567005157,0.02638007327914238,0.11201288551092148],"perturbed_probs":[0.014046115800738335,0.13295266032218933,0.0228534284979105,0.18216459453105927,0.647983193397522],"base_argmax":0,"perturbed_argmax":4,"oracle_argmax":0},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":"tv4x_paws_00","tokens":52,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9879871606826782,"mean_abs_dp_vs_base":0.9879871486336924,"base_probs":[0.010109007358551025,0.9898909330368042],"perturbed_probs":[0.9980961680412292,0.0019037964520975947],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1}]}},"base_vs_gate_run":[{"row":"tv4_000:q0","bit_equal_gate_run":true},{"row":"semif_a3f18f3a63d45345942b:q0","bit_equal_gate_run":true},{"row":"semif_0b43ea8e24e74d621f1b:q0","bit_equal_gate_run":true},{"row":"own_t01:q0","bit_equal_gate_run":true},{"row":"tv4x_qnli_00:q0","bit_equal_gate_run":true},{"row":"tv4x_tweet_offensive_00:q0","bit_equal_gate_run":true},{"row":"tv4x_paws_00:q0","bit_equal_gate_run":true},{"row":"tv4x_legacy_holdout_00:q0","bit_equal_gate_run":true},{"row":"own_t05:q2","bit_equal_gate_run":true}],"base_bit_equal_all":true,"all_arms_red":true,"reset_bit_equal":true,"result":"RED (every arm)","seconds":48.8168548749818,"generated_at":"2026-10-03T11:18:33+00:00"},"r4_summary":{"schema":"kev-r4-summary-4b/1","bar":"FACTS §7: argmax equal on every question whose oracle top-2 margin > 0.02, max |dp| <= 0.02, mean of the rows' mean |dp| <= 0.002, reset bit-equal, finite (oracle = the author's fp32 Kev-4B)","fixture_434":{"fp16":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_fp16_pf16_4b.json","transcript_sha256":"f018154ed4ea0e5d94a1ab63f45118de01f4119844bad23eb60c49f2a9776b51","result":"PASS","rows":434,"argmax_non_near_tie":[422,422],"argmax_near_tie":[9,12],"max_abs_dp":0.015406966209411621,"mean_of_run_mean_abs_dp":0.0007834579897192555,"row_max_abs_dp_quantiles":{"p50":0.000530630350112915,"p90":0.0028353899717330927,"max":0.015406966209411621},"rows_above":{"0.01":2,"0.02":0},"worst_10":[{"id":"semif_1105577da4c8dad4609d","q":0,"source":"semif_authored144","T":88,"max_abs_dp":0.015406966209411621,"near_tie":false},{"id":"semif_8a4c3de28be95c105ca1","q":0,"source":"semif_authored144","T":72,"max_abs_dp":0.013801813125610352,"near_tie":false},{"id":"tv4x_composition_holdout_11","q":0,"source":"transfer_v4_dev_sources","T":166,"max_abs_dp":0.009346753358840942,"near_tie":false},{"id":"tv4s_16","q":0,"source":"transfer_v4_dev_score","T":100,"max_abs_dp":0.008689403533935547,"near_tie":false},{"id":"tv4x_composition_holdout_08","q":0,"source":"transfer_v4_dev_sources","T":166,"max_abs_dp":0.00756305456161499,"near_tie":false},{"id":"tv4x_composition_holdout_10","q":0,"source":"transfer_v4_dev_sources","T":166,"max_abs_dp":0.00756305456161499,"near_tie":false},{"id":"tv4_041","q":0,"source":"transfer_v4_dev_head","T":208,"max_abs_dp":0.007135868072509766,"near_tie":false},{"id":"semif_942ec65dac0bfb9874f9","q":0,"source":"semif_authored144","T":60,"max_abs_dp":0.0066162943840026855,"near_tie":false},{"id":"semif_97d2ed2591fc7638806c","q":0,"source":"semif_authored144","T":82,"max_abs_dp":0.0063040852546691895,"near_tie":false},{"id":"tv4_007","q":0,"source":"transfer_v4_dev_head","T":80,"max_abs_dp":0.005870342254638672,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":0.9951249147360116,"ms_per_call_warm":{"n":3972,"median":47.738416484207846,"p10":45.19430801155977,"p90":50.899087500874884},"load_seconds_median":1.5881113749928772,"contended":true,"other_gpu_processes_start":2,"gpu_lock_start":0,"bundle":{"name":"kev_4b_decode_fp16_pf16","main_mlirb_bytes":8414007636,"main_mlirb_sha256":"e38dc3caaeae1ccb21eebea3148a2aa0010185ae8fcdd7a757bb5a1926b05806","aimodelc_bytes":15557770108,"aimodelc_tree_sha256":"e1155cab0743b8d4482ad6aa2e6beb2db740eb50bb78eba4b912e578a7d0e663","compression":null},"export":{"seconds":{"load":0.7,"quantize":0.0,"export":57.3,"optimize":0.6,"export_optimize":57.9,"save":6.4,"total":65.2},"aot_seconds":31.9,"du_bundle":"7.9G","du_aimodelc":"15G","int8_linear_modules":null,"int8_params":null,"fp16_layers":null},"vs_fp16":{"rows":434,"max_abs_dp":0.0,"mean_row_max_abs_dp":0.0,"p50":0.0,"p90":0.0,"argmax_equal":434},"red_arms_v2":{"result":"RED (every arm)","transcript_sha256":"742bee7fa189cf79696d487470a9ba870f2026b3939f33552aa36459cec167df","arms":{"instr_negate_v2":{"argmax_moved":3,"rows":5,"max_abs_dp_vs_base":0.9575627236627042,"mean_of_run_mean_abs_dp_vs_base":0.555058476049453,"red":true},"state_swap":{"argmax_moved":5,"rows":5,"max_abs_dp_vs_base":0.9879291027318686,"mean_of_run_mean_abs_dp_vs_base":0.528297510870422,"red":true}},"base_bit_equal_gate_run":true,"reset":true},"graph_seconds_all_rows":220.8317540830467,"graph_seconds_warm_rows":193.1360742090328,"warm_rows":423,"first_call_ms_per_process":[8204,1612,1557,1510,1474,1678,1541,1672,1661,1669,1700],"calls":4055},"int8lin":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16_4b.json","transcript_sha256":"d430a661d8dbeec4f0a81466b3049ebe4ab5594f1f5d1bb7e781f9fc0c92dbe5","result":"FAIL","rows":434,"argmax_non_near_tie":[422,422],"argmax_near_tie":[9,12],"max_abs_dp":0.060132622718811035,"mean_of_run_mean_abs_dp":0.001917580727576835,"row_max_abs_dp_quantiles":{"p50":0.0011676326394081116,"p90":0.007969838380813598,"max":0.060132622718811035},"rows_above":{"0.01":28,"0.02":5},"worst_10":[{"id":"semif_1105577da4c8dad4609d","q":0,"source":"semif_authored144","T":88,"max_abs_dp":0.060132622718811035,"near_tie":false},{"id":"semif_d880e65fef5e2993fdd0","q":0,"source":"semif_authored144","T":85,"max_abs_dp":0.029746264219284058,"near_tie":false},{"id":"semif_7ed75d854d99b3a81916","q":0,"source":"semif_authored144","T":68,"max_abs_dp":0.025355279445648193,"near_tie":false},{"id":"semif_ef3e0f8e657bf59f8a29","q":0,"source":"semif_authored144","T":83,"max_abs_dp":0.022765278816223145,"near_tie":false},{"id":"tv4_012","q":0,"source":"transfer_v4_dev_head","T":83,"max_abs_dp":0.02085888385772705,"near_tie":false},{"id":"tv4_007","q":0,"source":"transfer_v4_dev_head","T":80,"max_abs_dp":0.019733965396881104,"near_tie":false},{"id":"semif_8e8c3804a3c15ebb31e3","q":0,"source":"semif_authored144","T":69,"max_abs_dp":0.01799994707107544,"near_tie":false},{"id":"semif_f756e1cb04e11766341e","q":0,"source":"semif_authored144","T":89,"max_abs_dp":0.01778966188430786,"near_tie":false},{"id":"tv4x_qnli_19","q":0,"source":"transfer_v4_dev_sources","T":85,"max_abs_dp":0.016363978385925293,"near_tie":false},{"id":"semif_33d6e5da58f0fee2490d","q":0,"source":"semif_authored144","T":73,"max_abs_dp":0.014631807804107666,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":0.9241736357846124,"ms_per_call_warm":{"n":3972,"median":79.4767295155907,"p10":53.222950611962005,"p90":115.98405421536881},"load_seconds_median":1.7113432089972775,"contended":true,"other_gpu_processes_start":10,"gpu_lock_start":0,"bundle":{"name":"kev_4b_decode_int8lin_pf16","main_mlirb_bytes":5068103201,"main_mlirb_sha256":"daecfad609b49d8e34db423fb6120c884d7278f3f34da7f174f62cb3976f571a","aimodelc_bytes":12211947289,"aimodelc_tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":248,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"}},"export":{"seconds":{"load":0.9,"quantize":32.4,"export":98.5,"optimize":0.6,"export_optimize":99.2,"save":13.1,"total":145.8},"aot_seconds":218.8,"du_bundle":"4.8G","du_aimodelc":"11G","int8_linear_modules":248,"int8_params":3569090560,"fp16_layers":[]},"vs_fp16":{"rows":434,"max_abs_dp":0.07553958892822266,"mean_row_max_abs_dp":0.0029618223970283835,"p50":0.0010183341801166534,"p90":0.008269301056861874,"argmax_equal":432},"red_arms_v2":{"result":"RED (every arm)","transcript_sha256":"f0ed7aee1a33c1ad53276a1f20d7fd20c60ad962c84d242c4ba2c5ade7c3a05c","arms":{"instr_negate_v2":{"argmax_moved":3,"rows":5,"max_abs_dp_vs_base":0.9572172090411186,"mean_of_run_mean_abs_dp_vs_base":0.5553174579981714,"red":true},"state_swap":{"argmax_moved":5,"rows":5,"max_abs_dp_vs_base":0.9879871606826782,"mean_of_run_mean_abs_dp_vs_base":0.5279587732014867,"red":true}},"base_bit_equal_gate_run":true,"reset":true},"graph_seconds_all_rows":375.23181703913724,"graph_seconds_warm_rows":329.2472207881219,"warm_rows":423,"first_call_ms_per_process":[11377,2022,2043,2017,2048,2674,3272,2597,2797,4264,6271],"calls":4055}},"heldout_130":{"fp16":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_heldout_fp16_pf16_4b.json","transcript_sha256":"2360d48eb9369fc4a429f489d943a08248eb657e8a8423279a13c5441c49a684","result":"PASS","rows":130,"argmax_non_near_tie":[126,126],"argmax_near_tie":[2,4],"max_abs_dp":0.010867595672607422,"mean_of_run_mean_abs_dp":0.0007171148204547651,"row_max_abs_dp_quantiles":{"p50":0.0004649041220545769,"p90":0.0027806222438812277,"max":0.010867595672607422},"rows_above":{"0.01":1,"0.02":0},"worst_10":[{"id":"tv4sh_11","q":0,"source":"transfer_v4_dev_score_heldout","T":97,"max_abs_dp":0.010867595672607422,"near_tie":false},{"id":"tv4h_06","q":0,"source":"transfer_v4_dev_head_heldout","T":90,"max_abs_dp":0.008257865905761719,"near_tie":false},{"id":"tv4sh_12","q":0,"source":"transfer_v4_dev_score_heldout","T":98,"max_abs_dp":0.006889283657073975,"near_tie":false},{"id":"tv4sh_08","q":0,"source":"transfer_v4_dev_score_heldout","T":97,"max_abs_dp":0.0061570703983306885,"near_tie":false},{"id":"tv4sh_16","q":0,"source":"transfer_v4_dev_score_heldout","T":97,"max_abs_dp":0.004568547010421753,"near_tie":false},{"id":"tv4xh_emotion_03","q":0,"source":"transfer_v4_dev_sources_heldout","T":49,"max_abs_dp":0.004074275493621826,"near_tie":false},{"id":"tv4h_19","q":0,"source":"transfer_v4_dev_head_heldout","T":97,"max_abs_dp":0.003862142562866211,"near_tie":false},{"id":"tv4h_31","q":0,"source":"transfer_v4_dev_head_heldout","T":75,"max_abs_dp":0.0037256479263305664,"near_tie":false},{"id":"tv4sh_18","q":0,"source":"transfer_v4_dev_score_heldout","T":99,"max_abs_dp":0.003527820110321045,"near_tie":false},{"id":"tv4h_02","q":0,"source":"transfer_v4_dev_head_heldout","T":84,"max_abs_dp":0.0034202933311462402,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":null,"ms_per_call_warm":{"n":792,"median":55.46641651017126,"p10":50.557908607879654,"p90":82.2863083856646},"load_seconds_median":15.66007595800329,"contended":true,"other_gpu_processes_start":8,"gpu_lock_start":0,"bundle":{"name":"kev_4b_decode_fp16_pf16","main_mlirb_bytes":8414007636,"main_mlirb_sha256":"e38dc3caaeae1ccb21eebea3148a2aa0010185ae8fcdd7a757bb5a1926b05806","aimodelc_bytes":15557770108,"aimodelc_tree_sha256":"e1155cab0743b8d4482ad6aa2e6beb2db740eb50bb78eba4b912e578a7d0e663","compression":null},"vs_fp16":{"rows":130,"max_abs_dp":0.0,"mean_row_max_abs_dp":0.0,"p50":0.0,"p90":0.0,"argmax_equal":130},"graph_seconds_all_rows":132.01406111576944,"graph_seconds_warm_rows":49.66361590879387,"warm_rows":126,"first_call_ms_per_process":[12316,19457,3689,46072]},"int8lin":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_heldout_int8lin_pf16_4b.json","transcript_sha256":"66d1659855187b67a7050d990dae5fde326fcb7e501598f05c98cc0241a4ec0b","result":"FAIL","rows":130,"argmax_non_near_tie":[126,126],"argmax_near_tie":[2,4],"max_abs_dp":0.028466999530792236,"mean_of_run_mean_abs_dp":0.0016867079466278366,"row_max_abs_dp_quantiles":{"p50":0.0012055635452270508,"p90":0.006838726997375491,"max":0.028466999530792236},"rows_above":{"0.01":5,"0.02":1},"worst_10":[{"id":"tv4sh_11","q":0,"source":"transfer_v4_dev_score_heldout","T":97,"max_abs_dp":0.028466999530792236,"near_tie":false},{"id":"tv4h_19","q":0,"source":"transfer_v4_dev_head_heldout","T":97,"max_abs_dp":0.016419708728790283,"near_tie":false},{"id":"tv4h_06","q":0,"source":"transfer_v4_dev_head_heldout","T":90,"max_abs_dp":0.014124870300292969,"near_tie":false},{"id":"tv4sh_18","q":0,"source":"transfer_v4_dev_score_heldout","T":99,"max_abs_dp":0.01083335280418396,"near_tie":false},{"id":"tv4xh_emotion_02","q":0,"source":"transfer_v4_dev_sources_heldout","T":53,"max_abs_dp":0.010415822267532349,"near_tie":true},{"id":"tv4h_14","q":0,"source":"transfer_v4_dev_head_heldout","T":60,"max_abs_dp":0.009692668914794922,"near_tie":false},{"id":"tv4sh_08","q":0,"source":"transfer_v4_dev_score_heldout","T":97,"max_abs_dp":0.009523570537567139,"near_tie":false},{"id":"tv4xh_paws_04","q":0,"source":"transfer_v4_dev_sources_heldout","T":76,"max_abs_dp":0.009478181600570679,"near_tie":false},{"id":"tv4xh_sciq_05","q":0,"source":"transfer_v4_dev_sources_heldout","T":51,"max_abs_dp":0.009422540664672852,"near_tie":false},{"id":"tv4sh_07","q":0,"source":"transfer_v4_dev_score_heldout","T":100,"max_abs_dp":0.008283793926239014,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":null,"ms_per_call_warm":{"n":792,"median":106.13749999902211,"p10":88.52106279227883,"p90":117.78658721596001},"load_seconds_median":3.307429457985563,"contended":true,"other_gpu_processes_start":24,"gpu_lock_start":0,"bundle":{"name":"kev_4b_decode_int8lin_pf16","main_mlirb_bytes":5068103201,"main_mlirb_sha256":"daecfad609b49d8e34db423fb6120c884d7278f3f34da7f174f62cb3976f571a","aimodelc_bytes":12211947289,"aimodelc_tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":248,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"}},"vs_fp16":{"rows":130,"max_abs_dp":0.017599403858184814,"mean_row_max_abs_dp":0.002362516003016096,"p50":0.0010527782142162323,"p90":0.006056779623031619,"argmax_equal":128},"graph_seconds_all_rows":107.88500707829371,"graph_seconds_warm_rows":82.56871257929015,"warm_rows":126,"first_call_ms_per_process":[6856,5084,6272,5779]}},"table":[{"mode":"fp16","result_fixture":"PASS","rows":434,"argmax_non_near_tie":[422,422],"argmax_near_tie":[9,12],"max_abs_dp":0.015406966209411621,"mean_of_run_mean_abs_dp":0.0007834579897192555,"heldout":{"result":"PASS","rows":130,"argmax_non_near_tie":[126,126],"argmax_near_tie":[2,4],"max_abs_dp":0.010867595672607422,"mean_of_run_mean_abs_dp":0.0007171148204547651},"red_arms_v2":"RED (every arm)","ms_per_call_warm_median_contended":47.738416484207846,"main_mlirb_bytes":8414007636,"aimodelc_bytes":15557770108,"load_seconds_median":1.5881113749928772,"graph_seconds_all_rows":220.8317540830467,"graph_seconds_warm_rows":193.1360742090328,"first_call_ms_per_process":[8204,1612,1557,1510,1474,1678,1541,1672,1661,1669,1700],"iphone_reference":{"bytes":6400000000.0,"main_mlirb_over_reference":1.314688693125,"aimodelc_over_reference":2.430901579375}},{"mode":"int8lin","result_fixture":"FAIL","rows":434,"argmax_non_near_tie":[422,422],"argmax_near_tie":[9,12],"max_abs_dp":0.060132622718811035,"mean_of_run_mean_abs_dp":0.001917580727576835,"heldout":{"result":"FAIL","rows":130,"argmax_non_near_tie":[126,126],"argmax_near_tie":[2,4],"max_abs_dp":0.028466999530792236,"mean_of_run_mean_abs_dp":0.0016867079466278366},"red_arms_v2":"RED (every arm)","ms_per_call_warm_median_contended":79.4767295155907,"main_mlirb_bytes":5068103201,"aimodelc_bytes":12211947289,"load_seconds_median":1.7113432089972775,"graph_seconds_all_rows":375.23181703913724,"graph_seconds_warm_rows":329.2472207881219,"first_call_ms_per_process":[11377,2022,2043,2017,2048,2674,3272,2597,2797,4264,6271],"iphone_reference":{"bytes":6400000000.0,"main_mlirb_over_reference":0.79189112515625,"aimodelc_over_reference":1.90811676390625}}],"torch_side":{"oracle":{"questions":434,"near_ties":12,"prefix_check":{"subset":30,"max_abs_dp":1.9669532775878906e-06,"argmax_equal_all":true,"path":"model.probs(enc): DecisionModel.probs -> probs_and_prefix (state once, rows on its cache)"},"determinism":{"record":"tv4_000","logits_bit_equal":true},"loop_seconds":395.6,"sha256":"eb3031df83a75b746beec3480a4bbb47787e13f8a12412ff135feb9f609c70f9","_path":"$ZOO_WORK_ROOT/_kev/results/oracle_summary_4b.json","_sha256":"460f5ab3dbd9ebd6a79de12cd2ec734164d61050eded897c9f0899994e2d12fe"},"merged_parity":{"questions":434,"argmax_equal":434,"near_ties":12,"near_ties_argmax_equal":12,"max_abs_dp":0.0,"bit_equal_logits_questions":434,"hidden_max_abs_diff":{"tv4_000":0.0,"semif_a3f18f3a63d45345942b":0.0,"own_t01":0.0,"own_j03":0.0,"own_m01":0.0,"own_L02":0.0},"pass":true,"_path":"$ZOO_WORK_ROOT/_kev/results/parity_merged_vs_adapter_4b.json","_sha256":"d21c22c4076ab235578a6fbea184e131c017a6d140c1e4a0a3b2904a6f9f69ae"},"merge_formula":{"tensors":[{"tensor":"layers.0.linear_attn.in_proj_qkv","bit_equal":true,"max_abs_diff":0.0,"delta_rel_to_W":0.020087778568267822},{"tensor":"layers.3.self_attn.q_proj","bit_equal":true,"max_abs_diff":0.0,"delta_rel_to_W":0.013592205010354519},{"tensor":"layers.9.mlp.down_proj","bit_equal":true,"max_abs_diff":0.0,"delta_rel_to_W":0.01572984643280506}],"scaling":2.0,"_path":"$ZOO_WORK_ROOT/_kev/results/merge_formula_check_4b.json","_sha256":"9813fd3741a137390b8b48dfe3fd37e81345c63440a0f58e01dac36173aa9a86"},"module_load":{"result":"PASS","load_report":{"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[]},"_path":"$ZOO_WORK_ROOT/_kev/results/module_load_4b.json","_sha256":"e2c77241cdbd3d2c63caa176b748ea3bc23369254e3a09bbd89cf3e76acfe5bf"},"torch_parity":{"p1":"PASS","questions":434,"argmax_equal":434,"max_abs_dp":6.556510925292969e-06,"min_pos_cos":0.9999999965183307,"p2":"PASS","p3":{"16":{"rows":3,"argmax_equal_oracle":3,"vs_oracle_max_abs_dp":6.556510925292969e-07,"vs_oracle_min_pos_cos":0.9999999999696568,"vs_oracle_hidden_max_abs_diff":0.000148773193359375,"seconds":260.2945987910207,"chunks":106,"bit_equal_p1_run":1},"64":{"rows":3,"argmax_equal_oracle":3,"vs_oracle_max_abs_dp":6.556510925292969e-07,"vs_oracle_min_pos_cos":0.9999999999696568,"vs_oracle_hidden_max_abs_diff":0.000148773193359375,"seconds":102.71531437596423,"chunks":28,"vs_s16_hidden_max_abs_diff":0.0,"vs_s16_max_abs_dp":0.0}},"_path":"$ZOO_WORK_ROOT/_kev/results/parity_decoder_torch_4b.json","_sha256":"a173a0e142d6a515d2b30b7ade9f8d808249005ff93a87da9666e4974bee3805"}},"kev_0_8b_round3_reference":{"fp16":{"fixture":{"result":"PASS","argmax_non_near_tie":[420,420],"argmax_near_tie":[12,14],"max_abs_dp":0.011678963899612427,"mean_of_run_mean_abs_dp":0.0009687347178998231},"heldout":{"result":"PASS","max_abs_dp":0.005739271640777588,"mean_of_run_mean_abs_dp":0.0008054346284375359},"main_mlirb_bytes":1506481909,"aimodelc_bytes":2505682731},"int8lin":{"fixture":{"result":"FAIL","argmax_non_near_tie":[420,420],"argmax_near_tie":[11,14],"max_abs_dp":0.04313388466835022,"mean_of_run_mean_abs_dp":0.0032562481012639575},"heldout":{"result":"FAIL","max_abs_dp":0.02733546495437622,"mean_of_run_mean_abs_dp":0.0029399254602956205},"main_mlirb_bytes":1040063350,"aimodelc_bytes":2039283435},"_sha256":"a1f743aece6eeceea1700e9bb780a9a8f4dd3952509155442d1cafc82ee3ab62"},"generated_at":"2026-10-03T20:48:08+09:00"},"quant_api":{"schema":"kev-r6-quant-api/1","started":"2026-10-03T21:12:05+09:00","versions":{"coreai_opt":"0.2.1","torch":"2.9.0"},"toy":"Embedding(4096,1024) + Linear(1024,1024) x 2, fp16 weights ~ StudentT(df 4) x 0.02, seed 20261003; forward input_ids [1, T] -> [1, T, 1024]; export with T dynamic (2..64)","api_reading":{"qscheme":"coreai_opt.quantization.spec.qscheme.QuantizationScheme: 'symmetric', 'asymmetric', 'symmetric_with_clipping' (no 'affine')","qformulation":"QuantizationFormulation: 'zp' (default) and 'minval'; the CoreAI backend takes both, CoreML only zp","granularity":"per_tensor | per_channel(axis) | per_block(axis 0 or 1, block_size int > 0, or a tuple with axis None); the axis length must be divisible by block_size (_BlockSizeMismatchError)","dtype":"int8 int4 int2 uint8 uint4 uint2 float8_e4m3fn float8_e5m2 float4_e2m1fn_x2","embedding":"the eager quantizer registers F.embedding (supported_ops_registry 'embedding'); the per-block default axis for nn.Embedding is 1 (embedding_dim); the exporter excludes it by type ('torch.nn.modules.sparse.Embedding': None)","palettization":"compression.py also has palettize_pytorch_model (KMeansPalettizer); not exercised here (not in this round's variant list)"},"variants":{"b32_swc":{"why":"control: the exporter's int8lin linear spec","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.013065916980849579},"modules":{"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.00730543676763773,"max_abs_err":0.00384521484375},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.007340621668845415,"max_abs_err":0.00482940673828125}},"export":{"ok":true,"seconds":0.3970830420148559,"main_mlirb_bytes":10623440},"eager_vs_fp16_out_rel":0.010374062694609165,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.11904950000462122,"stderr_tail":"","bytes":14837422,"resources_bin":14811176,"storage_types":[{"name":"Float16","count":4292609},{"count":2129920,"name":"Int8"},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.11621775000821799,"stderr_tail":"","bytes":14837418,"resources_bin":14811176,"storage_types":[{"count":4292609,"name":"Float16"},{"count":2129920,"name":"Int8"},{"name":"UInt32","count":4},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.037424916023155674,"stderr_tail":"","bytes":10626986,"resources_bin":10616856,"storage_types":[{"count":4292609,"name":"Float16"},{"name":"Int8","count":2129920},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}]}]},"b16_swc":{"why":"block 16, symmetric_with_clipping","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":16,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.00861533300485462},"modules":{"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,64],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,64],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.00586461927741766,"max_abs_err":0.0037841796875},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,64],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,64],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.0058895680122077465,"max_abs_err":0.00482940673828125}},"export":{"ok":true,"seconds":0.10971879100543447,"main_mlirb_bytes":10754527},"eager_vs_fp16_out_rel":0.008312258869409561,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.11879204100114293,"stderr_tail":"","bytes":14968492,"resources_bin":14942248,"storage_types":[{"name":"Float16","count":4390913},{"count":2162688,"name":"Int8"},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.11673079099273309,"stderr_tail":"","bytes":14968488,"resources_bin":14942248,"storage_types":[{"count":4390913,"name":"Float16"},{"name":"Int8","count":2162688},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03819787499378435,"stderr_tail":"","bytes":10758055,"resources_bin":10747928,"storage_types":[{"count":4390913,"name":"Float16"},{"count":2162688,"name":"Int8"},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]}]},"b32_asym":{"why":"block 32, asymmetric (qformulation default zp)","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.007052540982840583},"modules":{"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":119},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.0056922826915979385,"max_abs_err":0.002166748046875},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":119},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.005700419656932354,"max_abs_err":0.002460479736328125}},"export":{"ok":true,"seconds":0.11478687499766238,"main_mlirb_bytes":10689118},"eager_vs_fp16_out_rel":0.00813352968543768,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.11909383299644105,"stderr_tail":"","bytes":14903285,"resources_bin":14876720,"storage_types":[{"count":4292609,"name":"Float16"},{"name":"Int8","count":2162688},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.12045483398833312,"stderr_tail":"","bytes":14903281,"resources_bin":14876720,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2162688},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03737354101031087,"stderr_tail":"","bytes":10692854,"resources_bin":10682400,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2162688},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]}]},"b32_asym_minval":{"why":"block 32, asymmetric, qformulation minval","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1},"qformulation":"minval"}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.0062738750129938126},"modules":{"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":null,"minval":[1024,32],"dequant_bit_equal_constexpr_op":true,"rel_err":0.0055404105223715305,"max_abs_err":0.0020904541015625},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":null,"minval":[1024,32],"dequant_bit_equal_constexpr_op":true,"rel_err":0.005548891611397266,"max_abs_err":0.0025968551635742188}},"export":{"ok":true,"seconds":0.11184204200981185,"main_mlirb_bytes":10754676},"eager_vs_fp16_out_rel":0.007846838794648647,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.11790504201781005,"stderr_tail":"","bytes":14968821,"resources_bin":14942256,"storage_types":[{"count":4325378,"name":"Float16"},{"count":2129920,"name":"Int8"},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.127060208003968,"stderr_tail":"","bytes":14968817,"resources_bin":14942256,"storage_types":[{"count":4325378,"name":"Float16"},{"name":"Int8","count":2129920},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03902704099891707,"stderr_tail":"","bytes":10758392,"resources_bin":10747936,"storage_types":[{"count":4325378,"name":"Float16"},{"count":2129920,"name":"Int8"},{"name":"UInt32","count":4},{"name":"Int32","count":3}]}]},"b32_sym":{"why":"block 32, symmetric (absmax / 127.5)","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.0062579170044045895},"modules":{"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.0076099359430372715,"max_abs_err":0.00396728515625},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-128,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.007644154597073793,"max_abs_err":0.0048828125}},"export":{"ok":true,"seconds":0.17911820800509304,"main_mlirb_bytes":10623436},"eager_vs_fp16_out_rel":0.010881621390581131,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.11672691701096483,"stderr_tail":"","bytes":14837410,"resources_bin":14811176,"storage_types":[{"count":4292609,"name":"Float16"},{"name":"Int8","count":2129920},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.11702933398191817,"stderr_tail":"","bytes":14837406,"resources_bin":14811176,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2129920},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03838429198367521,"stderr_tail":"","bytes":10626975,"resources_bin":10616856,"storage_types":[{"count":4292609,"name":"Float16"},{"name":"Int8","count":2129920},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]}]},"affine_name":{"why":"qscheme 'affine' (the launch's word; not an API name)","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"affine","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":null},"module_name_configs":{}},"quantize":{"ok":false,"seconds":0.0002491670020390302,"error":"ValidationError: 1 validation error for QuantizerConfig\nglobal_config.op_state_spec.weight.qscheme\n  Input should be 'symmetric', 'asymmetric' or 'symmetric_with_clipping' [type=enum, input_value='affine', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.14/v/enum","traceback_tail":"Traceback (most recent call last):\n  File \"$ZOO_WORK_ROOT/_kev/scripts/r6_quant_api.py\", line 201, in main\n    q = quantize_pytorch_model(m, (ids,), None, copy.deepcopy(c))\n        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"$ZOO_WORK_ROOT/coreai-models/python/src/coreai_models/export/compression.py\", line 141, in quantize_pytorch_model\n    config = QuantizerConfig.from_dict({\"quantization_config\": quantization_config})\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"$ZOO_WORK_ROOT/coreai-models/.venv/lib/python3.11/site-packages/coreai_opt/config/compression_config.py\", line 1156, in from_dict\n    return cls(**config_dict[config_key])\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"$ZOO_WORK_ROOT/coreai-models/.venv/lib/python3.11/site-packages/pydantic/main.py\", line 263, in __init__\n    validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\npydantic_core._pydantic_core.ValidationError: 1 validation error for QuantizerConfig\nglobal_config.op_state_spec.weight.qscheme\n  Input should be 'symmetric', 'asymmetric' or 'symmetric_with_clipping' [type=enum, input_value='affine', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.14/v/enum\n"}},"embed_b32_swc":{"why":"Embedding per-block-32 (axis 1 = hidden) int8 swc, linears int8 b32","config":{"execution_mode":"eager","global_config":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null}},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.009955957997590303},"modules":{"embed":{"type":"ParametrizedEmbedding","weight_shape":[4096,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[4096,32],"scale_dtype":"torch.float16","zero_point":{"shape":[4096,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.007286697626113892,"max_abs_err":0.0052490234375},"l1":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.00730543676763773,"max_abs_err":0.00384521484375},"l2":{"type":"ParametrizedLinear","weight_shape":[1024,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[1024,32],"scale_dtype":"torch.float16","zero_point":{"shape":[1024,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.007340621668845415,"max_abs_err":0.00482940673828125}},"export":{"ok":true,"seconds":0.11494395899353549,"main_mlirb_bytes":6691780},"eager_vs_fp16_out_rel":0.012589088641107082,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.12008120800601318,"stderr_tail":"","bytes":10906926,"resources_bin":10879020,"storage_types":[{"count":6422528,"name":"Int8"},{"name":"Float16","count":229377},{"name":"UInt32","count":4},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.11787116600316949,"stderr_tail":"","bytes":10906922,"resources_bin":10879020,"storage_types":[{"count":6422528,"name":"Int8"},{"name":"Float16","count":229377},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03694233298301697,"stderr_tail":"","bytes":6695328,"resources_bin":6684700,"storage_types":[{"count":6422528,"name":"Int8"},{"name":"Float16","count":229377},{"name":"UInt32","count":4},{"name":"Int32","count":3}]}]},"embed_only_b32_swc":{"why":"Embedding per-block-32 int8 swc only, linears fp16","config":{"execution_mode":"eager","global_config":null,"module_type_configs":{"torch.nn.modules.conv.Conv1d":null,"torch.nn.modules.sparse.Embedding":{"op_state_spec":{"weight":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}}},"op_input_spec":null,"op_output_spec":null},"torch.nn.modules.linear.Linear":null},"module_name_configs":{}},"quantize":{"ok":true,"seconds":0.0077889579988550395},"modules":{"embed":{"type":"ParametrizedEmbedding","weight_shape":[4096,1024],"dequant_dtype":"torch.float16","parametrizations":["WeightDequantizeModuleParametrization"],"codes_dtype":"torch.int8","codes_min":-127,"codes_max":127,"scale_shape":[4096,32],"scale_dtype":"torch.float16","zero_point":{"shape":[4096,32],"absmax":0},"minval":null,"dequant_bit_equal_constexpr_op":true,"rel_err":0.007286697626113892,"max_abs_err":0.0052490234375}},"export":{"ok":true,"seconds":0.10560408301535062,"main_mlirb_bytes":8656985},"eager_vs_fp16_out_rel":0.007385397795587778,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.05239570900448598,"stderr_tail":"","bytes":12871924,"resources_bin":12845100,"storage_types":[{"name":"Int8","count":4325376},{"name":"Float16","count":2359297},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.05184091700357385,"stderr_tail":"","bytes":12871920,"resources_bin":12845100,"storage_types":[{"name":"Int8","count":4325376},{"count":2359297,"name":"Float16"},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.037844874983420596,"stderr_tail":"","bytes":8660408,"resources_bin":8650780,"storage_types":[{"name":"Int8","count":4325376},{"count":2359297,"name":"Float16"},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]}]}},"fp16_reference":{"export":{"seconds":0.1698446660011541,"main_mlirb_bytes":12587721},"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.05422870800248347,"stderr_tail":"","bytes":16800478,"resources_bin":16777256,"storage_types":[{"count":6291458,"name":"Float16"},{"count":4,"name":"UInt32"},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.05149270800757222,"stderr_tail":"","bytes":16800474,"resources_bin":16777256,"storage_types":[{"name":"Float16","count":6291458},{"name":"UInt32","count":4},{"name":"Int32","count":3}]},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.03814849999616854,"stderr_tail":"","bytes":12590581,"resources_bin":12582936,"storage_types":[{"name":"Float16","count":6291458},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}]}]},"toy_bytes":{"embedding_fp16":8388608,"linears_fp16":4194304,"linears_int8_codes":2097152},"finished":"2026-10-03T21:12:09+09:00"},"static_toy":{"schema":"kev-r6-static-toy/1","variants":{"static16":{"main_mlirb":10623080,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.608065375010483,"stderr_tail":"","bytes":12600629,"resources_bin":12582936,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2129920},{"name":"UInt32","count":12},{"count":4,"name":"Int32"}],"files":["main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_0.mpsgraph","main-h16c.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/static16/macOS_efr/toy_static16.h16c.aimodelc"},{"platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":0.144867166003678,"stderr_tail":"","bytes":12600752,"resources_bin":12582936,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2129920},{"count":12,"name":"UInt32"},{"name":"Int32","count":4}],"files":["main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_0.mpsgraph","main-h16c.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/static16/macOS_noefr/toy_static16.h16c.aimodelc"},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.1417770000116434,"stderr_tail":"","bytes":12600625,"resources_bin":12582936,"storage_types":[{"name":"Float16","count":4292609},{"count":2129920,"name":"Int8"},{"name":"UInt32","count":12},{"count":4,"name":"Int32"}],"files":["main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_0.mpsgraph","main-h19p.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/static16/iOS_efr/toy_static16.h19p.aimodelc"},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.14717541699064896,"stderr_tail":"","bytes":12600748,"resources_bin":12582936,"storage_types":[{"count":4292609,"name":"Float16"},{"count":2129920,"name":"Int8"},{"name":"UInt32","count":12},{"count":4,"name":"Int32"}],"files":["main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_0.mpsgraph","main-h19p.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/static16/iOS_noefr/toy_static16.h19p.aimodelc"}],"mac_run_noefr":{"load_s":0.08742383299977519,"call_ms":[92.26433301228099,1.1290840047877282,0.823833019239828],"out_shape":[1,16,1024],"finite":true,"scratch_dirs":[{"dir":"$TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_33_56-2910397884","kb":0}],"ane_compile_failed_lines":0},"mac_run_efr":{"load_s":0.08946170800481923,"call_ms":[13.705249992199242,1.3254579971544445,1.2034169922117144],"out_shape":[1,16,1024],"finite":true,"scratch_dirs":[{"dir":"$TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_33_56-3084907826","kb":0}],"ane_compile_failed_lines":0}},"dynamic":{"main_mlirb":10623440,"compile":[{"platform":"macOS","arch":"h16c","efr":true,"returncode":0,"seconds":0.20316450000973418,"stderr_tail":"","bytes":14837416,"resources_bin":14811176,"storage_types":[{"name":"Float16","count":4292609},{"count":2129920,"name":"Int8"},{"count":4,"name":"UInt32"},{"count":3,"name":"Int32"}],"files":["main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph","main-h16c.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/dynamic/macOS_efr/toy_dynamic.h16c.aimodelc"},{"platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":0.05147591698914766,"stderr_tail":"","bytes":10626983,"resources_bin":10616856,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2129920},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}],"files":["main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/reflection.fb","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h16c.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/dynamic/macOS_noefr/toy_dynamic.h16c.aimodelc"},{"platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":0.14581533300224692,"stderr_tail":"","bytes":14837412,"resources_bin":14811176,"storage_types":[{"count":4292609,"name":"Float16"},{"count":2129920,"name":"Int8"},{"name":"UInt32","count":4},{"count":3,"name":"Int32"}],"files":["main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph","main-h19p.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/dynamic/iOS_efr/toy_dynamic.h19p.aimodelc"},{"platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":0.05264825001358986,"stderr_tail":"","bytes":10626983,"resources_bin":10616856,"storage_types":[{"name":"Float16","count":4292609},{"name":"Int8","count":2129920},{"name":"UInt32","count":4},{"name":"Int32","count":3}],"files":["main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/reflection.fb","main-h19p-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin","main-h19p.mlirb","main.hash","metadata.json","stats.json"],"asset":"$SCRATCH/r6_static_toy/dynamic/iOS_noefr/toy_dynamic.h19p.aimodelc"}],"mac_run_noefr":{"load_s":0.08511970797553658,"call_ms":[88.80624998710118,1.3109169958624989,1.0678329854272306],"out_shape":[1,16,1024],"finite":true,"scratch_dirs":[{"dir":"$TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_33_57-3523771932","kb":2188}],"ane_compile_failed_lines":0},"mac_run_efr":{"load_s":0.0882500410079956,"call_ms":[21.712666988605633,1.4732499839738011,1.2747080181725323],"out_shape":[1,16,1024],"finite":true,"scratch_dirs":[{"dir":"$TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_33_57-1640287828","kb":0}],"ane_compile_failed_lines":0}}}},"bisect_rule":{"schema":"kev-int8-bisect-rule-4b/1","model":"kev-4b","rule":{"rows":"the top 20 rows of the 4B int8lin gate (results/readout_int8lin_pf16_4b.json) by max|dp| (ties: oracle order), then 20 rows drawn from the other rows by source (proportional, largest remainder, at least one per source; numpy default_rng(20261003).choice without replacement inside each source, sources in sorted order) - round 3's procedure with 20 + 20","instrument":"fp32 torch on the CPU (parity_decoder_torch.Runner --model kev-4b, 1 thread, bmm depthwise conv), S = 16 chunks from fresh zero states, the 4B oracle's row ids, the 4B fp32 pointer head; per config every body linear and the embedding table carry either the merged checkpoint's fp32 weight or the weight the exporter's quantize_pytorch_model call computes for that variant (the finalized module's dequantization, fp16, cast to fp32)","reproduction_check":"before any selection: all_int8_b32's worst max|dp| over the 40 rows >= 0.5 x the int8lin gate transcript's worst over the same rows (0.0601); otherwise stop and report","instrument_floor":"exact reproduces the round-4 P1 run (parity_4b/p1_shard*of2.jsonl) of every row bit for bit (probabilities), and every process's swap-back re-run likewise","target":"worst max|dp| over the 40 rows <= 0.006 AND the mean over the rows of the row's mean |dp| <= 0.0012 (every option of every question, near-ties included), this instrument","ranking":"configs ranked by the mean (ascending; ties: the worst, then the name); the worst is recorded","size":"a config's size = the params of the body linears it keeps out of int8 (fp16); the cap = 25 % of the 248 body linears' params; the embedding table's precision does not count against the cap; the estimated main.mlirb = the round-4 fp16 main.mlirb - 2 B x body params + per-linear bytes (fp16 2 B, b32 1 + 2/32, b16 1 + 2/16, aff32 1 + 2/32 + 1/32) - (2 - (1 + 2/32)) B x table params when the table is int8","search":["(fixed) exact, all_int8_b32, all_int8_b16, all_int8_affine_b32, embed_int8_only, proj_<kind>_fp16 x 12 (that kind fp16, every other linear int8 b32), layers_<a>-<b>_fp16 x 8 (that four-layer group fp16, the rest b32)","(i) the variants alone: all_int8_b32 / b16 / affine_b32 (0 % fp16)","(ii) kind sets K_j (int8 part b32): the 12 kinds in ranking order, each added while the set stays within the cap (a kind that would cross it is skipped and the walk goes on); K_j = the first j added; K_2, K_3, ... evaluated in order, at most 4","(iii) group sets G_k (int8 part b32): the 8 groups in ranking order; G_k = the top k, k = 2, 3, ... while within the cap","(iv) only if the best variant by mean is not b32: that variant as the int8 part of every K_j / G_k (j, k >= 1) evaluated in (ii)-(iii), ascending size, at most 4","(v) unions K_j + G_k (j, k >= 1, int8 part = the best variant) within the cap, ascending size (ties: smaller k, then smaller j), at most 8","a stage is entered only while no evaluated config within the cap meets the target","(reference, never chosen) at most 2 sets beyond the cap: G_3 and G_4 by ranking (int8 part b32)"],"choose":"among the evaluated configs within the cap that meet the target, the smallest size (ties: the smaller estimated main.mlirb, then the lower mean); if none, the best two within the cap by the ranking (exact and embed_int8_only excluded: they keep the whole body fp32)","embed":"then the chosen set(s) with the table int8 (+E), one config each; +E is adopted when its worst exceeds the set's own by at most 0.001","then":"the chosen set(s) (and their +E when adopted) are exported (export_decoder.py int8mix / int8lin with --fp16-layers / --fp16-kinds / --quant-block / --quant-scheme / --embed-int8, --model kev-4b) and gated on all 434 rows with the gate's own bar, the 130 held-out rows (never used to choose) and the v2 red arms"},"target":{"worst_max_abs_dp":0.006,"mean_of_row_mean_abs_dp":0.0012},"cap_fraction":0.25,"cap_params":892272640.0,"limits":{"ii":4,"iv":4,"v":8,"beyond_cap":2},"embed_slack":0.001,"rows":[["semif_1105577da4c8dad4609d",0],["semif_d880e65fef5e2993fdd0",0],["semif_7ed75d854d99b3a81916",0],["semif_ef3e0f8e657bf59f8a29",0],["tv4_012",0],["tv4_007",0],["semif_8e8c3804a3c15ebb31e3",0],["semif_f756e1cb04e11766341e",0],["tv4x_qnli_19",0],["semif_33d6e5da58f0fee2490d",0],["tv4_045",0],["semif_32f36cd0fbbfb3604586",0],["tv4s_01",0],["semif_cdf7f428857e1b841cb3",0],["tv4_058",0],["own_t06",0],["semif_ea7ec66a5e8eac886548",0],["own_m01",2],["semif_a22dab8ba073ff6cdb8d",0],["semif_42384af9ae56beb37ff5",0],["own_j02",1],["own_j04",1],["own_L02",3],["own_t04",2],["semif_e0c140e9222d1bce5647",0],["semif_3226202ba1ada2570e9f",0],["semif_a0128e551d095f3449f8",0],["semif_6db17cc22bd656d78558",0],["semif_a9230edfc6ec9e524af0",0],["semif_4089c199b8b796ec3ed8",0],["tv4_008",0],["tv4_028",0],["tv4s_05",0],["tv4s_14",0],["tv4x_emotion_13",0],["tv4x_qnli_00",0],["tv4x_paws_12",0],["tv4x_sciq_03",0],["tv4x_composition_holdout_01",0],["tv4x_composition_holdout_19",0]],"rows_detail":[{"id":"semif_1105577da4c8dad4609d","q":0,"set":"top","source":"semif_authored144","row_len":88,"near_tie":false,"gate_int8lin_max_abs_dp":0.060132622718811035},{"id":"semif_d880e65fef5e2993fdd0","q":0,"set":"top","source":"semif_authored144","row_len":85,"near_tie":false,"gate_int8lin_max_abs_dp":0.029746264219284058},{"id":"semif_7ed75d854d99b3a81916","q":0,"set":"top","source":"semif_authored144","row_len":68,"near_tie":false,"gate_int8lin_max_abs_dp":0.025355279445648193},{"id":"semif_ef3e0f8e657bf59f8a29","q":0,"set":"top","source":"semif_authored144","row_len":83,"near_tie":false,"gate_int8lin_max_abs_dp":0.022765278816223145},{"id":"tv4_012","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":83,"near_tie":false,"gate_int8lin_max_abs_dp":0.02085888385772705},{"id":"tv4_007","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":80,"near_tie":false,"gate_int8lin_max_abs_dp":0.019733965396881104},{"id":"semif_8e8c3804a3c15ebb31e3","q":0,"set":"top","source":"semif_authored144","row_len":69,"near_tie":false,"gate_int8lin_max_abs_dp":0.01799994707107544},{"id":"semif_f756e1cb04e11766341e","q":0,"set":"top","source":"semif_authored144","row_len":89,"near_tie":false,"gate_int8lin_max_abs_dp":0.01778966188430786},{"id":"tv4x_qnli_19","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":85,"near_tie":false,"gate_int8lin_max_abs_dp":0.016363978385925293},{"id":"semif_33d6e5da58f0fee2490d","q":0,"set":"top","source":"semif_authored144","row_len":73,"near_tie":false,"gate_int8lin_max_abs_dp":0.014631807804107666},{"id":"tv4_045","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":90,"near_tie":false,"gate_int8lin_max_abs_dp":0.01393580436706543},{"id":"semif_32f36cd0fbbfb3604586","q":0,"set":"top","source":"semif_authored144","row_len":66,"near_tie":false,"gate_int8lin_max_abs_dp":0.013280987739562988},{"id":"tv4s_01","q":0,"set":"top","source":"transfer_v4_dev_score","row_len":98,"near_tie":false,"gate_int8lin_max_abs_dp":0.012888491153717041},{"id":"semif_cdf7f428857e1b841cb3","q":0,"set":"top","source":"semif_authored144","row_len":77,"near_tie":false,"gate_int8lin_max_abs_dp":0.012779414653778076},{"id":"tv4_058","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":129,"near_tie":false,"gate_int8lin_max_abs_dp":0.012551069259643555},{"id":"own_t06","q":0,"set":"top","source":"own_text","row_len":301,"near_tie":false,"gate_int8lin_max_abs_dp":0.012297570705413818},{"id":"semif_ea7ec66a5e8eac886548","q":0,"set":"top","source":"semif_authored144","row_len":75,"near_tie":false,"gate_int8lin_max_abs_dp":0.012225091457366943},{"id":"own_m01","q":2,"set":"top","source":"own_text","row_len":159,"near_tie":false,"gate_int8lin_max_abs_dp":0.012172996997833252},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"set":"top","source":"semif_authored144","row_len":84,"near_tie":false,"gate_int8lin_max_abs_dp":0.012049198150634766},{"id":"semif_42384af9ae56beb37ff5","q":0,"set":"top","source":"semif_authored144","row_len":84,"near_tie":false,"gate_int8lin_max_abs_dp":0.011975705623626709},{"id":"own_j02","q":1,"set":"random","source":"own_json","row_len":232,"near_tie":false,"gate_int8lin_max_abs_dp":0.005702078342437744},{"id":"own_j04","q":1,"set":"random","source":"own_json","row_len":171,"near_tie":false,"gate_int8lin_max_abs_dp":0.00011241436004638672},{"id":"own_L02","q":3,"set":"random","source":"own_long","row_len":1499,"near_tie":false,"gate_int8lin_max_abs_dp":8.698366582393646e-05},{"id":"own_t04","q":2,"set":"random","source":"own_text","row_len":191,"near_tie":false,"gate_int8lin_max_abs_dp":0.0017088651657104492},{"id":"semif_e0c140e9222d1bce5647","q":0,"set":"random","source":"semif_authored144","row_len":72,"near_tie":false,"gate_int8lin_max_abs_dp":6.195297464728355e-05},{"id":"semif_3226202ba1ada2570e9f","q":0,"set":"random","source":"semif_authored144","row_len":76,"near_tie":false,"gate_int8lin_max_abs_dp":0.0034356117248535156},{"id":"semif_a0128e551d095f3449f8","q":0,"set":"random","source":"semif_authored144","row_len":66,"near_tie":false,"gate_int8lin_max_abs_dp":0.0005913972854614258},{"id":"semif_6db17cc22bd656d78558","q":0,"set":"random","source":"semif_authored144","row_len":82,"near_tie":false,"gate_int8lin_max_abs_dp":0.0032169222831726074},{"id":"semif_a9230edfc6ec9e524af0","q":0,"set":"random","source":"semif_authored144","row_len":68,"near_tie":false,"gate_int8lin_max_abs_dp":9.232759475708008e-05},{"id":"semif_4089c199b8b796ec3ed8","q":0,"set":"random","source":"semif_authored144","row_len":88,"near_tie":true,"gate_int8lin_max_abs_dp":0.0024436116218566895},{"id":"tv4_008","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":96,"near_tie":false,"gate_int8lin_max_abs_dp":0.00801151990890503},{"id":"tv4_028","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":290,"near_tie":false,"gate_int8lin_max_abs_dp":0.010508567094802856},{"id":"tv4s_05","q":0,"set":"random","source":"transfer_v4_dev_score","row_len":98,"near_tie":false,"gate_int8lin_max_abs_dp":0.00735783576965332},{"id":"tv4s_14","q":0,"set":"random","source":"transfer_v4_dev_score","row_len":98,"near_tie":false,"gate_int8lin_max_abs_dp":0.0067758262157440186},{"id":"tv4x_emotion_13","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":60,"near_tie":false,"gate_int8lin_max_abs_dp":0.006184518337249756},{"id":"tv4x_qnli_00","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":94,"near_tie":false,"gate_int8lin_max_abs_dp":7.757917046546936e-06},{"id":"tv4x_paws_12","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":93,"near_tie":false,"gate_int8lin_max_abs_dp":0.005915597081184387},{"id":"tv4x_sciq_03","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":97,"near_tie":false,"gate_int8lin_max_abs_dp":0.0006895288825035095},{"id":"tv4x_composition_holdout_01","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":145,"near_tie":false,"gate_int8lin_max_abs_dp":0.0003303922712802887},{"id":"tv4x_composition_holdout_19","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":149,"near_tie":false,"gate_int8lin_max_abs_dp":0.0005483627319335938}],"random_quota":{"transfer_v4_dev_head":2,"transfer_v4_dev_sources":6,"transfer_v4_dev_score":2,"semif_authored144":6,"own_text":1,"own_json":2,"own_long":1},"seed":20261003,"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16_4b.json","sha256":"d430a661d8dbeec4f0a81466b3049ebe4ab5594f1f5d1bb7e781f9fc0c92dbe5","result":"FAIL","max_abs_dp":0.060132622718811035,"bundle":"kev_4b_decode_int8lin_pf16","aimodelc_tree_sha256":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403"},"configs_fixed":["exact","all_int8_b32","all_int8_b16","all_int8_affine_b32","embed_int8_only","proj_qkv_fp16","proj_z_fp16","proj_a_fp16","proj_b_fp16","proj_out_fp16","proj_q_fp16","proj_k_fp16","proj_v_fp16","proj_o_fp16","proj_gate_fp16","proj_up_fp16","proj_down_fp16","layers_0-3_fp16","layers_4-7_fp16","layers_8-11_fp16","layers_12-15_fp16","layers_16-19_fp16","layers_20-23_fp16","layers_24-27_fp16","layers_28-31_fp16"],"configs_fixed_canonical":{"exact":"set_fp32","all_int8_b32":"set_b32","all_int8_b16":"set_b16","all_int8_affine_b32":"set_aff32","embed_int8_only":"set_fp32__E","proj_qkv_fp16":"set_b32__K_qkv","proj_z_fp16":"set_b32__K_z","proj_a_fp16":"set_b32__K_a","proj_b_fp16":"set_b32__K_b","proj_out_fp16":"set_b32__K_out","proj_q_fp16":"set_b32__K_q","proj_k_fp16":"set_b32__K_k","proj_v_fp16":"set_b32__K_v","proj_o_fp16":"set_b32__K_o","proj_gate_fp16":"set_b32__K_gate","proj_up_fp16":"set_b32__K_up","proj_down_fp16":"set_b32__K_down","layers_0-3_fp16":"set_b32__G_0","layers_4-7_fp16":"set_b32__G_1","layers_8-11_fp16":"set_b32__G_2","layers_12-15_fp16":"set_b32__G_3","layers_16-19_fp16":"set_b32__G_4","layers_20-23_fp16":"set_b32__G_5","layers_24-27_fp16":"set_b32__G_6","layers_28-31_fp16":"set_b32__G_7"},"variants":{"b32":{"block":32,"qscheme":"symmetric_with_clipping","bytes_per_weight":1.0625},"b16":{"block":16,"qscheme":"symmetric_with_clipping","bytes_per_weight":1.125},"aff32":{"block":32,"qscheme":"asymmetric","bytes_per_weight":1.09375}},"embedding_variant":{"embed32":"per-block-32 symmetric_with_clipping along axis 1 (hidden)","bytes_per_weight":1.0625},"body_linear_params":3569090560,"table_params":635699200,"params_by_kind":{"qkv":503316480,"z":251658240,"a":1966080,"b":1966080,"out":251658240,"q":167772160,"k":20971520,"v":20971520,"o":83886080,"gate":754974720,"up":754974720,"down":754974720},"params_by_group":{"0":446136320,"1":446136320,"2":446136320,"3":446136320,"4":446136320,"5":446136320,"6":446136320,"7":446136320},"sizes_fixed":{"exact":{"fp16_params":3569090560,"fp16_fraction":1.0,"est_main_mlirb_bytes":8414007636},"all_int8_b32":{"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5067985236},"all_int8_b16":{"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5291053396},"all_int8_affine_b32":{"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5179519316},"embed_int8_only":{"fp16_params":3569090560,"fp16_fraction":1.0,"est_main_mlirb_bytes":7818039636},"proj_qkv_fp16":{"fp16_params":503316480,"fp16_fraction":0.14102093279471173,"est_main_mlirb_bytes":5539844436},"proj_z_fp16":{"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"est_main_mlirb_bytes":5303914836},"proj_a_fp16":{"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"est_main_mlirb_bytes":5069828436},"proj_b_fp16":{"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"est_main_mlirb_bytes":5069828436},"proj_out_fp16":{"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"est_main_mlirb_bytes":5303914836},"proj_q_fp16":{"fp16_params":167772160,"fp16_fraction":0.047006977598237236,"est_main_mlirb_bytes":5225271636},"proj_k_fp16":{"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"est_main_mlirb_bytes":5087646036},"proj_v_fp16":{"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"est_main_mlirb_bytes":5087646036},"proj_o_fp16":{"fp16_params":83886080,"fp16_fraction":0.023503488799118618,"est_main_mlirb_bytes":5146628436},"proj_gate_fp16":{"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036},"proj_up_fp16":{"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036},"proj_down_fp16":{"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036},"layers_0-3_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_4-7_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_8-11_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_12-15_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_16-19_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_20-23_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_24-27_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036},"layers_28-31_fp16":{"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036}},"chunk":16,"tokens":5731,"estimate":"5,731 tokens per config; at round 4's 38 ms per token (1 thread, bmm conv, alone) about 218 s per config","script_sha256":"e5863fcb74b131ff8c907b5ada408be3107d58835aa4a2414aa157065dfe1674","written_at":"2026-10-03T21:22:08+09:00"},"bisect":{"schema":"kev-int8-bisect-4b/1","model":"kev-4b","rule":{"rows":"the top 20 rows of the 4B int8lin gate (results/readout_int8lin_pf16_4b.json) by max|dp| (ties: oracle order), then 20 rows drawn from the other rows by source (proportional, largest remainder, at least one per source; numpy default_rng(20261003).choice without replacement inside each source, sources in sorted order) - round 3's procedure with 20 + 20","instrument":"fp32 torch on the CPU (parity_decoder_torch.Runner --model kev-4b, 1 thread, bmm depthwise conv), S = 16 chunks from fresh zero states, the 4B oracle's row ids, the 4B fp32 pointer head; per config every body linear and the embedding table carry either the merged checkpoint's fp32 weight or the weight the exporter's quantize_pytorch_model call computes for that variant (the finalized module's dequantization, fp16, cast to fp32)","reproduction_check":"before any selection: all_int8_b32's worst max|dp| over the 40 rows >= 0.5 x the int8lin gate transcript's worst over the same rows (0.0601); otherwise stop and report","instrument_floor":"exact reproduces the round-4 P1 run (parity_4b/p1_shard*of2.jsonl) of every row bit for bit (probabilities), and every process's swap-back re-run likewise","target":"worst max|dp| over the 40 rows <= 0.006 AND the mean over the rows of the row's mean |dp| <= 0.0012 (every option of every question, near-ties included), this instrument","ranking":"configs ranked by the mean (ascending; ties: the worst, then the name); the worst is recorded","size":"a config's size = the params of the body linears it keeps out of int8 (fp16); the cap = 25 % of the 248 body linears' params; the embedding table's precision does not count against the cap; the estimated main.mlirb = the round-4 fp16 main.mlirb - 2 B x body params + per-linear bytes (fp16 2 B, b32 1 + 2/32, b16 1 + 2/16, aff32 1 + 2/32 + 1/32) - (2 - (1 + 2/32)) B x table params when the table is int8","search":["(fixed) exact, all_int8_b32, all_int8_b16, all_int8_affine_b32, embed_int8_only, proj_<kind>_fp16 x 12 (that kind fp16, every other linear int8 b32), layers_<a>-<b>_fp16 x 8 (that four-layer group fp16, the rest b32)","(i) the variants alone: all_int8_b32 / b16 / affine_b32 (0 % fp16)","(ii) kind sets K_j (int8 part b32): the 12 kinds in ranking order, each added while the set stays within the cap (a kind that would cross it is skipped and the walk goes on); K_j = the first j added; K_2, K_3, ... evaluated in order, at most 4","(iii) group sets G_k (int8 part b32): the 8 groups in ranking order; G_k = the top k, k = 2, 3, ... while within the cap","(iv) only if the best variant by mean is not b32: that variant as the int8 part of every K_j / G_k (j, k >= 1) evaluated in (ii)-(iii), ascending size, at most 4","(v) unions K_j + G_k (j, k >= 1, int8 part = the best variant) within the cap, ascending size (ties: smaller k, then smaller j), at most 8","a stage is entered only while no evaluated config within the cap meets the target","(reference, never chosen) at most 2 sets beyond the cap: G_3 and G_4 by ranking (int8 part b32)"],"choose":"among the evaluated configs within the cap that meet the target, the smallest size (ties: the smaller estimated main.mlirb, then the lower mean); if none, the best two within the cap by the ranking (exact and embed_int8_only excluded: they keep the whole body fp32)","embed":"then the chosen set(s) with the table int8 (+E), one config each; +E is adopted when its worst exceeds the set's own by at most 0.001","then":"the chosen set(s) (and their +E when adopted) are exported (export_decoder.py int8mix / int8lin with --fp16-layers / --fp16-kinds / --quant-block / --quant-scheme / --embed-int8, --model kev-4b) and gated on all 434 rows with the gate's own bar, the 130 held-out rows (never used to choose) and the v2 red arms"},"rule_file":"$ZOO_WORK_ROOT/_kev/bisect_4b/rule.json","rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","rule_written_at":"2026-10-03T21:22:08+09:00","rows":[["semif_1105577da4c8dad4609d",0],["semif_d880e65fef5e2993fdd0",0],["semif_7ed75d854d99b3a81916",0],["semif_ef3e0f8e657bf59f8a29",0],["tv4_012",0],["tv4_007",0],["semif_8e8c3804a3c15ebb31e3",0],["semif_f756e1cb04e11766341e",0],["tv4x_qnli_19",0],["semif_33d6e5da58f0fee2490d",0],["tv4_045",0],["semif_32f36cd0fbbfb3604586",0],["tv4s_01",0],["semif_cdf7f428857e1b841cb3",0],["tv4_058",0],["own_t06",0],["semif_ea7ec66a5e8eac886548",0],["own_m01",2],["semif_a22dab8ba073ff6cdb8d",0],["semif_42384af9ae56beb37ff5",0],["own_j02",1],["own_j04",1],["own_L02",3],["own_t04",2],["semif_e0c140e9222d1bce5647",0],["semif_3226202ba1ada2570e9f",0],["semif_a0128e551d095f3449f8",0],["semif_6db17cc22bd656d78558",0],["semif_a9230edfc6ec9e524af0",0],["semif_4089c199b8b796ec3ed8",0],["tv4_008",0],["tv4_028",0],["tv4s_05",0],["tv4s_14",0],["tv4x_emotion_13",0],["tv4x_qnli_00",0],["tv4x_paws_12",0],["tv4x_sciq_03",0],["tv4x_composition_holdout_01",0],["tv4x_composition_holdout_19",0]],"target":{"worst_max_abs_dp":0.006,"mean_of_row_mean_abs_dp":0.0012},"cap_params":892272640.0,"body_linear_params":3569090560,"script":{"path":"conversion/kev/int8_bisect_torch.py","sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc"},"dumps":{},"reproduction_check":{"gate_worst":0.060132622718811035,"torch_all_int8_b32_worst":0.07101064920425415,"ratio":1.1809005826389838,"reproduced":true,"pearson_per_row":0.973113998763023,"rows_above_bar_gate":5,"rows_above_bar_torch":5,"per_row":[{"row":"semif_1105577da4c8dad4609d:q0","gate":0.060132622718811035,"torch":0.07101064920425415},{"row":"semif_d880e65fef5e2993fdd0:q0","gate":0.029746264219284058,"torch":0.026368260383605957},{"row":"semif_7ed75d854d99b3a81916:q0","gate":0.025355279445648193,"torch":0.02151423692703247},{"row":"semif_ef3e0f8e657bf59f8a29:q0","gate":0.022765278816223145,"torch":0.01827031373977661},{"row":"tv4_012:q0","gate":0.02085888385772705,"torch":0.02032250165939331},{"row":"tv4_007:q0","gate":0.019733965396881104,"torch":0.025035321712493896},{"row":"semif_8e8c3804a3c15ebb31e3:q0","gate":0.01799994707107544,"torch":0.017290115356445312},{"row":"semif_f756e1cb04e11766341e:q0","gate":0.01778966188430786,"torch":0.0195961594581604},{"row":"tv4x_qnli_19:q0","gate":0.016363978385925293,"torch":0.015782535076141357},{"row":"semif_33d6e5da58f0fee2490d:q0","gate":0.014631807804107666,"torch":0.01576271653175354},{"row":"tv4_045:q0","gate":0.01393580436706543,"torch":0.01375359296798706},{"row":"semif_32f36cd0fbbfb3604586:q0","gate":0.013280987739562988,"torch":0.010137081146240234},{"row":"tv4s_01:q0","gate":0.012888491153717041,"torch":0.01088804006576538},{"row":"semif_cdf7f428857e1b841cb3:q0","gate":0.012779414653778076,"torch":0.007764458656311035},{"row":"tv4_058:q0","gate":0.012551069259643555,"torch":0.009904086589813232},{"row":"own_t06:q0","gate":0.012297570705413818,"torch":0.011173248291015625},{"row":"semif_ea7ec66a5e8eac886548:q0","gate":0.012225091457366943,"torch":0.010933995246887207},{"row":"own_m01:q2","gate":0.012172996997833252,"torch":0.011166274547576904},{"row":"semif_a22dab8ba073ff6cdb8d:q0","gate":0.012049198150634766,"torch":0.019027352333068848},{"row":"semif_42384af9ae56beb37ff5:q0","gate":0.011975705623626709,"torch":0.00863867998123169},{"row":"own_j02:q1","gate":0.005702078342437744,"torch":0.007485777139663696},{"row":"own_j04:q1","gate":0.00011241436004638672,"torch":4.655122756958008e-05},{"row":"own_L02:q3","gate":8.698366582393646e-05,"torch":0.00017404556274414062},{"row":"own_t04:q2","gate":0.0017088651657104492,"torch":0.0007988251745700836},{"row":"semif_e0c140e9222d1bce5647:q0","gate":6.195297464728355e-05,"torch":8.291006088256836e-05},{"row":"semif_3226202ba1ada2570e9f:q0","gate":0.0034356117248535156,"torch":0.0023995041847229004},{"row":"semif_a0128e551d095f3449f8:q0","gate":0.0005913972854614258,"torch":0.00019553117454051971},{"row":"semif_6db17cc22bd656d78558:q0","gate":0.0032169222831726074,"torch":0.0013602972030639648},{"row":"semif_a9230edfc6ec9e524af0:q0","gate":9.232759475708008e-05,"torch":1.9947532564401627e-05},{"row":"semif_4089c199b8b796ec3ed8:q0","gate":0.0024436116218566895,"torch":0.005082130432128906},{"row":"tv4_008:q0","gate":0.00801151990890503,"torch":0.00774693489074707},{"row":"tv4_028:q0","gate":0.010508567094802856,"torch":0.01022648811340332},{"row":"tv4s_05:q0","gate":0.00735783576965332,"torch":0.005208015441894531},{"row":"tv4s_14:q0","gate":0.0067758262157440186,"torch":0.008822783827781677},{"row":"tv4x_emotion_13:q0","gate":0.006184518337249756,"torch":0.008360326290130615},{"row":"tv4x_qnli_00:q0","gate":7.757917046546936e-06,"torch":0.00010109599679708481},{"row":"tv4x_paws_12:q0","gate":0.005915597081184387,"torch":0.0036816000938415527},{"row":"tv4x_sciq_03:q0","gate":0.0006895288825035095,"torch":0.00041506439447402954},{"row":"tv4x_composition_holdout_01:q0","gate":0.0003303922712802887,"torch":0.00036722421646118164},{"row":"tv4x_composition_holdout_19:q0","gate":0.0005483627319335938,"torch":0.0024015307426452637}]},"instrument_floor":{"worst_max_abs_dp":4.0531158447265625e-06,"bit_equal_p1_rows":40,"rows":40},"processes":[{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_a.json","threads":1,"load_seconds":5.688849041995127,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T21:23:09+09:00","finished":"2026-10-03T22:37:37+09:00","conv":{"fast_calls":118704,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:37:37+09:00"},"configs":["exact","embed_int8_only","all_int8_b16","proj_qkv_fp16","proj_z_fp16","proj_a_fp16","proj_b_fp16","proj_out_fp16","proj_q_fp16","layers_0-3_fp16","layers_4-7_fp16","layers_8-11_fp16","layers_12-15_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_b.json","threads":1,"load_seconds":5.687672125000972,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T21:23:09+09:00","finished":"2026-10-03T22:32:19+09:00","conv":{"fast_calls":109584,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:32:19+09:00"},"configs":["all_int8_b32","proj_k_fp16","all_int8_affine_b32","proj_v_fp16","proj_o_fp16","proj_gate_fp16","proj_up_fp16","proj_down_fp16","layers_16-19_fp16","layers_20-23_fp16","layers_24-27_fp16","layers_28-31_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s1_ii_a.json","threads":1,"load_seconds":7.29205762501806,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:38:03+09:00","finished":"2026-10-03T22:45:54+09:00","conv":{"fast_calls":18384,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:45:54+09:00"},"configs":["set_b32__K_qkv_out","set_b32__K_qkv_b_out_v"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s1_ii_b.json","threads":1,"load_seconds":7.290043207991403,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:38:03+09:00","finished":"2026-10-03T22:45:54+09:00","conv":{"fast_calls":18384,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:45:54+09:00"},"configs":["set_b32__K_qkv_out_v","set_b32__K_qkv_a_b_out_v"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s2_iii_a.json","threads":1,"load_seconds":5.527781917015091,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:46:10+09:00","finished":"2026-10-03T22:49:16+09:00","conv":{"fast_calls":9264,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:49:16+09:00"},"configs":["set_b32__G_0_1"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s3_iv_a.json","threads":1,"load_seconds":2.8808161659981124,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:49:30+09:00","finished":"2026-10-03T22:57:04+09:00","conv":{"fast_calls":18384,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:57:04+09:00"},"configs":["set_aff32__K_out","set_aff32__K_qkv_out"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s3_iv_b.json","threads":1,"load_seconds":2.880017791991122,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:49:30+09:00","finished":"2026-10-03T22:57:04+09:00","conv":{"fast_calls":18384,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T22:57:04+09:00"},"configs":["set_aff32__G_0","set_aff32__K_qkv_out_v"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s4_v_a.json","threads":1,"load_seconds":7.758657541999128,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T22:57:21+09:00","finished":"2026-10-03T23:01:49+09:00","conv":{"fast_calls":9264,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T23:01:49+09:00"},"configs":["set_aff32__G_0__K_out"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s5_embed_a.json","threads":1,"load_seconds":4.1081967909995,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T23:02:07+09:00","finished":"2026-10-03T23:08:09+09:00","conv":{"fast_calls":9264,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T23:08:09+09:00"},"configs":["set_aff32__K_qkv_out_v__E"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect_4b/parts/part_s5_embed_b.json","threads":1,"load_seconds":4.071284291014308,"rule_sha256":"debf07091c69887be0e994467b966a1e863ccdf96814b0f8736c066e9260c1cd","script_sha256":"f524ad207ef966ebc3b6774872e4cdf48bd82978b5025a3452ed4e241a684fdc","dumps":{"b32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b32.json","sha256":"d2e014c456f3b249ada5fc4c2eac283c198287245010dded8534057cc681da5f"},"b16":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/b16.json","sha256":"5ac7acc6754cce8321a32c12c075299bb1996679bf86b54624825913bb44e305"},"aff32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/aff32.json","sha256":"babd789ff1c670f674af956aca22d32e6d1ffe58601363866a8c8960ead144b4"},"embed32":{"path":"$ZOO_WORK_ROOT/_kev/bisect_4b/dump/embed32.json","sha256":"796edca2895b0a98ce68484a97bba28317753afe4ebb474f1b2a3eef86bb7ac7"}},"probe_bit_equal_checkpoint":true,"started":"2026-10-03T23:02:07+09:00","finished":"2026-10-03T23:08:08+09:00","conv":{"fast_calls":9264,"checked_vs_F_conv1d":24,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":2.384185791015625e-06,"at":"2026-10-03T23:08:08+09:00"},"configs":["set_aff32__K_qkv_out__E"]}],"plan":{"cap_params":892272640.0,"next":["set_b32__G_0_1_2","set_b32__G_0_1_2_3"],"reproduction":{"gate_worst":0.060132622718811035,"torch_all_int8_b32_worst":0.07101064920425415,"ratio":1.1809005826389838,"reproduced":true},"kind_ranking":["out","qkv","up","down","gate","z","v","q","b","a","k","o"],"group_ranking":[0,1,2,3,4,5,6,7],"variant_ranking":["aff32","b16","b32"],"best_variant":"aff32","stages":{"ii":["set_b32__K_qkv_out","set_b32__K_qkv_out_v","set_b32__K_qkv_b_out_v","set_b32__K_qkv_a_b_out_v"],"iii":["set_b32__G_0_1"],"iv":["set_aff32__K_out","set_aff32__G_0","set_aff32__K_qkv_out","set_aff32__K_qkv_out_v"],"v":["set_aff32__G_0__K_out"]},"beyond_cap_reference":["set_b32__G_0_1_2","set_b32__G_0_1_2_3"],"meeting":[],"chosen":["set_aff32__K_qkv_out_v","set_aff32__K_qkv_out"],"chosen_meets_target":false,"embed_candidates":["set_aff32__K_qkv_out_v__E","set_aff32__K_qkv_out__E"],"embed_adopted":{"set_aff32__K_qkv_out_v":true,"set_aff32__K_qkv_out":true},"stage":"chosen"},"chosen":["set_aff32__K_qkv_out_v","set_aff32__K_qkv_out"],"table":[{"config":"exact","canonical":"set_fp32","variant":"fp32","groups":[],"kinds":[],"embed":false,"fp16_params":3569090560,"fp16_fraction":1.0,"within_cap":false,"est_main_mlirb_bytes":8414007636,"worst_max_abs_dp":4.0531158447265625e-06,"mean_of_row_mean_abs_dp":7.05648223326231e-07,"worst_top20":4.0531158447265625e-06,"worst_random20":1.9669532775878906e-06,"mean_random20":3.9389948748673006e-07,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":40,"meets_target":true,"seconds":235.8534802920185,"rank_by_mean":1},{"config":"embed_int8_only","canonical":"set_fp32__E","variant":"fp32","groups":[],"kinds":[],"embed":true,"fp16_params":3569090560,"fp16_fraction":1.0,"within_cap":false,"est_main_mlirb_bytes":7818039636,"worst_max_abs_dp":0.005160510540008545,"mean_of_row_mean_abs_dp":0.0005552305745853421,"worst_top20":0.005160510540008545,"worst_random20":0.0015743374824523926,"mean_random20":0.0002690992056159303,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":39,"meets_target":true,"seconds":209.13520808299654,"rank_by_mean":2},{"config":"set_aff32__K_qkv_out_v__E","canonical":"set_aff32__K_qkv_out_v__E","variant":"aff32","groups":[],"kinds":["qkv","out","v"],"embed":true,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5286752596,"worst_max_abs_dp":0.015426576137542725,"mean_of_row_mean_abs_dp":0.0018798819457879292,"worst_top20":0.015426576137542725,"worst_random20":0.005517542362213135,"mean_random20":0.001021396283370753,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":34,"meets_target":false,"seconds":352.3266149999981,"rank_by_mean":3},{"config":"set_aff32__K_qkv_out_v","canonical":"set_aff32__K_qkv_out_v","variant":"aff32","groups":[],"kinds":["qkv","out","v"],"embed":false,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5882720596,"worst_max_abs_dp":0.01644355058670044,"mean_of_row_mean_abs_dp":0.0019123301824826439,"worst_top20":0.01644355058670044,"worst_random20":0.006155461072921753,"mean_random20":0.0011755989020457492,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":34,"meets_target":false,"seconds":219.261534417019,"rank_by_mean":4},{"config":"set_aff32__K_qkv_out__E","canonical":"set_aff32__K_qkv_out__E","variant":"aff32","groups":[],"kinds":["qkv","out"],"embed":true,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5267747156,"worst_max_abs_dp":0.015333414077758789,"mean_of_row_mean_abs_dp":0.002010115250062275,"worst_top20":0.015333414077758789,"worst_random20":0.005572497844696045,"mean_random20":0.001181590159831103,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":33,"meets_target":false,"seconds":348.9667994589836,"rank_by_mean":5},{"config":"set_aff32__K_qkv_out","canonical":"set_aff32__K_qkv_out","variant":"aff32","groups":[],"kinds":["qkv","out"],"embed":false,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5863715156,"worst_max_abs_dp":0.016337275505065918,"mean_of_row_mean_abs_dp":0.0020814303058432414,"worst_top20":0.016337275505065918,"worst_random20":0.0062052905559539795,"mean_random20":0.0011926212639082224,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":33,"meets_target":false,"seconds":217.49563470800058,"rank_by_mean":6},{"config":"set_b32__G_0_1","canonical":"set_b32__G_0_1","variant":"b32","groups":[0,1],"kinds":[],"embed":false,"fp16_params":892272640,"fp16_fraction":0.25,"within_cap":true,"est_main_mlirb_bytes":5904490836,"worst_max_abs_dp":0.018460214138031006,"mean_of_row_mean_abs_dp":0.0024146552666691906,"worst_top20":0.018460214138031006,"worst_random20":0.0063607096672058105,"mean_random20":0.0011191794042436716,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":33,"meets_target":false,"seconds":182.4436264160031,"rank_by_mean":7},{"config":"set_aff32__G_0","canonical":"set_aff32__G_0","variant":"aff32","groups":[0],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5583830356,"worst_max_abs_dp":0.024686098098754883,"mean_of_row_mean_abs_dp":0.002719062058798348,"worst_top20":0.024686098098754883,"worst_random20":0.00907924771308899,"mean_random20":0.001276409449443842,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":33,"meets_target":false,"seconds":211.0626966250129,"rank_by_mean":8},{"config":"set_aff32__K_out","canonical":"set_aff32__K_out","variant":"aff32","groups":[],"kinds":["out"],"embed":false,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5407584596,"worst_max_abs_dp":0.035969555377960205,"mean_of_row_mean_abs_dp":0.002786534290741353,"worst_top20":0.035969555377960205,"worst_random20":0.007065504789352417,"mean_random20":0.0013025979404725756,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":33,"meets_target":false,"seconds":212.75119783400442,"rank_by_mean":9},{"config":"set_aff32__G_0__K_out","canonical":"set_aff32__G_0__K_out","variant":"aff32","groups":[0],"kinds":["out"],"embed":false,"fp16_params":666337280,"fp16_fraction":0.18669665809768637,"within_cap":true,"est_main_mlirb_bytes":5783387476,"worst_max_abs_dp":0.038170695304870605,"mean_of_row_mean_abs_dp":0.0028898118715248227,"worst_top20":0.038170695304870605,"worst_random20":0.008086919784545898,"mean_random20":0.0010962368663361606,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":34,"meets_target":false,"seconds":260.04487779099145,"rank_by_mean":10},{"config":"all_int8_affine_b32","canonical":"set_aff32","variant":"aff32","groups":[],"kinds":[],"embed":false,"fp16_params":0,"fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5179519316,"worst_max_abs_dp":0.032596468925476074,"mean_of_row_mean_abs_dp":0.0030056005895797476,"worst_top20":0.032596468925476074,"worst_random20":0.010189294815063477,"mean_random20":0.001579397337627597,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":197.64485462501762,"rank_by_mean":11},{"config":"set_b32__K_qkv_a_b_out_v","canonical":"set_b32__K_qkv_a_b_out_v","variant":"b32","groups":[],"kinds":["qkv","a","b","out","v"],"embed":false,"fp16_params":779878400,"fp16_fraction":0.2185089974293059,"within_cap":true,"est_main_mlirb_bytes":5799121236,"worst_max_abs_dp":0.017477869987487793,"mean_of_row_mean_abs_dp":0.0036948911649233194,"worst_top20":0.017477869987487793,"worst_random20":0.010927081108093262,"mean_random20":0.0016133496047890127,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":31,"meets_target":false,"seconds":209.79919770901324,"rank_by_mean":12},{"config":"set_b32__K_qkv_b_out_v","canonical":"set_b32__K_qkv_b_out_v","variant":"b32","groups":[],"kinds":["qkv","b","out","v"],"embed":false,"fp16_params":777912320,"fp16_fraction":0.21795813441057657,"within_cap":true,"est_main_mlirb_bytes":5797278036,"worst_max_abs_dp":0.019581615924835205,"mean_of_row_mean_abs_dp":0.0037199081680349385,"worst_top20":0.019581615924835205,"worst_random20":0.011025786399841309,"mean_random20":0.0016021091917840142,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":29,"meets_target":false,"seconds":209.8121754580061,"rank_by_mean":13},{"config":"set_b32__K_qkv_out_v","canonical":"set_b32__K_qkv_out_v","variant":"b32","groups":[],"kinds":["qkv","out","v"],"embed":false,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5795434836,"worst_max_abs_dp":0.02115654945373535,"mean_of_row_mean_abs_dp":0.0038814362924798236,"worst_top20":0.02115654945373535,"worst_random20":0.011215507984161377,"mean_random20":0.0016067490413358126,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":236.144449000014,"rank_by_mean":14},{"config":"set_b32__K_qkv_out","canonical":"set_b32__K_qkv_out","variant":"b32","groups":[],"kinds":["qkv","out"],"embed":false,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.018633544445037842,"mean_of_row_mean_abs_dp":0.003931028543884167,"worst_top20":0.018633544445037842,"worst_random20":0.010448873043060303,"mean_random20":0.0016753391939952657,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":32,"meets_target":false,"seconds":236.12805604099412,"rank_by_mean":15},{"config":"layers_0-3_fp16","canonical":"set_b32__G_0","variant":"b32","groups":[0],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.02719932794570923,"mean_of_row_mean_abs_dp":0.0042045773240776425,"worst_top20":0.02719932794570923,"worst_random20":0.008894920349121094,"mean_random20":0.0017918850828815874,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":560.1727806669951,"rank_by_mean":16},{"config":"all_int8_b16","canonical":"set_b16","variant":"b16","groups":[],"kinds":[],"embed":false,"fp16_params":0,"fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5291053396,"worst_max_abs_dp":0.04704171419143677,"mean_of_row_mean_abs_dp":0.004532722396979807,"worst_top20":0.04704171419143677,"worst_random20":0.008319184184074402,"mean_random20":0.001494376854680013,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":199.70211466599721,"rank_by_mean":17},{"config":"proj_out_fp16","canonical":"set_b32__K_out","variant":"b32","groups":[],"kinds":["out"],"embed":false,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5303914836,"worst_max_abs_dp":0.024914264678955078,"mean_of_row_mean_abs_dp":0.004831143981573405,"worst_top20":0.024914264678955078,"worst_random20":0.011762857437133789,"mean_random20":0.0019981799234907766,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":490.278810000018,"rank_by_mean":18},{"config":"layers_4-7_fp16","canonical":"set_b32__G_1","variant":"b32","groups":[1],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.04578268527984619,"mean_of_row_mean_abs_dp":0.004949301713398501,"worst_top20":0.04578268527984619,"worst_random20":0.011314094066619873,"mean_random20":0.0015047356825865182,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":32,"meets_target":false,"seconds":474.50616370901116,"rank_by_mean":19},{"config":"proj_qkv_fp16","canonical":"set_b32__K_qkv","variant":"b32","groups":[],"kinds":["qkv"],"embed":false,"fp16_params":503316480,"fp16_fraction":0.14102093279471173,"within_cap":true,"est_main_mlirb_bytes":5539844436,"worst_max_abs_dp":0.058991432189941406,"mean_of_row_mean_abs_dp":0.005999668049286508,"worst_top20":0.058991432189941406,"worst_random20":0.009840577840805054,"mean_random20":0.0019351860571381013,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":30,"meets_target":false,"seconds":207.4134852499992,"rank_by_mean":20},{"config":"proj_up_fp16","canonical":"set_b32__K_up","variant":"b32","groups":[],"kinds":["up"],"embed":false,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.07001703977584839,"mean_of_row_mean_abs_dp":0.006040579349113008,"worst_top20":0.07001703977584839,"worst_random20":0.009782075881958008,"mean_random20":0.0018814160663168876,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":31,"meets_target":false,"seconds":373.84406879200833,"rank_by_mean":21},{"config":"proj_down_fp16","canonical":"set_b32__K_down","variant":"b32","groups":[],"kinds":["down"],"embed":false,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.07015883922576904,"mean_of_row_mean_abs_dp":0.006056326911154125,"worst_top20":0.07015883922576904,"worst_random20":0.008356630802154541,"mean_random20":0.002126792893492772,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":510.9828269589925,"rank_by_mean":22},{"config":"layers_8-11_fp16","canonical":"set_b32__G_2","variant":"b32","groups":[2],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.06477582454681396,"mean_of_row_mean_abs_dp":0.006092905547508659,"worst_top20":0.06477582454681396,"worst_random20":0.008303612470626831,"mean_random20":0.0019388142138874777,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":31,"meets_target":false,"seconds":410.3863262090017,"rank_by_mean":23},{"config":"proj_gate_fp16","canonical":"set_b32__K_gate","variant":"b32","groups":[],"kinds":["gate"],"embed":false,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.05993455648422241,"mean_of_row_mean_abs_dp":0.006121997949376236,"worst_top20":0.05993455648422241,"worst_random20":0.009798780083656311,"mean_random20":0.0019216134533053264,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":24,"meets_target":false,"seconds":230.01575470800162,"rank_by_mean":24},{"config":"layers_12-15_fp16","canonical":"set_b32__G_3","variant":"b32","groups":[3],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.0745232105255127,"mean_of_row_mean_abs_dp":0.006196226572210435,"worst_top20":0.0745232105255127,"worst_random20":0.009913146495819092,"mean_random20":0.0018882591306464747,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":366.15587212500395,"rank_by_mean":25},{"config":"proj_z_fp16","canonical":"set_b32__K_z","variant":"b32","groups":[],"kinds":["z"],"embed":false,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5303914836,"worst_max_abs_dp":0.07238650321960449,"mean_of_row_mean_abs_dp":0.006343265754743091,"worst_top20":0.07238650321960449,"worst_random20":0.011164292693138123,"mean_random20":0.0020222710246647086,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":259.53927225002553,"rank_by_mean":26},{"config":"layers_16-19_fp16","canonical":"set_b32__G_4","variant":"b32","groups":[4],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.0730968713760376,"mean_of_row_mean_abs_dp":0.006627808520085335,"worst_top20":0.0730968713760376,"worst_random20":0.00919640064239502,"mean_random20":0.00214334764071585,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":484.28784824998,"rank_by_mean":27},{"config":"proj_v_fp16","canonical":"set_b32__K_v","variant":"b32","groups":[],"kinds":["v"],"embed":false,"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"within_cap":true,"est_main_mlirb_bytes":5087646036,"worst_max_abs_dp":0.0703086256980896,"mean_of_row_mean_abs_dp":0.006661105860354534,"worst_top20":0.0703086256980896,"worst_random20":0.009902596473693848,"mean_random20":0.0019843684131046757,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":204.7319894169923,"rank_by_mean":28},{"config":"proj_q_fp16","canonical":"set_b32__K_q","variant":"b32","groups":[],"kinds":["q"],"embed":false,"fp16_params":167772160,"fp16_fraction":0.047006977598237236,"within_cap":true,"est_main_mlirb_bytes":5225271636,"worst_max_abs_dp":0.06540495157241821,"mean_of_row_mean_abs_dp":0.006704179285491894,"worst_top20":0.06540495157241821,"worst_random20":0.009820669889450073,"mean_random20":0.002143552075237191,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":25,"meets_target":false,"seconds":479.9542120829865,"rank_by_mean":29},{"config":"proj_b_fp16","canonical":"set_b32__K_b","variant":"b32","groups":[],"kinds":["b"],"embed":false,"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"within_cap":true,"est_main_mlirb_bytes":5069828436,"worst_max_abs_dp":0.06486314535140991,"mean_of_row_mean_abs_dp":0.0067299234731278065,"worst_top20":0.06486314535140991,"worst_random20":0.01032152771949768,"mean_random20":0.002134489799694469,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":22,"meets_target":false,"seconds":338.58007441600785,"rank_by_mean":30},{"config":"proj_a_fp16","canonical":"set_b32__K_a","variant":"b32","groups":[],"kinds":["a"],"embed":false,"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"within_cap":true,"est_main_mlirb_bytes":5069828436,"worst_max_abs_dp":0.06977176666259766,"mean_of_row_mean_abs_dp":0.006796375088015338,"worst_top20":0.06977176666259766,"worst_random20":0.011193007230758667,"mean_random20":0.002160367214916429,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":24,"meets_target":false,"seconds":228.7779359999986,"rank_by_mean":31},{"config":"layers_20-23_fp16","canonical":"set_b32__G_5","variant":"b32","groups":[5],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07091116905212402,"mean_of_row_mean_abs_dp":0.006808550176307715,"worst_top20":0.07091116905212402,"worst_random20":0.010197639465332031,"mean_random20":0.002137279763701372,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":24,"meets_target":false,"seconds":553.9259412500251,"rank_by_mean":32},{"config":"proj_k_fp16","canonical":"set_b32__K_k","variant":"b32","groups":[],"kinds":["k"],"embed":false,"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"within_cap":true,"est_main_mlirb_bytes":5087646036,"worst_max_abs_dp":0.07084512710571289,"mean_of_row_mean_abs_dp":0.0068797385679014635,"worst_top20":0.07084512710571289,"worst_random20":0.00920674204826355,"mean_random20":0.0020649644546210768,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":22,"meets_target":false,"seconds":196.86913270800142,"rank_by_mean":33},{"config":"all_int8_b32","canonical":"set_b32","variant":"b32","groups":[],"kinds":[],"embed":false,"fp16_params":0,"fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5067985236,"worst_max_abs_dp":0.07101064920425415,"mean_of_row_mean_abs_dp":0.006890123262807417,"worst_top20":0.07101064920425415,"worst_random20":0.01022648811340332,"mean_random20":0.002187334212552135,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":40,"meets_target":false,"seconds":238.0854216250009,"rank_by_mean":34},{"config":"layers_24-27_fp16","canonical":"set_b32__G_6","variant":"b32","groups":[6],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07066327333450317,"mean_of_row_mean_abs_dp":0.006893168229192573,"worst_top20":0.07066327333450317,"worst_random20":0.010075300931930542,"mean_random20":0.0021898970711238994,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":24,"meets_target":false,"seconds":454.8524019999895,"rank_by_mean":35},{"config":"layers_28-31_fp16","canonical":"set_b32__G_7","variant":"b32","groups":[7],"kinds":[],"embed":false,"fp16_params":446136320,"fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07120299339294434,"mean_of_row_mean_abs_dp":0.006918687499128282,"worst_top20":0.07120299339294434,"worst_random20":0.01006174087524414,"mean_random20":0.0021833432236841566,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":13,"meets_target":false,"seconds":401.49867716699373,"rank_by_mean":36},{"config":"proj_o_fp16","canonical":"set_b32__K_o","variant":"b32","groups":[],"kinds":["o"],"embed":false,"fp16_params":83886080,"fp16_fraction":0.023503488799118618,"within_cap":true,"est_main_mlirb_bytes":5146628436,"worst_max_abs_dp":0.08551710844039917,"mean_of_row_mean_abs_dp":0.00697625177000494,"worst_top20":0.08551710844039917,"worst_random20":0.010287106037139893,"mean_random20":0.002228544380826255,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8_b32":20,"meets_target":false,"seconds":258.60858891601674,"rank_by_mean":37}],"configs":{"set_fp32":{"canonical":"set_fp32","variant":"fp32","groups":[],"kinds":[],"embed":false,"fp16_modules":248,"int8_modules":0,"fp16_params":3569090560,"fp16_fraction":1.0,"est_main_mlirb_bytes":8414007636,"modules_swapped":0,"swap_seconds":7.258402183651924e-05,"worst_max_abs_dp":4.0531158447265625e-06,"mean_of_row_mean_abs_dp":7.05648223326231e-07,"argmax_equal_non_near_tie":true,"seconds":235.8534802920185,"finished":"2026-10-03T21:27:04+09:00","name":"exact"},"set_fp32__E":{"canonical":"set_fp32__E","variant":"fp32","groups":[],"kinds":[],"embed":true,"fp16_modules":248,"int8_modules":0,"fp16_params":3569090560,"fp16_fraction":1.0,"est_main_mlirb_bytes":7818039636,"modules_swapped":1,"swap_seconds":2.641464457992697,"worst_max_abs_dp":0.005160510540008545,"mean_of_row_mean_abs_dp":0.0005552305745853421,"argmax_equal_non_near_tie":true,"seconds":209.13520808299654,"finished":"2026-10-03T21:30:34+09:00","name":"embed_int8_only"},"set_b16":{"canonical":"set_b16","variant":"b16","groups":[],"kinds":[],"embed":false,"fp16_modules":0,"int8_modules":248,"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5291053396,"modules_swapped":249,"swap_seconds":13.160504833009327,"worst_max_abs_dp":0.04704171419143677,"mean_of_row_mean_abs_dp":0.004532722396979807,"argmax_equal_non_near_tie":true,"seconds":199.70211466599721,"finished":"2026-10-03T21:33:53+09:00","name":"all_int8_b16"},"set_b32__K_qkv":{"canonical":"set_b32__K_qkv","variant":"b32","groups":[],"kinds":["qkv"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":503316480,"fp16_fraction":0.14102093279471173,"est_main_mlirb_bytes":5539844436,"modules_swapped":248,"swap_seconds":8.945315084012691,"worst_max_abs_dp":0.058991432189941406,"mean_of_row_mean_abs_dp":0.005999668049286508,"argmax_equal_non_near_tie":true,"seconds":207.4134852499992,"finished":"2026-10-03T21:37:21+09:00","name":"proj_qkv_fp16"},"set_b32__K_z":{"canonical":"set_b32__K_z","variant":"b32","groups":[],"kinds":["z"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"est_main_mlirb_bytes":5303914836,"modules_swapped":48,"swap_seconds":3.137954666017322,"worst_max_abs_dp":0.07238650321960449,"mean_of_row_mean_abs_dp":0.006343265754743091,"argmax_equal_non_near_tie":true,"seconds":259.53927225002553,"finished":"2026-10-03T21:41:40+09:00","name":"proj_z_fp16"},"set_b32__K_a":{"canonical":"set_b32__K_a","variant":"b32","groups":[],"kinds":["a"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"est_main_mlirb_bytes":5069828436,"modules_swapped":48,"swap_seconds":0.7379625409957953,"worst_max_abs_dp":0.06977176666259766,"mean_of_row_mean_abs_dp":0.006796375088015338,"argmax_equal_non_near_tie":true,"seconds":228.7779359999986,"finished":"2026-10-03T21:45:29+09:00","name":"proj_a_fp16"},"set_b32__K_b":{"canonical":"set_b32__K_b","variant":"b32","groups":[],"kinds":["b"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":1966080,"fp16_fraction":0.0005508630187293427,"est_main_mlirb_bytes":5069828436,"modules_swapped":48,"swap_seconds":0.04276195800048299,"worst_max_abs_dp":0.06486314535140991,"mean_of_row_mean_abs_dp":0.0067299234731278065,"argmax_equal_non_near_tie":true,"seconds":338.58007441600785,"finished":"2026-10-03T21:51:08+09:00","name":"proj_b_fp16"},"set_b32__K_out":{"canonical":"set_b32__K_out","variant":"b32","groups":[],"kinds":["out"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"est_main_mlirb_bytes":5303914836,"modules_swapped":48,"swap_seconds":2.5126293750072364,"worst_max_abs_dp":0.024914264678955078,"mean_of_row_mean_abs_dp":0.004831143981573405,"argmax_equal_non_near_tie":true,"seconds":490.278810000018,"finished":"2026-10-03T21:59:18+09:00","name":"proj_out_fp16"},"set_b32__K_q":{"canonical":"set_b32__K_q","variant":"b32","groups":[],"kinds":["q"],"embed":false,"fp16_modules":8,"int8_modules":240,"fp16_params":167772160,"fp16_fraction":0.047006977598237236,"est_main_mlirb_bytes":5225271636,"modules_swapped":32,"swap_seconds":1.9108644170046318,"worst_max_abs_dp":0.06540495157241821,"mean_of_row_mean_abs_dp":0.006704179285491894,"argmax_equal_non_near_tie":true,"seconds":479.9542120829865,"finished":"2026-10-03T22:07:18+09:00","name":"proj_q_fp16"},"set_b32__G_0":{"canonical":"set_b32__G_0","variant":"b32","groups":[0],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":37,"swap_seconds":9.18897437499254,"worst_max_abs_dp":0.02719932794570923,"mean_of_row_mean_abs_dp":0.0042045773240776425,"argmax_equal_non_near_tie":true,"seconds":560.1727806669951,"finished":"2026-10-03T22:16:38+09:00","name":"layers_0-3_fp16"},"set_b32__G_1":{"canonical":"set_b32__G_1","variant":"b32","groups":[1],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":5.1323822090053,"worst_max_abs_dp":0.04578268527984619,"mean_of_row_mean_abs_dp":0.004949301713398501,"argmax_equal_non_near_tie":true,"seconds":474.50616370901116,"finished":"2026-10-03T22:24:33+09:00","name":"layers_4-7_fp16"},"set_b32__G_2":{"canonical":"set_b32__G_2","variant":"b32","groups":[2],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":5.2431293340050615,"worst_max_abs_dp":0.06477582454681396,"mean_of_row_mean_abs_dp":0.006092905547508659,"argmax_equal_non_near_tie":true,"seconds":410.3863262090017,"finished":"2026-10-03T22:31:23+09:00","name":"layers_8-11_fp16"},"set_b32__G_3":{"canonical":"set_b32__G_3","variant":"b32","groups":[3],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":2.3485775419976562,"worst_max_abs_dp":0.0745232105255127,"mean_of_row_mean_abs_dp":0.006196226572210435,"argmax_equal_non_near_tie":true,"seconds":366.15587212500395,"finished":"2026-10-03T22:37:29+09:00","name":"layers_12-15_fp16"},"set_b32":{"canonical":"set_b32","variant":"b32","groups":[],"kinds":[],"embed":false,"fp16_modules":0,"int8_modules":248,"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5067985236,"modules_swapped":248,"swap_seconds":2.892110042012064,"worst_max_abs_dp":0.07101064920425415,"mean_of_row_mean_abs_dp":0.006890123262807417,"argmax_equal_non_near_tie":true,"seconds":238.0854216250009,"finished":"2026-10-03T21:27:37+09:00","name":"all_int8_b32"},"set_b32__K_k":{"canonical":"set_b32__K_k","variant":"b32","groups":[],"kinds":["k"],"embed":false,"fp16_modules":8,"int8_modules":240,"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"est_main_mlirb_bytes":5087646036,"modules_swapped":8,"swap_seconds":0.011979083006735891,"worst_max_abs_dp":0.07084512710571289,"mean_of_row_mean_abs_dp":0.0068797385679014635,"argmax_equal_non_near_tie":true,"seconds":196.86913270800142,"finished":"2026-10-03T21:30:54+09:00","name":"proj_k_fp16"},"set_aff32":{"canonical":"set_aff32","variant":"aff32","groups":[],"kinds":[],"embed":false,"fp16_modules":0,"int8_modules":248,"fp16_params":0,"fp16_fraction":0.0,"est_main_mlirb_bytes":5179519316,"modules_swapped":248,"swap_seconds":9.449734666995937,"worst_max_abs_dp":0.032596468925476074,"mean_of_row_mean_abs_dp":0.0030056005895797476,"argmax_equal_non_near_tie":true,"seconds":197.64485462501762,"finished":"2026-10-03T21:34:11+09:00","name":"all_int8_affine_b32"},"set_b32__K_v":{"canonical":"set_b32__K_v","variant":"b32","groups":[],"kinds":["v"],"embed":false,"fp16_modules":8,"int8_modules":240,"fp16_params":20971520,"fp16_fraction":0.0058758721997796545,"est_main_mlirb_bytes":5087646036,"modules_swapped":248,"swap_seconds":2.4002699999837205,"worst_max_abs_dp":0.0703086256980896,"mean_of_row_mean_abs_dp":0.006661105860354534,"argmax_equal_non_near_tie":true,"seconds":204.7319894169923,"finished":"2026-10-03T21:37:36+09:00","name":"proj_v_fp16"},"set_b32__K_o":{"canonical":"set_b32__K_o","variant":"b32","groups":[],"kinds":["o"],"embed":false,"fp16_modules":8,"int8_modules":240,"fp16_params":83886080,"fp16_fraction":0.023503488799118618,"est_main_mlirb_bytes":5146628436,"modules_swapped":16,"swap_seconds":0.5794667080044746,"worst_max_abs_dp":0.08551710844039917,"mean_of_row_mean_abs_dp":0.00697625177000494,"argmax_equal_non_near_tie":true,"seconds":258.60858891601674,"finished":"2026-10-03T21:41:55+09:00","name":"proj_o_fp16"},"set_b32__K_gate":{"canonical":"set_b32__K_gate","variant":"b32","groups":[],"kinds":["gate"],"embed":false,"fp16_modules":32,"int8_modules":216,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036,"modules_swapped":40,"swap_seconds":3.911157541006105,"worst_max_abs_dp":0.05993455648422241,"mean_of_row_mean_abs_dp":0.006121997949376236,"argmax_equal_non_near_tie":true,"seconds":230.01575470800162,"finished":"2026-10-03T21:45:45+09:00","name":"proj_gate_fp16"},"set_b32__K_up":{"canonical":"set_b32__K_up","variant":"b32","groups":[],"kinds":["up"],"embed":false,"fp16_modules":32,"int8_modules":216,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036,"modules_swapped":64,"swap_seconds":6.6046641250140965,"worst_max_abs_dp":0.07001703977584839,"mean_of_row_mean_abs_dp":0.006040579349113008,"argmax_equal_non_near_tie":true,"seconds":373.84406879200833,"finished":"2026-10-03T21:51:58+09:00","name":"proj_up_fp16"},"set_b32__K_down":{"canonical":"set_b32__K_down","variant":"b32","groups":[],"kinds":["down"],"embed":false,"fp16_modules":32,"int8_modules":216,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036,"modules_swapped":64,"swap_seconds":7.081261041981634,"worst_max_abs_dp":0.07015883922576904,"mean_of_row_mean_abs_dp":0.006056326911154125,"argmax_equal_non_near_tie":true,"seconds":510.9828269589925,"finished":"2026-10-03T22:00:29+09:00","name":"proj_down_fp16"},"set_b32__G_4":{"canonical":"set_b32__G_4","variant":"b32","groups":[4],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":55,"swap_seconds":7.0227454580017366,"worst_max_abs_dp":0.0730968713760376,"mean_of_row_mean_abs_dp":0.006627808520085335,"argmax_equal_non_near_tie":true,"seconds":484.28784824998,"finished":"2026-10-03T22:08:34+09:00","name":"layers_16-19_fp16"},"set_b32__G_5":{"canonical":"set_b32__G_5","variant":"b32","groups":[5],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":4.017618250014493,"worst_max_abs_dp":0.07091116905212402,"mean_of_row_mean_abs_dp":0.006808550176307715,"argmax_equal_non_near_tie":true,"seconds":553.9259412500251,"finished":"2026-10-03T22:17:48+09:00","name":"layers_20-23_fp16"},"set_b32__G_6":{"canonical":"set_b32__G_6","variant":"b32","groups":[6],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":3.629827208991628,"worst_max_abs_dp":0.07066327333450317,"mean_of_row_mean_abs_dp":0.006893168229192573,"argmax_equal_non_near_tie":true,"seconds":454.8524019999895,"finished":"2026-10-03T22:25:23+09:00","name":"layers_24-27_fp16"},"set_b32__G_7":{"canonical":"set_b32__G_7","variant":"b32","groups":[7],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5486238036,"modules_swapped":62,"swap_seconds":3.591278667008737,"worst_max_abs_dp":0.07120299339294434,"mean_of_row_mean_abs_dp":0.006918687499128282,"argmax_equal_non_near_tie":true,"seconds":401.49867716699373,"finished":"2026-10-03T22:32:04+09:00","name":"layers_28-31_fp16"},"set_b32__K_qkv_out":{"canonical":"set_b32__K_qkv_out","variant":"b32","groups":[],"kinds":["qkv","out"],"embed":false,"fp16_modules":48,"int8_modules":200,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5775774036,"modules_swapped":200,"swap_seconds":10.828139499993995,"worst_max_abs_dp":0.018633544445037842,"mean_of_row_mean_abs_dp":0.003931028543884167,"argmax_equal_non_near_tie":true,"seconds":236.12805604099412,"finished":"2026-10-03T22:41:59+09:00","name":"set_b32__K_qkv_out"},"set_b32__K_qkv_b_out_v":{"canonical":"set_b32__K_qkv_b_out_v","variant":"b32","groups":[],"kinds":["qkv","b","out","v"],"embed":false,"fp16_modules":80,"int8_modules":168,"fp16_params":777912320,"fp16_fraction":0.21795813441057657,"est_main_mlirb_bytes":5797278036,"modules_swapped":32,"swap_seconds":0.030229208001401275,"worst_max_abs_dp":0.019581615924835205,"mean_of_row_mean_abs_dp":0.0037199081680349385,"argmax_equal_non_near_tie":true,"seconds":209.8121754580061,"finished":"2026-10-03T22:45:29+09:00","name":"set_b32__K_qkv_b_out_v"},"set_b32__K_qkv_out_v":{"canonical":"set_b32__K_qkv_out_v","variant":"b32","groups":[],"kinds":["qkv","out","v"],"embed":false,"fp16_modules":56,"int8_modules":192,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"est_main_mlirb_bytes":5795434836,"modules_swapped":192,"swap_seconds":10.8301059580117,"worst_max_abs_dp":0.02115654945373535,"mean_of_row_mean_abs_dp":0.0038814362924798236,"argmax_equal_non_near_tie":true,"seconds":236.144449000014,"finished":"2026-10-03T22:41:59+09:00","name":"set_b32__K_qkv_out_v"},"set_b32__K_qkv_a_b_out_v":{"canonical":"set_b32__K_qkv_a_b_out_v","variant":"b32","groups":[],"kinds":["qkv","a","b","out","v"],"embed":false,"fp16_modules":104,"int8_modules":144,"fp16_params":779878400,"fp16_fraction":0.2185089974293059,"est_main_mlirb_bytes":5799121236,"modules_swapped":48,"swap_seconds":0.015223083988530561,"worst_max_abs_dp":0.017477869987487793,"mean_of_row_mean_abs_dp":0.0036948911649233194,"argmax_equal_non_near_tie":true,"seconds":209.79919770901324,"finished":"2026-10-03T22:45:29+09:00","name":"set_b32__K_qkv_a_b_out_v"},"set_b32__G_0_1":{"canonical":"set_b32__G_0_1","variant":"b32","groups":[0,1],"kinds":[],"embed":false,"fp16_modules":62,"int8_modules":186,"fp16_params":892272640,"fp16_fraction":0.25,"est_main_mlirb_bytes":5904490836,"modules_swapped":186,"swap_seconds":7.796173415990779,"worst_max_abs_dp":0.018460214138031006,"mean_of_row_mean_abs_dp":0.0024146552666691906,"argmax_equal_non_near_tie":true,"seconds":182.4436264160031,"finished":"2026-10-03T22:49:13+09:00","name":"set_b32__G_0_1"},"set_aff32__K_out":{"canonical":"set_aff32__K_out","variant":"aff32","groups":[],"kinds":["out"],"embed":false,"fp16_modules":24,"int8_modules":224,"fp16_params":251658240,"fp16_fraction":0.07051046639735586,"est_main_mlirb_bytes":5407584596,"modules_swapped":224,"swap_seconds":9.098056167014875,"worst_max_abs_dp":0.035969555377960205,"mean_of_row_mean_abs_dp":0.002786534290741353,"argmax_equal_non_near_tie":true,"seconds":212.75119783400442,"finished":"2026-10-03T22:53:03+09:00","name":"set_aff32__K_out"},"set_aff32__K_qkv_out":{"canonical":"set_aff32__K_qkv_out","variant":"aff32","groups":[],"kinds":["qkv","out"],"embed":false,"fp16_modules":48,"int8_modules":200,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5863715156,"modules_swapped":24,"swap_seconds":1.897479041974293,"worst_max_abs_dp":0.016337275505065918,"mean_of_row_mean_abs_dp":0.0020814303058432414,"argmax_equal_non_near_tie":true,"seconds":217.49563470800058,"finished":"2026-10-03T22:56:40+09:00","name":"set_aff32__K_qkv_out"},"set_aff32__G_0":{"canonical":"set_aff32__G_0","variant":"aff32","groups":[0],"kinds":[],"embed":false,"fp16_modules":31,"int8_modules":217,"fp16_params":446136320,"fp16_fraction":0.125,"est_main_mlirb_bytes":5583830356,"modules_swapped":217,"swap_seconds":9.098883167025633,"worst_max_abs_dp":0.024686098098754883,"mean_of_row_mean_abs_dp":0.002719062058798348,"argmax_equal_non_near_tie":true,"seconds":211.0626966250129,"finished":"2026-10-03T22:53:01+09:00","name":"set_aff32__G_0"},"set_aff32__K_qkv_out_v":{"canonical":"set_aff32__K_qkv_out_v","variant":"aff32","groups":[],"kinds":["qkv","out","v"],"embed":false,"fp16_modules":56,"int8_modules":192,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"est_main_mlirb_bytes":5882720596,"modules_swapped":73,"swap_seconds":4.234884833014803,"worst_max_abs_dp":0.01644355058670044,"mean_of_row_mean_abs_dp":0.0019123301824826439,"argmax_equal_non_near_tie":true,"seconds":219.261534417019,"finished":"2026-10-03T22:56:40+09:00","name":"set_aff32__K_qkv_out_v"},"set_aff32__G_0__K_out":{"canonical":"set_aff32__G_0__K_out","variant":"aff32","groups":[0],"kinds":["out"],"embed":false,"fp16_modules":52,"int8_modules":196,"fp16_params":666337280,"fp16_fraction":0.18669665809768637,"est_main_mlirb_bytes":5783387476,"modules_swapped":196,"swap_seconds":12.911560415988788,"worst_max_abs_dp":0.038170695304870605,"mean_of_row_mean_abs_dp":0.0028898118715248227,"argmax_equal_non_near_tie":true,"seconds":260.04487779099145,"finished":"2026-10-03T23:01:41+09:00","name":"set_aff32__G_0__K_out"},"set_aff32__K_qkv_out_v__E":{"canonical":"set_aff32__K_qkv_out_v__E","variant":"aff32","groups":[],"kinds":["qkv","out","v"],"embed":true,"fp16_modules":56,"int8_modules":192,"fp16_params":775946240,"fp16_fraction":0.21740727139184723,"est_main_mlirb_bytes":5286752596,"modules_swapped":193,"swap_seconds":23.921300292015076,"worst_max_abs_dp":0.015426576137542725,"mean_of_row_mean_abs_dp":0.0018798819457879292,"argmax_equal_non_near_tie":true,"seconds":352.3266149999981,"finished":"2026-10-03T23:08:00+09:00","name":"set_aff32__K_qkv_out_v__E"},"set_aff32__K_qkv_out__E":{"canonical":"set_aff32__K_qkv_out__E","variant":"aff32","groups":[],"kinds":["qkv","out"],"embed":true,"fp16_modules":48,"int8_modules":200,"fp16_params":754974720,"fp16_fraction":0.21153139919206757,"est_main_mlirb_bytes":5267747156,"modules_swapped":201,"swap_seconds":23.9578764999751,"worst_max_abs_dp":0.015333414077758789,"mean_of_row_mean_abs_dp":0.002010115250062275,"argmax_equal_non_near_tie":true,"seconds":348.9667994589836,"finished":"2026-10-03T23:07:56+09:00","name":"set_aff32__K_qkv_out__E"}},"generated_at":"2026-10-03T23:14:09+09:00"},"candidates":{"aff32_k_qkv_out_v":{"export":{"mode":"int8mix","model":"kev-4b","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out,v","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_v_pf16_4b.json"],"started":"2026-10-03T22:57:34+09:00","script_sha256":"a09c169b8b352ace3192f84db0a5b514f6c116661c0a9b0430baf371814d687b","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"embedding":null,"fp16_layers":[],"fp16_kinds":["qkv","out","v"],"fp16_layer_patterns":[],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj","model\\.layers\\.\\d+\\.self_attn\\.v_proj"],"int8_linear_modules":192,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.3.self_attn.v_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.7.self_attn.v_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.11.self_attn.v_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.15.self_attn.v_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.19.self_attn.v_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.23.self_attn.v_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.27.self_attn.v_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj","model.layers.31.self_attn.v_proj"],"int8_params":2793144320,"fp16_linear_params":775946240,"embedding_int8_params":0,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":26.012668084003963},"seconds":{"load":0.9431929159909487,"quantize":26.01269233401399,"export":78.940754249983,"optimize":0.5107688750140369,"export_optimize":79.45152312499704,"save":7.223417957982747,"total":113.99759366599028},"du_aimodel":"5.5G","du_bundle":"5.5G","aimodel_files":{"main.hash":32,"main.mlirb":5882832315,"metadata.json":105},"main_mlirb":{"bytes":5882832315,"sha256":"2fd6ade96e63f11cb29fdcad1ce1b3385bdfdecf74a3ccda4c645a2cc2175203"},"metadata_sha256":"96e9d6f855ce55ae6477d7b89d622c50bb18636a0f7451eb849996ec610276f5","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T22:57:38+09:00","free_bytes":243165814784,"free_gib":226.5,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_load":{"time":"2026-10-03T22:57:47+09:00","free_bytes":236114223104,"free_gib":219.9,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_export":{"time":"2026-10-03T22:59:32+09:00","free_bytes":218415288320,"free_gib":203.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_save":{"time":"2026-10-03T22:59:39+09:00","free_bytes":212473131008,"free_gib":197.9,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":200.56915374999517,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T22:59:43+09:00","free_bytes":212318019584,"free_gib":197.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after":{"time":"2026-10-03T23:03:03+09:00","free_bytes":188442812416,"free_gib":175.5,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}},"du_aimodelc":"12G","digest":{"bytes":13026681747,"tree_sha256":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"02c3e80bb2bb8515bbe8776daae3cd49e4245968fa9274f73b06b6bf64fc1485","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"d595fb7aa0686c671e853714121fc8a39087fa86fe8162785726a4e32622923f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a0a1991895776839bc1defb091b57b5bfb847b3f088aa8ee021c1ec26c0c1218","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"d133f40b094f642fa13857e8fcbe61dc20eeef69ed7fd5e80c572403635f5471","main-h16c.mlirb":"a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083","main.hash":"3ff7f7177e697d6fe532d942c6e8d6b7f87485d02d0dd72f19ba8dbb309ce96d","metadata.json":"b7bf9b94ccf6a431c029f106773e4a326c3089e1fe1d336ed5128b00bd0a0ec5","stats.json":"125a2401676ec4ace2475a298e07cc51200e6adf9fc7bd56bd4b84607f8d04d8"}}},"finished":"2026-10-03T23:03:10+09:00"},"readout":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 asymmetric (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":192,"fp16_layers":[],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.3.self_attn.v_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.7.self_attn.v_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.11.self_attn.v_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.15.self_attn.v_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.19.self_attn.v_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.23.self_attn.v_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.27.self_attn.v_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj","model.layers.31.self_attn.v_proj"],"head":"none in the graph (the pointer head runs on the host, head/)","linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_kinds":["qkv","out","v"],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj","model\\.layers\\.\\d+\\.self_attn\\.v_proj"],"fp16_linear_params":775946240,"int8_linear_params":2793144320,"embedding":"fp16","embedding_spec":null},"prefill_chunk":16,"metadata_sha256":"96e9d6f855ce55ae6477d7b89d622c50bb18636a0f7451eb849996ec610276f5","main_mlirb":{"bytes":5882832315,"sha256":"2fd6ade96e63f11cb29fdcad1ce1b3385bdfdecf74a3ccda4c645a2cc2175203"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.h16c.aimodelc","bytes":13026681747,"tree_sha256":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"02c3e80bb2bb8515bbe8776daae3cd49e4245968fa9274f73b06b6bf64fc1485","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"d595fb7aa0686c671e853714121fc8a39087fa86fe8162785726a4e32622923f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a0a1991895776839bc1defb091b57b5bfb847b3f088aa8ee021c1ec26c0c1218","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"d133f40b094f642fa13857e8fcbe61dc20eeef69ed7fd5e80c572403635f5471","main-h16c.mlirb":"a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083","main.hash":"3ff7f7177e697d6fe532d942c6e8d6b7f87485d02d0dd72f19ba8dbb309ce96d","metadata.json":"b7bf9b94ccf6a431c029f106773e4a326c3089e1fe1d336ed5128b00bd0a0ec5","stats.json":"125a2401676ec4ace2475a298e07cc51200e6adf9fc7bd56bd4b84607f8d04d8"}}},"chunk":16,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":432,"questions_non_near_tie":422,"argmax_equal_non_near_tie":422,"near_tie_questions":12,"argmax_equal_near_tie":10,"max_abs_dp":0.017461538314819336,"max_abs_dp_non_near_tie":0.017461538314819336,"mean_of_run_mean_abs_dp":0.0013533845412987554,"mean_of_record_mean_abs_dp":0.001393916890073778,"mean_question_max_abs_dp":0.0020065166231583854,"max_abs_dlogit_T":0.10048961639404297,"worst_run":{"id":"semif_fda0ca94652c6d739ecf","q":0,"max_abs_dp":0.017461538314819336},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9415156194925247,"hidden_max_abs_diff":2.7269492149353027,"positions":9168,"positions_below":{"0.99":1,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":230261.45657562301,"ms_per_call_median":45.52325000986457,"ms_per_call_warm_median":45.52024998702109},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.009758442640304565,"max_abs_dp_non_near_tie":0.009758442640304565,"mean_of_run_mean_abs_dp":0.0018859154508226808,"mean_of_record_mean_abs_dp":0.001728669985481629,"mean_question_max_abs_dp":0.002746379370556066,"max_abs_dlogit_T":0.054791390895843506,"worst_run":{"id":"own_j01","q":0,"max_abs_dp":0.009758442640304565},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9415156194925247,"hidden_max_abs_diff":2.7269492149353027,"positions":1027,"positions_below":{"0.99":1,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":16625.384162936825,"ms_per_call_median":45.00704200472683,"ms_per_call_warm_median":44.99812499852851},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.004800230264663696,"max_abs_dp_non_near_tie":0.004800230264663696,"mean_of_run_mean_abs_dp":0.00041017816966455994,"mean_of_record_mean_abs_dp":0.0004363045977969837,"mean_question_max_abs_dp":0.0007421751083298163,"max_abs_dlogit_T":0.025659680366516113,"worst_run":{"id":"own_L01","q":2,"max_abs_dp":0.004800230264663696},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9938577349674755,"hidden_max_abs_diff":0.983454704284668,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":49203.47574999323,"ms_per_call_median":45.2178539999295,"ms_per_call_warm_median":45.2178539999295},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":40,"questions_non_near_tie":38,"argmax_equal_non_near_tie":38,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.007106959819793701,"max_abs_dp_non_near_tie":0.007106959819793701,"mean_of_run_mean_abs_dp":0.0007981466338969768,"mean_of_record_mean_abs_dp":0.000771316358746385,"mean_question_max_abs_dp":0.0013167357072234153,"max_abs_dlogit_T":0.056304931640625,"worst_run":{"id":"own_t05","q":0,"max_abs_dp":0.007106959819793701},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9998213348579491,"hidden_max_abs_diff":0.454925537109375,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":21333.464667812223,"ms_per_call_median":45.45329199754633,"ms_per_call_warm_median":45.45329199754633},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":143,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.017461538314819336,"max_abs_dp_non_near_tie":0.017461538314819336,"mean_of_run_mean_abs_dp":0.0013706477984669618,"mean_of_record_mean_abs_dp":0.0013706477984669618,"mean_question_max_abs_dp":0.0020559726900602174,"max_abs_dlogit_T":0.09160768985748291,"worst_run":{"id":"semif_fda0ca94652c6d739ecf","q":0,"max_abs_dp":0.017461538314819336},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9996458301907165,"hidden_max_abs_diff":0.42974090576171875,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":41653.872580995085,"ms_per_call_median":45.72324999026023,"ms_per_call_warm_median":45.71000000578351},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":58,"argmax_equal_non_near_tie":58,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.01740133762359619,"max_abs_dp_non_near_tie":0.01740133762359619,"mean_of_run_mean_abs_dp":0.0015879287178904634,"mean_of_record_mean_abs_dp":0.0015879287178904634,"mean_question_max_abs_dp":0.0029923898982815444,"max_abs_dlogit_T":0.0971103310585022,"worst_run":{"id":"tv4_012","q":0,"max_abs_dp":0.01740133762359619},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.999521177553229,"hidden_max_abs_diff":1.1195526123046875,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":48072.38224698813,"ms_per_call_median":47.544333996484056,"ms_per_call_warm_median":47.546792004141025},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":19,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":0,"max_abs_dp":0.015772998332977295,"max_abs_dp_non_near_tie":0.015772998332977295,"mean_of_run_mean_abs_dp":0.0025667355318243308,"mean_of_record_mean_abs_dp":0.0025667355318243308,"mean_question_max_abs_dp":0.003850091900676489,"max_abs_dlogit_T":0.06114614009857178,"worst_run":{"id":"tv4s_07","q":0,"max_abs_dp":0.015772998332977295},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":7821.331206010655,"ms_per_call_median":45.209895994048566,"ms_per_call_warm_median":45.229167008074},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":137,"argmax_equal_non_near_tie":137,"near_tie_questions":3,"argmax_equal_near_tie":3,"max_abs_dp":0.014517366886138916,"max_abs_dp_non_near_tie":0.014517366886138916,"mean_of_run_mean_abs_dp":0.0012222496817620204,"mean_of_record_mean_abs_dp":0.0012222496817620204,"mean_question_max_abs_dp":0.0014657739475036839,"max_abs_dlogit_T":0.10048961639404297,"worst_run":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.014517366886138916},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":45551.54596088687,"ms_per_call_median":45.51587501191534,"ms_per_call_warm_median":45.512375014368445}},"near_ties":[{"id":"tv4_021","q":0,"qid":"answer","oracle_top2_margin":0.01855599880218506,"argmax_equal":true,"max_abs_dp":0.003992319107055664,"probs":[0.2865726053714752,0.15517184138298035,0.3061567544937134,0.25209876894950867]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.00373879075050354,"argmax_equal":true,"max_abs_dp":0.002202257513999939,"probs":[0.25567489862442017,0.2403239607810974,0.259280264377594,0.2447209656238556]},{"id":"tv4x_emotion_15","q":0,"qid":"emotion","oracle_top2_margin":0.007101953029632568,"argmax_equal":true,"max_abs_dp":0.0007041096687316895,"probs":[0.0006040267762728035,0.48937058448791504,0.4950677454471588,0.0008357945480383933,0.0008729963446967304,0.013248863629996777]},{"id":"tv4x_tweet_offensive_01","q":0,"qid":"offensive","oracle_top2_margin":0.002196580171585083,"argmax_equal":true,"max_abs_dp":0.0004487037658691406,"probs":[0.4984530508518219,0.5015469789505005]},{"id":"tv4x_paws_01","q":0,"qid":"paraphrase","oracle_top2_margin":0.007073014974594116,"argmax_equal":true,"max_abs_dp":0.0030095577239990234,"probs":[0.4934539496898651,0.5065460205078125]},{"id":"tv4s_11","q":0,"qid":"decision","oracle_top2_margin":0.001054227352142334,"argmax_equal":false,"max_abs_dp":0.002634674310684204,"probs":[0.010721319355070591,0.4967227578163147,0.4925559163093567]},{"id":"semif_ffee861474a9665a093a","q":0,"qid":"decision","oracle_top2_margin":0.0018825829029083252,"argmax_equal":true,"max_abs_dp":0.0011571943759918213,"probs":[0.3996962904930115,0.20004945993423462,0.4002542495727539]},{"id":"semif_2708193212d8a4a523c7","q":0,"qid":"decision","oracle_top2_margin":0.0197182297706604,"argmax_equal":true,"max_abs_dp":0.0034458041191101074,"probs":[0.2336970716714859,0.3732680678367615,0.39303484559059143]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0004886090755462646,"argmax_equal":false,"max_abs_dp":0.001360177993774414,"probs":[0.4362610876560211,0.12646442651748657,0.43727442622184753]},{"id":"own_t01","q":2,"qid":"upset","oracle_top2_margin":0.0016841888427734375,"argmax_equal":true,"max_abs_dp":0.001039355993270874,"probs":[0.3656497597694397,0.3691098093986511,0.19581156969070435,0.06942886859178543]},{"id":"own_t02","q":0,"qid":"impact","oracle_top2_margin":0.0024104714393615723,"argmax_equal":true,"max_abs_dp":0.0016285479068756104,"probs":[0.19274607300758362,0.3706933856010437,0.3710741698741913,0.0654863640666008]},{"id":"own_j05","q":2,"qid":"severity","oracle_top2_margin":0.0014663934707641602,"argmax_equal":true,"max_abs_dp":0.0012592524290084839,"probs":[0.11699266731739044,0.05302128940820694,0.11863147467374802,0.21336358785629272,0.20956841111183167,0.1687900573015213,0.11963255703449249]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"PASS","timing":{"contended":true,"S":16,"load_seconds":{"first_process":56.80768845797866,"median":1.1586471250047907,"all":[56.80768845797866,1.203329042007681,1.1043284590123221,1.0952729170094244,1.1822534169768915,1.020543833001284,1.1019314590084832,1.0033420420077164,1.1586471250047907,7.479524124995805,1.2710119590046816]},"ms_per_call_warm_runs":{"n":3972,"median":45.52024998702109,"p10":44.594962484552525,"p90":47.97514999809209},"ms_per_call_all_runs":{"n":4055,"median":45.52325000986457,"p10":44.59638299886137,"p90":48.03075880627148},"ms_per_token_warm_runs":{"n":423,"median":3.1219680667466796,"p10":2.893592436538297,"p90":3.5513146672880818},"run_decode_seconds_warm_runs":{"n":423,"median":0.27477358299074695,"p10":0.18515701659489422,"p90":0.5786928331886885},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":184584.6136973705}},"seconds":{"gpu_processes":317.9352423749806,"scoring":8.575821332982741,"total":326.7812737089989},"generated_at":"2026-10-03T14:12:21+00:00","note":"round 6 aff32_k_qkv_out_v: 434 rows, AOT h16c efr","compare_with":{"other":"$ZOO_WORK_ROOT/_kev/results/readout_fp16_pf16_4b.json","other_chunk":16,"other_bundle":"kev_4b_decode_fp16_pf16","runs":434,"max_abs_dp":0.01668757200241089,"argmax_equal_runs":433,"hidden_max_abs_diff":8.07421875,"hidden_bit_equal_runs":0,"vs_oracle_on_shared_runs":{"order":["other","this"],"max_abs_dp":[0.015406966209411621,0.017461538314819336]},"timing_warm_common_runs":{"other":{"S":16,"runs":423,"ms_per_call_median":47.738416484207846,"ms_per_token_median":3.1724670328772984,"decode_ms_total":193136.0742090328,"calls_total":3972},"this":{"S":16,"runs":423,"ms_per_call_median":45.52024998702109,"ms_per_token_median":3.1219680667466796,"decode_ms_total":184930.63116061967,"calls_total":3972}}}},"red":{"schema":"kev-decoder-readout-red-arms/1","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 asymmetric (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":192,"fp16_layers":[],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.3.self_attn.v_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.7.self_attn.v_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.11.self_attn.v_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.15.self_attn.v_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.19.self_attn.v_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.23.self_attn.v_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.27.self_attn.v_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj","model.layers.31.self_attn.v_proj"],"head":"none in the graph (the pointer head runs on the host, head/)","linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_kinds":["qkv","out","v"],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj","model\\.layers\\.\\d+\\.self_attn\\.v_proj"],"fp16_linear_params":775946240,"int8_linear_params":2793144320,"embedding":"fp16","embedding_spec":null},"prefill_chunk":16,"metadata_sha256":"96e9d6f855ce55ae6477d7b89d622c50bb18636a0f7451eb849996ec610276f5","main_mlirb":{"bytes":5882832315,"sha256":"2fd6ade96e63f11cb29fdcad1ce1b3385bdfdecf74a3ccda4c645a2cc2175203"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.h16c.aimodelc","bytes":13026681747,"tree_sha256":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"02c3e80bb2bb8515bbe8776daae3cd49e4245968fa9274f73b06b6bf64fc1485","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"d595fb7aa0686c671e853714121fc8a39087fa86fe8162785726a4e32622923f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a0a1991895776839bc1defb091b57b5bfb847b3f088aa8ee021c1ec26c0c1218","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"d133f40b094f642fa13857e8fcbe61dc20eeef69ed7fd5e80c572403635f5471","main-h16c.mlirb":"a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083","main.hash":"3ff7f7177e697d6fe532d942c6e8d6b7f87485d02d0dd72f19ba8dbb309ce96d","metadata.json":"b7bf9b94ccf6a431c029f106773e4a326c3089e1fe1d336ed5128b00bd0a0ec5","stats.json":"125a2401676ec4ace2475a298e07cc51200e6adf9fc7bd56bd4b84607f8d04d8"}}},"chunk":16,"max_ctx":4096,"rows_file":{"path":"$ZOO_WORK_ROOT/_kev/readout/red_arms_v2.json","sha256":"2de94c557e94ecf318242331d7bcba61e1372e4b4fb0ab8e95a695cd0ca0be89","version":2},"model":"kev-4b","head":{"head_safetensors":"$ZOO_WORK_ROOT/_kev/oracle_4b/head/head.safetensors","head_safetensors_sha256":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head_json_sha256":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416","scale":0.0625,"temperature":2.406050072164233,"dtype":"float32","device":"cpu"},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle_4b/records_oracle.json","sha256":"eb3031df83a75b746beec3480a4bbb47787e13f8a12412ff135feb9f609c70f9","checkpoint":"jaredpalmer/kev-4b@591dcb5bd6d05eb0b5131ea6608f93f10243335c"},"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8mix_aff32_k_qkv_out_v_pf16_4b.json","sha256":"1ff62f8c264c14d8bcd1d70e9226ccd8a029296f862ff0e642fd99c7a23b98dc","result":"PASS"},"script":{"path":"conversion/kev/readout_gate.py","sha256":"9fa297d1ac6cde05208026a582808d7ddc5d35674dba7d21efefabcbf686fd2b"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T22:49:59+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T22:49:59+09:00"}},"process":[{"shard":"red_arms_v2","runs":["tv4_000:q0:base","tv4_000:q0:state_swap","semif_a3f18f3a63d45345942b:q0:base","semif_a3f18f3a63d45345942b:q0:state_swap","semif_0b43ea8e24e74d621f1b:q0:base","semif_0b43ea8e24e74d621f1b:q0:state_swap","own_t01:q0:base","own_t01:q0:state_swap","tv4x_qnli_00:q0:base","tv4x_qnli_00:q0:state_swap","tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_00:q0:instr_negate_v2","tv4x_qnli_00:q0:instr_negate_v2","tv4x_paws_00:q0:base","tv4x_paws_00:q0:instr_negate_v2","tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_00:q0:instr_negate_v2","own_t05:q2:base","own_t05:q2:instr_negate_v2"],"prompts":20,"load_seconds":20.980415791011183,"load_split_seconds":{"model":0.011671541025862098,"main":20.96874424998532},"function_names":["main"],"process_wall_seconds":40.46464187500533,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.41710666599101387},"contract_mismatch":[]}],"rule":"red = the perturbed runs against the unperturbed ones fail the gate's own bar (FACTS §7): (a) an argmax moves on at least one question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over runs of the run's mean |dp| > 0.002","v1_rule":"argmax moved on >= 1 row AND max |dp| > 0.02","arms":{"instr_negate_v2":{"rows":5,"argmax_moved":3,"argmax_moved_non_near_tie":3,"max_abs_dp_vs_base":0.9570446009747684,"mean_of_run_mean_abs_dp_vs_base":0.5549261506646872,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4x_tweet_offensive_00:q0","type":"noul","state_from":null,"tokens":69,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.0007287859916687012,"mean_abs_dp_vs_base":0.0007287263870239258,"base_probs":[0.39821842312812805,0.6017814874649048],"perturbed_probs":[0.3974897563457489,0.6025102734565735],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":null,"tokens":95,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8349138349294662,"mean_abs_dp_vs_base":0.8349138027988374,"base_probs":[0.010238845832645893,0.9897611737251282],"perturbed_probs":[0.8451526165008545,0.15484733879566193],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"tv4x_paws_00:q0","type":"noul","state_from":null,"tokens":82,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.0838938057422638,"mean_abs_dp_vs_base":0.0838937908411026,"base_probs":[0.37249183654785156,0.6275081634521484],"perturbed_probs":[0.45638564229011536,0.543614387512207],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_legacy_holdout_00:q0","type":"noul","state_from":null,"tokens":79,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9570446009747684,"mean_abs_dp_vs_base":0.9570445937570184,"base_probs":[0.0065666441805660725,0.9934333562850952],"perturbed_probs":[0.9636112451553345,0.03638876974582672],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t05:q2","type":"noul","state_from":null,"tokens":118,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8980498551391065,"mean_abs_dp_vs_base":0.8980498395394534,"base_probs":[0.9958463311195374,0.004153645131736994],"perturbed_probs":[0.09779650717973709,0.9022035002708435],"base_argmax":0,"perturbed_argmax":1,"oracle_argmax":0}]},"state_swap":{"rows":5,"argmax_moved":5,"argmax_moved_non_near_tie":5,"max_abs_dp_vs_base":0.9879320068284869,"mean_of_run_mean_abs_dp_vs_base":0.5280939120651843,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4_000:q0","type":"choice","state_from":"tv4_001","tokens":80,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.08751535415649414,"mean_abs_dp_vs_base":0.048233356326818466,"base_probs":[0.31630006432533264,0.12723448872566223,0.2847510278224945,0.2717144191265106],"perturbed_probs":[0.2287847101688385,0.16504837572574615,0.2757996916770935,0.3303672671318054],"base_argmax":0,"perturbed_argmax":3,"oracle_argmax":0},{"row":"semif_a3f18f3a63d45345942b:q0","type":"choice","state_from":"semif_0b43ea8e24e74d621f1b","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9857998049119487,"mean_abs_dp_vs_base":0.6571998827081794,"base_probs":[0.0011719476897269487,0.011805960908532143,0.9870221018791199],"perturbed_probs":[0.9846547842025757,0.01412296760827303,0.0012222969671711326],"base_argmax":2,"perturbed_argmax":0,"oracle_argmax":2},{"row":"semif_0b43ea8e24e74d621f1b:q0","type":"choice","state_from":"semif_942ec65dac0bfb9874f9","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9788130572997034,"mean_abs_dp_vs_base":0.6525420407221342,"base_probs":[0.015727460384368896,0.9831636548042297,0.001108872122131288],"perturbed_probs":[0.38065606355667114,0.004350597504526377,0.6149933338165283],"base_argmax":1,"perturbed_argmax":2,"oracle_argmax":1},{"row":"own_t01:q0","type":"choice","state_from":"own_t03","tokens":245,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.5571587458252907,"mean_abs_dp_vs_base":0.29456227496266363,"base_probs":[0.5710565447807312,0.09102371335029602,0.20207776129245758,0.025980444625020027,0.10986163467168808],"perturbed_probs":[0.013897798955440521,0.13018763065338135,0.02283077873289585,0.1828690767288208,0.6502147316932678],"base_argmax":0,"perturbed_argmax":4,"oracle_argmax":0},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":"tv4x_paws_00","tokens":52,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9879320068284869,"mean_abs_dp_vs_base":0.987932005606126,"base_probs":[0.010238845832645893,0.9897611737251282],"perturbed_probs":[0.9981708526611328,0.0018291693413630128],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1}]}},"base_vs_gate_run":[{"row":"tv4_000:q0","bit_equal_gate_run":true},{"row":"semif_a3f18f3a63d45345942b:q0","bit_equal_gate_run":true},{"row":"semif_0b43ea8e24e74d621f1b:q0","bit_equal_gate_run":true},{"row":"own_t01:q0","bit_equal_gate_run":true},{"row":"tv4x_qnli_00:q0","bit_equal_gate_run":true},{"row":"tv4x_tweet_offensive_00:q0","bit_equal_gate_run":true},{"row":"tv4x_paws_00:q0","bit_equal_gate_run":true},{"row":"tv4x_legacy_holdout_00:q0","bit_equal_gate_run":true},{"row":"own_t05:q2","bit_equal_gate_run":true}],"base_bit_equal_all":true,"all_arms_red":true,"reset_bit_equal":true,"result":"RED (every arm)","seconds":49.82038904199726,"note":"round 6 aff32_k_qkv_out_v: red arms v2","generated_at":"2026-10-03T14:13:28+00:00"}},"aff32_k_qkv_out":{"export":{"mode":"int8mix","model":"kev-4b","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_pf16_4b.json"],"started":"2026-10-03T23:06:57+09:00","script_sha256":"a09c169b8b352ace3192f84db0a5b514f6c116661c0a9b0430baf371814d687b","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"embedding":null,"fp16_layers":[],"fp16_kinds":["qkv","out"],"fp16_layer_patterns":[],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"int8_linear_modules":200,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj"],"int8_params":2814115840,"fp16_linear_params":754974720,"embedding_int8_params":0,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":55.1019569170021},"seconds":{"load":3.2831740000110585,"quantize":55.10198633300024,"export":64.6792930419906,"optimize":0.5173500420060009,"export_optimize":65.1966430839966,"save":5.86821395799052,"total":129.78574254200794},"du_aimodel":"5.5G","du_bundle":"5.5G","aimodel_files":{"main.hash":32,"main.mlirb":5863831482,"metadata.json":105},"main_mlirb":{"bytes":5863831482,"sha256":"3f6273baddbf54762319573841c0211236ada523ba7d23312f1809cd26b84ca7"},"metadata_sha256":"d56899f9e443d366f0a0ff344e2d7d1ca9ef0c5c76d085d850a415cf6d146234","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T23:07:02+09:00","free_bytes":177939660800,"free_gib":165.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_load":{"time":"2026-10-03T23:07:15+09:00","free_bytes":177792204800,"free_gib":165.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_export":{"time":"2026-10-03T23:09:15+09:00","free_bytes":186312650752,"free_gib":173.5,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_save":{"time":"2026-10-03T23:09:21+09:00","free_bytes":178055290880,"free_gib":165.8,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":138.20918299999903,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T23:09:24+09:00","free_bytes":180298035200,"free_gib":167.9,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after":{"time":"2026-10-03T23:11:42+09:00","free_bytes":161109393408,"free_gib":150.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","721e14da868dfb065bde78ec88b4c87fdf96b4a96e610c2a5ca4b9d14c12d733":"5.4G","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b":"12G","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}},"du_aimodelc":"12G","digest":{"bytes":13007682053,"tree_sha256":"eb2fb7c909b6d39586161d70a581be96370fef30f5fee142267e387c075eb96b","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bb3f371e5956d5e6327e41b771980e52ccc2a0791a75ded459e6712e90adc2f3","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"a501bc2178f1b4d01a0dfb7526b58df7df6e0f0bbdb14d61f8b838ccbf449bef","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"ae0f21775abc88e5598f3b3dbbe7555b5c7af8f10d53924c6412eb64bd484075","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"83d3ccd67f0d5b8d48fac9233a68de604963b684a004503975aec5af35e9751d","main-h16c.mlirb":"46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c","main.hash":"c454ce8bf81d5c94de917623bd92b2f6014218bc0fc69e91ad278bf8f50b1cfa","metadata.json":"829674e4633ef202c3554f4585bce640fcfc6ea416349623cf0aade87644acff","stats.json":"db6dbe1f43284c6b2e433e9f5b1a1e886d0f6927a168e10f6a8f1611face8019"}}},"finished":"2026-10-03T23:11:47+09:00"},"readout":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 asymmetric (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":200,"fp16_layers":[],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj"],"head":"none in the graph (the pointer head runs on the host, head/)","linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_kinds":["qkv","out"],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"fp16_linear_params":754974720,"int8_linear_params":2814115840,"embedding":"fp16","embedding_spec":null},"prefill_chunk":16,"metadata_sha256":"d56899f9e443d366f0a0ff344e2d7d1ca9ef0c5c76d085d850a415cf6d146234","main_mlirb":{"bytes":5863831482,"sha256":"3f6273baddbf54762319573841c0211236ada523ba7d23312f1809cd26b84ca7"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.h16c.aimodelc","bytes":13007682053,"tree_sha256":"eb2fb7c909b6d39586161d70a581be96370fef30f5fee142267e387c075eb96b","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"bb3f371e5956d5e6327e41b771980e52ccc2a0791a75ded459e6712e90adc2f3","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"a501bc2178f1b4d01a0dfb7526b58df7df6e0f0bbdb14d61f8b838ccbf449bef","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"ae0f21775abc88e5598f3b3dbbe7555b5c7af8f10d53924c6412eb64bd484075","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"83d3ccd67f0d5b8d48fac9233a68de604963b684a004503975aec5af35e9751d","main-h16c.mlirb":"46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c","main.hash":"c454ce8bf81d5c94de917623bd92b2f6014218bc0fc69e91ad278bf8f50b1cfa","metadata.json":"829674e4633ef202c3554f4585bce640fcfc6ea416349623cf0aade87644acff","stats.json":"db6dbe1f43284c6b2e433e9f5b1a1e886d0f6927a168e10f6a8f1611face8019"}}},"chunk":16,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":432,"questions_non_near_tie":422,"argmax_equal_non_near_tie":422,"near_tie_questions":12,"argmax_equal_near_tie":10,"max_abs_dp":0.01744312047958374,"max_abs_dp_non_near_tie":0.01744312047958374,"mean_of_run_mean_abs_dp":0.0013839940496439224,"mean_of_record_mean_abs_dp":0.001421317362834233,"mean_question_max_abs_dp":0.002033599769569985,"max_abs_dlogit_T":0.11106133460998535,"worst_run":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.01744312047958374},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9301937457470312,"hidden_max_abs_diff":2.8880820274353027,"positions":9168,"positions_below":{"0.99":4,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":664261.3455129613,"ms_per_call_median":149.92716599954292,"ms_per_call_warm_median":149.89616700040642},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.009649157524108887,"max_abs_dp_non_near_tie":0.009649157524108887,"mean_of_run_mean_abs_dp":0.0018749845207404498,"mean_of_record_mean_abs_dp":0.0016974964370743244,"mean_question_max_abs_dp":0.002672838659859017,"max_abs_dlogit_T":0.05534029006958008,"worst_run":{"id":"own_j01","q":0,"max_abs_dp":0.009649157524108887},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9301937457470312,"hidden_max_abs_diff":2.8880820274353027,"positions":1027,"positions_below":{"0.99":4,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":54170.393205975415,"ms_per_call_median":147.4728749890346,"ms_per_call_warm_median":147.0304584945552},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.00547337532043457,"max_abs_dp_non_near_tie":0.00547337532043457,"mean_of_run_mean_abs_dp":0.0005136649745206038,"mean_of_record_mean_abs_dp":0.0005830846255263075,"mean_question_max_abs_dp":0.0009671583433042873,"max_abs_dlogit_T":0.021982669830322266,"worst_run":{"id":"own_L01","q":2,"max_abs_dp":0.00547337532043457},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9916974792902908,"hidden_max_abs_diff":1.151423454284668,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":168050.99720798898,"ms_per_call_median":149.72949950606562,"ms_per_call_warm_median":149.72949950606562},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":40,"questions_non_near_tie":38,"argmax_equal_non_near_tie":38,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.008698254823684692,"max_abs_dp_non_near_tie":0.008698254823684692,"mean_of_run_mean_abs_dp":0.0008661174956553927,"mean_of_record_mean_abs_dp":0.0008289034294426547,"mean_question_max_abs_dp":0.0014232947840355336,"max_abs_dlogit_T":0.06014108657836914,"worst_run":{"id":"own_m01","q":2,"max_abs_dp":0.008698254823684692},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9998153325643375,"hidden_max_abs_diff":0.4547443389892578,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":72180.94342001132,"ms_per_call_median":150.17820801585913,"ms_per_call_warm_median":150.17820801585913},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":143,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.0164794921875,"max_abs_dp_non_near_tie":0.0164794921875,"mean_of_run_mean_abs_dp":0.0013391640015335592,"mean_of_record_mean_abs_dp":0.0013391640015335592,"mean_question_max_abs_dp":0.0020087498907944085,"max_abs_dlogit_T":0.10269975662231445,"worst_run":{"id":"semif_8a4c3de28be95c105ca1","q":0,"max_abs_dp":0.0164794921875},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.999269316294879,"hidden_max_abs_diff":0.5432243347167969,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":118952.18342298176,"ms_per_call_median":148.76841599470936,"ms_per_call_warm_median":148.60012498684227},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":58,"argmax_equal_non_near_tie":58,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.01535874605178833,"max_abs_dp_non_near_tie":0.01535874605178833,"mean_of_run_mean_abs_dp":0.0016195259341960386,"mean_of_record_mean_abs_dp":0.0016195259341960386,"mean_question_max_abs_dp":0.003002736127624909,"max_abs_dlogit_T":0.09108740091323853,"worst_run":{"id":"tv4_012","q":0,"max_abs_dp":0.01535874605178833},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.999219510604577,"hidden_max_abs_diff":1.4320526123046875,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":85066.81175212725,"ms_per_call_median":151.3700420036912,"ms_per_call_warm_median":151.28650049155112},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":19,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":0,"max_abs_dp":0.0167694091796875,"max_abs_dp_non_near_tie":0.0167694091796875,"mean_of_run_mean_abs_dp":0.002795689452129106,"mean_of_record_mean_abs_dp":0.002795689452129106,"mean_question_max_abs_dp":0.004193531069904566,"max_abs_dlogit_T":0.07748398184776306,"worst_run":{"id":"tv4s_07","q":0,"max_abs_dp":0.0167694091796875},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":23262.118712969823,"ms_per_call_median":150.37495850992855,"ms_per_call_warm_median":152.03570798621513},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":137,"argmax_equal_non_near_tie":137,"near_tie_questions":3,"argmax_equal_near_tie":3,"max_abs_dp":0.01744312047958374,"max_abs_dp_non_near_tie":0.01744312047958374,"mean_of_run_mean_abs_dp":0.0012772052556448172,"mean_of_record_mean_abs_dp":0.0012772052556448172,"mean_question_max_abs_dp":0.0015066647070593067,"max_abs_dlogit_T":0.11106133460998535,"worst_run":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.01744312047958374},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":142577.89779090672,"ms_per_call_median":150.40412498638034,"ms_per_call_warm_median":150.36829098244198}},"near_ties":[{"id":"tv4_021","q":0,"qid":"answer","oracle_top2_margin":0.01855599880218506,"argmax_equal":true,"max_abs_dp":0.003582566976547241,"probs":[0.28719082474708557,0.1551506370306015,0.30515000224113464,0.2525085210800171]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.00373879075050354,"argmax_equal":true,"max_abs_dp":0.0020050853490829468,"probs":[0.25635093450546265,0.24034985899925232,0.25838106870651245,0.24491813778877258]},{"id":"tv4x_emotion_15","q":0,"qid":"emotion","oracle_top2_margin":0.007101953029632568,"argmax_equal":true,"max_abs_dp":0.0008075833320617676,"probs":[0.0006042288150638342,0.4894740581512451,0.4949743151664734,0.0008379850187338889,0.000876587990205735,0.01323280856013298]},{"id":"tv4x_tweet_offensive_01","q":0,"qid":"offensive","oracle_top2_margin":0.002196580171585083,"argmax_equal":true,"max_abs_dp":0.0011809468269348145,"probs":[0.4977208077907562,0.5022792220115662]},{"id":"tv4x_paws_01","q":0,"qid":"paraphrase","oracle_top2_margin":0.007073014974594116,"argmax_equal":true,"max_abs_dp":0.003036320209503174,"probs":[0.49342718720436096,0.5065727829933167]},{"id":"tv4s_11","q":0,"qid":"decision","oracle_top2_margin":0.001054227352142334,"argmax_equal":false,"max_abs_dp":0.002012699842453003,"probs":[0.010798851028084755,0.4960232973098755,0.4931778907775879]},{"id":"semif_ffee861474a9665a093a","q":0,"qid":"decision","oracle_top2_margin":0.0018825829029083252,"argmax_equal":true,"max_abs_dp":0.0015850961208343506,"probs":[0.3996427059173584,0.20053091645240784,0.3998263478279114]},{"id":"semif_2708193212d8a4a523c7","q":0,"qid":"decision","oracle_top2_margin":0.0197182297706604,"argmax_equal":true,"max_abs_dp":0.0033105164766311646,"probs":[0.23356178402900696,0.3732737898826599,0.39316442608833313]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0004886090755462646,"argmax_equal":false,"max_abs_dp":0.0011162161827087402,"probs":[0.4365299940109253,0.12636232376098633,0.4371076822280884]},{"id":"own_t01","q":2,"qid":"upset","oracle_top2_margin":0.0016841888427734375,"argmax_equal":true,"max_abs_dp":0.0008691251277923584,"probs":[0.36551713943481445,0.36886313557624817,0.1961292177438736,0.0694904699921608]},{"id":"own_t02","q":0,"qid":"impact","oracle_top2_margin":0.0024104714393615723,"argmax_equal":true,"max_abs_dp":0.0008884072303771973,"probs":[0.19230917096138,0.3694174587726593,0.37255799770355225,0.06571533530950546]},{"id":"own_j05","q":2,"qid":"severity","oracle_top2_margin":0.0014663934707641602,"argmax_equal":true,"max_abs_dp":0.0007435530424118042,"probs":[0.11637670546770096,0.05273645371198654,0.11804358661174774,0.2130376100540161,0.21035084128379822,0.16937729716300964,0.1200774610042572]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"PASS","timing":{"contended":true,"S":16,"load_seconds":{"first_process":28.69904058400425,"median":1.2692855419882108,"all":[28.69904058400425,1.245784165977966,1.2326305420137942,1.2692855419882108,1.3094890000065789,1.1472539579845034,1.1938712910050526,1.1956327910011169,1.2776919169991743,1.306107750016963,7.352110166015336]},"ms_per_call_warm_runs":{"n":3972,"median":149.89616700040642,"p10":120.55059968843125,"p90":190.82842017814983},"ms_per_call_all_runs":{"n":4055,"median":149.92716599954292,"p10":120.45320899342187,"p90":190.96606638631783},"ms_per_token_warm_runs":{"n":423,"median":10.388892123233271,"p10":9.658031591183784,"p90":11.552671231980705},"run_decode_seconds_warm_runs":{"n":423,"median":0.9048058330081403,"p10":0.6234073496190831,"p90":1.9644538578111685},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":607828.957186648}},"seconds":{"gpu_processes":733.8744237920037,"scoring":9.324042166990694,"total":743.4963517500146},"generated_at":"2026-10-03T14:27:45+00:00","note":"round 6 aff32_k_qkv_out: 434 rows, AOT h16c efr","compare_with":{"other":"$ZOO_WORK_ROOT/_kev/results/readout_fp16_pf16_4b.json","other_chunk":16,"other_bundle":"kev_4b_decode_fp16_pf16","runs":434,"max_abs_dp":0.019427970051765442,"argmax_equal_runs":433,"hidden_max_abs_diff":8.23828125,"hidden_bit_equal_runs":0,"vs_oracle_on_shared_runs":{"order":["other","this"],"max_abs_dp":[0.015406966209411621,0.01744312047958374]},"timing_warm_common_runs":{"other":{"S":16,"runs":423,"ms_per_call_median":47.738416484207846,"ms_per_token_median":3.1724670328772984,"decode_ms_total":193136.0742090328,"calls_total":3972},"this":{"S":16,"runs":423,"ms_per_call_median":149.89616700040642,"ms_per_token_median":10.388892123233271,"decode_ms_total":614880.857135955,"calls_total":3972}}}}},"aff32_g0_k_out":{"export":{"mode":"int8mix","model":"kev-4b","name":"kev_4b_decode_int8mix_aff32_g0_k_out_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-layers","0,1,2,3","--fp16-kinds","out","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_g0_k_out_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_g0_k_out_pf16_4b.json"],"started":"2026-10-03T22:25:52+09:00","script_sha256":"a09c169b8b352ace3192f84db0a5b514f6c116661c0a9b0430baf371814d687b","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16","name":"kev_4b_decode_int8mix_aff32_g0_k_out_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"embedding":null,"fp16_layers":[0,1,2,3],"fp16_kinds":["out"],"fp16_layer_patterns":["model\\.layers\\.0\\..*","model\\.layers\\.1\\..*","model\\.layers\\.2\\..*","model\\.layers\\.3\\..*"],"fp16_patterns":["model\\.layers\\.0\\..*","model\\.layers\\.1\\..*","model\\.layers\\.2\\..*","model\\.layers\\.3\\..*","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"int8_linear_modules":196,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.out_proj"],"int8_params":2902753280,"fp16_linear_params":666337280,"embedding_int8_params":0,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":50.98830795800313},"seconds":{"load":3.904600792011479,"quantize":50.98835341600352,"export":85.16875670899753,"optimize":0.4922861249942798,"export_optimize":85.66104283399181,"save":17.184449041000335,"total":158.02354083300452},"du_aimodel":"5.4G","du_bundle":"5.4G","aimodel_files":{"main.hash":32,"main.mlirb":5783501557,"metadata.json":105},"main_mlirb":{"bytes":5783501557,"sha256":"107c7a7ecbb384064f17e9928b2e1401240c97c4d7ae24f2546816df4f4eb037"},"metadata_sha256":"09223e6cde37b5d1913d09e70c8e763c231982363f7438ec1d1bfe81ffab4482","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T22:25:57+09:00","free_bytes":389121540096,"free_gib":362.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_load":{"time":"2026-10-03T22:26:09+09:00","free_bytes":389112934400,"free_gib":362.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_export":{"time":"2026-10-03T22:28:25+09:00","free_bytes":389135093760,"free_gib":362.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after_save":{"time":"2026-10-03T22:28:42+09:00","free_bytes":383349465088,"free_gib":357.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":174.3474397920072,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T22:28:45+09:00","free_bytes":383112286208,"free_gib":356.8,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}},"after":{"time":"2026-10-03T22:31:40+09:00","free_bytes":369066967040,"free_gib":343.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":"10M","634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":"14M","86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":"12M","91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":"7.8G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G","fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":"12M"}}},"du_aimodelc":"12G","digest":{"bytes":12927351705,"tree_sha256":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"436db3f7d31405e7eb591d122daeabfbf13203403fcf81ca03e459c1edd54646","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"a3c3acb40d7f3506b32a121a8d800b411804c730c3de7ea72dc62c90de7751bb","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b5358941479a664baceee7a9968e5ce472043ee0f0d8df5618bfc9f55d0007ef","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"6edf5a034bad3e5009e24df197c4a3cc958d03a7e01f200821c3653e0d645ad1","main-h16c.mlirb":"8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b","main.hash":"1b49c5f33a8165ebf40a19e3db3c75af3deb33c53aedd628c50172451dfb2262","metadata.json":"ae3a15edd477fd062a9e1ade845214d64f91787fc9414d5567c5b6b772cf1ce6","stats.json":"978efaf27d23501840d461fedaa02e0d7482dd8f0d0642b2cf024c607ef5d095"}}},"finished":"2026-10-03T22:31:46+09:00"},"readout":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16","name":"kev_4b_decode_int8mix_aff32_g0_k_out_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 asymmetric (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":196,"fp16_layers":[0,1,2,3],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.out_proj"],"head":"none in the graph (the pointer head runs on the host, head/)","linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_kinds":["out"],"fp16_patterns":["model\\.layers\\.0\\..*","model\\.layers\\.1\\..*","model\\.layers\\.2\\..*","model\\.layers\\.3\\..*","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"fp16_linear_params":666337280,"int8_linear_params":2902753280,"embedding":"fp16","embedding_spec":null},"prefill_chunk":16,"metadata_sha256":"09223e6cde37b5d1913d09e70c8e763c231982363f7438ec1d1bfe81ffab4482","main_mlirb":{"bytes":5783501557,"sha256":"107c7a7ecbb384064f17e9928b2e1401240c97c4d7ae24f2546816df4f4eb037"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.h16c.aimodelc","bytes":12927351705,"tree_sha256":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"436db3f7d31405e7eb591d122daeabfbf13203403fcf81ca03e459c1edd54646","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"a3c3acb40d7f3506b32a121a8d800b411804c730c3de7ea72dc62c90de7751bb","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b5358941479a664baceee7a9968e5ce472043ee0f0d8df5618bfc9f55d0007ef","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"6edf5a034bad3e5009e24df197c4a3cc958d03a7e01f200821c3653e0d645ad1","main-h16c.mlirb":"8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b","main.hash":"1b49c5f33a8165ebf40a19e3db3c75af3deb33c53aedd628c50172451dfb2262","metadata.json":"ae3a15edd477fd062a9e1ade845214d64f91787fc9414d5567c5b6b772cf1ce6","stats.json":"978efaf27d23501840d461fedaa02e0d7482dd8f0d0642b2cf024c607ef5d095"}}},"chunk":16,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":433,"questions_non_near_tie":422,"argmax_equal_non_near_tie":422,"near_tie_questions":12,"argmax_equal_near_tie":11,"max_abs_dp":0.027203857898712158,"max_abs_dp_non_near_tie":0.027203857898712158,"mean_of_run_mean_abs_dp":0.001441602148014375,"mean_of_record_mean_abs_dp":0.001500734554868702,"mean_question_max_abs_dp":0.0021407545344637974,"max_abs_dlogit_T":0.1620340347290039,"worst_run":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.027203857898712158},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9736495889727766,"hidden_max_abs_diff":2.1092233657836914,"positions":9168,"positions_below":{"0.99":4,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":233247.36080894945,"ms_per_call_median":48.9504580036737,"ms_per_call_warm_median":48.919395994744264},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.008127093315124512,"max_abs_dp_non_near_tie":0.008127093315124512,"mean_of_run_mean_abs_dp":0.0016206839820880673,"mean_of_record_mean_abs_dp":0.0015432534467348469,"mean_question_max_abs_dp":0.002437221401027943,"max_abs_dlogit_T":0.06655740737915039,"worst_run":{"id":"own_j01","q":0,"max_abs_dp":0.008127093315124512},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9736495889727766,"hidden_max_abs_diff":2.1092233657836914,"positions":1027,"positions_below":{"0.99":4,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":15111.407539981883,"ms_per_call_median":46.80158299743198,"ms_per_call_warm_median":46.62570799700916},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0027309060096740723,"max_abs_dp_non_near_tie":0.0027309060096740723,"mean_of_run_mean_abs_dp":0.00037673680203487023,"mean_of_record_mean_abs_dp":0.00039983944844008767,"mean_question_max_abs_dp":0.000638231084766713,"max_abs_dlogit_T":0.03240656852722168,"worst_run":{"id":"own_L01","q":2,"max_abs_dp":0.0027309060096740723},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9966011803848781,"hidden_max_abs_diff":0.8049607276916504,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":50868.172168091405,"ms_per_call_median":46.814417000859976,"ms_per_call_warm_median":46.814417000859976},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":40,"questions_non_near_tie":38,"argmax_equal_non_near_tie":38,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.007811397314071655,"max_abs_dp_non_near_tie":0.007811397314071655,"mean_of_run_mean_abs_dp":0.0008537366202411554,"mean_of_record_mean_abs_dp":0.0008412966452105902,"mean_question_max_abs_dp":0.0013538145634811372,"max_abs_dlogit_T":0.05486917495727539,"worst_run":{"id":"own_t07","q":0,"max_abs_dp":0.007811397314071655},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9996438168233984,"hidden_max_abs_diff":0.548675537109375,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":24101.63212698535,"ms_per_call_median":50.32374997972511,"ms_per_call_warm_median":50.32374997972511},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":143,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.01754772663116455,"max_abs_dp_non_near_tie":0.01754772663116455,"mean_of_run_mean_abs_dp":0.0013005750316315917,"mean_of_record_mean_abs_dp":0.0013005750316315917,"mean_question_max_abs_dp":0.0019508645903745976,"max_abs_dlogit_T":0.10721421241760254,"worst_run":{"id":"semif_8a4c3de28be95c105ca1","q":0,"max_abs_dp":0.01754772663116455},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9997184007991216,"hidden_max_abs_diff":0.31183815002441406,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":42906.49466193281,"ms_per_call_median":49.041541991755366,"ms_per_call_warm_median":49.03204200672917},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":58,"argmax_equal_non_near_tie":58,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.016187191009521484,"max_abs_dp_non_near_tie":0.016187191009521484,"mean_of_run_mean_abs_dp":0.0018800260787732745,"mean_of_record_mean_abs_dp":0.0018800260787732745,"mean_question_max_abs_dp":0.003604778414592147,"max_abs_dlogit_T":0.09377118945121765,"worst_run":{"id":"tv4_012","q":0,"max_abs_dp":0.016187191009521484},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9998388526043764,"hidden_max_abs_diff":0.5258026123046875,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":40726.14062996581,"ms_per_call_median":53.309666021959856,"ms_per_call_warm_median":53.34408349881414},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":20,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.017901182174682617,"max_abs_dp_non_near_tie":0.017901182174682617,"mean_of_run_mean_abs_dp":0.003430828219279647,"mean_of_record_mean_abs_dp":0.003430828219279647,"mean_question_max_abs_dp":0.00514625534415245,"max_abs_dlogit_T":0.08341601490974426,"worst_run":{"id":"tv4s_07","q":0,"max_abs_dp":0.017901182174682617},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":8743.828502047108,"ms_per_call_median":49.545187488547526,"ms_per_call_warm_median":49.54300000099465},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":137,"argmax_equal_non_near_tie":137,"near_tie_questions":3,"argmax_equal_near_tie":3,"max_abs_dp":0.027203857898712158,"max_abs_dp_non_near_tie":0.027203857898712158,"mean_of_run_mean_abs_dp":0.001341913094854265,"mean_of_record_mean_abs_dp":0.001341913094854265,"mean_question_max_abs_dp":0.0015819344578111279,"max_abs_dlogit_T":0.1620340347290039,"worst_run":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.027203857898712158},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":50789.68517994508,"ms_per_call_median":49.826790986116976,"ms_per_call_warm_median":49.811917007900774}},"near_ties":[{"id":"tv4_021","q":0,"qid":"answer","oracle_top2_margin":0.01855599880218506,"argmax_equal":true,"max_abs_dp":0.0035647451877593994,"probs":[0.2876395583152771,0.15522372722625732,0.3046104311943054,0.25252634286880493]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.00373879075050354,"argmax_equal":true,"max_abs_dp":0.0017859339714050293,"probs":[0.25493311882019043,0.2393946796655655,0.2602429986000061,0.2454291731119156]},{"id":"tv4x_emotion_15","q":0,"qid":"emotion","oracle_top2_margin":0.007101953029632568,"argmax_equal":true,"max_abs_dp":0.002295970916748047,"probs":[0.0005997169064357877,0.4909624457359314,0.49351441860198975,0.0008323815418407321,0.0008722509955987334,0.013218728825449944]},{"id":"tv4x_tweet_offensive_01","q":0,"qid":"offensive","oracle_top2_margin":0.002196580171585083,"argmax_equal":true,"max_abs_dp":0.0011425614356994629,"probs":[0.4977591931819916,0.5022408366203308]},{"id":"tv4x_paws_01","q":0,"qid":"paraphrase","oracle_top2_margin":0.007073014974594116,"argmax_equal":true,"max_abs_dp":0.0014911293983459473,"probs":[0.4949724078178406,0.5050276517868042]},{"id":"tv4s_11","q":0,"qid":"decision","oracle_top2_margin":0.001054227352142334,"argmax_equal":true,"max_abs_dp":0.0022298991680145264,"probs":[0.010745275765657425,0.49190646409988403,0.49734824895858765]},{"id":"semif_ffee861474a9665a093a","q":0,"qid":"decision","oracle_top2_margin":0.0018825829029083252,"argmax_equal":false,"max_abs_dp":0.0033592581748962402,"probs":[0.40185314416885376,0.20009465515613556,0.3980521857738495]},{"id":"semif_2708193212d8a4a523c7","q":0,"qid":"decision","oracle_top2_margin":0.0197182297706604,"argmax_equal":true,"max_abs_dp":0.0031692832708358765,"probs":[0.23342055082321167,0.3736460208892822,0.3929334580898285]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0004886090755462646,"argmax_equal":true,"max_abs_dp":0.0010405778884887695,"probs":[0.43809977173805237,0.12580817937850952,0.4360920786857605]},{"id":"own_t01","q":2,"qid":"upset","oracle_top2_margin":0.0016841888427734375,"argmax_equal":true,"max_abs_dp":0.0012609660625457764,"probs":[0.36512529850006104,0.36873430013656616,0.1964353770017624,0.06970500946044922]},{"id":"own_t02","q":0,"qid":"impact","oracle_top2_margin":0.0024104714393615723,"argmax_equal":true,"max_abs_dp":0.0005666911602020264,"probs":[0.1917160600423813,0.3697255551815033,0.3729309141635895,0.06562748551368713]},{"id":"own_j05","q":2,"qid":"severity","oracle_top2_margin":0.0014663934707641602,"argmax_equal":true,"max_abs_dp":0.0010511726140975952,"probs":[0.11662121117115021,0.052839793264865875,0.11810216307640076,0.2133452296257019,0.209990993142128,0.16933158040046692,0.11976896226406097]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":false,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"FAIL","timing":{"contended":true,"S":16,"load_seconds":{"first_process":30.883490875014104,"median":1.2425439579819795,"all":[30.883490875014104,1.3124847909784876,1.315686417015968,1.206084124976769,1.1958954579895362,1.1971607500163373,1.2472491670050658,1.19149650001782,1.249254916998325,1.2312940419942606,1.2425439579819795]},"ms_per_call_warm_runs":{"n":3972,"median":48.919395994744264,"p10":46.420966705773026,"p90":54.18967471632641},"ms_per_call_all_runs":{"n":4055,"median":48.9504580036737,"p10":46.432574989739805,"p90":54.274924797937274},"ms_per_token_warm_runs":{"n":423,"median":3.410700976382941,"p10":3.108510721000879,"p90":3.959064489803635},"run_decode_seconds_warm_runs":{"n":423,"median":0.2933983750117477,"p10":0.20063197499257512,"p90":0.6358215418178589},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":198368.150758688}},"seconds":{"gpu_processes":291.9960477910063,"scoring":9.364725749997888,"total":301.66049300000304},"generated_at":"2026-10-03T13:39:54+00:00","note":"round 6 aff32_g0_k_out: 434 rows, AOT h16c efr","compare_with":{"other":"$ZOO_WORK_ROOT/_kev/results/readout_fp16_pf16_4b.json","other_chunk":16,"other_bundle":"kev_4b_decode_fp16_pf16","runs":434,"max_abs_dp":0.031713783740997314,"argmax_equal_runs":430,"hidden_max_abs_diff":7.015625,"hidden_bit_equal_runs":0,"vs_oracle_on_shared_runs":{"order":["other","this"],"max_abs_dp":[0.015406966209411621,0.027203857898712158]},"timing_warm_common_runs":{"other":{"S":16,"runs":423,"ms_per_call_median":47.738416484207846,"ms_per_token_median":3.1724670328772984,"decode_ms_total":193136.0742090328,"calls_total":3972},"this":{"S":16,"runs":423,"ms_per_call_median":48.919395994744264,"ms_per_token_median":3.410700976382941,"decode_ms_total":199425.44051792356,"calls_total":3972}}}},"red":{"schema":"kev-decoder-readout-red-arms/1","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16","name":"kev_4b_decode_int8mix_aff32_g0_k_out_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 asymmetric (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":196,"fp16_layers":[0,1,2,3],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.out_proj"],"head":"none in the graph (the pointer head runs on the host, head/)","linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_kinds":["out"],"fp16_patterns":["model\\.layers\\.0\\..*","model\\.layers\\.1\\..*","model\\.layers\\.2\\..*","model\\.layers\\.3\\..*","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"fp16_linear_params":666337280,"int8_linear_params":2902753280,"embedding":"fp16","embedding_spec":null},"prefill_chunk":16,"metadata_sha256":"09223e6cde37b5d1913d09e70c8e763c231982363f7438ec1d1bfe81ffab4482","main_mlirb":{"bytes":5783501557,"sha256":"107c7a7ecbb384064f17e9928b2e1401240c97c4d7ae24f2546816df4f4eb037"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.h16c.aimodelc","bytes":12927351705,"tree_sha256":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"436db3f7d31405e7eb591d122daeabfbf13203403fcf81ca03e459c1edd54646","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"a3c3acb40d7f3506b32a121a8d800b411804c730c3de7ea72dc62c90de7751bb","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"b5358941479a664baceee7a9968e5ce472043ee0f0d8df5618bfc9f55d0007ef","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"6edf5a034bad3e5009e24df197c4a3cc958d03a7e01f200821c3653e0d645ad1","main-h16c.mlirb":"8b8e18feaa81530608c01fdb157eb561c7da98a45b70fa667989e6cad344307b","main.hash":"1b49c5f33a8165ebf40a19e3db3c75af3deb33c53aedd628c50172451dfb2262","metadata.json":"ae3a15edd477fd062a9e1ade845214d64f91787fc9414d5567c5b6b772cf1ce6","stats.json":"978efaf27d23501840d461fedaa02e0d7482dd8f0d0642b2cf024c607ef5d095"}}},"chunk":16,"max_ctx":4096,"rows_file":{"path":"$ZOO_WORK_ROOT/_kev/readout/red_arms_v2.json","sha256":"2de94c557e94ecf318242331d7bcba61e1372e4b4fb0ab8e95a695cd0ca0be89","version":2},"model":"kev-4b","head":{"head_safetensors":"$ZOO_WORK_ROOT/_kev/oracle_4b/head/head.safetensors","head_safetensors_sha256":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head_json_sha256":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416","scale":0.0625,"temperature":2.406050072164233,"dtype":"float32","device":"cpu"},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle_4b/records_oracle.json","sha256":"eb3031df83a75b746beec3480a4bbb47787e13f8a12412ff135feb9f609c70f9","checkpoint":"jaredpalmer/kev-4b@591dcb5bd6d05eb0b5131ea6608f93f10243335c"},"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8mix_aff32_g0_k_out_pf16_4b.json","sha256":"3466d71c6ea5f4af07971432d8d546eb1f2ae67d26c9eeacb20bdf06b5d1c8a2","result":"FAIL"},"script":{"path":"conversion/kev/readout_gate.py","sha256":"9fa297d1ac6cde05208026a582808d7ddc5d35674dba7d21efefabcbf686fd2b"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T22:39:17+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T22:39:17+09:00"}},"process":[{"shard":"red_arms_v2","runs":["tv4_000:q0:base","tv4_000:q0:state_swap","semif_a3f18f3a63d45345942b:q0:base","semif_a3f18f3a63d45345942b:q0:state_swap","semif_0b43ea8e24e74d621f1b:q0:base","semif_0b43ea8e24e74d621f1b:q0:state_swap","own_t01:q0:base","own_t01:q0:state_swap","tv4x_qnli_00:q0:base","tv4x_qnli_00:q0:state_swap","tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_00:q0:instr_negate_v2","tv4x_qnli_00:q0:instr_negate_v2","tv4x_paws_00:q0:base","tv4x_paws_00:q0:instr_negate_v2","tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_00:q0:instr_negate_v2","own_t05:q2:base","own_t05:q2:instr_negate_v2"],"prompts":20,"load_seconds":23.89520783399348,"load_split_seconds":{"model":0.007452166988514364,"main":23.887755667004967},"function_names":["main"],"process_wall_seconds":61.11846279198653,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.44046675000572577},"contract_mismatch":[]}],"rule":"red = the perturbed runs against the unperturbed ones fail the gate's own bar (FACTS §7): (a) an argmax moves on at least one question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over runs of the run's mean |dp| > 0.002","v1_rule":"argmax moved on >= 1 row AND max |dp| > 0.02","arms":{"instr_negate_v2":{"rows":5,"argmax_moved":3,"argmax_moved_non_near_tie":3,"max_abs_dp_vs_base":0.9566217139363289,"mean_of_run_mean_abs_dp_vs_base":0.5549906530417502,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4x_tweet_offensive_00:q0","type":"noul","state_from":null,"tokens":69,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.0011981725692749023,"mean_abs_dp_vs_base":0.0011981725692749023,"base_probs":[0.397928386926651,0.6020715832710266],"perturbed_probs":[0.3967302143573761,0.6032697558403015],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":null,"tokens":95,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.836440366692841,"mean_abs_dp_vs_base":0.8364403457380831,"base_probs":[0.010239261202514172,0.98976069688797],"perturbed_probs":[0.8466796278953552,0.15332037210464478],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"tv4x_paws_00:q0","type":"noul","state_from":null,"tokens":82,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08288723230361938,"mean_abs_dp_vs_base":0.08288723230361938,"base_probs":[0.37216371297836304,0.627836287021637],"perturbed_probs":[0.4550509452819824,0.5449490547180176],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_legacy_holdout_00:q0","type":"noul","state_from":null,"tokens":79,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9566217139363289,"mean_abs_dp_vs_base":0.9566217102110386,"base_probs":[0.006519138813018799,0.993480920791626],"perturbed_probs":[0.9631408452987671,0.03685920685529709],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t05:q2","type":"noul","state_from":null,"tokens":118,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8978058286011219,"mean_abs_dp_vs_base":0.897805804386735,"base_probs":[0.9958855509757996,0.004114430397748947],"perturbed_probs":[0.09807977080345154,0.9019202589988708],"base_argmax":0,"perturbed_argmax":1,"oracle_argmax":0}]},"state_swap":{"rows":5,"argmax_moved":5,"argmax_moved_non_near_tie":5,"max_abs_dp_vs_base":0.9879188956692815,"mean_of_run_mean_abs_dp_vs_base":0.5278631829586812,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4_000:q0","type":"choice","state_from":"tv4_001","tokens":80,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.087578684091568,"mean_abs_dp_vs_base":0.047397661954164505,"base_probs":[0.3178490400314331,0.12842625379562378,0.2829435169696808,0.2707812488079071],"perturbed_probs":[0.2302703559398651,0.16640697419643402,0.27572688460350037,0.32759585976600647],"base_argmax":0,"perturbed_argmax":3,"oracle_argmax":0},{"row":"semif_a3f18f3a63d45345942b:q0","type":"choice","state_from":"semif_0b43ea8e24e74d621f1b","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9857345080235973,"mean_abs_dp_vs_base":0.6571562881969536,"base_probs":[0.001171644777059555,0.011855751276016235,0.9869726300239563],"perturbed_probs":[0.9846014976501465,0.01416025497019291,0.0012381220003589988],"base_argmax":2,"perturbed_argmax":0,"oracle_argmax":2},{"row":"semif_0b43ea8e24e74d621f1b:q0","type":"choice","state_from":"semif_942ec65dac0bfb9874f9","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9788366165012121,"mean_abs_dp_vs_base":0.6525577182959145,"base_probs":[0.01563378795981407,0.9832536578178406,0.0011126416502520442],"perturbed_probs":[0.385482519865036,0.004417041316628456,0.6101004481315613],"base_argmax":1,"perturbed_argmax":2,"oracle_argmax":1},{"row":"own_t01:q0","type":"choice","state_from":"own_t03","tokens":245,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.5574034368619323,"mean_abs_dp_vs_base":0.2942853899672627,"base_probs":[0.5714820027351379,0.09097042679786682,0.20126184821128845,0.025868061929941177,0.1104176938533783],"perturbed_probs":[0.014078565873205662,0.13049767911434174,0.022951798513531685,0.1825798749923706,0.6498920917510986],"base_argmax":0,"perturbed_argmax":4,"oracle_argmax":0},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":"tv4x_paws_00","tokens":52,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9879188956692815,"mean_abs_dp_vs_base":0.9879188563791104,"base_probs":[0.010239261202514172,0.98976069688797],"perturbed_probs":[0.9981581568717957,0.0018418797990307212],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1}]}},"base_vs_gate_run":[{"row":"tv4_000:q0","bit_equal_gate_run":true},{"row":"semif_a3f18f3a63d45345942b:q0","bit_equal_gate_run":true},{"row":"semif_0b43ea8e24e74d621f1b:q0","bit_equal_gate_run":true},{"row":"own_t01:q0","bit_equal_gate_run":true},{"row":"tv4x_qnli_00:q0","bit_equal_gate_run":true},{"row":"tv4x_tweet_offensive_00:q0","bit_equal_gate_run":true},{"row":"tv4x_paws_00:q0","bit_equal_gate_run":true},{"row":"tv4x_legacy_holdout_00:q0","bit_equal_gate_run":true},{"row":"own_t05:q2","bit_equal_gate_run":true}],"base_bit_equal_all":true,"all_arms_red":true,"reset_bit_equal":true,"result":"RED (every arm)","seconds":69.90194370798417,"note":"round 6 aff32_g0_k_out: red arms v2","generated_at":"2026-10-03T13:41:19+00:00"}},"aff32_k_qkv_out_v_e":{"export":{"mode":"int8mix","model":"kev-4b","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out,v","--embed-int8","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_v_e_pf16_4b.json"],"started":"2026-10-03T23:16:02+09:00","script_sha256":"2462da9e89a348a544358765ddc0a046be3a34daba4ab7ae4d81f57f38825de1","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d"],"embedding":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_layers":[],"fp16_kinds":["qkv","out","v"],"fp16_layer_patterns":[],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj","model\\.layers\\.\\d+\\.self_attn\\.v_proj"],"int8_linear_modules":192,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.3.self_attn.v_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.7.self_attn.v_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.11.self_attn.v_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.15.self_attn.v_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.19.self_attn.v_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.23.self_attn.v_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.27.self_attn.v_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj","model.layers.31.self_attn.v_proj"],"int8_params":2793144320,"fp16_linear_params":775946240,"embedding_int8_params":635699200,"lm_head":"none in the graph (the tied table is the int8 embedding)","seconds":20.388884332991438},"seconds":{"load":1.0315297090273816,"quantize":20.388929499982623,"export":65.42960866598878,"optimize":0.4321262090234086,"export_optimize":65.86173487501219,"save":4.234692999976687,"total":91.70527550001862},"du_aimodel":"4.9G","du_bundle":"5.0G","aimodel_files":{"main.hash":32,"main.mlirb":5286864947,"metadata.json":105},"main_mlirb":{"bytes":5286864947,"sha256":"eebbdfc2196ba40b7b689433980801d1db24830aa1b2e8740e0223cd616ce2a2"},"metadata_sha256":"a57650802794cb62f3a39fa023b8999b3bc35aec891cacc1d1a49ff86ae369d2","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T23:16:05+09:00","free_bytes":183610888192,"free_gib":171.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_load":{"time":"2026-10-03T23:16:13+09:00","free_bytes":183593873408,"free_gib":171.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_export":{"time":"2026-10-03T23:17:39+09:00","free_bytes":187005919232,"free_gib":174.2,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_save":{"time":"2026-10-03T23:17:44+09:00","free_bytes":181705019392,"free_gib":169.2,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":133.87217504100408,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T23:17:46+09:00","free_bytes":179932938240,"free_gib":167.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after":{"time":"2026-10-03T23:20:00+09:00","free_bytes":168796966912,"free_gib":157.2,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}},"du_aimodelc":"12G","digest":{"bytes":12430715409,"tree_sha256":"7921a80fbd568ef59730f63d0369c269b0ed7f6a7332bd4faf3dc6d54deba38d","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"d403d3bb2c26bdb5d6bb11f6d81c4ce37d6c2cab35d7c175e3b5dbf410c655f6","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"7aa667839099b88f843ee975eb34e0324593dc0240525e0b6af87163cdeb0ac9","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"96c929529b652e675d343170efeaab8fb9735b2ae37ba64ebb724b27ca948ad0","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"e40e4f9babd7d22891299ee4a239a49ad94ef7cd804499c9bf53970fa67c1563","main-h16c.mlirb":"42c9672f29ad6c7f6afdb614c8e597608ba02435b720d5a03aa5c5b124ded2fd","main.hash":"619e54d8e57cccd9bb63611bffa0c6202d900a2a97f44860a43d77cc9fb11560","metadata.json":"050db79c18c9c5e7b06a983e1b2e60cb9991067726fc64ced2c654713b1286e0","stats.json":"dbaf0d99ec9571316191d37c7b2c63d2b0de83c9aabca8796e93ceafcf4ecd59"}}},"finished":"2026-10-03T23:20:06+09:00"}},"aff32_k_qkv_out_e":{"export":{"mode":"int8mix","model":"kev-4b","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","hf_id":"kev-local/kev-4b-v1.0-merged","weights_sha256":"904380cbf0be134e122f2abd7c7bab52f1e2a0f373d3a2e7947a3025e882f3f3","prefill_chunk":16,"argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out","--embed-int8","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_e_pf16_4b.json"],"started":"2026-10-03T23:22:42+09:00","script_sha256":"2462da9e89a348a544358765ddc0a046be3a34daba4ab7ae4d81f57f38825de1","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.aimodel","load_report":{"checkpoint_files":["model-00001-of-00004.safetensors","model-00002-of-00004.safetensors","model-00003-of-00004.safetensors","model-00004-of-00004.safetensors"],"checkpoint_keys":426,"module_tensors":426,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"weights_check":{"snapshot":"$ZOO_WORK_ROOT/_kev/hf/hub/models--kev-local--kev-4b-v1.0-merged/snapshots/904380cbf0be","sha256":{"model-00001-of-00004.safetensors":"45a919c6e0c4a907324e8735cd3be10ab8bdad2ba67fa5844367e0fe3a4bd18f","model-00002-of-00004.safetensors":"9e35c50caf92e662b03a092fc6b57db69683f9ae4076661a2aef962ea175aaab","model-00003-of-00004.safetensors":"c9036c7fed9c87d419f81435955b4a84e6803c1d8b7029e37cd9979aaf6815f3","model-00004-of-00004.safetensors":"47cde07c0ef1909177ea7c6f6bc75033443ca186f8567d72a9ef5bfd000d2b89"}},"loopfree_unrolled_linear_layers":24,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d"],"embedding":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"fp16_layers":[],"fp16_kinds":["qkv","out"],"fp16_layer_patterns":[],"fp16_patterns":["model\\.layers\\.\\d+\\.linear_attn\\.in_proj_qkv","model\\.layers\\.\\d+\\.linear_attn\\.out_proj"],"int8_linear_modules":200,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.out_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.out_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.out_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.out_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.out_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.out_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.out_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.out_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.out_proj","model.layers.12.linear_attn.in_proj_qkv","model.layers.12.linear_attn.out_proj","model.layers.13.linear_attn.in_proj_qkv","model.layers.13.linear_attn.out_proj","model.layers.14.linear_attn.in_proj_qkv","model.layers.14.linear_attn.out_proj","model.layers.16.linear_attn.in_proj_qkv","model.layers.16.linear_attn.out_proj","model.layers.17.linear_attn.in_proj_qkv","model.layers.17.linear_attn.out_proj","model.layers.18.linear_attn.in_proj_qkv","model.layers.18.linear_attn.out_proj","model.layers.20.linear_attn.in_proj_qkv","model.layers.20.linear_attn.out_proj","model.layers.21.linear_attn.in_proj_qkv","model.layers.21.linear_attn.out_proj","model.layers.22.linear_attn.in_proj_qkv","model.layers.22.linear_attn.out_proj","model.layers.24.linear_attn.in_proj_qkv","model.layers.24.linear_attn.out_proj","model.layers.25.linear_attn.in_proj_qkv","model.layers.25.linear_attn.out_proj","model.layers.26.linear_attn.in_proj_qkv","model.layers.26.linear_attn.out_proj","model.layers.28.linear_attn.in_proj_qkv","model.layers.28.linear_attn.out_proj","model.layers.29.linear_attn.in_proj_qkv","model.layers.29.linear_attn.out_proj","model.layers.30.linear_attn.in_proj_qkv","model.layers.30.linear_attn.out_proj"],"int8_params":2814115840,"fp16_linear_params":754974720,"embedding_int8_params":635699200,"lm_head":"none in the graph (the tied table is the int8 embedding)","seconds":20.545604999992065},"seconds":{"load":0.9885537909867708,"quantize":20.545626875013113,"export":61.05222695900011,"optimize":0.4913320409832522,"export_optimize":61.54355899998336,"save":4.896396124997409,"total":88.15310883297934},"du_aimodel":"4.9G","du_bundle":"4.9G","aimodel_files":{"main.hash":32,"main.mlirb":5267864084,"metadata.json":105},"main_mlirb":{"bytes":5267864084,"sha256":"d40673868e523dbbb85fde38acba3972394db08f9e8c6c68a3fa5c6f06569c49"},"metadata_sha256":"04c79a4cc2e926db09614dd755bfb050061d7641d77fb0b0e54120b351440720","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"disk":{"before_load":{"time":"2026-10-03T23:22:45+09:00","free_bytes":145317175296,"free_gib":135.3,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_load":{"time":"2026-10-03T23:22:54+09:00","free_bytes":140345901056,"free_gib":130.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_export":{"time":"2026-10-03T23:24:16+09:00","free_bytes":137149014016,"free_gib":127.7,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after_save":{"time":"2026-10-03T23:24:21+09:00","free_bytes":131313033216,"free_gib":122.3,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":136.1660434579826,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T23:24:23+09:00","free_bytes":132976971776,"free_gib":123.8,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}},"after":{"time":"2026-10-03T23:26:40+09:00","free_bytes":120263143424,"free_gib":112.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","46e3567731091463574878ae29ac03522cf303b9fbd3c1b86823d6ab17ff024c":"12G","a704b8e12d73a509b4e60a5fda8001aba2f399239a0cad486f124463dacfe083":"12G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":"14G"}}},"du_aimodelc":"12G","digest":{"bytes":12411715730,"tree_sha256":"b27879baa771f22ddbdac24a64f44d1c8e1e69d55a85d19c17a7d02dbd7e6b0b","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"175be39d6efc4f101823f26c7189995b7bdb169c6327ff6db7e79253310e5dc6","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"45a1685e230bf4fb21da33f6196176caacc6e213ae1ffac5932011b4e13ff2c5","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"e9849a51444eed3b7b9b94dcb2b8d1b12198f9e06baba074f85585eaed80b1f0","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"2b9a22c11ec39bb47a5b0bb07ecd5c856b9b100532495f6b4af259b3c20502a2","main-h16c.mlirb":"7cfb321b6dd0cfe8d397122a3a96670a40954b11fce8bf10dd8af5b0f7781670","main.hash":"c6e7f6b17d4cfe3d164808cb89a90f16bab2696f7f8645f80a16f46c8df63864","metadata.json":"7dcfe2c3df3d900f2bcda31da8564606a9fa3f2f1361ad55b67e96af7f46bd01","stats.json":"5a498595ac0d37c880e28dfac6eb5d8ab0f77b5d53ed5c721c703531b9f90a57"}}},"finished":"2026-10-03T23:26:45+09:00"}}},"summary":{"schema":"kev-r6-int8-4b-summary/1","bar":{"argmax":"every question with oracle top-2 margin > 0.02","max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002},"candidates":[{"tag":"aff32_k_qkv_out_v","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","mode":"int8mix","argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out,v","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_v_pf16_4b.json"],"fp16_layers":[],"fp16_kinds":["qkv","out","v"],"linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"embedding_int8":false,"fp16_linear_params":775946240,"fp16_fraction":0.21740727139184723,"main_mlirb_bytes":5882832315,"main_mlirb_sha256":"2fd6ade96e63f11cb29fdcad1ce1b3385bdfdecf74a3ccda4c645a2cc2175203","export_seconds":113.99759366599028,"mac_aot_efr":{"bytes":13026681747,"tree_sha256":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388","seconds":200.56915374999517},"mac_aot_noefr":{"bytes":5882606928,"tree_sha256":"2030cef23fed9db583dac0cdbb4747acc0012b4268b4fad862e538f676020466","seconds":6.428049041016493,"returncode":0,"bytes_over_6_4e9":0.9191573325,"bytes_minus_aimodel":-225524},"ios_aot_h19p_efr":{"bytes":13026670589,"tree_sha256":"57b66d9e02c46b5aab8f1b5c4a8093c2582b3ba01f973d4636f2eeb449e41ee0","seconds":188.71786787500605,"returncode":0,"bytes_over_6_4e9":2.03541727953125,"bytes_minus_aimodel":7143838137},"ios_aot_h19p_noefr":{"bytes":5882606928,"tree_sha256":"dbd9e60eeb572520472517782ebe08d8f3ec9eaa6e78f6472688ab861c1e489e","seconds":4.982264542020857,"returncode":0,"bytes_over_6_4e9":0.9191573325,"bytes_minus_aimodel":-225524},"gate_434":{"result":"PASS","rows":434,"argmax_non_near_tie":[422,422],"near_tie":[10,12],"max_abs_dp":0.017461538314819336,"mean_of_run_mean_abs_dp":0.0013533845412987554,"worst":{"id":"semif_fda0ca94652c6d739ecf","q":0,"max_abs_dp":0.017461538314819336},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":45.52024998702109,"ms_per_call_warm_p90_contended":47.97514999809209,"first_call_of_process_ms":[21903.23912500753,1657.780792011181,1519.0591250138823,1482.645042007789,1531.014292006148,1505.3124999976717,1512.0275410008617,1436.7074169858824,1911.708625004394,3402.4507499998435,4053.3110830001533],"load_seconds_median":1.1586471250047907,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.h16c.aimodelc","aimodelc_bytes":13026681747,"compare_with":{"max_abs_dp":0.01668757200241089,"argmax_equal_runs":433,"runs":434,"other":"kev_4b_decode_fp16_pf16"},"transcript_sha_hint":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388"},"heldout_130":{"result":"FAIL","rows":130,"argmax_non_near_tie":[126,126],"near_tie":[4,4],"max_abs_dp":0.027729809284210205,"mean_of_run_mean_abs_dp":0.0012334366360543453,"worst":{"id":"tv4sh_11","q":0,"max_abs_dp":0.027729809284210205},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":46.97656200733036,"ms_per_call_warm_p90_contended":73.41467530350201,"first_call_of_process_ms":[9863.430792000145,2134.5092919946183,1464.272292010719,1647.6309160061646],"load_seconds_median":1.3618676455080276,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.h16c.aimodelc","aimodelc_bytes":13026681747,"compare_with":null,"transcript_sha_hint":"1225a9da15e64281cd1ef9135b95120623fbcf4e5340f3365b4f2be4ff07e388"},"red_v2":{"result":"RED (every arm)","reset_ok":true,"base_bit_equal_gate":true,"arms":{"instr_negate_v2":{"argmax_moved":"3/5","max_abs_dp_vs_base":0.9570446009747684,"red":true},"state_swap":{"argmax_moved":"5/5","max_abs_dp_vs_base":0.9879320068284869,"red":true}}},"gate_noefr":null},{"tag":"aff32_k_qkv_out","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","mode":"int8mix","argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_pf16_4b.json"],"fp16_layers":[],"fp16_kinds":["qkv","out"],"linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"embedding_int8":false,"fp16_linear_params":754974720,"fp16_fraction":0.21153139919206757,"main_mlirb_bytes":5863831482,"main_mlirb_sha256":"3f6273baddbf54762319573841c0211236ada523ba7d23312f1809cd26b84ca7","export_seconds":129.78574254200794,"mac_aot_efr":{"bytes":13007682053,"tree_sha256":"eb2fb7c909b6d39586161d70a581be96370fef30f5fee142267e387c075eb96b","seconds":138.20918299999903},"mac_aot_noefr":{"bytes":5863607354,"tree_sha256":"1012cb914ca2944f78a13cf5e46e0ca36a15b38afff7e2c1cc7358d85723ca68","seconds":3.822517583001172,"returncode":0,"bytes_over_6_4e9":0.9161886490625,"bytes_minus_aimodel":-224265},"ios_aot_h19p_efr":{"bytes":13007670689,"tree_sha256":"13f93a88cabc888429405a99100dafd44a500444c763930fae3057ffaf863ab0","seconds":140.0149582499871,"returncode":0,"bytes_over_6_4e9":2.03244854515625,"bytes_minus_aimodel":7143839070},"ios_aot_h19p_noefr":{"bytes":5863607354,"tree_sha256":"886f453d94f5d2554d5bd9201c5f0701802504dedaee1c518ae172f3e5e741a6","seconds":4.338819541997509,"returncode":0,"bytes_over_6_4e9":0.9161886490625,"bytes_minus_aimodel":-224265},"gate_434":{"result":"PASS","rows":434,"argmax_non_near_tie":[422,422],"near_tie":[10,12],"max_abs_dp":0.01744312047958374,"mean_of_run_mean_abs_dp":0.0013839940496439224,"worst":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.01744312047958374},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":149.89616700040642,"ms_per_call_warm_p90_contended":190.82842017814983,"first_call_of_process_ms":[9953.04354201653,2133.45229200786,1734.9529589992017,1738.6575000127777,1875.4624169960152,1836.143833992537,1767.6562080159783,1682.7057920163497,1771.2631250033155,2062.8615830210038,11700.260957994033],"load_seconds_median":1.2692855419882108,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.h16c.aimodelc","aimodelc_bytes":13007682053,"compare_with":{"max_abs_dp":0.019427970051765442,"argmax_equal_runs":433,"runs":434,"other":"kev_4b_decode_fp16_pf16"},"transcript_sha_hint":"eb2fb7c909b6d39586161d70a581be96370fef30f5fee142267e387c075eb96b"},"heldout_130":null,"red_v2":null,"gate_noefr":null},{"tag":"aff32_k_qkv_out_v_e","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","mode":"int8mix","argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out,v","--embed-int8","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_v_e_pf16_4b.json"],"fp16_layers":[],"fp16_kinds":["qkv","out","v"],"linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"embedding_int8":true,"fp16_linear_params":775946240,"fp16_fraction":0.21740727139184723,"main_mlirb_bytes":5286864947,"main_mlirb_sha256":"eebbdfc2196ba40b7b689433980801d1db24830aa1b2e8740e0223cd616ce2a2","export_seconds":91.70527550001862,"mac_aot_efr":{"bytes":12430715409,"tree_sha256":"7921a80fbd568ef59730f63d0369c269b0ed7f6a7332bd4faf3dc6d54deba38d","seconds":133.87217504100408},"mac_aot_noefr":{"bytes":5286639669,"tree_sha256":"aa1827eb0556b3a259bb9189c9263673ed9c0088b454bca9dffdd3c00994f5cf","seconds":3.6468205000273883,"returncode":0,"bytes_over_6_4e9":0.82603744828125,"bytes_minus_aimodel":-225415},"ios_aot_h19p_efr":{"bytes":12430704328,"tree_sha256":"0ba4a2966a1611bd5599c7957805925b56c33f8d6531bf79b981a2479061c778","seconds":131.2038212500047,"returncode":0,"bytes_over_6_4e9":1.94229755125,"bytes_minus_aimodel":7143839244},"ios_aot_h19p_noefr":{"bytes":5286639669,"tree_sha256":"b48bc68de67677bdeafd7a3ccdbf44ce9d21667272bbf4b2729b176c771a3d7e","seconds":3.49811674997909,"returncode":0,"bytes_over_6_4e9":0.82603744828125,"bytes_minus_aimodel":-225415},"gate_434":null,"heldout_130":null,"red_v2":null,"gate_noefr":null},{"tag":"aff32_k_qkv_out_e","name":"kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","mode":"int8mix","argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-kinds","qkv,out","--embed-int8","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_k_qkv_out_e_pf16_4b.json"],"fp16_layers":[],"fp16_kinds":["qkv","out"],"linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"embedding_int8":true,"fp16_linear_params":754974720,"fp16_fraction":0.21153139919206757,"main_mlirb_bytes":5267864084,"main_mlirb_sha256":"d40673868e523dbbb85fde38acba3972394db08f9e8c6c68a3fa5c6f06569c49","export_seconds":88.15310883297934,"mac_aot_efr":{"bytes":12411715730,"tree_sha256":"b27879baa771f22ddbdac24a64f44d1c8e1e69d55a85d19c17a7d02dbd7e6b0b","seconds":136.1660434579826},"mac_aot_noefr":{"bytes":5267639984,"tree_sha256":"64980da9cd4bfc721db704e8358d4f6a9c7d9d6a8fdf6eb0960a2777826d0b4f","seconds":4.326805332995718,"returncode":0,"bytes_over_6_4e9":0.8230687475,"bytes_minus_aimodel":-224237},"ios_aot_h19p_efr":{"bytes":12411704850,"tree_sha256":"a9bdc64cb5d06097162704fa6e918fb7346a3a83e56aefa67ad6671247da1bef","seconds":133.41884025000036,"returncode":0,"bytes_over_6_4e9":1.9393288828125,"bytes_minus_aimodel":7143840629},"ios_aot_h19p_noefr":{"bytes":5267639984,"tree_sha256":"8d77ba5a6d8313ae04cecd4fb99b50785b50850ea0194542b75559ab93f3b879","seconds":4.44286229199497,"returncode":0,"bytes_over_6_4e9":0.8230687475,"bytes_minus_aimodel":-224237},"gate_434":null,"heldout_130":null,"red_v2":null,"gate_noefr":null},{"tag":"aff32_g0_k_out","name":"kev_4b_decode_int8mix_aff32_g0_k_out_pf16","mode":"int8mix","argv":["int8mix","--model","kev-4b","--quant-scheme","asymmetric","--fp16-layers","0,1,2,3","--fp16-kinds","out","--prefill-chunk","16","--aot","--name","kev_4b_decode_int8mix_aff32_g0_k_out_pf16","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_aff32_g0_k_out_pf16_4b.json"],"fp16_layers":[0,1,2,3],"fp16_kinds":["out"],"linear_spec":{"dtype":"int8","qscheme":"asymmetric","granularity":{"type":"per_block","block_size":32,"axis":1}},"embedding_int8":false,"fp16_linear_params":666337280,"fp16_fraction":0.18669665809768637,"main_mlirb_bytes":5783501557,"main_mlirb_sha256":"107c7a7ecbb384064f17e9928b2e1401240c97c4d7ae24f2546816df4f4eb037","export_seconds":158.02354083300452,"mac_aot_efr":{"bytes":12927351705,"tree_sha256":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff","seconds":174.3474397920072},"mac_aot_noefr":{"bytes":5783276818,"tree_sha256":"620ef7e7cb01a9eec4be00ff9af60f83bce017e3c46b2ca1bf05131e7713fa3e","seconds":5.08118441698025,"returncode":0,"bytes_over_6_4e9":0.9036370028125,"bytes_minus_aimodel":-224876},"ios_aot_h19p_efr":{"bytes":12927340923,"tree_sha256":"668c80f78b894fdf077d5fa224532ba80d6b1e71b3e3259340b14b0fda5cf820","seconds":149.83634054099093,"returncode":0,"bytes_over_6_4e9":2.01989701921875,"bytes_minus_aimodel":7143839229},"ios_aot_h19p_noefr":{"bytes":5783276818,"tree_sha256":"4cd081b69d58d8fb7a46299aa4c19256e9abe8a32c60fc9a9105a713ae975867","seconds":6.37895687497803,"returncode":0,"bytes_over_6_4e9":0.9036370028125,"bytes_minus_aimodel":-224876},"gate_434":{"result":"FAIL","rows":434,"argmax_non_near_tie":[422,422],"near_tie":[11,12],"max_abs_dp":0.027203857898712158,"mean_of_run_mean_abs_dp":0.001441602148014375,"worst":{"id":"tv4x_composition_holdout_13","q":0,"max_abs_dp":0.027203857898712158},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":48.919395994744264,"ms_per_call_warm_p90_contended":54.18967471632641,"first_call_of_process_ms":[12001.647875003982,1856.5701250045095,2071.0544169996865,1770.0634580105543,1800.9072089917026,1793.268999987049,1740.674125001533,1872.1863339887932,1794.0233749977779,1744.9479999777395,1753.3820839889813],"load_seconds_median":1.2425439579819795,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.h16c.aimodelc","aimodelc_bytes":12927351705,"compare_with":{"max_abs_dp":0.031713783740997314,"argmax_equal_runs":430,"runs":434,"other":"kev_4b_decode_fp16_pf16"},"transcript_sha_hint":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff"},"heldout_130":{"result":"FAIL","rows":130,"argmax_non_near_tie":[126,126],"near_tie":[4,4],"max_abs_dp":0.029297292232513428,"mean_of_run_mean_abs_dp":0.0013550090635581935,"worst":{"id":"tv4sh_11","q":0,"max_abs_dp":0.029297292232513428},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":46.602896007243544,"ms_per_call_warm_p90_contended":48.08562499820255,"first_call_of_process_ms":[1541.7207080172375,1776.705917000072,1620.728958019754,1590.872832981404],"load_seconds_median":1.0999067499942612,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.h16c.aimodelc","aimodelc_bytes":12927351705,"compare_with":null,"transcript_sha_hint":"76cb7437165ce8b8a4a3f9cb9d04e4a4207b3ff034d0f2a9596f3d99efaff2ff"},"red_v2":{"result":"RED (every arm)","reset_ok":true,"base_bit_equal_gate":true,"arms":{"instr_negate_v2":{"argmax_moved":"3/5","max_abs_dp_vs_base":0.9566217139363289,"red":true},"state_swap":{"argmax_moved":"5/5","max_abs_dp_vs_base":0.9879188956692815,"red":true}}},"gate_noefr":null}],"references":{"fp16_pf16_4b":{"gate_434":{"result":"PASS","rows":434,"argmax_non_near_tie":[422,422],"near_tie":[9,12],"max_abs_dp":0.015406966209411621,"mean_of_run_mean_abs_dp":0.0007834579897192555,"worst":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.015406966209411621},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":47.738416484207846,"ms_per_call_warm_p90_contended":50.899087500874884,"first_call_of_process_ms":[8203.586165996967,1611.9389589875937,1557.311375014251,1510.0004580162931,1474.0969160047825,1677.6825829874724,1541.10358300386,1671.7187920003198,1661.4506669866387,1668.60579198692,1700.4869579977822],"load_seconds_median":1.5881113749928772,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_fp16_pf16.h16c.aimodelc","aimodelc_bytes":15557770108,"compare_with":null,"transcript_sha_hint":"e1155cab0743b8d4482ad6aa2e6beb2db740eb50bb78eba4b912e578a7d0e663"},"heldout_130":{"result":"PASS","rows":130,"argmax_non_near_tie":[126,126],"near_tie":[2,4],"max_abs_dp":0.010867595672607422,"mean_of_run_mean_abs_dp":0.0007171148204547651,"worst":{"id":"tv4sh_11","q":0,"max_abs_dp":0.010867595672607422},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":55.46641651017126,"ms_per_call_warm_p90_contended":82.2863083856646,"first_call_of_process_ms":[12316.238208004506,19456.900083983783,3689.1148329887073,46071.71233301051],"load_seconds_median":15.66007595800329,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_fp16_pf16.h16c.aimodelc","aimodelc_bytes":15557770108,"compare_with":null,"transcript_sha_hint":"e1155cab0743b8d4482ad6aa2e6beb2db740eb50bb78eba4b912e578a7d0e663"},"main_mlirb_bytes":8414007636,"mac_aot_efr_bytes":15557770108,"mac_aot_noefr":{"bytes":8413701860,"tree_sha256":"c63f546b71345703ec9b4d35d5127152dd4fa93ff0ac2980e2cdb9e04714784b","seconds":6.48417262500152,"returncode":0,"bytes_over_6_4e9":1.314640915625,"bytes_minus_aimodel":-305913},"ios_aot_h19p_efr":{"bytes":15557759365,"tree_sha256":"86d9b615aca9f33733abc275574459870b8b8574065c30980fed9e02e3296848","seconds":27.409209374978673,"returncode":0,"bytes_over_6_4e9":2.43089990078125,"bytes_minus_aimodel":7143751592},"ios_aot_h19p_noefr":{"bytes":8413701860,"tree_sha256":"b795a314f8083c751ab071fea4c3fd9e69168d483842a816d6bdd96eac0bf535","seconds":3.333297290984774,"returncode":0,"bytes_over_6_4e9":1.314640915625,"bytes_minus_aimodel":-305913}},"int8lin_pf16_4b_round4":{"gate_434":{"result":"FAIL","rows":434,"argmax_non_near_tie":[422,422],"near_tie":[9,12],"max_abs_dp":0.060132622718811035,"mean_of_run_mean_abs_dp":0.001917580727576835,"worst":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.060132622718811035},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":79.4767295155907,"ms_per_call_warm_p90_contended":115.98405421536881,"first_call_of_process_ms":[11376.792792027118,2022.3009590117726,2042.648625007132,2017.0743750059046,2047.8446669876575,2673.6463329871185,3272.31358300196,2596.7562909936532,2796.722958999453,4264.315707987407,6271.182666998357],"load_seconds_median":1.7113432089972775,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8lin_pf16.h16c.aimodelc","aimodelc_bytes":12211947289,"compare_with":{"max_abs_dp":0.07553958892822266,"argmax_equal_runs":432,"runs":434,"other":"kev_4b_decode_fp16_pf16"},"transcript_sha_hint":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403"},"heldout_130":{"result":"FAIL","rows":130,"argmax_non_near_tie":[126,126],"near_tie":[2,4],"max_abs_dp":0.028466999530792236,"mean_of_run_mean_abs_dp":0.0016867079466278366,"worst":{"id":"tv4sh_11","q":0,"max_abs_dp":0.028466999530792236},"reset_ok":true,"finite":true,"ms_per_call_warm_median_contended":106.13749999902211,"ms_per_call_warm_p90_contended":117.78658721596001,"first_call_of_process_ms":[6856.40170800616,5083.55945799849,6272.365292010363,5779.135874996427],"load_seconds_median":3.307429457985563,"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_4b_decode_int8lin_pf16.h16c.aimodelc","aimodelc_bytes":12211947289,"compare_with":{"max_abs_dp":0.017599403858184814,"argmax_equal_runs":128,"runs":130,"other":"kev_4b_decode_fp16_pf16"},"transcript_sha_hint":"c96cd7b4a9aba3dd6fbabab6681196fd1dc7773b6506f22b0d12fc1e0409e403"},"main_mlirb_bytes":5068103201,"mac_aot_efr_bytes":12211947289},"fp16_pf16_0_8b":{"main_mlirb_bytes":1506481909,"ios_aot_h19p_efr":{"bytes":2505674625,"tree_sha256":"a9e593ea6e1bd7b0000826d63216ce8a323a04b7bd92f57b36d8f1e2c7caab76","seconds":8.540033166005742,"returncode":0,"bytes_over_6_4e9":0.39151166015625,"bytes_minus_aimodel":999192579},"ios_aot_h19p_noefr":{"bytes":1506400620,"tree_sha256":"c2731425c21fc905070ad65ac4222ab029e144e6e64a47ce30ad89083918bb3c","seconds":2.7169347919989377,"returncode":0,"bytes_over_6_4e9":0.235375096875,"bytes_minus_aimodel":-81426}}},"instrument":{"file":"$ZOO_WORK_ROOT/_kev/results/r6_int8_4b_bisect.json","reproduction_check":{"gate_worst":0.060132622718811035,"torch_all_int8_b32_worst":0.07101064920425415,"ratio":1.1809005826389838,"reproduced":true,"pearson_per_row":0.973113998763023,"rows_above_bar_gate":5,"rows_above_bar_torch":5,"per_row":[{"row":"semif_1105577da4c8dad4609d:q0","gate":0.060132622718811035,"torch":0.07101064920425415},{"row":"semif_d880e65fef5e2993fdd0:q0","gate":0.029746264219284058,"torch":0.026368260383605957},{"row":"semif_7ed75d854d99b3a81916:q0","gate":0.025355279445648193,"torch":0.02151423692703247},{"row":"semif_ef3e0f8e657bf59f8a29:q0","gate":0.022765278816223145,"torch":0.01827031373977661},{"row":"tv4_012:q0","gate":0.02085888385772705,"torch":0.02032250165939331},{"row":"tv4_007:q0","gate":0.019733965396881104,"torch":0.025035321712493896},{"row":"semif_8e8c3804a3c15ebb31e3:q0","gate":0.01799994707107544,"torch":0.017290115356445312},{"row":"semif_f756e1cb04e11766341e:q0","gate":0.01778966188430786,"torch":0.0195961594581604},{"row":"tv4x_qnli_19:q0","gate":0.016363978385925293,"torch":0.015782535076141357},{"row":"semif_33d6e5da58f0fee2490d:q0","gate":0.014631807804107666,"torch":0.01576271653175354},{"row":"tv4_045:q0","gate":0.01393580436706543,"torch":0.01375359296798706},{"row":"semif_32f36cd0fbbfb3604586:q0","gate":0.013280987739562988,"torch":0.010137081146240234},{"row":"tv4s_01:q0","gate":0.012888491153717041,"torch":0.01088804006576538},{"row":"semif_cdf7f428857e1b841cb3:q0","gate":0.012779414653778076,"torch":0.007764458656311035},{"row":"tv4_058:q0","gate":0.012551069259643555,"torch":0.009904086589813232},{"row":"own_t06:q0","gate":0.012297570705413818,"torch":0.011173248291015625},{"row":"semif_ea7ec66a5e8eac886548:q0","gate":0.012225091457366943,"torch":0.010933995246887207},{"row":"own_m01:q2","gate":0.012172996997833252,"torch":0.011166274547576904},{"row":"semif_a22dab8ba073ff6cdb8d:q0","gate":0.012049198150634766,"torch":0.019027352333068848},{"row":"semif_42384af9ae56beb37ff5:q0","gate":0.011975705623626709,"torch":0.00863867998123169},{"row":"own_j02:q1","gate":0.005702078342437744,"torch":0.007485777139663696},{"row":"own_j04:q1","gate":0.00011241436004638672,"torch":4.655122756958008e-05},{"row":"own_L02:q3","gate":8.698366582393646e-05,"torch":0.00017404556274414062},{"row":"own_t04:q2","gate":0.0017088651657104492,"torch":0.0007988251745700836},{"row":"semif_e0c140e9222d1bce5647:q0","gate":6.195297464728355e-05,"torch":8.291006088256836e-05},{"row":"semif_3226202ba1ada2570e9f:q0","gate":0.0034356117248535156,"torch":0.0023995041847229004},{"row":"semif_a0128e551d095f3449f8:q0","gate":0.0005913972854614258,"torch":0.00019553117454051971},{"row":"semif_6db17cc22bd656d78558:q0","gate":0.0032169222831726074,"torch":0.0013602972030639648},{"row":"semif_a9230edfc6ec9e524af0:q0","gate":9.232759475708008e-05,"torch":1.9947532564401627e-05},{"row":"semif_4089c199b8b796ec3ed8:q0","gate":0.0024436116218566895,"torch":0.005082130432128906},{"row":"tv4_008:q0","gate":0.00801151990890503,"torch":0.00774693489074707},{"row":"tv4_028:q0","gate":0.010508567094802856,"torch":0.01022648811340332},{"row":"tv4s_05:q0","gate":0.00735783576965332,"torch":0.005208015441894531},{"row":"tv4s_14:q0","gate":0.0067758262157440186,"torch":0.008822783827781677},{"row":"tv4x_emotion_13:q0","gate":0.006184518337249756,"torch":0.008360326290130615},{"row":"tv4x_qnli_00:q0","gate":7.757917046546936e-06,"torch":0.00010109599679708481},{"row":"tv4x_paws_12:q0","gate":0.005915597081184387,"torch":0.0036816000938415527},{"row":"tv4x_sciq_03:q0","gate":0.0006895288825035095,"torch":0.00041506439447402954},{"row":"tv4x_composition_holdout_01:q0","gate":0.0003303922712802887,"torch":0.00036722421646118164},{"row":"tv4x_composition_holdout_19:q0","gate":0.0005483627319335938,"torch":0.0024015307426452637}]},"instrument_floor":{"worst_max_abs_dp":4.0531158447265625e-06,"bit_equal_p1_rows":40,"rows":40},"chosen":["set_aff32__K_qkv_out_v","set_aff32__K_qkv_out"],"stage":"chosen","ranking":[{"rank_by_mean":1,"config":"exact","canonical":"set_fp32","fp16_fraction":1.0,"within_cap":false,"est_main_mlirb_bytes":8414007636,"worst_max_abs_dp":4.0531158447265625e-06,"mean_of_row_mean_abs_dp":7.05648223326231e-07,"meets_target":true},{"rank_by_mean":2,"config":"embed_int8_only","canonical":"set_fp32__E","fp16_fraction":1.0,"within_cap":false,"est_main_mlirb_bytes":7818039636,"worst_max_abs_dp":0.005160510540008545,"mean_of_row_mean_abs_dp":0.0005552305745853421,"meets_target":true},{"rank_by_mean":3,"config":"set_aff32__K_qkv_out_v__E","canonical":"set_aff32__K_qkv_out_v__E","fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5286752596,"worst_max_abs_dp":0.015426576137542725,"mean_of_row_mean_abs_dp":0.0018798819457879292,"meets_target":false},{"rank_by_mean":4,"config":"set_aff32__K_qkv_out_v","canonical":"set_aff32__K_qkv_out_v","fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5882720596,"worst_max_abs_dp":0.01644355058670044,"mean_of_row_mean_abs_dp":0.0019123301824826439,"meets_target":false},{"rank_by_mean":5,"config":"set_aff32__K_qkv_out__E","canonical":"set_aff32__K_qkv_out__E","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5267747156,"worst_max_abs_dp":0.015333414077758789,"mean_of_row_mean_abs_dp":0.002010115250062275,"meets_target":false},{"rank_by_mean":6,"config":"set_aff32__K_qkv_out","canonical":"set_aff32__K_qkv_out","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5863715156,"worst_max_abs_dp":0.016337275505065918,"mean_of_row_mean_abs_dp":0.0020814303058432414,"meets_target":false},{"rank_by_mean":7,"config":"set_b32__G_0_1","canonical":"set_b32__G_0_1","fp16_fraction":0.25,"within_cap":true,"est_main_mlirb_bytes":5904490836,"worst_max_abs_dp":0.018460214138031006,"mean_of_row_mean_abs_dp":0.0024146552666691906,"meets_target":false},{"rank_by_mean":8,"config":"set_aff32__G_0","canonical":"set_aff32__G_0","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5583830356,"worst_max_abs_dp":0.024686098098754883,"mean_of_row_mean_abs_dp":0.002719062058798348,"meets_target":false},{"rank_by_mean":9,"config":"set_aff32__K_out","canonical":"set_aff32__K_out","fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5407584596,"worst_max_abs_dp":0.035969555377960205,"mean_of_row_mean_abs_dp":0.002786534290741353,"meets_target":false},{"rank_by_mean":10,"config":"set_aff32__G_0__K_out","canonical":"set_aff32__G_0__K_out","fp16_fraction":0.18669665809768637,"within_cap":true,"est_main_mlirb_bytes":5783387476,"worst_max_abs_dp":0.038170695304870605,"mean_of_row_mean_abs_dp":0.0028898118715248227,"meets_target":false},{"rank_by_mean":11,"config":"all_int8_affine_b32","canonical":"set_aff32","fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5179519316,"worst_max_abs_dp":0.032596468925476074,"mean_of_row_mean_abs_dp":0.0030056005895797476,"meets_target":false},{"rank_by_mean":12,"config":"set_b32__K_qkv_a_b_out_v","canonical":"set_b32__K_qkv_a_b_out_v","fp16_fraction":0.2185089974293059,"within_cap":true,"est_main_mlirb_bytes":5799121236,"worst_max_abs_dp":0.017477869987487793,"mean_of_row_mean_abs_dp":0.0036948911649233194,"meets_target":false},{"rank_by_mean":13,"config":"set_b32__K_qkv_b_out_v","canonical":"set_b32__K_qkv_b_out_v","fp16_fraction":0.21795813441057657,"within_cap":true,"est_main_mlirb_bytes":5797278036,"worst_max_abs_dp":0.019581615924835205,"mean_of_row_mean_abs_dp":0.0037199081680349385,"meets_target":false},{"rank_by_mean":14,"config":"set_b32__K_qkv_out_v","canonical":"set_b32__K_qkv_out_v","fp16_fraction":0.21740727139184723,"within_cap":true,"est_main_mlirb_bytes":5795434836,"worst_max_abs_dp":0.02115654945373535,"mean_of_row_mean_abs_dp":0.0038814362924798236,"meets_target":false},{"rank_by_mean":15,"config":"set_b32__K_qkv_out","canonical":"set_b32__K_qkv_out","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.018633544445037842,"mean_of_row_mean_abs_dp":0.003931028543884167,"meets_target":false},{"rank_by_mean":16,"config":"layers_0-3_fp16","canonical":"set_b32__G_0","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.02719932794570923,"mean_of_row_mean_abs_dp":0.0042045773240776425,"meets_target":false},{"rank_by_mean":17,"config":"all_int8_b16","canonical":"set_b16","fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5291053396,"worst_max_abs_dp":0.04704171419143677,"mean_of_row_mean_abs_dp":0.004532722396979807,"meets_target":false},{"rank_by_mean":18,"config":"proj_out_fp16","canonical":"set_b32__K_out","fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5303914836,"worst_max_abs_dp":0.024914264678955078,"mean_of_row_mean_abs_dp":0.004831143981573405,"meets_target":false},{"rank_by_mean":19,"config":"layers_4-7_fp16","canonical":"set_b32__G_1","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.04578268527984619,"mean_of_row_mean_abs_dp":0.004949301713398501,"meets_target":false},{"rank_by_mean":20,"config":"proj_qkv_fp16","canonical":"set_b32__K_qkv","fp16_fraction":0.14102093279471173,"within_cap":true,"est_main_mlirb_bytes":5539844436,"worst_max_abs_dp":0.058991432189941406,"mean_of_row_mean_abs_dp":0.005999668049286508,"meets_target":false},{"rank_by_mean":21,"config":"proj_up_fp16","canonical":"set_b32__K_up","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.07001703977584839,"mean_of_row_mean_abs_dp":0.006040579349113008,"meets_target":false},{"rank_by_mean":22,"config":"proj_down_fp16","canonical":"set_b32__K_down","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.07015883922576904,"mean_of_row_mean_abs_dp":0.006056326911154125,"meets_target":false},{"rank_by_mean":23,"config":"layers_8-11_fp16","canonical":"set_b32__G_2","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.06477582454681396,"mean_of_row_mean_abs_dp":0.006092905547508659,"meets_target":false},{"rank_by_mean":24,"config":"proj_gate_fp16","canonical":"set_b32__K_gate","fp16_fraction":0.21153139919206757,"within_cap":true,"est_main_mlirb_bytes":5775774036,"worst_max_abs_dp":0.05993455648422241,"mean_of_row_mean_abs_dp":0.006121997949376236,"meets_target":false},{"rank_by_mean":25,"config":"layers_12-15_fp16","canonical":"set_b32__G_3","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.0745232105255127,"mean_of_row_mean_abs_dp":0.006196226572210435,"meets_target":false},{"rank_by_mean":26,"config":"proj_z_fp16","canonical":"set_b32__K_z","fp16_fraction":0.07051046639735586,"within_cap":true,"est_main_mlirb_bytes":5303914836,"worst_max_abs_dp":0.07238650321960449,"mean_of_row_mean_abs_dp":0.006343265754743091,"meets_target":false},{"rank_by_mean":27,"config":"layers_16-19_fp16","canonical":"set_b32__G_4","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.0730968713760376,"mean_of_row_mean_abs_dp":0.006627808520085335,"meets_target":false},{"rank_by_mean":28,"config":"proj_v_fp16","canonical":"set_b32__K_v","fp16_fraction":0.0058758721997796545,"within_cap":true,"est_main_mlirb_bytes":5087646036,"worst_max_abs_dp":0.0703086256980896,"mean_of_row_mean_abs_dp":0.006661105860354534,"meets_target":false},{"rank_by_mean":29,"config":"proj_q_fp16","canonical":"set_b32__K_q","fp16_fraction":0.047006977598237236,"within_cap":true,"est_main_mlirb_bytes":5225271636,"worst_max_abs_dp":0.06540495157241821,"mean_of_row_mean_abs_dp":0.006704179285491894,"meets_target":false},{"rank_by_mean":30,"config":"proj_b_fp16","canonical":"set_b32__K_b","fp16_fraction":0.0005508630187293427,"within_cap":true,"est_main_mlirb_bytes":5069828436,"worst_max_abs_dp":0.06486314535140991,"mean_of_row_mean_abs_dp":0.0067299234731278065,"meets_target":false},{"rank_by_mean":31,"config":"proj_a_fp16","canonical":"set_b32__K_a","fp16_fraction":0.0005508630187293427,"within_cap":true,"est_main_mlirb_bytes":5069828436,"worst_max_abs_dp":0.06977176666259766,"mean_of_row_mean_abs_dp":0.006796375088015338,"meets_target":false},{"rank_by_mean":32,"config":"layers_20-23_fp16","canonical":"set_b32__G_5","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07091116905212402,"mean_of_row_mean_abs_dp":0.006808550176307715,"meets_target":false},{"rank_by_mean":33,"config":"proj_k_fp16","canonical":"set_b32__K_k","fp16_fraction":0.0058758721997796545,"within_cap":true,"est_main_mlirb_bytes":5087646036,"worst_max_abs_dp":0.07084512710571289,"mean_of_row_mean_abs_dp":0.0068797385679014635,"meets_target":false},{"rank_by_mean":34,"config":"all_int8_b32","canonical":"set_b32","fp16_fraction":0.0,"within_cap":true,"est_main_mlirb_bytes":5067985236,"worst_max_abs_dp":0.07101064920425415,"mean_of_row_mean_abs_dp":0.006890123262807417,"meets_target":false},{"rank_by_mean":35,"config":"layers_24-27_fp16","canonical":"set_b32__G_6","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07066327333450317,"mean_of_row_mean_abs_dp":0.006893168229192573,"meets_target":false},{"rank_by_mean":36,"config":"layers_28-31_fp16","canonical":"set_b32__G_7","fp16_fraction":0.125,"within_cap":true,"est_main_mlirb_bytes":5486238036,"worst_max_abs_dp":0.07120299339294434,"mean_of_row_mean_abs_dp":0.006918687499128282,"meets_target":false},{"rank_by_mean":37,"config":"proj_o_fp16","canonical":"set_b32__K_o","fp16_fraction":0.023503488799118618,"within_cap":true,"est_main_mlirb_bytes":5146628436,"worst_max_abs_dp":0.08551710844039917,"mean_of_row_mean_abs_dp":0.00697625177000494,"meets_target":false}]},"quant_api":{"b32_swc":{"quantize_ok":true,"error":null,"export_ok":true},"b16_swc":{"quantize_ok":true,"error":null,"export_ok":true},"b32_asym":{"quantize_ok":true,"error":null,"export_ok":true},"b32_asym_minval":{"quantize_ok":true,"error":null,"export_ok":true},"b32_sym":{"quantize_ok":true,"error":null,"export_ok":true},"affine_name":{"quantize_ok":false,"error":"ValidationError: 1 validation error for QuantizerConfig\nglobal_config.op_state_spec.weight.qscheme\n  Input should be 'symmetric', 'asymmetric' or 'symmetric_with_clipping' [type=enum, input_value='affine', input_type=str]\n    For further information visit https://errors.pydantic.dev/2.14/v/enum","export_ok":null},"embed_b32_swc":{"quantize_ok":true,"error":null,"export_ok":true},"embed_only_b32_swc":{"quantize_ok":true,"error":null,"export_ok":true}},"ios_aot_bytes_file":"$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_bytes.json","efr_abba":{"file":"$ZOO_WORK_ROOT/_kev/results/r6_abba_efr_summary.json","arms":{"leg1_efr":{"load_seconds":1.7792652080242988,"load_split":{"model":0.008001416019396856,"main":1.771263792004902},"first_call_ms":2376.480208011344,"first_seen_length_ms":[[16,2376.5],[32,67.7],[48,69.2],[64,66.5],[80,71.1],[96,69.0],[112,73.0],[128,70.4],[144,71.3],[160,71.1]],"first_seen_q":{"n":10,"median":70.74372949136887,"p90":303.3797827170921,"p10":67.57404561212752,"max":2376.480208011344,"min":66.51283398969099},"repeat_length_ms":{"n":199,"median":70.46970899682492,"p90":73.2142084161751,"p10":65.34360040677711,"max":151.51116700144485,"min":58.43999999342486},"rows":33,"max_abs_dp_vs_oracle":0.005605041980743408,"mean_question_max_abs_dp":0.0014130505592082486,"hidden_bit_equal_vs_noefr_rows":0,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.2196338750072755},"started":1791030351.016547,"finished":1791030370.493758,"full_s80":{"result":"PASS","rows":130,"max_abs_dp":0.008689403533935547,"mean":0.0007635142046837955,"argmax_non_near_tie":[124,124],"warm_ms":{"n":2127,"median":68.77533299848437,"p10":46.96445839945227,"p90":71.86866680858657},"first_call_of_process_ms":[2376.480208011344,1758.9019580045715,1758.133707975503,2074.297499988461],"load_seconds":[1.7792652080242988,1.802317416004371,1.8056957079970744,1.803747250000015]}},"leg2_noefr":{"load_seconds":22.48512195798685,"load_split":{"model":12.302854540990666,"main":10.182267416996183},"first_call_ms":9638.490458019078,"first_seen_length_ms":[[16,9638.5],[32,12250.8],[48,12874.7],[64,13347.0],[80,13555.8],[96,13585.3],[112,13583.8],[128,14552.1],[144,14399.3],[160,14190.4]],"first_seen_q":{"n":10,"median":13569.828375504585,"p90":14414.581491710851,"p10":11989.529933597078,"max":14552.05416699755,"min":9638.490458019078},"repeat_length_ms":{"n":199,"median":46.42458297894336,"p90":47.42300899815746,"p10":45.91371718561277,"max":50.069874996552244,"min":45.265583001310006},"rows":33,"max_abs_dp_vs_oracle":0.0054892897605896,"mean_question_max_abs_dp":0.001396150772713802,"hidden_bit_equal_vs_noefr_rows":33,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.15115799999330193},"started":1791030518.2502851,"finished":1791030682.550563},"leg4_efr":{"load_seconds":13.625012750009773,"load_split":{"model":0.018970250006532297,"main":13.606042500003241},"first_call_ms":17102.8002089879,"first_seen_length_ms":[[16,17102.8],[32,68.9],[48,69.2],[64,60.3],[80,48.8],[96,70.8],[112,69.7],[128,69.5],[144,71.9],[160,70.8]],"first_seen_q":{"n":10,"median":69.61118748586159,"p90":1774.9742330954116,"p10":59.18538779951632,"max":17102.8002089879,"min":48.823875025846064},"repeat_length_ms":{"n":199,"median":70.26249999762513,"p90":72.72521599661559,"p10":57.88814200204797,"max":76.33533299667761,"min":47.81383302179165},"rows":33,"max_abs_dp_vs_oracle":0.005605041980743408,"mean_question_max_abs_dp":0.0014130505592082486,"hidden_bit_equal_vs_noefr_rows":0,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.2156941670109518},"started":1791030994.798642,"finished":1791031040.5523489,"full_s80":{"result":"PASS","rows":130,"max_abs_dp":0.008689403533935547,"mean":0.0007635142046837955,"argmax_non_near_tie":[124,124],"warm_ms":{"n":2127,"median":133.58391600195318,"p10":53.61095841508359,"p90":161.3509504008107},"first_call_of_process_ms":[17102.8002089879,5365.817957994295,2281.9418750004843,4485.588624986121],"load_seconds":[13.625012750009773,8.929044417018304,6.876254959002836,3.87619308399735]}}},"efr_vs_noefr_probs":null}},"ios_aot_bytes":{"schema":"kev-r6-ios-aot-bytes/1","what":"coreai-build compile --platform iOS (compile only; never loaded on the Mac); bytes against 6.4e9 are arithmetic only","rows":[{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_fp16_pf16/kev_0_8b_decode_fp16_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":2.7169347919989377,"main_mlirb_bytes":1506481909,"aimodel_bytes":1506482046,"asset_bytes":1506400620,"asset_tree_sha256":"c2731425c21fc905070ad65ac4222ab029e144e6e64a47ce30ad89083918bb3c","resources_bin_bytes":1504781908,"bytes_minus_aimodel":-81426,"bytes_over_6_4e9":0.235375096875,"stats_storage_types":[{"name":"Float16","count":752339206},{"count":53861,"name":"Float32"},{"count":747,"name":"Int32"},{"count":146,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_fp16_pf16/kev_0_8b_decode_fp16_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":8.540033166005742,"main_mlirb_bytes":1506481909,"aimodel_bytes":1506482046,"asset_bytes":2505674625,"asset_tree_sha256":"a9e593ea6e1bd7b0000826d63216ce8a323a04b7bd92f57b36d8f1e2c7caab76","resources_bin_bytes":2500897708,"bytes_minus_aimodel":999192579,"bytes_over_6_4e9":0.39151166015625,"stats_storage_types":[{"name":"Float16","count":752339206},{"name":"Float32","count":53861},{"name":"Int32","count":747},{"name":"UInt32","count":146},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_fp16_pf16/kev_4b_decode_fp16_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":3.333297290984774,"main_mlirb_bytes":8414007636,"aimodel_bytes":8414007773,"asset_bytes":8413701860,"asset_tree_sha256":"b795a314f8083c751ab071fea4c3fd9e69168d483842a816d6bdd96eac0bf535","resources_bin_bytes":8411495596,"bytes_minus_aimodel":-305913,"bytes_over_6_4e9":1.314640915625,"stats_storage_types":[{"count":4205579270,"name":"Float16"},{"count":172069,"name":"Float32"},{"count":916,"name":"Int32"},{"name":"UInt32","count":156},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_fp16_pf16/kev_4b_decode_fp16_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":27.409209374978673,"main_mlirb_bytes":8414007636,"aimodel_bytes":8414007773,"asset_bytes":15557759365,"asset_tree_sha256":"86d9b615aca9f33733abc275574459870b8b8574065c30980fed9e02e3296848","resources_bin_bytes":15551251468,"bytes_minus_aimodel":7143751592,"bytes_over_6_4e9":2.43089990078125,"stats_storage_types":[{"count":4205579270,"name":"Float16"},{"count":172069,"name":"Float32"},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":6.37895687497803,"main_mlirb_bytes":5783501557,"aimodel_bytes":5783501694,"asset_bytes":5783276818,"asset_tree_sha256":"4cd081b69d58d8fb7a46299aa4c19256e9abe8a32c60fc9a9105a713ae975867","resources_bin_bytes":5780877004,"bytes_minus_aimodel":-224876,"bytes_over_6_4e9":0.9036370028125,"stats_storage_types":[{"name":"Int8","count":2993464320},{"name":"Float16","count":1394192389},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":149.83634054099093,"main_mlirb_bytes":5783501557,"aimodel_bytes":5783501694,"asset_bytes":12927340923,"asset_tree_sha256":"668c80f78b894fdf077d5fa224532ba80d6b1e71b3e3259340b14b0fda5cf820","resources_bin_bytes":12920632876,"bytes_minus_aimodel":7143839229,"bytes_over_6_4e9":2.01989701921875,"stats_storage_types":[{"name":"Int8","count":2993464320},{"count":1394192389,"name":"Float16"},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"name":"UInt32","count":156},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":4.44286229199497,"main_mlirb_bytes":5267864084,"aimodel_bytes":5267864221,"asset_bytes":5267639984,"asset_tree_sha256":"8d77ba5a6d8313ae04cecd4fb99b50785b50850ea0194542b75559ab93f3b879","resources_bin_bytes":5265236724,"bytes_minus_aimodel":-224237,"bytes_over_6_4e9":0.8230687475,"stats_storage_types":[{"name":"Int8","count":3557621760},{"count":883436549,"name":"Float16"},{"name":"Float32","count":172069},{"count":916,"name":"Int32"},{"name":"UInt32","count":156},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":133.41884025000036,"main_mlirb_bytes":5267864084,"aimodel_bytes":5267864221,"asset_bytes":12411704850,"asset_tree_sha256":"a9bdc64cb5d06097162704fa6e918fb7346a3a83e56aefa67ad6671247da1bef","resources_bin_bytes":12404992596,"bytes_minus_aimodel":7143840629,"bytes_over_6_4e9":1.9393288828125,"stats_storage_types":[{"name":"Int8","count":3557621760},{"name":"Float16","count":883436549},{"count":172069,"name":"Float32"},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":4.338819541997509,"main_mlirb_bytes":5863831482,"aimodel_bytes":5863831619,"asset_bytes":5863607354,"asset_tree_sha256":"886f453d94f5d2554d5bd9201c5f0701802504dedaee1c518ae172f3e5e741a6","resources_bin_bytes":5861204716,"bytes_minus_aimodel":-224265,"bytes_over_6_4e9":0.9161886490625,"stats_storage_types":[{"count":2902056960,"name":"Int8"},{"count":1479732229,"name":"Float16"},{"name":"Float32","count":172069},{"count":916,"name":"Int32"},{"count":156,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":140.0149582499871,"main_mlirb_bytes":5863831482,"aimodel_bytes":5863831619,"asset_bytes":13007670689,"asset_tree_sha256":"13f93a88cabc888429405a99100dafd44a500444c763930fae3057ffaf863ab0","resources_bin_bytes":13000960588,"bytes_minus_aimodel":7143839070,"bytes_over_6_4e9":2.03244854515625,"stats_storage_types":[{"count":2902056960,"name":"Int8"},{"name":"Float16","count":1479732229},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":3.49811674997909,"main_mlirb_bytes":5286864947,"aimodel_bytes":5286865084,"asset_bytes":5286639669,"asset_tree_sha256":"b48bc68de67677bdeafd7a3ccdbf44ce9d21667272bbf4b2729b176c771a3d7e","resources_bin_bytes":5284242100,"bytes_minus_aimodel":-225415,"bytes_over_6_4e9":0.82603744828125,"stats_storage_types":[{"name":"Int8","count":3535994880},{"count":903752709,"name":"Float16"},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":131.2038212500047,"main_mlirb_bytes":5286864947,"aimodel_bytes":5286865084,"asset_bytes":12430704328,"asset_tree_sha256":"0ba4a2966a1611bd5599c7957805925b56c33f8d6531bf79b981a2479061c778","resources_bin_bytes":12423997972,"bytes_minus_aimodel":7143839244,"bytes_over_6_4e9":1.94229755125,"stats_storage_types":[{"count":3535994880,"name":"Int8"},{"count":903752709,"name":"Float16"},{"count":172069,"name":"Float32"},{"name":"Int32","count":916},{"name":"UInt32","count":156},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.aimodel","platform":"iOS","arch":"h19p","efr":false,"returncode":0,"seconds":4.982264542020857,"main_mlirb_bytes":5882832315,"aimodel_bytes":5882832452,"asset_bytes":5882606928,"asset_tree_sha256":"dbd9e60eeb572520472517782ebe08d8f3ec9eaa6e78f6472688ab861c1e489e","resources_bin_bytes":5880210092,"bytes_minus_aimodel":-225524,"bytes_over_6_4e9":0.9191573325,"stats_storage_types":[{"name":"Int8","count":2880430080},{"name":"Float16","count":1500048389},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"name":"UInt32","count":156},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.aimodel","platform":"iOS","arch":"h19p","efr":true,"returncode":0,"seconds":188.71786787500605,"main_mlirb_bytes":5882832315,"aimodel_bytes":5882832452,"asset_bytes":13026670589,"asset_tree_sha256":"57b66d9e02c46b5aab8f1b5c4a8093c2582b3ba01f973d4636f2eeb449e41ee0","resources_bin_bytes":13019965964,"bytes_minus_aimodel":7143838137,"bytes_over_6_4e9":2.03541727953125,"stats_storage_types":[{"name":"Int8","count":2880430080},{"count":1500048389,"name":"Float16"},{"name":"Float32","count":172069},{"count":916,"name":"Int32"},{"name":"UInt32","count":156},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_fp16_pf16/kev_4b_decode_fp16_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":6.48417262500152,"main_mlirb_bytes":8414007636,"aimodel_bytes":8414007773,"asset_bytes":8413701860,"asset_tree_sha256":"c63f546b71345703ec9b4d35d5127152dd4fa93ff0ac2980e2cdb9e04714784b","resources_bin_bytes":8411495596,"bytes_minus_aimodel":-305913,"bytes_over_6_4e9":1.314640915625,"stats_storage_types":[{"count":4205579270,"name":"Float16"},{"count":172069,"name":"Float32"},{"count":916,"name":"Int32"},{"name":"UInt32","count":156},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_g0_k_out_pf16/kev_4b_decode_int8mix_aff32_g0_k_out_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":5.08118441698025,"main_mlirb_bytes":5783501557,"aimodel_bytes":5783501694,"asset_bytes":5783276818,"asset_tree_sha256":"620ef7e7cb01a9eec4be00ff9af60f83bce017e3c46b2ca1bf05131e7713fa3e","resources_bin_bytes":5780877004,"bytes_minus_aimodel":-224876,"bytes_over_6_4e9":0.9036370028125,"stats_storage_types":[{"count":2993464320,"name":"Int8"},{"count":1394192389,"name":"Float16"},{"count":172069,"name":"Float32"},{"count":916,"name":"Int32"},{"count":156,"name":"UInt32"},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":4.326805332995718,"main_mlirb_bytes":5267864084,"aimodel_bytes":5267864221,"asset_bytes":5267639984,"asset_tree_sha256":"64980da9cd4bfc721db704e8358d4f6a9c7d9d6a8fdf6eb0960a2777826d0b4f","resources_bin_bytes":5265236724,"bytes_minus_aimodel":-224237,"bytes_over_6_4e9":0.8230687475,"stats_storage_types":[{"name":"Int8","count":3557621760},{"count":883436549,"name":"Float16"},{"count":172069,"name":"Float32"},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":3.822517583001172,"main_mlirb_bytes":5863831482,"aimodel_bytes":5863831619,"asset_bytes":5863607354,"asset_tree_sha256":"1012cb914ca2944f78a13cf5e46e0ca36a15b38afff7e2c1cc7358d85723ca68","resources_bin_bytes":5861204716,"bytes_minus_aimodel":-224265,"bytes_over_6_4e9":0.9161886490625,"stats_storage_types":[{"name":"Int8","count":2902056960},{"count":1479732229,"name":"Float16"},{"name":"Float32","count":172069},{"count":916,"name":"Int32"},{"name":"UInt32","count":156},{"count":1,"name":"UInt64"}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":3.6468205000273883,"main_mlirb_bytes":5286864947,"aimodel_bytes":5286865084,"asset_bytes":5286639669,"asset_tree_sha256":"aa1827eb0556b3a259bb9189c9263673ed9c0088b454bca9dffdd3c00994f5cf","resources_bin_bytes":5284242100,"bytes_minus_aimodel":-225415,"bytes_over_6_4e9":0.82603744828125,"stats_storage_types":[{"count":3535994880,"name":"Int8"},{"count":903752709,"name":"Float16"},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"name":"UInt64","count":1}]},{"aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16/kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16.aimodel","platform":"macOS","arch":"h16c","efr":false,"returncode":0,"seconds":6.428049041016493,"main_mlirb_bytes":5882832315,"aimodel_bytes":5882832452,"asset_bytes":5882606928,"asset_tree_sha256":"2030cef23fed9db583dac0cdbb4747acc0012b4268b4fad862e538f676020466","resources_bin_bytes":5880210092,"bytes_minus_aimodel":-225524,"bytes_over_6_4e9":0.9191573325,"stats_storage_types":[{"name":"Int8","count":2880430080},{"name":"Float16","count":1500048389},{"name":"Float32","count":172069},{"name":"Int32","count":916},{"count":156,"name":"UInt32"},{"name":"UInt64","count":1}]}],"records":["$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_0_8b_decode_fp16_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_0_8b_decode_fp16_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_fp16_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_fp16_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_g0_k_out_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_g0_k_out_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16_h19p_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_ios_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16_h19p.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_fp16_pf16_h16c_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_int8mix_aff32_g0_k_out_pf16_h16c_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_e_pf16_h16c_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_pf16_h16c_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_e_pf16_h16c_noefr.json","$ZOO_WORK_ROOT/_kev/results/r6_mac_aot_kev_4b_decode_int8mix_aff32_k_qkv_out_v_pf16_h16c_noefr.json"],"generated_at":"2026-10-03T23:44:47+09:00"},"efr_abba":{"schema":"kev-r6-abba-efr/2","rows":"s80 shard_00 (33 rows, position lengths 16..160): the only shard the no-efr leg finished before it was stopped","arms":{"leg1_efr":{"load_seconds":1.7792652080242988,"load_split":{"model":0.008001416019396856,"main":1.771263792004902},"first_call_ms":2376.480208011344,"first_seen_length_ms":[[16,2376.5],[32,67.7],[48,69.2],[64,66.5],[80,71.1],[96,69.0],[112,73.0],[128,70.4],[144,71.3],[160,71.1]],"first_seen_q":{"n":10,"median":70.74372949136887,"p90":303.3797827170921,"p10":67.57404561212752,"max":2376.480208011344,"min":66.51283398969099},"repeat_length_ms":{"n":199,"median":70.46970899682492,"p90":73.2142084161751,"p10":65.34360040677711,"max":151.51116700144485,"min":58.43999999342486},"rows":33,"max_abs_dp_vs_oracle":0.005605041980743408,"mean_question_max_abs_dp":0.0014130505592082486,"hidden_bit_equal_vs_noefr_rows":0,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.2196338750072755},"started":1791030351.016547,"finished":1791030370.493758,"full_s80":{"result":"PASS","rows":130,"max_abs_dp":0.008689403533935547,"mean":0.0007635142046837955,"argmax_non_near_tie":[124,124],"warm_ms":{"n":2127,"median":68.77533299848437,"p10":46.96445839945227,"p90":71.86866680858657},"first_call_of_process_ms":[2376.480208011344,1758.9019580045715,1758.133707975503,2074.297499988461],"load_seconds":[1.7792652080242988,1.802317416004371,1.8056957079970744,1.803747250000015]}},"leg2_noefr":{"load_seconds":22.48512195798685,"load_split":{"model":12.302854540990666,"main":10.182267416996183},"first_call_ms":9638.490458019078,"first_seen_length_ms":[[16,9638.5],[32,12250.8],[48,12874.7],[64,13347.0],[80,13555.8],[96,13585.3],[112,13583.8],[128,14552.1],[144,14399.3],[160,14190.4]],"first_seen_q":{"n":10,"median":13569.828375504585,"p90":14414.581491710851,"p10":11989.529933597078,"max":14552.05416699755,"min":9638.490458019078},"repeat_length_ms":{"n":199,"median":46.42458297894336,"p90":47.42300899815746,"p10":45.91371718561277,"max":50.069874996552244,"min":45.265583001310006},"rows":33,"max_abs_dp_vs_oracle":0.0054892897605896,"mean_question_max_abs_dp":0.001396150772713802,"hidden_bit_equal_vs_noefr_rows":33,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.15115799999330193},"started":1791030518.2502851,"finished":1791030682.550563},"leg4_efr":{"load_seconds":13.625012750009773,"load_split":{"model":0.018970250006532297,"main":13.606042500003241},"first_call_ms":17102.8002089879,"first_seen_length_ms":[[16,17102.8],[32,68.9],[48,69.2],[64,60.3],[80,48.8],[96,70.8],[112,69.7],[128,69.5],[144,71.9],[160,70.8]],"first_seen_q":{"n":10,"median":69.61118748586159,"p90":1774.9742330954116,"p10":59.18538779951632,"max":17102.8002089879,"min":48.823875025846064},"repeat_length_ms":{"n":199,"median":70.26249999762513,"p90":72.72521599661559,"p10":57.88814200204797,"max":76.33533299667761,"min":47.81383302179165},"rows":33,"max_abs_dp_vs_oracle":0.005605041980743408,"mean_question_max_abs_dp":0.0014130505592082486,"hidden_bit_equal_vs_noefr_rows":0,"reset":{"run":"tv4x_emotion_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.2156941670109518},"started":1791030994.798642,"finished":1791031040.5523489,"full_s80":{"result":"PASS","rows":130,"max_abs_dp":0.008689403533935547,"mean":0.0007635142046837955,"argmax_non_near_tie":[124,124],"warm_ms":{"n":2127,"median":133.58391600195318,"p10":53.61095841508359,"p90":161.3509504008107},"first_call_of_process_ms":[17102.8002089879,5365.817957994295,2281.9418750004843,4485.588624986121],"load_seconds":[13.625012750009773,8.929044417018304,6.876254959002836,3.87619308399735]}}},"snapshots":[{"label":"leg1_efr_pre","time":"2026-10-03T21:25:50+09:00","coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":2505703424,"c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":7312179200,"f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":15557791744},"mpsgraph_scratch":{"path":"$TMPDIR/com.apple.MetalPerformanceShadersGraph","dirs":1365,"bytes":0,"newest":["mpsgraph-<pid>-2026-10-03_21_23_02-816441501","mpsgraph-<pid>-2026-10-03_21_23_31-83811919","mpsgraph-<pid>-2026-10-03_21_23_55-2610029405","mpsgraph-<pid>-2026-10-03_21_24_33-3151589763","mpsgraph-<pid>-2026-10-03_21_25_02-2661926534"]},"gpu_lock":{"bytes":0,"content":""},"df_free_gib":414.5},{"label":"leg1_efr_post","time":"2026-10-03T21:28:37+09:00","coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":2505703424,"c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":7312179200,"f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":15557791744},"mpsgraph_scratch":{"path":"$TMPDIR/com.apple.MetalPerformanceShadersGraph","dirs":1369,"bytes":0,"newest":["mpsgraph-<pid>-2026-10-03_21_25_02-2661926534","mpsgraph-<pid>-2026-10-03_21_25_51-2482735587","mpsgraph-<pid>-2026-10-03_21_26_10-480584052","mpsgraph-<pid>-2026-10-03_21_26_30-4043145350","mpsgraph-<pid>-2026-10-03_21_27_05-2562882962"]},"gpu_lock":{"bytes":0,"content":""},"df_free_gib":420.7},{"label":"leg2_noefr_pre","time":"2026-10-03T21:28:37+09:00","coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":2505703424,"c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":7312179200,"f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":15557791744},"mpsgraph_scratch":{"path":"$TMPDIR/com.apple.MetalPerformanceShadersGraph","dirs":1369,"bytes":0,"newest":["mpsgraph-<pid>-2026-10-03_21_25_02-2661926534","mpsgraph-<pid>-2026-10-03_21_25_51-2482735587","mpsgraph-<pid>-2026-10-03_21_26_10-480584052","mpsgraph-<pid>-2026-10-03_21_26_30-4043145350","mpsgraph-<pid>-2026-10-03_21_27_05-2562882962"]},"gpu_lock":{"bytes":0,"content":""},"df_free_gib":420.7},{"label":"leg4_efr_pre","time":"2026-10-03T21:36:34+09:00","coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":2505703424,"5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":10653696,"634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":14864384,"86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":12615680,"91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":8413720576,"c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":7312179200,"f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":15557791744,"fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":12615680},"mpsgraph_scratch":{"path":"$TMPDIR/com.apple.MetalPerformanceShadersGraph","dirs":1374,"bytes":25341153280,"newest":["mpsgraph-<pid>-2026-10-03_21_33_56-2910397884","mpsgraph-<pid>-2026-10-03_21_33_56-3084907826","mpsgraph-<pid>-2026-10-03_21_33_57-3523771932","mpsgraph-<pid>-2026-10-03_21_33_57-1640287828","mpsgraph-<pid>-2026-10-03_21_31_22-721066252"]},"gpu_lock":{"bytes":0,"content":""},"df_free_gib":376.6},{"label":"leg4_efr_post","time":"2026-10-03T21:42:29+09:00","coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":2505703424,"5c31cd6cb4b43125c3d025b7f80832a1a3bdb977efe89d3b233568332eba7edc":10653696,"634d3e2b0bbbffe4f234f25f8c98686373a4c02cb170a2d52b9c4cd9d383953c":14864384,"86d022cd11a9dc127c9872a316c3cc3b034f3d6a000a1d7c68403e09860dc8ce":12615680,"91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4":8413720576,"c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":7312179200,"f6283a6dfbe1682b94b5da8edf55988f0a1f277725a7e6229d1149889f187bd4":15557791744,"fd9b43991fcfbc17cf585531dfb8e7e6ed161ee8cba8c3c2017f9cc41d2483b4":12615680},"mpsgraph_scratch":{"path":"$TMPDIR/com.apple.MetalPerformanceShadersGraph","dirs":1381,"bytes":2240512,"newest":["mpsgraph-<pid>-2026-10-03_21_37_35-3045585318","mpsgraph-<pid>-2026-10-03_21_37_48-1631968230","mpsgraph-<pid>-2026-10-03_21_38_39-2522051388","mpsgraph-<pid>-2026-10-03_21_39_00-3428038293","mpsgraph-<pid>-2026-10-03_21_39_51-981673013"]},"gpu_lock":{"bytes":56,"content":"kev timing r5 pid <n> since 2026-10-03T21:41:34+09:00\n"},"df_free_gib":375.6}],"scratch_removed":[{"line":"21:32:42 removed $TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_28_50-1186006713 pid <n> 13747240 KB (from $ZOO_WORK_ROOT/_kev/readout_4b/r6_fp16_4b_s80_2_noefr/shard_00.json)","kb":13747240},{"line":"21:36:53 removed $TMPDIR/com.apple.MetalPerformanceShadersGraph/mpsgraph-<pid>-2026-10-03_21_31_22-721066252 pid <n> 24745032 KB (killed leg-2 worker shard_01, no shard json)","kb":24745032}],"scratch_removed_gb_total":36.70909118652344,"stopped":"leg 2 (no efr) stopped at 21:36 after shard_00: every new position length re-specializes (9.6-14.6 s, ~1.4 GB of MPSGraph scratch each); shard_03 holds 113 lengths (~25 min, ~150 GB); leg 3 not run"},"noefr33_readout":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_4b_decode_fp16_pf16","name":"kev_4b_decode_fp16_pf16","kind":"decision-backbone","compression":null,"prefill_chunk":16,"metadata_sha256":"7618a1dca5711af450081138e303c154609346f8a973436c4e65d77bdee25a9a","main_mlirb":{"bytes":8414007636,"sha256":"e38dc3caaeae1ccb21eebea3148a2aa0010185ae8fcdd7a757bb5a1926b05806"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"3891e840d7dc5fca0af33d3a25083a735e36fe06214e3f707024820cb6b9f89c"},"head_sha256":{"head.safetensors":"36c392f75fa5eff2243b07664e1edb1dc4e771dfc9d145469ba175db54157106","kev_head.json":"5ff071ffaa5b620999f44770a014b4b5b1da2a8225acbde10b412652cfba7416"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc_noefr/kev_4b_decode_fp16_pf16.h16c.aimodelc","bytes":8413701860,"tree_sha256":"c63f546b71345703ec9b4d35d5127152dd4fa93ff0ac2980e2cdb9e04714784b","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"7a795545586a28be938e310cd2880bdd8ea0f882e26fd0684ee77b3c8a186a34","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"f956e830fe83a2a3a892612a5db9234f284ad54e3f62929d46f1b321c820a08f","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/reflection.fb":"7d2ea3b0766b2439f9f57b01c4265a14ea904db2ca6644de59b4bb426973a2f1","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"37e53325f0cdb1b0714bd4af2c65abc24e61daa8342034d32cde1148824dce67","main-h16c.mlirb":"91b53793d559c65a2e6803bf6d6140108186084a85a7c468eedea353780c15d4","main.hash":"80264ae40cdf02dc5d3337534aa47a0d9d7911f0d142afda048729215902fe8f","metadata.json":"58463d202b2e14047e5806c41bee8093df3c937bf1c094f1bedbad7534a98649","stats.json":"856d1e15a5a4f9ddd4b559ec362ac12a0c1d40fafa4e24284a4ed12d3a469857"}}},"chunk":16,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"summary":{"runs":33,"expected_runs":33,"missing_runs":[],"questions":33,"argmax_equal":32,"questions_non_near_tie":30,"argmax_equal_non_near_tie":30,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.0054892897605896,"max_abs_dp_non_near_tie":0.0054892897605896,"mean_of_run_mean_abs_dp":0.0010046793002108432,"mean_of_record_mean_abs_dp":0.0010046793002108432,"mean_question_max_abs_dp":0.001396150772713802,"max_abs_dlogit_T":0.029694080352783203,"worst_run":{"id":"tv4s_04","q":0,"max_abs_dp":0.0054892897605896},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":3010,"padded_tokens":3344,"calls":209,"pad_waste":0.09988038277511961,"decode_ms_total":240831.6257931583,"ms_per_call_median":58.14716700115241,"ms_per_call_warm_median":57.67147950245999},"by_source":{"transfer_v4_dev_score":{"runs":12,"expected_runs":12,"missing_runs":[],"questions":12,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":1,"argmax_equal_near_tie":0,"max_abs_dp":0.0054892897605896,"max_abs_dp_non_near_tie":0.0054892897605896,"mean_of_run_mean_abs_dp":0.0016775523415870138,"mean_of_record_mean_abs_dp":0.0016775523415870138,"mean_question_max_abs_dp":0.0025163311511278152,"max_abs_dlogit_T":0.029694080352783203,"worst_run":{"id":"tv4s_04","q":0,"max_abs_dp":0.0054892897605896},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1186,"padded_tokens":1344,"calls":84,"pad_waste":0.11755952380952381,"decode_ms_total":4937.48337609577,"ms_per_call_median":56.171874995925464,"ms_per_call_warm_median":56.171874995925464},"transfer_v4_dev_sources":{"runs":21,"expected_runs":21,"missing_runs":[],"questions":21,"argmax_equal":21,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":2,"argmax_equal_near_tie":2,"max_abs_dp":0.004446506500244141,"max_abs_dp_non_near_tie":0.004446506500244141,"mean_of_run_mean_abs_dp":0.00062018041942446,"mean_of_record_mean_abs_dp":0.00062018041942446,"mean_question_max_abs_dp":0.000756047699334366,"max_abs_dlogit_T":0.024653911590576172,"worst_run":{"id":"tv4x_paws_00","q":0,"max_abs_dp":0.004446506500244141},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1824,"padded_tokens":2000,"calls":125,"pad_waste":0.088,"decode_ms_total":235894.14241706254,"ms_per_call_median":60.65304100047797,"ms_per_call_warm_median":60.325958009343594}},"near_ties":[{"id":"tv4x_tweet_offensive_01","q":0,"qid":"offensive","oracle_top2_margin":0.002196580171585083,"argmax_equal":true,"max_abs_dp":0.0018206238746643066,"probs":[0.49708113074302673,0.5029188990592957]},{"id":"tv4x_paws_01","q":0,"qid":"paraphrase","oracle_top2_margin":0.007073014974594116,"argmax_equal":true,"max_abs_dp":0.0015058815479278564,"probs":[0.497969388961792,0.5020306706428528]},{"id":"tv4s_11","q":0,"qid":"decision","oracle_top2_margin":0.001054227352142334,"argmax_equal":false,"max_abs_dp":0.0011511147022247314,"probs":[0.010837554931640625,0.4951229691505432,0.49403947591781616]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"PASS","timing":{"contended":true,"S":16,"load_seconds":{"first_process":12.217824749997817,"median":12.217824749997817,"all":[12.217824749997817]},"ms_per_call_warm_runs":{"n":206,"median":57.67147950245999,"p10":50.55131249537226,"p90":78.05037499929313},"ms_per_call_all_runs":{"n":209,"median":58.14716700115241,"p10":50.56034999433905,"p90":81.44185819546696},"ms_per_token_warm_runs":{"n":32,"median":4.105282647126959,"p10":3.7800663771666785,"p90":258.6501651330988},"run_decode_seconds_warm_runs":{"n":32,"median":0.3924499170097988,"p10":0.25219860060315114,"p90":24.324616662212122},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":233857.84938247525}},"seconds":{"gpu_processes":255.67783800000325,"scoring":9.32606950000627,"total":265.10355520801386},"generated_at":"2026-10-03T13:24:57+00:00","note":"round 6 r6: the 33 probe rows, AOT h16c without efr"},"resident_memory":[{"file":"results/r6_mem_aff32_g0_k_out_gate434.jsonl","samples":89,"numeric_fields":{"graphics_dirty":{"n":89,"min":0,"max":280264704},"mapped_file_clean":{"n":88,"min":0,"max":19827703808},"phys_footprint":{"n":88,"min":30622224,"max":987950360},"phys_footprint_peak":{"n":88,"min":30622224,"max":1142385944},"rss_bytes":{"n":89,"min":38928384,"max":20140670976}},"first":{"time":"22:34:52","etime":"00:00"},"last":{"time":"22:39:42","etime":"01:08"}},{"file":"results/r6_mem_aff32_g0_k_out_heldout.jsonl","samples":16,"numeric_fields":{"graphics_dirty":{"n":16,"min":81920,"max":278020096},"mapped_file_clean":{"n":16,"min":445399040,"max":12937396224},"phys_footprint":{"n":16,"min":41534568,"max":805022712},"phys_footprint_peak":{"n":16,"min":41534568,"max":937749592},"rss_bytes":{"n":16,"min":239403008,"max":13454524416}},"first":{"time":"22:41:33","etime":"00:03"},"last":{"time":"22:42:21","etime":"00:13"}},{"file":"results/r6_mem_aff32_g0_k_out_noefr33.jsonl","samples":222,"numeric_fields":{"graphics_dirty":{"n":222,"min":16384,"max":183238656},"mapped_file_clean":{"n":222,"min":1683406848,"max":25558319104},"phys_footprint":{"n":222,"min":51548720,"max":2109439912},"phys_footprint_peak":{"n":222,"min":51548720,"max":2302074016},"rss_bytes":{"n":222,"min":1773338624,"max":27130806272}},"first":{"time":"22:42:43","etime":"00:03"},"last":{"time":"22:54:50","etime":"12:10"}},{"file":"results/r6_mem_aff32_g0_k_out_red.jsonl","samples":19,"numeric_fields":{"graphics_dirty":{"n":19,"min":81920,"max":281133056},"mapped_file_clean":{"n":19,"min":756334592,"max":18780307456},"phys_footprint":{"n":19,"min":48449024,"max":763833264},"phys_footprint_peak":{"n":19,"min":48449024,"max":922610608},"rss_bytes":{"n":19,"min":812793856,"max":18885558272}},"first":{"time":"22:40:10","etime":"00:01"},"last":{"time":"22:41:08","etime":"00:59"}},{"file":"results/r6_mem_aff32_k_qkv_out_gate434.jsonl","samples":226,"numeric_fields":{"graphics_dirty":{"n":226,"min":0,"max":287358976},"mapped_file_clean":{"n":225,"min":1366376448,"max":19890880512},"phys_footprint":{"n":225,"min":48597128,"max":1012149528},"phys_footprint_peak":{"n":225,"min":48597128,"max":1171090712},"rss_bytes":{"n":226,"min":0,"max":19930939392}},"first":{"time":"23:15:24","etime":"00:03"},"last":{"time":"23:27:32","etime":"03:55"}},{"file":"results/r6_mem_aff32_k_qkv_out_v_gate434.jsonl","samples":96,"numeric_fields":{"graphics_dirty":{"n":96,"min":0,"max":281067520},"mapped_file_clean":{"n":96,"min":65536,"max":19377897472},"phys_footprint":{"n":96,"min":32703016,"max":1000139984},"phys_footprint_peak":{"n":96,"min":32703016,"max":1159572688},"rss_bytes":{"n":96,"min":40763392,"max":19084738560}},"first":{"time":"23:06:57","etime":"00:03"},"last":{"time":"23:12:11","etime":"01:10"}},{"file":"results/r6_mem_aff32_k_qkv_out_v_heldout.jsonl","samples":23,"numeric_fields":{"graphics_dirty":{"n":23,"min":81920,"max":282050560},"mapped_file_clean":{"n":23,"min":1997930496,"max":19804520448},"phys_footprint":{"n":23,"min":44389680,"max":845196328},"phys_footprint_peak":{"n":23,"min":44389680,"max":943926144},"rss_bytes":{"n":23,"min":4452892672,"max":20069138432}},"first":{"time":"23:13:43","etime":"00:03"},"last":{"time":"23:14:58","etime":"00:13"}},{"file":"results/r6_mem_aff32_k_qkv_out_v_red.jsonl","samples":13,"numeric_fields":{"graphics_dirty":{"n":13,"min":81920,"max":282345472},"mapped_file_clean":{"n":13,"min":36798464,"max":18663505920},"phys_footprint":{"n":13,"min":47645440,"max":802057016},"phys_footprint_peak":{"n":13,"min":47645440,"max":931179392},"rss_bytes":{"n":13,"min":102547456,"max":18907758592}},"first":{"time":"23:12:39","etime":"00:01"},"last":{"time":"23:13:18","etime":"00:40"}},{"file":"results/r6_mem_fp16_4b_efr_leg4.jsonl","samples":65,"numeric_fields":{"graphics_dirty":{"n":65,"min":81920,"max":286392320},"mapped_file_clean":{"n":65,"min":11751358464,"max":16374857728},"phys_footprint":{"n":65,"min":53979384,"max":1148023264},"phys_footprint_peak":{"n":65,"min":53979384,"max":1149235680},"rss_bytes":{"n":65,"min":11754340352,"max":16631447552}},"first":{"time":"21:38:46","etime":"00:57"},"last":{"time":"21:42:14","etime":"03:15"}},{"file":"results/r6_mem_r6_noefr33.jsonl","samples":4,"numeric_fields":{"graphics_dirty":{"n":4,"min":0,"max":81920},"mapped_file_clean":{"n":4,"min":180224,"max":13688455168},"phys_footprint":{"n":4,"min":33063464,"max":53750152},"phys_footprint_peak":{"n":4,"min":33063464,"max":53750152},"rss_bytes":{"n":4,"min":42942464,"max":13687963648}},"first":{"time":"22:20:32","etime":"00:00"},"last":{"time":"22:20:42","etime":"00:10"}}]}