{"what":"Kev-0.8B: int8 measured, not shipped. int8lin (every decoder linear int8 per block of 32) fails the bar; the fp32 torch bisect with the exporter's own int8 weights under a rule written before any result (bisect/rule.json) chose no set; int8mix (layers 0-11 fp16, outside the rule) passes. Exports, gates, red arms, the bisect table and the summary. Every int8 graph here is the unrolled S = 16 form (the first upload's graph); the fp32 Metal-kernel forms were not measured in int8. Process ids removed in round 17 (no number or result changed).","sources":[{"file":"results/export_int8lin_pf16.json","bytes":7481,"sha256":"8c4a1ffee838d036368a9fb1efc9b0e110df06b57571d2893df815918fd0795b"},{"file":"results/readout_int8lin_pf16.json","bytes":748093,"sha256":"610053646a4571869615d0c9ac584f8ad8d5749c21f82597118ad4d03b3f5730"},{"file":"results/readout_int8lin_pf16_red_v2.json","bytes":15260,"sha256":"e20ab43f0b194d562642ef69bfdce4e50702f93e67fb10efd9e040e9bb8bc578"},{"file":"results/export_int8mix_pf16.json","bytes":12233,"sha256":"d0ed74fb511347b44046de9fad742df1d9d6cff72c9e08b77867f1e670b6f7a9"},{"file":"results/readout_int8mix_pf16.json","bytes":752617,"sha256":"0b4be0782bbacc28e00a10772b9cec13160dae5eb1fbcb9965e65bc58c1f17bb"},{"file":"results/readout_int8mix_pf16_red_v2.json","bytes":19509,"sha256":"4f52cac870950642535a15f16fea2e0e76bfb5d11fe396c1a39653c3cd467b47"},{"file":"bisect/rule.json","bytes":18552,"sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf"},{"file":"results/r3_int8_bisect.json","bytes":1747800,"sha256":"0a33b8f5974dd61ca24d365008818cc212f276e7b88fd01d0a77bed9ab1f34c4"},{"file":"results/r3_int8_summary.json","bytes":35001,"sha256":"a1f743aece6eeceea1700e9bb780a9a8f4dd3952509155442d1cafc82ee3ab62"}],"trimmed":{"removed":["bisect.configs.*.rows (per-row values of each config; each config keeps its worst / mean / argmax and the table row)","runs[] of the non-shipped readouts keep id, q, tokens, calls, finite, source, type, argmax, argmax_oracle, argmax_equal, max / mean |dp|, oracle top-2 margin, near_tie, probs and the hidden rows' min_pos_cos / positions_below (dropped: per-run seconds, shard and index, padded_tokens, max |dlogit|, the hidden-row difference details) (readout_int8lin, readout_int8mix)","runs[].call_ms and timing.first_call_of_process_ms (per-call ms; the timing block keeps the medians)","runs[].npz / npz_key, processes[].npz / npz_sha256, oracle.npz_sha256 (lane file names and hashes of hidden dumps)","runs[].hidden.lowest_positions (each run keeps min_pos_cos, its index and the position counts)","runs[].probs_oracle (the oracle's probabilities are in the fixture file)","processes[].descriptor (identical to the top-level descriptor: descriptor_same_in_every_process)","other_gpu_processes (command lines of other sessions' processes)","red arm items' instr_old / instr_new (transfer-v4 question text; each item keeps its row id and numbers)","every pid key (process ids)"],"serialization":"compact json (no indentation)","full_copies":"the source files above, in the lane directory $ZOO_WORK_ROOT/_kev/ (not in the repository)","paths":"local paths rewritten: the lane directory as $ZOO_WORK_ROOT/_kev, the repository relative, the home directory as ~, temporary directories as $TMPDIR, Xcode as $DEVELOPER_DIR, the phone's app container as <app container>"},"export_int8lin":{"mode":"int8lin","name":"kev_0_8b_decode_int8lin_pf16","hf_id":"kev-local/kev-0.8b-v1.0-merged","weights_sha256":"ab6bd41853ef2a45d56923202e15c8c9c3dcedaec152b1fb8e529f1ec208415f","prefill_chunk":16,"argv":["int8lin","--prefill-chunk","16","--aot","--record","$ZOO_WORK_ROOT/_kev/results/export_int8lin_pf16.json"],"started":"2026-10-03T17:22:58+09:00","script_sha256":"499095cd70ebdd0b9816865606d8de696e83d026ec6fc18200eb7007881f5040","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8lin_pf16","name":"kev_0_8b_decode_int8lin_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8lin_pf16/kev_0_8b_decode_int8lin_pf16.aimodel","load_report":{"checkpoint_files":["model.safetensors"],"checkpoint_keys":320,"module_tensors":320,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"loopfree_unrolled_linear_layers":18,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"fp16_layers":[],"fp16_layer_patterns":[],"int8_linear_modules":186,"fp16_linear_modules":[],"int8_params":497614848,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":12.274526249995688},"seconds":{"load":1.2339160410047043,"quantize":12.27455933400779,"export":46.315567457990255,"optimize":0.29773250001017004,"export_optimize":46.613299958000425,"save":1.106685832986841,"total":61.312744875001954},"du_aimodel":"992M","du_bundle":"1.0G","aimodel_files":{"main.hash":32,"main.mlirb":1040063350,"metadata.json":105},"main_mlirb":{"bytes":1040063350,"sha256":"2281e56d413cbc38cde4bd37953fd5476726a1f37953f940eba9dadcc0bfdb77"},"metadata_sha256":"a35475102eb71c97f7bf73cc39afcba2d4de14c1bb9986c2b1aa7025980d818e","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"disk":{"before_load":{"time":"2026-10-03T17:23:03+09:00","free_bytes":590942416896,"free_gib":550.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_load":{"time":"2026-10-03T17:23:04+09:00","free_bytes":590942351360,"free_gib":550.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_export":{"time":"2026-10-03T17:24:03+09:00","free_bytes":590940483584,"free_gib":550.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_save":{"time":"2026-10-03T17:24:04+09:00","free_bytes":589900394496,"free_gib":549.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8lin_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":24.39232283399906,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8lin_pf16/kev_0_8b_decode_int8lin_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T17:24:04+09:00","free_bytes":589885390848,"free_gib":549.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after":{"time":"2026-10-03T17:24:29+09:00","free_bytes":587880341504,"free_gib":547.5,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}}},"du_aimodelc":"1.9G","digest":{"bytes":2039283435,"tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"9917e9b4280745b24b9c0ae5d6d3091bc14cae450202661efa1880e223a3ae32","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"8f100f2739ee2c23945f6eb612cc072de70df00e5f3e00de73b4413ec09ddeda","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"1cda3ad44b1fdf2670ab96857fcd2b7833284c9672a32a6db12c36b6be40d8cf","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"be1be4361facb8d030824a04fac62e5d2d5a9935b5aab4b260203a6704f058e9","main-h16c.mlirb":"be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950","main.hash":"33638a7f1ce95a4fb9c85161d9f04c529ac3e95ef99d8d5cb5d471b0a72f0739","metadata.json":"47e965dcd7035971c4e5ffc59ae2473c390d6a32efe4f22d06ed6c54834e2c66","stats.json":"50309ea498f9c14ee6c2bf20909e32b57140b2ab81649885404ea5e6abfc4c19"}}},"finished":"2026-10-03T17:24:30+09:00"},"readout_int8lin":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8lin_pf16","name":"kev_0_8b_decode_int8lin_pf16","kind":"decision-backbone","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":186,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"a35475102eb71c97f7bf73cc39afcba2d4de14c1bb9986c2b1aa7025980d818e","main_mlirb":{"bytes":1040063350,"sha256":"2281e56d413cbc38cde4bd37953fd5476726a1f37953f940eba9dadcc0bfdb77"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8lin_pf16.h16c.aimodelc","bytes":2039283435,"tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"9917e9b4280745b24b9c0ae5d6d3091bc14cae450202661efa1880e223a3ae32","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"8f100f2739ee2c23945f6eb612cc072de70df00e5f3e00de73b4413ec09ddeda","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"1cda3ad44b1fdf2670ab96857fcd2b7833284c9672a32a6db12c36b6be40d8cf","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"be1be4361facb8d030824a04fac62e5d2d5a9935b5aab4b260203a6704f058e9","main-h16c.mlirb":"be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950","main.hash":"33638a7f1ce95a4fb9c85161d9f04c529ac3e95ef99d8d5cb5d471b0a72f0739","metadata.json":"47e965dcd7035971c4e5ffc59ae2473c390d6a32efe4f22d06ed6c54834e2c66","stats.json":"50309ea498f9c14ee6c2bf20909e32b57140b2ab81649885404ea5e6abfc4c19"}}},"chunk":16,"max_ctx":4096,"readout":"one row per question, fresh zero states per row; a row of T ids runs as ceil(T / 16) calls of 'main' (static S = 16): call k gets ids[16k : 16k + 16] with position_ids 0..16k+15; the last call is padded with <|endoftext|> (248044) and the hidden rows of the padded positions are discarded (causal: they cannot reach a real position). The hidden rows [1, 16, 1024] of every call, concatenated and cut to T, are the backbone's final-norm last_hidden_state [T, 1024] the head reads. A row fits when ceil(T / 16) * 16 <= 4095 (the position dim's upper bound; the KV sequence dim is allocated at 4096).","descriptor_same_in_every_process":true,"contract":{"inputs":{"input_ids":[[1,16],"int32"],"position_ids":[[1,-1],"int32"]},"outputs":{"hidden":[[1,16,1024],"float16"]},"states":{"keyCache":[[6,1,2,-1,256],"float16"],"valueCache":[[6,1,2,-1,256],"float16"],"convState":[[18,1,6144,3],"float16"],"recState":[[18,1,16,128,128],"float16"]}},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle/records_oracle.json","sha256":"1a72a45831eecc2af7ddfc68a86e9250f572d6bf9274362f96cc62bcb803cc6d","checkpoint":"jaredpalmer/kev-0.8b@788ddbdd65715bb03a56788c822f6c632c9a551d","versions":{"python":"3.12.13","torch":"2.8.0","transformers":"5.17.0","peft":"0.21.0","numpy":"2.5.3","safetensors":"0.8.0"}},"subset":"all","subset_file":null,"script":{"path":"conversion/kev/readout_gate.py","sha256":"2a31a63316d313ef3aa1f22addecbc94d55c7d0cd3527c6b8f39f21e4d1de61b"},"bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"}},"processes":[{"shard":"shard_00","runs":["tv4_000:q0:base","tv4_001:q0:base","tv4_002:q0:base","tv4_003:q0:base","tv4_004:q0:base","tv4_005:q0:base","tv4_006:q0:base","tv4_007:q0:base","tv4_008:q0:base","tv4_009:q0:base","tv4_010:q0:base","tv4_011:q0:base","tv4_012:q0:base","tv4_013:q0:base","tv4_014:q0:base","tv4_015:q0:base","tv4_016:q0:base","tv4_017:q0:base","tv4_018:q0:base","tv4_019:q0:base","tv4_020:q0:base","tv4_021:q0:base","tv4_022:q0:base","tv4_023:q0:base","tv4_024:q0:base","tv4_025:q0:base","tv4_026:q0:base","tv4_027:q0:base","tv4_028:q0:base","tv4_029:q0:base","tv4_030:q0:base","tv4_031:q0:base","tv4_032:q0:base","tv4_033:q0:base","tv4_034:q0:base","tv4_035:q0:base","tv4_036:q0:base","tv4_037:q0:base","tv4_038:q0:base","tv4_039:q0:base"],"prompts":41,"load_seconds":1.8636506660259329,"load_split_seconds":{"model":0.59131858299952,"main":1.2723320830264129},"function_names":["main"],"process_wall_seconds":10.877666832995601,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.15611474998877384},"contract_mismatch":[]},{"shard":"shard_01","runs":["tv4_040:q0:base","tv4_041:q0:base","tv4_042:q0:base","tv4_043:q0:base","tv4_044:q0:base","tv4_045:q0:base","tv4_046:q0:base","tv4_047:q0:base","tv4_048:q0:base","tv4_049:q0:base","tv4_050:q0:base","tv4_051:q0:base","tv4_052:q0:base","tv4_053:q0:base","tv4_054:q0:base","tv4_055:q0:base","tv4_056:q0:base","tv4_057:q0:base","tv4_058:q0:base","tv4_059:q0:base","tv4x_emotion_00:q0:base","tv4x_emotion_01:q0:base","tv4x_emotion_02:q0:base","tv4x_emotion_03:q0:base","tv4x_emotion_04:q0:base","tv4x_emotion_05:q0:base","tv4x_emotion_06:q0:base","tv4x_emotion_07:q0:base","tv4x_emotion_08:q0:base","tv4x_emotion_09:q0:base","tv4x_emotion_10:q0:base","tv4x_emotion_11:q0:base","tv4x_emotion_12:q0:base","tv4x_emotion_13:q0:base","tv4x_emotion_14:q0:base","tv4x_emotion_15:q0:base","tv4x_emotion_16:q0:base","tv4x_emotion_17:q0:base","tv4x_emotion_18:q0:base","tv4x_emotion_19:q0:base"],"prompts":41,"load_seconds":0.2692842500109691,"load_split_seconds":{"model":0.0067772920010611415,"main":0.262506958009908},"function_names":["main"],"process_wall_seconds":7.192300041002454,"reset_check":{"run":"tv4_040:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.3672026249987539},"contract_mismatch":[]},{"shard":"shard_02","runs":["tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_01:q0:base","tv4x_tweet_offensive_02:q0:base","tv4x_tweet_offensive_03:q0:base","tv4x_tweet_offensive_04:q0:base","tv4x_tweet_offensive_05:q0:base","tv4x_tweet_offensive_06:q0:base","tv4x_tweet_offensive_07:q0:base","tv4x_tweet_offensive_08:q0:base","tv4x_tweet_offensive_09:q0:base","tv4x_tweet_offensive_10:q0:base","tv4x_tweet_offensive_11:q0:base","tv4x_tweet_offensive_12:q0:base","tv4x_tweet_offensive_13:q0:base","tv4x_tweet_offensive_14:q0:base","tv4x_tweet_offensive_15:q0:base","tv4x_tweet_offensive_16:q0:base","tv4x_tweet_offensive_17:q0:base","tv4x_tweet_offensive_18:q0:base","tv4x_tweet_offensive_19:q0:base","tv4x_qnli_00:q0:base","tv4x_qnli_01:q0:base","tv4x_qnli_02:q0:base","tv4x_qnli_03:q0:base","tv4x_qnli_04:q0:base","tv4x_qnli_05:q0:base","tv4x_qnli_06:q0:base","tv4x_qnli_07:q0:base","tv4x_qnli_08:q0:base","tv4x_qnli_09:q0:base","tv4x_qnli_10:q0:base","tv4x_qnli_11:q0:base","tv4x_qnli_12:q0:base","tv4x_qnli_13:q0:base","tv4x_qnli_14:q0:base","tv4x_qnli_15:q0:base","tv4x_qnli_16:q0:base","tv4x_qnli_17:q0:base","tv4x_qnli_18:q0:base","tv4x_qnli_19:q0:base"],"prompts":41,"load_seconds":0.36750095899333246,"load_split_seconds":{"model":0.006739959004335105,"main":0.36076099998899736},"function_names":["main"],"process_wall_seconds":6.831603834027192,"reset_check":{"run":"tv4x_tweet_offensive_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13748591698822565},"contract_mismatch":[]},{"shard":"shard_03","runs":["tv4x_paws_00:q0:base","tv4x_paws_01:q0:base","tv4x_paws_02:q0:base","tv4x_paws_03:q0:base","tv4x_paws_04:q0:base","tv4x_paws_05:q0:base","tv4x_paws_06:q0:base","tv4x_paws_07:q0:base","tv4x_paws_08:q0:base","tv4x_paws_09:q0:base","tv4x_paws_10:q0:base","tv4x_paws_11:q0:base","tv4x_paws_12:q0:base","tv4x_paws_13:q0:base","tv4x_paws_14:q0:base","tv4x_paws_15:q0:base","tv4x_paws_16:q0:base","tv4x_paws_17:q0:base","tv4x_paws_18:q0:base","tv4x_paws_19:q0:base","tv4x_sciq_00:q0:base","tv4x_sciq_01:q0:base","tv4x_sciq_02:q0:base","tv4x_sciq_03:q0:base","tv4x_sciq_04:q0:base","tv4x_sciq_05:q0:base","tv4x_sciq_06:q0:base","tv4x_sciq_07:q0:base","tv4x_sciq_08:q0:base","tv4x_sciq_09:q0:base","tv4x_sciq_10:q0:base","tv4x_sciq_11:q0:base","tv4x_sciq_12:q0:base","tv4x_sciq_13:q0:base","tv4x_sciq_14:q0:base","tv4x_sciq_15:q0:base","tv4x_sciq_16:q0:base","tv4x_sciq_17:q0:base","tv4x_sciq_18:q0:base","tv4x_sciq_19:q0:base"],"prompts":41,"load_seconds":0.3053808330150787,"load_split_seconds":{"model":0.007432458020048216,"main":0.29794837499503046},"function_names":["main"],"process_wall_seconds":10.036305250017904,"reset_check":{"run":"tv4x_paws_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.18435774999670684},"contract_mismatch":[]},{"shard":"shard_04","runs":["tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_01:q0:base","tv4x_legacy_holdout_02:q0:base","tv4x_legacy_holdout_03:q0:base","tv4x_legacy_holdout_04:q0:base","tv4x_legacy_holdout_05:q0:base","tv4x_legacy_holdout_06:q0:base","tv4x_legacy_holdout_07:q0:base","tv4x_legacy_holdout_08:q0:base","tv4x_legacy_holdout_09:q0:base","tv4x_legacy_holdout_10:q0:base","tv4x_legacy_holdout_11:q0:base","tv4x_legacy_holdout_12:q0:base","tv4x_legacy_holdout_13:q0:base","tv4x_legacy_holdout_14:q0:base","tv4x_legacy_holdout_15:q0:base","tv4x_legacy_holdout_16:q0:base","tv4x_legacy_holdout_17:q0:base","tv4x_legacy_holdout_18:q0:base","tv4x_legacy_holdout_19:q0:base","tv4x_composition_holdout_00:q0:base","tv4x_composition_holdout_01:q0:base","tv4x_composition_holdout_02:q0:base","tv4x_composition_holdout_03:q0:base","tv4x_composition_holdout_04:q0:base","tv4x_composition_holdout_05:q0:base","tv4x_composition_holdout_06:q0:base","tv4x_composition_holdout_07:q0:base","tv4x_composition_holdout_08:q0:base","tv4x_composition_holdout_09:q0:base","tv4x_composition_holdout_10:q0:base","tv4x_composition_holdout_11:q0:base","tv4x_composition_holdout_12:q0:base","tv4x_composition_holdout_13:q0:base","tv4x_composition_holdout_14:q0:base","tv4x_composition_holdout_15:q0:base","tv4x_composition_holdout_16:q0:base","tv4x_composition_holdout_17:q0:base","tv4x_composition_holdout_18:q0:base","tv4x_composition_holdout_19:q0:base"],"prompts":41,"load_seconds":0.3299044169834815,"load_split_seconds":{"model":0.008006166986888275,"main":0.3218982499965932},"function_names":["main"],"process_wall_seconds":9.148747374973027,"reset_check":{"run":"tv4x_legacy_holdout_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13522483297856525},"contract_mismatch":[]},{"shard":"shard_05","runs":["tv4s_00:q0:base","tv4s_01:q0:base","tv4s_02:q0:base","tv4s_03:q0:base","tv4s_04:q0:base","tv4s_05:q0:base","tv4s_06:q0:base","tv4s_07:q0:base","tv4s_08:q0:base","tv4s_09:q0:base","tv4s_10:q0:base","tv4s_11:q0:base","tv4s_12:q0:base","tv4s_13:q0:base","tv4s_14:q0:base","tv4s_15:q0:base","tv4s_16:q0:base","tv4s_17:q0:base","tv4s_18:q0:base","tv4s_19:q0:base","semif_a3f18f3a63d45345942b:q0:base","semif_f40beba9088c8db8bbd6:q0:base","semif_0b43ea8e24e74d621f1b:q0:base","semif_942ec65dac0bfb9874f9:q0:base","semif_e0c140e9222d1bce5647:q0:base","semif_8e8c3804a3c15ebb31e3:q0:base","semif_56c419b12cd37cbe790c:q0:base","semif_75b470586834df8c568d:q0:base","semif_834f3b3e10cc61b33d25:q0:base","semif_76c7a79b1ff2363a60c5:q0:base","semif_36c71bb649b7d5eccb1f:q0:base","semif_eac5121a4a1bf391a08c:q0:base","semif_44574c44e243ee1b2f52:q0:base","semif_2608370d01783735c3ea:q0:base","semif_ec94ef909f7918695ab3:q0:base","semif_03d919fd0e9b3a7cd1e6:q0:base","semif_86a15618ba4cb90a307e:q0:base","semif_3dd9c3ac98f8ddf7bdd3:q0:base","semif_36273dd4777876a26338:q0:base","semif_07ee70cfe4b01330ce89:q0:base"],"prompts":41,"load_seconds":0.30040495900902897,"load_split_seconds":{"model":0.00897100000292994,"main":0.29143395900609903},"function_names":["main"],"process_wall_seconds":7.528707874997053,"reset_check":{"run":"tv4s_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.18003679197863676},"contract_mismatch":[]},{"shard":"shard_06","runs":["semif_51ceab3ce622c7c759f1:q0:base","semif_b8091be929b1e489fcc4:q0:base","semif_c89b86e3f55123bfa506:q0:base","semif_3b0958cabc339005cf66:q0:base","semif_f2b4ec4930322fe45118:q0:base","semif_61ac20b07a5189231e15:q0:base","semif_4b3d794c004566610181:q0:base","semif_b7c4e62d07986286fce0:q0:base","semif_9473603b44af63beab79:q0:base","semif_d41072dc35d8cd14c056:q0:base","semif_33d6e5da58f0fee2490d:q0:base","semif_823bf736b92e9d8fa66d:q0:base","semif_8a4c3de28be95c105ca1:q0:base","semif_ea7ec66a5e8eac886548:q0:base","semif_25172cd6ddc49ab0ca70:q0:base","semif_fb73a85d386cf59b0c51:q0:base","semif_a02fd582d2881cc462b2:q0:base","semif_19f37d5a97034bd7d04f:q0:base","semif_fda0ca94652c6d739ecf:q0:base","semif_d0e36c4cb10a29b3191c:q0:base","semif_a0a39ace7e5337d038a7:q0:base","semif_eeae283841ce75dd25bf:q0:base","semif_00bb4cc043f7d2b288d4:q0:base","semif_1a81ead69e93e5d0c86a:q0:base","semif_4444dad25e438f128673:q0:base","semif_3226202ba1ada2570e9f:q0:base","semif_3f783eec5c90812479bc:q0:base","semif_ffee861474a9665a093a:q0:base","semif_8c992e17e4fb54b044f5:q0:base","semif_695dd61195ca025c4f4c:q0:base","semif_e6155431ab7b0f8170d4:q0:base","semif_b49ecda75086d8c68d2a:q0:base","semif_587ced261c6092cd1325:q0:base","semif_533d4423d311de82b27d:q0:base","semif_2bf540d515a51755b25c:q0:base","semif_fdb9fc214aea2710763a:q0:base","semif_f6d68126b43f8c9ed183:q0:base","semif_44bae0c8a1615a096c43:q0:base","semif_2175379f4c69e6207626:q0:base","semif_def9fe59d262357a673a:q0:base"],"prompts":41,"load_seconds":0.28505320800468326,"load_split_seconds":{"model":0.0068884580105077475,"main":0.2781647499941755},"function_names":["main"],"process_wall_seconds":5.9639452080009505,"reset_check":{"run":"semif_51ceab3ce622c7c759f1:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.12723791698226705},"contract_mismatch":[]},{"shard":"shard_07","runs":["semif_712f6ffb763eef7c9a12:q0:base","semif_e1d610bd14e3d16c09b9:q0:base","semif_22ec57822db50ec121cf:q0:base","semif_3e75f2b84e8e08e3d3f2:q0:base","semif_ebd74aa05324a1556288:q0:base","semif_d7e84e4245a29817da1f:q0:base","semif_f41ae66b3c09630c7a42:q0:base","semif_82c2cd97c160ad27951b:q0:base","semif_7ada97b6f99f4788e5c7:q0:base","semif_4d7a003342db5b0a17e3:q0:base","semif_19d3d8d0bf773e88be7b:q0:base","semif_d6de731cca52359b1c87:q0:base","semif_b45c63fb92729af27f05:q0:base","semif_46b7029b9a704138b77a:q0:base","semif_caf773b23934fa7362a1:q0:base","semif_f46f392ef9e9e9df564b:q0:base","semif_005dcf6d2c0f21d64796:q0:base","semif_edb708bd0fe5fd67d6ee:q0:base","semif_97a35499bd8717f3e786:q0:base","semif_31eb4533d2c89959ec46:q0:base","semif_0aa24a6701ac9c56c4a0:q0:base","semif_796d5c0da6eaadae6a3f:q0:base","semif_599b3092b6fd18312d23:q0:base","semif_960a4bcd27363e6db4b9:q0:base","semif_c6a411e7f2bbdf377173:q0:base","semif_e332aa927d0c2ed044aa:q0:base","semif_a0128e551d095f3449f8:q0:base","semif_46d6a0731bd288c370f0:q0:base","semif_21ca7a8b970e36b8e20f:q0:base","semif_4aa83af95bb69e90bc2d:q0:base","semif_cdfa9eed51fafcf09b72:q0:base","semif_d51b05bfb9d6d7f09331:q0:base","semif_fdd902cc6e65fad7cb4f:q0:base","semif_42384af9ae56beb37ff5:q0:base","semif_cdf7f428857e1b841cb3:q0:base","semif_774f7f9245f83c796d44:q0:base","semif_92ec2ffa3dd46ada1335:q0:base","semif_6149a17bc154f9c5b4a0:q0:base","semif_be866e1dbec5d42d9423:q0:base","semif_97d2ed2591fc7638806c:q0:base"],"prompts":41,"load_seconds":0.2922575830016285,"load_split_seconds":{"model":0.006929916009539738,"main":0.2853276669920888},"function_names":["main"],"process_wall_seconds":6.575138334010262,"reset_check":{"run":"semif_712f6ffb763eef7c9a12:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1147107079741545},"contract_mismatch":[]},{"shard":"shard_08","runs":["semif_4533dd91267b4d815690:q0:base","semif_d880e65fef5e2993fdd0:q0:base","semif_90494c76078ffc3c717a:q0:base","semif_9ac5cc82c4126c91174a:q0:base","semif_386e10b1c1ce906473a4:q0:base","semif_4da6aedc1139224eeff2:q0:base","semif_a22dab8ba073ff6cdb8d:q0:base","semif_42c54fa2ad45743e24d8:q0:base","semif_f3d13cf184f916677def:q0:base","semif_1105577da4c8dad4609d:q0:base","semif_0ebd299739302d8b057b:q0:base","semif_eafc22c8c40df3932a8e:q0:base","semif_0aa1447680e4f25862bb:q0:base","semif_980e76145c925747220c:q0:base","semif_4aee37a6a155747766aa:q0:base","semif_6db17cc22bd656d78558:q0:base","semif_ef3e0f8e657bf59f8a29:q0:base","semif_f756e1cb04e11766341e:q0:base","semif_d167e51bb1f499099645:q0:base","semif_674ca4e9346216255466:q0:base","semif_1df851a202994a07f123:q0:base","semif_b5e6788d95b8395d2888:q0:base","semif_32f36cd0fbbfb3604586:q0:base","semif_843185706adff0c60f8b:q0:base","semif_b531dd603bf080dab820:q0:base","semif_be2dd67bc733c1213bef:q0:base","semif_b8e2750307df9761e716:q0:base","semif_5d31b4bf2d0fc193f53b:q0:base","semif_a9230edfc6ec9e524af0:q0:base","semif_c100e84585149a04be49:q0:base","semif_579beed184ffc1104aa9:q0:base","semif_c4e9bd463b6fde5be743:q0:base","semif_b863998d4be2721a5dfd:q0:base","semif_37cebc3792e98049439f:q0:base","semif_64d63f86e328aff1481c:q0:base","semif_6d64a8459f64eac7f4cc:q0:base","semif_2708193212d8a4a523c7:q0:base","semif_5a37d1dfcad20810a76a:q0:base","semif_acacd8b46acd25cb954b:q0:base","semif_4089c199b8b796ec3ed8:q0:base"],"prompts":41,"load_seconds":0.3078007909934968,"load_split_seconds":{"model":0.008111749979434535,"main":0.29968904101406224},"function_names":["main"],"process_wall_seconds":5.916500332998112,"reset_check":{"run":"semif_4533dd91267b4d815690:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.10870754200732335},"contract_mismatch":[]},{"shard":"shard_09","runs":["semif_02f2b95a5c5a4777a45c:q0:base","semif_ebea4681e3dcc7d92ca2:q0:base","semif_b8c5b9b285cbdddc9f8d:q0:base","semif_7ed75d854d99b3a81916:q0:base","own_t01:q0:base","own_t01:q1:base","own_t01:q2:base","own_t01:q3:base","own_t02:q0:base","own_t02:q1:base","own_t02:q2:base","own_t03:q0:base","own_t03:q1:base","own_t03:q2:base","own_t04:q0:base","own_t04:q1:base","own_t04:q2:base","own_t04:q3:base","own_t05:q0:base","own_t05:q1:base","own_t05:q2:base","own_t05:q3:base","own_t06:q0:base","own_t06:q1:base","own_t06:q2:base","own_t07:q0:base","own_t07:q1:base","own_t07:q2:base","own_t08:q0:base","own_t08:q1:base","own_t08:q2:base","own_t09:q0:base","own_t09:q1:base","own_t10:q0:base","own_t10:q1:base","own_t10:q2:base","own_j01:q0:base","own_j01:q1:base","own_j01:q2:base","own_j01:q3:base"],"prompts":41,"load_seconds":0.22877495901775546,"load_split_seconds":{"model":0.005500958999618888,"main":0.22327400001813658},"function_names":["main"],"process_wall_seconds":9.697510750003858,"reset_check":{"run":"semif_02f2b95a5c5a4777a45c:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.10497987500275485},"contract_mismatch":[]},{"shard":"shard_10","runs":["own_j02:q0:base","own_j02:q1:base","own_j02:q2:base","own_j03:q0:base","own_j03:q1:base","own_j03:q2:base","own_j04:q0:base","own_j04:q1:base","own_j04:q2:base","own_j05:q0:base","own_j05:q1:base","own_j05:q2:base","own_a01:q0:base","own_a01:q1:base","own_a01:q2:base","own_m01:q0:base","own_m01:q1:base","own_m01:q2:base","own_m01:q3:base","own_m01:q4:base","own_m01:q5:base","own_m01:q6:base","own_m01:q7:base","own_L01:q0:base","own_L01:q1:base","own_L01:q2:base","own_L02:q0:base","own_L02:q1:base","own_L02:q2:base","own_L02:q3:base","own_L03:q0:base","own_L03:q1:base","own_L03:q2:base","own_L03:q3:base"],"prompts":35,"load_seconds":0.23304875000030734,"load_split_seconds":{"model":0.0057236250140704215,"main":0.22732512498623691},"function_names":["main"],"process_wall_seconds":28.866272209008457,"reset_check":{"run":"own_j02:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.4521169590007048},"contract_mismatch":[]}],"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":431,"questions_non_near_tie":420,"argmax_equal_non_near_tie":420,"near_tie_questions":14,"argmax_equal_near_tie":11,"max_abs_dp":0.04313388466835022,"max_abs_dp_non_near_tie":0.04313388466835022,"mean_of_run_mean_abs_dp":0.0032562481012639575,"mean_of_record_mean_abs_dp":0.0033821680378596044,"mean_question_max_abs_dp":0.004676064381951011,"max_abs_dlogit_T":0.1422673463821411,"worst_run":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.04313388466835022},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9990858151789371,"hidden_max_abs_diff":0.497005820274353,"positions":9168,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":98209.05354668503,"ms_per_call_median":21.9651669904124,"ms_per_call_warm_median":21.899353989283554},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.018736541271209717,"max_abs_dp_non_near_tie":0.018736541271209717,"mean_of_run_mean_abs_dp":0.002540853831515575,"mean_of_record_mean_abs_dp":0.002247256835210745,"mean_question_max_abs_dp":0.003878929113086901,"max_abs_dlogit_T":0.06842005252838135,"worst_run":{"id":"own_j04","q":0,"max_abs_dp":0.018736541271209717},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9998040192160517,"hidden_max_abs_diff":0.2418193817138672,"positions":1027,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":5626.417041145032,"ms_per_call_median":18.83070802432485,"ms_per_call_warm_median":18.76541701494716},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.016770899295806885,"max_abs_dp_non_near_tie":0.016770899295806885,"mean_of_run_mean_abs_dp":0.003103576024824923,"mean_of_record_mean_abs_dp":0.003085295466753917,"mean_question_max_abs_dp":0.005163125355135311,"max_abs_dlogit_T":0.06045961380004883,"worst_run":{"id":"own_L03","q":0,"max_abs_dp":0.016770899295806885},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9990858151789371,"hidden_max_abs_diff":0.497005820274353,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":21744.858042045962,"ms_per_call_median":19.90902099350933,"ms_per_call_warm_median":19.90902099350933},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":40,"questions_non_near_tie":39,"argmax_equal_non_near_tie":39,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.01196315884590149,"max_abs_dp_non_near_tie":0.01196315884590149,"mean_of_run_mean_abs_dp":0.0018738969543483108,"mean_of_record_mean_abs_dp":0.0018338964021642793,"mean_question_max_abs_dp":0.0028853118419647217,"max_abs_dlogit_T":0.07553759217262268,"worst_run":{"id":"own_t06","q":0,"max_abs_dp":0.01196315884590149},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9996967220801186,"hidden_max_abs_diff":0.3991870880126953,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":8939.690250059357,"ms_per_call_median":18.72979200561531,"ms_per_call_warm_median":18.72979200561531},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":143,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.04313388466835022,"max_abs_dp_non_near_tie":0.04313388466835022,"mean_of_run_mean_abs_dp":0.0038571930997056397,"mean_of_record_mean_abs_dp":0.0038571930997056397,"mean_question_max_abs_dp":0.005785790918808844,"max_abs_dlogit_T":0.1274890899658203,"worst_run":{"id":"semif_1105577da4c8dad4609d","q":0,"max_abs_dp":0.04313388466835022},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.999760318900587,"hidden_max_abs_diff":0.1971721649169922,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":19192.84200205584,"ms_per_call_median":24.391166982240975,"ms_per_call_warm_median":24.391166982240975},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":58,"questions_non_near_tie":52,"argmax_equal_non_near_tie":52,"near_tie_questions":8,"argmax_equal_near_tie":6,"max_abs_dp":0.026517271995544434,"max_abs_dp_non_near_tie":0.017518997192382812,"mean_of_run_mean_abs_dp":0.002489552361900375,"mean_of_record_mean_abs_dp":0.002489552361900375,"mean_question_max_abs_dp":0.004632989193002383,"max_abs_dlogit_T":0.10893160104751587,"worst_run":{"id":"tv4_023","q":0,"max_abs_dp":0.026517271995544434},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9998533746578387,"hidden_max_abs_diff":0.1933727264404297,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":12870.933628146304,"ms_per_call_median":23.40945799369365,"ms_per_call_warm_median":23.299937514821067},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":20,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.004956483840942383,"max_abs_dp_non_near_tie":0.004956483840942383,"mean_of_run_mean_abs_dp":0.001808550332983335,"mean_of_record_mean_abs_dp":0.001808550332983335,"mean_question_max_abs_dp":0.0027128301560878755,"max_abs_dlogit_T":0.033730149269104004,"worst_run":{"id":"tv4s_09","q":0,"max_abs_dp":0.004956483840942383},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":4401.22429098119,"ms_per_call_median":28.118625006754883,"ms_per_call_warm_median":27.99858301295899},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":140,"argmax_equal_non_near_tie":140,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.026461392641067505,"max_abs_dp_non_near_tie":0.026461392641067505,"mean_of_run_mean_abs_dp":0.0036775734569389554,"mean_of_record_mean_abs_dp":0.0036775734569389554,"mean_question_max_abs_dp":0.004415111350161689,"max_abs_dlogit_T":0.1422673463821411,"worst_run":{"id":"tv4x_composition_holdout_15","q":0,"max_abs_dp":0.026461392641067505},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":25433.08829225134,"ms_per_call_median":27.028000011341646,"ms_per_call_warm_median":26.989624981069937}},"near_ties":[{"id":"tv4_003","q":0,"qid":"answer","oracle_top2_margin":0.016063004732131958,"argmax_equal":true,"max_abs_dp":0.004495948553085327,"probs":[0.4062272906303406,0.3976673185825348,0.07255612313747406,0.12354932725429535]},{"id":"tv4_017","q":0,"qid":"answer","oracle_top2_margin":0.01221814751625061,"argmax_equal":true,"max_abs_dp":0.004297226667404175,"probs":[0.2615928649902344,0.23841449618339539,0.2568325996398926,0.24316006898880005]},{"id":"tv4_023","q":0,"qid":"answer","oracle_top2_margin":0.0019783377647399902,"argmax_equal":false,"max_abs_dp":0.026517271995544434,"probs":[0.39101436734199524,0.34842997789382935,0.14739468693733215,0.11316091567277908]},{"id":"tv4_024","q":0,"qid":"answer","oracle_top2_margin":0.015634745359420776,"argmax_equal":true,"max_abs_dp":0.0033638477325439453,"probs":[0.3425765931606293,0.2676292061805725,0.03687611594796181,0.3529181182384491]},{"id":"tv4_027","q":0,"qid":"answer","oracle_top2_margin":0.009622842073440552,"argmax_equal":true,"max_abs_dp":0.001272737979888916,"probs":[0.16761399805545807,0.3338010907173157,0.34227389097213745,0.1563110202550888]},{"id":"tv4_028","q":0,"qid":"answer","oracle_top2_margin":0.01672416925430298,"argmax_equal":true,"max_abs_dp":0.004708617925643921,"probs":[0.16981390118598938,0.31957295536994934,0.21111741662025452,0.29949572682380676]},{"id":"tv4_051","q":0,"qid":"answer","oracle_top2_margin":0.006280571222305298,"argmax_equal":true,"max_abs_dp":0.0025317519903182983,"probs":[0.05768092721700668,0.18390388786792755,0.3830457925796509,0.3753693401813507]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.0028791427612304688,"argmax_equal":false,"max_abs_dp":0.0043344199657440186,"probs":[0.13899265229701996,0.276436984539032,0.2924071252346039,0.2921631932258606]},{"id":"tv4s_14","q":0,"qid":"decision","oracle_top2_margin":0.01574954390525818,"argmax_equal":true,"max_abs_dp":0.0011319220066070557,"probs":[0.3958556354045868,0.41349661350250244,0.19064776599407196]},{"id":"semif_2175379f4c69e6207626","q":0,"qid":"decision","oracle_top2_margin":0.01896810531616211,"argmax_equal":true,"max_abs_dp":0.0016297996044158936,"probs":[0.41866299510002136,0.39789846539497375,0.1834385246038437]},{"id":"semif_d167e51bb1f499099645","q":0,"qid":"decision","oracle_top2_margin":0.01671499013900757,"argmax_equal":false,"max_abs_dp":0.010080963373184204,"probs":[0.346190482378006,0.3045433461666107,0.3492661714553833]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0032414793968200684,"argmax_equal":true,"max_abs_dp":0.0018231868743896484,"probs":[0.4404924511909485,0.4406611919403076,0.11884637176990509]},{"id":"own_t10","q":2,"qid":"escalation","oracle_top2_margin":0.011951148509979248,"argmax_equal":true,"max_abs_dp":0.004240356385707855,"probs":[0.31916216015815735,0.2712917625904083,0.30745646357536316,0.10208962112665176]},{"id":"own_j02","q":0,"qid":"next_action","oracle_top2_margin":0.007524773478507996,"argmax_equal":true,"max_abs_dp":0.0020061135292053223,"probs":[0.2073727548122406,0.08935251832008362,0.19707722961902618,0.08295747637748718,0.02874712087213993,0.004249794874340296,0.01963174343109131,0.15918785333633423,0.015149169601500034,0.19627439975738525]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":false,"mean_of_run_mean_abs_dp":false,"finite":true,"reset_bit_equal_all_processes":true},"result":"FAIL","red_arm":null,"timing":{"contended":true,"S":16,"load_seconds":{"first_process":1.8636506660259329,"median":0.30040495900902897,"all":[1.8636506660259329,0.2692842500109691,0.36750095899333246,0.3053808330150787,0.3299044169834815,0.30040495900902897,0.28505320800468326,0.2922575830016285,0.3078007909934968,0.22877495901775546,0.23304875000030734]},"ms_per_call_warm_runs":{"n":3972,"median":21.899353989283554,"p10":18.218545787385665,"p90":28.547662499477156},"ms_per_call_all_runs":{"n":4055,"median":21.9651669904124,"p10":18.223341403063387,"p90":28.615233401069418},"ms_per_token_warm_runs":{"n":423,"median":1.717643047120114,"p10":1.2276228333339243,"p90":1.995633606191541},"run_decode_seconds_warm_runs":{"n":423,"median":0.14118895799038,"p10":0.10074629199225456,"p90":0.28722954978584314},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":88801.88042654481}},"seconds":{"gpu_processes":108.68899208301445,"scoring":2.0754527089884505,"total":110.91896741601522},"runs":[{"id":"tv4_000","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004012078046798706,"mean_abs_dp":0.002006053924560547,"oracle_top2_margin":0.2241945117712021,"near_tie":false,"probs":[0.14792905747890472,0.2063399702310562,0.4324827790260315,0.21324819326400757],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4_001","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003998875617980957,"mean_abs_dp":0.00019992668239865452,"oracle_top2_margin":0.9833351373672485,"near_tie":false,"probs":[0.006235084030777216,0.002635754644870758,0.9902555346488953,0.0008736865129321814]},{"id":"tv4_002","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0059996843338012695,"mean_abs_dp":0.0029998384416103363,"oracle_top2_margin":0.10138940811157227,"near_tie":false,"probs":[0.1768374741077423,0.3640696704387665,0.2560829818248749,0.20300987362861633]},{"id":"tv4_003","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004495948553085327,"mean_abs_dp":0.0022708717733621597,"oracle_top2_margin":0.016063004732131958,"near_tie":true,"probs":[0.4062272906303406,0.3976673185825348,0.07255612313747406,0.12354932725429535]},{"id":"tv4_004","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0016527175903320312,"mean_abs_dp":0.000886741909198463,"oracle_top2_margin":0.9105252027511597,"near_tie":false,"probs":[0.0057381740771234035,0.028252121061086655,0.9415088891983032,0.024500755593180656]},{"id":"tv4_005","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003984361886978149,"mean_abs_dp":0.002032437827438116,"oracle_top2_margin":0.23442357778549194,"near_tie":false,"probs":[0.32915452122688293,0.5711400508880615,0.022669656202197075,0.07703584432601929]},{"id":"tv4_006","q":0,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.010095983743667603,"mean_abs_dp":0.005047975108027458,"oracle_top2_margin":0.12790140509605408,"near_tie":false,"probs":[0.20154951512813568,0.2870020270347595,0.07856094837188721,0.43288758397102356]},{"id":"tv4_007","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00558057427406311,"mean_abs_dp":0.004372086375951767,"oracle_top2_margin":0.2037426233291626,"near_tie":false,"probs":[0.23612408339977264,0.1317918747663498,0.43930283188819885,0.19278118014335632]},{"id":"tv4_008","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001433342695236206,"mean_abs_dp":0.0007476486498489976,"oracle_top2_margin":0.8541439175605774,"near_tie":false,"probs":[0.049622006714344025,0.903619110584259,0.005585627164691687,0.041173238307237625]},{"id":"tv4_009","q":0,"variant":"base","tokens":321,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001700282096862793,"mean_abs_dp":0.0008501391857862473,"oracle_top2_margin":0.787467360496521,"near_tie":false,"probs":[0.03076532855629921,0.07468190044164658,0.8648245334625244,0.029728172346949577]},{"id":"tv4_010","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0036230087280273438,"mean_abs_dp":0.001811489462852478,"oracle_top2_margin":0.11676552891731262,"near_tie":false,"probs":[0.07709261775016785,0.41617581248283386,0.20255400240421295,0.30417755246162415]},{"id":"tv4_011","q":0,"variant":"base","tokens":105,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0032656192779541016,"mean_abs_dp":0.002158176153898239,"oracle_top2_margin":0.07526007294654846,"near_tie":false,"probs":[0.1668730080127716,0.16832169890403748,0.29744696617126465,0.36735835671424866]},{"id":"tv4_012","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005900263786315918,"mean_abs_dp":0.0030542267486453056,"oracle_top2_margin":0.13543769717216492,"near_tie":false,"probs":[0.31941303610801697,0.02703586407005787,0.4630146026611328,0.1905365288257599]},{"id":"tv4_013","q":0,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.005666404962539673,"mean_abs_dp":0.0029819770716130733,"oracle_top2_margin":0.15590348839759827,"near_tie":false,"probs":[0.05265102908015251,0.01629490777850151,0.38219112157821655,0.5488629341125488]},{"id":"tv4_014","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004725217819213867,"mean_abs_dp":0.0023626014590263367,"oracle_top2_margin":0.06324279308319092,"near_tie":false,"probs":[0.1854899674654007,0.23733994364738464,0.25367599725723267,0.3234940767288208]},{"id":"tv4_015","q":0,"variant":"base","tokens":177,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002215176820755005,"mean_abs_dp":0.0016162581741809845,"oracle_top2_margin":0.26414579153060913,"near_tie":false,"probs":[0.15968553721904755,0.19782108068466187,0.46076902747154236,0.18172435462474823]},{"id":"tv4_016","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002340465784072876,"mean_abs_dp":0.0013190582394599915,"oracle_top2_margin":0.1596996784210205,"near_tie":false,"probs":[0.16293670237064362,0.24424400925636292,0.19314426183700562,0.3996749818325043]},{"id":"tv4_017","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004297226667404175,"mean_abs_dp":0.0021486058831214905,"oracle_top2_margin":0.01221814751625061,"near_tie":true,"probs":[0.2615928649902344,0.23841449618339539,0.2568325996398926,0.24316006898880005]},{"id":"tv4_018","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0082053542137146,"mean_abs_dp":0.00410268921405077,"oracle_top2_margin":0.7636825442314148,"near_tie":false,"probs":[0.012926855124533176,0.09372782707214355,0.8723523020744324,0.02099299244582653]},{"id":"tv4_019","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002485021948814392,"mean_abs_dp":0.0014908500015735626,"oracle_top2_margin":0.06632544100284576,"near_tie":false,"probs":[0.2322787046432495,0.3032502830028534,0.2326459437608719,0.2318250834941864]},{"id":"tv4_020","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.017518997192382812,"mean_abs_dp":0.00875951093621552,"oracle_top2_margin":0.1631980538368225,"near_tie":false,"probs":[0.541876494884491,0.09886481612920761,0.34705328941345215,0.012205332517623901]},{"id":"tv4_021","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002603888511657715,"mean_abs_dp":0.0013019274920225143,"oracle_top2_margin":0.4488777816295624,"near_tie":false,"probs":[0.08708503842353821,0.159738227725029,0.14079958200454712,0.6123771667480469]},{"id":"tv4_022","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005656331777572632,"mean_abs_dp":0.004131171852350235,"oracle_top2_margin":0.05077701807022095,"near_tie":false,"probs":[0.3618391454219818,0.3128662407398224,0.13916915655136108,0.18612539768218994]},{"id":"tv4_023","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":1,"argmax_equal":false,"max_abs_dp":0.026517271995544434,"mean_abs_dp":0.013258632272481918,"oracle_top2_margin":0.0019783377647399902,"near_tie":true,"probs":[0.39101436734199524,0.34842997789382935,0.14739468693733215,0.11316091567277908]},{"id":"tv4_024","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0033638477325439453,"mean_abs_dp":0.001681930385529995,"oracle_top2_margin":0.015634745359420776,"near_tie":true,"probs":[0.3425765931606293,0.2676292061805725,0.03687611594796181,0.3529181182384491]},{"id":"tv4_025","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0023564398288726807,"mean_abs_dp":0.0016117431223392487,"oracle_top2_margin":0.07681050896644592,"near_tie":false,"probs":[0.3022761940956116,0.38057610392570496,0.101571224629879,0.21557646989822388]},{"id":"tv4_026","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002385959029197693,"mean_abs_dp":0.0013521946966648102,"oracle_top2_margin":0.06704592704772949,"near_tie":false,"probs":[0.1976228952407837,0.23586657643318176,0.24932774901390076,0.3171828091144562]},{"id":"tv4_027","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001272737979888916,"mean_abs_dp":0.0007210224866867065,"oracle_top2_margin":0.009622842073440552,"near_tie":true,"probs":[0.16761399805545807,0.3338010907173157,0.34227389097213745,0.1563110202550888]},{"id":"tv4_028","q":0,"variant":"base","tokens":290,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004708617925643921,"mean_abs_dp":0.003032080829143524,"oracle_top2_margin":0.01672416925430298,"near_tie":true,"probs":[0.16981390118598938,0.31957295536994934,0.21111741662025452,0.29949572682380676]},{"id":"tv4_029","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002639979124069214,"mean_abs_dp":0.0013200193643569946,"oracle_top2_margin":0.04676586389541626,"near_tie":false,"probs":[0.3030582368373871,0.34860897064208984,0.16304950416088104,0.1852833479642868]},{"id":"tv4_030","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00505964457988739,"mean_abs_dp":0.003268815577030182,"oracle_top2_margin":0.559301495552063,"near_tie":false,"probs":[0.10806303471326828,0.04458969086408615,0.7003349661827087,0.14701229333877563]},{"id":"tv4_031","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006056368350982666,"mean_abs_dp":0.00302819162607193,"oracle_top2_margin":0.2643141746520996,"near_tie":false,"probs":[0.22904568910598755,0.48348599672317505,0.1864677369594574,0.10100056976079941]},{"id":"tv4_032","q":0,"variant":"base","tokens":135,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001034855842590332,"mean_abs_dp":0.0005827480927109718,"oracle_top2_margin":0.07778683304786682,"near_tie":false,"probs":[0.05781615152955055,0.1917404979467392,0.4133433401584625,0.3371000289916992]},{"id":"tv4_033","q":0,"variant":"base","tokens":293,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002655714750289917,"mean_abs_dp":0.0013278704136610031,"oracle_top2_margin":0.10813379287719727,"near_tie":false,"probs":[0.08106568455696106,0.09701375663280487,0.35875385999679565,0.4631666839122772]},{"id":"tv4_034","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003359675407409668,"mean_abs_dp":0.0018390985205769539,"oracle_top2_margin":0.1505841612815857,"near_tie":false,"probs":[0.09987957775592804,0.05218437314033508,0.5025705099105835,0.34536558389663696]},{"id":"tv4_035","q":0,"variant":"base","tokens":108,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0016131401062011719,"mean_abs_dp":0.0008779503405094147,"oracle_top2_margin":0.18563473224639893,"near_tie":false,"probs":[0.18778279423713684,0.2152312994003296,0.3993956446647644,0.19759026169776917]},{"id":"tv4_036","q":0,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.014157116413116455,"mean_abs_dp":0.007078571245074272,"oracle_top2_margin":0.07235273718833923,"near_tie":false,"probs":[0.40009188652038574,0.34676405787467957,0.18316815793514252,0.06997591257095337]},{"id":"tv4_037","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.010501444339752197,"mean_abs_dp":0.005250699818134308,"oracle_top2_margin":0.05972352623939514,"near_tie":false,"probs":[0.20538513362407684,0.25922515988349915,0.20444345474243164,0.33094629645347595]},{"id":"tv4_038","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006163731217384338,"mean_abs_dp":0.0035173483192920685,"oracle_top2_margin":0.16796669363975525,"near_tie":false,"probs":[0.4131634533405304,0.24787522852420807,0.1874740868806839,0.15148721635341644]},{"id":"tv4_039","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003702901303768158,"mean_abs_dp":0.0002543441951274872,"oracle_top2_margin":0.35326528549194336,"near_tie":false,"probs":[0.29289621114730835,0.6460286378860474,0.03829319775104523,0.022781992331147194]},{"id":"tv4_040","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002925373613834381,"mean_abs_dp":0.0015857573598623276,"oracle_top2_margin":0.052141427993774414,"near_tie":false,"probs":[0.10596513003110886,0.33816906809806824,0.16751015186309814,0.38835564255714417]},{"id":"tv4_041","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019587576389312744,"mean_abs_dp":0.0010231509804725647,"oracle_top2_margin":0.06237095594406128,"near_tie":false,"probs":[0.1406014859676361,0.3599461317062378,0.29663458466529846,0.20281781256198883]},{"id":"tv4_042","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006620258092880249,"mean_abs_dp":0.003310123458504677,"oracle_top2_margin":0.02623698115348816,"near_tie":false,"probs":[0.32736849784851074,0.09917232394218445,0.30514857172966003,0.26831069588661194]},{"id":"tv4_043","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0021358877420425415,"mean_abs_dp":0.0010679364204406738,"oracle_top2_margin":0.12820479273796082,"near_tie":false,"probs":[0.3812345564365387,0.25240951776504517,0.22657053172588348,0.13978540897369385]},{"id":"tv4_044","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0042781829833984375,"mean_abs_dp":0.0021390793845057487,"oracle_top2_margin":0.7342252731323242,"near_tie":false,"probs":[0.04234972968697548,0.0782523900270462,0.0612364187836647,0.8181614875793457]},{"id":"tv4_045","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001821611076593399,"mean_abs_dp":0.0012429151684045792,"oracle_top2_margin":0.311641126871109,"near_tie":false,"probs":[0.03737141564488411,0.03630131110548973,0.619089663028717,0.3072375953197479]},{"id":"tv4_046","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011722967028617859,"mean_abs_dp":0.0006329112220555544,"oracle_top2_margin":0.8037196397781372,"near_tie":false,"probs":[0.8857616186141968,0.019194120541214943,0.0841749757528305,0.010869305580854416]},{"id":"tv4_047","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006850004196166992,"mean_abs_dp":0.003425002098083496,"oracle_top2_margin":0.10399001836776733,"near_tie":false,"probs":[0.30345404148101807,0.3945807218551636,0.12563566863536835,0.17632947862148285]},{"id":"tv4_048","q":0,"variant":"base","tokens":119,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002561911940574646,"mean_abs_dp":0.001280955970287323,"oracle_top2_margin":0.040600210428237915,"near_tie":false,"probs":[0.20809054374694824,0.3118944466114044,0.2086552232503891,0.27135974168777466]},{"id":"tv4_049","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004991844296455383,"mean_abs_dp":0.0026232730597257614,"oracle_top2_margin":0.14964529871940613,"near_tie":false,"probs":[0.21970932185649872,0.42251208424568176,0.08254785090684891,0.2752307951450348]},{"id":"tv4_050","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004951536655426025,"mean_abs_dp":0.0024757692590355873,"oracle_top2_margin":0.2277095913887024,"near_tie":false,"probs":[0.3335062265396118,0.5522769093513489,0.045247480273246765,0.06896934658288956]},{"id":"tv4_051","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025317519903182983,"mean_abs_dp":0.0017975633963942528,"oracle_top2_margin":0.006280571222305298,"near_tie":true,"probs":[0.05768092721700668,0.18390388786792755,0.3830457925796509,0.3753693401813507]},{"id":"tv4_052","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001508273184299469,"mean_abs_dp":0.000776803120970726,"oracle_top2_margin":0.5888446569442749,"near_tie":false,"probs":[0.7166076898574829,0.10301590710878372,0.05554603785276413,0.12483035773038864]},{"id":"tv4_053","q":0,"variant":"base","tokens":358,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0043964385986328125,"mean_abs_dp":0.0023812614381313324,"oracle_top2_margin":0.06682348251342773,"near_tie":false,"probs":[0.32807213068008423,0.2624736726284027,0.19111399352550507,0.21834023296833038]},{"id":"tv4_054","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":3,"argmax_equal":false,"max_abs_dp":0.0043344199657440186,"mean_abs_dp":0.0021838508546352386,"oracle_top2_margin":0.0028791427612304688,"near_tie":true,"probs":[0.13899265229701996,0.276436984539032,0.2924071252346039,0.2921631932258606]},{"id":"tv4_055","q":0,"variant":"base","tokens":189,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004767805337905884,"mean_abs_dp":0.003096851520240307,"oracle_top2_margin":0.14189845323562622,"near_tie":false,"probs":[0.27580463886260986,0.4152843654155731,0.2627182602882385,0.04619278758764267]},{"id":"tv4_056","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003444582223892212,"mean_abs_dp":0.0022194385528564453,"oracle_top2_margin":0.1477002501487732,"near_tie":false,"probs":[0.25762757658958435,0.21303841471672058,0.4004143178462982,0.12891969084739685]},{"id":"tv4_057","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0017376244068145752,"mean_abs_dp":0.000996941700577736,"oracle_top2_margin":0.042515188455581665,"near_tie":false,"probs":[0.2683759331703186,0.2988198697566986,0.08929335325956345,0.34351080656051636]},{"id":"tv4_058","q":0,"variant":"base","tokens":129,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0025154054164886475,"mean_abs_dp":0.001257680356502533,"oracle_top2_margin":0.045424774289131165,"near_tie":false,"probs":[0.228025883436203,0.291144996881485,0.24276867508888245,0.23806050419807434]},{"id":"tv4_059","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010400712490081787,"mean_abs_dp":0.005200352519750595,"oracle_top2_margin":0.11904534697532654,"near_tie":false,"probs":[0.423258900642395,0.19526128470897675,0.2875556945800781,0.09392406046390533]},{"id":"tv4x_emotion_00","q":0,"variant":"base","tokens":42,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.010186612606048584,"mean_abs_dp":0.0036469921469688416,"oracle_top2_margin":0.056429117918014526,"near_tie":false,"probs":[0.09872494637966156,0.2784610390663147,0.3501954972743988,0.06340787559747696,0.05952855944633484,0.14968203008174896]},{"id":"tv4x_emotion_01","q":0,"variant":"base","tokens":38,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002611905336380005,"mean_abs_dp":0.001686019202073415,"oracle_top2_margin":0.0766339898109436,"near_tie":false,"probs":[0.16157932579517365,0.10680545121431351,0.07480307668447495,0.31393706798553467,0.11076702922582626,0.23210805654525757]},{"id":"tv4x_emotion_02","q":0,"variant":"base","tokens":36,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024014487862586975,"mean_abs_dp":0.0008313328338166078,"oracle_top2_margin":0.6756487488746643,"near_tie":false,"probs":[0.7564907670021057,0.02019726298749447,0.01469787210226059,0.08054830133914948,0.081187903881073,0.04687798395752907]},{"id":"tv4x_emotion_03","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010986745357513428,"mean_abs_dp":0.0036697331815958023,"oracle_top2_margin":0.4306091070175171,"near_tie":false,"probs":[0.6868003010749817,0.03416033834218979,0.23603981733322144,0.015908146277070045,0.01533554308116436,0.011755802668631077]},{"id":"tv4x_emotion_04","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00405508279800415,"mean_abs_dp":0.0013516925585766633,"oracle_top2_margin":0.8624813556671143,"near_tie":false,"probs":[0.0052094911225140095,0.9148070812225342,0.045825496315956116,0.006981291808187962,0.009106445126235485,0.018070170655846596]},{"id":"tv4x_emotion_05","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.012595832347869873,"mean_abs_dp":0.004198603332042694,"oracle_top2_margin":0.10235360264778137,"near_tie":false,"probs":[0.19455210864543915,0.292235404253006,0.04908927530050278,0.37850138545036316,0.03293079137802124,0.05269096791744232]},{"id":"tv4x_emotion_06","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004428446292877197,"mean_abs_dp":0.001476150006055832,"oracle_top2_margin":0.3278556764125824,"near_tie":false,"probs":[0.017386019229888916,0.24068039655685425,0.57486492395401,0.03561976179480553,0.035550717264413834,0.09589824080467224]},{"id":"tv4x_emotion_07","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003618597984313965,"mean_abs_dp":0.0017231002760430176,"oracle_top2_margin":0.6330275535583496,"near_tie":false,"probs":[0.7235214710235596,0.025039702653884888,0.015316609293222427,0.08842601627111435,0.0831325501203537,0.06456364691257477]},{"id":"tv4x_emotion_08","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.010573267936706543,"mean_abs_dp":0.0035244235768914223,"oracle_top2_margin":0.1790621280670166,"near_tie":false,"probs":[0.02169351652264595,0.5656298995018005,0.3673349618911743,0.012971069663763046,0.01156502403318882,0.020805547013878822]},{"id":"tv4x_emotion_09","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":5,"argmax_oracle":5,"argmax_equal":true,"max_abs_dp":0.003945499658584595,"mean_abs_dp":0.0016937019924322765,"oracle_top2_margin":0.02565602958202362,"near_tie":false,"probs":[0.19008572399616241,0.03994644060730934,0.14287656545639038,0.2064003050327301,0.19341902434825897,0.22727195918560028]},{"id":"tv4x_emotion_10","q":0,"variant":"base","tokens":40,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020286738872528076,"mean_abs_dp":0.0008536924918492635,"oracle_top2_margin":0.034467101097106934,"near_tie":false,"probs":[0.3225802481174469,0.05048045888543129,0.08041079342365265,0.2864372730255127,0.11588969081640244,0.14420151710510254]},{"id":"tv4x_emotion_11","q":0,"variant":"base","tokens":48,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005059242248535156,"mean_abs_dp":0.0016864410135895014,"oracle_top2_margin":0.7190135717391968,"near_tie":false,"probs":[0.007621716242283583,0.8335050940513611,0.10573771595954895,0.010632060468196869,0.009405425749719143,0.03309790417551994]},{"id":"tv4x_emotion_12","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003951072692871094,"mean_abs_dp":0.0014341383551557858,"oracle_top2_margin":0.28088271617889404,"near_tie":false,"probs":[0.025351937860250473,0.3050636053085327,0.5895460844039917,0.02877296879887581,0.03067423775792122,0.02059120498597622]},{"id":"tv4x_emotion_13","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005307137966156006,"mean_abs_dp":0.0026750322431325912,"oracle_top2_margin":0.2596837878227234,"near_tie":false,"probs":[0.4466152489185333,0.09960184246301651,0.11783985048532486,0.03147852420806885,0.10876510292291641,0.19569948315620422]},{"id":"tv4x_emotion_14","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00044433027505874634,"mean_abs_dp":0.00014810617237041393,"oracle_top2_margin":0.7910157442092896,"near_tie":false,"probs":[0.0027697025798261166,0.09455547481775284,0.8850083351135254,0.006491268053650856,0.006076479330658913,0.005098724737763405]},{"id":"tv4x_emotion_15","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000925600528717041,"mean_abs_dp":0.0003085383990158637,"oracle_top2_margin":0.25465235114097595,"near_tie":false,"probs":[0.005487137474119663,0.5967130661010742,0.3433379828929901,0.010770267806947231,0.011775819584727287,0.03191567584872246]},{"id":"tv4x_emotion_16","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019068121910095215,"mean_abs_dp":0.000662381915996472,"oracle_top2_margin":0.8546378016471863,"near_tie":false,"probs":[0.8951876163482666,0.008829527534544468,0.024034913629293442,0.03813852369785309,0.016791576519608498,0.017017852514982224]},{"id":"tv4x_emotion_17","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019863396883010864,"mean_abs_dp":0.0013434973855813344,"oracle_top2_margin":0.2597692310810089,"near_tie":false,"probs":[0.44867244362831116,0.02429567649960518,0.18981455266475677,0.1331474483013153,0.06493488699197769,0.13913494348526]},{"id":"tv4x_emotion_18","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0034050345420837402,"mean_abs_dp":0.0011350312270224094,"oracle_top2_margin":0.06271809339523315,"near_tie":false,"probs":[0.35473036766052246,0.014437204226851463,0.03851829841732979,0.4124804437160492,0.10635685175657272,0.07347685843706131]},{"id":"tv4x_emotion_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0033885836601257324,"mean_abs_dp":0.0011295182630419731,"oracle_top2_margin":0.4918869137763977,"near_tie":false,"probs":[0.6298291087150574,0.028506159782409668,0.04688236489892006,0.13400334119796753,0.09594196826219559,0.06483707576990128]},{"id":"tv4x_tweet_offensive_00","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004985332489013672,"mean_abs_dp":0.0004985332489013672,"oracle_top2_margin":0.4777187407016754,"near_tie":false,"probs":[0.26064208149909973,0.7393578886985779]},{"id":"tv4x_tweet_offensive_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0022057071328163147,"mean_abs_dp":0.0022056885063648224,"oracle_top2_margin":0.7617751359939575,"near_tie":false,"probs":[0.11690669506788254,0.8830932378768921]},{"id":"tv4x_tweet_offensive_02","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009493231773376465,"mean_abs_dp":0.0009493082761764526,"oracle_top2_margin":0.6824735403060913,"near_tie":false,"probs":[0.840287446975708,0.1597125232219696]},{"id":"tv4x_tweet_offensive_03","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010250210762023926,"mean_abs_dp":0.0010249912738800049,"oracle_top2_margin":0.3328827917575836,"near_tie":false,"probs":[0.667466402053833,0.3325336277484894]},{"id":"tv4x_tweet_offensive_04","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001671314239501953,"mean_abs_dp":0.00016711652278900146,"oracle_top2_margin":0.2086932361125946,"near_tie":false,"probs":[0.6045137643814087,0.3954862952232361]},{"id":"tv4x_tweet_offensive_05","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018425583839416504,"mean_abs_dp":0.0018425583839416504,"oracle_top2_margin":0.38322776556015015,"near_tie":false,"probs":[0.6934564113616943,0.3065435290336609]},{"id":"tv4x_tweet_offensive_06","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011975765228271484,"mean_abs_dp":0.0011975765228271484,"oracle_top2_margin":0.028999686241149902,"near_tie":false,"probs":[0.5156974196434021,0.4843025803565979]},{"id":"tv4x_tweet_offensive_07","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005835294723510742,"mean_abs_dp":0.0005835145711898804,"oracle_top2_margin":0.020770907402038574,"near_tie":false,"probs":[0.489031046628952,0.5109689831733704]},{"id":"tv4x_tweet_offensive_08","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003588080406188965,"mean_abs_dp":0.0035880357027053833,"oracle_top2_margin":0.2947831451892853,"near_tie":false,"probs":[0.6509796380996704,0.34902042150497437]},{"id":"tv4x_tweet_offensive_09","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009412467479705811,"mean_abs_dp":0.0009412020444869995,"oracle_top2_margin":0.07451754808425903,"near_tie":false,"probs":[0.4636824429035187,0.5363175868988037]},{"id":"tv4x_tweet_offensive_10","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004619181156158447,"mean_abs_dp":0.00461915135383606,"oracle_top2_margin":0.1919628381729126,"near_tie":false,"probs":[0.39939945936203003,0.6006006002426147]},{"id":"tv4x_tweet_offensive_11","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.843273639678955e-05,"mean_abs_dp":1.8425285816192627e-05,"oracle_top2_margin":0.6438125371932983,"near_tie":false,"probs":[0.8219246864318848,0.17807532846927643]},{"id":"tv4x_tweet_offensive_12","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001556694507598877,"mean_abs_dp":0.001556694507598877,"oracle_top2_margin":0.03468778729438782,"near_tie":false,"probs":[0.4842127859592438,0.5157871842384338]},{"id":"tv4x_tweet_offensive_13","q":0,"variant":"base","tokens":43,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010249018669128418,"mean_abs_dp":0.001024872064590454,"oracle_top2_margin":0.1575031578540802,"near_tie":false,"probs":[0.5777267217636108,0.42227330803871155]},{"id":"tv4x_tweet_offensive_14","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005565226078033447,"mean_abs_dp":0.005565226078033447,"oracle_top2_margin":0.08823934197425842,"near_tie":false,"probs":[0.46144554018974304,0.5385544300079346]},{"id":"tv4x_tweet_offensive_15","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005900859832763672,"mean_abs_dp":0.0005900859832763672,"oracle_top2_margin":0.23108571767807007,"near_tie":false,"probs":[0.616132915019989,0.3838670253753662]},{"id":"tv4x_tweet_offensive_16","q":0,"variant":"base","tokens":104,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.8908252716064453e-05,"mean_abs_dp":2.8908252716064453e-05,"oracle_top2_margin":0.5775153040885925,"near_tie":false,"probs":[0.2112712413072586,0.7887287139892578]},{"id":"tv4x_tweet_offensive_17","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003132462501525879,"mean_abs_dp":0.003132462501525879,"oracle_top2_margin":0.15798139572143555,"near_tie":false,"probs":[0.4241417646408081,0.5758582353591919]},{"id":"tv4x_tweet_offensive_18","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004009127616882324,"mean_abs_dp":0.004009127616882324,"oracle_top2_margin":0.09931710362434387,"near_tie":false,"probs":[0.4543505609035492,0.5456494092941284]},{"id":"tv4x_tweet_offensive_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003488302230834961,"mean_abs_dp":0.0034882724285125732,"oracle_top2_margin":0.18660569190979004,"near_tie":false,"probs":[0.5967910885810852,0.40320885181427]},{"id":"tv4x_qnli_00","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008150413632392883,"mean_abs_dp":0.0008150376379489899,"oracle_top2_margin":0.7700709700584412,"near_tie":false,"probs":[0.11414948105812073,0.8858505487442017]},{"id":"tv4x_qnli_01","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003735840320587158,"mean_abs_dp":0.0037358254194259644,"oracle_top2_margin":0.5029501914978027,"near_tie":false,"probs":[0.7552109360694885,0.24478912353515625]},{"id":"tv4x_qnli_02","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008853554725646973,"mean_abs_dp":0.008853539824485779,"oracle_top2_margin":0.6557972431182861,"near_tie":false,"probs":[0.1809549182653427,0.8190450668334961]},{"id":"tv4x_qnli_03","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008892565965652466,"mean_abs_dp":0.0008892491459846497,"oracle_top2_margin":0.6964544057846069,"near_tie":false,"probs":[0.8491164445877075,0.1508835405111313]},{"id":"tv4x_qnli_04","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006228461861610413,"mean_abs_dp":0.000622827559709549,"oracle_top2_margin":0.9449087381362915,"near_tie":false,"probs":[0.028168465942144394,0.9718315601348877]},{"id":"tv4x_qnli_05","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.5391578674316406e-05,"mean_abs_dp":2.5384128093719482e-05,"oracle_top2_margin":0.8874053359031677,"near_tie":false,"probs":[0.05632273480296135,0.9436773061752319]},{"id":"tv4x_qnli_06","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005094774067401886,"mean_abs_dp":0.0050947405397892,"oracle_top2_margin":0.8620169758796692,"near_tie":false,"probs":[0.07408623397350311,0.9259137511253357]},{"id":"tv4x_qnli_07","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.417533874511719e-05,"mean_abs_dp":9.417533874511719e-05,"oracle_top2_margin":0.18525761365890503,"near_tie":false,"probs":[0.5927229523658752,0.40727698802948]},{"id":"tv4x_qnli_08","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0067716240882873535,"mean_abs_dp":0.0067716240882873535,"oracle_top2_margin":0.35944250226020813,"near_tie":false,"probs":[0.6729496121406555,0.3270503580570221]},{"id":"tv4x_qnli_09","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006310760974884033,"mean_abs_dp":0.006310760974884033,"oracle_top2_margin":0.2295372486114502,"near_tie":false,"probs":[0.6084578633308411,0.39154213666915894]},{"id":"tv4x_qnli_10","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003969728946685791,"mean_abs_dp":0.003969714045524597,"oracle_top2_margin":0.43258559703826904,"near_tie":false,"probs":[0.7123230695724487,0.2876769006252289]},{"id":"tv4x_qnli_11","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00406646728515625,"mean_abs_dp":0.00406646728515625,"oracle_top2_margin":0.409393310546875,"near_tie":false,"probs":[0.29936981201171875,0.7006301879882812]},{"id":"tv4x_qnli_12","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008989036083221436,"mean_abs_dp":0.008989006280899048,"oracle_top2_margin":0.10790503025054932,"near_tie":false,"probs":[0.5449634790420532,0.455036461353302]},{"id":"tv4x_qnli_13","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003126382827758789,"mean_abs_dp":0.0031263530254364014,"oracle_top2_margin":0.35351717472076416,"near_tie":false,"probs":[0.6798849105834961,0.32011502981185913]},{"id":"tv4x_qnli_14","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0026055574417114258,"mean_abs_dp":0.002605527639389038,"oracle_top2_margin":0.28486254811286926,"near_tie":false,"probs":[0.6450368165969849,0.3549632132053375]},{"id":"tv4x_qnli_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013523995876312256,"mean_abs_dp":0.0013523846864700317,"oracle_top2_margin":0.9124271273612976,"near_tie":false,"probs":[0.045138828456401825,0.9548611640930176]},{"id":"tv4x_qnli_16","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0022853687405586243,"mean_abs_dp":0.002285365015268326,"oracle_top2_margin":0.7679163217544556,"near_tie":false,"probs":[0.11375647783279419,0.8862435221672058]},{"id":"tv4x_qnli_17","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024195313453674316,"mean_abs_dp":0.002419516444206238,"oracle_top2_margin":0.7737533450126648,"near_tie":false,"probs":[0.11070380359888077,0.8892961740493774]},{"id":"tv4x_qnli_18","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009066462516784668,"mean_abs_dp":0.000906631350517273,"oracle_top2_margin":0.17956295609474182,"near_tie":false,"probs":[0.411125123500824,0.5888748168945312]},{"id":"tv4x_qnli_19","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0045626163482666016,"mean_abs_dp":0.0045626163482666016,"oracle_top2_margin":0.44058558344841003,"near_tie":false,"probs":[0.7157301902770996,0.2842698395252228]},{"id":"tv4x_paws_00","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006009280681610107,"mean_abs_dp":0.006009265780448914,"oracle_top2_margin":0.38785746693611145,"near_tie":false,"probs":[0.3120805025100708,0.6879194378852844]},{"id":"tv4x_paws_01","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007960021495819092,"mean_abs_dp":0.007959991693496704,"oracle_top2_margin":0.483722984790802,"near_tie":false,"probs":[0.2660984992980957,0.7339015007019043]},{"id":"tv4x_paws_02","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006059587001800537,"mean_abs_dp":0.006059557199478149,"oracle_top2_margin":0.21775135397911072,"near_tie":false,"probs":[0.602816104888916,0.3971838653087616]},{"id":"tv4x_paws_03","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0068367719650268555,"mean_abs_dp":0.0068367719650268555,"oracle_top2_margin":0.2834298312664032,"near_tie":false,"probs":[0.36512187123298645,0.6348781585693359]},{"id":"tv4x_paws_04","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006959080696105957,"mean_abs_dp":0.006959050893783569,"oracle_top2_margin":0.210747629404068,"near_tie":false,"probs":[0.40158525109291077,0.5984147787094116]},{"id":"tv4x_paws_05","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008675038814544678,"mean_abs_dp":0.00867500901222229,"oracle_top2_margin":0.5033949613571167,"near_tie":false,"probs":[0.25697752833366394,0.7430225014686584]},{"id":"tv4x_paws_06","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008843556046485901,"mean_abs_dp":0.008843518793582916,"oracle_top2_margin":0.5668308734893799,"near_tie":false,"probs":[0.22542810440063477,0.77457195520401]},{"id":"tv4x_paws_07","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005845487117767334,"mean_abs_dp":0.005845487117767334,"oracle_top2_margin":0.4673820734024048,"near_tie":false,"probs":[0.27215445041656494,0.7278455495834351]},{"id":"tv4x_paws_08","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013265907764434814,"mean_abs_dp":0.0013265758752822876,"oracle_top2_margin":0.3115227520465851,"near_tie":false,"probs":[0.34556519985198975,0.6544348001480103]},{"id":"tv4x_paws_09","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004891932010650635,"mean_abs_dp":0.004891887307167053,"oracle_top2_margin":0.2519233822822571,"near_tie":false,"probs":[0.3789301812648773,0.6210697889328003]},{"id":"tv4x_paws_10","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005866587162017822,"mean_abs_dp":0.0005866438150405884,"oracle_top2_margin":0.3951742947101593,"near_tie":false,"probs":[0.30299949645996094,0.6970005035400391]},{"id":"tv4x_paws_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004056006669998169,"mean_abs_dp":0.004055991768836975,"oracle_top2_margin":0.16973772644996643,"near_tie":false,"probs":[0.41918712854385376,0.5808128714561462]},{"id":"tv4x_paws_12","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006837636232376099,"mean_abs_dp":0.006837621331214905,"oracle_top2_margin":0.43726658821105957,"near_tie":false,"probs":[0.2882043421268463,0.7117956876754761]},{"id":"tv4x_paws_13","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002532064914703369,"mean_abs_dp":0.0025320500135421753,"oracle_top2_margin":0.33207952976226807,"near_tie":false,"probs":[0.33649227023124695,0.6635076999664307]},{"id":"tv4x_paws_14","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007983386516571045,"mean_abs_dp":0.007983356714248657,"oracle_top2_margin":0.22963646054267883,"near_tie":false,"probs":[0.39316514134407043,0.606834888458252]},{"id":"tv4x_paws_15","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0033729076385498047,"mean_abs_dp":0.003372892737388611,"oracle_top2_margin":0.38555964827537537,"near_tie":false,"probs":[0.3105930685997009,0.6894069314002991]},{"id":"tv4x_paws_16","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004571139812469482,"mean_abs_dp":0.004571124911308289,"oracle_top2_margin":0.42694148421287537,"near_tie":false,"probs":[0.2911003828048706,0.7088996171951294]},{"id":"tv4x_paws_17","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00028961896896362305,"mean_abs_dp":0.00028958916664123535,"oracle_top2_margin":0.20490708947181702,"near_tie":false,"probs":[0.39725688099861145,0.6027431488037109]},{"id":"tv4x_paws_18","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006960034370422363,"mean_abs_dp":0.0006960034370422363,"oracle_top2_margin":0.20717203617095947,"near_tie":false,"probs":[0.604282021522522,0.395717978477478]},{"id":"tv4x_paws_19","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002976030111312866,"mean_abs_dp":0.0029760152101516724,"oracle_top2_margin":0.25582870841026306,"near_tie":false,"probs":[0.3691096305847168,0.6308903694152832]},{"id":"tv4x_sciq_00","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005608797073364258,"mean_abs_dp":0.0002804504765663296,"oracle_top2_margin":0.9667011499404907,"near_tie":false,"probs":[0.001758051454089582,0.9789263606071472,0.013205413706600666,0.0061102015897631645]},{"id":"tv4x_sciq_01","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0060510337352752686,"mean_abs_dp":0.003135092556476593,"oracle_top2_margin":0.11972805857658386,"near_tie":false,"probs":[0.1301324963569641,0.2999867796897888,0.13859768211841583,0.43128305673599243]},{"id":"tv4x_sciq_02","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00013124942779541016,"mean_abs_dp":7.359765004366636e-05,"oracle_top2_margin":0.9568026661872864,"near_tie":false,"probs":[0.9705632328987122,0.013532422482967377,0.00715647405013442,0.008747860789299011]},{"id":"tv4x_sciq_03","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0002657175064086914,"mean_abs_dp":0.00013285549357533455,"oracle_top2_margin":0.9384280443191528,"near_tie":false,"probs":[0.002785054035484791,0.003115270286798477,0.027586448937654495,0.9665132164955139]},{"id":"tv4x_sciq_04","q":0,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.019666969776153564,"mean_abs_dp":0.009833474643528461,"oracle_top2_margin":0.0839184820652008,"near_tie":false,"probs":[0.05537472292780876,0.4142361283302307,0.4640745520591736,0.06631466746330261]},{"id":"tv4x_sciq_05","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008951961994171143,"mean_abs_dp":0.004723438993096352,"oracle_top2_margin":0.3225005269050598,"near_tie":false,"probs":[0.5764788389205933,0.2676735520362854,0.1170770600438118,0.03877054154872894]},{"id":"tv4x_sciq_06","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.940696716308594e-05,"mean_abs_dp":5.3580530220642686e-05,"oracle_top2_margin":0.9903356432914734,"near_tie":false,"probs":[0.0017470872262492776,0.9933488965034485,0.00308487331494689,0.0018190792761743069]},{"id":"tv4x_sciq_07","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00015181303024291992,"mean_abs_dp":7.591379107907414e-05,"oracle_top2_margin":0.9827399849891663,"near_tie":false,"probs":[0.001617893110960722,0.9889233708381653,0.0030546230264008045,0.006404046900570393]},{"id":"tv4x_sciq_08","q":0,"variant":"base","tokens":162,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012099146842956543,"mean_abs_dp":0.0006049516960047185,"oracle_top2_margin":0.8441504836082458,"near_tie":false,"probs":[0.9135335087776184,0.0716085433959961,0.00316237541846931,0.01169554702937603]},{"id":"tv4x_sciq_09","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0011547207832336426,"mean_abs_dp":0.0005773771554231644,"oracle_top2_margin":0.7690727114677429,"near_tie":false,"probs":[0.058151908218860626,0.04908382147550583,0.06263919174671173,0.8301251530647278]},{"id":"tv4x_sciq_10","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0037484243512153625,"mean_abs_dp":0.0018742198590189219,"oracle_top2_margin":0.7186641693115234,"near_tie":false,"probs":[0.12195593863725662,0.015418317168951035,0.014893871732056141,0.8477318286895752]},{"id":"tv4x_sciq_11","q":0,"variant":"base","tokens":366,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.007786154747009277,"mean_abs_dp":0.0038930955925025046,"oracle_top2_margin":0.7560468316078186,"near_tie":false,"probs":[0.05173474922776222,0.10108571499586105,0.844752848148346,0.0024266685359179974]},{"id":"tv4x_sciq_12","q":0,"variant":"base","tokens":192,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013521909713745117,"mean_abs_dp":0.0006761078257113695,"oracle_top2_margin":0.872226357460022,"near_tie":false,"probs":[0.011433972977101803,0.012508459389209747,0.9230833053588867,0.052974265068769455]},{"id":"tv4x_sciq_13","q":0,"variant":"base","tokens":147,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.01044851541519165,"mean_abs_dp":0.0052242790115997195,"oracle_top2_margin":0.8092012405395508,"near_tie":false,"probs":[0.886976420879364,0.09715577960014343,0.005049346946179867,0.010818426497280598]},{"id":"tv4x_sciq_14","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0015496015548706055,"mean_abs_dp":0.0007748180069029331,"oracle_top2_margin":0.8467167019844055,"near_tie":false,"probs":[0.03812318295240402,0.009899204596877098,0.05392837151885033,0.8980492353439331]},{"id":"tv4x_sciq_15","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007801055908203125,"mean_abs_dp":0.0003900638548657298,"oracle_top2_margin":0.9471396207809448,"near_tie":false,"probs":[0.0055908747017383575,0.9653024673461914,0.009550430811941624,0.019556235522031784]},{"id":"tv4x_sciq_16","q":0,"variant":"base","tokens":335,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002252340316772461,"mean_abs_dp":0.001163287553936243,"oracle_top2_margin":0.45971256494522095,"near_tie":false,"probs":[0.22484616935253143,0.019670363515615463,0.07518827170133591,0.6802951693534851]},{"id":"tv4x_sciq_17","q":0,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.01787745952606201,"mean_abs_dp":0.008938733953982592,"oracle_top2_margin":0.3270365595817566,"near_tie":false,"probs":[0.5522506833076477,0.2554621696472168,0.17085455358028412,0.021432576701045036]},{"id":"tv4x_sciq_18","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005038127303123474,"mean_abs_dp":0.0026353932917118073,"oracle_top2_margin":0.5720489621162415,"near_tie":false,"probs":[0.6990587115287781,0.06721634417772293,0.10635385662317276,0.12737101316452026]},{"id":"tv4x_sciq_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012392997741699219,"mean_abs_dp":0.000619651866145432,"oracle_top2_margin":0.9556483626365662,"near_tie":false,"probs":[0.9736985564231873,0.0033913457300513983,0.002655232325196266,0.02025483176112175]},{"id":"tv4x_legacy_holdout_00","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014951825141906738,"mean_abs_dp":0.001495175063610077,"oracle_top2_margin":0.6975672245025635,"near_tie":false,"probs":[0.14972122013568878,0.8502787947654724]},{"id":"tv4x_legacy_holdout_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012741684913635254,"mean_abs_dp":0.0012741535902023315,"oracle_top2_margin":0.03042253851890564,"near_tie":false,"probs":[0.5139371156692505,0.4860628843307495]},{"id":"tv4x_legacy_holdout_02","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003974452614784241,"mean_abs_dp":0.003974445164203644,"oracle_top2_margin":0.6348230838775635,"near_tie":false,"probs":[0.17861400544643402,0.8213859796524048]},{"id":"tv4x_legacy_holdout_03","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.4036893844604492e-05,"mean_abs_dp":1.4021992683410645e-05,"oracle_top2_margin":0.04561471939086914,"near_tie":false,"probs":[0.5227933526039124,0.47720667719841003]},{"id":"tv4x_legacy_holdout_04","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0030246376991271973,"mean_abs_dp":0.0030246302485466003,"oracle_top2_margin":0.5876218676567078,"near_tie":false,"probs":[0.20316441357135773,0.7968355417251587]},{"id":"tv4x_legacy_holdout_05","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004510998725891113,"mean_abs_dp":0.004510968923568726,"oracle_top2_margin":0.0782371461391449,"near_tie":false,"probs":[0.5346075892448425,0.4653923809528351]},{"id":"tv4x_legacy_holdout_06","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005597501993179321,"mean_abs_dp":0.0055974870920181274,"oracle_top2_margin":0.5711351633071899,"near_tie":false,"probs":[0.22002989053726196,0.779970109462738]},{"id":"tv4x_legacy_holdout_07","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010971426963806152,"mean_abs_dp":0.010971412062644958,"oracle_top2_margin":0.7396279573440552,"near_tie":false,"probs":[0.8588425517082214,0.14115743339061737]},{"id":"tv4x_legacy_holdout_08","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010778307914733887,"mean_abs_dp":0.001077800989151001,"oracle_top2_margin":0.7547731399536133,"near_tie":false,"probs":[0.12153561413288116,0.8784643411636353]},{"id":"tv4x_legacy_holdout_09","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020726919174194336,"mean_abs_dp":0.002072662115097046,"oracle_top2_margin":0.24087288975715637,"near_tie":false,"probs":[0.38163623213768005,0.6183637976646423]},{"id":"tv4x_legacy_holdout_10","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00274580717086792,"mean_abs_dp":0.002745792269706726,"oracle_top2_margin":0.6889723539352417,"near_tie":false,"probs":[0.15276804566383362,0.8472319841384888]},{"id":"tv4x_legacy_holdout_11","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00015604496002197266,"mean_abs_dp":0.00015601515769958496,"oracle_top2_margin":0.1864413619041443,"near_tie":false,"probs":[0.5930646657943726,0.40693533420562744]},{"id":"tv4x_legacy_holdout_12","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012638270854949951,"mean_abs_dp":0.0012638121843338013,"oracle_top2_margin":0.7034794092178345,"near_tie":false,"probs":[0.14952410757541656,0.8504759073257446]},{"id":"tv4x_legacy_holdout_13","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.02400749921798706,"mean_abs_dp":0.024007484316825867,"oracle_top2_margin":0.19806906580924988,"near_tie":false,"probs":[0.3769580125808716,0.6230420470237732]},{"id":"tv4x_legacy_holdout_14","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003228902816772461,"mean_abs_dp":0.0032288730144500732,"oracle_top2_margin":0.5819854140281677,"near_tie":false,"probs":[0.2122361809015274,0.7877638339996338]},{"id":"tv4x_legacy_holdout_15","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.016536161303520203,"mean_abs_dp":0.016536138951778412,"oracle_top2_margin":0.6110415458679199,"near_tie":false,"probs":[0.7889846563339233,0.21101535856723785]},{"id":"tv4x_legacy_holdout_16","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.762411117553711e-05,"mean_abs_dp":4.757195711135864e-05,"oracle_top2_margin":0.6085996031761169,"near_tie":false,"probs":[0.19565266370773315,0.8043473958969116]},{"id":"tv4x_legacy_holdout_17","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005377531051635742,"mean_abs_dp":0.0005377233028411865,"oracle_top2_margin":0.2321089208126068,"near_tie":false,"probs":[0.615516722202301,0.3844832479953766]},{"id":"tv4x_legacy_holdout_18","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004385992884635925,"mean_abs_dp":0.0043859705328941345,"oracle_top2_margin":0.6196736097335815,"near_tie":false,"probs":[0.19454917311668396,0.8054508566856384]},{"id":"tv4x_legacy_holdout_19","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0182955265045166,"mean_abs_dp":0.018295519053936005,"oracle_top2_margin":0.57481849193573,"near_tie":false,"probs":[0.7691137194633484,0.23088623583316803]},{"id":"tv4x_composition_holdout_00","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00689290463924408,"mean_abs_dp":0.006892882287502289,"oracle_top2_margin":0.5304081439971924,"near_tie":false,"probs":[0.7720969319343567,0.22790299355983734]},{"id":"tv4x_composition_holdout_01","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0035462379455566406,"mean_abs_dp":0.0035462118685245514,"oracle_top2_margin":0.785683274269104,"near_tie":false,"probs":[0.8892953991889954,0.11070454865694046]},{"id":"tv4x_composition_holdout_02","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00689290463924408,"mean_abs_dp":0.006892882287502289,"oracle_top2_margin":0.5304081439971924,"near_tie":false,"probs":[0.7720969319343567,0.22790299355983734]},{"id":"tv4x_composition_holdout_03","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010257124900817871,"mean_abs_dp":0.010257124900817871,"oracle_top2_margin":0.45854881405830383,"near_tie":false,"probs":[0.7395315170288086,0.260468453168869]},{"id":"tv4x_composition_holdout_04","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020045042037963867,"mean_abs_dp":0.00020043551921844482,"oracle_top2_margin":0.3756886124610901,"near_tie":false,"probs":[0.3123561441898346,0.6876438856124878]},{"id":"tv4x_composition_holdout_05","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0023857206106185913,"mean_abs_dp":0.0023856982588768005,"oracle_top2_margin":0.7627792358398438,"near_tie":false,"probs":[0.8790039420127869,0.12099607288837433]},{"id":"tv4x_composition_holdout_06","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020045042037963867,"mean_abs_dp":0.00020043551921844482,"oracle_top2_margin":0.3756886124610901,"near_tie":false,"probs":[0.3123561441898346,0.6876438856124878]},{"id":"tv4x_composition_holdout_07","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006743669509887695,"mean_abs_dp":0.0006743371486663818,"oracle_top2_margin":0.3700430393218994,"near_tie":false,"probs":[0.3156527876853943,0.6843471527099609]},{"id":"tv4x_composition_holdout_08","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016316473484039307,"mean_abs_dp":0.0016316324472427368,"oracle_top2_margin":0.7355459928512573,"near_tie":false,"probs":[0.8694046139717102,0.13059532642364502]},{"id":"tv4x_composition_holdout_09","q":0,"variant":"base","tokens":167,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011724531650543213,"mean_abs_dp":0.0011724382638931274,"oracle_top2_margin":0.8914244174957275,"near_tie":false,"probs":[0.9445397853851318,0.05546022206544876]},{"id":"tv4x_composition_holdout_10","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016316473484039307,"mean_abs_dp":0.0016316324472427368,"oracle_top2_margin":0.7355459928512573,"near_tie":false,"probs":[0.8694046139717102,0.13059532642364502]},{"id":"tv4x_composition_holdout_11","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0021796822547912598,"mean_abs_dp":0.002179645001888275,"oracle_top2_margin":0.7365687489509583,"near_tie":false,"probs":[0.870464026927948,0.1295359879732132]},{"id":"tv4x_composition_holdout_12","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.024287700653076172,"mean_abs_dp":0.024287700653076172,"oracle_top2_margin":0.28672489523887634,"near_tie":false,"probs":[0.3809252679347992,0.6190747618675232]},{"id":"tv4x_composition_holdout_13","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005535900592803955,"mean_abs_dp":0.0055358558893203735,"oracle_top2_margin":0.8679777383804321,"near_tie":false,"probs":[0.07154696434736252,0.9284529685974121]},{"id":"tv4x_composition_holdout_14","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.024287700653076172,"mean_abs_dp":0.024287700653076172,"oracle_top2_margin":0.28672489523887634,"near_tie":false,"probs":[0.3809252679347992,0.6190747618675232]},{"id":"tv4x_composition_holdout_15","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.026461392641067505,"mean_abs_dp":0.02646137773990631,"oracle_top2_margin":0.1411517858505249,"near_tie":false,"probs":[0.45588549971580505,0.5441145300865173]},{"id":"tv4x_composition_holdout_16","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002110302448272705,"mean_abs_dp":0.002110302448272705,"oracle_top2_margin":0.7229863405227661,"near_tie":false,"probs":[0.14061711728572845,0.8593828678131104]},{"id":"tv4x_composition_holdout_17","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018329024314880371,"mean_abs_dp":0.0018328726291656494,"oracle_top2_margin":0.571626603603363,"near_tie":false,"probs":[0.21235382556915283,0.7876461744308472]},{"id":"tv4x_composition_holdout_18","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002110302448272705,"mean_abs_dp":0.002110302448272705,"oracle_top2_margin":0.7229863405227661,"near_tie":false,"probs":[0.14061711728572845,0.8593828678131104]},{"id":"tv4x_composition_holdout_19","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0026557445526123047,"mean_abs_dp":0.0026557445526123047,"oracle_top2_margin":0.719793438911438,"near_tie":false,"probs":[0.14275901019573212,0.8572409749031067]},{"id":"tv4s_00","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020653605461120605,"mean_abs_dp":0.0013769020636876423,"oracle_top2_margin":0.3157815933227539,"near_tie":false,"probs":[0.23423005640506744,0.5529204607009888,0.21284951269626617]},{"id":"tv4s_01","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004526972770690918,"mean_abs_dp":0.003017966945966085,"oracle_top2_margin":0.4067290723323822,"near_tie":false,"probs":[0.2560013234615326,0.6705372333526611,0.07346145063638687]},{"id":"tv4s_02","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009917765855789185,"mean_abs_dp":0.0006611893574396769,"oracle_top2_margin":0.11895173788070679,"near_tie":false,"probs":[0.31629687547683716,0.4356347620487213,0.24806837737560272]},{"id":"tv4s_03","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020799338817596436,"mean_abs_dp":0.0013865729173024495,"oracle_top2_margin":0.09898605942726135,"near_tie":false,"probs":[0.32994362711906433,0.428750216960907,0.24130606651306152]},{"id":"tv4s_04","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004687592387199402,"mean_abs_dp":0.0031250715255737305,"oracle_top2_margin":0.41046595573425293,"near_tie":false,"probs":[0.19677004218101501,0.6060600876808167,0.19716989994049072]},{"id":"tv4s_05","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017088055610656738,"mean_abs_dp":0.0011392136414845784,"oracle_top2_margin":0.5455268621444702,"near_tie":false,"probs":[0.19341154396533966,0.7423111796379089,0.06427731364965439]},{"id":"tv4s_06","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0045926570892333984,"mean_abs_dp":0.0030617515246073404,"oracle_top2_margin":0.024926811456680298,"near_tie":false,"probs":[0.2606208622455597,0.38601016998291016,0.35336893796920776]},{"id":"tv4s_07","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0034439116716384888,"mean_abs_dp":0.0022959460814793906,"oracle_top2_margin":0.07871943712234497,"near_tie":false,"probs":[0.34010761976242065,0.41821786761283875,0.2416745126247406]},{"id":"tv4s_08","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020674914121627808,"mean_abs_dp":0.0013783226410547893,"oracle_top2_margin":0.11677592992782593,"near_tie":false,"probs":[0.33105140924453735,0.44721487164497375,0.22173374891281128]},{"id":"tv4s_09","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004956483840942383,"mean_abs_dp":0.0033043324947357178,"oracle_top2_margin":0.05665457248687744,"near_tie":false,"probs":[0.34656816720962524,0.40231379866600037,0.2511180639266968]},{"id":"tv4s_10","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002914637327194214,"mean_abs_dp":0.0019431014855702717,"oracle_top2_margin":0.10606691241264343,"near_tie":false,"probs":[0.2995741069316864,0.40770184993743896,0.292724072933197]},{"id":"tv4s_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002370685338973999,"mean_abs_dp":0.001580427090326945,"oracle_top2_margin":0.0634344220161438,"near_tie":false,"probs":[0.33298707008361816,0.4007273018360138,0.2662856876850128]},{"id":"tv4s_12","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015564411878585815,"mean_abs_dp":0.001037632425626119,"oracle_top2_margin":0.4871361255645752,"near_tie":false,"probs":[0.2121465653181076,0.6983429193496704,0.0895104929804802]},{"id":"tv4s_13","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009128451347351074,"mean_abs_dp":0.0006085882584253947,"oracle_top2_margin":0.5980459451675415,"near_tie":false,"probs":[0.1698119044303894,0.7660450339317322,0.06414308398962021]},{"id":"tv4s_14","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011319220066070557,"mean_abs_dp":0.0007546146710713705,"oracle_top2_margin":0.01574954390525818,"near_tie":true,"probs":[0.3958556354045868,0.41349661350250244,0.19064776599407196]},{"id":"tv4s_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016981661319732666,"mean_abs_dp":0.0011321107546488445,"oracle_top2_margin":0.07872390747070312,"near_tie":false,"probs":[0.3343343436717987,0.4143408536911011,0.251324862241745]},{"id":"tv4s_16","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002271682024002075,"mean_abs_dp":0.0015144050121307373,"oracle_top2_margin":0.18242567777633667,"near_tie":false,"probs":[0.323442280292511,0.5071736574172974,0.16938401758670807]},{"id":"tv4s_17","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0042798519134521484,"mean_abs_dp":0.002853264411290487,"oracle_top2_margin":0.07194459438323975,"near_tie":false,"probs":[0.2530069947242737,0.41299203038215637,0.33400097489356995]},{"id":"tv4s_18","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004442095756530762,"mean_abs_dp":0.0029613872369130454,"oracle_top2_margin":0.3651939928531647,"near_tie":false,"probs":[0.2714269459247589,0.6452169418334961,0.0833560973405838]},{"id":"tv4s_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015572905540466309,"mean_abs_dp":0.001038206120332082,"oracle_top2_margin":0.29585587978363037,"near_tie":false,"probs":[0.3176325559616089,0.6158719062805176,0.06649555265903473]},{"id":"semif_a3f18f3a63d45345942b","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0026427656412124634,"mean_abs_dp":0.0017618549366792042,"oracle_top2_margin":0.6473773717880249,"near_tie":false,"probs":[0.020859764888882637,0.16328774392604828,0.8158525228500366],"hidden":{"min_pos_cos":0.999760318900587,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"semif_f40beba9088c8db8bbd6","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018395185470581055,"mean_abs_dp":0.0012263115495443344,"oracle_top2_margin":0.8147870898246765,"near_tie":false,"probs":[0.898598849773407,0.08017496019601822,0.021226204931735992]},{"id":"semif_0b43ea8e24e74d621f1b","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007878802716732025,"mean_abs_dp":0.0005252820750077566,"oracle_top2_margin":0.9045717716217041,"near_tie":false,"probs":[0.03737448900938034,0.9412896633148193,0.021335849538445473]},{"id":"semif_942ec65dac0bfb9874f9","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010996848344802856,"mean_abs_dp":0.007331204911073049,"oracle_top2_margin":0.03487250208854675,"near_tie":false,"probs":[0.49577122926712036,0.06405827403068542,0.4401704668998718]},{"id":"semif_e0c140e9222d1bce5647","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002534806728363037,"mean_abs_dp":0.0016898568719625473,"oracle_top2_margin":0.7138881683349609,"near_tie":false,"probs":[0.845664918422699,0.01797582022845745,0.13635924458503723]},{"id":"semif_8e8c3804a3c15ebb31e3","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0036323070526123047,"mean_abs_dp":0.0024214982986450195,"oracle_top2_margin":0.6479207277297974,"near_tie":false,"probs":[0.14305032789707184,0.7855930924415588,0.07135648280382156]},{"id":"semif_56c419b12cd37cbe790c","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0036872774362564087,"mean_abs_dp":0.0024581526716550193,"oracle_top2_margin":0.7378895282745361,"near_tie":false,"probs":[0.8283462524414062,0.07551781088113785,0.0961359515786171]},{"id":"semif_75b470586834df8c568d","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0071908533573150635,"mean_abs_dp":0.004793904721736908,"oracle_top2_margin":0.028810232877731323,"near_tie":false,"probs":[0.04106474295258522,0.47210219502449036,0.4868330657482147]},{"id":"semif_834f3b3e10cc61b33d25","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.01570206880569458,"mean_abs_dp":0.010468098645408949,"oracle_top2_margin":0.6174204349517822,"near_tie":false,"probs":[0.7779820561408997,0.03092591091990471,0.19109207391738892]},{"id":"semif_76c7a79b1ff2363a60c5","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000874936580657959,"mean_abs_dp":0.0005833053340514501,"oracle_top2_margin":0.9108183979988098,"near_tie":false,"probs":[0.9449470639228821,0.03553685545921326,0.01951613463461399]},{"id":"semif_36c71bb649b7d5eccb1f","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007202811539173126,"mean_abs_dp":0.0004801725347836812,"oracle_top2_margin":0.8975834846496582,"near_tie":false,"probs":[0.9331232309341431,0.032663293182849884,0.03421349078416824]},{"id":"semif_eac5121a4a1bf391a08c","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009585469961166382,"mean_abs_dp":0.0006390412648518881,"oracle_top2_margin":0.6141079664230347,"near_tie":false,"probs":[0.1610090285539627,0.7732524275779724,0.06573855131864548]},{"id":"semif_44574c44e243ee1b2f52","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013433098793029785,"mean_abs_dp":0.0008955287436644236,"oracle_top2_margin":0.7463966608047485,"near_tie":false,"probs":[0.10828288644552231,0.03926673159003258,0.8524503707885742]},{"id":"semif_2608370d01783735c3ea","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.01745012402534485,"mean_abs_dp":0.01163343588511149,"oracle_top2_margin":0.05861121416091919,"near_tie":false,"probs":[0.2897630035877228,0.37494027614593506,0.3352966904640198]},{"id":"semif_ec94ef909f7918695ab3","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002229034900665283,"mean_abs_dp":0.001486022025346756,"oracle_top2_margin":0.7390934228897095,"near_tie":false,"probs":[0.025761887431144714,0.11932258307933807,0.8549154996871948]},{"id":"semif_03d919fd0e9b3a7cd1e6","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003353297710418701,"mean_abs_dp":0.0022354970375696817,"oracle_top2_margin":0.5888532400131226,"near_tie":false,"probs":[0.7351629734039307,0.12480108439922333,0.1400359719991684]},{"id":"semif_86a15618ba4cb90a307e","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005804300308227539,"mean_abs_dp":0.00038697198033332825,"oracle_top2_margin":0.8870839476585388,"near_tie":false,"probs":[0.9350093007087708,0.04695684462785721,0.018033890053629875]},{"id":"semif_3dd9c3ac98f8ddf7bdd3","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003867805004119873,"mean_abs_dp":0.0025785788893699646,"oracle_top2_margin":0.6225321292877197,"near_tie":false,"probs":[0.7708819508552551,0.08730892837047577,0.14180907607078552]},{"id":"semif_36273dd4777876a26338","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00019407272338867188,"mean_abs_dp":0.00012934363136688867,"oracle_top2_margin":0.9606409072875977,"near_tie":false,"probs":[0.004923136439174414,0.9780420660972595,0.017034856602549553]},{"id":"semif_07ee70cfe4b01330ce89","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00262296199798584,"mean_abs_dp":0.0017486661672592163,"oracle_top2_margin":0.12762874364852905,"near_tie":false,"probs":[0.0806957483291626,0.3973904252052307,0.5219138860702515]},{"id":"semif_51ceab3ce622c7c759f1","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005183219909667969,"mean_abs_dp":0.0003455498566230138,"oracle_top2_margin":0.9027648568153381,"near_tie":false,"probs":[0.009600392542779446,0.04421387240290642,0.9461857080459595]},{"id":"semif_b8091be929b1e489fcc4","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008707344532012939,"mean_abs_dp":0.0005804992591341337,"oracle_top2_margin":0.8441397547721863,"near_tie":false,"probs":[0.013046018779277802,0.916387677192688,0.07056631147861481]},{"id":"semif_c89b86e3f55123bfa506","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00013846158981323242,"mean_abs_dp":9.234560032685597e-05,"oracle_top2_margin":0.9237306118011475,"near_tie":false,"probs":[0.011066460981965065,0.9564358592033386,0.03249761089682579]},{"id":"semif_3b0958cabc339005cf66","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0044325292110443115,"mean_abs_dp":0.0029550443092981973,"oracle_top2_margin":0.358396053314209,"near_tie":false,"probs":[0.5885096788406372,0.1751343011856079,0.2363560050725937]},{"id":"semif_f2b4ec4930322fe45118","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001984536647796631,"mean_abs_dp":0.0013230405747890472,"oracle_top2_margin":0.6658164858818054,"near_tie":false,"probs":[0.1390702724456787,0.8082730770111084,0.052656613290309906]},{"id":"semif_61ac20b07a5189231e15","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006647229194641113,"mean_abs_dp":0.004431476195653279,"oracle_top2_margin":0.5127555131912231,"near_tie":false,"probs":[0.0253997053951025,0.7378273606300354,0.23677287995815277]},{"id":"semif_4b3d794c004566610181","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0018676966428756714,"mean_abs_dp":0.0012451214715838432,"oracle_top2_margin":0.7644643187522888,"near_tie":false,"probs":[0.011901226826012135,0.11003235727548599,0.87806636095047]},{"id":"semif_b7c4e62d07986286fce0","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003857627511024475,"mean_abs_dp":0.002571736772855123,"oracle_top2_margin":0.18547630310058594,"near_tie":false,"probs":[0.5271489024162292,0.1277085691690445,0.34514251351356506]},{"id":"semif_9473603b44af63beab79","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004277527332305908,"mean_abs_dp":0.002851709723472595,"oracle_top2_margin":0.4208753705024719,"near_tie":false,"probs":[0.6792829632759094,0.2500956356525421,0.0706213191151619]},{"id":"semif_d41072dc35d8cd14c056","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.013136208057403564,"mean_abs_dp":0.008757467071215311,"oracle_top2_margin":0.3230390250682831,"near_tie":false,"probs":[0.1417761594057083,0.5794675946235657,0.27875620126724243]},{"id":"semif_33d6e5da58f0fee2490d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.009425908327102661,"mean_abs_dp":0.006283914049466451,"oracle_top2_margin":0.03421613574028015,"near_tie":false,"probs":[0.3501538932323456,0.40105587244033813,0.24879023432731628]},{"id":"semif_823bf736b92e9d8fa66d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0051011741161346436,"mean_abs_dp":0.003400797645250956,"oracle_top2_margin":0.33118629455566406,"near_tie":false,"probs":[0.21298451721668243,0.5590951442718506,0.22792038321495056]},{"id":"semif_8a4c3de28be95c105ca1","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005613446235656738,"mean_abs_dp":0.003742267688115438,"oracle_top2_margin":0.4625657796859741,"near_tie":false,"probs":[0.10517657548189163,0.22081255912780762,0.6740108132362366]},{"id":"semif_ea7ec66a5e8eac886548","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008949339389801025,"mean_abs_dp":0.0059662361939748125,"oracle_top2_margin":0.21984949707984924,"near_tie":false,"probs":[0.5328294038772583,0.3286515772342682,0.1385190486907959]},{"id":"semif_25172cd6ddc49ab0ca70","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.013986349105834961,"mean_abs_dp":0.009324230253696442,"oracle_top2_margin":0.29460233449935913,"near_tie":false,"probs":[0.09238239377737045,0.31742075085639954,0.5901968479156494]},{"id":"semif_fb73a85d386cf59b0c51","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0034686028957366943,"mean_abs_dp":0.00231242428223292,"oracle_top2_margin":0.20669782161712646,"near_tie":false,"probs":[0.5463327169418335,0.10796212404966354,0.3457051217556]},{"id":"semif_a02fd582d2881cc462b2","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005388736724853516,"mean_abs_dp":0.0035924936334292092,"oracle_top2_margin":0.17390939593315125,"near_tie":false,"probs":[0.11946943402290344,0.522304356098175,0.3582262098789215]},{"id":"semif_19f37d5a97034bd7d04f","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001548483967781067,"mean_abs_dp":0.0010323400298754375,"oracle_top2_margin":0.8648225665092468,"near_tie":false,"probs":[0.05053037032485008,0.9124056100845337,0.03706398978829384]},{"id":"semif_fda0ca94652c6d739ecf","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001592099666595459,"mean_abs_dp":0.0010614047447840373,"oracle_top2_margin":0.8368752598762512,"near_tie":false,"probs":[0.9010158181190491,0.0318327471613884,0.06715142726898193]},{"id":"semif_d0e36c4cb10a29b3191c","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0022559165954589844,"mean_abs_dp":0.001503994067509969,"oracle_top2_margin":0.2865877151489258,"near_tie":false,"probs":[0.5712660551071167,0.14085565507411957,0.2878783643245697]},{"id":"semif_a0a39ace7e5337d038a7","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014776363968849182,"mean_abs_dp":9.850040078163147e-05,"oracle_top2_margin":0.8597530126571655,"near_tie":false,"probs":[0.9126294851303101,0.03443494439125061,0.05293566733598709]},{"id":"semif_eeae283841ce75dd25bf","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00850909948348999,"mean_abs_dp":0.005672715604305267,"oracle_top2_margin":0.4696289002895355,"near_tie":false,"probs":[0.2164122313261032,0.6724936962127686,0.11109405010938644]},{"id":"semif_00bb4cc043f7d2b288d4","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00046466849744319916,"mean_abs_dp":0.00030976223448912304,"oracle_top2_margin":0.7398609519004822,"near_tie":false,"probs":[0.11504767090082169,0.030299920588731766,0.8546524047851562]},{"id":"semif_1a81ead69e93e5d0c86a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003090381622314453,"mean_abs_dp":0.0020602792501449585,"oracle_top2_margin":0.24788936972618103,"near_tie":false,"probs":[0.14760801196098328,0.2994978725910187,0.5528940558433533]},{"id":"semif_4444dad25e438f128673","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004919707775115967,"mean_abs_dp":0.0032798051834106445,"oracle_top2_margin":0.15798577666282654,"near_tie":false,"probs":[0.28719204664230347,0.26865148544311523,0.4441564977169037]},{"id":"semif_3226202ba1ada2570e9f","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0037698745727539062,"mean_abs_dp":0.0025132298469543457,"oracle_top2_margin":0.058840811252593994,"near_tie":false,"probs":[0.22142602503299713,0.35634714365005493,0.42222684621810913]},{"id":"semif_3f783eec5c90812479bc","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002129882574081421,"mean_abs_dp":0.0014198968807856243,"oracle_top2_margin":0.1389794647693634,"near_tie":false,"probs":[0.21765869855880737,0.32298213243484497,0.45935919880867004]},{"id":"semif_ffee861474a9665a093a","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0030732005834579468,"mean_abs_dp":0.002048790454864502,"oracle_top2_margin":0.36492863297462463,"near_tie":false,"probs":[0.22812964022159576,0.18442896008491516,0.5874413847923279]},{"id":"semif_8c992e17e4fb54b044f5","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0038269013166427612,"mean_abs_dp":0.002551312247912089,"oracle_top2_margin":0.24903073906898499,"near_tie":false,"probs":[0.21454881131649017,0.5154661536216736,0.2699849307537079]},{"id":"semif_695dd61195ca025c4f4c","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008753478527069092,"mean_abs_dp":0.005835644900798798,"oracle_top2_margin":0.6265780925750732,"near_tie":false,"probs":[0.7806803584098816,0.14008435606956482,0.07923529297113419]},{"id":"semif_e6155431ab7b0f8170d4","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018589124083518982,"mean_abs_dp":0.0012392674883206685,"oracle_top2_margin":0.5309679508209229,"near_tie":false,"probs":[0.18936163187026978,0.7218835353851318,0.08875483274459839]},{"id":"semif_b49ecda75086d8c68d2a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005085617303848267,"mean_abs_dp":0.0033904314041137695,"oracle_top2_margin":0.16984423995018005,"near_tie":false,"probs":[0.25553107261657715,0.29156985878944397,0.4528990387916565]},{"id":"semif_587ced261c6092cd1325","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000364154577255249,"mean_abs_dp":0.00024275481700897217,"oracle_top2_margin":0.29910629987716675,"near_tie":false,"probs":[0.20806793868541718,0.5454195737838745,0.24651239812374115]},{"id":"semif_533d4423d311de82b27d","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003022134304046631,"mean_abs_dp":0.0020147462685902915,"oracle_top2_margin":0.20385906100273132,"near_tie":false,"probs":[0.1318938434123993,0.5329845547676086,0.3351215720176697]},{"id":"semif_2bf540d515a51755b25c","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.013102889060974121,"mean_abs_dp":0.008735281725724539,"oracle_top2_margin":0.7265769839286804,"near_tie":false,"probs":[0.09006571769714355,0.0721297562122345,0.8378044962882996]},{"id":"semif_fdb9fc214aea2710763a","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006450265645980835,"mean_abs_dp":0.004300177097320557,"oracle_top2_margin":0.1179611086845398,"near_tie":false,"probs":[0.16251370310783386,0.36514419317245483,0.4723420739173889]},{"id":"semif_f6d68126b43f8c9ed183","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0030102133750915527,"mean_abs_dp":0.002006813883781433,"oracle_top2_margin":0.799848735332489,"near_tie":false,"probs":[0.06938475370407104,0.8645904064178467,0.06602485477924347]},{"id":"semif_44bae0c8a1615a096c43","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005407899618148804,"mean_abs_dp":0.0036052515109380088,"oracle_top2_margin":0.3203999996185303,"near_tie":false,"probs":[0.5635067224502563,0.245199054479599,0.19129428267478943]},{"id":"semif_2175379f4c69e6207626","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016297996044158936,"mean_abs_dp":0.0010864983002344768,"oracle_top2_margin":0.01896810531616211,"near_tie":true,"probs":[0.41866299510002136,0.39789846539497375,0.1834385246038437]},{"id":"semif_def9fe59d262357a673a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012326836585998535,"mean_abs_dp":0.0008218089739481608,"oracle_top2_margin":0.09732639789581299,"near_tie":false,"probs":[0.41015225648880005,0.509647786617279,0.08019998669624329]},{"id":"semif_712f6ffb763eef7c9a12","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0023659169673919678,"mean_abs_dp":0.0015772382418314617,"oracle_top2_margin":0.13914558291435242,"near_tie":false,"probs":[0.2814801335334778,0.42846471071243286,0.29005521535873413]},{"id":"semif_e1d610bd14e3d16c09b9","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013651400804519653,"mean_abs_dp":0.0009100760022799174,"oracle_top2_margin":0.6573848724365234,"near_tie":false,"probs":[0.7976839542388916,0.13786213099956512,0.06445392221212387]},{"id":"semif_22ec57822db50ec121cf","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00427091121673584,"mean_abs_dp":0.002847264210383097,"oracle_top2_margin":0.04997369647026062,"near_tie":false,"probs":[0.24698704481124878,0.3556533455848694,0.39735960960388184]},{"id":"semif_3e75f2b84e8e08e3d3f2","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007579922676086426,"mean_abs_dp":0.0005053530136744181,"oracle_top2_margin":0.2756141722202301,"near_tie":false,"probs":[0.16848328709602356,0.2785704433917999,0.5529462099075317]},{"id":"semif_ebd74aa05324a1556288","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001018226146697998,"mean_abs_dp":0.000678760310014089,"oracle_top2_margin":0.749722421169281,"near_tie":false,"probs":[0.06756703555583954,0.8416138291358948,0.09081920981407166]},{"id":"semif_d7e84e4245a29817da1f","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.016526520252227783,"mean_abs_dp":0.011017707486947378,"oracle_top2_margin":0.43940702080726624,"near_tie":false,"probs":[0.10238894820213318,0.24412678182125092,0.6534842848777771]},{"id":"semif_f41ae66b3c09630c7a42","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.007601439952850342,"mean_abs_dp":0.0050676316022872925,"oracle_top2_margin":0.4598276913166046,"near_tie":false,"probs":[0.17900203168392181,0.17313237488269806,0.6478655934333801]},{"id":"semif_82c2cd97c160ad27951b","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0030087530612945557,"mean_abs_dp":0.002005830407142639,"oracle_top2_margin":0.14875605702400208,"near_tie":false,"probs":[0.5006176233291626,0.15298005938529968,0.34640228748321533]},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.018214523792266846,"mean_abs_dp":0.012143010894457499,"oracle_top2_margin":0.2047100067138672,"near_tie":false,"probs":[0.470397025346756,0.2956225275993347,0.2339804321527481]},{"id":"semif_4d7a003342db5b0a17e3","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016157031059265137,"mean_abs_dp":0.001077177623907725,"oracle_top2_margin":0.7220965027809143,"near_tie":false,"probs":[0.10798745602369308,0.8276680111885071,0.06434459239244461]},{"id":"semif_19d3d8d0bf773e88be7b","q":0,"variant":"base","tokens":55,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004124343395233154,"mean_abs_dp":0.0027495678514242172,"oracle_top2_margin":0.844078779220581,"near_tie":false,"probs":[0.9023512005805969,0.03322567790746689,0.06442306190729141]},{"id":"semif_d6de731cca52359b1c87","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.009973526000976562,"mean_abs_dp":0.006649027268091838,"oracle_top2_margin":0.088623046875,"near_tie":false,"probs":[0.4387437701225281,0.36358603835105896,0.19767019152641296]},{"id":"semif_b45c63fb92729af27f05","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005959272384643555,"mean_abs_dp":0.003972843289375305,"oracle_top2_margin":0.617178201675415,"near_tie":false,"probs":[0.7635613679885864,0.09712207317352295,0.139316588640213]},{"id":"semif_46b7029b9a704138b77a","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.026026636362075806,"mean_abs_dp":0.017351080973943073,"oracle_top2_margin":0.10240557789802551,"near_tie":false,"probs":[0.12257806956768036,0.5123765468597412,0.36504536867141724]},{"id":"semif_caf773b23934fa7362a1","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005137138068675995,"mean_abs_dp":0.00034246717890103656,"oracle_top2_margin":0.7860848903656006,"near_tie":false,"probs":[0.8779681324958801,0.03021855279803276,0.09181339293718338]},{"id":"semif_f46f392ef9e9e9df564b","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003931522369384766,"mean_abs_dp":0.0026210322976112366,"oracle_top2_margin":0.502530574798584,"near_tie":false,"probs":[0.017935186624526978,0.24352183938026428,0.7385430335998535]},{"id":"semif_005dcf6d2c0f21d64796","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0029805898666381836,"mean_abs_dp":0.0019870524605115256,"oracle_top2_margin":0.6591771841049194,"near_tie":false,"probs":[0.047108244150877,0.8032755851745605,0.14961615204811096]},{"id":"semif_edb708bd0fe5fd67d6ee","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018695592880249023,"mean_abs_dp":0.001246395210425059,"oracle_top2_margin":0.29818880558013916,"near_tie":false,"probs":[0.08695562928915024,0.603823184967041,0.3092212378978729]},{"id":"semif_97a35499bd8717f3e786","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0031740963459014893,"mean_abs_dp":0.0021160642306009927,"oracle_top2_margin":0.08557590842247009,"near_tie":false,"probs":[0.2658678889274597,0.41244596242904663,0.32168617844581604]},{"id":"semif_31eb4533d2c89959ec46","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012247264385223389,"mean_abs_dp":0.0008164693911870321,"oracle_top2_margin":0.29386773705482483,"near_tie":false,"probs":[0.5710992217063904,0.27921032905578613,0.1496904343366623]},{"id":"semif_0aa24a6701ac9c56c4a0","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002118706703186035,"mean_abs_dp":0.0014124773442745209,"oracle_top2_margin":0.887073278427124,"near_tie":false,"probs":[0.04673198610544205,0.0231621190905571,0.9301058650016785]},{"id":"semif_796d5c0da6eaadae6a3f","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020717382431030273,"mean_abs_dp":0.001381170004606247,"oracle_top2_margin":0.8756633400917053,"near_tie":false,"probs":[0.014576539397239685,0.9287246465682983,0.05669880658388138]},{"id":"semif_599b3092b6fd18312d23","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.01459193229675293,"mean_abs_dp":0.00972796231508255,"oracle_top2_margin":0.4057760238647461,"near_tie":false,"probs":[0.22921304404735565,0.11000137031078339,0.6607855558395386]},{"id":"semif_960a4bcd27363e6db4b9","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00484071671962738,"mean_abs_dp":0.0032271395126978555,"oracle_top2_margin":0.23092997074127197,"near_tie":false,"probs":[0.2806316912174225,0.2047179788351059,0.5146503448486328]},{"id":"semif_c6a411e7f2bbdf377173","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.015746235847473145,"mean_abs_dp":0.010497475663820902,"oracle_top2_margin":0.4384426474571228,"near_tie":false,"probs":[0.6592381000518799,0.19766825437545776,0.14309369027614594]},{"id":"semif_e332aa927d0c2ed044aa","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00304257869720459,"mean_abs_dp":0.002028346061706543,"oracle_top2_margin":0.025122374296188354,"near_tie":false,"probs":[0.36376529932022095,0.3434635102748871,0.29277125000953674]},{"id":"semif_a0128e551d095f3449f8","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00020719505846500397,"mean_abs_dp":0.0001380634494125843,"oracle_top2_margin":0.9322084188461304,"near_tie":false,"probs":[0.9631165266036987,0.03061399795114994,0.006269404664635658]},{"id":"semif_46d6a0731bd288c370f0","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010342836380004883,"mean_abs_dp":0.0068952540556589765,"oracle_top2_margin":0.4198586940765381,"near_tie":false,"probs":[0.6485851407051086,0.13961690664291382,0.21179795265197754]},{"id":"semif_21ca7a8b970e36b8e20f","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012861192226409912,"mean_abs_dp":0.0008574028809865316,"oracle_top2_margin":0.043576329946517944,"near_tie":false,"probs":[0.3426211178302765,0.3846345841884613,0.2727442681789398]},{"id":"semif_4aa83af95bb69e90bc2d","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0032129883766174316,"mean_abs_dp":0.002141989767551422,"oracle_top2_margin":0.3831571936607361,"near_tie":false,"probs":[0.6757340431213379,0.03741474449634552,0.2868512272834778]},{"id":"semif_cdfa9eed51fafcf09b72","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.014052867889404297,"mean_abs_dp":0.009368558724721273,"oracle_top2_margin":0.6123061776161194,"near_tie":false,"probs":[0.16679130494594574,0.028413889929652214,0.8047947883605957]},{"id":"semif_d51b05bfb9d6d7f09331","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006421774625778198,"mean_abs_dp":0.004281193017959595,"oracle_top2_margin":0.4349525570869446,"near_tie":false,"probs":[0.21585913002490997,0.662446916103363,0.12169402092695236]},{"id":"semif_fdd902cc6e65fad7cb4f","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0030311644077301025,"mean_abs_dp":0.002020778755346934,"oracle_top2_margin":0.50705486536026,"near_tie":false,"probs":[0.12681615352630615,0.69217449426651,0.18100930750370026]},{"id":"semif_42384af9ae56beb37ff5","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.01421266794204712,"mean_abs_dp":0.009475151697794596,"oracle_top2_margin":0.06503075361251831,"near_tie":false,"probs":[0.3776892125606537,0.3299669623374939,0.2923438549041748]},{"id":"semif_cdf7f428857e1b841cb3","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006398022174835205,"mean_abs_dp":0.004265318314234416,"oracle_top2_margin":0.3410225212574005,"near_tie":false,"probs":[0.1727619469165802,0.24838010966777802,0.5788578987121582]},{"id":"semif_774f7f9245f83c796d44","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0017609596252441406,"mean_abs_dp":0.0011739581823349,"oracle_top2_margin":0.7296704649925232,"near_tie":false,"probs":[0.10257606953382492,0.06846793740987778,0.8289560079574585]},{"id":"semif_92ec2ffa3dd46ada1335","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.011353135108947754,"mean_abs_dp":0.007568761706352234,"oracle_top2_margin":0.36772847175598145,"near_tie":false,"probs":[0.14918190240859985,0.24946701526641846,0.6013510823249817]},{"id":"semif_6149a17bc154f9c5b4a0","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.010267555713653564,"mean_abs_dp":0.006845059494177501,"oracle_top2_margin":0.6154326796531677,"near_tie":false,"probs":[0.7522668838500977,0.1519550383090973,0.09577807784080505]},{"id":"semif_be866e1dbec5d42d9423","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00394517183303833,"mean_abs_dp":0.0026301120718320212,"oracle_top2_margin":0.6972379684448242,"near_tie":false,"probs":[0.06462875008583069,0.11580581963062286,0.8195654153823853]},{"id":"semif_97d2ed2591fc7638806c","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002026796340942383,"mean_abs_dp":0.0013511801759401958,"oracle_top2_margin":0.31745457649230957,"near_tie":false,"probs":[0.29247307777404785,0.6134445667266846,0.0940823033452034]},{"id":"semif_4533dd91267b4d815690","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002110123634338379,"mean_abs_dp":0.0014067192872365315,"oracle_top2_margin":0.4459984302520752,"near_tie":false,"probs":[0.1812114119529724,0.18817955255508423,0.6306089758872986]},{"id":"semif_d880e65fef5e2993fdd0","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00820082426071167,"mean_abs_dp":0.00546721617380778,"oracle_top2_margin":0.05805802345275879,"near_tie":false,"probs":[0.2850084602832794,0.32256779074668884,0.39242371916770935]},{"id":"semif_90494c76078ffc3c717a","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0027776360511779785,"mean_abs_dp":0.0018517772356669109,"oracle_top2_margin":0.022894471883773804,"near_tie":false,"probs":[0.3519638478755951,0.3771188259124756,0.27091729640960693]},{"id":"semif_9ac5cc82c4126c91174a","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025995373725891113,"mean_abs_dp":0.001733014980951945,"oracle_top2_margin":0.042064547538757324,"near_tie":false,"probs":[0.2592650055885315,0.34761762619018555,0.39311739802360535]},{"id":"semif_386e10b1c1ce906473a4","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.024856865406036377,"mean_abs_dp":0.016571228702863056,"oracle_top2_margin":0.43534374237060547,"near_tie":false,"probs":[0.2075207680463791,0.6020828485488892,0.19039635360240936]},{"id":"semif_4da6aedc1139224eeff2","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0028258562088012695,"mean_abs_dp":0.001883906622727712,"oracle_top2_margin":0.7471664547920227,"near_tie":false,"probs":[0.07488502562046051,0.834364652633667,0.09075025469064713]},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.04081737995147705,"mean_abs_dp":0.027211596568425495,"oracle_top2_margin":0.05925852060317993,"near_tie":false,"probs":[0.21672122180461884,0.3226245939731598,0.4606541693210602]},{"id":"semif_42c54fa2ad45743e24d8","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006586551666259766,"mean_abs_dp":0.004391017059485118,"oracle_top2_margin":0.22642815113067627,"near_tie":false,"probs":[0.12473386526107788,0.5449090600013733,0.33035704493522644]},{"id":"semif_f3d13cf184f916677def","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.02583444118499756,"mean_abs_dp":0.017222970724105835,"oracle_top2_margin":0.05370849370956421,"near_tie":false,"probs":[0.32354459166526794,0.37449532747268677,0.3019600212574005]},{"id":"semif_1105577da4c8dad4609d","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.04313388466835022,"mean_abs_dp":0.028755903244018555,"oracle_top2_margin":0.14877480268478394,"near_tie":false,"probs":[0.3235653340816498,0.2806474268436432,0.39578720927238464]},{"id":"semif_0ebd299739302d8b057b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.014297336339950562,"mean_abs_dp":0.009531527757644653,"oracle_top2_margin":0.05391588807106018,"near_tie":false,"probs":[0.27466434240341187,0.38802939653396606,0.33730629086494446]},{"id":"semif_eafc22c8c40df3932a8e","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025859475135803223,"mean_abs_dp":0.0017239550749460857,"oracle_top2_margin":0.0383472740650177,"near_tie":false,"probs":[0.206257626414299,0.375928670167923,0.41781365871429443]},{"id":"semif_0aa1447680e4f25862bb","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025915205478668213,"mean_abs_dp":0.001727670431137085,"oracle_top2_margin":0.044051527976989746,"near_tie":false,"probs":[0.37979602813720703,0.28727957606315613,0.33292439579963684]},{"id":"semif_980e76145c925747220c","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.007738590240478516,"mean_abs_dp":0.005159010489781697,"oracle_top2_margin":0.48375236988067627,"near_tie":false,"probs":[0.22696200013160706,0.074718177318573,0.6983197331428528]},{"id":"semif_4aee37a6a155747766aa","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012968331575393677,"mean_abs_dp":0.0008645405371983846,"oracle_top2_margin":0.13445758819580078,"near_tie":false,"probs":[0.3087027668952942,0.24731028079986572,0.4439869225025177]},{"id":"semif_6db17cc22bd656d78558","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00512772798538208,"mean_abs_dp":0.0034184902906417847,"oracle_top2_margin":0.40535932779312134,"near_tie":false,"probs":[0.21952737867832184,0.16345393657684326,0.6170186996459961]},{"id":"semif_ef3e0f8e657bf59f8a29","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.009519189596176147,"mean_abs_dp":0.006346156199773152,"oracle_top2_margin":0.15905368328094482,"near_tie":false,"probs":[0.3116495609283447,0.21754051744937897,0.4708098769187927]},{"id":"semif_f756e1cb04e11766341e","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.008287221193313599,"mean_abs_dp":0.005524814128875732,"oracle_top2_margin":0.1089421808719635,"near_tie":false,"probs":[0.2750186324119568,0.3106433153152466,0.41433805227279663]},{"id":"semif_d167e51bb1f499099645","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":0,"argmax_equal":false,"max_abs_dp":0.010080963373184204,"mean_abs_dp":0.0067206621170043945,"oracle_top2_margin":0.01671499013900757,"near_tie":true,"probs":[0.346190482378006,0.3045433461666107,0.3492661714553833]},{"id":"semif_674ca4e9346216255466","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0041877031326293945,"mean_abs_dp":0.002791831890741984,"oracle_top2_margin":0.5046945810317993,"near_tie":false,"probs":[0.6943565607070923,0.1822805255651474,0.12336281687021255]},{"id":"semif_1df851a202994a07f123","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004023313522338867,"mean_abs_dp":0.00026822245369354886,"oracle_top2_margin":0.9035253524780273,"near_tie":false,"probs":[0.9473530054092407,0.04462796077132225,0.00801901426166296]},{"id":"semif_b5e6788d95b8395d2888","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006696581840515137,"mean_abs_dp":0.0004464574158191681,"oracle_top2_margin":0.8871046304702759,"near_tie":false,"probs":[0.04115220531821251,0.9292663335800171,0.0295814611017704]},{"id":"semif_32f36cd0fbbfb3604586","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011666938662528992,"mean_abs_dp":0.0007777586579322815,"oracle_top2_margin":0.6213997006416321,"near_tie":false,"probs":[0.7684829831123352,0.14791595935821533,0.08360110968351364]},{"id":"semif_843185706adff0c60f8b","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0032514333724975586,"mean_abs_dp":0.002167621006568273,"oracle_top2_margin":0.12877494096755981,"near_tie":false,"probs":[0.5621827840805054,0.010832476429641247,0.4269847571849823]},{"id":"semif_b531dd603bf080dab820","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.012893199920654297,"mean_abs_dp":0.008595431844393412,"oracle_top2_margin":0.35339561104774475,"near_tie":false,"probs":[0.6219592094421387,0.08614465594291687,0.2918960750102997]},{"id":"semif_be2dd67bc733c1213bef","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006429404020309448,"mean_abs_dp":0.004286294182141622,"oracle_top2_margin":0.12457886338233948,"near_tie":false,"probs":[0.12172918766736984,0.49631741642951965,0.3819534182548523]},{"id":"semif_b8e2750307df9761e716","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012512803077697754,"mean_abs_dp":0.0008341707289218903,"oracle_top2_margin":0.4760705828666687,"near_tie":false,"probs":[0.7217902541160583,0.03085591271519661,0.24735380709171295]},{"id":"semif_5d31b4bf2d0fc193f53b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024169087409973145,"mean_abs_dp":0.0016112476587295532,"oracle_top2_margin":0.2525418996810913,"near_tie":false,"probs":[0.1355939507484436,0.5602473616600037,0.3041587471961975]},{"id":"semif_a9230edfc6ec9e524af0","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004920482635498047,"mean_abs_dp":0.00328029195467631,"oracle_top2_margin":0.12013593316078186,"near_tie":false,"probs":[0.2512340545654297,0.4329686462879181,0.31579723954200745]},{"id":"semif_c100e84585149a04be49","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.011261343955993652,"mean_abs_dp":0.007507552703221639,"oracle_top2_margin":0.628185510635376,"near_tie":false,"probs":[0.7864094376564026,0.035419099032878876,0.17817145586013794]},{"id":"semif_579beed184ffc1104aa9","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00526273250579834,"mean_abs_dp":0.00350849578777949,"oracle_top2_margin":0.16769906878471375,"near_tie":false,"probs":[0.5315226912498474,0.09486360847949982,0.37361374497413635]},{"id":"semif_c4e9bd463b6fde5be743","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00328826904296875,"mean_abs_dp":0.0021921868125597634,"oracle_top2_margin":0.7536282539367676,"near_tie":false,"probs":[0.09054077416658401,0.8389317989349365,0.07052747160196304]},{"id":"semif_b863998d4be2721a5dfd","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009967386722564697,"mean_abs_dp":0.0006645222504933675,"oracle_top2_margin":0.08771461248397827,"near_tie":false,"probs":[0.35369011759757996,0.4407995641231537,0.20551033318042755]},{"id":"semif_37cebc3792e98049439f","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022699497640132904,"mean_abs_dp":0.00015136289099852243,"oracle_top2_margin":0.8853868246078491,"near_tie":false,"probs":[0.051344286650419235,0.9369518756866455,0.011703752912580967]},{"id":"semif_64d63f86e328aff1481c","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003199707716703415,"mean_abs_dp":0.00021334302922089896,"oracle_top2_margin":0.9121700525283813,"near_tie":false,"probs":[0.019315527752041817,0.0343572162091732,0.9463272094726562]},{"id":"semif_6d64a8459f64eac7f4cc","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004674121737480164,"mean_abs_dp":0.0031161109606424966,"oracle_top2_margin":0.39074867963790894,"near_tie":false,"probs":[0.6235161423683167,0.1436479389667511,0.23283588886260986]},{"id":"semif_2708193212d8a4a523c7","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0037764012813568115,"mean_abs_dp":0.002517630656560262,"oracle_top2_margin":0.20737019181251526,"near_tie":false,"probs":[0.22756372392177582,0.28036531805992126,0.4920710027217865]},{"id":"semif_5a37d1dfcad20810a76a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011462569236755371,"mean_abs_dp":0.0007641588648160299,"oracle_top2_margin":0.4521888494491577,"near_tie":false,"probs":[0.02842242829501629,0.7111488580703735,0.26042866706848145]},{"id":"semif_acacd8b46acd25cb954b","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.015568524599075317,"mean_abs_dp":0.010379031300544739,"oracle_top2_margin":0.06195080280303955,"near_tie":false,"probs":[0.4320998191833496,0.34581220149993896,0.22208799421787262]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018231868743896484,"mean_abs_dp":0.0012154554327329,"oracle_top2_margin":0.0032414793968200684,"near_tie":true,"probs":[0.4404924511909485,0.4406611919403076,0.11884637176990509]},{"id":"semif_02f2b95a5c5a4777a45c","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.011606395244598389,"mean_abs_dp":0.007737591862678528,"oracle_top2_margin":0.6085776090621948,"near_tie":false,"probs":[0.09859045594930649,0.7462278008460999,0.1551816761493683]},{"id":"semif_ebea4681e3dcc7d92ca2","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003974229097366333,"mean_abs_dp":0.0026494661966959634,"oracle_top2_margin":0.13673824071884155,"near_tie":false,"probs":[0.2977369725704193,0.2736370265483856,0.4286259710788727]},{"id":"semif_b8c5b9b285cbdddc9f8d","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008846521377563477,"mean_abs_dp":0.0005897680918375651,"oracle_top2_margin":0.0788612961769104,"near_tie":false,"probs":[0.3124503493309021,0.29475104808807373,0.39279866218566895]},{"id":"semif_7ed75d854d99b3a81916","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003912687301635742,"mean_abs_dp":0.002608430882294973,"oracle_top2_margin":0.4379068613052368,"near_tie":false,"probs":[0.6660853028297424,0.10080329328775406,0.23311135172843933]},{"id":"own_t01","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019826292991638184,"mean_abs_dp":0.000892358273267746,"oracle_top2_margin":0.3381020724773407,"near_tie":false,"probs":[0.5425770282745361,0.03400913253426552,0.14745785295963287,0.06974665075540543,0.20620925724506378],"hidden":{"min_pos_cos":0.9996967220801186,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":1,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014037489891052246,"mean_abs_dp":0.0014037340879440308,"oracle_top2_margin":0.30454543232917786,"near_tie":false,"probs":[0.34632354974746704,0.653676450252533],"hidden":{"min_pos_cos":0.9998069951793994,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":2,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00139559805393219,"mean_abs_dp":0.00073280930519104,"oracle_top2_margin":0.22719793021678925,"near_tie":false,"probs":[0.1080050989985466,0.23810306191444397,0.4636754095554352,0.19021637737751007],"hidden":{"min_pos_cos":0.9998069951793994,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":3,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016316920518875122,"mean_abs_dp":0.0016316547989845276,"oracle_top2_margin":0.7503683567047119,"near_tie":false,"probs":[0.12644748389720917,0.8735525608062744],"hidden":{"min_pos_cos":0.9998069951793994,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t02","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003397703170776367,"mean_abs_dp":0.0018532127141952515,"oracle_top2_margin":0.12554225325584412,"near_tie":false,"probs":[0.41272079944610596,0.2823317050933838,0.1944597214460373,0.11048773676156998]},{"id":"own_t02","q":1,"variant":"base","tokens":303,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014523863792419434,"mean_abs_dp":0.0006017275154590607,"oracle_top2_margin":0.7914202213287354,"near_tie":false,"probs":[0.011292759329080582,0.8677960634231567,0.02403523586690426,0.018300121650099754,0.07857590168714523]},{"id":"own_t02","q":2,"variant":"base","tokens":262,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018345192074775696,"mean_abs_dp":0.0018345154821872711,"oracle_top2_margin":0.8067709803581238,"near_tie":false,"probs":[0.9015509486198425,0.09844899922609329]},{"id":"own_t03","q":0,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007103562355041504,"mean_abs_dp":0.00236785722275575,"oracle_top2_margin":0.5557763576507568,"near_tie":false,"probs":[0.04317395016551018,0.718421459197998,0.03777197748422623,0.02875157818198204,0.018011948093771935,0.15386907756328583]},{"id":"own_t03","q":1,"variant":"base","tokens":214,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004126787185668945,"mean_abs_dp":0.004126757383346558,"oracle_top2_margin":0.7302511930465698,"near_tie":false,"probs":[0.13074766099452972,0.8692523837089539]},{"id":"own_t03","q":2,"variant":"base","tokens":245,"calls":16,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00263860821723938,"mean_abs_dp":0.001759631559252739,"oracle_top2_margin":0.18012985587120056,"near_tie":false,"probs":[0.30195119976997375,0.48143404722213745,0.17054589092731476,0.04606889560818672]},{"id":"own_t04","q":0,"variant":"base","tokens":190,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0031719207763671875,"mean_abs_dp":0.001585950143635273,"oracle_top2_margin":0.8690025806427002,"near_tie":false,"probs":[0.042929768562316895,0.9066300392150879,0.02621227502822876,0.02422790601849556]},{"id":"own_t04","q":1,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018109679222106934,"mean_abs_dp":0.0018109679222106934,"oracle_top2_margin":0.106167733669281,"near_tie":false,"probs":[0.4451051354408264,0.5548948049545288]},{"id":"own_t04","q":2,"variant":"base","tokens":191,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014303624629974365,"mean_abs_dp":0.0008112061768770218,"oracle_top2_margin":0.05060693621635437,"near_tie":false,"probs":[0.1401994824409485,0.04020736366510391,0.433919757604599,0.3856733441352844]},{"id":"own_t04","q":3,"variant":"base","tokens":175,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0026178359985351562,"mean_abs_dp":0.0017452736695607503,"oracle_top2_margin":0.14521116018295288,"near_tie":false,"probs":[0.2047233134508133,0.47263863682746887,0.32263797521591187]},{"id":"own_t05","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002777576446533203,"mean_abs_dp":0.0017372369766235352,"oracle_top2_margin":0.08131647109985352,"near_tie":false,"probs":[0.2648012042045593,0.3411100208759308,0.25113868713378906,0.10264191031455994,0.0403081513941288]},{"id":"own_t05","q":1,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001680716872215271,"mean_abs_dp":0.0005974161128203074,"oracle_top2_margin":0.397593230009079,"near_tie":false,"probs":[0.020131954923272133,0.6290848255157471,0.23078881204128265,0.024872809648513794,0.0131571339443326,0.081964410841465]},{"id":"own_t05","q":2,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012743473052978516,"mean_abs_dp":0.00012738816440105438,"oracle_top2_margin":0.9450063109397888,"near_tie":false,"probs":[0.972375750541687,0.02762422524392605]},{"id":"own_t05","q":3,"variant":"base","tokens":125,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001328699290752411,"mean_abs_dp":0.0001328643411397934,"oracle_top2_margin":0.9648926258087158,"near_tie":false,"probs":[0.017686564475297928,0.9823134541511536]},{"id":"own_t06","q":0,"variant":"base","tokens":301,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.01196315884590149,"mean_abs_dp":0.0027110394090414046,"oracle_top2_margin":0.048377931118011475,"near_tie":false,"probs":[0.03218561410903931,0.2912707030773163,0.00702153705060482,0.2634766101837158,0.022607704624533653,0.07051414251327515,0.014771086163818836,0.018963351845741272,0.03449355810880661,0.24469567835330963]},{"id":"own_t06","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0033268630504608154,"mean_abs_dp":0.0033268481492996216,"oracle_top2_margin":0.5263986587524414,"near_tie":false,"probs":[0.7665261626243591,0.23347382247447968]},{"id":"own_t06","q":2,"variant":"base","tokens":224,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009157955646514893,"mean_abs_dp":0.0006105701128641764,"oracle_top2_margin":0.04287439584732056,"near_tie":false,"probs":[0.39195913076400757,0.436604768037796,0.17143604159355164]},{"id":"own_t07","q":0,"variant":"base","tokens":219,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025977492332458496,"mean_abs_dp":0.0013573020696640015,"oracle_top2_margin":0.13217660784721375,"near_tie":false,"probs":[0.12773926556110382,0.25480911135673523,0.381980299949646,0.23547136783599854]},{"id":"own_t07","q":1,"variant":"base","tokens":174,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006479322910308838,"mean_abs_dp":0.006479315459728241,"oracle_top2_margin":0.6053133010864258,"near_tie":false,"probs":[0.20382264256477356,0.796177327632904]},{"id":"own_t07","q":2,"variant":"base","tokens":187,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.009774863719940186,"mean_abs_dp":0.00651658574740092,"oracle_top2_margin":0.2717403173446655,"near_tie":false,"probs":[0.546354353427887,0.26091280579566956,0.1927328109741211]},{"id":"own_t08","q":0,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011895298957824707,"mean_abs_dp":0.0011895149946212769,"oracle_top2_margin":0.1572883427143097,"near_tie":false,"probs":[0.5798336863517761,0.4201663136482239]},{"id":"own_t08","q":1,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00321120023727417,"mean_abs_dp":0.001605592668056488,"oracle_top2_margin":0.03466594219207764,"near_tie":false,"probs":[0.31244930624961853,0.3427073359489441,0.21665455400943756,0.1281888484954834]},{"id":"own_t08","q":2,"variant":"base","tokens":170,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003873184323310852,"mean_abs_dp":0.001936584711074829,"oracle_top2_margin":0.25134599208831787,"near_tie":false,"probs":[0.49005886912345886,0.2388146072626114,0.11389904469251633,0.15722741186618805]},{"id":"own_t09","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002621591091156006,"mean_abs_dp":0.002621561288833618,"oracle_top2_margin":0.6116014122962952,"near_tie":false,"probs":[0.19682085514068604,0.803179144859314]},{"id":"own_t09","q":1,"variant":"base","tokens":151,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020749568939208984,"mean_abs_dp":0.0010374881094321609,"oracle_top2_margin":0.7431982159614563,"near_tie":false,"probs":[0.8629664778709412,0.004676468204706907,0.008692961186170578,0.12366412580013275]},{"id":"own_t10","q":0,"variant":"base","tokens":161,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006288327276706696,"mean_abs_dp":0.0002515189116820693,"oracle_top2_margin":0.9358961582183838,"near_tie":false,"probs":[0.9571694135665894,0.01826452650129795,0.02031525783240795,0.0017456552013754845,0.002505086362361908]},{"id":"own_t10","q":1,"variant":"base","tokens":140,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016608834266662598,"mean_abs_dp":0.0016608387231826782,"oracle_top2_margin":0.07992240786552429,"near_tie":false,"probs":[0.4583779573440552,0.5416220426559448]},{"id":"own_t10","q":2,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004240356385707855,"mean_abs_dp":0.002812465652823448,"oracle_top2_margin":0.011951148509979248,"near_tie":true,"probs":[0.31916216015815735,0.2712917625904083,0.30745646357536316,0.10208962112665176]},{"id":"own_j01","q":0,"variant":"base","tokens":212,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019022226333618164,"mean_abs_dp":0.0019021928310394287,"oracle_top2_margin":0.36813127994537354,"near_tie":false,"probs":[0.3140321373939514,0.6859678030014038]},{"id":"own_j01","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024046599864959717,"mean_abs_dp":0.001203194260597229,"oracle_top2_margin":0.145786851644516,"near_tie":false,"probs":[0.2274700552225113,0.3697919547557831,0.2054738998413086,0.19726401567459106]},{"id":"own_j01","q":2,"variant":"base","tokens":242,"calls":16,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002497762441635132,"mean_abs_dp":0.001248873770236969,"oracle_top2_margin":0.06988182663917542,"near_tie":false,"probs":[0.13282763957977295,0.21568410098552704,0.29221972823143005,0.35926851630210876]},{"id":"own_j01","q":3,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005257993936538696,"mean_abs_dp":0.0052579790353775024,"oracle_top2_margin":0.15178239345550537,"near_tie":false,"probs":[0.4188508093357086,0.581149160861969]},{"id":"own_j02","q":0,"variant":"base","tokens":332,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020061135292053223,"mean_abs_dp":0.0006391927599906921,"oracle_top2_margin":0.007524773478507996,"near_tie":true,"probs":[0.2073727548122406,0.08935251832008362,0.19707722961902618,0.08295747637748718,0.02874712087213993,0.004249794874340296,0.01963174343109131,0.15918785333633423,0.015149169601500034,0.19627439975738525]},{"id":"own_j02","q":1,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003012537956237793,"mean_abs_dp":0.003012523055076599,"oracle_top2_margin":0.10447829961776733,"near_tie":false,"probs":[0.4507733881473541,0.5492266416549683]},{"id":"own_j02","q":2,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0034804344177246094,"mean_abs_dp":0.0034804195165634155,"oracle_top2_margin":0.08897575736045837,"near_tie":false,"probs":[0.45899254083633423,0.5410074591636658]},{"id":"own_j03","q":0,"variant":"base","tokens":380,"calls":24,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008165925741195679,"mean_abs_dp":0.004319570958614349,"oracle_top2_margin":0.06177407503128052,"near_tie":false,"probs":[0.3490881323814392,0.22608262300491333,0.15197402238845825,0.2728552520275116],"hidden":{"min_pos_cos":0.9998040192160517,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j03","q":1,"variant":"base","tokens":333,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0029489994049072266,"mean_abs_dp":0.0008425712585449219,"oracle_top2_margin":0.020112618803977966,"near_tie":false,"probs":[0.22976645827293396,0.06642156094312668,0.15098026394844055,0.20954620838165283,0.11453499644994736,0.08417529612779617,0.14457523822784424],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j03","q":2,"variant":"base","tokens":314,"calls":20,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.006011068820953369,"mean_abs_dp":0.006011053919792175,"oracle_top2_margin":0.02395007014274597,"near_tie":false,"probs":[0.4940360188484192,0.5059639811515808],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j04","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.018736541271209717,"mean_abs_dp":0.009368272498250008,"oracle_top2_margin":0.5144549608230591,"near_tie":false,"probs":[0.11355031281709671,0.12579573690891266,0.6247901916503906,0.1358637511730194]},{"id":"own_j04","q":1,"variant":"base","tokens":171,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.217028617858887e-05,"mean_abs_dp":4.215538501739502e-05,"oracle_top2_margin":0.32075631618499756,"near_tie":false,"probs":[0.660420298576355,0.33957967162132263]},{"id":"own_j04","q":2,"variant":"base","tokens":196,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002406388521194458,"mean_abs_dp":0.001468762755393982,"oracle_top2_margin":0.08215877413749695,"near_tie":false,"probs":[0.3835221827030182,0.3045971989631653,0.15743833780288696,0.15444226562976837]},{"id":"own_j05","q":0,"variant":"base","tokens":198,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00538790225982666,"mean_abs_dp":0.003591919938723246,"oracle_top2_margin":0.4507710933685303,"near_tie":false,"probs":[0.17985734343528748,0.6316659450531006,0.18847665190696716]},{"id":"own_j05","q":1,"variant":"base","tokens":153,"calls":10,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013417601585388184,"mean_abs_dp":0.0013417601585388184,"oracle_top2_margin":0.024675756692886353,"near_tie":false,"probs":[0.5136796236038208,0.4863203465938568]},{"id":"own_j05","q":2,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012962818145751953,"mean_abs_dp":0.00037038219826562066,"oracle_top2_margin":0.09147655963897705,"near_tie":false,"probs":[0.24858823418617249,0.09259825944900513,0.11897934973239899,0.11708119511604309,0.1522981971502304,0.1554824560880661,0.11497224867343903]},{"id":"own_a01","q":0,"variant":"base","tokens":199,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005250900983810425,"mean_abs_dp":0.0026254691183567047,"oracle_top2_margin":0.12012860178947449,"near_tie":false,"probs":[0.3514038324356079,0.46219998598098755,0.058404114097356796,0.12799206376075745]},{"id":"own_a01","q":1,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012416243553161621,"mean_abs_dp":0.0012415796518325806,"oracle_top2_margin":0.31672245264053345,"near_tie":false,"probs":[0.3403972089290619,0.6596028208732605]},{"id":"own_a01","q":2,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003083646297454834,"mean_abs_dp":0.00030834972858428955,"oracle_top2_margin":0.09236004948616028,"near_tie":false,"probs":[0.546488344669342,0.4535115957260132]},{"id":"own_m01","q":0,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015125870704650879,"mean_abs_dp":0.0010084037979443867,"oracle_top2_margin":0.8812302947044373,"near_tie":false,"probs":[0.0316176600754261,0.9236109852790833,0.04477139934897423],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":1,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001052163541316986,"mean_abs_dp":0.0010521523654460907,"oracle_top2_margin":0.8975095748901367,"near_tie":false,"probs":[0.052297357469797134,0.9477026462554932],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":2,"variant":"base","tokens":159,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0036519765853881836,"mean_abs_dp":0.0024346361557642617,"oracle_top2_margin":0.3613918125629425,"near_tie":false,"probs":[0.6492382884025574,0.2937619090080261,0.05699983984231949],"hidden":{"min_pos_cos":0.9998373467790606,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":3,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002207458019256592,"mean_abs_dp":0.001471645509203275,"oracle_top2_margin":0.4848095774650574,"near_tie":false,"probs":[0.027755746617913246,0.2417151778936386,0.7305290699005127],"hidden":{"min_pos_cos":0.9998196159990311,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":4,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00023170560598373413,"mean_abs_dp":0.00023169443011283875,"oracle_top2_margin":0.9664559364318848,"near_tie":false,"probs":[0.9829962849617004,0.017003759741783142],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":5,"variant":"base","tokens":168,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0031392574310302734,"mean_abs_dp":0.0022099584341049193,"oracle_top2_margin":0.22992277145385742,"near_tie":false,"probs":[0.035180702805519104,0.03645547106862068,0.555886447429657,0.045760106295347214,0.3267172873020172],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":6,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0049925148487091064,"mean_abs_dp":0.004992499947547913,"oracle_top2_margin":0.44855186343193054,"near_tie":false,"probs":[0.2707315683364868,0.7292684316635132],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":7,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003349602222442627,"mean_abs_dp":0.0016850996762514114,"oracle_top2_margin":0.6617178320884705,"near_tie":false,"probs":[0.04542052745819092,0.0830879881978035,0.7644464373588562,0.10704505443572998],"hidden":{"min_pos_cos":0.9998533746578387,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L01","q":0,"variant":"base","tokens":1793,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008969306945800781,"mean_abs_dp":0.0004743847995996475,"oracle_top2_margin":0.5732508897781372,"near_tie":false,"probs":[0.03346693888306618,0.7462090849876404,0.03301747888326645,0.012732859700918198,0.17457368969917297]},{"id":"own_L01","q":1,"variant":"base","tokens":1766,"calls":111,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005126595497131348,"mean_abs_dp":0.00512656569480896,"oracle_top2_margin":0.3969528377056122,"near_tie":false,"probs":[0.3066501319408417,0.6933498382568359]},{"id":"own_L01","q":2,"variant":"base","tokens":1802,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.011092573404312134,"mean_abs_dp":0.0038332268595695496,"oracle_top2_margin":0.03610238432884216,"near_tie":false,"probs":[0.052647143602371216,0.0761098563671112,0.10979142785072327,0.14263837039470673,0.32211509346961975,0.2966981530189514]},{"id":"own_L02","q":0,"variant":"base","tokens":1518,"calls":95,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.008796393871307373,"mean_abs_dp":0.003518563136458397,"oracle_top2_margin":0.43417489528656006,"near_tie":false,"probs":[0.17836809158325195,0.030267436057329178,0.07325278967618942,0.5989121794700623,0.11919949948787689],"hidden":{"min_pos_cos":0.9990858151789371,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":1,"variant":"base","tokens":1497,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003178715705871582,"mean_abs_dp":0.0031786859035491943,"oracle_top2_margin":0.7794755697250366,"near_tie":false,"probs":[0.8929164409637451,0.1070835143327713],"hidden":{"min_pos_cos":0.9990858151789371,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":2,"variant":"base","tokens":1539,"calls":97,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012988448143005371,"mean_abs_dp":0.0006494414992630482,"oracle_top2_margin":0.735587477684021,"near_tie":false,"probs":[0.8549135327339172,0.010349120944738388,0.12146366387605667,0.01327376626431942],"hidden":{"min_pos_cos":0.9990858151789371,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":3,"variant":"base","tokens":1499,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.470805525779724e-05,"mean_abs_dp":9.468011558055878e-05,"oracle_top2_margin":0.8828529715538025,"near_tie":false,"probs":[0.058668218553066254,0.9413318037986755],"hidden":{"min_pos_cos":0.9990858151789371,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L03","q":0,"variant":"base","tokens":1475,"calls":93,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.016770899295806885,"mean_abs_dp":0.008432215545326471,"oracle_top2_margin":0.4104243218898773,"near_tie":false,"probs":[0.02679143100976944,0.6322270035743713,0.25505930185317993,0.08592233061790466]},{"id":"own_L03","q":1,"variant":"base","tokens":1445,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0047508180141448975,"mean_abs_dp":0.004750773310661316,"oracle_top2_margin":0.6987161636352539,"near_tie":false,"probs":[0.15539270639419556,0.8446073532104492]},{"id":"own_L03","q":2,"variant":"base","tokens":1451,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0030238404870033264,"mean_abs_dp":0.003023821860551834,"oracle_top2_margin":0.8095381259918213,"near_tie":false,"probs":[0.907792866230011,0.09220708906650543]},{"id":"own_L03","q":3,"variant":"base","tokens":1463,"calls":92,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001764059066772461,"mean_abs_dp":0.0010569775477051735,"oracle_top2_margin":0.2239849865436554,"near_tie":false,"probs":[0.07622689008712769,0.32369473576545715,0.5455651879310608,0.054513219743967056]}],"generated_at":"2026-10-03T08:26:42+00:00"},"red_int8lin":{"schema":"kev-decoder-readout-red-arms/1","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8lin_pf16","name":"kev_0_8b_decode_int8lin_pf16","kind":"decision-backbone","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":186,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"a35475102eb71c97f7bf73cc39afcba2d4de14c1bb9986c2b1aa7025980d818e","main_mlirb":{"bytes":1040063350,"sha256":"2281e56d413cbc38cde4bd37953fd5476726a1f37953f940eba9dadcc0bfdb77"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8lin_pf16.h16c.aimodelc","bytes":2039283435,"tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"9917e9b4280745b24b9c0ae5d6d3091bc14cae450202661efa1880e223a3ae32","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"8f100f2739ee2c23945f6eb612cc072de70df00e5f3e00de73b4413ec09ddeda","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"1cda3ad44b1fdf2670ab96857fcd2b7833284c9672a32a6db12c36b6be40d8cf","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"be1be4361facb8d030824a04fac62e5d2d5a9935b5aab4b260203a6704f058e9","main-h16c.mlirb":"be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950","main.hash":"33638a7f1ce95a4fb9c85161d9f04c529ac3e95ef99d8d5cb5d471b0a72f0739","metadata.json":"47e965dcd7035971c4e5ffc59ae2473c390d6a32efe4f22d06ed6c54834e2c66","stats.json":"50309ea498f9c14ee6c2bf20909e32b57140b2ab81649885404ea5e6abfc4c19"}}},"chunk":16,"max_ctx":4096,"rows_file":{"path":"$ZOO_WORK_ROOT/_kev/readout/red_arms_v2.json","sha256":"2de94c557e94ecf318242331d7bcba61e1372e4b4fb0ab8e95a695cd0ca0be89","version":2},"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16.json","sha256":"610053646a4571869615d0c9ac584f8ad8d5749c21f82597118ad4d03b3f5730","result":"FAIL"},"script":{"path":"conversion/kev/readout_gate.py","sha256":"2a31a63316d313ef3aa1f22addecbc94d55c7d0cd3527c6b8f39f21e4d1de61b"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"}},"process":[{"shard":"red_arms_v2","runs":["tv4_000:q0:base","tv4_000:q0:state_swap","semif_a3f18f3a63d45345942b:q0:base","semif_a3f18f3a63d45345942b:q0:state_swap","semif_0b43ea8e24e74d621f1b:q0:base","semif_0b43ea8e24e74d621f1b:q0:state_swap","own_t01:q0:base","own_t01:q0:state_swap","tv4x_qnli_00:q0:base","tv4x_qnli_00:q0:state_swap","tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_00:q0:instr_negate_v2","tv4x_qnli_00:q0:instr_negate_v2","tv4x_paws_00:q0:base","tv4x_paws_00:q0:instr_negate_v2","tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_00:q0:instr_negate_v2","own_t05:q2:base","own_t05:q2:instr_negate_v2"],"prompts":20,"load_seconds":1.1536205419979524,"load_split_seconds":{"model":0.006396999990101904,"main":1.1472235420078505},"function_names":["main"],"process_wall_seconds":5.299351749999914,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13075983300223015},"contract_mismatch":[]}],"rule":"red = the perturbed runs against the unperturbed ones fail the gate's own bar (FACTS §7): (a) an argmax moves on at least one question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over runs of the run's mean |dp| > 0.002","v1_rule":"argmax moved on >= 1 row AND max |dp| > 0.02","arms":{"instr_negate_v2":{"rows":5,"argmax_moved":2,"argmax_moved_non_near_tie":2,"max_abs_dp_vs_base":0.7648195866495371,"mean_of_run_mean_abs_dp_vs_base":0.29320039693266153,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4x_tweet_offensive_00:q0","type":"noul","state_from":null,"tokens":69,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08203236758708954,"mean_abs_dp_vs_base":0.08203234523534775,"base_probs":[0.26064208149909973,0.7393578886985779],"perturbed_probs":[0.1786097139120102,0.8213902115821838],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":null,"tokens":95,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08581054210662842,"mean_abs_dp_vs_base":0.08581051975488663,"base_probs":[0.11414948105812073,0.8858505487442017],"perturbed_probs":[0.19995997846126556,0.8000400066375732],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_paws_00:q0","type":"noul","state_from":null,"tokens":82,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":5.933642387390137e-05,"mean_abs_dp_vs_base":5.932152271270752e-05,"base_probs":[0.3120805025100708,0.6879194378852844],"perturbed_probs":[0.3121398389339447,0.6878601312637329],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_legacy_holdout_00:q0","type":"noul","state_from":null,"tokens":79,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.5332802385091782,"mean_abs_dp_vs_base":0.5332802161574364,"base_probs":[0.14972122013568878,0.8502787947654724],"perturbed_probs":[0.6830014586448669,0.31699860095977783],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t05:q2","type":"noul","state_from":null,"tokens":118,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.7648195866495371,"mean_abs_dp_vs_base":0.7648195819929242,"base_probs":[0.972375750541687,0.02762422524392605],"perturbed_probs":[0.20755617320537567,0.7924438118934631],"base_argmax":0,"perturbed_argmax":1,"oracle_argmax":0}]},"state_swap":{"rows":5,"argmax_moved":4,"argmax_moved_non_near_tie":4,"max_abs_dp_vs_base":0.9048093687742949,"mean_of_run_mean_abs_dp_vs_base":0.45287297501539187,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4_000:q0","type":"choice","state_from":"tv4_001","tokens":80,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.022904187440872192,"mean_abs_dp_vs_base":0.01834271103143692,"base_probs":[0.14792905747890472,0.2063399702310562,0.4324827790260315,0.21324819326400757],"perturbed_probs":[0.13088423013687134,0.2292441576719284,0.41284218430519104,0.22702942788600922],"base_argmax":2,"perturbed_argmax":2,"oracle_argmax":2},{"row":"semif_a3f18f3a63d45345942b:q0","type":"choice","state_from":"semif_0b43ea8e24e74d621f1b","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9048093687742949,"mean_abs_dp_vs_base":0.6032062731683254,"base_probs":[0.020859764888882637,0.16328774392604828,0.8158525228500366],"perturbed_probs":[0.9256691336631775,0.049448706209659576,0.024882109835743904],"base_argmax":2,"perturbed_argmax":0,"oracle_argmax":2},{"row":"semif_0b43ea8e24e74d621f1b:q0","type":"choice","state_from":"semif_942ec65dac0bfb9874f9","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8772313892841339,"mean_abs_dp_vs_base":0.5848209156344334,"base_probs":[0.03737448900938034,0.9412896633148193,0.021335849538445473],"perturbed_probs":[0.49577122926712036,0.06405827403068542,0.4401704668998718],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t01:q0","type":"choice","state_from":"own_t03","tokens":245,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.6905156821012497,"mean_abs_dp_vs_base":0.2806746275164187,"base_probs":[0.5425770282745361,0.03400913253426552,0.14745785295963287,0.06974665075540543,0.20620925724506378],"perturbed_probs":[0.00613804766908288,0.045180048793554306,0.007473149336874485,0.04448379576206207,0.8967249393463135],"base_argmax":0,"perturbed_argmax":4,"oracle_argmax":0},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":"tv4x_paws_00","tokens":52,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.7773203551769257,"mean_abs_dp_vs_base":0.7773203477263451,"base_probs":[0.11414948105812073,0.8858505487442017],"perturbed_probs":[0.8914698362350464,0.1085302084684372],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1}]}},"base_vs_gate_run":[{"row":"tv4_000:q0","bit_equal_gate_run":true},{"row":"semif_a3f18f3a63d45345942b:q0","bit_equal_gate_run":true},{"row":"semif_0b43ea8e24e74d621f1b:q0","bit_equal_gate_run":true},{"row":"own_t01:q0","bit_equal_gate_run":true},{"row":"tv4x_qnli_00:q0","bit_equal_gate_run":true},{"row":"tv4x_tweet_offensive_00:q0","bit_equal_gate_run":true},{"row":"tv4x_paws_00:q0","bit_equal_gate_run":true},{"row":"tv4x_legacy_holdout_00:q0","bit_equal_gate_run":true},{"row":"own_t05:q2","bit_equal_gate_run":true}],"base_bit_equal_all":true,"all_arms_red":true,"reset_bit_equal":true,"result":"RED (every arm)","seconds":7.47131883300608,"generated_at":"2026-10-03T08:27:01+00:00"},"export_int8mix":{"mode":"int8mix","name":"kev_0_8b_decode_int8mix_pf16","hf_id":"kev-local/kev-0.8b-v1.0-merged","weights_sha256":"ab6bd41853ef2a45d56923202e15c8c9c3dcedaec152b1fb8e529f1ec208415f","prefill_chunk":16,"argv":["int8mix","--fp16-layers","0,1,2,3,4,5,6,7,8,9,10,11","--prefill-chunk","16","--aot","--record","$ZOO_WORK_ROOT/_kev/results/export_int8mix_pf16.json"],"started":"2026-10-03T19:29:45+09:00","script_sha256":"499095cd70ebdd0b9816865606d8de696e83d026ec6fc18200eb7007881f5040","module_sha256":"99a32b2d59b9186d8575433075e959a26ac111c9b2d05a9496330aa739c4e41f","export":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8mix_pf16","name":"kev_0_8b_decode_int8mix_pf16","aimodel":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8mix_pf16/kev_0_8b_decode_int8mix_pf16.aimodel","load_report":{"checkpoint_files":["model.safetensors"],"checkpoint_keys":320,"module_tensors":320,"unread_checkpoint_keys":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_word_embeddings":true,"module_has_lm_head_weight":false,"meta_params":[],"dtype":"float16"},"loopfree_unrolled_linear_layers":18,"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":16,"gdn_scan":"unroll","output_names":["hidden"],"quantization":{"linear":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_layer_patterns":["model\\.layers\\.0\\..*","model\\.layers\\.1\\..*","model\\.layers\\.2\\..*","model\\.layers\\.3\\..*","model\\.layers\\.4\\..*","model\\.layers\\.5\\..*","model\\.layers\\.6\\..*","model\\.layers\\.7\\..*","model\\.layers\\.8\\..*","model\\.layers\\.9\\..*","model\\.layers\\.10\\..*","model\\.layers\\.11\\..*"],"int8_linear_modules":93,"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"int8_params":248807424,"lm_head":"none in the graph (the tied table is the fp16 embedding)","seconds":11.838807874999475},"seconds":{"load":0.16355145801207982,"quantize":11.838860999996541,"export":43.815435917000286,"optimize":0.28958229199633934,"export_optimize":44.105018208996626,"save":1.1992726250027772,"total":57.397415792016545},"du_aimodel":"1.2G","du_bundle":"1.2G","aimodel_files":{"main.hash":32,"main.mlirb":1273276920,"metadata.json":105},"main_mlirb":{"bytes":1273276920,"sha256":"84b37b90783782a80f4a348a69e7cccbe96156d1b8a5907b6f25a399beb7dadf"},"metadata_sha256":"218cf045abff64f6d7ffaa4df3673c5e6852a3890f7335267d8b67746bc333b1","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"tokenizer_verbatim_from_snapshot":true,"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"disk":{"before_load":{"time":"2026-10-03T19:29:48+09:00","free_bytes":600887758848,"free_gib":559.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_load":{"time":"2026-10-03T19:29:48+09:00","free_bytes":600887758848,"free_gib":559.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_export":{"time":"2026-10-03T19:30:44+09:00","free_bytes":600888492032,"free_gib":559.6,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after_save":{"time":"2026-10-03T19:30:46+09:00","free_bytes":599108575232,"free_gib":558.0,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8mix_pf16.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":15.708862250001403,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8mix_pf16/kev_0_8b_decode_int8mix_pf16.aimodel","--output","$ZOO_WORK_ROOT/_kev/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-03T19:30:46+09:00","free_bytes":599602163712,"free_gib":558.4,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}},"after":{"time":"2026-10-03T19:31:02+09:00","free_bytes":597331267584,"free_gib":556.3,"coreai_cache_python":{"456862b877df0b5d1a7f2562243f2cdd425cf9512035082058f874361eb21857":"2.3G","be38a5bfef3aeace8e0a7ecd3dc144f8b321c60d27e8a91598acd72d89987950":"1.9G","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G"}}},"du_aimodelc":"2.1G","digest":{"bytes":2272483979,"tree_sha256":"a551fda4c99dea9db6841efa22cec3d7067deb61a4ae7d8c6791b2d2832ef000","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"d0206168f8bdf0d9cc30e09076e05ed93883c7c62ac8052c3369219196fd8e75","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"aa6b981d712f3c85f36aab05556dda7dd26ab8f53d980bbc7e20ced7590b6c69","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a2b898e43b32f163f81c0b13548abade65af7dcfdc57a6df85bee5e3a0e7a195","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"0347cebeef326444687ae7f0fac61d0d890bb220a43fa4175c499e2017adc390","main-h16c.mlirb":"717762adc6e3fb54b617c9860ac4d1467d175148878bf01b2831d744f7763506","main.hash":"97a6ba5f28134877a053ac43c66f2a59a4d9e4b47dd58e0557a7b09278e560e7","metadata.json":"f9ecebe00b0ded7788a14464033ef5f1656a804102ad0833949f284cb7737e58","stats.json":"edd7c3ed4bc2d810160fe4159fac84793290fb66725645b4013d56c61db877d6"}}},"finished":"2026-10-03T19:31:03+09:00"},"readout_int8mix":{"schema":"kev-decoder-readout-gate/1","gate":"decoder alone on the Mac GPU (oracle row ids), the fp16 hidden read through the fp32 pointer head","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8mix_pf16","name":"kev_0_8b_decode_int8mix_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":93,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"218cf045abff64f6d7ffaa4df3673c5e6852a3890f7335267d8b67746bc333b1","main_mlirb":{"bytes":1273276920,"sha256":"84b37b90783782a80f4a348a69e7cccbe96156d1b8a5907b6f25a399beb7dadf"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8mix_pf16.h16c.aimodelc","bytes":2272483979,"tree_sha256":"a551fda4c99dea9db6841efa22cec3d7067deb61a4ae7d8c6791b2d2832ef000","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"d0206168f8bdf0d9cc30e09076e05ed93883c7c62ac8052c3369219196fd8e75","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"aa6b981d712f3c85f36aab05556dda7dd26ab8f53d980bbc7e20ced7590b6c69","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a2b898e43b32f163f81c0b13548abade65af7dcfdc57a6df85bee5e3a0e7a195","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"0347cebeef326444687ae7f0fac61d0d890bb220a43fa4175c499e2017adc390","main-h16c.mlirb":"717762adc6e3fb54b617c9860ac4d1467d175148878bf01b2831d744f7763506","main.hash":"97a6ba5f28134877a053ac43c66f2a59a4d9e4b47dd58e0557a7b09278e560e7","metadata.json":"f9ecebe00b0ded7788a14464033ef5f1656a804102ad0833949f284cb7737e58","stats.json":"edd7c3ed4bc2d810160fe4159fac84793290fb66725645b4013d56c61db877d6"}}},"chunk":16,"max_ctx":4096,"readout":"one row per question, fresh zero states per row; a row of T ids runs as ceil(T / 16) calls of 'main' (static S = 16): call k gets ids[16k : 16k + 16] with position_ids 0..16k+15; the last call is padded with <|endoftext|> (248044) and the hidden rows of the padded positions are discarded (causal: they cannot reach a real position). The hidden rows [1, 16, 1024] of every call, concatenated and cut to T, are the backbone's final-norm last_hidden_state [T, 1024] the head reads. A row fits when ceil(T / 16) * 16 <= 4095 (the position dim's upper bound; the KV sequence dim is allocated at 4096).","descriptor_same_in_every_process":true,"contract":{"inputs":{"input_ids":[[1,16],"int32"],"position_ids":[[1,-1],"int32"]},"outputs":{"hidden":[[1,16,1024],"float16"]},"states":{"keyCache":[[6,1,2,-1,256],"float16"],"valueCache":[[6,1,2,-1,256],"float16"],"convState":[[18,1,6144,3],"float16"],"recState":[[18,1,16,128,128],"float16"]}},"oracle":{"path":"$ZOO_WORK_ROOT/_kev/oracle/records_oracle.json","sha256":"1a72a45831eecc2af7ddfc68a86e9250f572d6bf9274362f96cc62bcb803cc6d","checkpoint":"jaredpalmer/kev-0.8b@788ddbdd65715bb03a56788c822f6c632c9a551d","versions":{"python":"3.12.13","torch":"2.8.0","transformers":"5.17.0","peft":"0.21.0","numpy":"2.5.3","safetensors":"0.8.0"}},"subset":"all","subset_file":null,"script":{"path":"conversion/kev/readout_gate.py","sha256":"4bdc3b1f670a4a2abb0104472bde8aac682ac0b0a0b9d2ff5df0aefd9c3cfd91"},"bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","reset":"bit-equal hidden rows in every process","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"}},"processes":[{"shard":"shard_00","runs":["tv4_000:q0:base","tv4_001:q0:base","tv4_002:q0:base","tv4_003:q0:base","tv4_004:q0:base","tv4_005:q0:base","tv4_006:q0:base","tv4_007:q0:base","tv4_008:q0:base","tv4_009:q0:base","tv4_010:q0:base","tv4_011:q0:base","tv4_012:q0:base","tv4_013:q0:base","tv4_014:q0:base","tv4_015:q0:base","tv4_016:q0:base","tv4_017:q0:base","tv4_018:q0:base","tv4_019:q0:base","tv4_020:q0:base","tv4_021:q0:base","tv4_022:q0:base","tv4_023:q0:base","tv4_024:q0:base","tv4_025:q0:base","tv4_026:q0:base","tv4_027:q0:base","tv4_028:q0:base","tv4_029:q0:base","tv4_030:q0:base","tv4_031:q0:base","tv4_032:q0:base","tv4_033:q0:base","tv4_034:q0:base","tv4_035:q0:base","tv4_036:q0:base","tv4_037:q0:base","tv4_038:q0:base","tv4_039:q0:base"],"prompts":41,"load_seconds":2.0243704589956906,"load_split_seconds":{"model":0.5778396670066286,"main":1.446530791989062},"function_names":["main"],"process_wall_seconds":10.714852875011275,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1391192500013858},"contract_mismatch":[]},{"shard":"shard_01","runs":["tv4_040:q0:base","tv4_041:q0:base","tv4_042:q0:base","tv4_043:q0:base","tv4_044:q0:base","tv4_045:q0:base","tv4_046:q0:base","tv4_047:q0:base","tv4_048:q0:base","tv4_049:q0:base","tv4_050:q0:base","tv4_051:q0:base","tv4_052:q0:base","tv4_053:q0:base","tv4_054:q0:base","tv4_055:q0:base","tv4_056:q0:base","tv4_057:q0:base","tv4_058:q0:base","tv4_059:q0:base","tv4x_emotion_00:q0:base","tv4x_emotion_01:q0:base","tv4x_emotion_02:q0:base","tv4x_emotion_03:q0:base","tv4x_emotion_04:q0:base","tv4x_emotion_05:q0:base","tv4x_emotion_06:q0:base","tv4x_emotion_07:q0:base","tv4x_emotion_08:q0:base","tv4x_emotion_09:q0:base","tv4x_emotion_10:q0:base","tv4x_emotion_11:q0:base","tv4x_emotion_12:q0:base","tv4x_emotion_13:q0:base","tv4x_emotion_14:q0:base","tv4x_emotion_15:q0:base","tv4x_emotion_16:q0:base","tv4x_emotion_17:q0:base","tv4x_emotion_18:q0:base","tv4x_emotion_19:q0:base"],"prompts":41,"load_seconds":0.28978074999758974,"load_split_seconds":{"model":0.006401332997484133,"main":0.2833794170001056},"function_names":["main"],"process_wall_seconds":6.421525332989404,"reset_check":{"run":"tv4_040:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.28945941699203104},"contract_mismatch":[]},{"shard":"shard_02","runs":["tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_01:q0:base","tv4x_tweet_offensive_02:q0:base","tv4x_tweet_offensive_03:q0:base","tv4x_tweet_offensive_04:q0:base","tv4x_tweet_offensive_05:q0:base","tv4x_tweet_offensive_06:q0:base","tv4x_tweet_offensive_07:q0:base","tv4x_tweet_offensive_08:q0:base","tv4x_tweet_offensive_09:q0:base","tv4x_tweet_offensive_10:q0:base","tv4x_tweet_offensive_11:q0:base","tv4x_tweet_offensive_12:q0:base","tv4x_tweet_offensive_13:q0:base","tv4x_tweet_offensive_14:q0:base","tv4x_tweet_offensive_15:q0:base","tv4x_tweet_offensive_16:q0:base","tv4x_tweet_offensive_17:q0:base","tv4x_tweet_offensive_18:q0:base","tv4x_tweet_offensive_19:q0:base","tv4x_qnli_00:q0:base","tv4x_qnli_01:q0:base","tv4x_qnli_02:q0:base","tv4x_qnli_03:q0:base","tv4x_qnli_04:q0:base","tv4x_qnli_05:q0:base","tv4x_qnli_06:q0:base","tv4x_qnli_07:q0:base","tv4x_qnli_08:q0:base","tv4x_qnli_09:q0:base","tv4x_qnli_10:q0:base","tv4x_qnli_11:q0:base","tv4x_qnli_12:q0:base","tv4x_qnli_13:q0:base","tv4x_qnli_14:q0:base","tv4x_qnli_15:q0:base","tv4x_qnli_16:q0:base","tv4x_qnli_17:q0:base","tv4x_qnli_18:q0:base","tv4x_qnli_19:q0:base"],"prompts":41,"load_seconds":0.29209058298147283,"load_split_seconds":{"model":0.007880582998041064,"main":0.28420999998343177},"function_names":["main"],"process_wall_seconds":5.463241332996404,"reset_check":{"run":"tv4x_tweet_offensive_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.11157466701115482},"contract_mismatch":[]},{"shard":"shard_03","runs":["tv4x_paws_00:q0:base","tv4x_paws_01:q0:base","tv4x_paws_02:q0:base","tv4x_paws_03:q0:base","tv4x_paws_04:q0:base","tv4x_paws_05:q0:base","tv4x_paws_06:q0:base","tv4x_paws_07:q0:base","tv4x_paws_08:q0:base","tv4x_paws_09:q0:base","tv4x_paws_10:q0:base","tv4x_paws_11:q0:base","tv4x_paws_12:q0:base","tv4x_paws_13:q0:base","tv4x_paws_14:q0:base","tv4x_paws_15:q0:base","tv4x_paws_16:q0:base","tv4x_paws_17:q0:base","tv4x_paws_18:q0:base","tv4x_paws_19:q0:base","tv4x_sciq_00:q0:base","tv4x_sciq_01:q0:base","tv4x_sciq_02:q0:base","tv4x_sciq_03:q0:base","tv4x_sciq_04:q0:base","tv4x_sciq_05:q0:base","tv4x_sciq_06:q0:base","tv4x_sciq_07:q0:base","tv4x_sciq_08:q0:base","tv4x_sciq_09:q0:base","tv4x_sciq_10:q0:base","tv4x_sciq_11:q0:base","tv4x_sciq_12:q0:base","tv4x_sciq_13:q0:base","tv4x_sciq_14:q0:base","tv4x_sciq_15:q0:base","tv4x_sciq_16:q0:base","tv4x_sciq_17:q0:base","tv4x_sciq_18:q0:base","tv4x_sciq_19:q0:base"],"prompts":41,"load_seconds":0.2926525000075344,"load_split_seconds":{"model":0.006697792006889358,"main":0.28595470800064504},"function_names":["main"],"process_wall_seconds":8.063783332996536,"reset_check":{"run":"tv4x_paws_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13740037500974722},"contract_mismatch":[]},{"shard":"shard_04","runs":["tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_01:q0:base","tv4x_legacy_holdout_02:q0:base","tv4x_legacy_holdout_03:q0:base","tv4x_legacy_holdout_04:q0:base","tv4x_legacy_holdout_05:q0:base","tv4x_legacy_holdout_06:q0:base","tv4x_legacy_holdout_07:q0:base","tv4x_legacy_holdout_08:q0:base","tv4x_legacy_holdout_09:q0:base","tv4x_legacy_holdout_10:q0:base","tv4x_legacy_holdout_11:q0:base","tv4x_legacy_holdout_12:q0:base","tv4x_legacy_holdout_13:q0:base","tv4x_legacy_holdout_14:q0:base","tv4x_legacy_holdout_15:q0:base","tv4x_legacy_holdout_16:q0:base","tv4x_legacy_holdout_17:q0:base","tv4x_legacy_holdout_18:q0:base","tv4x_legacy_holdout_19:q0:base","tv4x_composition_holdout_00:q0:base","tv4x_composition_holdout_01:q0:base","tv4x_composition_holdout_02:q0:base","tv4x_composition_holdout_03:q0:base","tv4x_composition_holdout_04:q0:base","tv4x_composition_holdout_05:q0:base","tv4x_composition_holdout_06:q0:base","tv4x_composition_holdout_07:q0:base","tv4x_composition_holdout_08:q0:base","tv4x_composition_holdout_09:q0:base","tv4x_composition_holdout_10:q0:base","tv4x_composition_holdout_11:q0:base","tv4x_composition_holdout_12:q0:base","tv4x_composition_holdout_13:q0:base","tv4x_composition_holdout_14:q0:base","tv4x_composition_holdout_15:q0:base","tv4x_composition_holdout_16:q0:base","tv4x_composition_holdout_17:q0:base","tv4x_composition_holdout_18:q0:base","tv4x_composition_holdout_19:q0:base"],"prompts":41,"load_seconds":0.29000483301933855,"load_split_seconds":{"model":0.006794208020437509,"main":0.28321062499890104},"function_names":["main"],"process_wall_seconds":7.419386333000148,"reset_check":{"run":"tv4x_legacy_holdout_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.11063733301125467},"contract_mismatch":[]},{"shard":"shard_05","runs":["tv4s_00:q0:base","tv4s_01:q0:base","tv4s_02:q0:base","tv4s_03:q0:base","tv4s_04:q0:base","tv4s_05:q0:base","tv4s_06:q0:base","tv4s_07:q0:base","tv4s_08:q0:base","tv4s_09:q0:base","tv4s_10:q0:base","tv4s_11:q0:base","tv4s_12:q0:base","tv4s_13:q0:base","tv4s_14:q0:base","tv4s_15:q0:base","tv4s_16:q0:base","tv4s_17:q0:base","tv4s_18:q0:base","tv4s_19:q0:base","semif_a3f18f3a63d45345942b:q0:base","semif_f40beba9088c8db8bbd6:q0:base","semif_0b43ea8e24e74d621f1b:q0:base","semif_942ec65dac0bfb9874f9:q0:base","semif_e0c140e9222d1bce5647:q0:base","semif_8e8c3804a3c15ebb31e3:q0:base","semif_56c419b12cd37cbe790c:q0:base","semif_75b470586834df8c568d:q0:base","semif_834f3b3e10cc61b33d25:q0:base","semif_76c7a79b1ff2363a60c5:q0:base","semif_36c71bb649b7d5eccb1f:q0:base","semif_eac5121a4a1bf391a08c:q0:base","semif_44574c44e243ee1b2f52:q0:base","semif_2608370d01783735c3ea:q0:base","semif_ec94ef909f7918695ab3:q0:base","semif_03d919fd0e9b3a7cd1e6:q0:base","semif_86a15618ba4cb90a307e:q0:base","semif_3dd9c3ac98f8ddf7bdd3:q0:base","semif_36273dd4777876a26338:q0:base","semif_07ee70cfe4b01330ce89:q0:base"],"prompts":41,"load_seconds":0.29440587499993853,"load_split_seconds":{"model":0.006572999991476536,"main":0.287832875008462},"function_names":["main"],"process_wall_seconds":6.08174229200813,"reset_check":{"run":"tv4s_00:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1616046660055872},"contract_mismatch":[]},{"shard":"shard_06","runs":["semif_51ceab3ce622c7c759f1:q0:base","semif_b8091be929b1e489fcc4:q0:base","semif_c89b86e3f55123bfa506:q0:base","semif_3b0958cabc339005cf66:q0:base","semif_f2b4ec4930322fe45118:q0:base","semif_61ac20b07a5189231e15:q0:base","semif_4b3d794c004566610181:q0:base","semif_b7c4e62d07986286fce0:q0:base","semif_9473603b44af63beab79:q0:base","semif_d41072dc35d8cd14c056:q0:base","semif_33d6e5da58f0fee2490d:q0:base","semif_823bf736b92e9d8fa66d:q0:base","semif_8a4c3de28be95c105ca1:q0:base","semif_ea7ec66a5e8eac886548:q0:base","semif_25172cd6ddc49ab0ca70:q0:base","semif_fb73a85d386cf59b0c51:q0:base","semif_a02fd582d2881cc462b2:q0:base","semif_19f37d5a97034bd7d04f:q0:base","semif_fda0ca94652c6d739ecf:q0:base","semif_d0e36c4cb10a29b3191c:q0:base","semif_a0a39ace7e5337d038a7:q0:base","semif_eeae283841ce75dd25bf:q0:base","semif_00bb4cc043f7d2b288d4:q0:base","semif_1a81ead69e93e5d0c86a:q0:base","semif_4444dad25e438f128673:q0:base","semif_3226202ba1ada2570e9f:q0:base","semif_3f783eec5c90812479bc:q0:base","semif_ffee861474a9665a093a:q0:base","semif_8c992e17e4fb54b044f5:q0:base","semif_695dd61195ca025c4f4c:q0:base","semif_e6155431ab7b0f8170d4:q0:base","semif_b49ecda75086d8c68d2a:q0:base","semif_587ced261c6092cd1325:q0:base","semif_533d4423d311de82b27d:q0:base","semif_2bf540d515a51755b25c:q0:base","semif_fdb9fc214aea2710763a:q0:base","semif_f6d68126b43f8c9ed183:q0:base","semif_44bae0c8a1615a096c43:q0:base","semif_2175379f4c69e6207626:q0:base","semif_def9fe59d262357a673a:q0:base"],"prompts":41,"load_seconds":0.2889024169999175,"load_split_seconds":{"model":0.006785417004721239,"main":0.28211699999519624},"function_names":["main"],"process_wall_seconds":5.2982688340125605,"reset_check":{"run":"semif_51ceab3ce622c7c759f1:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.11174516699975356},"contract_mismatch":[]},{"shard":"shard_07","runs":["semif_712f6ffb763eef7c9a12:q0:base","semif_e1d610bd14e3d16c09b9:q0:base","semif_22ec57822db50ec121cf:q0:base","semif_3e75f2b84e8e08e3d3f2:q0:base","semif_ebd74aa05324a1556288:q0:base","semif_d7e84e4245a29817da1f:q0:base","semif_f41ae66b3c09630c7a42:q0:base","semif_82c2cd97c160ad27951b:q0:base","semif_7ada97b6f99f4788e5c7:q0:base","semif_4d7a003342db5b0a17e3:q0:base","semif_19d3d8d0bf773e88be7b:q0:base","semif_d6de731cca52359b1c87:q0:base","semif_b45c63fb92729af27f05:q0:base","semif_46b7029b9a704138b77a:q0:base","semif_caf773b23934fa7362a1:q0:base","semif_f46f392ef9e9e9df564b:q0:base","semif_005dcf6d2c0f21d64796:q0:base","semif_edb708bd0fe5fd67d6ee:q0:base","semif_97a35499bd8717f3e786:q0:base","semif_31eb4533d2c89959ec46:q0:base","semif_0aa24a6701ac9c56c4a0:q0:base","semif_796d5c0da6eaadae6a3f:q0:base","semif_599b3092b6fd18312d23:q0:base","semif_960a4bcd27363e6db4b9:q0:base","semif_c6a411e7f2bbdf377173:q0:base","semif_e332aa927d0c2ed044aa:q0:base","semif_a0128e551d095f3449f8:q0:base","semif_46d6a0731bd288c370f0:q0:base","semif_21ca7a8b970e36b8e20f:q0:base","semif_4aa83af95bb69e90bc2d:q0:base","semif_cdfa9eed51fafcf09b72:q0:base","semif_d51b05bfb9d6d7f09331:q0:base","semif_fdd902cc6e65fad7cb4f:q0:base","semif_42384af9ae56beb37ff5:q0:base","semif_cdf7f428857e1b841cb3:q0:base","semif_774f7f9245f83c796d44:q0:base","semif_92ec2ffa3dd46ada1335:q0:base","semif_6149a17bc154f9c5b4a0:q0:base","semif_be866e1dbec5d42d9423:q0:base","semif_97d2ed2591fc7638806c:q0:base"],"prompts":41,"load_seconds":0.2962748330028262,"load_split_seconds":{"model":0.007544166001025587,"main":0.2887306670018006},"function_names":["main"],"process_wall_seconds":5.603783458005637,"reset_check":{"run":"semif_712f6ffb763eef7c9a12:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.08956208301242441},"contract_mismatch":[]},{"shard":"shard_08","runs":["semif_4533dd91267b4d815690:q0:base","semif_d880e65fef5e2993fdd0:q0:base","semif_90494c76078ffc3c717a:q0:base","semif_9ac5cc82c4126c91174a:q0:base","semif_386e10b1c1ce906473a4:q0:base","semif_4da6aedc1139224eeff2:q0:base","semif_a22dab8ba073ff6cdb8d:q0:base","semif_42c54fa2ad45743e24d8:q0:base","semif_f3d13cf184f916677def:q0:base","semif_1105577da4c8dad4609d:q0:base","semif_0ebd299739302d8b057b:q0:base","semif_eafc22c8c40df3932a8e:q0:base","semif_0aa1447680e4f25862bb:q0:base","semif_980e76145c925747220c:q0:base","semif_4aee37a6a155747766aa:q0:base","semif_6db17cc22bd656d78558:q0:base","semif_ef3e0f8e657bf59f8a29:q0:base","semif_f756e1cb04e11766341e:q0:base","semif_d167e51bb1f499099645:q0:base","semif_674ca4e9346216255466:q0:base","semif_1df851a202994a07f123:q0:base","semif_b5e6788d95b8395d2888:q0:base","semif_32f36cd0fbbfb3604586:q0:base","semif_843185706adff0c60f8b:q0:base","semif_b531dd603bf080dab820:q0:base","semif_be2dd67bc733c1213bef:q0:base","semif_b8e2750307df9761e716:q0:base","semif_5d31b4bf2d0fc193f53b:q0:base","semif_a9230edfc6ec9e524af0:q0:base","semif_c100e84585149a04be49:q0:base","semif_579beed184ffc1104aa9:q0:base","semif_c4e9bd463b6fde5be743:q0:base","semif_b863998d4be2721a5dfd:q0:base","semif_37cebc3792e98049439f:q0:base","semif_64d63f86e328aff1481c:q0:base","semif_6d64a8459f64eac7f4cc:q0:base","semif_2708193212d8a4a523c7:q0:base","semif_5a37d1dfcad20810a76a:q0:base","semif_acacd8b46acd25cb954b:q0:base","semif_4089c199b8b796ec3ed8:q0:base"],"prompts":41,"load_seconds":0.2863593329966534,"load_split_seconds":{"model":0.006503582990262657,"main":0.27985575000639074},"function_names":["main"],"process_wall_seconds":5.942246584017994,"reset_check":{"run":"semif_4533dd91267b4d815690:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13521112501621246},"contract_mismatch":[]},{"shard":"shard_09","runs":["semif_02f2b95a5c5a4777a45c:q0:base","semif_ebea4681e3dcc7d92ca2:q0:base","semif_b8c5b9b285cbdddc9f8d:q0:base","semif_7ed75d854d99b3a81916:q0:base","own_t01:q0:base","own_t01:q1:base","own_t01:q2:base","own_t01:q3:base","own_t02:q0:base","own_t02:q1:base","own_t02:q2:base","own_t03:q0:base","own_t03:q1:base","own_t03:q2:base","own_t04:q0:base","own_t04:q1:base","own_t04:q2:base","own_t04:q3:base","own_t05:q0:base","own_t05:q1:base","own_t05:q2:base","own_t05:q3:base","own_t06:q0:base","own_t06:q1:base","own_t06:q2:base","own_t07:q0:base","own_t07:q1:base","own_t07:q2:base","own_t08:q0:base","own_t08:q1:base","own_t08:q2:base","own_t09:q0:base","own_t09:q1:base","own_t10:q0:base","own_t10:q1:base","own_t10:q2:base","own_j01:q0:base","own_j01:q1:base","own_j01:q2:base","own_j01:q3:base"],"prompts":41,"load_seconds":0.2836708330141846,"load_split_seconds":{"model":0.006568915996467695,"main":0.2771019170177169},"function_names":["main"],"process_wall_seconds":11.136234041012358,"reset_check":{"run":"semif_02f2b95a5c5a4777a45c:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.11312958298367448},"contract_mismatch":[]},{"shard":"shard_10","runs":["own_j02:q0:base","own_j02:q1:base","own_j02:q2:base","own_j03:q0:base","own_j03:q1:base","own_j03:q2:base","own_j04:q0:base","own_j04:q1:base","own_j04:q2:base","own_j05:q0:base","own_j05:q1:base","own_j05:q2:base","own_a01:q0:base","own_a01:q1:base","own_a01:q2:base","own_m01:q0:base","own_m01:q1:base","own_m01:q2:base","own_m01:q3:base","own_m01:q4:base","own_m01:q5:base","own_m01:q6:base","own_m01:q7:base","own_L01:q0:base","own_L01:q1:base","own_L01:q2:base","own_L02:q0:base","own_L02:q1:base","own_L02:q2:base","own_L02:q3:base","own_L03:q0:base","own_L03:q1:base","own_L03:q2:base","own_L03:q3:base"],"prompts":35,"load_seconds":0.28494929199223407,"load_split_seconds":{"model":0.006652832991676405,"main":0.27829645900055766},"function_names":["main"],"process_wall_seconds":31.482866250007646,"reset_check":{"run":"own_j02:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.4606764999916777},"contract_mismatch":[]}],"summary":{"runs":434,"expected_runs":434,"missing_runs":[],"questions":434,"argmax_equal":432,"questions_non_near_tie":420,"argmax_equal_non_near_tie":420,"near_tie_questions":14,"argmax_equal_near_tie":12,"max_abs_dp":0.00888979434967041,"max_abs_dp_non_near_tie":0.00888979434967041,"mean_of_run_mean_abs_dp":0.0010320237972326677,"mean_of_record_mean_abs_dp":0.0010586527016070856,"mean_question_max_abs_dp":0.0015197324047234201,"max_abs_dlogit_T":0.06255793571472168,"worst_run":{"id":"semif_46b7029b9a704138b77a","q":0,"max_abs_dp":0.00888979434967041},"finite_all":true,"all_zero_runs":0,"hidden_rows":21,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.23488569259643555,"positions":9168,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":61556,"padded_tokens":64880,"calls":4055,"pad_waste":0.05123304562268804,"decode_ms_total":93402.73120478378,"ms_per_call_median":21.749290986917913,"ms_per_call_warm_median":21.733832996687852},"by_source":{"own_json":{"runs":19,"expected_runs":19,"missing_runs":[],"questions":19,"argmax_equal":19,"questions_non_near_tie":18,"argmax_equal_non_near_tie":18,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.0042130351066589355,"max_abs_dp_non_near_tie":0.0042130351066589355,"mean_of_run_mean_abs_dp":0.0007849429375760463,"mean_of_record_mean_abs_dp":0.0007080117775859614,"mean_question_max_abs_dp":0.001204981615668849,"max_abs_dlogit_T":0.01808631420135498,"worst_run":{"id":"own_j04","q":0,"max_abs_dp":0.0042130351066589355},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.20166587829589844,"positions":1027,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":4323,"padded_tokens":4464,"calls":279,"pad_waste":0.031586021505376344,"decode_ms_total":6446.340248978231,"ms_per_call_median":21.774499997263774,"ms_per_call_warm_median":21.720499513321556},"own_long":{"runs":11,"expected_runs":11,"missing_runs":[],"questions":11,"argmax_equal":11,"questions_non_near_tie":11,"argmax_equal_non_near_tie":11,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.006616711616516113,"max_abs_dp_non_near_tie":0.006616711616516113,"mean_of_run_mean_abs_dp":0.0011827149174430153,"mean_of_record_mean_abs_dp":0.0012399833410596235,"mean_question_max_abs_dp":0.002016228031028401,"max_abs_dlogit_T":0.03536725044250488,"worst_run":{"id":"own_L02","q":0,"max_abs_dp":0.006616711616516113},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.23488569259643555,"positions":6053,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":17248,"padded_tokens":17344,"calls":1084,"pad_waste":0.005535055350553505,"decode_ms_total":23464.569498959463,"ms_per_call_median":21.68104199517984,"ms_per_call_warm_median":21.68104199517984},"own_text":{"runs":40,"expected_runs":40,"missing_runs":[],"questions":40,"argmax_equal":40,"questions_non_near_tie":39,"argmax_equal_non_near_tie":39,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.005302995443344116,"max_abs_dp_non_near_tie":0.005302995443344116,"mean_of_run_mean_abs_dp":0.0007270420944162955,"mean_of_record_mean_abs_dp":0.000696522946538034,"mean_question_max_abs_dp":0.0011390318162739278,"max_abs_dlogit_T":0.0301588773727417,"worst_run":{"id":"own_t04","q":3,"max_abs_dp":0.005302995443344116},"finite_all":true,"all_zero_runs":0,"hidden_rows":12,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.21360397338867188,"positions":1926,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7247,"padded_tokens":7472,"calls":467,"pad_waste":0.03011241970021413,"decode_ms_total":10134.433666884433,"ms_per_call_median":21.693582995794713,"ms_per_call_warm_median":21.693582995794713},"semif_authored144":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":143,"questions_non_near_tie":141,"argmax_equal_non_near_tie":141,"near_tie_questions":3,"argmax_equal_near_tie":2,"max_abs_dp":0.00888979434967041,"max_abs_dp_non_near_tie":0.00888979434967041,"mean_of_run_mean_abs_dp":0.0013074599954523837,"mean_of_record_mean_abs_dp":0.0013074599954523837,"mean_question_max_abs_dp":0.001961186398855514,"max_abs_dlogit_T":0.04139852523803711,"worst_run":{"id":"semif_46b7029b9a704138b77a","q":0,"max_abs_dp":0.00888979434967041},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.1971721649169922,"positions":68,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":10574,"padded_tokens":11664,"calls":729,"pad_waste":0.09344993141289437,"decode_ms_total":17403.64858493558,"ms_per_call_median":21.759958006441593,"ms_per_call_warm_median":21.752333996118978},"transfer_v4_dev_head":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":59,"questions_non_near_tie":52,"argmax_equal_non_near_tie":52,"near_tie_questions":8,"argmax_equal_near_tie":7,"max_abs_dp":0.006994843482971191,"max_abs_dp_non_near_tie":0.006994843482971191,"mean_of_run_mean_abs_dp":0.0008637683308431103,"mean_of_record_mean_abs_dp":0.0008637683308431103,"mean_question_max_abs_dp":0.0016228961758315563,"max_abs_dlogit_T":0.03673839569091797,"worst_run":{"id":"tv4_020","q":0,"max_abs_dp":0.006994843482971191},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9998692690427006,"hidden_max_abs_diff":0.1505298614501953,"positions":94,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":7021,"padded_tokens":7504,"calls":469,"pad_waste":0.06436567164179105,"decode_ms_total":12115.230126015376,"ms_per_call_median":21.986916981404647,"ms_per_call_warm_median":21.983104496030137},"transfer_v4_dev_score":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":20,"questions_non_near_tie":19,"argmax_equal_non_near_tie":19,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.002020508050918579,"max_abs_dp_non_near_tie":0.0017922818660736084,"mean_of_run_mean_abs_dp":0.0007784509410460789,"mean_of_record_mean_abs_dp":0.0007784509410460789,"mean_question_max_abs_dp":0.0011676758527755736,"max_abs_dlogit_T":0.013297557830810547,"worst_run":{"id":"tv4s_14","q":0,"max_abs_dp":0.002020508050918579},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":1979,"padded_tokens":2240,"calls":140,"pad_waste":0.11651785714285715,"decode_ms_total":3437.4012930493336,"ms_per_call_median":21.9752495031571,"ms_per_call_warm_median":21.92216698313132},"transfer_v4_dev_sources":{"runs":140,"expected_runs":140,"missing_runs":[],"questions":140,"argmax_equal":140,"questions_non_near_tie":140,"argmax_equal_non_near_tie":140,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.004893302917480469,"max_abs_dp_non_near_tie":0.004893302917480469,"mean_of_run_mean_abs_dp":0.0009658821878561174,"mean_of_record_mean_abs_dp":0.0009658821878561174,"mean_question_max_abs_dp":0.0011842235922813415,"max_abs_dlogit_T":0.06255793571472168,"worst_run":{"id":"tv4x_sciq_11","q":0,"max_abs_dp":0.004893302917480469},"finite_all":true,"all_zero_runs":0,"hidden_rows":0,"min_pos_cos":null,"hidden_max_abs_diff":null,"positions":0,"positions_below":{"0.99":0,"0.9":0,"0.5":0},"tokens":13164,"padded_tokens":14192,"calls":887,"pad_waste":0.07243517474633597,"decode_ms_total":20401.107785961358,"ms_per_call_median":21.743624994996935,"ms_per_call_warm_median":21.72416599933058}},"near_ties":[{"id":"tv4_003","q":0,"qid":"answer","oracle_top2_margin":0.016063004732131958,"argmax_equal":true,"max_abs_dp":0.0014964640140533447,"probs":[0.41221970319747925,0.3933868706226349,0.07267779856920242,0.12171569466590881]},{"id":"tv4_017","q":0,"qid":"answer","oracle_top2_margin":0.01221814751625061,"argmax_equal":true,"max_abs_dp":0.0011511743068695068,"probs":[0.2636023461818695,0.2393730878829956,0.2525893747806549,0.24443519115447998]},{"id":"tv4_023","q":0,"qid":"answer","oracle_top2_margin":0.0019783377647399902,"argmax_equal":false,"max_abs_dp":0.0012638568878173828,"probs":[0.3657609522342682,0.3653772175312042,0.15247812867164612,0.11638376116752625]},{"id":"tv4_024","q":0,"qid":"answer","oracle_top2_margin":0.015634745359420776,"argmax_equal":true,"max_abs_dp":0.0013806819915771484,"probs":[0.3386755585670471,0.27016597986221313,0.03741179406642914,0.3537467122077942]},{"id":"tv4_027","q":0,"qid":"answer","oracle_top2_margin":0.009622842073440552,"argmax_equal":true,"max_abs_dp":0.0012668073177337646,"probs":[0.16842885315418243,0.33257701992988586,0.3437099754810333,0.15528403222560883]},{"id":"tv4_028","q":0,"qid":"answer","oracle_top2_margin":0.01672416925430298,"argmax_equal":true,"max_abs_dp":0.0005431473255157471,"probs":[0.16780264675617218,0.3211795389652252,0.20735663175582886,0.30366119742393494]},{"id":"tv4_051","q":0,"qid":"answer","oracle_top2_margin":0.006280571222305298,"argmax_equal":true,"max_abs_dp":0.0011369287967681885,"probs":[0.05906115472316742,0.18529871106147766,0.3808095455169678,0.37483057379722595]},{"id":"tv4_054","q":0,"qid":"answer","oracle_top2_margin":0.0028791427612304688,"argmax_equal":true,"max_abs_dp":0.0006154179573059082,"probs":[0.13885273039340973,0.27271798253059387,0.29270830750465393,0.29572102427482605]},{"id":"tv4s_14","q":0,"qid":"decision","oracle_top2_margin":0.01574954390525818,"argmax_equal":true,"max_abs_dp":0.002020508050918579,"probs":[0.39459463953971863,0.413241446018219,0.19216391444206238]},{"id":"semif_2175379f4c69e6207626","q":0,"qid":"decision","oracle_top2_margin":0.01896810531616211,"argmax_equal":true,"max_abs_dp":0.002835020422935486,"probs":[0.4181659519672394,0.39976751804351807,0.18206657469272614]},{"id":"semif_d167e51bb1f499099645","q":0,"qid":"decision","oracle_top2_margin":0.01671499013900757,"argmax_equal":true,"max_abs_dp":0.0025804340839385986,"probs":[0.3537586033344269,0.3041044771671295,0.3421368896961212]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"qid":"decision","oracle_top2_margin":0.0032414793968200684,"argmax_equal":false,"max_abs_dp":0.0039997100830078125,"probs":[0.4431862533092499,0.43848466873168945,0.11832913011312485]},{"id":"own_t10","q":2,"qid":"escalation","oracle_top2_margin":0.011951148509979248,"argmax_equal":true,"max_abs_dp":0.0009426772594451904,"probs":[0.3215273320674896,0.27070480585098267,0.30920353531837463,0.09856431931257248]},{"id":"own_j02","q":0,"qid":"next_action","oracle_top2_margin":0.007524773478507996,"argmax_equal":true,"max_abs_dp":0.001283407211303711,"probs":[0.20408323407173157,0.08876383304595947,0.19799970090389252,0.08309917151927948,0.0279974527657032,0.004260205663740635,0.01964416541159153,0.16099944710731506,0.015448244288563728,0.19770458340644836]}],"checks":{"all_runs":true,"argmax_non_near_tie":true,"max_abs_dp":true,"mean_of_run_mean_abs_dp":true,"finite":true,"reset_bit_equal_all_processes":true},"result":"PASS","red_arm":null,"timing":{"contended":true,"S":16,"load_seconds":{"first_process":2.0243704589956906,"median":0.29000483301933855,"all":[2.0243704589956906,0.28978074999758974,0.29209058298147283,0.2926525000075344,0.29000483301933855,0.29440587499993853,0.2889024169999175,0.2962748330028262,0.2863593329966534,0.2836708330141846,0.28494929199223407]},"ms_per_call_warm_runs":{"n":3972,"median":21.733832996687852,"p10":19.7982288984349,"p90":23.81210390594788},"ms_per_call_all_runs":{"n":4055,"median":21.749290986917913,"p10":19.803041801787913,"p90":23.858750017825514},"ms_per_token_warm_runs":{"n":423,"median":1.4607589461390653,"p10":1.3736052211490588,"p90":1.6213200968035206},"run_decode_seconds_warm_runs":{"n":423,"median":0.1282964580168482,"p10":0.08748413319699466,"p90":0.2615696750115603},"fixture_434_rows":{"calls":4055,"tokens":61556,"padded_tokens":64880,"pad_waste":0.05123304562268804,"projected_ms_at_warm_median_per_call":88130.69280156924}},"seconds":{"gpu_processes":103.68053208300262,"scoring":2.159618125006091,"total":105.98662587499712},"runs":[{"id":"tv4_000","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0016155093908309937,"mean_abs_dp":0.0011539682745933533,"oracle_top2_margin":0.2241945117712021,"near_tie":false,"probs":[0.1486000418663025,0.2049126923084259,0.43501436710357666,0.21147295832633972],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4_001","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.1081180572509766e-05,"mean_abs_dp":3.0451323254965246e-05,"oracle_top2_margin":0.9833351373672485,"near_tie":false,"probs":[0.0064821853302419186,0.002714826026931405,0.9899067282676697,0.0008962587453424931]},{"id":"tv4_002","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007145106792449951,"mean_abs_dp":0.00035726651549339294,"oracle_top2_margin":0.10138940811157227,"near_tie":false,"probs":[0.17872202396392822,0.3581911325454712,0.256827175617218,0.20625969767570496]},{"id":"tv4_003","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014964640140533447,"mean_abs_dp":0.0007861703634262085,"oracle_top2_margin":0.016063004732131958,"near_tie":true,"probs":[0.41221970319747925,0.3933868706226349,0.07267779856920242,0.12171569466590881]},{"id":"tv4_004","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00039699487388134,"mean_abs_dp":0.0001984762493520975,"oracle_top2_margin":0.9105252027511597,"near_tie":false,"probs":[0.00561577919870615,0.029727933928370476,0.939608633518219,0.025047723203897476]},{"id":"tv4_005","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003255307674407959,"mean_abs_dp":0.001627644058316946,"oracle_top2_margin":0.23442357778549194,"near_tie":false,"probs":[0.33058807253837585,0.570817768573761,0.022649472579360008,0.075944684445858]},{"id":"tv4_006","q":0,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0035729706287384033,"mean_abs_dp":0.0017864890396595001,"oracle_top2_margin":0.12790140509605408,"near_tie":false,"probs":[0.20180048048496246,0.2984631657600403,0.07934961467981339,0.42038673162460327]},{"id":"tv4_007","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009180158376693726,"mean_abs_dp":0.0007948577404022217,"oracle_top2_margin":0.2037426233291626,"near_tie":false,"probs":[0.23238809406757355,0.13668636977672577,0.4343090355396271,0.19661648571491241]},{"id":"tv4_008","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.996948599815369e-05,"mean_abs_dp":4.496879409998655e-05,"oracle_top2_margin":0.8541439175605774,"near_tie":false,"probs":[0.04889090359210968,0.9028663635253906,0.005642142612487078,0.042600590735673904]},{"id":"tv4_009","q":0,"variant":"base","tokens":321,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009253025054931641,"mean_abs_dp":0.0004626195877790451,"oracle_top2_margin":0.787467360496521,"near_tie":false,"probs":[0.030835764482617378,0.07527299970388412,0.8640495538711548,0.02984173595905304]},{"id":"tv4_010","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0027551352977752686,"mean_abs_dp":0.0013775471597909927,"oracle_top2_margin":0.11676552891731262,"near_tie":false,"probs":[0.07629967480897903,0.41704368591308594,0.20207057893276215,0.3045860230922699]},{"id":"tv4_011","q":0,"variant":"base","tokens":105,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0007356405258178711,"mean_abs_dp":0.0003678053617477417,"oracle_top2_margin":0.07526007294654846,"near_tie":false,"probs":[0.16435520350933075,0.16926220059394836,0.29502302408218384,0.37135961651802063]},{"id":"tv4_012","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002628326416015625,"mean_abs_dp":0.001314179040491581,"oracle_top2_margin":0.13543769717216492,"near_tie":false,"probs":[0.3253781497478485,0.02738797850906849,0.4581226706504822,0.18911121785640717]},{"id":"tv4_013","q":0,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.000537574291229248,"mean_abs_dp":0.0002783332020044327,"oracle_top2_margin":0.15590348839759827,"near_tie":false,"probs":[0.05182981863617897,0.016573341563344002,0.387319952249527,0.5442768335342407]},{"id":"tv4_014","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0023405253887176514,"mean_abs_dp":0.001217089593410492,"oracle_top2_margin":0.06324279308319092,"near_tie":false,"probs":[0.18399763107299805,0.23813612759113312,0.2560606896877289,0.321805477142334]},{"id":"tv4_015","q":0,"variant":"base","tokens":177,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007926672697067261,"mean_abs_dp":0.0003963448107242584,"oracle_top2_margin":0.26414579153060913,"near_tie":false,"probs":[0.15754275023937225,0.19837458431720734,0.4626891016960144,0.18139353394508362]},{"id":"tv4_016","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.000319063663482666,"mean_abs_dp":0.00017202645540237427,"oracle_top2_margin":0.1596996784210205,"near_tie":false,"probs":[0.1622018665075302,0.24199670553207397,0.193535715341568,0.402265727519989]},{"id":"tv4_017","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011511743068695068,"mean_abs_dp":0.0005755871534347534,"oracle_top2_margin":0.01221814751625061,"near_tie":true,"probs":[0.2636023461818695,0.2393730878829956,0.2525893747806549,0.24443519115447998]},{"id":"tv4_018","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0031005144119262695,"mean_abs_dp":0.001550253015011549,"oracle_top2_margin":0.7636825442314148,"near_tie":false,"probs":[0.013203592039644718,0.0976053774356842,0.867247462272644,0.021943610161542892]},{"id":"tv4_019","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013884752988815308,"mean_abs_dp":0.0010200180113315582,"oracle_top2_margin":0.06632544100284576,"near_tie":false,"probs":[0.23577432334423065,0.30211853981018066,0.23249100148677826,0.22961603105068207]},{"id":"tv4_020","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.006994843482971191,"mean_abs_dp":0.0034974347800016403,"oracle_top2_margin":0.1631980538368225,"near_tie":false,"probs":[0.5313523411750793,0.10132889449596405,0.3546988368034363,0.012619857676327229]},{"id":"tv4_021","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0022206902503967285,"mean_abs_dp":0.0011103581637144089,"oracle_top2_margin":0.4488777816295624,"near_tie":false,"probs":[0.08745673298835754,0.1622854620218277,0.14270521700382233,0.6075525879859924]},{"id":"tv4_022","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010744929313659668,"mean_abs_dp":0.0006603486835956573,"oracle_top2_margin":0.05077701807022095,"near_tie":false,"probs":[0.3588562607765198,0.30691203474998474,0.14137548208236694,0.19285622239112854]},{"id":"tv4_023","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":1,"argmax_equal":false,"max_abs_dp":0.0012638568878173828,"mean_abs_dp":0.0007604341953992844,"oracle_top2_margin":0.0019783377647399902,"near_tie":true,"probs":[0.3657609522342682,0.3653772175312042,0.15247812867164612,0.11638376116752625]},{"id":"tv4_024","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0013806819915771484,"mean_abs_dp":0.0008189789950847626,"oracle_top2_margin":0.015634745359420776,"near_tie":true,"probs":[0.3386755585670471,0.27016597986221313,0.03741179406642914,0.3537467122077942]},{"id":"tv4_025","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011830329895019531,"mean_abs_dp":0.0007677972316741943,"oracle_top2_margin":0.07681050896644592,"near_tie":false,"probs":[0.3025921881198883,0.37709587812423706,0.10320857912302017,0.21710339188575745]},{"id":"tv4_026","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0008314549922943115,"mean_abs_dp":0.0004157423973083496,"oracle_top2_margin":0.06704592704772949,"near_tie":false,"probs":[0.19550547003746033,0.23850177228450775,0.24898695945739746,0.31700581312179565]},{"id":"tv4_027","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012668073177337646,"mean_abs_dp":0.0006334185600280762,"oracle_top2_margin":0.009622842073440552,"near_tie":true,"probs":[0.16842885315418243,0.33257701992988586,0.3437099754810333,0.15528403222560883]},{"id":"tv4_028","q":0,"variant":"base","tokens":290,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005431473255157471,"mean_abs_dp":0.0002715699374675751,"oracle_top2_margin":0.01672416925430298,"near_tie":true,"probs":[0.16780264675617218,0.3211795389652252,0.20735663175582886,0.30366119742393494]},{"id":"tv4_029","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005552470684051514,"mean_abs_dp":0.0003414228558540344,"oracle_top2_margin":0.04676586389541626,"near_tie":false,"probs":[0.3011783957481384,0.348834365606308,0.16534151136875153,0.18464569747447968]},{"id":"tv4_030","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014666914939880371,"mean_abs_dp":0.000733325257897377,"oracle_top2_margin":0.559301495552063,"near_tie":false,"probs":[0.10203228145837784,0.046132415533065796,0.7063024640083313,0.14553287625312805]},{"id":"tv4_031","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0028488337993621826,"mean_abs_dp":0.0014244206249713898,"oracle_top2_margin":0.2643141746520996,"near_tie":false,"probs":[0.22567042708396912,0.48669353127479553,0.18690115213394165,0.10073486715555191]},{"id":"tv4_032","q":0,"variant":"base","tokens":135,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00030472874641418457,"mean_abs_dp":0.000227990560233593,"oracle_top2_margin":0.07778683304786682,"near_tie":false,"probs":[0.05748194828629494,0.19270208477973938,0.4140032231807709,0.3358127474784851]},{"id":"tv4_033","q":0,"variant":"base","tokens":293,"calls":19,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0020120739936828613,"mean_abs_dp":0.001006055623292923,"oracle_top2_margin":0.10813379287719727,"near_tie":false,"probs":[0.0801900178194046,0.09709455817937851,0.3589051067829132,0.4638103246688843]},{"id":"tv4_034","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012533366680145264,"mean_abs_dp":0.0008817827329039574,"oracle_top2_margin":0.1505841612815857,"near_tie":false,"probs":[0.10079970210790634,0.053126342594623566,0.4979574978351593,0.348116397857666]},{"id":"tv4_035","q":0,"variant":"base","tokens":108,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007853209972381592,"mean_abs_dp":0.00039677321910858154,"oracle_top2_margin":0.18563473224639893,"near_tie":false,"probs":[0.18924131989479065,0.21362636983394623,0.398521363735199,0.19861093163490295]},{"id":"tv4_036","q":0,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011899769306182861,"mean_abs_dp":0.0007472038269042969,"oracle_top2_margin":0.07235273718833923,"near_tie":false,"probs":[0.4136674404144287,0.34308624267578125,0.17723120748996735,0.0660151019692421]},{"id":"tv4_037","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0022120773792266846,"mean_abs_dp":0.001170516014099121,"oracle_top2_margin":0.05972352623939514,"near_tie":false,"probs":[0.20974168181419373,0.2624477744102478,0.2074946016073227,0.3203158974647522]},{"id":"tv4_038","q":0,"variant":"base","tokens":130,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012812316417694092,"mean_abs_dp":0.0008316971361637115,"oracle_top2_margin":0.16796669363975525,"near_tie":false,"probs":[0.41705361008644104,0.24935634434223175,0.18259158730506897,0.15099838376045227]},{"id":"tv4_039","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002278655767440796,"mean_abs_dp":0.0011393236927688122,"oracle_top2_margin":0.35326528549194336,"near_tie":false,"probs":[0.2902967631816864,0.6477354764938354,0.038611702620983124,0.023356005549430847]},{"id":"tv4_040","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00047825276851654053,"mean_abs_dp":0.00023913010954856873,"oracle_top2_margin":0.052141427993774414,"near_tie":false,"probs":[0.10292653739452362,0.33860164880752563,0.16774225234985352,0.39072951674461365]},{"id":"tv4_041","q":0,"variant":"base","tokens":208,"calls":13,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00107535719871521,"mean_abs_dp":0.0005406476557254791,"oracle_top2_margin":0.06237095594406128,"near_tie":false,"probs":[0.1421525925397873,0.3598526418209076,0.29641225934028625,0.20158250629901886]},{"id":"tv4_042","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009586811065673828,"mean_abs_dp":0.00047935545444488525,"oracle_top2_margin":0.02623698115348816,"near_tie":false,"probs":[0.3322635293006897,0.09912151843309402,0.3059658408164978,0.2626491189002991]},{"id":"tv4_043","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011763274669647217,"mean_abs_dp":0.000924527645111084,"oracle_top2_margin":0.12820479273796082,"near_tie":false,"probs":[0.38112738728523254,0.2505880296230316,0.22705410420894623,0.14123043417930603]},{"id":"tv4_044","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.001613914966583252,"mean_abs_dp":0.0008069593459367752,"oracle_top2_margin":0.7342252731323242,"near_tie":false,"probs":[0.043021608144044876,0.0792010948061943,0.06228004768490791,0.8154972195625305]},{"id":"tv4_045","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006981581449508667,"mean_abs_dp":0.0034907683730125427,"oracle_top2_margin":0.311641126871109,"near_tie":false,"probs":[0.03966842591762543,0.03708619624376297,0.6241267323493958,0.2991185784339905]},{"id":"tv4_046","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006157830357551575,"mean_abs_dp":0.0003078640438616276,"oracle_top2_margin":0.8037196397781372,"near_tie":false,"probs":[0.8873147368431091,0.019106175750494003,0.08238689601421356,0.011192121542990208]},{"id":"tv4_047","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0030972063541412354,"mean_abs_dp":0.0015835464000701904,"oracle_top2_margin":0.10399001836776733,"near_tie":false,"probs":[0.2991897165775299,0.3974968492984772,0.12654130160808563,0.1767721027135849]},{"id":"tv4_048","q":0,"variant":"base","tokens":119,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000909656286239624,"mean_abs_dp":0.0005538053810596466,"oracle_top2_margin":0.040600210428237915,"near_tie":false,"probs":[0.2115621119737625,0.31109923124313354,0.20792056620121002,0.26941803097724915]},{"id":"tv4_049","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018898248672485352,"mean_abs_dp":0.0009449087083339691,"oracle_top2_margin":0.14964529871940613,"near_tie":false,"probs":[0.21594056487083435,0.4264119863510132,0.0828651636838913,0.2747822105884552]},{"id":"tv4_050","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010973215103149414,"mean_abs_dp":0.0005486421287059784,"oracle_top2_margin":0.2277095913887024,"near_tie":false,"probs":[0.3277333378791809,0.5573616027832031,0.04535195603966713,0.06955314427614212]},{"id":"tv4_051","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0011369287967681885,"mean_abs_dp":0.0005684671923518181,"oracle_top2_margin":0.006280571222305298,"near_tie":true,"probs":[0.05906115472316742,0.18529871106147766,0.3808095455169678,0.37483057379722595]},{"id":"tv4_052","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004993557929992676,"mean_abs_dp":0.0024967873468995094,"oracle_top2_margin":0.5888446569442749,"near_tie":false,"probs":[0.7201768755912781,0.10197113454341888,0.05503232404589653,0.12281963229179382]},{"id":"tv4_053","q":0,"variant":"base","tokens":358,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00292244553565979,"mean_abs_dp":0.001471344381570816,"oracle_top2_margin":0.06682348251342773,"near_tie":false,"probs":[0.3236418664455414,0.2595001757144928,0.19522929191589355,0.22162872552871704]},{"id":"tv4_054","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0006154179573059082,"mean_abs_dp":0.00030770525336265564,"oracle_top2_margin":0.0028791427612304688,"near_tie":true,"probs":[0.13885273039340973,0.27271798253059387,0.29270830750465393,0.29572102427482605]},{"id":"tv4_055","q":0,"variant":"base","tokens":189,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014485418796539307,"mean_abs_dp":0.0008312994614243507,"oracle_top2_margin":0.14189845323562622,"near_tie":false,"probs":[0.2706543505191803,0.412578284740448,0.26893460750579834,0.0478326752781868]},{"id":"tv4_056","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00121268630027771,"mean_abs_dp":0.0006122961640357971,"oracle_top2_margin":0.1477002501487732,"near_tie":false,"probs":[0.2549459636211395,0.21032832562923431,0.4048236310482025,0.12990207970142365]},{"id":"tv4_057","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0009570717811584473,"mean_abs_dp":0.0005218833684921265,"oracle_top2_margin":0.042515188455581665,"near_tie":false,"probs":[0.2672346234321594,0.29960042238235474,0.0901789665222168,0.34298598766326904]},{"id":"tv4_058","q":0,"variant":"base","tokens":129,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001010894775390625,"mean_abs_dp":0.0005225054919719696,"oracle_top2_margin":0.045424774289131165,"near_tie":false,"probs":[0.2303963452577591,0.2893342673778534,0.24219392240047455,0.23807545006275177]},{"id":"tv4_059","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_head","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014499127864837646,"mean_abs_dp":0.0007249657064676285,"oracle_top2_margin":0.11904534697532654,"near_tie":false,"probs":[0.41398411989212036,0.199211984872818,0.29236292839050293,0.09444092959165573]},{"id":"tv4x_emotion_00","q":0,"variant":"base","tokens":42,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014435946941375732,"mean_abs_dp":0.0008980818092823029,"oracle_top2_margin":0.056429117918014526,"near_tie":false,"probs":[0.09529443085193634,0.2872040569782257,0.3465140163898468,0.06238111853599548,0.059342339634895325,0.14926400780677795]},{"id":"tv4x_emotion_01","q":0,"variant":"base","tokens":38,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0011102557182312012,"mean_abs_dp":0.0005154547592004141,"oracle_top2_margin":0.0766339898109436,"near_tie":false,"probs":[0.15971678495407104,0.1073416993021965,0.07515289634466171,0.3121541738510132,0.11205349862575531,0.23358091711997986]},{"id":"tv4x_emotion_02","q":0,"variant":"base","tokens":36,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011586546897888184,"mean_abs_dp":0.0004029935225844383,"oracle_top2_margin":0.6756487488746643,"near_tie":false,"probs":[0.7566699385643005,0.020213020965456963,0.014861873351037502,0.07891049236059189,0.0825003981590271,0.046844277530908585]},{"id":"tv4x_emotion_03","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007624030113220215,"mean_abs_dp":0.0002541414772470792,"oracle_top2_margin":0.4306091070175171,"near_tie":false,"probs":[0.6750511527061462,0.03576159477233887,0.24538980424404144,0.016023043543100357,0.01574527844786644,0.012029158882796764]},{"id":"tv4x_emotion_04","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.497759699821472e-05,"mean_abs_dp":5.536278088887533e-05,"oracle_top2_margin":0.8624813556671143,"near_tie":false,"probs":[0.005436768755316734,0.9106729030609131,0.048185642808675766,0.007295018527656794,0.009641665033996105,0.018767960369586945]},{"id":"tv4x_emotion_05","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.002183198928833008,"mean_abs_dp":0.0008498579263687134,"oracle_top2_margin":0.10235360264778137,"near_tie":false,"probs":[0.1889888048171997,0.2898675501346588,0.04749619960784912,0.3889140188694,0.03251685947179794,0.0522165447473526]},{"id":"tv4x_emotion_06","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0010173618793487549,"mean_abs_dp":0.00043363838146130246,"oracle_top2_margin":0.3278556764125824,"near_tie":false,"probs":[0.0180195402354002,0.24156343936920166,0.5701528787612915,0.03638022392988205,0.03693448007106781,0.09694941341876984]},{"id":"tv4x_emotion_07","q":0,"variant":"base","tokens":37,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001608133316040039,"mean_abs_dp":0.0005360348150134087,"oracle_top2_margin":0.6330275535583496,"near_tie":false,"probs":[0.7215110063552856,0.025226041674613953,0.015498696826398373,0.08665971457958221,0.0850195363163948,0.06608501076698303]},{"id":"tv4x_emotion_08","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024527013301849365,"mean_abs_dp":0.0008175487940510114,"oracle_top2_margin":0.1790621280670166,"near_tie":false,"probs":[0.02282405085861683,0.556826114654541,0.37354180216789246,0.013471893034875393,0.01193863432854414,0.021397419273853302]},{"id":"tv4x_emotion_09","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":5,"argmax_oracle":5,"argmax_equal":true,"max_abs_dp":0.0012561976909637451,"mean_abs_dp":0.00046385452151298523,"oracle_top2_margin":0.02565602958202362,"near_tie":false,"probs":[0.1862509548664093,0.04087664559483528,0.14352428913116455,0.2053321897983551,0.19333158433437347,0.2306843101978302]},{"id":"tv4x_emotion_10","q":0,"variant":"base","tokens":40,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012404322624206543,"mean_abs_dp":0.000504969929655393,"oracle_top2_margin":0.034467101097106934,"near_tie":false,"probs":[0.32179200649261475,0.05063674598932266,0.07949387282133102,0.286180704832077,0.11715059727430344,0.1447460651397705]},{"id":"tv4x_emotion_11","q":0,"variant":"base","tokens":48,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014516711235046387,"mean_abs_dp":0.000513856764882803,"oracle_top2_margin":0.7190135717391968,"near_tie":false,"probs":[0.007938779890537262,0.8269941806793213,0.1106434166431427,0.01111913938075304,0.009819075465202332,0.033485397696495056]},{"id":"tv4x_emotion_12","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00055655837059021,"mean_abs_dp":0.00027459270010391873,"oracle_top2_margin":0.28088271617889404,"near_tie":false,"probs":[0.02430153079330921,0.3084581196308136,0.5896301865577698,0.02755715511739254,0.030036119744181633,0.020016886293888092]},{"id":"tv4x_emotion_13","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001751929521560669,"mean_abs_dp":0.000716112864514192,"oracle_top2_margin":0.2596837878227234,"near_tie":false,"probs":[0.45367431640625,0.09693339467048645,0.11614951491355896,0.0308766420930624,0.11108897626399994,0.19127708673477173]},{"id":"tv4x_emotion_14","q":0,"variant":"base","tokens":39,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005893334746360779,"mean_abs_dp":0.00020252137134472528,"oracle_top2_margin":0.7910157442092896,"near_tie":false,"probs":[0.002865477232262492,0.09352181106805801,0.8856046795845032,0.006592904683202505,0.006227686069905758,0.005187378264963627]},{"id":"tv4x_emotion_15","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001151740550994873,"mean_abs_dp":0.00041791175802548725,"oracle_top2_margin":0.25465235114097595,"near_tie":false,"probs":[0.005529662128537893,0.5982164740562439,0.341275155544281,0.010856241919100285,0.012094934470951557,0.03202749043703079]},{"id":"tv4x_emotion_16","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007630586624145508,"mean_abs_dp":0.00032568226257960003,"oracle_top2_margin":0.8546378016471863,"near_tie":false,"probs":[0.8940438628196716,0.00898166373372078,0.024168558418750763,0.03797433152794838,0.017313385382294655,0.017518188804388046]},{"id":"tv4x_emotion_17","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007981956005096436,"mean_abs_dp":0.00034928228706121445,"oracle_top2_margin":0.2597692310810089,"near_tie":false,"probs":[0.44718706607818604,0.024394800886511803,0.18897147476673126,0.13527342677116394,0.06703270226716995,0.13714052736759186]},{"id":"tv4x_emotion_18","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004338502883911133,"mean_abs_dp":0.0014461949467658997,"oracle_top2_margin":0.06271809339523315,"near_tie":false,"probs":[0.3569484353065491,0.014430890791118145,0.03861790895462036,0.4115469753742218,0.10524303466081619,0.07321282476186752]},{"id":"tv4x_emotion_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007581114768981934,"mean_abs_dp":0.00025268395741780597,"oracle_top2_margin":0.4918869137763977,"near_tie":false,"probs":[0.6271986365318298,0.028979608789086342,0.04814891889691353,0.13430550694465637,0.0959121584892273,0.06545525044202805]},{"id":"tv4x_tweet_offensive_00","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00036257505416870117,"mean_abs_dp":0.00036257505416870117,"oracle_top2_margin":0.4777187407016754,"near_tie":false,"probs":[0.2607780396938324,0.7392219305038452]},{"id":"tv4x_tweet_offensive_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.249462366104126e-05,"mean_abs_dp":1.247599720954895e-05,"oracle_top2_margin":0.7617751359939575,"near_tie":false,"probs":[0.1191248968243599,0.8808751106262207]},{"id":"tv4x_tweet_offensive_02","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009202063083648682,"mean_abs_dp":0.0009201914072036743,"oracle_top2_margin":0.6824735403060913,"near_tie":false,"probs":[0.8403165936470032,0.1596834361553192]},{"id":"tv4x_tweet_offensive_03","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010437369346618652,"mean_abs_dp":0.0010437071323394775,"oracle_top2_margin":0.3328827917575836,"near_tie":false,"probs":[0.6653977036476135,0.33460232615470886]},{"id":"tv4x_tweet_offensive_04","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024309158325195312,"mean_abs_dp":0.0024309009313583374,"oracle_top2_margin":0.2086932361125946,"near_tie":false,"probs":[0.601915717124939,0.39808428287506104]},{"id":"tv4x_tweet_offensive_05","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011359453201293945,"mean_abs_dp":0.0011359453201293945,"oracle_top2_margin":0.38322776556015015,"near_tie":false,"probs":[0.6904779076576233,0.30952203273773193]},{"id":"tv4x_tweet_offensive_06","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00031235814094543457,"mean_abs_dp":0.0003123432397842407,"oracle_top2_margin":0.028999686241149902,"near_tie":false,"probs":[0.5141875147819519,0.4858125150203705]},{"id":"tv4x_tweet_offensive_07","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003998279571533203,"mean_abs_dp":0.0003997981548309326,"oracle_top2_margin":0.020770907402038574,"near_tie":false,"probs":[0.48921477794647217,0.5107852816581726]},{"id":"tv4x_tweet_offensive_08","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012553930282592773,"mean_abs_dp":0.0012553632259368896,"oracle_top2_margin":0.2947831451892853,"near_tie":false,"probs":[0.6461362242698669,0.35386380553245544]},{"id":"tv4x_tweet_offensive_09","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015403032302856445,"mean_abs_dp":0.001540258526802063,"oracle_top2_margin":0.07451754808425903,"near_tie":false,"probs":[0.4612009823322296,0.5387990474700928]},{"id":"tv4x_tweet_offensive_10","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00040283799171447754,"mean_abs_dp":0.0004028230905532837,"oracle_top2_margin":0.1919628381729126,"near_tie":false,"probs":[0.4036157429218292,0.5963842272758484]},{"id":"tv4x_tweet_offensive_11","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008295178413391113,"mean_abs_dp":0.0008294656872749329,"oracle_top2_margin":0.6438125371932983,"near_tie":false,"probs":[0.8210767507553101,0.17892317473888397]},{"id":"tv4x_tweet_offensive_12","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004773139953613281,"mean_abs_dp":0.00047728419303894043,"oracle_top2_margin":0.03468778729438782,"near_tie":false,"probs":[0.48217883706092834,0.517821192741394]},{"id":"tv4x_tweet_offensive_13","q":0,"variant":"base","tokens":43,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.4378299713134766e-05,"mean_abs_dp":2.4378299713134766e-05,"oracle_top2_margin":0.1575031578540802,"near_tie":false,"probs":[0.578775942325592,0.42122402787208557]},{"id":"tv4x_tweet_offensive_14","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007702410221099854,"mean_abs_dp":0.0007702261209487915,"oracle_top2_margin":0.08823934197425842,"near_tie":false,"probs":[0.4551100730895996,0.5448898673057556]},{"id":"tv4x_tweet_offensive_15","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001653224229812622,"mean_abs_dp":0.0016532093286514282,"oracle_top2_margin":0.23108571767807007,"near_tie":false,"probs":[0.6138896346092224,0.3861103355884552]},{"id":"tv4x_tweet_offensive_16","q":0,"variant":"base","tokens":104,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007571130990982056,"mean_abs_dp":0.000757075846195221,"oracle_top2_margin":0.5775153040885925,"near_tie":false,"probs":[0.21199944615364075,0.7880005836486816]},{"id":"tv4x_tweet_offensive_17","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0025031566619873047,"mean_abs_dp":0.002503126859664917,"oracle_top2_margin":0.15798139572143555,"near_tie":false,"probs":[0.42351245880126953,0.5764876008033752]},{"id":"tv4x_tweet_offensive_18","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001613020896911621,"mean_abs_dp":0.001613020896911621,"oracle_top2_margin":0.09931710362434387,"near_tie":false,"probs":[0.44872841238975525,0.5512715578079224]},{"id":"tv4x_tweet_offensive_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007212162017822266,"mean_abs_dp":0.0007212162017822266,"oracle_top2_margin":0.18660569190979004,"near_tie":false,"probs":[0.5925816297531128,0.4074183702468872]},{"id":"tv4x_qnli_00","q":0,"variant":"base","tokens":94,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004703998565673828,"mean_abs_dp":0.0004703737795352936,"oracle_top2_margin":0.7700709700584412,"near_tie":false,"probs":[0.11449417471885681,0.8855059146881104]},{"id":"tv4x_qnli_01","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00042705237865448,"mean_abs_dp":0.0004270300269126892,"oracle_top2_margin":0.5029501914978027,"near_tie":false,"probs":[0.7519021034240723,0.24809788167476654]},{"id":"tv4x_qnli_02","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002105236053466797,"mean_abs_dp":0.00021050125360488892,"oracle_top2_margin":0.6557972431182861,"near_tie":false,"probs":[0.17189091444015503,0.8281091451644897]},{"id":"tv4x_qnli_03","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007594823837280273,"mean_abs_dp":0.0007594674825668335,"oracle_top2_margin":0.6964544057846069,"near_tie":false,"probs":[0.8489866852760315,0.1510133445262909]},{"id":"tv4x_qnli_04","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002998374402523041,"mean_abs_dp":0.00029982440173625946,"oracle_top2_margin":0.9449087381362915,"near_tie":false,"probs":[0.027845457196235657,0.9721545577049255]},{"id":"tv4x_qnli_05","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009099431335926056,"mean_abs_dp":0.0009099040180444717,"oracle_top2_margin":0.8874053359031677,"near_tie":false,"probs":[0.05538741499185562,0.9446125626564026]},{"id":"tv4x_qnli_06","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000911414623260498,"mean_abs_dp":0.0009113848209381104,"oracle_top2_margin":0.8620169758796692,"near_tie":false,"probs":[0.06990287452936172,0.9300971031188965]},{"id":"tv4x_qnli_07","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00028693675994873047,"mean_abs_dp":0.0002869069576263428,"oracle_top2_margin":0.18525761365890503,"near_tie":false,"probs":[0.5929157137870789,0.40708428621292114]},{"id":"tv4x_qnli_08","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001128077507019043,"mean_abs_dp":0.0011280328035354614,"oracle_top2_margin":0.35944250226020813,"near_tie":false,"probs":[0.6808493137359619,0.31915074586868286]},{"id":"tv4x_qnli_09","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.134411811828613e-05,"mean_abs_dp":9.132921695709229e-05,"oracle_top2_margin":0.2295372486114502,"near_tie":false,"probs":[0.6146773099899292,0.3853227198123932]},{"id":"tv4x_qnli_10","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000995039939880371,"mean_abs_dp":0.0009950250387191772,"oracle_top2_margin":0.43258559703826904,"near_tie":false,"probs":[0.7152977585792542,0.28470221161842346]},{"id":"tv4x_qnli_11","q":0,"variant":"base","tokens":50,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018752515316009521,"mean_abs_dp":0.0018752366304397583,"oracle_top2_margin":0.409393310546875,"near_tie":false,"probs":[0.29717859625816345,0.7028214335441589]},{"id":"tv4x_qnli_12","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00033786892890930176,"mean_abs_dp":0.0003378540277481079,"oracle_top2_margin":0.10790503025054932,"near_tie":false,"probs":[0.5536146759986877,0.44638535380363464]},{"id":"tv4x_qnli_13","q":0,"variant":"base","tokens":52,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0028818249702453613,"mean_abs_dp":0.0028818249702453613,"oracle_top2_margin":0.35351717472076416,"near_tie":false,"probs":[0.6796404123306274,0.32035958766937256]},{"id":"tv4x_qnli_14","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003263235092163086,"mean_abs_dp":0.003263235092163086,"oracle_top2_margin":0.28486254811286926,"near_tie":false,"probs":[0.6456944942474365,0.3543054759502411]},{"id":"tv4x_qnli_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007860064506530762,"mean_abs_dp":0.0007859934121370316,"oracle_top2_margin":0.9124271273612976,"near_tie":false,"probs":[0.044572409242391586,0.9554275274276733]},{"id":"tv4x_qnli_16","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002346038818359375,"mean_abs_dp":0.0002345740795135498,"oracle_top2_margin":0.7679163217544556,"near_tie":false,"probs":[0.11580730229616165,0.8841927647590637]},{"id":"tv4x_qnli_17","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006034001708030701,"mean_abs_dp":0.000603359192609787,"oracle_top2_margin":0.7737533450126648,"near_tie":false,"probs":[0.11372670531272888,0.8862733244895935]},{"id":"tv4x_qnli_18","q":0,"variant":"base","tokens":44,"calls":3,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001953303813934326,"mean_abs_dp":0.0019532740116119385,"oracle_top2_margin":0.17956295609474182,"near_tie":false,"probs":[0.41217175126075745,0.5878281593322754]},{"id":"tv4x_qnli_19","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001499772071838379,"mean_abs_dp":0.0014997273683547974,"oracle_top2_margin":0.44058558344841003,"near_tie":false,"probs":[0.7187930345535278,0.2812069058418274]},{"id":"tv4x_paws_00","q":0,"variant":"base","tokens":81,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011883974075317383,"mean_abs_dp":0.0011883974075317383,"oracle_top2_margin":0.38785746693611145,"near_tie":false,"probs":[0.30488285422325134,0.6951171159744263]},{"id":"tv4x_paws_01","q":0,"variant":"base","tokens":113,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002600550651550293,"mean_abs_dp":0.0002600252628326416,"oracle_top2_margin":0.483722984790802,"near_tie":false,"probs":[0.25787848234176636,0.7421215176582336]},{"id":"tv4x_paws_02","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018310546875,"mean_abs_dp":0.00018309056758880615,"oracle_top2_margin":0.21775135397911072,"near_tie":false,"probs":[0.6086925864219666,0.39130741357803345]},{"id":"tv4x_paws_03","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004774034023284912,"mean_abs_dp":0.00047735869884490967,"oracle_top2_margin":0.2834298312664032,"near_tie":false,"probs":[0.3578076958656311,0.6421922445297241]},{"id":"tv4x_paws_04","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000582575798034668,"mean_abs_dp":0.000582575798034668,"oracle_top2_margin":0.210747629404068,"near_tie":false,"probs":[0.39404359459877014,0.6059563755989075]},{"id":"tv4x_paws_05","q":0,"variant":"base","tokens":96,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00018143653869628906,"mean_abs_dp":0.00018142908811569214,"oracle_top2_margin":0.5033949613571167,"near_tie":false,"probs":[0.24812106788158417,0.7518789172172546]},{"id":"tv4x_paws_06","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00029841065406799316,"mean_abs_dp":0.0002983957529067993,"oracle_top2_margin":0.5668308734893799,"near_tie":false,"probs":[0.21688295900821686,0.7831170558929443]},{"id":"tv4x_paws_07","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002790004014968872,"mean_abs_dp":0.0027899891138076782,"oracle_top2_margin":0.4673820734024048,"near_tie":false,"probs":[0.26351895928382874,0.7364810109138489]},{"id":"tv4x_paws_08","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00015422701835632324,"mean_abs_dp":0.0001542121171951294,"oracle_top2_margin":0.3115227520465851,"near_tie":false,"probs":[0.34408438205718994,0.6559155583381653]},{"id":"tv4x_paws_09","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00033399462699890137,"mean_abs_dp":0.0003339797258377075,"oracle_top2_margin":0.2519233822822571,"near_tie":false,"probs":[0.37370434403419495,0.6262956857681274]},{"id":"tv4x_paws_10","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016091465950012207,"mean_abs_dp":0.0016091316938400269,"oracle_top2_margin":0.3951742947101593,"near_tie":false,"probs":[0.3008037209510803,0.6991962790489197]},{"id":"tv4x_paws_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.390711784362793e-05,"mean_abs_dp":9.389221668243408e-05,"oracle_top2_margin":0.16973772644996643,"near_tie":false,"probs":[0.4152250289916992,0.5847749710083008]},{"id":"tv4x_paws_12","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007879137992858887,"mean_abs_dp":0.0007879137992858887,"oracle_top2_margin":0.43726658821105957,"near_tie":false,"probs":[0.2805787920951843,0.7194212079048157]},{"id":"tv4x_paws_13","q":0,"variant":"base","tokens":107,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012268424034118652,"mean_abs_dp":0.0012268126010894775,"oracle_top2_margin":0.33207952976226807,"near_tie":false,"probs":[0.3327334523200989,0.6672666072845459]},{"id":"tv4x_paws_14","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002289891242980957,"mean_abs_dp":0.0022898614406585693,"oracle_top2_margin":0.22963646054267883,"near_tie":false,"probs":[0.38289186358451843,0.6171080470085144]},{"id":"tv4x_paws_15","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019438862800598145,"mean_abs_dp":0.0019438862800598145,"oracle_top2_margin":0.38555964827537537,"near_tie":false,"probs":[0.3052763044834137,0.6947237253189087]},{"id":"tv4x_paws_16","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.8789043426513672e-05,"mean_abs_dp":2.8789043426513672e-05,"oracle_top2_margin":0.42694148421287537,"near_tie":false,"probs":[0.28655806183815,0.7134419679641724]},{"id":"tv4x_paws_17","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002099364995956421,"mean_abs_dp":0.002099350094795227,"oracle_top2_margin":0.20490708947181702,"near_tie":false,"probs":[0.39544713497161865,0.6045529246330261]},{"id":"tv4x_paws_18","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0028151869773864746,"mean_abs_dp":0.002815157175064087,"oracle_top2_margin":0.20717203617095947,"near_tie":false,"probs":[0.6064011454582214,0.3935987949371338]},{"id":"tv4x_paws_19","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015604794025421143,"mean_abs_dp":0.0015604645013809204,"oracle_top2_margin":0.25582870841026306,"near_tie":false,"probs":[0.37052518129348755,0.6294748187065125]},{"id":"tv4x_sciq_00","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00023496150970458984,"mean_abs_dp":0.00011747394455596805,"oracle_top2_margin":0.9667011499404907,"near_tie":false,"probs":[0.0017045121639966965,0.979252278804779,0.012895501218736172,0.006147665437310934]},{"id":"tv4x_sciq_01","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0015627145767211914,"mean_abs_dp":0.0007813684642314911,"oracle_top2_margin":0.11972805857658386,"near_tie":false,"probs":[0.132448211312294,0.3041198253631592,0.1383114606142044,0.42512047290802]},{"id":"tv4x_sciq_02","q":0,"variant":"base","tokens":102,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00021690130233764648,"mean_abs_dp":0.00010845519136637449,"oracle_top2_margin":0.9568026661872864,"near_tie":false,"probs":[0.9706488847732544,0.013510250486433506,0.007113439496606588,0.008727433159947395]},{"id":"tv4x_sciq_03","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":3.7729740142822266e-05,"mean_abs_dp":1.8880469724535942e-05,"oracle_top2_margin":0.9384280443191528,"near_tie":false,"probs":[0.0028128977864980698,0.0031122788786888123,0.027789508923888206,0.966285228729248]},{"id":"tv4x_sciq_04","q":0,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006245672702789307,"mean_abs_dp":0.00031229667365550995,"oracle_top2_margin":0.0839184820652008,"near_tie":false,"probs":[0.05577481538057327,0.39439988136291504,0.4791122078895569,0.07071307301521301]},{"id":"tv4x_sciq_05","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005551576614379883,"mean_abs_dp":0.0002838745713233948,"oracle_top2_margin":0.3225005269050598,"near_tie":false,"probs":[0.5856528282165527,0.2623751163482666,0.1123608648777008,0.039611175656318665]},{"id":"tv4x_sciq_06","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.632110595703125e-05,"mean_abs_dp":4.816908040083945e-05,"oracle_top2_margin":0.9903356432914734,"near_tie":false,"probs":[0.0017238827422261238,0.9933419823646545,0.0031310338526964188,0.0018031151266768575]},{"id":"tv4x_sciq_07","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.4405460357666016e-05,"mean_abs_dp":2.2215652279555798e-05,"oracle_top2_margin":0.9827399849891663,"near_tie":false,"probs":[0.0015541440807282925,0.9890307784080505,0.0030705274548381567,0.0063445065170526505]},{"id":"tv4x_sciq_08","q":0,"variant":"base","tokens":162,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000680387020111084,"mean_abs_dp":0.0003402022994123399,"oracle_top2_margin":0.8441504836082458,"near_tie":false,"probs":[0.914063036441803,0.07121897488832474,0.0031405144836753607,0.011577506549656391]},{"id":"tv4x_sciq_09","q":0,"variant":"base","tokens":144,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0029648542404174805,"mean_abs_dp":0.0014824355021119118,"oracle_top2_margin":0.7690727114677429,"near_tie":false,"probs":[0.058574821799993515,0.04979259520769119,0.06331760436296463,0.828315019607544]},{"id":"tv4x_sciq_10","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.004339814186096191,"mean_abs_dp":0.0021699315402656794,"oracle_top2_margin":0.7186641693115234,"near_tie":false,"probs":[0.12961885333061218,0.015561962500214577,0.014790504239499569,0.840028703212738]},{"id":"tv4x_sciq_11","q":0,"variant":"base","tokens":366,"calls":23,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004893302917480469,"mean_abs_dp":0.0024466808536089957,"oracle_top2_margin":0.7560468316078186,"near_tie":false,"probs":[0.05238313227891922,0.09758970886468887,0.8476456999778748,0.002381484489887953]},{"id":"tv4x_sciq_12","q":0,"variant":"base","tokens":192,"calls":12,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001088559627532959,"mean_abs_dp":0.0005442823749035597,"oracle_top2_margin":0.872226357460022,"near_tie":false,"probs":[0.011177790351212025,0.012587156146764755,0.9233469367027283,0.05288808047771454]},{"id":"tv4x_sciq_13","q":0,"variant":"base","tokens":147,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004371032118797302,"mean_abs_dp":0.00021896976977586746,"oracle_top2_margin":0.8092012405395508,"near_tie":false,"probs":[0.8969968557357788,0.088660828769207,0.004772558342665434,0.00956979114562273]},{"id":"tv4x_sciq_14","q":0,"variant":"base","tokens":127,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0018621087074279785,"mean_abs_dp":0.0009310273453593254,"oracle_top2_margin":0.8467167019844055,"near_tie":false,"probs":[0.03728846460580826,0.009303495287895203,0.051947128027677536,0.9014609456062317]},{"id":"tv4x_sciq_15","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003826245665550232,"mean_abs_dp":0.00020858785137534142,"oracle_top2_margin":0.9471396207809448,"near_tie":false,"probs":[0.0055365483276546,0.9664249420166016,0.009478237479925156,0.018560316413640976]},{"id":"tv4x_sciq_16","q":0,"variant":"base","tokens":335,"calls":21,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0028686225414276123,"mean_abs_dp":0.001434312667697668,"oracle_top2_margin":0.45971256494522095,"near_tie":false,"probs":[0.22546245157718658,0.01995650678873062,0.07502233982086182,0.6795586943626404]},{"id":"tv4x_sciq_17","q":0,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012093037366867065,"mean_abs_dp":0.0006522876210510731,"oracle_top2_margin":0.3270365595817566,"near_tie":false,"probs":[0.5690882802009583,0.24430087208747864,0.1656859815120697,0.02092490904033184]},{"id":"tv4x_sciq_18","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004133760929107666,"mean_abs_dp":0.0020668432116508484,"oracle_top2_margin":0.5720489621162415,"near_tie":false,"probs":[0.700737714767456,0.06673169881105423,0.10995592921972275,0.12257474660873413]},{"id":"tv4x_sciq_19","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000549018383026123,"mean_abs_dp":0.00027452653739601374,"oracle_top2_margin":0.9556483626365662,"near_tie":false,"probs":[0.974388837814331,0.0033390040043741465,0.002570390235632658,0.019701795652508736]},{"id":"tv4x_legacy_holdout_00","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003916621208190918,"mean_abs_dp":0.0003916621208190918,"oracle_top2_margin":0.6975672245025635,"near_tie":false,"probs":[0.15160804986953735,0.8483919501304626]},{"id":"tv4x_legacy_holdout_01","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006021559238433838,"mean_abs_dp":0.0006021112203598022,"oracle_top2_margin":0.03042253851890564,"near_tie":false,"probs":[0.5158133506774902,0.484186589717865]},{"id":"tv4x_legacy_holdout_02","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005991458892822266,"mean_abs_dp":0.0005991458892822266,"oracle_top2_margin":0.6348230838775635,"near_tie":false,"probs":[0.18198931217193604,0.818010687828064]},{"id":"tv4x_legacy_holdout_03","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0039871931076049805,"mean_abs_dp":0.003987163305282593,"oracle_top2_margin":0.04561471939086914,"near_tie":false,"probs":[0.5188202261924744,0.4811798334121704]},{"id":"tv4x_legacy_holdout_04","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013933181762695312,"mean_abs_dp":0.0013932809233665466,"oracle_top2_margin":0.5876218676567078,"near_tie":false,"probs":[0.20479579269886017,0.795204222202301]},{"id":"tv4x_legacy_holdout_05","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018530786037445068,"mean_abs_dp":0.001853063702583313,"oracle_top2_margin":0.0782371461391449,"near_tie":false,"probs":[0.5372655391693115,0.46273452043533325]},{"id":"tv4x_legacy_holdout_06","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019799917936325073,"mean_abs_dp":0.0019799694418907166,"oracle_top2_margin":0.5711351633071899,"near_tie":false,"probs":[0.21641238033771515,0.783587634563446]},{"id":"tv4x_legacy_holdout_07","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00047260522842407227,"mean_abs_dp":0.00047256797552108765,"oracle_top2_margin":0.7396279573440552,"near_tie":false,"probs":[0.8693413734436035,0.1306585669517517]},{"id":"tv4x_legacy_holdout_08","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004119873046875,"mean_abs_dp":0.00041197240352630615,"oracle_top2_margin":0.7547731399536133,"near_tie":false,"probs":[0.12220148742198944,0.8777985572814941]},{"id":"tv4x_legacy_holdout_09","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00035637617111206055,"mean_abs_dp":0.0003563612699508667,"oracle_top2_margin":0.24087288975715637,"near_tie":false,"probs":[0.37920719385147095,0.620792806148529]},{"id":"tv4x_legacy_holdout_10","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022874772548675537,"mean_abs_dp":0.0002287253737449646,"oracle_top2_margin":0.6889723539352417,"near_tie":false,"probs":[0.1557425707578659,0.8442574739456177]},{"id":"tv4x_legacy_holdout_11","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007867813110351562,"mean_abs_dp":0.0007867515087127686,"oracle_top2_margin":0.1864413619041443,"near_tie":false,"probs":[0.5924339294433594,0.4075660705566406]},{"id":"tv4x_legacy_holdout_12","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019432902336120605,"mean_abs_dp":0.0019432827830314636,"oracle_top2_margin":0.7034794092178345,"near_tie":false,"probs":[0.15020355582237244,0.8497964143753052]},{"id":"tv4x_legacy_holdout_13","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00131303071975708,"mean_abs_dp":0.00131303071975708,"oracle_top2_margin":0.19806906580924988,"near_tie":false,"probs":[0.40227851271629333,0.597721517086029]},{"id":"tv4x_legacy_holdout_14","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0022650957107543945,"mean_abs_dp":0.002265065908432007,"oracle_top2_margin":0.5819854140281677,"near_tie":false,"probs":[0.21127237379550934,0.7887276411056519]},{"id":"tv4x_legacy_holdout_15","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002278953790664673,"mean_abs_dp":0.002278938889503479,"oracle_top2_margin":0.6110415458679199,"near_tie":false,"probs":[0.8032418489456177,0.19675815105438232]},{"id":"tv4x_legacy_holdout_16","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.1324882507324219e-06,"mean_abs_dp":1.080334186553955e-06,"oracle_top2_margin":0.6085996031761169,"near_tie":false,"probs":[0.19569915533065796,0.8043009042739868]},{"id":"tv4x_legacy_holdout_17","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004559755325317383,"mean_abs_dp":0.00045596063137054443,"oracle_top2_margin":0.2321089208126068,"near_tie":false,"probs":[0.6155984997749329,0.38440150022506714]},{"id":"tv4x_legacy_holdout_18","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013013482093811035,"mean_abs_dp":0.0013013407588005066,"oracle_top2_margin":0.6196736097335815,"near_tie":false,"probs":[0.19146451354026794,0.8085354566574097]},{"id":"tv4x_legacy_holdout_19","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025588274002075195,"mean_abs_dp":0.002558797597885132,"oracle_top2_margin":0.57481849193573,"near_tie":false,"probs":[0.7848504781723022,0.21514955163002014]},{"id":"tv4x_composition_holdout_00","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004768490791320801,"mean_abs_dp":0.004768475890159607,"oracle_top2_margin":0.5304081439971924,"near_tie":false,"probs":[0.769972562789917,0.230027437210083]},{"id":"tv4x_composition_holdout_01","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000919729471206665,"mean_abs_dp":0.0009197145700454712,"oracle_top2_margin":0.785683274269104,"near_tie":false,"probs":[0.8937613368034363,0.10623863339424133]},{"id":"tv4x_composition_holdout_02","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004768490791320801,"mean_abs_dp":0.004768475890159607,"oracle_top2_margin":0.5304081439971924,"near_tie":false,"probs":[0.769972562789917,0.230027437210083]},{"id":"tv4x_composition_holdout_03","q":0,"variant":"base","tokens":145,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004546821117401123,"mean_abs_dp":0.004546791315078735,"oracle_top2_margin":0.45854881405830383,"near_tie":false,"probs":[0.7338212132453918,0.26617881655693054]},{"id":"tv4x_composition_holdout_04","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009115338325500488,"mean_abs_dp":0.0009114742279052734,"oracle_top2_margin":0.3756886124610901,"near_tie":false,"probs":[0.3112441897392273,0.6887557506561279]},{"id":"tv4x_composition_holdout_05","q":0,"variant":"base","tokens":137,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012612342834472656,"mean_abs_dp":0.00012612342834472656,"oracle_top2_margin":0.7627792358398438,"near_tie":false,"probs":[0.8812634944915771,0.11873647570610046]},{"id":"tv4x_composition_holdout_06","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009115338325500488,"mean_abs_dp":0.0009114742279052734,"oracle_top2_margin":0.3756886124610901,"near_tie":false,"probs":[0.3112441897392273,0.6887557506561279]},{"id":"tv4x_composition_holdout_07","q":0,"variant":"base","tokens":136,"calls":9,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002601444721221924,"mean_abs_dp":0.00026012957096099854,"oracle_top2_margin":0.3700430393218994,"near_tie":false,"probs":[0.3152386248111725,0.6847614049911499]},{"id":"tv4x_composition_holdout_08","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006085336208343506,"mean_abs_dp":0.000608488917350769,"oracle_top2_margin":0.7355459928512573,"near_tie":false,"probs":[0.8671645522117615,0.1328355073928833]},{"id":"tv4x_composition_holdout_09","q":0,"variant":"base","tokens":167,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012626759707927704,"mean_abs_dp":0.001262640580534935,"oracle_top2_margin":0.8914244174957275,"near_tie":false,"probs":[0.9444496035575867,0.05555044487118721]},{"id":"tv4x_composition_holdout_10","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006085336208343506,"mean_abs_dp":0.000608488917350769,"oracle_top2_margin":0.7355459928512573,"near_tie":false,"probs":[0.8671645522117615,0.1328355073928833]},{"id":"tv4x_composition_holdout_11","q":0,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006043463945388794,"mean_abs_dp":0.0006043091416358948,"oracle_top2_margin":0.7365687489509583,"near_tie":false,"probs":[0.8676800727844238,0.13231994211673737]},{"id":"tv4x_composition_holdout_12","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00013691186904907227,"mean_abs_dp":0.00013688206672668457,"oracle_top2_margin":0.28672489523887634,"near_tie":false,"probs":[0.35650065541267395,0.6434993147850037]},{"id":"tv4x_composition_holdout_13","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001271963119506836,"mean_abs_dp":0.0012719221413135529,"oracle_top2_margin":0.8679777383804321,"near_tie":false,"probs":[0.067283034324646,0.9327169060707092]},{"id":"tv4x_composition_holdout_14","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00013691186904907227,"mean_abs_dp":0.00013688206672668457,"oracle_top2_margin":0.28672489523887634,"near_tie":false,"probs":[0.35650065541267395,0.6434993147850037]},{"id":"tv4x_composition_holdout_15","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008018314838409424,"mean_abs_dp":0.0008017867803573608,"oracle_top2_margin":0.1411517858505249,"near_tie":false,"probs":[0.4286222755908966,0.5713776350021362]},{"id":"tv4x_composition_holdout_16","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004698038101196289,"mean_abs_dp":0.000469796359539032,"oracle_top2_margin":0.7229863405227661,"near_tie":false,"probs":[0.13897660374641418,0.8610233664512634]},{"id":"tv4x_composition_holdout_17","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010326504707336426,"mean_abs_dp":0.0010326206684112549,"oracle_top2_margin":0.571626603603363,"near_tie":false,"probs":[0.21521931886672974,0.7847806811332703]},{"id":"tv4x_composition_holdout_18","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004698038101196289,"mean_abs_dp":0.000469796359539032,"oracle_top2_margin":0.7229863405227661,"near_tie":false,"probs":[0.13897660374641418,0.8610233664512634]},{"id":"tv4x_composition_holdout_19","q":0,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"transfer_v4_dev_sources","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007384270429611206,"mean_abs_dp":0.0007384046912193298,"oracle_top2_margin":0.719793438911438,"near_tie":false,"probs":[0.14084169268608093,0.8591583371162415]},{"id":"tv4s_00","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001684233546257019,"mean_abs_dp":0.0011228422323862712,"oracle_top2_margin":0.3157815933227539,"near_tie":false,"probs":[0.23609736561775208,0.5505907535552979,0.21331186592578888]},{"id":"tv4s_01","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011622309684753418,"mean_abs_dp":0.0007748007774353027,"oracle_top2_margin":0.4067290723323822,"near_tie":false,"probs":[0.25936606526374817,0.6677227020263672,0.07291122525930405]},{"id":"tv4s_02","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013920366764068604,"mean_abs_dp":0.0009280343850453695,"oracle_top2_margin":0.11895173788070679,"near_tie":false,"probs":[0.3146020174026489,0.4353678226470947,0.25003018975257874]},{"id":"tv4s_03","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012005865573883057,"mean_abs_dp":0.0008004059394200643,"oracle_top2_margin":0.09898605942726135,"near_tie":false,"probs":[0.32794421911239624,0.4290006458759308,0.24305514991283417]},{"id":"tv4s_04","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013869255781173706,"mean_abs_dp":0.0009246468544006348,"oracle_top2_margin":0.41046595573425293,"near_tie":false,"probs":[0.1983300745487213,0.6078006625175476,0.1938692331314087]},{"id":"tv4s_05","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000592350959777832,"mean_abs_dp":0.0003948758045832316,"oracle_top2_margin":0.5455268621444702,"near_tie":false,"probs":[0.19520440697669983,0.7400100231170654,0.06478556245565414]},{"id":"tv4s_06","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007935464382171631,"mean_abs_dp":0.000529030958811442,"oracle_top2_margin":0.024926811456680298,"near_tie":false,"probs":[0.2583564817905426,0.38330134749412537,0.3583422005176544]},{"id":"tv4s_07","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015017688274383545,"mean_abs_dp":0.0010011841853459675,"oracle_top2_margin":0.07871943712234497,"near_tie":false,"probs":[0.336579293012619,0.41721466183662415,0.24620604515075684]},{"id":"tv4s_08","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014452636241912842,"mean_abs_dp":0.0009634892145792643,"oracle_top2_margin":0.11677592992782593,"near_tie":false,"probs":[0.3282661736011505,0.44659408926963806,0.22513972222805023]},{"id":"tv4s_09","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008346140384674072,"mean_abs_dp":0.000556419293085734,"oracle_top2_margin":0.05665457248687744,"near_tie":false,"probs":[0.3428988754749298,0.40112462639808655,0.25597646832466125]},{"id":"tv4s_10","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007771849632263184,"mean_abs_dp":0.000518113374710083,"oracle_top2_margin":0.10606691241264343,"near_tie":false,"probs":[0.29837000370025635,0.4057398736476898,0.29589009284973145]},{"id":"tv4s_11","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012253522872924805,"mean_abs_dp":0.0008169114589691162,"oracle_top2_margin":0.0634344220161438,"near_tie":false,"probs":[0.3336968421936035,0.3991801142692566,0.2671230435371399]},{"id":"tv4s_12","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008018016815185547,"mean_abs_dp":0.000534507135550181,"oracle_top2_margin":0.4871361255645752,"near_tie":false,"probs":[0.21249212324619293,0.6987892389297485,0.08871854841709137]},{"id":"tv4s_13","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007930994033813477,"mean_abs_dp":0.0005287428696950277,"oracle_top2_margin":0.5980459451675415,"near_tie":false,"probs":[0.16906824707984924,0.7661647796630859,0.06476695090532303]},{"id":"tv4s_14","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002020508050918579,"mean_abs_dp":0.0013470004002253215,"oracle_top2_margin":0.01574954390525818,"near_tie":true,"probs":[0.39459463953971863,0.413241446018219,0.19216391444206238]},{"id":"tv4s_15","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017309784889221191,"mean_abs_dp":0.0011539657910664876,"oracle_top2_margin":0.07872390747070312,"near_tie":false,"probs":[0.3323955833911896,0.41359034180641174,0.2540140748023987]},{"id":"tv4s_16","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017922818660736084,"mean_abs_dp":0.0011948347091674805,"oracle_top2_margin":0.18242567777633667,"near_tie":false,"probs":[0.32116708159446716,0.5063302516937256,0.17250271141529083]},{"id":"tv4s_17","q":0,"variant":"base","tokens":100,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004589855670928955,"mean_abs_dp":0.00030601024627685547,"oracle_top2_margin":0.07194459438323975,"near_tie":false,"probs":[0.251034677028656,0.4106757640838623,0.3382895290851593]},{"id":"tv4s_18","q":0,"variant":"base","tokens":98,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011906027793884277,"mean_abs_dp":0.0007937575380007426,"oracle_top2_margin":0.3651939928531647,"near_tie":false,"probs":[0.27573785185813904,0.6395842432975769,0.08467792719602585]},{"id":"tv4s_19","q":0,"variant":"base","tokens":99,"calls":7,"finite":true,"all_zero":false,"source":"transfer_v4_dev_score","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005691647529602051,"mean_abs_dp":0.0003794456521670024,"oracle_top2_margin":0.29585587978363037,"near_tie":false,"probs":[0.3186206817626953,0.6152265071868896,0.06615279614925385]},{"id":"semif_a3f18f3a63d45345942b","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008739829063415527,"mean_abs_dp":0.0005826534082492193,"oracle_top2_margin":0.6473773717880249,"near_tie":false,"probs":[0.021132148802280426,0.16643394529819489,0.8124338984489441],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"semif_f40beba9088c8db8bbd6","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000749051570892334,"mean_abs_dp":0.0004993534336487452,"oracle_top2_margin":0.8147870898246765,"near_tie":false,"probs":[0.8975083827972412,0.08131039142608643,0.021181181073188782]},{"id":"semif_0b43ea8e24e74d621f1b","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00014358386397361755,"mean_abs_dp":9.576665858427684e-05,"oracle_top2_margin":0.9045717716217041,"near_tie":false,"probs":[0.03758374974131584,0.9418790936470032,0.020537111908197403]},{"id":"semif_942ec65dac0bfb9874f9","q":0,"variant":"base","tokens":60,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.004744023084640503,"mean_abs_dp":0.0031626696387926736,"oracle_top2_margin":0.03487250208854675,"near_tie":false,"probs":[0.49051374197006226,0.06306298077106476,0.4464232921600342]},{"id":"semif_e0c140e9222d1bce5647","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007934123277664185,"mean_abs_dp":0.0005289328595002493,"oracle_top2_margin":0.7138881683349609,"near_tie":false,"probs":[0.84897780418396,0.01750408113002777,0.13351811468601227]},{"id":"semif_8e8c3804a3c15ebb31e3","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001273319125175476,"mean_abs_dp":0.0008488794167836508,"oracle_top2_margin":0.6479207277297974,"near_tie":false,"probs":[0.14230431616306305,0.789499044418335,0.06819666177034378]},{"id":"semif_56c419b12cd37cbe790c","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000350266695022583,"mean_abs_dp":0.00023350616296132407,"oracle_top2_margin":0.7378895282745361,"near_tie":false,"probs":[0.8304153680801392,0.07748612761497498,0.0920984074473381]},{"id":"semif_75b470586834df8c568d","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009698569774627686,"mean_abs_dp":0.000646587461233139,"oracle_top2_margin":0.028810232877731323,"near_tie":false,"probs":[0.04135113209486008,0.46588119864463806,0.4927676320075989]},{"id":"semif_834f3b3e10cc61b33d25","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016164332628250122,"mean_abs_dp":0.0010777053733666737,"oracle_top2_margin":0.6174204349517822,"near_tie":false,"probs":[0.7950324416160583,0.03032044880092144,0.1746472418308258]},{"id":"semif_76c7a79b1ff2363a60c5","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004431009292602539,"mean_abs_dp":0.00029537205894788104,"oracle_top2_margin":0.9108183979988098,"near_tie":false,"probs":[0.9453788995742798,0.03540675342082977,0.019214272499084473]},{"id":"semif_36c71bb649b7d5eccb1f","q":0,"variant":"base","tokens":57,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006605871021747589,"mean_abs_dp":0.0004403914014498393,"oracle_top2_margin":0.8975834846496582,"near_tie":false,"probs":[0.9328095316886902,0.03291734308004379,0.0342731848359108]},{"id":"semif_eac5121a4a1bf391a08c","q":0,"variant":"base","tokens":56,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00023952126502990723,"mean_abs_dp":0.00015969077746073404,"oracle_top2_margin":0.6141079664230347,"near_tie":false,"probs":[0.15981096029281616,0.7743120789527893,0.0658770278096199]},{"id":"semif_44574c44e243ee1b2f52","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005004554986953735,"mean_abs_dp":0.000333623339732488,"oracle_top2_margin":0.7463966608047485,"near_tie":false,"probs":[0.10789750516414642,0.03878762200474739,0.8533149361610413]},{"id":"semif_2608370d01783735c3ea","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013041198253631592,"mean_abs_dp":0.0008693933486938477,"oracle_top2_margin":0.05861121416091919,"near_tie":false,"probs":[0.30553826689720154,0.3777619004249573,0.31669992208480835]},{"id":"semif_ec94ef909f7918695ab3","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006614327430725098,"mean_abs_dp":0.0004409762720266978,"oracle_top2_margin":0.7390934228897095,"near_tie":false,"probs":[0.02500046044588089,0.1185164749622345,0.8564831018447876]},{"id":"semif_03d919fd0e9b3a7cd1e6","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012288093566894531,"mean_abs_dp":0.0008192211389541626,"oracle_top2_margin":0.5888532400131226,"near_tie":false,"probs":[0.7305808663368225,0.12629114091396332,0.14312796294689178]},{"id":"semif_86a15618ba4cb90a307e","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00021463632583618164,"mean_abs_dp":0.00014311820268630981,"oracle_top2_margin":0.8870839476585388,"near_tie":false,"probs":[0.9343640208244324,0.04755954444408417,0.018076444044709206]},{"id":"semif_3dd9c3ac98f8ddf7bdd3","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002844512462615967,"mean_abs_dp":0.0018963366746902466,"oracle_top2_margin":0.6225321292877197,"near_tie":false,"probs":[0.7641696333885193,0.0887136235833168,0.14711681008338928]},{"id":"semif_36273dd4777876a26338","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011658668518066406,"mean_abs_dp":7.774898161490758e-05,"oracle_top2_margin":0.9606409072875977,"near_tie":false,"probs":[0.004951061215251684,0.9777314066886902,0.017317550256848335]},{"id":"semif_07ee70cfe4b01330ce89","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0027700066566467285,"mean_abs_dp":0.0018466686209042866,"oracle_top2_margin":0.12762874364852905,"near_tie":false,"probs":[0.07928327471017838,0.3941380977630615,0.5265786051750183]},{"id":"semif_51ceab3ce622c7c759f1","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":4.924461245536804e-05,"mean_abs_dp":3.286171704530716e-05,"oracle_top2_margin":0.9027648568153381,"near_tie":false,"probs":[0.00938548892736435,0.04388992115855217,0.9467246532440186]},{"id":"semif_b8091be929b1e489fcc4","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00030118227005004883,"mean_abs_dp":0.00020079904546340308,"oracle_top2_margin":0.8441397547721863,"near_tie":false,"probs":[0.013155531138181686,0.9152756333351135,0.07156884670257568]},{"id":"semif_c89b86e3f55123bfa506","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001275930553674698,"mean_abs_dp":8.506389955679576e-05,"oracle_top2_margin":0.9237306118011475,"near_tie":false,"probs":[0.011263484135270119,0.9561738967895508,0.032562658190727234]},{"id":"semif_3b0958cabc339005cf66","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013087987899780273,"mean_abs_dp":0.0008725672960281372,"oracle_top2_margin":0.358396053314209,"near_tie":false,"probs":[0.5916283130645752,0.1764867901802063,0.23188485205173492]},{"id":"semif_f2b4ec4930322fe45118","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0029619932174682617,"mean_abs_dp":0.001974675804376602,"oracle_top2_margin":0.6658164858818054,"near_tie":false,"probs":[0.14284829795360565,0.8033265471458435,0.05382520705461502]},{"id":"semif_61ac20b07a5189231e15","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017731189727783203,"mean_abs_dp":0.0011820898701747258,"oracle_top2_margin":0.5127555131912231,"near_tie":false,"probs":[0.02421928197145462,0.7427014708518982,0.23307925462722778]},{"id":"semif_4b3d794c004566610181","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013071298599243164,"mean_abs_dp":0.0008714292198419571,"oracle_top2_margin":0.7644643187522888,"near_tie":false,"probs":[0.012029202654957771,0.112913578748703,0.8750572204589844]},{"id":"semif_b7c4e62d07986286fce0","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0026782751083374023,"mean_abs_dp":0.001785541574160258,"oracle_top2_margin":0.18547630310058594,"near_tie":false,"probs":[0.5296333432197571,0.13127322494983673,0.3390933871269226]},{"id":"semif_9473603b44af63beab79","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0017295479774475098,"mean_abs_dp":0.001153022050857544,"oracle_top2_margin":0.4208753705024719,"near_tie":false,"probs":[0.676734983921051,0.25266870856285095,0.07059632986783981]},{"id":"semif_d41072dc35d8cd14c056","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003768622875213623,"mean_abs_dp":0.0025124053160349527,"oracle_top2_margin":0.3230390250682831,"near_tie":false,"probs":[0.13881006836891174,0.5888351798057556,0.27235469222068787]},{"id":"semif_33d6e5da58f0fee2490d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004960179328918457,"mean_abs_dp":0.0033067713181177774,"oracle_top2_margin":0.03421613574028015,"near_tie":false,"probs":[0.3546196222305298,0.3960658311843872,0.2493145763874054]},{"id":"semif_823bf736b92e9d8fa66d","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002740025520324707,"mean_abs_dp":0.0001827081044514974,"oracle_top2_margin":0.33118629455566406,"near_tie":false,"probs":[0.20767293870449066,0.5619255304336548,0.23040150105953217]},{"id":"semif_8a4c3de28be95c105ca1","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006723523139953613,"mean_abs_dp":0.0044823214411735535,"oracle_top2_margin":0.4625657796859741,"near_tie":false,"probs":[0.10486884415149689,0.22223037481307983,0.6729007363319397]},{"id":"semif_ea7ec66a5e8eac886548","q":0,"variant":"base","tokens":75,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0022856593132019043,"mean_abs_dp":0.0015237579743067424,"oracle_top2_margin":0.21984949707984924,"near_tie":false,"probs":[0.5394930839538574,0.3240213990211487,0.13648547232151031]},{"id":"semif_25172cd6ddc49ab0ca70","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.003188490867614746,"mean_abs_dp":0.002125687897205353,"oracle_top2_margin":0.29460233449935913,"near_tie":false,"probs":[0.08737631887197495,0.311629056930542,0.6009947061538696]},{"id":"semif_fb73a85d386cf59b0c51","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024036169052124023,"mean_abs_dp":0.0016024286548296611,"oracle_top2_margin":0.20669782161712646,"near_tie":false,"probs":[0.5513379573822021,0.10835123807191849,0.3403107821941376]},{"id":"semif_a02fd582d2881cc462b2","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002878338098526001,"mean_abs_dp":0.001918882131576538,"oracle_top2_margin":0.17390939593315125,"near_tie":false,"probs":[0.12021354585886002,0.524070680141449,0.355715811252594]},{"id":"semif_19f37d5a97034bd7d04f","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016061067581176758,"mean_abs_dp":0.001070740322271983,"oracle_top2_margin":0.8648225665092468,"near_tie":false,"probs":[0.050234850496053696,0.9121983647346497,0.037566814571619034]},{"id":"semif_fda0ca94652c6d739ecf","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000990360975265503,"mean_abs_dp":0.0006602282325426737,"oracle_top2_margin":0.8368752598762512,"near_tie":false,"probs":[0.9016873836517334,0.031589604914188385,0.06672302633523941]},{"id":"semif_d0e36c4cb10a29b3191c","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005268752574920654,"mean_abs_dp":0.00035127003987630207,"oracle_top2_margin":0.2865877151489258,"near_tie":false,"probs":[0.5737104415893555,0.1398821622133255,0.28640738129615784]},{"id":"semif_a0a39ace7e5337d038a7","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006613731384277344,"mean_abs_dp":0.00044090549151102704,"oracle_top2_margin":0.8597530126571655,"near_tie":false,"probs":[0.9120715856552124,0.03433377295732498,0.0535946823656559]},{"id":"semif_eeae283841ce75dd25bf","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004655063152313232,"mean_abs_dp":0.00310336301724116,"oracle_top2_margin":0.4696289002895355,"near_tie":false,"probs":[0.21452850103378296,0.6763477325439453,0.10912375897169113]},{"id":"semif_00bb4cc043f7d2b288d4","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0010671019554138184,"mean_abs_dp":0.000711426759759585,"oracle_top2_margin":0.7398609519004822,"near_tie":false,"probs":[0.11607329547405243,0.029980994760990143,0.8539457321166992]},{"id":"semif_1a81ead69e93e5d0c86a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00204429030418396,"mean_abs_dp":0.0013628651698430379,"oracle_top2_margin":0.24788936972618103,"near_tie":false,"probs":[0.14839467406272888,0.29987001419067383,0.5517353415489197]},{"id":"semif_4444dad25e438f128673","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0011555254459381104,"mean_abs_dp":0.000770340363184611,"oracle_top2_margin":0.15798577666282654,"near_tie":false,"probs":[0.2830660045146942,0.2739805579185486,0.4429534375667572]},{"id":"semif_3226202ba1ada2570e9f","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0032674968242645264,"mean_abs_dp":0.0021783113479614258,"oracle_top2_margin":0.058840811252593994,"near_tie":false,"probs":[0.2218131572008133,0.3568495213985443,0.4213373363018036]},{"id":"semif_3f783eec5c90812479bc","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00041171908378601074,"mean_abs_dp":0.0002745091915130615,"oracle_top2_margin":0.1389794647693634,"near_tie":false,"probs":[0.21951602399349213,0.32044053077697754,0.4600434899330139]},{"id":"semif_ffee861474a9665a093a","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012098252773284912,"mean_abs_dp":0.0008065352837244669,"oracle_top2_margin":0.36492863297462463,"near_tie":false,"probs":[0.22490404546260834,0.18616826832294464,0.588927686214447]},{"id":"semif_8c992e17e4fb54b044f5","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018062591552734375,"mean_abs_dp":0.0012041727701822917,"oracle_top2_margin":0.24903073906898499,"near_tie":false,"probs":[0.21228384971618652,0.5173481702804565,0.2703680098056793]},{"id":"semif_695dd61195ca025c4f4c","q":0,"variant":"base","tokens":62,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008338093757629395,"mean_abs_dp":0.0005558754007021586,"oracle_top2_margin":0.6265780925750732,"near_tie":false,"probs":[0.7710930705070496,0.1461125612258911,0.08279436081647873]},{"id":"semif_e6155431ab7b0f8170d4","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003970488905906677,"mean_abs_dp":0.0002646918098131816,"oracle_top2_margin":0.5309679508209229,"near_tie":false,"probs":[0.19070476293563843,0.7220022678375244,0.08729296922683716]},{"id":"semif_b49ecda75086d8c68d2a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004101097583770752,"mean_abs_dp":0.0002734263737996419,"oracle_top2_margin":0.16984423995018005,"near_tie":false,"probs":[0.257597416639328,0.28625035285949707,0.45615220069885254]},{"id":"semif_587ced261c6092cd1325","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004987657070159912,"mean_abs_dp":0.003325134515762329,"oracle_top2_margin":0.29910629987716675,"near_tie":false,"probs":[0.21066825091838837,0.5403494238853455,0.24898236989974976]},{"id":"semif_533d4423d311de82b27d","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006881654262542725,"mean_abs_dp":0.00045875708262125653,"oracle_top2_margin":0.20385906100273132,"near_tie":false,"probs":[0.13253384828567505,0.5355870723724365,0.3318791389465332]},{"id":"semif_2bf540d515a51755b25c","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0019118785858154297,"mean_abs_dp":0.0012745832403500874,"oracle_top2_margin":0.7265769839286804,"near_tie":false,"probs":[0.0990930125117302,0.07811728864908218,0.82278972864151]},{"id":"semif_fdb9fc214aea2710763a","q":0,"variant":"base","tokens":51,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006535351276397705,"mean_abs_dp":0.00043569008509318036,"oracle_top2_margin":0.1179611086845398,"near_tie":false,"probs":[0.1653045415878296,0.35823678970336914,0.4764586389064789]},{"id":"semif_f6d68126b43f8c9ed183","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0026540756225585938,"mean_abs_dp":0.0017694085836410522,"oracle_top2_margin":0.799848735332489,"near_tie":false,"probs":[0.06920363754034042,0.8649465441703796,0.06584989279508591]},{"id":"semif_44bae0c8a1615a096c43","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0021639466285705566,"mean_abs_dp":0.0014426509539286296,"oracle_top2_margin":0.3203999996185303,"near_tie":false,"probs":[0.5694207549095154,0.2449731081724167,0.18560609221458435]},{"id":"semif_2175379f4c69e6207626","q":0,"variant":"base","tokens":58,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002835020422935486,"mean_abs_dp":0.001890068252881368,"oracle_top2_margin":0.01896810531616211,"near_tie":true,"probs":[0.4181659519672394,0.39976751804351807,0.18206657469272614]},{"id":"semif_def9fe59d262357a673a","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00445246696472168,"mean_abs_dp":0.0029682591557502747,"oracle_top2_margin":0.09732639789581299,"near_tie":false,"probs":[0.4147816300392151,0.5039626359939575,0.08125566691160202]},{"id":"semif_712f6ffb763eef7c9a12","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0056025683879852295,"mean_abs_dp":0.003735025723775228,"oracle_top2_margin":0.13914558291435242,"near_tie":false,"probs":[0.2741396725177765,0.43561822175979614,0.29024210572242737]},{"id":"semif_e1d610bd14e3d16c09b9","q":0,"variant":"base","tokens":64,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014511942863464355,"mean_abs_dp":0.0009674703081448873,"oracle_top2_margin":0.6573848724365234,"near_tie":false,"probs":[0.798063337802887,0.1389365792274475,0.06300012022256851]},{"id":"semif_22ec57822db50ec121cf","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00034399330615997314,"mean_abs_dp":0.00022932390371958414,"oracle_top2_margin":0.04997369647026062,"near_tie":false,"probs":[0.24691741168498993,0.351386696100235,0.4016958475112915]},{"id":"semif_3e75f2b84e8e08e3d3f2","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007080435752868652,"mean_abs_dp":0.00047202905019124347,"oracle_top2_margin":0.2756141722202301,"near_tie":false,"probs":[0.16805289685726166,0.27845194935798645,0.5534951686859131]},{"id":"semif_ebd74aa05324a1556288","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008635148406028748,"mean_abs_dp":0.0005756641427675883,"oracle_top2_margin":0.749722421169281,"near_tie":false,"probs":[0.06939462572336197,0.8404587507247925,0.09014656394720078]},{"id":"semif_d7e84e4245a29817da1f","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00632476806640625,"mean_abs_dp":0.004216539363066356,"oracle_top2_margin":0.43940702080726624,"near_tie":false,"probs":[0.1009969711303711,0.23531700670719147,0.6636860370635986]},{"id":"semif_f41ae66b3c09630c7a42","q":0,"variant":"base","tokens":61,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013136863708496094,"mean_abs_dp":0.0008757909138997396,"oracle_top2_margin":0.4598276913166046,"near_tie":false,"probs":[0.18116424977779388,0.17988529801368713,0.6389504671096802]},{"id":"semif_82c2cd97c160ad27951b","q":0,"variant":"base","tokens":54,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019789934158325195,"mean_abs_dp":0.0013193090756734212,"oracle_top2_margin":0.14875605702400208,"near_tie":false,"probs":[0.5001460909843445,0.15233688056468964,0.34751707315444946]},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00721016526222229,"mean_abs_dp":0.004806766907374064,"oracle_top2_margin":0.2047100067138672,"near_tie":false,"probs":[0.48140138387680054,0.28950580954551697,0.2290927767753601]},{"id":"semif_4d7a003342db5b0a17e3","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015421956777572632,"mean_abs_dp":0.0010281304518381755,"oracle_top2_margin":0.7220965027809143,"near_tie":false,"probs":[0.10872942954301834,0.8281897306442261,0.06308077275753021]},{"id":"semif_19d3d8d0bf773e88be7b","q":0,"variant":"base","tokens":55,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002379119396209717,"mean_abs_dp":0.0015861236800750096,"oracle_top2_margin":0.844078779220581,"near_tie":false,"probs":[0.9040964245796204,0.03192972019314766,0.06397391110658646]},{"id":"semif_d6de731cca52359b1c87","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001369178295135498,"mean_abs_dp":0.0009127755959828695,"oracle_top2_margin":0.088623046875,"near_tie":false,"probs":[0.44230756163597107,0.3522433340549469,0.20544910430908203]},{"id":"semif_b45c63fb92729af27f05","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004521980881690979,"mean_abs_dp":0.00030147284269332886,"oracle_top2_margin":0.617178201675415,"near_tie":false,"probs":[0.7579770088195801,0.101521797478199,0.1405012309551239]},{"id":"semif_46b7029b9a704138b77a","q":0,"variant":"base","tokens":78,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00888979434967041,"mean_abs_dp":0.0059265196323394775,"oracle_top2_margin":0.10240557789802551,"near_tie":false,"probs":[0.12858839333057404,0.4952397048473358,0.37617188692092896]},{"id":"semif_caf773b23934fa7362a1","q":0,"variant":"base","tokens":73,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006782412528991699,"mean_abs_dp":0.0004521925002336502,"oracle_top2_margin":0.7860848903656006,"near_tie":false,"probs":[0.8788682818412781,0.029687995091080666,0.09144368022680283]},{"id":"semif_f46f392ef9e9e9df564b","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005304813385009766,"mean_abs_dp":0.00035365670919418335,"oracle_top2_margin":0.502530574798584,"near_tie":false,"probs":[0.01769918203353882,0.24035675823688507,0.7419440746307373]},{"id":"semif_005dcf6d2c0f21d64796","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011554956436157227,"mean_abs_dp":0.0007703341543674469,"oracle_top2_margin":0.6591771841049194,"near_tie":false,"probs":[0.047102853655815125,0.805100679397583,0.14779648184776306]},{"id":"semif_edb708bd0fe5fd67d6ee","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00036197900772094727,"mean_abs_dp":0.00024132182200749716,"oracle_top2_margin":0.29818880558013916,"near_tie":false,"probs":[0.0869922861456871,0.605330765247345,0.30767694115638733]},{"id":"semif_97a35499bd8717f3e786","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0037092864513397217,"mean_abs_dp":0.002472837766011556,"oracle_top2_margin":0.08557590842247009,"near_tie":false,"probs":[0.26810914278030396,0.41190415620803833,0.3199866712093353]},{"id":"semif_31eb4533d2c89959ec46","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001517951488494873,"mean_abs_dp":0.001011987527211507,"oracle_top2_margin":0.29386773705482483,"near_tie":false,"probs":[0.5731081366539001,0.2764676511287689,0.15042421221733093]},{"id":"semif_0aa24a6701ac9c56c4a0","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0015094280242919922,"mean_abs_dp":0.001006318877140681,"oracle_top2_margin":0.887073278427124,"near_tie":false,"probs":[0.04640694335103035,0.02287796512246132,0.9307151436805725]},{"id":"semif_796d5c0da6eaadae6a3f","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016192793846130371,"mean_abs_dp":0.001079557773967584,"oracle_top2_margin":0.8756633400917053,"near_tie":false,"probs":[0.014456040225923061,0.9291771054267883,0.056366927921772]},{"id":"semif_599b3092b6fd18312d23","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00238250195980072,"mean_abs_dp":0.0015883346398671467,"oracle_top2_margin":0.4057760238647461,"near_tie":false,"probs":[0.24280008673667908,0.1127302274107933,0.644469678401947]},{"id":"semif_960a4bcd27363e6db4b9","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001461416482925415,"mean_abs_dp":0.0009742875893910726,"oracle_top2_margin":0.23092997074127197,"near_tie":false,"probs":[0.28605780005455017,0.19939595460891724,0.5145462155342102]},{"id":"semif_c6a411e7f2bbdf377173","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0022770166397094727,"mean_abs_dp":0.0015180259943008423,"oracle_top2_margin":0.4384426474571228,"near_tie":false,"probs":[0.6412148475646973,0.2067606896162033,0.152024507522583]},{"id":"semif_e332aa927d0c2ed044aa","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014484822750091553,"mean_abs_dp":0.000965644915898641,"oracle_top2_margin":0.025122374296188354,"near_tie":false,"probs":[0.36511683464050293,0.34186941385269165,0.29301372170448303]},{"id":"semif_a0128e551d095f3449f8","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012292899191379547,"mean_abs_dp":8.194753900170326e-05,"oracle_top2_margin":0.9322084188461304,"near_tie":false,"probs":[0.9631202816963196,0.03069826401770115,0.006181567907333374]},{"id":"semif_46d6a0731bd288c370f0","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007600486278533936,"mean_abs_dp":0.0005067139863967896,"oracle_top2_margin":0.4198586940765381,"near_tie":false,"probs":[0.6423293352127075,0.13628987967967987,0.22138074040412903]},{"id":"semif_21ca7a8b970e36b8e20f","q":0,"variant":"base","tokens":97,"calls":7,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002145439386367798,"mean_abs_dp":0.001430282990137736,"oracle_top2_margin":0.043576329946517944,"near_tie":false,"probs":[0.34448981285095215,0.3844691514968872,0.27104106545448303]},{"id":"semif_4aa83af95bb69e90bc2d","q":0,"variant":"base","tokens":95,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010218620300292969,"mean_abs_dp":0.0006812425951162974,"oracle_top2_margin":0.3831571936607361,"near_tie":false,"probs":[0.6714991927146912,0.03845364972949028,0.29004716873168945]},{"id":"semif_cdfa9eed51fafcf09b72","q":0,"variant":"base","tokens":80,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005556046962738037,"mean_abs_dp":0.00037042051553726196,"oracle_top2_margin":0.6123061776161194,"near_tie":false,"probs":[0.17788012325763702,0.030964931473135948,0.7911549210548401]},{"id":"semif_d51b05bfb9d6d7f09331","q":0,"variant":"base","tokens":86,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015180110931396484,"mean_abs_dp":0.0010119924942652385,"oracle_top2_margin":0.4349525570869446,"near_tie":false,"probs":[0.22076289355754852,0.6580715179443359,0.12116558104753494]},{"id":"semif_fdd902cc6e65fad7cb4f","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0017919614911079407,"mean_abs_dp":0.0011946186423301697,"oracle_top2_margin":0.50705486536026,"near_tie":false,"probs":[0.12665608525276184,0.6904308199882507,0.18291310966014862]},{"id":"semif_42384af9ae56beb37ff5","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0034764111042022705,"mean_abs_dp":0.0023176372051239014,"oracle_top2_margin":0.06503075361251831,"near_tie":false,"probs":[0.38842546939849854,0.3279004395008087,0.28367409110069275]},{"id":"semif_cdf7f428857e1b841cb3","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0027874112129211426,"mean_abs_dp":0.0018582393725713093,"oracle_top2_margin":0.3410225212574005,"near_tie":false,"probs":[0.17200760543346405,0.2455238252878189,0.5824685096740723]},{"id":"semif_774f7f9245f83c796d44","q":0,"variant":"base","tokens":63,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00259476900100708,"mean_abs_dp":0.0017298211654027302,"oracle_top2_margin":0.7296704649925232,"near_tie":false,"probs":[0.10275521874427795,0.0691225677728653,0.8281221985816956]},{"id":"semif_92ec2ffa3dd46ada1335","q":0,"variant":"base","tokens":87,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012932419776916504,"mean_abs_dp":0.0008621762196222941,"oracle_top2_margin":0.36772847175598145,"near_tie":false,"probs":[0.1431562751531601,0.2454327791929245,0.6114109754562378]},{"id":"semif_6149a17bc154f9c5b4a0","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00024406611919403076,"mean_abs_dp":0.00016274054845174155,"oracle_top2_margin":0.6154326796531677,"near_tie":false,"probs":[0.7627754807472229,0.14685769379138947,0.09036684781312943]},{"id":"semif_be866e1dbec5d42d9423","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0024535655975341797,"mean_abs_dp":0.0016357079148292542,"oracle_top2_margin":0.6972379684448242,"near_tie":false,"probs":[0.06523808091878891,0.11668809503316879,0.8180738091468811]},{"id":"semif_97d2ed2591fc7638806c","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003792107105255127,"mean_abs_dp":0.0025280937552452087,"oracle_top2_margin":0.31745457649230957,"near_tie":false,"probs":[0.2969872057437897,0.6081623435020447,0.09485051780939102]},{"id":"semif_4533dd91267b4d815690","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004549801349639893,"mean_abs_dp":0.0030332008997599282,"oracle_top2_margin":0.4459984302520752,"near_tie":false,"probs":[0.1823236644268036,0.18950706720352173,0.6281692981719971]},{"id":"semif_d880e65fef5e2993fdd0","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0010447502136230469,"mean_abs_dp":0.0006965001424153646,"oracle_top2_margin":0.05805802345275879,"near_tie":false,"probs":[0.2900380790233612,0.32512012124061584,0.38484176993370056]},{"id":"semif_90494c76078ffc3c717a","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019719600677490234,"mean_abs_dp":0.001314630111058553,"oracle_top2_margin":0.022894471883773804,"near_tie":false,"probs":[0.3525109887123108,0.3783930242061615,0.2690959870815277]},{"id":"semif_9ac5cc82c4126c91174a","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000849306583404541,"mean_abs_dp":0.0005661944548288981,"oracle_top2_margin":0.042064547538757324,"near_tie":false,"probs":[0.2610919773578644,0.34923943877220154,0.3896685540676117]},{"id":"semif_386e10b1c1ce906473a4","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0069460272789001465,"mean_abs_dp":0.004630679885546367,"oracle_top2_margin":0.43534374237060547,"near_tie":false,"probs":[0.19839569926261902,0.6199936866760254,0.1816106140613556]},{"id":"semif_4da6aedc1139224eeff2","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010564029216766357,"mean_abs_dp":0.0007043058673540751,"oracle_top2_margin":0.7471664547920227,"near_tie":false,"probs":[0.07306884229183197,0.8379635214805603,0.08896767348051071]},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004833519458770752,"mean_abs_dp":0.0032223363717397055,"oracle_top2_margin":0.05925852060317993,"near_tie":false,"probs":[0.22038467228412628,0.35574474930763245,0.4238705635070801]},{"id":"semif_42c54fa2ad45743e24d8","q":0,"variant":"base","tokens":71,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005747675895690918,"mean_abs_dp":0.0003831783930460612,"oracle_top2_margin":0.22642815113067627,"near_tie":false,"probs":[0.12288548797369003,0.5514723062515259,0.3256422281265259]},{"id":"semif_f3d13cf184f916677def","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003911226987838745,"mean_abs_dp":0.0026074747244517007,"oracle_top2_margin":0.05370849370956421,"near_tie":false,"probs":[0.29379892349243164,0.37806960940361023,0.32813140749931335]},{"id":"semif_1105577da4c8dad4609d","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014439821243286133,"mean_abs_dp":0.0009626547495524088,"oracle_top2_margin":0.14877480268478394,"near_tie":false,"probs":[0.2887023091316223,0.27191904187202454,0.43937867879867554]},{"id":"semif_0ebd299739302d8b057b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010683536529541016,"mean_abs_dp":0.0007122457027435303,"oracle_top2_margin":0.05391588807106018,"near_tie":false,"probs":[0.2611001431941986,0.39710965752601624,0.34179016947746277]},{"id":"semif_eafc22c8c40df3932a8e","q":0,"variant":"base","tokens":72,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025572776794433594,"mean_abs_dp":0.0017048319180806477,"oracle_top2_margin":0.0383472740650177,"near_tie":false,"probs":[0.20306040346622467,0.38107189536094666,0.41586774587631226]},{"id":"semif_0aa1447680e4f25862bb","q":0,"variant":"base","tokens":91,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019999444484710693,"mean_abs_dp":0.0013332962989807129,"oracle_top2_margin":0.044051527976989746,"near_tie":false,"probs":[0.3815673887729645,0.2839500606060028,0.3344825804233551]},{"id":"semif_980e76145c925747220c","q":0,"variant":"base","tokens":92,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0027616024017333984,"mean_abs_dp":0.001841050883134206,"oracle_top2_margin":0.48375236988067627,"near_tie":false,"probs":[0.22425661981105804,0.07244666665792465,0.7032967209815979]},{"id":"semif_4aee37a6a155747766aa","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0029226690530776978,"mean_abs_dp":0.0019484410683314006,"oracle_top2_margin":0.13445758819580078,"near_tie":false,"probs":[0.31046658754348755,0.2456844449043274,0.44384896755218506]},{"id":"semif_6db17cc22bd656d78558","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0013927668333053589,"mean_abs_dp":0.0009285261233647665,"oracle_top2_margin":0.40535932779312134,"near_tie":false,"probs":[0.21643486618995667,0.16245923936367035,0.6211058497428894]},{"id":"semif_ef3e0f8e657bf59f8a29","q":0,"variant":"base","tokens":83,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0026321113109588623,"mean_abs_dp":0.0017547359069188435,"oracle_top2_margin":0.15905368328094482,"near_tie":false,"probs":[0.3138304054737091,0.21063348650932312,0.47553613781929016]},{"id":"semif_f756e1cb04e11766341e","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00279998779296875,"mean_abs_dp":0.0018666585286458333,"oracle_top2_margin":0.1089421808719635,"near_tie":false,"probs":[0.2693185806274414,0.3123760223388672,0.4183053970336914]},{"id":"semif_d167e51bb1f499099645","q":0,"variant":"base","tokens":69,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025804340839385986,"mean_abs_dp":0.0017202794551849365,"oracle_top2_margin":0.01671499013900757,"near_tie":true,"probs":[0.3537586033344269,0.3041044771671295,0.3421368896961212]},{"id":"semif_674ca4e9346216255466","q":0,"variant":"base","tokens":59,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009690523147583008,"mean_abs_dp":0.0006460274259249369,"oracle_top2_margin":0.5046945810317993,"near_tie":false,"probs":[0.6911379098892212,0.18521557748317719,0.12364652752876282]},{"id":"semif_1df851a202994a07f123","q":0,"variant":"base","tokens":77,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00033435598015785217,"mean_abs_dp":0.00022287201136350632,"oracle_top2_margin":0.9035253524780273,"near_tie":false,"probs":[0.9474858641624451,0.04456434026360512,0.00794986728578806]},{"id":"semif_b5e6788d95b8395d2888","q":0,"variant":"base","tokens":76,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006784573197364807,"mean_abs_dp":0.00045228687425454456,"oracle_top2_margin":0.8871046304702759,"near_tie":false,"probs":[0.04217051714658737,0.9284937977790833,0.02933579497039318]},{"id":"semif_32f36cd0fbbfb3604586","q":0,"variant":"base","tokens":66,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024565458297729492,"mean_abs_dp":0.0016377419233322144,"oracle_top2_margin":0.6213997006416321,"near_tie":false,"probs":[0.7670260667800903,0.14973711967468262,0.08323688805103302]},{"id":"semif_843185706adff0c60f8b","q":0,"variant":"base","tokens":67,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002083122730255127,"mean_abs_dp":0.0013887376214067142,"oracle_top2_margin":0.12877494096755981,"near_tie":false,"probs":[0.5610144734382629,0.010846512392163277,0.4281390607357025]},{"id":"semif_b531dd603bf080dab820","q":0,"variant":"base","tokens":82,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002045273780822754,"mean_abs_dp":0.0013634935021400452,"oracle_top2_margin":0.35339561104774475,"near_tie":false,"probs":[0.6328071355819702,0.08405262976884842,0.2831402122974396]},{"id":"semif_be2dd67bc733c1213bef","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003005683422088623,"mean_abs_dp":0.0020037939151128135,"oracle_top2_margin":0.12457886338233948,"near_tie":false,"probs":[0.12150075286626816,0.4997411370277405,0.3787580728530884]},{"id":"semif_b8e2750307df9761e716","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0039313435554504395,"mean_abs_dp":0.0026208901156981788,"oracle_top2_margin":0.4760705828666687,"near_tie":false,"probs":[0.7191101908683777,0.030755674466490746,0.250134140253067]},{"id":"semif_5d31b4bf2d0fc193f53b","q":0,"variant":"base","tokens":84,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0028066039085388184,"mean_abs_dp":0.0018710543711980183,"oracle_top2_margin":0.2525418996810913,"near_tie":false,"probs":[0.13606292009353638,0.5606370568275452,0.30330005288124084]},{"id":"semif_a9230edfc6ec9e524af0","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007675588130950928,"mean_abs_dp":0.0005117058753967285,"oracle_top2_margin":0.12013593316078186,"near_tie":false,"probs":[0.2561784088611603,0.4327344000339508,0.3110872209072113]},{"id":"semif_c100e84585149a04be49","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005041956901550293,"mean_abs_dp":0.0003360969324906667,"oracle_top2_margin":0.628185510635376,"near_tie":false,"probs":[0.7971665859222412,0.033107828348875046,0.16972550749778748]},{"id":"semif_579beed184ffc1104aa9","q":0,"variant":"base","tokens":65,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0037153959274291992,"mean_abs_dp":0.0024768958489100137,"oracle_top2_margin":0.16769906878471375,"near_tie":false,"probs":[0.5323346853256226,0.09689363092184067,0.3707716464996338]},{"id":"semif_c4e9bd463b6fde5be743","q":0,"variant":"base","tokens":53,"calls":4,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0021488070487976074,"mean_abs_dp":0.001432540516058604,"oracle_top2_margin":0.7536282539367676,"near_tie":false,"probs":[0.08998070657253265,0.8400712609291077,0.06994806230068207]},{"id":"semif_b863998d4be2721a5dfd","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002282261848449707,"mean_abs_dp":0.00015212595462799072,"oracle_top2_margin":0.08771461248397827,"near_tie":false,"probs":[0.35283708572387695,0.4408319592475891,0.20633095502853394]},{"id":"semif_37cebc3792e98049439f","q":0,"variant":"base","tokens":85,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004991292953491211,"mean_abs_dp":0.0003327510009209315,"oracle_top2_margin":0.8853868246078491,"near_tie":false,"probs":[0.051967766135931015,0.9364559054374695,0.011576336808502674]},{"id":"semif_64d63f86e328aff1481c","q":0,"variant":"base","tokens":93,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006350278854370117,"mean_abs_dp":0.00042335006097952527,"oracle_top2_margin":0.9121700525283813,"near_tie":false,"probs":[0.01927849091589451,0.03476930782198906,0.9459522366523743]},{"id":"semif_6d64a8459f64eac7f4cc","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005237460136413574,"mean_abs_dp":0.00034913917382558185,"oracle_top2_margin":0.39074867963790894,"near_tie":false,"probs":[0.6253637075424194,0.139352485537529,0.23528379201889038]},{"id":"semif_2708193212d8a4a523c7","q":0,"variant":"base","tokens":90,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009089410305023193,"mean_abs_dp":0.0006059755881627401,"oracle_top2_margin":0.20737019181251526,"near_tie":false,"probs":[0.22384843230247498,0.2837306261062622,0.49242085218429565]},{"id":"semif_5a37d1dfcad20810a76a","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000521540641784668,"mean_abs_dp":0.0003477223217487335,"oracle_top2_margin":0.4521888494491577,"near_tie":false,"probs":[0.02779342420399189,0.7126219272613525,0.2595847249031067]},{"id":"semif_acacd8b46acd25cb954b","q":0,"variant":"base","tokens":89,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0022705048322677612,"mean_abs_dp":0.0015136748552322388,"oracle_top2_margin":0.06195080280303955,"near_tie":false,"probs":[0.4253982603549957,0.36158451437950134,0.21301721036434174]},{"id":"semif_4089c199b8b796ec3ed8","q":0,"variant":"base","tokens":88,"calls":6,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":1,"argmax_equal":false,"max_abs_dp":0.0039997100830078125,"mean_abs_dp":0.0026664833227793374,"oracle_top2_margin":0.0032414793968200684,"near_tie":true,"probs":[0.4431862533092499,0.43848466873168945,0.11832913011312485]},{"id":"semif_02f2b95a5c5a4777a45c","q":0,"variant":"base","tokens":74,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0051749348640441895,"mean_abs_dp":0.003449981411298116,"oracle_top2_margin":0.6085776090621948,"near_tie":false,"probs":[0.09541357308626175,0.752659261226654,0.151927188038826]},{"id":"semif_ebea4681e3dcc7d92ca2","q":0,"variant":"base","tokens":79,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0025035738945007324,"mean_abs_dp":0.0016690691312154133,"oracle_top2_margin":0.13673824071884155,"near_tie":false,"probs":[0.2954706847667694,0.2732326090335846,0.4312966763973236]},{"id":"semif_b8c5b9b285cbdddc9f8d","q":0,"variant":"base","tokens":70,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002310842275619507,"mean_abs_dp":0.0015405515829722087,"oracle_top2_margin":0.0788612961769104,"near_tie":false,"probs":[0.3107418715953827,0.29627224802970886,0.39298591017723083]},{"id":"semif_7ed75d854d99b3a81916","q":0,"variant":"base","tokens":68,"calls":5,"finite":true,"all_zero":false,"source":"semif_authored144","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0032462477684020996,"mean_abs_dp":0.002164157728354136,"oracle_top2_margin":0.4379068613052368,"near_tie":false,"probs":[0.6667517423629761,0.0992695763707161,0.23397868871688843]},{"id":"own_t01","q":0,"variant":"base","tokens":184,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001388251781463623,"mean_abs_dp":0.0005927108228206635,"oracle_top2_margin":0.3381020724773407,"near_tie":false,"probs":[0.5431714057922363,0.03420301154255867,0.1478623002767563,0.0680856928229332,0.20667757093906403],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":1,"variant":"base","tokens":149,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.459785461425781e-05,"mean_abs_dp":5.455315113067627e-05,"oracle_top2_margin":0.30454543232917786,"near_tie":false,"probs":[0.3476727604866028,0.652327299118042],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":2,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006360039114952087,"mean_abs_dp":0.0003180112689733505,"oracle_top2_margin":0.22719793021678925,"near_tie":false,"probs":[0.10987676680088043,0.23651953041553497,0.46350929141044617,0.19009438157081604],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t01","q":3,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005526095628738403,"mean_abs_dp":0.0005526021122932434,"oracle_top2_margin":0.7503683567047119,"near_tie":false,"probs":[0.1253684014081955,0.8746315836906433],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_t02","q":0,"variant":"base","tokens":270,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001035258173942566,"mean_abs_dp":0.0008247904479503632,"oracle_top2_margin":0.12554225325584412,"near_tie":false,"probs":[0.4101420044898987,0.2846115231513977,0.19610269367694855,0.10914375633001328]},{"id":"own_t02","q":1,"variant":"base","tokens":303,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00012560002505779266,"mean_abs_dp":6.677825003862382e-05,"oracle_top2_margin":0.7914202213287354,"near_tie":false,"probs":[0.011157718487083912,0.869373083114624,0.023322641849517822,0.01831062138080597,0.07783584296703339]},{"id":"own_t02","q":2,"variant":"base","tokens":262,"calls":17,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006864070892333984,"mean_abs_dp":0.0006863735616207123,"oracle_top2_margin":0.8067709803581238,"near_tie":false,"probs":[0.9040718674659729,0.0959281399846077]},{"id":"own_t03","q":0,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010297894477844238,"mean_abs_dp":0.0003432457645734151,"oracle_top2_margin":0.5557763576507568,"near_tie":false,"probs":[0.045662786811590195,0.7102881073951721,0.03981516510248184,0.029522627592086792,0.018603390082716942,0.1561078280210495]},{"id":"own_t03","q":1,"variant":"base","tokens":214,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00015550851821899414,"mean_abs_dp":0.00015550851821899414,"oracle_top2_margin":0.7302511930465698,"near_tie":false,"probs":[0.1350298970937729,0.8649700880050659]},{"id":"own_t03","q":2,"variant":"base","tokens":245,"calls":16,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002009779214859009,"mean_abs_dp":0.0010048924013972282,"oracle_top2_margin":0.18012985587120056,"near_tie":false,"probs":[0.30350539088249207,0.48150739073753357,0.16952252388000488,0.045464687049388885]},{"id":"own_t04","q":0,"variant":"base","tokens":190,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001896202564239502,"mean_abs_dp":0.0009480924345552921,"oracle_top2_margin":0.8690025806427002,"near_tie":false,"probs":[0.04197196662425995,0.9079057574272156,0.025986433029174805,0.024135837331414223]},{"id":"own_t04","q":1,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005757808685302734,"mean_abs_dp":0.0005757510662078857,"oracle_top2_margin":0.106167733669281,"near_tie":false,"probs":[0.4474918842315674,0.5525081157684326]},{"id":"own_t04","q":2,"variant":"base","tokens":191,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004046261310577393,"mean_abs_dp":0.0020231101661920547,"oracle_top2_margin":0.05060693621635437,"near_tie":false,"probs":[0.14019638299942017,0.04069250077009201,0.43130385875701904,0.3878072202205658]},{"id":"own_t04","q":3,"variant":"base","tokens":175,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005302995443344116,"mean_abs_dp":0.0035353004932403564,"oracle_top2_margin":0.14521116018295288,"near_tie":false,"probs":[0.20906473696231842,0.4647178053855896,0.3262174427509308]},{"id":"own_t05","q":0,"variant":"base","tokens":132,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016525685787200928,"mean_abs_dp":0.0006817840039730072,"oracle_top2_margin":0.08131647109985352,"near_tie":false,"probs":[0.26422369480133057,0.34273281693458557,0.25222474336624146,0.10156071186065674,0.03925806283950806]},{"id":"own_t05","q":1,"variant":"base","tokens":158,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0019702166318893433,"mean_abs_dp":0.0007586365876098474,"oracle_top2_margin":0.397593230009079,"near_tie":false,"probs":[0.020597800612449646,0.6297528147697449,0.2284589260816574,0.024707572534680367,0.013143431395292282,0.08333943039178848]},{"id":"own_t05","q":2,"variant":"base","tokens":117,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00045449286699295044,"mean_abs_dp":0.000454489141702652,"oracle_top2_margin":0.9450063109397888,"near_tie":false,"probs":[0.9720486998558044,0.027951376512646675]},{"id":"own_t05","q":3,"variant":"base","tokens":125,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00014748238027095795,"mean_abs_dp":0.00014747213572263718,"oracle_top2_margin":0.9648926258087158,"near_tie":false,"probs":[0.017701176926493645,0.9822988510131836]},{"id":"own_t06","q":0,"variant":"base","tokens":301,"calls":19,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0031300783157348633,"mean_abs_dp":0.0006292612757533789,"oracle_top2_margin":0.048377931118011475,"near_tie":false,"probs":[0.031482115387916565,0.30302146077156067,0.006566839758306742,0.2509351372718811,0.0226809773594141,0.07181160151958466,0.014437448233366013,0.018341490998864174,0.03504466265439987,0.24567827582359314]},{"id":"own_t06","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.000301361083984e-05,"mean_abs_dp":9.000301361083984e-05,"oracle_top2_margin":0.5263986587524414,"near_tie":false,"probs":[0.7632893323898315,0.23671068251132965]},{"id":"own_t06","q":2,"variant":"base","tokens":224,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007384717464447021,"mean_abs_dp":0.0004923045635223389,"oracle_top2_margin":0.04287439584732056,"near_tie":false,"probs":[0.3931240141391754,0.4361179769039154,0.17075803875923157]},{"id":"own_t07","q":0,"variant":"base","tokens":219,"calls":14,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007802098989486694,"mean_abs_dp":0.0006193816661834717,"oracle_top2_margin":0.13217660784721375,"near_tie":false,"probs":[0.12808090448379517,0.2517339587211609,0.3836266100406647,0.2365584522485733]},{"id":"own_t07","q":1,"variant":"base","tokens":174,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0015332996845245361,"mean_abs_dp":0.0015332847833633423,"oracle_top2_margin":0.6053133010864258,"near_tie":false,"probs":[0.19887663424015045,0.8011233806610107]},{"id":"own_t07","q":2,"variant":"base","tokens":187,"calls":12,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013476014137268066,"mean_abs_dp":0.0008984108765920004,"oracle_top2_margin":0.2717403173446655,"near_tie":false,"probs":[0.5377475023269653,0.26349157094955444,0.1987609565258026]},{"id":"own_t08","q":0,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001704096794128418,"mean_abs_dp":0.0017040669918060303,"oracle_top2_margin":0.1572883427143097,"near_tie":false,"probs":[0.57694011926651,0.4230599105358124]},{"id":"own_t08","q":1,"variant":"base","tokens":155,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007762610912322998,"mean_abs_dp":0.0004231072962284088,"oracle_top2_margin":0.03466594219207764,"near_tie":false,"probs":[0.3098284602165222,0.3431277871131897,0.2188054621219635,0.12823829054832458]},{"id":"own_t08","q":2,"variant":"base","tokens":170,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002612382173538208,"mean_abs_dp":0.0014043357223272324,"oracle_top2_margin":0.25134599208831787,"near_tie":false,"probs":[0.4861803352832794,0.2372504323720932,0.11441926658153534,0.16214995086193085]},{"id":"own_t09","q":0,"variant":"base","tokens":122,"calls":8,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00042472779750823975,"mean_abs_dp":0.0004246756434440613,"oracle_top2_margin":0.6116014122962952,"near_tie":false,"probs":[0.19377459585666656,0.8062253594398499]},{"id":"own_t09","q":1,"variant":"base","tokens":151,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00021463632583618164,"mean_abs_dp":0.0001236594980582595,"oracle_top2_margin":0.7431982159614563,"near_tie":false,"probs":[0.8652536273002625,0.004669331479817629,0.00844839308410883,0.12162858247756958]},{"id":"own_t10","q":0,"variant":"base","tokens":161,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.995698928833008e-05,"mean_abs_dp":4.245538730174303e-05,"oracle_top2_margin":0.9358961582183838,"near_tie":false,"probs":[0.956940233707428,0.01801576092839241,0.02093448117375374,0.0017063971608877182,0.0024030825588852167]},{"id":"own_t10","q":1,"variant":"base","tokens":140,"calls":9,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013127923011779785,"mean_abs_dp":0.0013127326965332031,"oracle_top2_margin":0.07992240786552429,"near_tie":false,"probs":[0.46135154366493225,0.5386484861373901]},{"id":"own_t10","q":2,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009426772594451904,"mean_abs_dp":0.0007563401013612747,"oracle_top2_margin":0.011951148509979248,"near_tie":true,"probs":[0.3215273320674896,0.27070480585098267,0.30920353531837463,0.09856431931257248]},{"id":"own_j01","q":0,"variant":"base","tokens":212,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012857913970947266,"mean_abs_dp":0.001285746693611145,"oracle_top2_margin":0.36813127994537354,"near_tie":false,"probs":[0.31464865803718567,0.6853514313697815]},{"id":"own_j01","q":1,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00043255090713500977,"mean_abs_dp":0.00021626800298690796,"oracle_top2_margin":0.145786851644516,"near_tie":false,"probs":[0.2267099916934967,0.37176406383514404,0.2041897028684616,0.19733619689941406]},{"id":"own_j01","q":2,"variant":"base","tokens":242,"calls":16,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0021840035915374756,"mean_abs_dp":0.0010919906198978424,"oracle_top2_margin":0.06988182663917542,"near_tie":false,"probs":[0.1316101998090744,0.21587003767490387,0.2929374575614929,0.3595822751522064]},{"id":"own_j01","q":3,"variant":"base","tokens":223,"calls":14,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.887052536010742e-05,"mean_abs_dp":8.885562419891357e-05,"oracle_top2_margin":0.15178239345550537,"near_tie":false,"probs":[0.4240199625492096,0.5759800672531128]},{"id":"own_j02","q":0,"variant":"base","tokens":332,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001283407211303711,"mean_abs_dp":0.0003684727009385824,"oracle_top2_margin":0.007524773478507996,"near_tie":true,"probs":[0.20408323407173157,0.08876383304595947,0.19799970090389252,0.08309917151927948,0.0279974527657032,0.004260205663740635,0.01964416541159153,0.16099944710731506,0.015448244288563728,0.19770458340644836]},{"id":"own_j02","q":1,"variant":"base","tokens":232,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011616945266723633,"mean_abs_dp":0.0011616647243499756,"oracle_top2_margin":0.10447829961776733,"near_tie":false,"probs":[0.4489225149154663,0.5510774850845337]},{"id":"own_j02","q":2,"variant":"base","tokens":237,"calls":15,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0011234283447265625,"mean_abs_dp":0.0011234134435653687,"oracle_top2_margin":0.08897575736045837,"near_tie":false,"probs":[0.45438873767852783,0.5456113219261169]},{"id":"own_j03","q":0,"variant":"base","tokens":380,"calls":24,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005763769149780273,"mean_abs_dp":0.0002881847321987152,"oracle_top2_margin":0.06177407503128052,"near_tie":false,"probs":[0.34090062975883484,0.22801019251346588,0.15207713842391968,0.279011994600296],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j03","q":1,"variant":"base","tokens":333,"calls":21,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00036203861236572266,"mean_abs_dp":0.00016595636095319475,"oracle_top2_margin":0.020112618803977966,"near_tie":false,"probs":[0.2297314554452896,0.06666494160890579,0.14797312021255493,0.20979756116867065,0.11441891640424728,0.08414887636899948,0.14726512134075165],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j03","q":2,"variant":"base","tokens":314,"calls":20,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008125901222229004,"mean_abs_dp":0.0008125901222229004,"oracle_top2_margin":0.02395007014274597,"near_tie":false,"probs":[0.4872123897075653,0.5127876400947571],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_j04","q":0,"variant":"base","tokens":180,"calls":12,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0042130351066589355,"mean_abs_dp":0.002106519415974617,"oracle_top2_margin":0.5144549608230591,"near_tie":false,"probs":[0.10975559800863266,0.12029726803302765,0.6393136978149414,0.1306334286928177]},{"id":"own_j04","q":1,"variant":"base","tokens":171,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007322132587432861,"mean_abs_dp":0.0007321983575820923,"oracle_top2_margin":0.32075631618499756,"near_tie":false,"probs":[0.6611103415489197,0.33888962864875793]},{"id":"own_j04","q":2,"variant":"base","tokens":196,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025519728660583496,"mean_abs_dp":0.001276060938835144,"oracle_top2_margin":0.08215877413749695,"near_tie":false,"probs":[0.3833765983581543,0.30376961827278137,0.15931902825832367,0.15353477001190186]},{"id":"own_j05","q":0,"variant":"base","tokens":198,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0036762505769729614,"mean_abs_dp":0.002450828750928243,"oracle_top2_margin":0.4507710933685303,"near_tie":false,"probs":[0.17603537440299988,0.634005606174469,0.18995901942253113]},{"id":"own_j05","q":1,"variant":"base","tokens":153,"calls":10,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006572306156158447,"mean_abs_dp":0.0006571859121322632,"oracle_top2_margin":0.024675756692886353,"near_tie":false,"probs":[0.5116807222366333,0.4883193373680115]},{"id":"own_j05","q":2,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007919520139694214,"mean_abs_dp":0.0003613308072090149,"oracle_top2_margin":0.09147655963897705,"near_tie":false,"probs":[0.2480839043855667,0.092924565076828,0.11953124403953552,0.11747381836175919,0.15167199075222015,0.15538614988327026,0.11492831259965897]},{"id":"own_a01","q":0,"variant":"base","tokens":199,"calls":13,"finite":true,"all_zero":false,"source":"own_json","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005387365818023682,"mean_abs_dp":0.00030421558767557144,"oracle_top2_margin":0.12012860178947449,"near_tie":false,"probs":[0.3460831642150879,0.46669793128967285,0.05941498279571533,0.12780384719371796]},{"id":"own_a01","q":1,"variant":"base","tokens":166,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003209114074707031,"mean_abs_dp":0.00032085180282592773,"oracle_top2_margin":0.31672245264053345,"near_tie":false,"probs":[0.3419596552848816,0.6580404043197632]},{"id":"own_a01","q":2,"variant":"base","tokens":163,"calls":11,"finite":true,"all_zero":false,"source":"own_json","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010159611701965332,"mean_abs_dp":0.00010158121585845947,"oracle_top2_margin":0.09236004948616028,"near_tie":false,"probs":[0.5462815761566162,0.453718364238739]},{"id":"own_m01","q":0,"variant":"base","tokens":156,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004343986511230469,"mean_abs_dp":0.000289591650168101,"oracle_top2_margin":0.8812302947044373,"near_tie":false,"probs":[0.03134109824895859,0.9246891736984253,0.043969713151454926],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":1,"variant":"base","tokens":150,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007940530776977539,"mean_abs_dp":0.0007940474897623062,"oracle_top2_margin":0.8975095748901367,"near_tie":false,"probs":[0.05203923583030701,0.9479607343673706],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":2,"variant":"base","tokens":159,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003481060266494751,"mean_abs_dp":0.0002320359150568644,"oracle_top2_margin":0.3613918125629425,"near_tie":false,"probs":[0.653232753276825,0.2915039658546448,0.05526325851678848],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":3,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00029587745666503906,"mean_abs_dp":0.00019726715981960297,"oracle_top2_margin":0.4848095774650574,"near_tie":false,"probs":[0.028377370908856392,0.24359695613384247,0.7280257344245911],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":4,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.29425048828125e-05,"mean_abs_dp":6.290338933467865e-05,"oracle_top2_margin":0.9664559364318848,"near_tie":false,"probs":[0.9831650257110596,0.016834918409585953],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":5,"variant":"base","tokens":168,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"choice","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012432336807250977,"mean_abs_dp":0.0005555517971515655,"oracle_top2_margin":0.22992277145385742,"near_tie":false,"probs":[0.033709317445755005,0.03499662131071091,0.5577824711799622,0.04455430433154106,0.3289572596549988],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":6,"variant":"base","tokens":154,"calls":10,"finite":true,"all_zero":false,"source":"own_text","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002255380153656006,"mean_abs_dp":0.002255380153656006,"oracle_top2_margin":0.44855186343193054,"near_tie":false,"probs":[0.27797946333885193,0.7220205664634705],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_m01","q":7,"variant":"base","tokens":169,"calls":11,"finite":true,"all_zero":false,"source":"own_text","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007002800703048706,"mean_abs_dp":0.0005167843773961067,"oracle_top2_margin":0.6617178320884705,"near_tie":false,"probs":[0.045733217149972916,0.08619163930416107,0.7649799585342407,0.10309518873691559],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L01","q":0,"variant":"base","tokens":1793,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00026644766330718994,"mean_abs_dp":0.00013228636234998702,"oracle_top2_margin":0.5732508897781372,"near_tie":false,"probs":[0.03332134336233139,0.7471098303794861,0.03302256390452385,0.012957594357430935,0.17358866333961487]},{"id":"own_L01","q":1,"variant":"base","tokens":1766,"calls":111,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014964938163757324,"mean_abs_dp":0.0014964640140533447,"oracle_top2_margin":0.3969528377056122,"near_tie":false,"probs":[0.30002710223197937,0.6999728679656982]},{"id":"own_L01","q":2,"variant":"base","tokens":1802,"calls":113,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.00162428617477417,"mean_abs_dp":0.0009150430560112,"oracle_top2_margin":0.03610238432884216,"near_tie":false,"probs":[0.05405528098344803,0.07842056453227997,0.11042151600122452,0.14704398810863495,0.32282882928848267,0.28722986578941345]},{"id":"own_L02","q":0,"variant":"base","tokens":1518,"calls":95,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.006616711616516113,"mean_abs_dp":0.0026467077434062956,"oracle_top2_margin":0.43417489528656006,"near_tie":false,"probs":[0.17698770761489868,0.030158283188939095,0.07300910353660583,0.6010918617248535,0.11875312775373459],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":1,"variant":"base","tokens":1497,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.1444091796875e-05,"mean_abs_dp":1.1436641216278076e-05,"oracle_top2_margin":0.7794755697250366,"near_tie":false,"probs":[0.8897263407707214,0.11027365922927856],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":2,"variant":"base","tokens":1539,"calls":97,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0014500021934509277,"mean_abs_dp":0.0007249945774674416,"oracle_top2_margin":0.735587477684021,"near_tie":false,"probs":[0.8547623753547668,0.01040624175220728,0.12172850221395493,0.013102862052619457],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L02","q":3,"variant":"base","tokens":1499,"calls":94,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000919535756111145,"mean_abs_dp":0.0009195283055305481,"oracle_top2_margin":0.8828529715538025,"near_tie":false,"probs":[0.0594930462539196,0.9405069351196289],"hidden":{"min_pos_cos":0.9998692690427006,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_L03","q":0,"variant":"base","tokens":1475,"calls":93,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003718733787536621,"mean_abs_dp":0.0018593594431877136,"oracle_top2_margin":0.4104243218898773,"near_tie":false,"probs":[0.026993775740265846,0.6452791690826416,0.24117042124271393,0.08655660599470139]},{"id":"own_L03","q":1,"variant":"base","tokens":1445,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"noul","argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0012344419956207275,"mean_abs_dp":0.0012344270944595337,"oracle_top2_margin":0.6987161636352539,"near_tie":false,"probs":[0.1518763303756714,0.8481236696243286]},{"id":"own_L03","q":2,"variant":"base","tokens":1451,"calls":91,"finite":true,"all_zero":false,"source":"own_long","type":"choice","argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012988820672035217,"mean_abs_dp":0.0012988634407520294,"oracle_top2_margin":0.8095381259918213,"near_tie":false,"probs":[0.9060679078102112,0.09393204748630524]},{"id":"own_L03","q":3,"variant":"base","tokens":1463,"calls":92,"finite":true,"all_zero":false,"source":"own_long","type":"score","argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0035415291786193848,"mean_abs_dp":0.001770753413438797,"oracle_top2_margin":0.2239849865436554,"near_tie":false,"probs":[0.07892508059740067,0.32422763109207153,0.5423741340637207,0.054473064839839935]}],"note":"int8mix with layers 0-11 fp16: outside the bisect rule's search (the rule chose no set); measured as the supervisor's reference","generated_at":"2026-10-03T10:32:51+00:00"},"red_int8mix":{"schema":"kev-decoder-readout-red-arms/1","bundle":{"bundle":"$ZOO_WORK_ROOT/_kev/exports/bundles/kev_0_8b_decode_int8mix_pf16","name":"kev_0_8b_decode_int8mix_pf16","kind":"decision-backbone","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":93,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the pointer head runs on the host, head/)"},"prefill_chunk":16,"metadata_sha256":"218cf045abff64f6d7ffaa4df3673c5e6852a3890f7335267d8b67746bc333b1","main_mlirb":{"bytes":1273276920,"sha256":"84b37b90783782a80f4a348a69e7cccbe96156d1b8a5907b6f25a399beb7dadf"},"tokenizer_sha256":{"tokenizer.json":"fe000e3ed39ed12b8d2481d527d44f93c65d37e87645d2dcc80d1bf9d50d2927","tokenizer_config.json":"e611fbccc7c29ef3b1cafb1cb7ea548d189968632901d678fd62be68c47885de"},"head_sha256":{"head.safetensors":"12038b02ecf7ece3ffb008b22ae97988293e521a86e7aadc8036c9c700893a42","kev_head.json":"73d510708790422d3c84bfc915e311e536d527bc032f0a07394e76999d5c9d7c"},"aimodelc":{"path":"$ZOO_WORK_ROOT/_kev/exports/bundles_aotc/kev_0_8b_decode_int8mix_pf16.h16c.aimodelc","bytes":2272483979,"tree_sha256":"a551fda4c99dea9db6841efa22cec3d7067deb61a4ae7d8c6791b2d2832ef000","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"d0206168f8bdf0d9cc30e09076e05ed93883c7c62ac8052c3369219196fd8e75","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"aa6b981d712f3c85f36aab05556dda7dd26ab8f53d980bbc7e20ced7590b6c69","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"a2b898e43b32f163f81c0b13548abade65af7dcfdc57a6df85bee5e3a0e7a195","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"0347cebeef326444687ae7f0fac61d0d890bb220a43fa4175c499e2017adc390","main-h16c.mlirb":"717762adc6e3fb54b617c9860ac4d1467d175148878bf01b2831d744f7763506","main.hash":"97a6ba5f28134877a053ac43c66f2a59a4d9e4b47dd58e0557a7b09278e560e7","metadata.json":"f9ecebe00b0ded7788a14464033ef5f1656a804102ad0833949f284cb7737e58","stats.json":"edd7c3ed4bc2d810160fe4159fac84793290fb66725645b4013d56c61db877d6"}}},"chunk":16,"max_ctx":4096,"rows_file":{"path":"$ZOO_WORK_ROOT/_kev/readout/red_arms_v2.json","sha256":"2de94c557e94ecf318242331d7bcba61e1372e4b4fb0ab8e95a695cd0ca0be89","version":2},"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8mix_pf16.json","sha256":"0b4be0782bbacc28e00a10772b9cec13160dae5eb1fbcb9965e65bc58c1f17bb","result":"PASS"},"script":{"path":"conversion/kev/readout_gate.py","sha256":"4bdc3b1f670a4a2abb0104472bde8aac682ac0b0a0b9d2ff5df0aefd9c3cfd91"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","head":"the pointer head from head.safetensors, fp32, CPU (parity_decoder_torch.KevHead)","gpu":"shared with other sessions, no _GPU_LOCK held (times are contended reference values)"},"gpu_lock":{"start":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"},"end":{"path":"$ZOO_WORK_ROOT/_GPU_LOCK","exists":true,"bytes":0,"content":"","mtime":"2026-10-03T15:28:17+09:00"}},"process":[{"shard":"red_arms_v2","runs":["tv4_000:q0:base","tv4_000:q0:state_swap","semif_a3f18f3a63d45345942b:q0:base","semif_a3f18f3a63d45345942b:q0:state_swap","semif_0b43ea8e24e74d621f1b:q0:base","semif_0b43ea8e24e74d621f1b:q0:state_swap","own_t01:q0:base","own_t01:q0:state_swap","tv4x_qnli_00:q0:base","tv4x_qnli_00:q0:state_swap","tv4x_tweet_offensive_00:q0:base","tv4x_tweet_offensive_00:q0:instr_negate_v2","tv4x_qnli_00:q0:instr_negate_v2","tv4x_paws_00:q0:base","tv4x_paws_00:q0:instr_negate_v2","tv4x_legacy_holdout_00:q0:base","tv4x_legacy_holdout_00:q0:instr_negate_v2","own_t05:q2:base","own_t05:q2:instr_negate_v2"],"prompts":20,"load_seconds":0.29289979199529625,"load_split_seconds":{"model":0.007125417003408074,"main":0.2857743749918882},"function_names":["main"],"process_wall_seconds":3.6375809170131106,"reset_check":{"run":"tv4_000:q0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1328229999926407},"contract_mismatch":[]}],"rule":"red = the perturbed runs against the unperturbed ones fail the gate's own bar (FACTS §7): (a) an argmax moves on at least one question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over runs of the run's mean |dp| > 0.002","v1_rule":"argmax moved on >= 1 row AND max |dp| > 0.02","arms":{"instr_negate_v2":{"rows":5,"argmax_moved":2,"argmax_moved_non_near_tie":2,"max_abs_dp_vs_base":0.756274864077568,"mean_of_run_mean_abs_dp_vs_base":0.29028557520359755,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4x_tweet_offensive_00:q0","type":"noul","state_from":null,"tokens":69,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08343744277954102,"mean_abs_dp_vs_base":0.08343743532896042,"base_probs":[0.2607780396938324,0.7392219305038452],"perturbed_probs":[0.17734061181545258,0.8226593732833862],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":null,"tokens":95,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.08787530660629272,"mean_abs_dp_vs_base":0.08787522464990616,"base_probs":[0.11449417471885681,0.8855059146881104],"perturbed_probs":[0.2023693174123764,0.7976306080818176],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_paws_00:q0","type":"noul","state_from":null,"tokens":82,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":7.832050323486328e-05,"mean_abs_dp_vs_base":7.830560207366943e-05,"base_probs":[0.30488285422325134,0.6951171159744263],"perturbed_probs":[0.30480456352233887,0.6951954364776611],"base_argmax":1,"perturbed_argmax":1,"oracle_argmax":1},{"row":"tv4x_legacy_holdout_00:q0","type":"noul","state_from":null,"tokens":79,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.5237621068954468,"mean_abs_dp_vs_base":0.5237621068954468,"base_probs":[0.15160804986953735,0.8483919501304626],"perturbed_probs":[0.6753701567649841,0.32462984323501587],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t05:q2","type":"noul","state_from":null,"tokens":118,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.756274864077568,"mean_abs_dp_vs_base":0.7562748035416007,"base_probs":[0.9720486998558044,0.027951376512646675],"perturbed_probs":[0.2157738357782364,0.78422611951828],"base_argmax":0,"perturbed_argmax":1,"oracle_argmax":0}]},"state_swap":{"rows":5,"argmax_moved":4,"argmax_moved_non_near_tie":4,"max_abs_dp_vs_base":0.9049020633101463,"mean_of_run_mean_abs_dp_vs_base":0.4531575363688171,"red_facts7":{"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_means":true},"red":true,"red_v1_rule":true,"items":[{"row":"tv4_000:q0","type":"choice","state_from":"tv4_001","tokens":80,"near_tie":false,"argmax_moved":false,"max_abs_dp_vs_base":0.02585768699645996,"mean_abs_dp_vs_base":0.02105548605322838,"base_probs":[0.1486000418663025,0.2049126923084259,0.43501436710357666,0.21147295832633972],"perturbed_probs":[0.13231992721557617,0.23077037930488586,0.4091835021972656,0.22772623598575592],"base_argmax":2,"perturbed_argmax":2,"oracle_argmax":2},{"row":"semif_a3f18f3a63d45345942b:q0","type":"choice","state_from":"semif_0b43ea8e24e74d621f1b","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.9049020633101463,"mean_abs_dp_vs_base":0.6032680210967859,"base_probs":[0.021132148802280426,0.16643394529819489,0.8124338984489441],"perturbed_probs":[0.9260342121124268,0.04999573901295662,0.0239701047539711],"base_argmax":2,"perturbed_argmax":0,"oracle_argmax":2},{"row":"semif_0b43ea8e24e74d621f1b:q0","type":"choice","state_from":"semif_942ec65dac0bfb9874f9","tokens":60,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.8788161128759384,"mean_abs_dp_vs_base":0.5858774284521738,"base_probs":[0.03758374974131584,0.9418790936470032,0.020537111908197403],"perturbed_probs":[0.49051374197006226,0.06306298077106476,0.4464232921600342],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1},{"row":"own_t01:q0","type":"choice","state_from":"own_t03","tokens":245,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.6922697871923447,"mean_abs_dp_vs_base":0.28022061502560974,"base_probs":[0.5431714057922363,0.03420301154255867,0.1478623002767563,0.0680856928229332,0.20667757093906403],"perturbed_probs":[0.006127014756202698,0.04248477518558502,0.007401399780064821,0.04503946006298065,0.8989473581314087],"base_argmax":0,"perturbed_argmax":4,"oracle_argmax":0},{"row":"tv4x_qnli_00:q0","type":"noul","state_from":"tv4x_paws_00","tokens":52,"near_tie":false,"argmax_moved":true,"max_abs_dp_vs_base":0.7753661647439003,"mean_abs_dp_vs_base":0.7753661312162876,"base_probs":[0.11449417471885681,0.8855059146881104],"perturbed_probs":[0.8898602724075317,0.11013974994421005],"base_argmax":1,"perturbed_argmax":0,"oracle_argmax":1}]}},"base_vs_gate_run":[{"row":"tv4_000:q0","bit_equal_gate_run":true},{"row":"semif_a3f18f3a63d45345942b:q0","bit_equal_gate_run":true},{"row":"semif_0b43ea8e24e74d621f1b:q0","bit_equal_gate_run":true},{"row":"own_t01:q0","bit_equal_gate_run":true},{"row":"tv4x_qnli_00:q0","bit_equal_gate_run":true},{"row":"tv4x_tweet_offensive_00:q0","bit_equal_gate_run":true},{"row":"tv4x_paws_00:q0","bit_equal_gate_run":true},{"row":"tv4x_legacy_holdout_00:q0","bit_equal_gate_run":true},{"row":"own_t05:q2","bit_equal_gate_run":true}],"base_bit_equal_all":true,"all_arms_red":true,"reset_bit_equal":true,"result":"RED (every arm)","seconds":5.718735458009178,"generated_at":"2026-10-03T10:32:58+00:00"},"bisect_rule":{"schema":"kev-int8-bisect-rule/1","rule":{"target":"worst max|dp| over the 60 bisect rows < 0.010 (0.5 x the gate bar 0.02; every option of every question, near-ties included), this instrument","reproduction_check":"before any selection: all_int8's worst max|dp| over the bisect rows (this instrument) >= 0.5 x the int8lin gate transcript's worst over the same rows; otherwise stop and report (the gate's error would not be the int8 weights')","instrument_floor":"exact must reproduce the round-2 P1 run (parity/p1_shard*.jsonl) of every bisect row bit for bit (probabilities), and every process's swap-back re-run likewise","size":"a set's size = the params of the linears it keeps out of int8 (the bytes it costs)","cap":"a candidate keeps at most 50% of the body's linear params out of int8","search":["1. fixed configs: exact, all_int8, layer_NN_fp16 (24), gdn_fp16, attn_fp16, mlp_fp16, proj_<kind>_fp16 (12)","2. layer sets: the 24 layers ranked by their layer_NN_fp16 worst (ascending; ties: the lower index); L_k = the top k, for k = 2, 3, ... while within the cap; at the first k whose L_k meets the target (k = 1: the best single), also every k-subset of the top k+1 layers","3. kind sets: the 12 kinds ranked by their proj_<kind>_fp16 worst (ascending; ties: the order qkv z a b out q k v o gate up down); K_j = the top j, for j = 2, 3, ... while within the cap; at the first j whose K_j meets the target, also every j-subset of the top j+1 kinds; gdn_fp16 / attn_fp16 / mlp_fp16 are kind sets too","4. only if no set of steps 1-3 meets the target: unions L_k + K_j (k, j >= 1, the top-k / top-j sets of 2 and 3) within the cap, in ascending size (ties: smaller k, then smaller j), evaluated in that order until the first that meets the target, at most 12 of them"],"choose":"among the layer and kind sets of steps 1-3 that meet the target, the smallest size (ties: the layer set, then the lexicographically smallest sorted list); if none, the first union of step 4 that meets it; if none, no set (report the numbers: the ship form is the supervisor's call)","then":"the chosen set is exported (export_decoder.py int8mix) and gated on all 434 rows with the gate's own bar, then on the 130 held-out rows (never used to choose)"},"target":0.01,"cap_fraction":0.5,"rows":[["semif_1105577da4c8dad4609d",0],["semif_a22dab8ba073ff6cdb8d",0],["tv4_023",0],["tv4x_composition_holdout_15",0],["semif_46b7029b9a704138b77a",0],["semif_f3d13cf184f916677def",0],["semif_386e10b1c1ce906473a4",0],["tv4x_composition_holdout_12",0],["tv4x_composition_holdout_14",0],["tv4x_legacy_holdout_13",0],["tv4x_sciq_04",0],["own_j04",0],["tv4x_legacy_holdout_19",0],["semif_7ada97b6f99f4788e5c7",0],["tv4x_sciq_17",0],["tv4_020",0],["semif_2608370d01783735c3ea",0],["own_L03",0],["tv4x_legacy_holdout_15",0],["semif_d7e84e4245a29817da1f",0],["semif_c6a411e7f2bbdf377173",0],["semif_834f3b3e10cc61b33d25",0],["semif_acacd8b46acd25cb954b",0],["semif_599b3092b6fd18312d23",0],["semif_0ebd299739302d8b057b",0],["semif_42384af9ae56beb37ff5",0],["tv4_036",0],["semif_cdfa9eed51fafcf09b72",0],["semif_25172cd6ddc49ab0ca70",0],["semif_d41072dc35d8cd14c056",0],["own_j02",1],["own_L02",2],["own_L03",2],["own_t04",2],["own_m01",4],["own_m01",6],["semif_56c419b12cd37cbe790c",0],["semif_07ee70cfe4b01330ce89",0],["semif_00bb4cc043f7d2b288d4",0],["semif_533d4423d311de82b27d",0],["semif_3e75f2b84e8e08e3d3f2",0],["semif_d51b05bfb9d6d7f09331",0],["semif_980e76145c925747220c",0],["semif_37cebc3792e98049439f",0],["semif_5a37d1dfcad20810a76a",0],["tv4_028",0],["tv4_034",0],["tv4_050",0],["tv4_057",0],["tv4s_05",0],["tv4x_tweet_offensive_03",0],["tv4x_tweet_offensive_13",0],["tv4x_paws_15",0],["tv4x_sciq_01",0],["tv4x_sciq_12",0],["tv4x_sciq_16",0],["tv4x_legacy_holdout_03",0],["tv4x_legacy_holdout_06",0],["tv4x_composition_holdout_06",0],["tv4x_composition_holdout_11",0]],"rows_detail":[{"id":"semif_1105577da4c8dad4609d","q":0,"set":"top","source":"semif_authored144","row_len":88,"near_tie":false,"gate_int8lin_max_abs_dp":0.04313388466835022},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"set":"top","source":"semif_authored144","row_len":84,"near_tie":false,"gate_int8lin_max_abs_dp":0.04081737995147705},{"id":"tv4_023","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":92,"near_tie":true,"gate_int8lin_max_abs_dp":0.026517271995544434},{"id":"tv4x_composition_holdout_15","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":122,"near_tie":false,"gate_int8lin_max_abs_dp":0.026461392641067505},{"id":"semif_46b7029b9a704138b77a","q":0,"set":"top","source":"semif_authored144","row_len":78,"near_tie":false,"gate_int8lin_max_abs_dp":0.026026636362075806},{"id":"semif_f3d13cf184f916677def","q":0,"set":"top","source":"semif_authored144","row_len":91,"near_tie":false,"gate_int8lin_max_abs_dp":0.02583444118499756},{"id":"semif_386e10b1c1ce906473a4","q":0,"set":"top","source":"semif_authored144","row_len":88,"near_tie":false,"gate_int8lin_max_abs_dp":0.024856865406036377},{"id":"tv4x_composition_holdout_12","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":122,"near_tie":false,"gate_int8lin_max_abs_dp":0.024287700653076172},{"id":"tv4x_composition_holdout_14","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":122,"near_tie":false,"gate_int8lin_max_abs_dp":0.024287700653076172},{"id":"tv4x_legacy_holdout_13","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":78,"near_tie":false,"gate_int8lin_max_abs_dp":0.02400749921798706},{"id":"tv4x_sciq_04","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":117,"near_tie":false,"gate_int8lin_max_abs_dp":0.019666969776153564},{"id":"own_j04","q":0,"set":"top","source":"own_json","row_len":180,"near_tie":false,"gate_int8lin_max_abs_dp":0.018736541271209717},{"id":"tv4x_legacy_holdout_19","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":76,"near_tie":false,"gate_int8lin_max_abs_dp":0.0182955265045166},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"set":"top","source":"semif_authored144","row_len":67,"near_tie":false,"gate_int8lin_max_abs_dp":0.018214523792266846},{"id":"tv4x_sciq_17","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":169,"near_tie":false,"gate_int8lin_max_abs_dp":0.01787745952606201},{"id":"tv4_020","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":82,"near_tie":false,"gate_int8lin_max_abs_dp":0.017518997192382812},{"id":"semif_2608370d01783735c3ea","q":0,"set":"top","source":"semif_authored144","row_len":69,"near_tie":false,"gate_int8lin_max_abs_dp":0.01745012402534485},{"id":"own_L03","q":0,"set":"top","source":"own_long","row_len":1475,"near_tie":false,"gate_int8lin_max_abs_dp":0.016770899295806885},{"id":"tv4x_legacy_holdout_15","q":0,"set":"top","source":"transfer_v4_dev_sources","row_len":77,"near_tie":false,"gate_int8lin_max_abs_dp":0.016536161303520203},{"id":"semif_d7e84e4245a29817da1f","q":0,"set":"top","source":"semif_authored144","row_len":67,"near_tie":false,"gate_int8lin_max_abs_dp":0.016526520252227783},{"id":"semif_c6a411e7f2bbdf377173","q":0,"set":"top","source":"semif_authored144","row_len":90,"near_tie":false,"gate_int8lin_max_abs_dp":0.015746235847473145},{"id":"semif_834f3b3e10cc61b33d25","q":0,"set":"top","source":"semif_authored144","row_len":58,"near_tie":false,"gate_int8lin_max_abs_dp":0.01570206880569458},{"id":"semif_acacd8b46acd25cb954b","q":0,"set":"top","source":"semif_authored144","row_len":89,"near_tie":false,"gate_int8lin_max_abs_dp":0.015568524599075317},{"id":"semif_599b3092b6fd18312d23","q":0,"set":"top","source":"semif_authored144","row_len":70,"near_tie":false,"gate_int8lin_max_abs_dp":0.01459193229675293},{"id":"semif_0ebd299739302d8b057b","q":0,"set":"top","source":"semif_authored144","row_len":84,"near_tie":false,"gate_int8lin_max_abs_dp":0.014297336339950562},{"id":"semif_42384af9ae56beb37ff5","q":0,"set":"top","source":"semif_authored144","row_len":84,"near_tie":false,"gate_int8lin_max_abs_dp":0.01421266794204712},{"id":"tv4_036","q":0,"set":"top","source":"transfer_v4_dev_head","row_len":150,"near_tie":false,"gate_int8lin_max_abs_dp":0.014157116413116455},{"id":"semif_cdfa9eed51fafcf09b72","q":0,"set":"top","source":"semif_authored144","row_len":80,"near_tie":false,"gate_int8lin_max_abs_dp":0.014052867889404297},{"id":"semif_25172cd6ddc49ab0ca70","q":0,"set":"top","source":"semif_authored144","row_len":66,"near_tie":false,"gate_int8lin_max_abs_dp":0.013986349105834961},{"id":"semif_d41072dc35d8cd14c056","q":0,"set":"top","source":"semif_authored144","row_len":78,"near_tie":false,"gate_int8lin_max_abs_dp":0.013136208057403564},{"id":"own_j02","q":1,"set":"random","source":"own_json","row_len":232,"near_tie":false,"gate_int8lin_max_abs_dp":0.003012537956237793},{"id":"own_L02","q":2,"set":"random","source":"own_long","row_len":1539,"near_tie":false,"gate_int8lin_max_abs_dp":0.0012988448143005371},{"id":"own_L03","q":2,"set":"random","source":"own_long","row_len":1451,"near_tie":false,"gate_int8lin_max_abs_dp":0.0030238404870033264},{"id":"own_t04","q":2,"set":"random","source":"own_text","row_len":191,"near_tie":false,"gate_int8lin_max_abs_dp":0.0014303624629974365},{"id":"own_m01","q":4,"set":"random","source":"own_text","row_len":154,"near_tie":false,"gate_int8lin_max_abs_dp":0.00023170560598373413},{"id":"own_m01","q":6,"set":"random","source":"own_text","row_len":154,"near_tie":false,"gate_int8lin_max_abs_dp":0.0049925148487091064},{"id":"semif_56c419b12cd37cbe790c","q":0,"set":"random","source":"semif_authored144","row_len":71,"near_tie":false,"gate_int8lin_max_abs_dp":0.0036872774362564087},{"id":"semif_07ee70cfe4b01330ce89","q":0,"set":"random","source":"semif_authored144","row_len":58,"near_tie":false,"gate_int8lin_max_abs_dp":0.00262296199798584},{"id":"semif_00bb4cc043f7d2b288d4","q":0,"set":"random","source":"semif_authored144","row_len":72,"near_tie":false,"gate_int8lin_max_abs_dp":0.00046466849744319916},{"id":"semif_533d4423d311de82b27d","q":0,"set":"random","source":"semif_authored144","row_len":72,"near_tie":false,"gate_int8lin_max_abs_dp":0.003022134304046631},{"id":"semif_3e75f2b84e8e08e3d3f2","q":0,"set":"random","source":"semif_authored144","row_len":53,"near_tie":false,"gate_int8lin_max_abs_dp":0.0007579922676086426},{"id":"semif_d51b05bfb9d6d7f09331","q":0,"set":"random","source":"semif_authored144","row_len":86,"near_tie":false,"gate_int8lin_max_abs_dp":0.006421774625778198},{"id":"semif_980e76145c925747220c","q":0,"set":"random","source":"semif_authored144","row_len":92,"near_tie":false,"gate_int8lin_max_abs_dp":0.007738590240478516},{"id":"semif_37cebc3792e98049439f","q":0,"set":"random","source":"semif_authored144","row_len":85,"near_tie":false,"gate_int8lin_max_abs_dp":0.00022699497640132904},{"id":"semif_5a37d1dfcad20810a76a","q":0,"set":"random","source":"semif_authored144","row_len":79,"near_tie":false,"gate_int8lin_max_abs_dp":0.0011462569236755371},{"id":"tv4_028","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":290,"near_tie":true,"gate_int8lin_max_abs_dp":0.004708617925643921},{"id":"tv4_034","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":77,"near_tie":false,"gate_int8lin_max_abs_dp":0.003359675407409668},{"id":"tv4_050","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":73,"near_tie":false,"gate_int8lin_max_abs_dp":0.004951536655426025},{"id":"tv4_057","q":0,"set":"random","source":"transfer_v4_dev_head","row_len":63,"near_tie":false,"gate_int8lin_max_abs_dp":0.0017376244068145752},{"id":"tv4s_05","q":0,"set":"random","source":"transfer_v4_dev_score","row_len":98,"near_tie":false,"gate_int8lin_max_abs_dp":0.0017088055610656738},{"id":"tv4x_tweet_offensive_03","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":93,"near_tie":false,"gate_int8lin_max_abs_dp":0.0010250210762023926},{"id":"tv4x_tweet_offensive_13","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":43,"near_tie":false,"gate_int8lin_max_abs_dp":0.0010249018669128418},{"id":"tv4x_paws_15","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":74,"near_tie":false,"gate_int8lin_max_abs_dp":0.0033729076385498047},{"id":"tv4x_sciq_01","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":57,"near_tie":false,"gate_int8lin_max_abs_dp":0.0060510337352752686},{"id":"tv4x_sciq_12","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":192,"near_tie":false,"gate_int8lin_max_abs_dp":0.0013521909713745117},{"id":"tv4x_sciq_16","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":335,"near_tie":false,"gate_int8lin_max_abs_dp":0.002252340316772461},{"id":"tv4x_legacy_holdout_03","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":77,"near_tie":false,"gate_int8lin_max_abs_dp":1.4036893844604492e-05},{"id":"tv4x_legacy_holdout_06","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":75,"near_tie":false,"gate_int8lin_max_abs_dp":0.005597501993179321},{"id":"tv4x_composition_holdout_06","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":136,"near_tie":false,"gate_int8lin_max_abs_dp":0.00020045042037963867},{"id":"tv4x_composition_holdout_11","q":0,"set":"random","source":"transfer_v4_dev_sources","row_len":166,"near_tie":false,"gate_int8lin_max_abs_dp":0.0021796822547912598}],"why_rows":"the top 30 rows of the int8lin gate by max|dp| (ties: oracle order), then 30 rows drawn from the other 404 rows by source (proportional, largest remainder, at least one per source; numpy default_rng(20261003).choice without replacement inside each source, sources in sorted order)","random_quota":{"transfer_v4_dev_head":4,"transfer_v4_dev_sources":10,"transfer_v4_dev_score":1,"semif_authored144":9,"own_text":3,"own_json":1,"own_long":2},"seed":20261003,"gate_transcript":{"path":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16.json","sha256":"610053646a4571869615d0c9ac584f8ad8d5749c21f82597118ad4d03b3f5730","result":"FAIL","max_abs_dp":0.04313388466835022,"bundle":"kev_0_8b_decode_int8lin_pf16","aimodelc_tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e"},"configs_fixed":["all_int8","exact","layer_00_fp16","layer_01_fp16","layer_02_fp16","layer_03_fp16","layer_04_fp16","layer_05_fp16","layer_06_fp16","layer_07_fp16","layer_08_fp16","layer_09_fp16","layer_10_fp16","layer_11_fp16","layer_12_fp16","layer_13_fp16","layer_14_fp16","layer_15_fp16","layer_16_fp16","layer_17_fp16","layer_18_fp16","layer_19_fp16","layer_20_fp16","layer_21_fp16","layer_22_fp16","layer_23_fp16","gdn_fp16","attn_fp16","mlp_fp16","proj_qkv_fp16","proj_z_fp16","proj_a_fp16","proj_b_fp16","proj_out_fp16","proj_q_fp16","proj_k_fp16","proj_v_fp16","proj_o_fp16","proj_gate_fp16","proj_up_fp16","proj_down_fp16"],"chunk":16,"tokens":10431,"estimate":"10,431 tokens per config; at round 2's 7.5 ms per token (1 thread, bmm conv) about 78 s per config","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","written_at":"2026-10-03T18:54:14+09:00"},"bisect":{"schema":"kev-int8-bisect/1","rule":{"target":"worst max|dp| over the 60 bisect rows < 0.010 (0.5 x the gate bar 0.02; every option of every question, near-ties included), this instrument","reproduction_check":"before any selection: all_int8's worst max|dp| over the bisect rows (this instrument) >= 0.5 x the int8lin gate transcript's worst over the same rows; otherwise stop and report (the gate's error would not be the int8 weights')","instrument_floor":"exact must reproduce the round-2 P1 run (parity/p1_shard*.jsonl) of every bisect row bit for bit (probabilities), and every process's swap-back re-run likewise","size":"a set's size = the params of the linears it keeps out of int8 (the bytes it costs)","cap":"a candidate keeps at most 50% of the body's linear params out of int8","search":["1. fixed configs: exact, all_int8, layer_NN_fp16 (24), gdn_fp16, attn_fp16, mlp_fp16, proj_<kind>_fp16 (12)","2. layer sets: the 24 layers ranked by their layer_NN_fp16 worst (ascending; ties: the lower index); L_k = the top k, for k = 2, 3, ... while within the cap; at the first k whose L_k meets the target (k = 1: the best single), also every k-subset of the top k+1 layers","3. kind sets: the 12 kinds ranked by their proj_<kind>_fp16 worst (ascending; ties: the order qkv z a b out q k v o gate up down); K_j = the top j, for j = 2, 3, ... while within the cap; at the first j whose K_j meets the target, also every j-subset of the top j+1 kinds; gdn_fp16 / attn_fp16 / mlp_fp16 are kind sets too","4. only if no set of steps 1-3 meets the target: unions L_k + K_j (k, j >= 1, the top-k / top-j sets of 2 and 3) within the cap, in ascending size (ties: smaller k, then smaller j), evaluated in that order until the first that meets the target, at most 12 of them"],"choose":"among the layer and kind sets of steps 1-3 that meet the target, the smallest size (ties: the layer set, then the lexicographically smallest sorted list); if none, the first union of step 4 that meets it; if none, no set (report the numbers: the ship form is the supervisor's call)","then":"the chosen set is exported (export_decoder.py int8mix) and gated on all 434 rows with the gate's own bar, then on the 130 held-out rows (never used to choose)"},"rule_file":"$ZOO_WORK_ROOT/_kev/bisect/rule.json","rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","rule_written_at":"2026-10-03T18:54:14+09:00","rows":[["semif_1105577da4c8dad4609d",0],["semif_a22dab8ba073ff6cdb8d",0],["tv4_023",0],["tv4x_composition_holdout_15",0],["semif_46b7029b9a704138b77a",0],["semif_f3d13cf184f916677def",0],["semif_386e10b1c1ce906473a4",0],["tv4x_composition_holdout_12",0],["tv4x_composition_holdout_14",0],["tv4x_legacy_holdout_13",0],["tv4x_sciq_04",0],["own_j04",0],["tv4x_legacy_holdout_19",0],["semif_7ada97b6f99f4788e5c7",0],["tv4x_sciq_17",0],["tv4_020",0],["semif_2608370d01783735c3ea",0],["own_L03",0],["tv4x_legacy_holdout_15",0],["semif_d7e84e4245a29817da1f",0],["semif_c6a411e7f2bbdf377173",0],["semif_834f3b3e10cc61b33d25",0],["semif_acacd8b46acd25cb954b",0],["semif_599b3092b6fd18312d23",0],["semif_0ebd299739302d8b057b",0],["semif_42384af9ae56beb37ff5",0],["tv4_036",0],["semif_cdfa9eed51fafcf09b72",0],["semif_25172cd6ddc49ab0ca70",0],["semif_d41072dc35d8cd14c056",0],["own_j02",1],["own_L02",2],["own_L03",2],["own_t04",2],["own_m01",4],["own_m01",6],["semif_56c419b12cd37cbe790c",0],["semif_07ee70cfe4b01330ce89",0],["semif_00bb4cc043f7d2b288d4",0],["semif_533d4423d311de82b27d",0],["semif_3e75f2b84e8e08e3d3f2",0],["semif_d51b05bfb9d6d7f09331",0],["semif_980e76145c925747220c",0],["semif_37cebc3792e98049439f",0],["semif_5a37d1dfcad20810a76a",0],["tv4_028",0],["tv4_034",0],["tv4_050",0],["tv4_057",0],["tv4s_05",0],["tv4x_tweet_offensive_03",0],["tv4x_tweet_offensive_13",0],["tv4x_paws_15",0],["tv4x_sciq_01",0],["tv4x_sciq_12",0],["tv4x_sciq_16",0],["tv4x_legacy_holdout_03",0],["tv4x_legacy_holdout_06",0],["tv4x_composition_holdout_06",0],["tv4x_composition_holdout_11",0]],"why_rows":"the top 30 rows of the int8lin gate by max|dp| (ties: oracle order), then 30 rows drawn from the other 404 rows by source (proportional, largest remainder, at least one per source; numpy default_rng(20261003).choice without replacement inside each source, sources in sorted order)","instrument":"fp32 torch on the CPU (parity_decoder_torch.Runner, 1 thread, bmm depthwise conv), S = 16 chunks from fresh zero states, the oracle's row ids, the author's fp32 pointer head; per config the listed linears carry their fp32 checkpoint weight, every other one the exporter's int8 weight (quantize_pytorch_model on the fp16 model, dequantized by the finalized module)","script":{"path":"conversion/kev/int8_bisect_torch.py","sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2"},"dump":{"meta":"$ZOO_WORK_ROOT/_kev/bisect/dump/int8_clipping.json","sha256":"5d9e8a8b694eed4c7a3eaa0cf9e56461816a8212c23a86aa8f34cfbe052bf22b"},"reproduction_check":{"gate_worst":0.04313388466835022,"torch_all_int8_worst":0.04200497269630432,"ratio":0.9738277231293697,"reproduced":true,"pearson_per_row":0.9723470816657634,"rows_above_bar_gate":10,"rows_above_bar_torch":9,"per_row":[{"row":"semif_1105577da4c8dad4609d:q0","gate":0.04313388466835022,"torch":0.04200497269630432},{"row":"semif_a22dab8ba073ff6cdb8d:q0","gate":0.04081737995147705,"torch":0.039111047983169556},{"row":"tv4_023:q0","gate":0.026517271995544434,"torch":0.027171224355697632},{"row":"tv4x_composition_holdout_15:q0","gate":0.026461392641067505,"torch":0.030447542667388916},{"row":"semif_46b7029b9a704138b77a:q0","gate":0.026026636362075806,"torch":0.016141802072525024},{"row":"semif_f3d13cf184f916677def:q0","gate":0.02583444118499756,"torch":0.026892870664596558},{"row":"semif_386e10b1c1ce906473a4:q0","gate":0.024856865406036377,"torch":0.024194002151489258},{"row":"tv4x_composition_holdout_12:q0","gate":0.024287700653076172,"torch":0.026147067546844482},{"row":"tv4x_composition_holdout_14:q0","gate":0.024287700653076172,"torch":0.026147067546844482},{"row":"tv4x_legacy_holdout_13:q0","gate":0.02400749921798706,"torch":0.02320876717567444},{"row":"tv4x_sciq_04:q0","gate":0.019666969776153564,"torch":0.019648849964141846},{"row":"own_j04:q0","gate":0.018736541271209717,"torch":0.016225695610046387},{"row":"tv4x_legacy_holdout_19:q0","gate":0.0182955265045166,"torch":0.016691535711288452},{"row":"semif_7ada97b6f99f4788e5c7:q0","gate":0.018214523792266846,"torch":0.013745903968811035},{"row":"tv4x_sciq_17:q0","gate":0.01787745952606201,"torch":0.01806318759918213},{"row":"tv4_020:q0","gate":0.017518997192382812,"torch":0.010664403438568115},{"row":"semif_2608370d01783735c3ea:q0","gate":0.01745012402534485,"torch":0.014944136142730713},{"row":"own_L03:q0","gate":0.016770899295806885,"torch":0.013764619827270508},{"row":"tv4x_legacy_holdout_15:q0","gate":0.016536161303520203,"torch":0.013957247138023376},{"row":"semif_d7e84e4245a29817da1f:q0","gate":0.016526520252227783,"torch":0.010027289390563965},{"row":"semif_c6a411e7f2bbdf377173:q0","gate":0.015746235847473145,"torch":0.012790799140930176},{"row":"semif_834f3b3e10cc61b33d25:q0","gate":0.01570206880569458,"torch":0.014819502830505371},{"row":"semif_acacd8b46acd25cb954b:q0","gate":0.015568524599075317,"torch":0.014681726694107056},{"row":"semif_599b3092b6fd18312d23:q0","gate":0.01459193229675293,"torch":0.013873815536499023},{"row":"semif_0ebd299739302d8b057b:q0","gate":0.014297336339950562,"torch":0.01130858063697815},{"row":"semif_42384af9ae56beb37ff5:q0","gate":0.01421266794204712,"torch":0.009709745645523071},{"row":"tv4_036:q0","gate":0.014157116413116455,"torch":0.01094365119934082},{"row":"semif_cdfa9eed51fafcf09b72:q0","gate":0.014052867889404297,"torch":0.014418542385101318},{"row":"semif_25172cd6ddc49ab0ca70:q0","gate":0.013986349105834961,"torch":0.009421706199645996},{"row":"semif_d41072dc35d8cd14c056:q0","gate":0.013136208057403564,"torch":0.011030912399291992},{"row":"own_j02:q1","gate":0.003012537956237793,"torch":0.0029041171073913574},{"row":"own_L02:q2","gate":0.0012988448143005371,"torch":0.0005309581756591797},{"row":"own_L03:q2","gate":0.0030238404870033264,"torch":0.0007517337799072266},{"row":"own_t04:q2","gate":0.0014303624629974365,"torch":0.002186417579650879},{"row":"own_m01:q4","gate":0.00023170560598373413,"torch":0.00023277290165424347},{"row":"own_m01:q6","gate":0.0049925148487091064,"torch":0.006142795085906982},{"row":"semif_56c419b12cd37cbe790c:q0","gate":0.0036872774362564087,"torch":0.003954902291297913},{"row":"semif_07ee70cfe4b01330ce89:q0","gate":0.00262296199798584,"torch":0.004449069499969482},{"row":"semif_00bb4cc043f7d2b288d4:q0","gate":0.00046466849744319916,"torch":0.00036193057894706726},{"row":"semif_533d4423d311de82b27d:q0","gate":0.003022134304046631,"torch":0.0038840770721435547},{"row":"semif_3e75f2b84e8e08e3d3f2:q0","gate":0.0007579922676086426,"torch":0.00027485191822052},{"row":"semif_d51b05bfb9d6d7f09331:q0","gate":0.006421774625778198,"torch":0.005586534738540649},{"row":"semif_980e76145c925747220c:q0","gate":0.007738590240478516,"torch":0.003861844539642334},{"row":"semif_37cebc3792e98049439f:q0","gate":0.00022699497640132904,"torch":0.00031411275267601013},{"row":"semif_5a37d1dfcad20810a76a:q0","gate":0.0011462569236755371,"torch":0.002461671829223633},{"row":"tv4_028:q0","gate":0.004708617925643921,"torch":0.00456923246383667},{"row":"tv4_034:q0","gate":0.003359675407409668,"torch":0.003295600414276123},{"row":"tv4_050:q0","gate":0.004951536655426025,"torch":0.0049651265144348145},{"row":"tv4_057:q0","gate":0.0017376244068145752,"torch":0.0014074444770812988},{"row":"tv4s_05:q0","gate":0.0017088055610656738,"torch":0.002974092960357666},{"row":"tv4x_tweet_offensive_03:q0","gate":0.0010250210762023926,"torch":0.0008094310760498047},{"row":"tv4x_tweet_offensive_13:q0","gate":0.0010249018669128418,"torch":0.0010031163692474365},{"row":"tv4x_paws_15:q0","gate":0.0033729076385498047,"torch":0.005306124687194824},{"row":"tv4x_sciq_01:q0","gate":0.0060510337352752686,"torch":0.00591963529586792},{"row":"tv4x_sciq_12:q0","gate":0.0013521909713745117,"torch":0.00038683414459228516},{"row":"tv4x_sciq_16:q0","gate":0.002252340316772461,"torch":0.0009837150573730469},{"row":"tv4x_legacy_holdout_03:q0","gate":1.4036893844604492e-05,"torch":0.0035926103591918945},{"row":"tv4x_legacy_holdout_06:q0","gate":0.005597501993179321,"torch":0.0033920705318450928},{"row":"tv4x_composition_holdout_06:q0","gate":0.00020045042037963867,"torch":0.0016388297080993652},{"row":"tv4x_composition_holdout_11:q0","gate":0.0021796822547912598,"torch":0.0021622776985168457}]},"instrument_floor":{"worst_max_abs_dp":3.4570693969726562e-06,"bit_equal_p1_rows":60,"rows":60},"processes":[{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_a.json","threads":1,"load_seconds":2.322464917000616,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:17:56+09:00","conv":{"fast_calls":85662,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:17:56+09:00"},"configs":["all_int8","exact","layer_00_fp16","layer_06_fp16","layer_12_fp16","layer_18_fp16","proj_qkv_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_b.json","threads":1,"load_seconds":2.322464958007913,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:17:54+09:00","conv":{"fast_calls":85662,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:17:54+09:00"},"configs":["layer_01_fp16","layer_07_fp16","layer_13_fp16","layer_19_fp16","gdn_fp16","proj_z_fp16","proj_down_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_c.json","threads":1,"load_seconds":2.3069650840188842,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:17:52+09:00","conv":{"fast_calls":85662,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:17:52+09:00"},"configs":["layer_02_fp16","layer_08_fp16","layer_14_fp16","layer_20_fp16","attn_fp16","proj_a_fp16","proj_up_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_d.json","threads":1,"load_seconds":2.3068159589893185,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:17:53+09:00","conv":{"fast_calls":85662,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:17:53+09:00"},"configs":["layer_03_fp16","layer_09_fp16","layer_15_fp16","layer_21_fp16","mlp_fp16","proj_b_fp16","proj_gate_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_e.json","threads":1,"load_seconds":2.3009337909752503,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:14:14+09:00","conv":{"fast_calls":73440,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:14:14+09:00"},"configs":["layer_04_fp16","layer_10_fp16","layer_16_fp16","layer_22_fp16","proj_out_fp16","proj_q_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_f.json","threads":1,"load_seconds":2.305324917018879,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"382f71e22fa87ab64266921fb2039ab28ebe0f92ecdb221f41054818a397bbac","started":"2026-10-03T18:55:29+09:00","finished":"2026-10-03T19:17:55+09:00","conv":{"fast_calls":85662,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:17:55+09:00"},"configs":["layer_05_fp16","layer_11_fp16","layer_17_fp16","layer_23_fp16","proj_k_fp16","proj_v_fp16","proj_o_fp16"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_g.json","threads":1,"load_seconds":5.048638333013514,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:15:22+09:00","finished":"2026-10-03T19:26:36+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:26:36+09:00"},"configs":["set_L01_L03","set_L01_L02_L03_L04_L08_L09","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L23"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_h.json","threads":1,"load_seconds":4.851315750012873,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:15:22+09:00","finished":"2026-10-03T19:26:36+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:26:36+09:00"},"configs":["set_L01_L03_L08","set_L01_L02_L03_L04_L06_L08_L09","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L23"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_i.json","threads":1,"load_seconds":5.0743250000232365,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:15:22+09:00","finished":"2026-10-03T19:26:35+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:26:35+09:00"},"configs":["set_L01_L03_L08_L09","set_L01_L02_L03_L04_L06_L08_L09_L12","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L18_L23"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_j.json","threads":1,"load_seconds":5.012542542011943,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:15:22+09:00","finished":"2026-10-03T19:26:34+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:26:34+09:00"},"configs":["set_L01_L03_L04_L08_L09","set_L01_L02_L03_L04_L05_L06_L08_L09_L12","set_L00_L01_L02_L03_L04_L05_L06_L07_L08_L09_L10_L11"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_k.json","threads":1,"load_seconds":3.389969875017414,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:18:33+09:00","finished":"2026-10-03T19:22:17+09:00","conv":{"fast_calls":12330,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:22:17+09:00"},"configs":["set_L01_L03_L04_L08_L09__K_up"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_l.json","threads":1,"load_seconds":3.541908333019819,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:18:33+09:00","finished":"2026-10-03T19:29:10+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:29:10+09:00"},"configs":["set_K_up_down","set_L03__K_up","set_L03__K_up_down"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_m.json","threads":1,"load_seconds":3.418304040998919,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:18:33+09:00","finished":"2026-10-03T19:29:09+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:29:09+09:00"},"configs":["set_K_out_up_down","set_L01_L03__K_up","set_L01_L02_L03_L04_L08_L09__K_up"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_n.json","threads":1,"load_seconds":3.5387517500203103,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:18:33+09:00","finished":"2026-10-03T19:29:09+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:29:09+09:00"},"configs":["set_L01_L03_L08__K_up","set_L01_L03__K_up_down","set_L01_L02_L03_L04_L06_L08_L09__K_up"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_o.json","threads":1,"load_seconds":3.551751291990513,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:18:33+09:00","finished":"2026-10-03T19:29:10+09:00","conv":{"fast_calls":36774,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:29:10+09:00"},"configs":["set_L01_L03_L08_L09__K_up","set_L01_L03_L08__K_up_down","set_L03__K_out_up_down"]},{"part":"$ZOO_WORK_ROOT/_kev/bisect/parts/part_p.json","threads":1,"load_seconds":3.631275416992139,"rule_sha256":"77e406d48f27676656b0fe8dfa570b44939ee12db69cc44f819d7d3532d76fcf","script_sha256":"9984507e043c3bda4747dc95ae67d7e44eaab1f7b32a9a962f6c8171ceae85d2","started":"2026-10-03T19:22:25+09:00","finished":"2026-10-03T19:29:14+09:00","conv":{"fast_calls":24552,"checked_vs_F_conv1d":18,"mismatched":0,"max_abs_diff":0.0},"swap_back_check":{"row":"semif_1105577da4c8dad4609d:q0","bit_equal_p1":true,"max_abs_dp_vs_oracle":3.5762786865234375e-07,"at":"2026-10-03T19:29:14+09:00"},"configs":["set_K_out_o_up_down","set_L01_L02_L03_L04_L06_L08_L09_L12__K_up"]}],"plan":{"layer_ranking":[3,1,8,9,4,2,6,12,5,23,15,18,20,17,16,22,19,21,0,10,13,14,7,11],"kind_ranking":["up","down","out","o","z","a","v","k","b","qkv","q","gate"],"body_linear_params":497614848,"cap_params":248807424.0,"next":[],"layers":{"seq":[{"k":1,"set":"layer_03_fp16","size":18350080},{"k":2,"set":"set_L01_L03","size":39878656},{"k":3,"set":"set_L01_L03_L08","size":61407232},{"k":4,"set":"set_L01_L03_L08_L09","size":82935808},{"k":5,"set":"set_L01_L03_L04_L08_L09","size":104464384},{"k":6,"set":"set_L01_L02_L03_L04_L08_L09","size":125992960},{"k":7,"set":"set_L01_L02_L03_L04_L06_L08_L09","size":147521536},{"k":8,"set":"set_L01_L02_L03_L04_L06_L08_L09_L12","size":169050112},{"k":9,"set":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12","size":190578688},{"k":10,"set":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L23","size":208928768},{"k":11,"set":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L23","size":227278848},{"k":12,"set":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L18_L23","size":248807424}],"next":[],"first_meeting_k":null,"exhausted":true},"kinds":{"seq":[{"k":1,"set":"proj_up_fp16","size":88080384},{"k":2,"set":"set_K_up_down","size":176160768},{"k":3,"set":"set_K_out_up_down","size":213909504},{"k":4,"set":"set_K_out_o_up_down","size":226492416}],"next":[],"first_meeting_k":null,"exhausted":true},"rule_search_sets":["attn_fp16","gdn_fp16","layer_00_fp16","layer_01_fp16","layer_02_fp16","layer_03_fp16","layer_04_fp16","layer_05_fp16","layer_06_fp16","layer_07_fp16","layer_08_fp16","layer_09_fp16","layer_10_fp16","layer_11_fp16","layer_12_fp16","layer_13_fp16","layer_14_fp16","layer_15_fp16","layer_16_fp16","layer_17_fp16","layer_18_fp16","layer_19_fp16","layer_20_fp16","layer_21_fp16","layer_22_fp16","layer_23_fp16","mlp_fp16","proj_a_fp16","proj_b_fp16","proj_down_fp16","proj_gate_fp16","proj_k_fp16","proj_o_fp16","proj_out_fp16","proj_q_fp16","proj_qkv_fp16","proj_up_fp16","proj_v_fp16","proj_z_fp16","set_K_out_o_up_down","set_K_out_up_down","set_K_up_down","set_L01_L02_L03_L04_L05_L06_L08_L09_L12","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L18_L23","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L23","set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L23","set_L01_L02_L03_L04_L06_L08_L09","set_L01_L02_L03_L04_L06_L08_L09_L12","set_L01_L02_L03_L04_L08_L09","set_L01_L03","set_L01_L03_L04_L08_L09","set_L01_L03_L08","set_L01_L03_L08_L09"],"meeting_steps_1_3":[],"unions":[{"k":1,"j":1,"set":"set_L03__K_up","size":102760448},{"k":2,"j":1,"set":"set_L01_L03__K_up","size":120619008},{"k":3,"j":1,"set":"set_L01_L03_L08__K_up","size":138477568},{"k":4,"j":1,"set":"set_L01_L03_L08_L09__K_up","size":156336128},{"k":5,"j":1,"set":"set_L01_L03_L04_L08_L09__K_up","size":174194688},{"k":1,"j":2,"set":"set_L03__K_up_down","size":187170816},{"k":6,"j":1,"set":"set_L01_L02_L03_L04_L08_L09__K_up","size":192053248},{"k":2,"j":2,"set":"set_L01_L03__K_up_down","size":201359360},{"k":7,"j":1,"set":"set_L01_L02_L03_L04_L06_L08_L09__K_up","size":209911808},{"k":3,"j":2,"set":"set_L01_L03_L08__K_up_down","size":215547904},{"k":1,"j":3,"set":"set_L03__K_out_up_down","size":224919552},{"k":8,"j":1,"set":"set_L01_L02_L03_L04_L06_L08_L09_L12__K_up","size":227770368}],"stage":"none","chosen":null},"chosen":null,"table":[{"config":"all_int8","layers":[],"kinds":[],"fp16_params":0,"fp16_fraction":0.0,"worst_max_abs_dp":0.04200497269630432,"worst_top30":0.04200497269630432,"worst_random30":0.006142795085906982,"mean_of_row_mean_abs_dp":0.00756838682702639,"mean_of_row_mean_abs_dp_random30":0.0019936240795585844,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":60,"meets_target":false,"seconds":128.1759374999965,"in_rule_search":true},{"config":"exact","layers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23],"kinds":["qkv","z","a","b","out","q","k","v","o","gate","up","down"],"fp16_params":497614848,"fp16_fraction":1.0,"worst_max_abs_dp":3.4570693969726562e-06,"worst_top30":3.4570693969726562e-06,"worst_random30":1.3113021850585938e-06,"mean_of_row_mean_abs_dp":5.437889032893711e-07,"mean_of_row_mean_abs_dp_random30":3.6691263731982967e-07,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":60,"meets_target":true,"seconds":126.05388329099515,"in_rule_search":true},{"config":"attn_fp16","layers":[],"kinds":["q","k","v","o"],"fp16_params":44040192,"fp16_fraction":0.08850256815487949,"worst_max_abs_dp":0.038986384868621826,"worst_top30":0.038986384868621826,"worst_random30":0.007090866565704346,"mean_of_row_mean_abs_dp":0.006371087520124598,"mean_of_row_mean_abs_dp_random30":0.0017663144480644,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":42,"meets_target":false,"seconds":229.3899149580102,"in_rule_search":true},{"config":"gdn_fp16","layers":[],"kinds":["qkv","z","a","b","out"],"fp16_params":189333504,"fp16_fraction":0.38048202291584354,"worst_max_abs_dp":0.03588011860847473,"worst_top30":0.03588011860847473,"worst_random30":0.005678236484527588,"mean_of_row_mean_abs_dp":0.003978107683360576,"mean_of_row_mean_abs_dp_random30":0.001153851918772691,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":52,"meets_target":false,"seconds":228.22680691699497,"in_rule_search":true},{"config":"layer_00_fp16","layers":[0],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.0420394241809845,"worst_top30":0.0420394241809845,"worst_random30":0.007158875465393066,"mean_of_row_mean_abs_dp":0.006223095319647755,"mean_of_row_mean_abs_dp_random30":0.001773483766656783,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":38,"meets_target":false,"seconds":205.4471021249774,"in_rule_search":true},{"config":"layer_01_fp16","layers":[1],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.037010788917541504,"worst_top30":0.037010788917541504,"worst_random30":0.006377100944519043,"mean_of_row_mean_abs_dp":0.006792243646406051,"mean_of_row_mean_abs_dp_random30":0.001921760515930752,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":41,"meets_target":false,"seconds":126.63871725002537,"in_rule_search":true},{"config":"layer_02_fp16","layers":[2],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04087334871292114,"worst_top30":0.04087334871292114,"worst_random30":0.006198525428771973,"mean_of_row_mean_abs_dp":0.006853713290507181,"mean_of_row_mean_abs_dp_random30":0.0018203739542514085,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":42,"meets_target":false,"seconds":126.85620041700895,"in_rule_search":true},{"config":"layer_03_fp16","layers":[3],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.03599494695663452,"worst_top30":0.03599494695663452,"worst_random30":0.0065337419509887695,"mean_of_row_mean_abs_dp":0.007035507009519885,"mean_of_row_mean_abs_dp_random30":0.0019513470405298802,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":39,"meets_target":false,"seconds":126.76952974998858,"in_rule_search":true},{"config":"layer_04_fp16","layers":[4],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.040255963802337646,"worst_top30":0.040255963802337646,"worst_random30":0.005676209926605225,"mean_of_row_mean_abs_dp":0.007258782786731091,"mean_of_row_mean_abs_dp_random30":0.0019127048189855284,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":35,"meets_target":false,"seconds":126.03203416697215,"in_rule_search":true},{"config":"layer_05_fp16","layers":[5],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04173511266708374,"worst_top30":0.04173511266708374,"worst_random30":0.008000552654266357,"mean_of_row_mean_abs_dp":0.007302109721220202,"mean_of_row_mean_abs_dp_random30":0.00190140409136398,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":38,"meets_target":false,"seconds":127.64735924999695,"in_rule_search":true},{"config":"layer_06_fp16","layers":[6],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04147791862487793,"worst_top30":0.04147791862487793,"worst_random30":0.0092373788356781,"mean_of_row_mean_abs_dp":0.007306201810327669,"mean_of_row_mean_abs_dp_random30":0.0021835078071388934,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":33,"meets_target":false,"seconds":221.8788394999865,"in_rule_search":true},{"config":"layer_07_fp16","layers":[7],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.04297250509262085,"worst_top30":0.04297250509262085,"worst_random30":0.007170915603637695,"mean_of_row_mean_abs_dp":0.006851477265202751,"mean_of_row_mean_abs_dp_random30":0.0018498598649683928,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":36,"meets_target":false,"seconds":126.46683029201813,"in_rule_search":true},{"config":"layer_08_fp16","layers":[8],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.039775848388671875,"worst_top30":0.039775848388671875,"worst_random30":0.008383989334106445,"mean_of_row_mean_abs_dp":0.006776957744215097,"mean_of_row_mean_abs_dp_random30":0.0018663344258028599,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":39,"meets_target":false,"seconds":124.97607512501418,"in_rule_search":true},{"config":"layer_09_fp16","layers":[9],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.03989619016647339,"worst_top30":0.03989619016647339,"worst_random30":0.0066946446895599365,"mean_of_row_mean_abs_dp":0.007235660237104942,"mean_of_row_mean_abs_dp_random30":0.0022226650681760577,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":29,"meets_target":false,"seconds":124.71683854202274,"in_rule_search":true},{"config":"layer_10_fp16","layers":[10],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04209446907043457,"worst_top30":0.04209446907043457,"worst_random30":0.007410407066345215,"mean_of_row_mean_abs_dp":0.0071556266837029,"mean_of_row_mean_abs_dp_random30":0.001965766335423622,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":35,"meets_target":false,"seconds":125.59593425001367,"in_rule_search":true},{"config":"layer_11_fp16","layers":[11],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.0443778932094574,"worst_top30":0.0443778932094574,"worst_random30":0.007427513599395752,"mean_of_row_mean_abs_dp":0.0071738071105856855,"mean_of_row_mean_abs_dp_random30":0.001940636874900924,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":35,"meets_target":false,"seconds":124.88982379200752,"in_rule_search":true},{"config":"layer_12_fp16","layers":[12],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04162922501564026,"worst_top30":0.04162922501564026,"worst_random30":0.006324589252471924,"mean_of_row_mean_abs_dp":0.007568677098283337,"mean_of_row_mean_abs_dp_random30":0.0019947395375412374,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":30,"meets_target":false,"seconds":228.50496708301944,"in_rule_search":true},{"config":"layer_13_fp16","layers":[13],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.042102664709091187,"worst_top30":0.042102664709091187,"worst_random30":0.006359755992889404,"mean_of_row_mean_abs_dp":0.0075539088898545345,"mean_of_row_mean_abs_dp_random30":0.0019404191239219574,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":29,"meets_target":false,"seconds":204.1163794999884,"in_rule_search":true},{"config":"layer_14_fp16","layers":[14],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04212489724159241,"worst_top30":0.04212489724159241,"worst_random30":0.0059725940227508545,"mean_of_row_mean_abs_dp":0.007581123698037118,"mean_of_row_mean_abs_dp_random30":0.00198386724676109,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":28,"meets_target":false,"seconds":201.48478954200982,"in_rule_search":true},{"config":"layer_15_fp16","layers":[15],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.04188081622123718,"worst_top30":0.04188081622123718,"worst_random30":0.005954742431640625,"mean_of_row_mean_abs_dp":0.007553124144518127,"mean_of_row_mean_abs_dp_random30":0.0019599984891505704,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":35,"meets_target":false,"seconds":202.47977812500903,"in_rule_search":true},{"config":"layer_16_fp16","layers":[16],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04198223352432251,"worst_top30":0.04198223352432251,"worst_random30":0.006070524454116821,"mean_of_row_mean_abs_dp":0.007562959320946699,"mean_of_row_mean_abs_dp_random30":0.001986659737303853,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":36,"meets_target":false,"seconds":202.29436454200186,"in_rule_search":true},{"config":"layer_17_fp16","layers":[17],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.041974663734436035,"worst_top30":0.041974663734436035,"worst_random30":0.006207764148712158,"mean_of_row_mean_abs_dp":0.007565615939286848,"mean_of_row_mean_abs_dp_random30":0.0019968349590069718,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":34,"meets_target":false,"seconds":203.48503924999386,"in_rule_search":true},{"config":"layer_18_fp16","layers":[18],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04195845127105713,"worst_top30":0.04195845127105713,"worst_random30":0.006097853183746338,"mean_of_row_mean_abs_dp":0.007576041523781088,"mean_of_row_mean_abs_dp_random30":0.0019936825873123273,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":25,"meets_target":false,"seconds":218.05861949999235,"in_rule_search":true},{"config":"layer_19_fp16","layers":[19],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.04201266169548035,"worst_top30":0.04201266169548035,"worst_random30":0.006098628044128418,"mean_of_row_mean_abs_dp":0.007581064535770565,"mean_of_row_mean_abs_dp_random30":0.001991619637960361,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":24,"meets_target":false,"seconds":221.61396312501165,"in_rule_search":true},{"config":"layer_20_fp16","layers":[20],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04197445511817932,"worst_top30":0.04197445511817932,"worst_random30":0.006128191947937012,"mean_of_row_mean_abs_dp":0.007572263142921859,"mean_of_row_mean_abs_dp_random30":0.0019894702515254417,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":31,"meets_target":false,"seconds":221.28106870802003,"in_rule_search":true},{"config":"layer_21_fp16","layers":[21],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04202812910079956,"worst_top30":0.04202812910079956,"worst_random30":0.006158292293548584,"mean_of_row_mean_abs_dp":0.007566178242106819,"mean_of_row_mean_abs_dp_random30":0.001992587421813773,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":31,"meets_target":false,"seconds":222.2626408750075,"in_rule_search":true},{"config":"layer_22_fp16","layers":[22],"kinds":[],"fp16_params":21528576,"fp16_fraction":0.043263532200711184,"worst_max_abs_dp":0.04199141263961792,"worst_top30":0.04199141263961792,"worst_random30":0.006136834621429443,"mean_of_row_mean_abs_dp":0.0075619155049531,"mean_of_row_mean_abs_dp_random30":0.001991177601222363,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":36,"meets_target":false,"seconds":221.9284342500032,"in_rule_search":true},{"config":"layer_23_fp16","layers":[23],"kinds":[],"fp16_params":18350080,"fp16_fraction":0.036876070064533126,"worst_max_abs_dp":0.04179176688194275,"worst_top30":0.04179176688194275,"worst_random30":0.006297081708908081,"mean_of_row_mean_abs_dp":0.007540275058191683,"mean_of_row_mean_abs_dp_random30":0.001987522581799163,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":38,"meets_target":false,"seconds":222.10541516699595,"in_rule_search":true},{"config":"mlp_fp16","layers":[],"kinds":["gate","up","down"],"fp16_params":264241152,"fp16_fraction":0.531015408929277,"worst_max_abs_dp":0.030813872814178467,"worst_top30":0.030813872814178467,"worst_random30":0.008037060499191284,"mean_of_row_mean_abs_dp":0.005494893321560489,"mean_of_row_mean_abs_dp_random30":0.0019204262307741574,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":45,"meets_target":false,"seconds":228.88682149999659,"in_rule_search":true},{"config":"proj_a_fp16","layers":[],"kinds":["a"],"fp16_params":294912,"fp16_fraction":0.0005926511260371395,"worst_max_abs_dp":0.04090067744255066,"worst_top30":0.04090067744255066,"worst_random30":0.006015509366989136,"mean_of_row_mean_abs_dp":0.0074820101920825745,"mean_of_row_mean_abs_dp_random30":0.002001557390516003,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":35,"meets_target":false,"seconds":218.17107954100356,"in_rule_search":true},{"config":"proj_b_fp16","layers":[],"kinds":["b"],"fp16_params":294912,"fp16_fraction":0.0005926511260371395,"worst_max_abs_dp":0.042357563972473145,"worst_top30":0.042357563972473145,"worst_random30":0.006322771310806274,"mean_of_row_mean_abs_dp":0.007582917575362241,"mean_of_row_mean_abs_dp_random30":0.0020348375249240134,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":29,"meets_target":false,"seconds":218.7027008330042,"in_rule_search":true},{"config":"proj_down_fp16","layers":[],"kinds":["down"],"fp16_params":88080384,"fp16_fraction":0.17700513630975898,"worst_max_abs_dp":0.03599792718887329,"worst_top30":0.03599792718887329,"worst_random30":0.007972627878189087,"mean_of_row_mean_abs_dp":0.006839977918813626,"mean_of_row_mean_abs_dp_random30":0.0020269396766606305,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":37,"meets_target":false,"seconds":218.17271216597874,"in_rule_search":true},{"config":"proj_gate_fp16","layers":[],"kinds":["gate"],"fp16_params":88080384,"fp16_fraction":0.17700513630975898,"worst_max_abs_dp":0.04419216513633728,"worst_top30":0.04419216513633728,"worst_random30":0.007690101861953735,"mean_of_row_mean_abs_dp":0.007541425392911252,"mean_of_row_mean_abs_dp_random30":0.0021799121000286607,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":34,"meets_target":false,"seconds":219.02879762501107,"in_rule_search":true},{"config":"proj_k_fp16","layers":[],"kinds":["k"],"fp16_params":3145728,"fp16_fraction":0.006321612011062821,"worst_max_abs_dp":0.041714996099472046,"worst_top30":0.041714996099472046,"worst_random30":0.006091445684432983,"mean_of_row_mean_abs_dp":0.0073561636498197915,"mean_of_row_mean_abs_dp_random30":0.0018863049258167545,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":37,"meets_target":false,"seconds":228.92390487500234,"in_rule_search":true},{"config":"proj_o_fp16","layers":[],"kinds":["o"],"fp16_params":12582912,"fp16_fraction":0.025286448044251283,"worst_max_abs_dp":0.03897479176521301,"worst_top30":0.03897479176521301,"worst_random30":0.006365299224853516,"mean_of_row_mean_abs_dp":0.006933213919142468,"mean_of_row_mean_abs_dp_random30":0.0018104395441090068,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":40,"meets_target":false,"seconds":218.7406123339897,"in_rule_search":true},{"config":"proj_out_fp16","layers":[],"kinds":["out"],"fp16_params":37748736,"fp16_fraction":0.07585934413275386,"worst_max_abs_dp":0.03739839792251587,"worst_top30":0.03739839792251587,"worst_random30":0.009765267372131348,"mean_of_row_mean_abs_dp":0.0057090379776329634,"mean_of_row_mean_abs_dp_random30":0.0017361093571202622,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":42,"meets_target":false,"seconds":229.14073658298003,"in_rule_search":true},{"config":"proj_q_fp16","layers":[],"kinds":["q"],"fp16_params":25165824,"fp16_fraction":0.050572896088502566,"worst_max_abs_dp":0.04338905215263367,"worst_top30":0.04338905215263367,"worst_random30":0.007149457931518555,"mean_of_row_mean_abs_dp":0.007281774973186354,"mean_of_row_mean_abs_dp_random30":0.002085924109754463,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":38,"meets_target":false,"seconds":218.22597887500888,"in_rule_search":true},{"config":"proj_qkv_fp16","layers":[],"kinds":["qkv"],"fp16_params":113246208,"fp16_fraction":0.22757803239826155,"worst_max_abs_dp":0.042523086071014404,"worst_top30":0.042523086071014404,"worst_random30":0.006158411502838135,"mean_of_row_mean_abs_dp":0.006901354854926467,"mean_of_row_mean_abs_dp_random30":0.0019282295472092096,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":34,"meets_target":false,"seconds":218.02642895799363,"in_rule_search":true},{"config":"proj_up_fp16","layers":[],"kinds":["up"],"fp16_params":88080384,"fp16_fraction":0.17700513630975898,"worst_max_abs_dp":0.03438436985015869,"worst_top30":0.03438436985015869,"worst_random30":0.005800127983093262,"mean_of_row_mean_abs_dp":0.006294841403400319,"mean_of_row_mean_abs_dp_random30":0.0018336713909068041,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":40,"meets_target":false,"seconds":218.78095387501526,"in_rule_search":true},{"config":"proj_v_fp16","layers":[],"kinds":["v"],"fp16_params":3145728,"fp16_fraction":0.006321612011062821,"worst_max_abs_dp":0.04097825288772583,"worst_top30":0.04097825288772583,"worst_random30":0.005870342254638672,"mean_of_row_mean_abs_dp":0.007402294815983623,"mean_of_row_mean_abs_dp_random30":0.001933541663715409,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":32,"meets_target":false,"seconds":218.27096525000525,"in_rule_search":true},{"config":"proj_z_fp16","layers":[],"kinds":["z"],"fp16_params":37748736,"fp16_fraction":0.07585934413275386,"worst_max_abs_dp":0.04017055034637451,"worst_top30":0.04017055034637451,"worst_random30":0.006214022636413574,"mean_of_row_mean_abs_dp":0.006738179469377631,"mean_of_row_mean_abs_dp_random30":0.0018964263455321392,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":36,"meets_target":false,"seconds":218.12617749997298,"in_rule_search":true},{"config":"set_K_out_o_up_down","layers":[],"kinds":["out","o","up","down"],"fp16_params":226492416,"fp16_fraction":0.4551560647965231,"worst_max_abs_dp":0.014053970575332642,"worst_top30":0.014053970575332642,"worst_random30":0.0043520331382751465,"mean_of_row_mean_abs_dp":0.003161626150055478,"mean_of_row_mean_abs_dp_random30":0.0010718355498587093,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":53,"meets_target":false,"seconds":257.20702387500205,"in_rule_search":true},{"config":"set_K_out_up_down","layers":[],"kinds":["out","up","down"],"fp16_params":213909504,"fp16_fraction":0.4298696167522718,"worst_max_abs_dp":0.01806086301803589,"worst_top30":0.01806086301803589,"worst_random30":0.005434006452560425,"mean_of_row_mean_abs_dp":0.0037061840903738308,"mean_of_row_mean_abs_dp_random30":0.001294774207700458,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":52,"meets_target":false,"seconds":222.79026666699792,"in_rule_search":true},{"config":"set_K_up_down","layers":[],"kinds":["up","down"],"fp16_params":176160768,"fp16_fraction":0.35401027261951795,"worst_max_abs_dp":0.028537988662719727,"worst_top30":0.028537988662719727,"worst_random30":0.006544053554534912,"mean_of_row_mean_abs_dp":0.005623602278582339,"mean_of_row_mean_abs_dp_random30":0.0019393934169784188,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":41,"meets_target":false,"seconds":222.69342045899248,"in_rule_search":true},{"config":"set_L00_L01_L02_L03_L04_L05_L06_L07_L08_L09_L10_L11","layers":[0,1,2,3,4,5,6,7,8,9,10,11],"kinds":[],"fp16_params":248807424,"fp16_fraction":0.5,"worst_max_abs_dp":0.00460052490234375,"worst_top30":0.00460052490234375,"worst_random30":0.0013185739517211914,"mean_of_row_mean_abs_dp":0.0008129597589787511,"mean_of_row_mean_abs_dp_random30":0.00046005151234567164,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":56,"meets_target":true,"seconds":255.67591674998403,"in_rule_search":false},{"config":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12","layers":[1,2,3,4,5,6,8,9,12],"kinds":[],"fp16_params":190578688,"fp16_fraction":0.38298432767022256,"worst_max_abs_dp":0.017789050936698914,"worst_top30":0.017789050936698914,"worst_random30":0.004630148410797119,"mean_of_row_mean_abs_dp":0.0033890123414393103,"mean_of_row_mean_abs_dp_random30":0.0013665739886669648,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":47,"meets_target":false,"seconds":221.54927470898838,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L18_L23","layers":[1,2,3,4,5,6,8,9,12,15,18,23],"kinds":[],"fp16_params":248807424,"fp16_fraction":0.5,"worst_max_abs_dp":0.017874151468276978,"worst_top30":0.017874151468276978,"worst_random30":0.004502415657043457,"mean_of_row_mean_abs_dp":0.0033582762013086016,"mean_of_row_mean_abs_dp_random30":0.0013401429298230342,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":48,"meets_target":false,"seconds":255.85228962500696,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L23","layers":[1,2,3,4,5,6,8,9,12,15,23],"kinds":[],"fp16_params":227278848,"fp16_fraction":0.45673646779928884,"worst_max_abs_dp":0.01791420578956604,"worst_top30":0.01791420578956604,"worst_random30":0.004547178745269775,"mean_of_row_mean_abs_dp":0.0033478968453386593,"mean_of_row_mean_abs_dp_random30":0.0013356278365891838,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":49,"meets_target":false,"seconds":257.0984012919944,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L23","layers":[1,2,3,4,5,6,8,9,12,23],"kinds":[],"fp16_params":208928768,"fp16_fraction":0.4198603977347557,"worst_max_abs_dp":0.017971426248550415,"worst_top30":0.017971426248550415,"worst_random30":0.004641056060791016,"mean_of_row_mean_abs_dp":0.003354770364239812,"mean_of_row_mean_abs_dp_random30":0.0013473163772788313,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":48,"meets_target":false,"seconds":256.32279149998794,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L06_L08_L09","layers":[1,2,3,4,6,8,9],"kinds":[],"fp16_params":147521536,"fp16_fraction":0.29645726326880023,"worst_max_abs_dp":0.021508246660232544,"worst_top30":0.021508246660232544,"worst_random30":0.006312727928161621,"mean_of_row_mean_abs_dp":0.003879685015676336,"mean_of_row_mean_abs_dp_random30":0.0015651940393985972,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":45,"meets_target":false,"seconds":222.39799125000718,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L06_L08_L09_L12","layers":[1,2,3,4,6,8,9,12],"kinds":[],"fp16_params":169050112,"fp16_fraction":0.3397207954695114,"worst_max_abs_dp":0.021167218685150146,"worst_top30":0.021167218685150146,"worst_random30":0.005643784999847412,"mean_of_row_mean_abs_dp":0.0038070319957720735,"mean_of_row_mean_abs_dp_random30":0.0015441808668482636,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":44,"meets_target":false,"seconds":222.17397395899752,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L06_L08_L09_L12__K_up","layers":[1,2,3,4,6,8,9,12],"kinds":["up"],"fp16_params":227770368,"fp16_fraction":0.4577242196760174,"worst_max_abs_dp":0.02080148458480835,"worst_top30":0.02080148458480835,"worst_random30":0.004785448312759399,"mean_of_row_mean_abs_dp":0.0031743595130845075,"mean_of_row_mean_abs_dp_random30":0.0012926471548982794,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":46,"meets_target":false,"seconds":150.79950641599135,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L06_L08_L09__K_up","layers":[1,2,3,4,6,8,9],"kinds":["up"],"fp16_params":209911808,"fp16_fraction":0.4218359014882128,"worst_max_abs_dp":0.02111712098121643,"worst_top30":0.02111712098121643,"worst_random30":0.005547314882278442,"mean_of_row_mean_abs_dp":0.003266810121325155,"mean_of_row_mean_abs_dp_random30":0.0013356003522252042,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":45,"meets_target":false,"seconds":155.8864904580114,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L08_L09","layers":[1,2,3,4,8,9],"kinds":[],"fp16_params":125992960,"fp16_fraction":0.253193731068089,"worst_max_abs_dp":0.022240281105041504,"worst_top30":0.022240281105041504,"worst_random30":0.008351325988769531,"mean_of_row_mean_abs_dp":0.004424009834312731,"mean_of_row_mean_abs_dp_random30":0.0018231643674274285,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":41,"meets_target":false,"seconds":222.2155937079806,"in_rule_search":true},{"config":"set_L01_L02_L03_L04_L08_L09__K_up","layers":[1,2,3,4,8,9],"kinds":["up"],"fp16_params":192053248,"fp16_fraction":0.3859475833004083,"worst_max_abs_dp":0.019790172576904297,"worst_top30":0.019790172576904297,"worst_random30":0.0052389055490493774,"mean_of_row_mean_abs_dp":0.003925238018079351,"mean_of_row_mean_abs_dp_random30":0.0016647329424611394,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":42,"meets_target":false,"seconds":156.05421550001483,"in_rule_search":true},{"config":"set_L01_L03","layers":[1,3],"kinds":[],"fp16_params":39878656,"fp16_fraction":0.0801396022652443,"worst_max_abs_dp":0.032437294721603394,"worst_top30":0.032437294721603394,"worst_random30":0.008166491985321045,"mean_of_row_mean_abs_dp":0.006240192965035223,"mean_of_row_mean_abs_dp_random30":0.0019004849328969917,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":45,"meets_target":false,"seconds":193.02699195800233,"in_rule_search":true},{"config":"set_L01_L03_L04_L08_L09","layers":[1,3,4,8,9],"kinds":[],"fp16_params":104464384,"fp16_fraction":0.20993019886737785,"worst_max_abs_dp":0.027990341186523438,"worst_top30":0.027990341186523438,"worst_random30":0.007595181465148926,"mean_of_row_mean_abs_dp":0.004847974239641593,"mean_of_row_mean_abs_dp_random30":0.0018877431077675686,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":37,"meets_target":false,"seconds":192.28684341697954,"in_rule_search":true},{"config":"set_L01_L03_L04_L08_L09__K_up","layers":[1,3,4,8,9],"kinds":["up"],"fp16_params":174194688,"fp16_fraction":0.3500592651126037,"worst_max_abs_dp":0.020550429821014404,"worst_top30":0.020550429821014404,"worst_random30":0.006256759166717529,"mean_of_row_mean_abs_dp":0.004073258165994452,"mean_of_row_mean_abs_dp_random30":0.0017381938950469097,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":41,"meets_target":false,"seconds":222.39581545899273,"in_rule_search":true},{"config":"set_L01_L03_L08","layers":[1,3,8],"kinds":[],"fp16_params":61407232,"fp16_fraction":0.12340313446595548,"worst_max_abs_dp":0.031371086835861206,"worst_top30":0.031371086835861206,"worst_random30":0.007818043231964111,"mean_of_row_mean_abs_dp":0.005511130190764864,"mean_of_row_mean_abs_dp_random30":0.0018633860085780422,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":43,"meets_target":false,"seconds":192.71587833299418,"in_rule_search":true},{"config":"set_L01_L03_L08_L09","layers":[1,3,8,9],"kinds":[],"fp16_params":82935808,"fp16_fraction":0.16666666666666666,"worst_max_abs_dp":0.028752028942108154,"worst_top30":0.028752028942108154,"worst_random30":0.007809579372406006,"mean_of_row_mean_abs_dp":0.005207888113283035,"mean_of_row_mean_abs_dp_random30":0.0019250293391653234,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":41,"meets_target":false,"seconds":192.74772216699785,"in_rule_search":true},{"config":"set_L01_L03_L08_L09__K_up","layers":[1,3,8,9],"kinds":["up"],"fp16_params":156336128,"fp16_fraction":0.31417094692479913,"worst_max_abs_dp":0.023656219244003296,"worst_top30":0.023656219244003296,"worst_random30":0.006729006767272949,"mean_of_row_mean_abs_dp":0.004516828031693068,"mean_of_row_mean_abs_dp_random30":0.0018733479703466097,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":40,"meets_target":false,"seconds":223.16445804198156,"in_rule_search":true},{"config":"set_L01_L03_L08__K_up","layers":[1,3,8],"kinds":["up"],"fp16_params":138477568,"fp16_fraction":0.2782826287369946,"worst_max_abs_dp":0.025286227464675903,"worst_top30":0.025286227464675903,"worst_random30":0.0073764026165008545,"mean_of_row_mean_abs_dp":0.004736099319739474,"mean_of_row_mean_abs_dp_random30":0.0017166628786880111,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":44,"meets_target":false,"seconds":222.87768050000886,"in_rule_search":true},{"config":"set_L01_L03_L08__K_up_down","layers":[1,3,8],"kinds":["up","down"],"fp16_params":215547904,"fp16_fraction":0.4331621230080337,"worst_max_abs_dp":0.018236875534057617,"worst_top30":0.018236875534057617,"worst_random30":0.007295131683349609,"mean_of_row_mean_abs_dp":0.003991934094422807,"mean_of_row_mean_abs_dp_random30":0.0016051926339666048,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":47,"meets_target":false,"seconds":256.48233541700756,"in_rule_search":true},{"config":"set_L01_L03__K_up","layers":[1,3],"kinds":["up"],"fp16_params":120619008,"fp16_fraction":0.24239431054919006,"worst_max_abs_dp":0.02643844485282898,"worst_top30":0.02643844485282898,"worst_random30":0.006985992193222046,"mean_of_row_mean_abs_dp":0.0050451034576528605,"mean_of_row_mean_abs_dp_random30":0.0017193501892810066,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":42,"meets_target":false,"seconds":256.24770770798204,"in_rule_search":true},{"config":"set_L01_L03__K_up_down","layers":[1,3],"kinds":["up","down"],"fp16_params":201359360,"fp16_fraction":0.4046490188331358,"worst_max_abs_dp":0.021231919527053833,"worst_top30":0.021231919527053833,"worst_random30":0.007364362478256226,"mean_of_row_mean_abs_dp":0.0043264456300271875,"mean_of_row_mean_abs_dp_random30":0.001679843899586962,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":44,"meets_target":false,"seconds":256.0227040000027,"in_rule_search":true},{"config":"set_L03__K_out_up_down","layers":[3],"kinds":["out","up","down"],"fp16_params":224919552,"fp16_fraction":0.4519952587909917,"worst_max_abs_dp":0.018612265586853027,"worst_top30":0.018612265586853027,"worst_random30":0.005311369895935059,"mean_of_row_mean_abs_dp":0.0032404107921239407,"mean_of_row_mean_abs_dp_random30":0.0013467762438166475,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":50,"meets_target":false,"seconds":155.789031332999,"in_rule_search":true},{"config":"set_L03__K_up","layers":[3],"kinds":["up"],"fp16_params":102760448,"fp16_fraction":0.20650599236138548,"worst_max_abs_dp":0.02965041995048523,"worst_top30":0.02965041995048523,"worst_random30":0.006143331527709961,"mean_of_row_mean_abs_dp":0.005739943670212394,"mean_of_row_mean_abs_dp_random30":0.0017859751901899775,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":40,"meets_target":false,"seconds":256.4957508750085,"in_rule_search":true},{"config":"set_L03__K_up_down","layers":[3],"kinds":["up","down"],"fp16_params":187170816,"fp16_fraction":0.37613591465823787,"worst_max_abs_dp":0.022936761379241943,"worst_top30":0.022936761379241943,"worst_random30":0.00647619366645813,"mean_of_row_mean_abs_dp":0.005030889932014462,"mean_of_row_mean_abs_dp_random30":0.0017295817353038326,"argmax_equal_non_near_tie":true,"rows_not_worse_than_all_int8":43,"meets_target":false,"seconds":156.23032595901168,"in_rule_search":true}],"configs":{"all_int8":{"why":"the export's int8lin body: every linear int8","layers":[],"kinds":[],"fp16_modules":0,"int8_modules":186,"fp16_params":0,"int8_params":497614848,"fp16_fraction":0.0,"modules_swapped":186,"swap_seconds":0.49949633301002905,"worst_max_abs_dp":0.04200497269630432,"mean_of_row_mean_abs_dp":0.00756838682702639,"argmax_equal_non_near_tie":true,"seconds":128.1759374999965,"finished":"2026-10-03T18:57:37+09:00"},"exact":{"why":"no int8: the instrument's floor vs the oracle","layers":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23],"kinds":["qkv","z","a","b","out","q","k","v","o","gate","up","down"],"fp16_modules":186,"int8_modules":0,"fp16_params":497614848,"int8_params":0,"fp16_fraction":1.0,"modules_swapped":186,"swap_seconds":0.04434729099739343,"worst_max_abs_dp":3.4570693969726562e-06,"mean_of_row_mean_abs_dp":5.437889032893711e-07,"argmax_equal_non_near_tie":true,"seconds":126.05388329099515,"finished":"2026-10-03T18:59:43+09:00"},"layer_00_fp16":{"why":"layer 0's linears fp32, every other linear int8","layers":[0],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":178,"swap_seconds":0.07376641599694267,"worst_max_abs_dp":0.0420394241809845,"mean_of_row_mean_abs_dp":0.006223095319647755,"argmax_equal_non_near_tie":true,"seconds":205.4471021249774,"finished":"2026-10-03T19:03:09+09:00"},"layer_06_fp16":{"why":"layer 6's linears fp32, every other linear int8","layers":[6],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.007207124988781288,"worst_max_abs_dp":0.04147791862487793,"mean_of_row_mean_abs_dp":0.007306201810327669,"argmax_equal_non_near_tie":true,"seconds":221.8788394999865,"finished":"2026-10-03T19:06:51+09:00"},"layer_12_fp16":{"why":"layer 12's linears fp32, every other linear int8","layers":[12],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.014587583020329475,"worst_max_abs_dp":0.04162922501564026,"mean_of_row_mean_abs_dp":0.007568677098283337,"argmax_equal_non_near_tie":true,"seconds":228.50496708301944,"finished":"2026-10-03T19:10:39+09:00"},"layer_18_fp16":{"why":"layer 18's linears fp32, every other linear int8","layers":[18],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.006147375010186806,"worst_max_abs_dp":0.04195845127105713,"mean_of_row_mean_abs_dp":0.007576041523781088,"argmax_equal_non_near_tie":true,"seconds":218.05861949999235,"finished":"2026-10-03T19:14:17+09:00"},"proj_qkv_fp16":{"why":"in_proj_qkv fp32 in every layer, the rest int8","layers":[],"kinds":["qkv"],"fp16_modules":18,"int8_modules":168,"fp16_params":113246208,"int8_params":384368640,"fp16_fraction":0.22757803239826155,"modules_swapped":24,"swap_seconds":0.017056499986210838,"worst_max_abs_dp":0.042523086071014404,"mean_of_row_mean_abs_dp":0.006901354854926467,"argmax_equal_non_near_tie":true,"seconds":218.02642895799363,"finished":"2026-10-03T19:17:55+09:00"},"layer_01_fp16":{"why":"layer 1's linears fp32, every other linear int8","layers":[1],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":178,"swap_seconds":0.49943104200065136,"worst_max_abs_dp":0.037010788917541504,"mean_of_row_mean_abs_dp":0.006792243646406051,"argmax_equal_non_near_tie":true,"seconds":126.63871725002537,"finished":"2026-10-03T18:57:36+09:00"},"layer_07_fp16":{"why":"layer 7's linears fp32, every other linear int8","layers":[7],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":15,"swap_seconds":0.020822125021368265,"worst_max_abs_dp":0.04297250509262085,"mean_of_row_mean_abs_dp":0.006851477265202751,"argmax_equal_non_near_tie":true,"seconds":126.46683029201813,"finished":"2026-10-03T18:59:42+09:00"},"layer_13_fp16":{"why":"layer 13's linears fp32, every other linear int8","layers":[13],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":15,"swap_seconds":0.004542167007457465,"worst_max_abs_dp":0.042102664709091187,"mean_of_row_mean_abs_dp":0.0075539088898545345,"argmax_equal_non_near_tie":true,"seconds":204.1163794999884,"finished":"2026-10-03T19:03:06+09:00"},"layer_19_fp16":{"why":"layer 19's linears fp32, every other linear int8","layers":[19],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":15,"swap_seconds":0.010153000010177493,"worst_max_abs_dp":0.04201266169548035,"mean_of_row_mean_abs_dp":0.007581064535770565,"argmax_equal_non_near_tie":true,"seconds":221.61396312501165,"finished":"2026-10-03T19:06:48+09:00"},"gdn_fp16":{"why":"every gdn linear fp32, the rest int8","layers":[],"kinds":["qkv","z","a","b","out"],"fp16_modules":90,"int8_modules":96,"fp16_params":189333504,"int8_params":308281344,"fp16_fraction":0.38048202291584354,"modules_swapped":97,"swap_seconds":0.042974584008334205,"worst_max_abs_dp":0.03588011860847473,"mean_of_row_mean_abs_dp":0.003978107683360576,"argmax_equal_non_near_tie":true,"seconds":228.22680691699497,"finished":"2026-10-03T19:10:36+09:00"},"proj_z_fp16":{"why":"in_proj_z fp32 in every layer, the rest int8","layers":[],"kinds":["z"],"fp16_modules":18,"int8_modules":168,"fp16_params":37748736,"int8_params":459866112,"fp16_fraction":0.07585934413275386,"modules_swapped":72,"swap_seconds":0.03915833399514668,"worst_max_abs_dp":0.04017055034637451,"mean_of_row_mean_abs_dp":0.006738179469377631,"argmax_equal_non_near_tie":true,"seconds":218.12617749997298,"finished":"2026-10-03T19:14:14+09:00"},"proj_down_fp16":{"why":"down_proj fp32 in every layer, the rest int8","layers":[],"kinds":["down"],"fp16_modules":24,"int8_modules":162,"fp16_params":88080384,"int8_params":409534464,"fp16_fraction":0.17700513630975898,"modules_swapped":42,"swap_seconds":0.018990915996255353,"worst_max_abs_dp":0.03599792718887329,"mean_of_row_mean_abs_dp":0.006839977918813626,"argmax_equal_non_near_tie":true,"seconds":218.17271216597874,"finished":"2026-10-03T19:17:52+09:00"},"layer_02_fp16":{"why":"layer 2's linears fp32, every other linear int8","layers":[2],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":178,"swap_seconds":0.4987977090058848,"worst_max_abs_dp":0.04087334871292114,"mean_of_row_mean_abs_dp":0.006853713290507181,"argmax_equal_non_near_tie":true,"seconds":126.85620041700895,"finished":"2026-10-03T18:57:36+09:00"},"layer_08_fp16":{"why":"layer 8's linears fp32, every other linear int8","layers":[8],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.021771416009869426,"worst_max_abs_dp":0.039775848388671875,"mean_of_row_mean_abs_dp":0.006776957744215097,"argmax_equal_non_near_tie":true,"seconds":124.97607512501418,"finished":"2026-10-03T18:59:41+09:00"},"layer_14_fp16":{"why":"layer 14's linears fp32, every other linear int8","layers":[14],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.004768042010255158,"worst_max_abs_dp":0.04212489724159241,"mean_of_row_mean_abs_dp":0.007581123698037118,"argmax_equal_non_near_tie":true,"seconds":201.48478954200982,"finished":"2026-10-03T19:03:02+09:00"},"layer_20_fp16":{"why":"layer 20's linears fp32, every other linear int8","layers":[20],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.008099625003524125,"worst_max_abs_dp":0.04197445511817932,"mean_of_row_mean_abs_dp":0.007572263142921859,"argmax_equal_non_near_tie":true,"seconds":221.28106870802003,"finished":"2026-10-03T19:06:44+09:00"},"attn_fp16":{"why":"every attn linear fp32, the rest int8","layers":[],"kinds":["q","k","v","o"],"fp16_modules":24,"int8_modules":162,"fp16_params":44040192,"int8_params":453574656,"fp16_fraction":0.08850256815487949,"modules_swapped":32,"swap_seconds":0.018131707998691127,"worst_max_abs_dp":0.038986384868621826,"mean_of_row_mean_abs_dp":0.006371087520124598,"argmax_equal_non_near_tie":true,"seconds":229.3899149580102,"finished":"2026-10-03T19:10:33+09:00"},"proj_a_fp16":{"why":"in_proj_a fp32 in every layer, the rest int8","layers":[],"kinds":["a"],"fp16_modules":18,"int8_modules":168,"fp16_params":294912,"int8_params":497319936,"fp16_fraction":0.0005926511260371395,"modules_swapped":42,"swap_seconds":0.008824707998428494,"worst_max_abs_dp":0.04090067744255066,"mean_of_row_mean_abs_dp":0.0074820101920825745,"argmax_equal_non_near_tie":true,"seconds":218.17107954100356,"finished":"2026-10-03T19:14:11+09:00"},"proj_up_fp16":{"why":"up_proj fp32 in every layer, the rest int8","layers":[],"kinds":["up"],"fp16_modules":24,"int8_modules":162,"fp16_params":88080384,"int8_params":409534464,"fp16_fraction":0.17700513630975898,"modules_swapped":42,"swap_seconds":0.014353000005939975,"worst_max_abs_dp":0.03438436985015869,"mean_of_row_mean_abs_dp":0.006294841403400319,"argmax_equal_non_near_tie":true,"seconds":218.78095387501526,"finished":"2026-10-03T19:17:50+09:00"},"layer_03_fp16":{"why":"layer 3's linears fp32, every other linear int8","layers":[3],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":179,"swap_seconds":0.49891233298694715,"worst_max_abs_dp":0.03599494695663452,"mean_of_row_mean_abs_dp":0.007035507009519885,"argmax_equal_non_near_tie":true,"seconds":126.76952974998858,"finished":"2026-10-03T18:57:36+09:00"},"layer_09_fp16":{"why":"layer 9's linears fp32, every other linear int8","layers":[9],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":15,"swap_seconds":0.01710575001197867,"worst_max_abs_dp":0.03989619016647339,"mean_of_row_mean_abs_dp":0.007235660237104942,"argmax_equal_non_near_tie":true,"seconds":124.71683854202274,"finished":"2026-10-03T18:59:41+09:00"},"layer_15_fp16":{"why":"layer 15's linears fp32, every other linear int8","layers":[15],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":15,"swap_seconds":0.0057135420211125165,"worst_max_abs_dp":0.04188081622123718,"mean_of_row_mean_abs_dp":0.007553124144518127,"argmax_equal_non_near_tie":true,"seconds":202.47977812500903,"finished":"2026-10-03T19:03:03+09:00"},"layer_21_fp16":{"why":"layer 21's linears fp32, every other linear int8","layers":[21],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":15,"swap_seconds":0.011025417014025152,"worst_max_abs_dp":0.04202812910079956,"mean_of_row_mean_abs_dp":0.007566178242106819,"argmax_equal_non_near_tie":true,"seconds":222.2626408750075,"finished":"2026-10-03T19:06:45+09:00"},"mlp_fp16":{"why":"every mlp linear fp32, the rest int8","layers":[],"kinds":["gate","up","down"],"fp16_modules":72,"int8_modules":114,"fp16_params":264241152,"int8_params":233373696,"fp16_fraction":0.531015408929277,"modules_swapped":74,"swap_seconds":0.04601712501607835,"worst_max_abs_dp":0.030813872814178467,"mean_of_row_mean_abs_dp":0.005494893321560489,"argmax_equal_non_near_tie":true,"seconds":228.88682149999659,"finished":"2026-10-03T19:10:34+09:00"},"proj_b_fp16":{"why":"in_proj_b fp32 in every layer, the rest int8","layers":[],"kinds":["b"],"fp16_modules":18,"int8_modules":168,"fp16_params":294912,"int8_params":497319936,"fp16_fraction":0.0005926511260371395,"modules_swapped":90,"swap_seconds":0.06355912500293925,"worst_max_abs_dp":0.042357563972473145,"mean_of_row_mean_abs_dp":0.007582917575362241,"argmax_equal_non_near_tie":true,"seconds":218.7027008330042,"finished":"2026-10-03T19:14:13+09:00"},"proj_gate_fp16":{"why":"gate_proj fp32 in every layer, the rest int8","layers":[],"kinds":["gate"],"fp16_modules":24,"int8_modules":162,"fp16_params":88080384,"int8_params":409534464,"fp16_fraction":0.17700513630975898,"modules_swapped":42,"swap_seconds":0.01134137500775978,"worst_max_abs_dp":0.04419216513633728,"mean_of_row_mean_abs_dp":0.007541425392911252,"argmax_equal_non_near_tie":true,"seconds":219.02879762501107,"finished":"2026-10-03T19:17:52+09:00"},"layer_04_fp16":{"why":"layer 4's linears fp32, every other linear int8","layers":[4],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":178,"swap_seconds":0.5022433329722844,"worst_max_abs_dp":0.040255963802337646,"mean_of_row_mean_abs_dp":0.007258782786731091,"argmax_equal_non_near_tie":true,"seconds":126.03203416697215,"finished":"2026-10-03T18:57:35+09:00"},"layer_10_fp16":{"why":"layer 10's linears fp32, every other linear int8","layers":[10],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.019984417012892663,"worst_max_abs_dp":0.04209446907043457,"mean_of_row_mean_abs_dp":0.0071556266837029,"argmax_equal_non_near_tie":true,"seconds":125.59593425001367,"finished":"2026-10-03T18:59:41+09:00"},"layer_16_fp16":{"why":"layer 16's linears fp32, every other linear int8","layers":[16],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.005473625002196059,"worst_max_abs_dp":0.04198223352432251,"mean_of_row_mean_abs_dp":0.007562959320946699,"argmax_equal_non_near_tie":true,"seconds":202.29436454200186,"finished":"2026-10-03T19:03:03+09:00"},"layer_22_fp16":{"why":"layer 22's linears fp32, every other linear int8","layers":[22],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":16,"swap_seconds":0.01750654200441204,"worst_max_abs_dp":0.04199141263961792,"mean_of_row_mean_abs_dp":0.0075619155049531,"argmax_equal_non_near_tie":true,"seconds":221.9284342500032,"finished":"2026-10-03T19:06:45+09:00"},"proj_out_fp16":{"why":"out_proj fp32 in every layer, the rest int8","layers":[],"kinds":["out"],"fp16_modules":18,"int8_modules":168,"fp16_params":37748736,"int8_params":459866112,"fp16_fraction":0.07585934413275386,"modules_swapped":24,"swap_seconds":0.010687957983464003,"worst_max_abs_dp":0.03739839792251587,"mean_of_row_mean_abs_dp":0.0057090379776329634,"argmax_equal_non_near_tie":true,"seconds":229.14073658298003,"finished":"2026-10-03T19:10:34+09:00"},"proj_q_fp16":{"why":"q_proj fp32 in every layer, the rest int8","layers":[],"kinds":["q"],"fp16_modules":6,"int8_modules":180,"fp16_params":25165824,"int8_params":472449024,"fp16_fraction":0.050572896088502566,"modules_swapped":24,"swap_seconds":0.01401154202176258,"worst_max_abs_dp":0.04338905215263367,"mean_of_row_mean_abs_dp":0.007281774973186354,"argmax_equal_non_near_tie":true,"seconds":218.22597887500888,"finished":"2026-10-03T19:14:12+09:00"},"layer_05_fp16":{"why":"layer 5's linears fp32, every other linear int8","layers":[5],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":178,"swap_seconds":0.49878424999769777,"worst_max_abs_dp":0.04173511266708374,"mean_of_row_mean_abs_dp":0.007302109721220202,"argmax_equal_non_near_tie":true,"seconds":127.64735924999695,"finished":"2026-10-03T18:57:37+09:00"},"layer_11_fp16":{"why":"layer 11's linears fp32, every other linear int8","layers":[11],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":15,"swap_seconds":0.02154041701578535,"worst_max_abs_dp":0.0443778932094574,"mean_of_row_mean_abs_dp":0.0071738071105856855,"argmax_equal_non_near_tie":true,"seconds":124.88982379200752,"finished":"2026-10-03T18:59:42+09:00"},"layer_17_fp16":{"why":"layer 17's linears fp32, every other linear int8","layers":[17],"kinds":[],"fp16_modules":8,"int8_modules":178,"fp16_params":21528576,"int8_params":476086272,"fp16_fraction":0.043263532200711184,"modules_swapped":15,"swap_seconds":0.0048491249908693135,"worst_max_abs_dp":0.041974663734436035,"mean_of_row_mean_abs_dp":0.007565615939286848,"argmax_equal_non_near_tie":true,"seconds":203.48503924999386,"finished":"2026-10-03T19:03:05+09:00"},"layer_23_fp16":{"why":"layer 23's linears fp32, every other linear int8","layers":[23],"kinds":[],"fp16_modules":7,"int8_modules":179,"fp16_params":18350080,"int8_params":479264768,"fp16_fraction":0.036876070064533126,"modules_swapped":15,"swap_seconds":0.008102208987111226,"worst_max_abs_dp":0.04179176688194275,"mean_of_row_mean_abs_dp":0.007540275058191683,"argmax_equal_non_near_tie":true,"seconds":222.10541516699595,"finished":"2026-10-03T19:06:47+09:00"},"proj_k_fp16":{"why":"k_proj fp32 in every layer, the rest int8","layers":[],"kinds":["k"],"fp16_modules":6,"int8_modules":180,"fp16_params":3145728,"int8_params":494469120,"fp16_fraction":0.006321612011062821,"modules_swapped":11,"swap_seconds":0.007111499988241121,"worst_max_abs_dp":0.041714996099472046,"mean_of_row_mean_abs_dp":0.0073561636498197915,"argmax_equal_non_near_tie":true,"seconds":228.92390487500234,"finished":"2026-10-03T19:10:36+09:00"},"proj_v_fp16":{"why":"v_proj fp32 in every layer, the rest int8","layers":[],"kinds":["v"],"fp16_modules":6,"int8_modules":180,"fp16_params":3145728,"int8_params":494469120,"fp16_fraction":0.006321612011062821,"modules_swapped":12,"swap_seconds":0.001346500008367002,"worst_max_abs_dp":0.04097825288772583,"mean_of_row_mean_abs_dp":0.007402294815983623,"argmax_equal_non_near_tie":true,"seconds":218.27096525000525,"finished":"2026-10-03T19:14:14+09:00"},"proj_o_fp16":{"why":"o_proj fp32 in every layer, the rest int8","layers":[],"kinds":["o"],"fp16_modules":6,"int8_modules":180,"fp16_params":12582912,"int8_params":485031936,"fp16_fraction":0.025286448044251283,"modules_swapped":12,"swap_seconds":0.002666291984496638,"worst_max_abs_dp":0.03897479176521301,"mean_of_row_mean_abs_dp":0.006933213919142468,"argmax_equal_non_near_tie":true,"seconds":218.7406123339897,"finished":"2026-10-03T19:17:53+09:00"},"set_L01_L03":{"why":"layers [1, 3] and kinds [] fp32, the rest int8","layers":[1,3],"kinds":[],"fp16_modules":15,"int8_modules":171,"fp16_params":39878656,"int8_params":457736192,"fp16_fraction":0.0801396022652443,"modules_swapped":171,"swap_seconds":0.8192487920168787,"worst_max_abs_dp":0.032437294721603394,"mean_of_row_mean_abs_dp":0.006240192965035223,"argmax_equal_non_near_tie":true,"seconds":193.02699195800233,"finished":"2026-10-03T19:18:35+09:00"},"set_L01_L02_L03_L04_L08_L09":{"why":"layers [1, 2, 3, 4, 8, 9] and kinds [] fp32, the rest int8","layers":[1,2,3,4,8,9],"kinds":[],"fp16_modules":47,"int8_modules":139,"fp16_params":125992960,"int8_params":371621888,"fp16_fraction":0.253193731068089,"modules_swapped":32,"swap_seconds":0.020510582980932668,"worst_max_abs_dp":0.022240281105041504,"mean_of_row_mean_abs_dp":0.004424009834312731,"argmax_equal_non_near_tie":true,"seconds":222.2155937079806,"finished":"2026-10-03T19:22:17+09:00"},"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L23":{"why":"layers [1, 2, 3, 4, 5, 6, 8, 9, 12, 23] and kinds [] fp32, the rest int8","layers":[1,2,3,4,5,6,8,9,12,23],"kinds":[],"fp16_modules":78,"int8_modules":108,"fp16_params":208928768,"int8_params":288686080,"fp16_fraction":0.4198603977347557,"modules_swapped":31,"swap_seconds":0.019650417001685128,"worst_max_abs_dp":0.017971426248550415,"mean_of_row_mean_abs_dp":0.003354770364239812,"argmax_equal_non_near_tie":true,"seconds":256.32279149998794,"finished":"2026-10-03T19:26:34+09:00"},"set_L01_L03_L08":{"why":"layers [1, 3, 8] and kinds [] fp32, the rest int8","layers":[1,3,8],"kinds":[],"fp16_modules":23,"int8_modules":163,"fp16_params":61407232,"int8_params":436207616,"fp16_fraction":0.12340313446595548,"modules_swapped":163,"swap_seconds":0.8185310830012895,"worst_max_abs_dp":0.031371086835861206,"mean_of_row_mean_abs_dp":0.005511130190764864,"argmax_equal_non_near_tie":true,"seconds":192.71587833299418,"finished":"2026-10-03T19:18:34+09:00"},"set_L01_L02_L03_L04_L06_L08_L09":{"why":"layers [1, 2, 3, 4, 6, 8, 9] and kinds [] fp32, the rest int8","layers":[1,2,3,4,6,8,9],"kinds":[],"fp16_modules":55,"int8_modules":131,"fp16_params":147521536,"int8_params":350093312,"fp16_fraction":0.29645726326880023,"modules_swapped":32,"swap_seconds":0.02355616699787788,"worst_max_abs_dp":0.021508246660232544,"mean_of_row_mean_abs_dp":0.003879685015676336,"argmax_equal_non_near_tie":true,"seconds":222.39799125000718,"finished":"2026-10-03T19:22:17+09:00"},"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L23":{"why":"layers [1, 2, 3, 4, 5, 6, 8, 9, 12, 15, 23] and kinds [] fp32, the rest int8","layers":[1,2,3,4,5,6,8,9,12,15,23],"kinds":[],"fp16_modules":85,"int8_modules":101,"fp16_params":227278848,"int8_params":270336000,"fp16_fraction":0.45673646779928884,"modules_swapped":30,"swap_seconds":0.02041945801465772,"worst_max_abs_dp":0.01791420578956604,"mean_of_row_mean_abs_dp":0.0033478968453386593,"argmax_equal_non_near_tie":true,"seconds":257.0984012919944,"finished":"2026-10-03T19:26:34+09:00"},"set_L01_L03_L08_L09":{"why":"layers [1, 3, 8, 9] and kinds [] fp32, the rest int8","layers":[1,3,8,9],"kinds":[],"fp16_modules":31,"int8_modules":155,"fp16_params":82935808,"int8_params":414679040,"fp16_fraction":0.16666666666666666,"modules_swapped":155,"swap_seconds":0.7149915419868194,"worst_max_abs_dp":0.028752028942108154,"mean_of_row_mean_abs_dp":0.005207888113283035,"argmax_equal_non_near_tie":true,"seconds":192.74772216699785,"finished":"2026-10-03T19:18:35+09:00"},"set_L01_L02_L03_L04_L06_L08_L09_L12":{"why":"layers [1, 2, 3, 4, 6, 8, 9, 12] and kinds [] fp32, the rest int8","layers":[1,2,3,4,6,8,9,12],"kinds":[],"fp16_modules":63,"int8_modules":123,"fp16_params":169050112,"int8_params":328564736,"fp16_fraction":0.3397207954695114,"modules_swapped":32,"swap_seconds":0.02336275001289323,"worst_max_abs_dp":0.021167218685150146,"mean_of_row_mean_abs_dp":0.0038070319957720735,"argmax_equal_non_near_tie":true,"seconds":222.17397395899752,"finished":"2026-10-03T19:22:17+09:00"},"set_L01_L02_L03_L04_L05_L06_L08_L09_L12_L15_L18_L23":{"why":"layers [1, 2, 3, 4, 5, 6, 8, 9, 12, 15, 18, 23] and kinds [] fp32, the rest int8","layers":[1,2,3,4,5,6,8,9,12,15,18,23],"kinds":[],"fp16_modules":93,"int8_modules":93,"fp16_params":248807424,"int8_params":248807424,"fp16_fraction":0.5,"modules_swapped":30,"swap_seconds":0.018293166998773813,"worst_max_abs_dp":0.017874151468276978,"mean_of_row_mean_abs_dp":0.0033582762013086016,"argmax_equal_non_near_tie":true,"seconds":255.85228962500696,"finished":"2026-10-03T19:26:33+09:00"},"set_L01_L03_L04_L08_L09":{"why":"layers [1, 3, 4, 8, 9] and kinds [] fp32, the rest int8","layers":[1,3,4,8,9],"kinds":[],"fp16_modules":39,"int8_modules":147,"fp16_params":104464384,"int8_params":393150464,"fp16_fraction":0.20993019886737785,"modules_swapped":147,"swap_seconds":0.7154462919861544,"worst_max_abs_dp":0.027990341186523438,"mean_of_row_mean_abs_dp":0.004847974239641593,"argmax_equal_non_near_tie":true,"seconds":192.28684341697954,"finished":"2026-10-03T19:18:34+09:00"},"set_L01_L02_L03_L04_L05_L06_L08_L09_L12":{"why":"layers [1, 2, 3, 4, 5, 6, 8, 9, 12] and kinds [] fp32, the rest int8","layers":[1,2,3,4,5,6,8,9,12],"kinds":[],"fp16_modules":71,"int8_modules":115,"fp16_params":190578688,"int8_params":307036160,"fp16_fraction":0.38298432767022256,"modules_swapped":32,"swap_seconds":0.01976004199241288,"worst_max_abs_dp":0.017789050936698914,"mean_of_row_mean_abs_dp":0.0033890123414393103,"argmax_equal_non_near_tie":true,"seconds":221.54927470898838,"finished":"2026-10-03T19:22:16+09:00"},"set_L00_L01_L02_L03_L04_L05_L06_L07_L08_L09_L10_L11":{"why":"layers [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] and kinds [] fp32, the rest int8","layers":[0,1,2,3,4,5,6,7,8,9,10,11],"kinds":[],"fp16_modules":93,"int8_modules":93,"fp16_params":248807424,"int8_params":248807424,"fp16_fraction":0.5,"modules_swapped":38,"swap_seconds":0.021594166988506913,"worst_max_abs_dp":0.00460052490234375,"mean_of_row_mean_abs_dp":0.0008129597589787511,"argmax_equal_non_near_tie":true,"seconds":255.67591674998403,"finished":"2026-10-03T19:26:31+09:00"},"set_L01_L03_L04_L08_L09__K_up":{"why":"layers [1, 3, 4, 8, 9] and kinds ['up'] fp32, the rest int8","layers":[1,3,4,8,9],"kinds":["up"],"fp16_modules":58,"int8_modules":128,"fp16_params":174194688,"int8_params":323420160,"fp16_fraction":0.3500592651126037,"modules_swapped":128,"swap_seconds":0.49778558398247696,"worst_max_abs_dp":0.020550429821014404,"mean_of_row_mean_abs_dp":0.004073258165994452,"argmax_equal_non_near_tie":true,"seconds":222.39581545899273,"finished":"2026-10-03T19:22:16+09:00"},"set_K_up_down":{"why":"layers [] and kinds ['up', 'down'] fp32, the rest int8","layers":[],"kinds":["up","down"],"fp16_modules":48,"int8_modules":138,"fp16_params":176160768,"int8_params":321454080,"fp16_fraction":0.35401027261951795,"modules_swapped":138,"swap_seconds":0.5016302919830196,"worst_max_abs_dp":0.028537988662719727,"mean_of_row_mean_abs_dp":0.005623602278582339,"argmax_equal_non_near_tie":true,"seconds":222.69342045899248,"finished":"2026-10-03T19:22:16+09:00"},"set_L03__K_up":{"why":"layers [3] and kinds ['up'] fp32, the rest int8","layers":[3],"kinds":["up"],"fp16_modules":30,"int8_modules":156,"fp16_params":102760448,"int8_params":394854400,"fp16_fraction":0.20650599236138548,"modules_swapped":28,"swap_seconds":0.1287136669852771,"worst_max_abs_dp":0.02965041995048523,"mean_of_row_mean_abs_dp":0.005739943670212394,"argmax_equal_non_near_tie":true,"seconds":256.4957508750085,"finished":"2026-10-03T19:26:32+09:00"},"set_L03__K_up_down":{"why":"layers [3] and kinds ['up', 'down'] fp32, the rest int8","layers":[3],"kinds":["up","down"],"fp16_modules":53,"int8_modules":133,"fp16_params":187170816,"int8_params":310444032,"fp16_fraction":0.37613591465823787,"modules_swapped":23,"swap_seconds":0.01663925001048483,"worst_max_abs_dp":0.022936761379241943,"mean_of_row_mean_abs_dp":0.005030889932014462,"argmax_equal_non_near_tie":true,"seconds":156.23032595901168,"finished":"2026-10-03T19:29:09+09:00"},"set_K_out_up_down":{"why":"layers [] and kinds ['out', 'up', 'down'] fp32, the rest int8","layers":[],"kinds":["out","up","down"],"fp16_modules":66,"int8_modules":120,"fp16_params":213909504,"int8_params":283705344,"fp16_fraction":0.4298696167522718,"modules_swapped":120,"swap_seconds":0.43554074998246506,"worst_max_abs_dp":0.01806086301803589,"mean_of_row_mean_abs_dp":0.0037061840903738308,"argmax_equal_non_near_tie":true,"seconds":222.79026666699792,"finished":"2026-10-03T19:22:16+09:00"},"set_L01_L03__K_up":{"why":"layers [1, 3] and kinds ['up'] fp32, the rest int8","layers":[1,3],"kinds":["up"],"fp16_modules":37,"int8_modules":149,"fp16_params":120619008,"int8_params":376995840,"fp16_fraction":0.24239431054919006,"modules_swapped":49,"swap_seconds":0.17695187497884035,"worst_max_abs_dp":0.02643844485282898,"mean_of_row_mean_abs_dp":0.0050451034576528605,"argmax_equal_non_near_tie":true,"seconds":256.24770770798204,"finished":"2026-10-03T19:26:32+09:00"},"set_L01_L02_L03_L04_L08_L09__K_up":{"why":"layers [1, 2, 3, 4, 8, 9] and kinds ['up'] fp32, the rest int8","layers":[1,2,3,4,8,9],"kinds":["up"],"fp16_modules":65,"int8_modules":121,"fp16_params":192053248,"int8_params":305561600,"fp16_fraction":0.3859475833004083,"modules_swapped":28,"swap_seconds":0.012640875007491559,"worst_max_abs_dp":0.019790172576904297,"mean_of_row_mean_abs_dp":0.003925238018079351,"argmax_equal_non_near_tie":true,"seconds":156.05421550001483,"finished":"2026-10-03T19:29:08+09:00"},"set_L01_L03_L08__K_up":{"why":"layers [1, 3, 8] and kinds ['up'] fp32, the rest int8","layers":[1,3,8],"kinds":["up"],"fp16_modules":44,"int8_modules":142,"fp16_params":138477568,"int8_params":359137280,"fp16_fraction":0.2782826287369946,"modules_swapped":142,"swap_seconds":0.5407261250074953,"worst_max_abs_dp":0.025286227464675903,"mean_of_row_mean_abs_dp":0.004736099319739474,"argmax_equal_non_near_tie":true,"seconds":222.87768050000886,"finished":"2026-10-03T19:22:16+09:00"},"set_L01_L03__K_up_down":{"why":"layers [1, 3] and kinds ['up', 'down'] fp32, the rest int8","layers":[1,3],"kinds":["up","down"],"fp16_modules":59,"int8_modules":127,"fp16_params":201359360,"int8_params":296255488,"fp16_fraction":0.4046490188331358,"modules_swapped":27,"swap_seconds":0.029282000003149733,"worst_max_abs_dp":0.021231919527053833,"mean_of_row_mean_abs_dp":0.0043264456300271875,"argmax_equal_non_near_tie":true,"seconds":256.0227040000027,"finished":"2026-10-03T19:26:32+09:00"},"set_L01_L02_L03_L04_L06_L08_L09__K_up":{"why":"layers [1, 2, 3, 4, 6, 8, 9] and kinds ['up'] fp32, the rest int8","layers":[1,2,3,4,6,8,9],"kinds":["up"],"fp16_modules":72,"int8_modules":114,"fp16_params":209911808,"int8_params":287703040,"fp16_fraction":0.4218359014882128,"modules_swapped":47,"swap_seconds":0.036986749997595325,"worst_max_abs_dp":0.02111712098121643,"mean_of_row_mean_abs_dp":0.003266810121325155,"argmax_equal_non_near_tie":true,"seconds":155.8864904580114,"finished":"2026-10-03T19:29:08+09:00"},"set_L01_L03_L08_L09__K_up":{"why":"layers [1, 3, 8, 9] and kinds ['up'] fp32, the rest int8","layers":[1,3,8,9],"kinds":["up"],"fp16_modules":51,"int8_modules":135,"fp16_params":156336128,"int8_params":341278720,"fp16_fraction":0.31417094692479913,"modules_swapped":135,"swap_seconds":0.51125879198662,"worst_max_abs_dp":0.023656219244003296,"mean_of_row_mean_abs_dp":0.004516828031693068,"argmax_equal_non_near_tie":true,"seconds":223.16445804198156,"finished":"2026-10-03T19:22:16+09:00"},"set_L01_L03_L08__K_up_down":{"why":"layers [1, 3, 8] and kinds ['up', 'down'] fp32, the rest int8","layers":[1,3,8],"kinds":["up","down"],"fp16_modules":65,"int8_modules":121,"fp16_params":215547904,"int8_params":282066944,"fp16_fraction":0.4331621230080337,"modules_swapped":26,"swap_seconds":0.027727375010726973,"worst_max_abs_dp":0.018236875534057617,"mean_of_row_mean_abs_dp":0.003991934094422807,"argmax_equal_non_near_tie":true,"seconds":256.48233541700756,"finished":"2026-10-03T19:26:33+09:00"},"set_L03__K_out_up_down":{"why":"layers [3] and kinds ['out', 'up', 'down'] fp32, the rest int8","layers":[3],"kinds":["out","up","down"],"fp16_modules":71,"int8_modules":115,"fp16_params":224919552,"int8_params":272695296,"fp16_fraction":0.4519952587909917,"modules_swapped":26,"swap_seconds":0.04494212500867434,"worst_max_abs_dp":0.018612265586853027,"mean_of_row_mean_abs_dp":0.0032404107921239407,"argmax_equal_non_near_tie":true,"seconds":155.789031332999,"finished":"2026-10-03T19:29:09+09:00"},"set_K_out_o_up_down":{"why":"layers [] and kinds ['out', 'o', 'up', 'down'] fp32, the rest int8","layers":[],"kinds":["out","o","up","down"],"fp16_modules":72,"int8_modules":114,"fp16_params":226492416,"int8_params":271122432,"fp16_fraction":0.4551560647965231,"modules_swapped":114,"swap_seconds":0.3807548340118956,"worst_max_abs_dp":0.014053970575332642,"mean_of_row_mean_abs_dp":0.003161626150055478,"argmax_equal_non_near_tie":true,"seconds":257.20702387500205,"finished":"2026-10-03T19:26:43+09:00"},"set_L01_L02_L03_L04_L06_L08_L09_L12__K_up":{"why":"layers [1, 2, 3, 4, 6, 8, 9, 12] and kinds ['up'] fp32, the rest int8","layers":[1,2,3,4,6,8,9,12],"kinds":["up"],"fp16_modules":79,"int8_modules":107,"fp16_params":227770368,"int8_params":269844480,"fp16_fraction":0.4577242196760174,"modules_swapped":71,"swap_seconds":0.1345116659940686,"worst_max_abs_dp":0.02080148458480835,"mean_of_row_mean_abs_dp":0.0031743595130845075,"argmax_equal_non_near_tie":true,"seconds":150.79950641599135,"finished":"2026-10-03T19:29:13+09:00"}},"generated_at":"2026-10-03T19:29:45+09:00"},"summary":{"schema":"kev-r3-int8-summary/1","bar":"FACTS §7: argmax equal on every question whose oracle top-2 margin > 0.02, max |dp| <= 0.02, mean of the rows' mean |dp| <= 0.002, reset bit-equal, finite","fixture_434":{"fp16":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_fp16_pf16.json","transcript_sha256":"2c6a7846081de2e83bea6c6df792418aa8f2d5f846b96dadb7358fd0bf66219a","result":"PASS","rows":434,"argmax_non_near_tie":[420,420],"argmax_near_tie":[12,14],"max_abs_dp":0.011678963899612427,"mean_of_run_mean_abs_dp":0.0009687347178998231,"row_max_abs_dp_quantiles":{"p50":0.0010115653276443481,"p90":0.003131607174873352,"max":0.011678963899612427},"rows_above":{"0.01":1,"0.02":0},"worst_10":[{"id":"semif_46b7029b9a704138b77a","q":0,"source":"semif_authored144","T":78,"max_abs_dp":0.011678963899612427,"near_tie":false},{"id":"tv4_045","q":0,"source":"transfer_v4_dev_head","T":90,"max_abs_dp":0.0073184967041015625,"near_tie":false},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"source":"semif_authored144","T":67,"max_abs_dp":0.006962984800338745,"near_tie":false},{"id":"semif_d7e84e4245a29817da1f","q":0,"source":"semif_authored144","T":67,"max_abs_dp":0.006792604923248291,"near_tie":false},{"id":"tv4_020","q":0,"source":"transfer_v4_dev_head","T":82,"max_abs_dp":0.0062604546546936035,"near_tie":false},{"id":"semif_942ec65dac0bfb9874f9","q":0,"source":"semif_authored144","T":60,"max_abs_dp":0.00599294900894165,"near_tie":false},{"id":"semif_97d2ed2591fc7638806c","q":0,"source":"semif_authored144","T":82,"max_abs_dp":0.005683779716491699,"near_tie":false},{"id":"semif_eafc22c8c40df3932a8e","q":0,"source":"semif_authored144","T":72,"max_abs_dp":0.005271822214126587,"near_tie":false},{"id":"tv4x_legacy_holdout_03","q":0,"source":"transfer_v4_dev_sources","T":77,"max_abs_dp":0.0052561163902282715,"near_tie":false},{"id":"semif_8a4c3de28be95c105ca1","q":0,"source":"semif_authored144","T":72,"max_abs_dp":0.005201578140258789,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":0.9999578776962144,"ms_per_call_warm":{"n":3972,"median":18.78837500407826,"p10":18.04734579345677,"p90":20.165566698415205},"load_seconds_median":0.29674316599266604,"contended":true,"other_gpu_processes_start":0,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_fp16_pf16","main_mlirb_bytes":1506481909,"main_mlirb_sha256":"4131d0284899787bab6cae6ca62a706886478b4e25c4d8cb971c5ba2ad72682d","aimodelc_bytes":2505682731,"aimodelc_tree_sha256":"de1b6a1a12a0a7a3726bd043e9e5859b3ddea295e5c5dcec7fa87b33d25cef02","compression":null},"export":{"seconds":{"load":0.5,"quantize":0.0,"export":42.3,"optimize":0.3,"export_optimize":42.6,"save":1.3,"total":44.5},"aot_seconds":5.9,"du_bundle":"1.4G","du_aimodelc":"2.3G","int8_linear_modules":null,"int8_params":null,"fp16_layers":null},"red_arms_v2":{"result":"RED (every arm)","transcript_sha256":"a7a750823303ddf7044dc2619f6d80e7a4fb210669660fa0da876a4d32214744","arms":{"instr_negate_v2":{"argmax_moved":2,"rows":5,"max_abs_dp_vs_base":0.7587456461042166,"mean_of_run_mean_abs_dp_vs_base":0.29107410442084075,"red":true},"state_swap":{"argmax_moved":4,"rows":5,"max_abs_dp_vs_base":0.9051760490983725,"mean_of_run_mean_abs_dp_vs_base":0.4530959569786985,"red":true}},"base_bit_equal_gate_run":true,"reset":true}},"int8lin":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_int8lin_pf16.json","transcript_sha256":"610053646a4571869615d0c9ac584f8ad8d5749c21f82597118ad4d03b3f5730","result":"FAIL","rows":434,"argmax_non_near_tie":[420,420],"argmax_near_tie":[11,14],"max_abs_dp":0.04313388466835022,"mean_of_run_mean_abs_dp":0.0032562481012639575,"row_max_abs_dp_quantiles":{"p50":0.003036871552467346,"p90":0.010434174537658691,"max":0.04313388466835022},"rows_above":{"0.01":51,"0.02":10},"worst_10":[{"id":"semif_1105577da4c8dad4609d","q":0,"source":"semif_authored144","T":88,"max_abs_dp":0.04313388466835022,"near_tie":false},{"id":"semif_a22dab8ba073ff6cdb8d","q":0,"source":"semif_authored144","T":84,"max_abs_dp":0.04081737995147705,"near_tie":false},{"id":"tv4_023","q":0,"source":"transfer_v4_dev_head","T":92,"max_abs_dp":0.026517271995544434,"near_tie":true},{"id":"tv4x_composition_holdout_15","q":0,"source":"transfer_v4_dev_sources","T":122,"max_abs_dp":0.026461392641067505,"near_tie":false},{"id":"semif_46b7029b9a704138b77a","q":0,"source":"semif_authored144","T":78,"max_abs_dp":0.026026636362075806,"near_tie":false},{"id":"semif_f3d13cf184f916677def","q":0,"source":"semif_authored144","T":91,"max_abs_dp":0.02583444118499756,"near_tie":false},{"id":"semif_386e10b1c1ce906473a4","q":0,"source":"semif_authored144","T":88,"max_abs_dp":0.024856865406036377,"near_tie":false},{"id":"tv4x_composition_holdout_12","q":0,"source":"transfer_v4_dev_sources","T":122,"max_abs_dp":0.024287700653076172,"near_tie":false},{"id":"tv4x_composition_holdout_14","q":0,"source":"transfer_v4_dev_sources","T":122,"max_abs_dp":0.024287700653076172,"near_tie":false},{"id":"tv4x_legacy_holdout_13","q":0,"source":"transfer_v4_dev_sources","T":78,"max_abs_dp":0.02400749921798706,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":0.9990858151789371,"ms_per_call_warm":{"n":3972,"median":21.899353989283554,"p10":18.218545787385665,"p90":28.547662499477156},"load_seconds_median":0.30040495900902897,"contended":true,"other_gpu_processes_start":4,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_int8lin_pf16","main_mlirb_bytes":1040063350,"main_mlirb_sha256":"2281e56d413cbc38cde4bd37953fd5476726a1f37953f940eba9dadcc0bfdb77","aimodelc_bytes":2039283435,"aimodelc_tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":186,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"}},"export":{"seconds":{"load":1.2,"quantize":12.3,"export":46.3,"optimize":0.3,"export_optimize":46.6,"save":1.1,"total":61.3},"aot_seconds":24.4,"du_bundle":"1.0G","du_aimodelc":"1.9G","int8_linear_modules":186,"int8_params":497614848,"fp16_layers":[]},"vs_fp16":{"rows":434,"max_abs_dp":0.044226497411727905,"mean_row_max_abs_dp":0.004442330672993638,"p50":0.0029132962226867676,"p90":0.009966552257537842,"argmax_equal":431},"red_arms_v2":{"result":"RED (every arm)","transcript_sha256":"e20ab43f0b194d562642ef69bfdce4e50702f93e67fb10efd9e040e9bb8bc578","arms":{"instr_negate_v2":{"argmax_moved":2,"rows":5,"max_abs_dp_vs_base":0.7648195866495371,"mean_of_run_mean_abs_dp_vs_base":0.29320039693266153,"red":true},"state_swap":{"argmax_moved":4,"rows":5,"max_abs_dp_vs_base":0.9048093687742949,"mean_of_run_mean_abs_dp_vs_base":0.45287297501539187,"red":true}},"base_bit_equal_gate_run":true,"reset":true}},"int8mix":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_int8mix_pf16.json","transcript_sha256":"0b4be0782bbacc28e00a10772b9cec13160dae5eb1fbcb9965e65bc58c1f17bb","result":"PASS","rows":434,"argmax_non_near_tie":[420,420],"argmax_near_tie":[12,14],"max_abs_dp":0.00888979434967041,"mean_of_run_mean_abs_dp":0.0010320237972326677,"row_max_abs_dp_quantiles":{"p50":0.0011536180973052979,"p90":0.0032525897026062013,"max":0.00888979434967041},"rows_above":{"0.01":0,"0.02":0},"worst_10":[{"id":"semif_46b7029b9a704138b77a","q":0,"source":"semif_authored144","T":78,"max_abs_dp":0.00888979434967041,"near_tie":false},{"id":"semif_7ada97b6f99f4788e5c7","q":0,"source":"semif_authored144","T":67,"max_abs_dp":0.00721016526222229,"near_tie":false},{"id":"tv4_020","q":0,"source":"transfer_v4_dev_head","T":82,"max_abs_dp":0.006994843482971191,"near_tie":false},{"id":"tv4_045","q":0,"source":"transfer_v4_dev_head","T":90,"max_abs_dp":0.006981581449508667,"near_tie":false},{"id":"semif_386e10b1c1ce906473a4","q":0,"source":"semif_authored144","T":88,"max_abs_dp":0.0069460272789001465,"near_tie":false},{"id":"semif_8a4c3de28be95c105ca1","q":0,"source":"semif_authored144","T":72,"max_abs_dp":0.006723523139953613,"near_tie":false},{"id":"own_L02","q":0,"source":"own_long","T":1518,"max_abs_dp":0.006616711616516113,"near_tie":false},{"id":"semif_d7e84e4245a29817da1f","q":0,"source":"semif_authored144","T":67,"max_abs_dp":0.00632476806640625,"near_tie":false},{"id":"semif_712f6ffb763eef7c9a12","q":0,"source":"semif_authored144","T":64,"max_abs_dp":0.0056025683879852295,"near_tie":false},{"id":"own_t04","q":3,"source":"own_text","T":175,"max_abs_dp":0.005302995443344116,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":0.9998692690427006,"ms_per_call_warm":{"n":3972,"median":21.733832996687852,"p10":19.7982288984349,"p90":23.81210390594788},"load_seconds_median":0.29000483301933855,"contended":true,"other_gpu_processes_start":1,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_int8mix_pf16","main_mlirb_bytes":1273276920,"main_mlirb_sha256":"84b37b90783782a80f4a348a69e7cccbe96156d1b8a5907b6f25a399beb7dadf","aimodelc_bytes":2272483979,"aimodelc_tree_sha256":"a551fda4c99dea9db6841efa22cec3d7067deb61a4ae7d8c6791b2d2832ef000","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":93,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the pointer head runs on the host, head/)"}},"export":{"seconds":{"load":0.2,"quantize":11.8,"export":43.8,"optimize":0.3,"export_optimize":44.1,"save":1.2,"total":57.4},"aot_seconds":15.7,"du_bundle":"1.2G","du_aimodelc":"2.1G","int8_linear_modules":93,"int8_params":248807424,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11]},"vs_fp16":{"rows":434,"max_abs_dp":0.005405724048614502,"mean_row_max_abs_dp":0.0008100075915472985,"p50":0.0006482899188995361,"p90":0.0016523212194442746,"argmax_equal":434},"red_arms_v2":{"result":"RED (every arm)","transcript_sha256":"4f52cac870950642535a15f16fea2e0e76bfb5d11fe396c1a39653c3cd467b47","arms":{"instr_negate_v2":{"argmax_moved":2,"rows":5,"max_abs_dp_vs_base":0.756274864077568,"mean_of_run_mean_abs_dp_vs_base":0.29028557520359755,"red":true},"state_swap":{"argmax_moved":4,"rows":5,"max_abs_dp_vs_base":0.9049020633101463,"mean_of_run_mean_abs_dp_vs_base":0.4531575363688171,"red":true}},"base_bit_equal_gate_run":true,"reset":true}}},"heldout_130":{"fp16":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_heldout_fp16_pf16.json","transcript_sha256":"0cbfed2c8e3a9c1cb58ecbb11de5668e7c5c2744e925be08980f589746bad603","result":"PASS","rows":130,"argmax_non_near_tie":[125,125],"argmax_near_tie":[5,5],"max_abs_dp":0.005739271640777588,"mean_of_run_mean_abs_dp":0.0008054346284375359,"row_max_abs_dp_quantiles":{"p50":0.0009092092514038086,"p90":0.002331995964050293,"max":0.005739271640777588},"rows_above":{"0.01":0,"0.02":0},"worst_10":[{"id":"tv4h_11","q":0,"source":"transfer_v4_dev_head_heldout","T":299,"max_abs_dp":0.005739271640777588,"near_tie":false},{"id":"tv4xh_paws_00","q":0,"source":"transfer_v4_dev_sources_heldout","T":94,"max_abs_dp":0.0037997663021087646,"near_tie":false},{"id":"tv4xh_qnli_06","q":0,"source":"transfer_v4_dev_sources_heldout","T":48,"max_abs_dp":0.003721684217453003,"near_tie":false},{"id":"tv4xh_paws_04","q":0,"source":"transfer_v4_dev_sources_heldout","T":76,"max_abs_dp":0.0035116970539093018,"near_tie":false},{"id":"tv4xh_composition_holdout_09","q":0,"source":"transfer_v4_dev_sources_heldout","T":122,"max_abs_dp":0.0034955739974975586,"near_tie":false},{"id":"tv4h_00","q":0,"source":"transfer_v4_dev_head_heldout","T":59,"max_abs_dp":0.0031816959381103516,"near_tie":false},{"id":"tv4sh_00","q":0,"source":"transfer_v4_dev_score_heldout","T":98,"max_abs_dp":0.003098130226135254,"near_tie":false},{"id":"tv4xh_emotion_02","q":0,"source":"transfer_v4_dev_sources_heldout","T":53,"max_abs_dp":0.0027331411838531494,"near_tie":false},{"id":"tv4xh_legacy_holdout_09","q":0,"source":"transfer_v4_dev_sources_heldout","T":76,"max_abs_dp":0.0024182796478271484,"near_tie":false},{"id":"tv4xh_sciq_02","q":0,"source":"transfer_v4_dev_sources_heldout","T":44,"max_abs_dp":0.0024171173572540283,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":null,"ms_per_call_warm":{"n":792,"median":36.75862499221694,"p10":32.622311895829625,"p90":42.677337501663715},"load_seconds_median":0.47685272950911894,"contended":true,"other_gpu_processes_start":5,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_fp16_pf16","main_mlirb_bytes":1506481909,"main_mlirb_sha256":"4131d0284899787bab6cae6ca62a706886478b4e25c4d8cb971c5ba2ad72682d","aimodelc_bytes":2505682731,"aimodelc_tree_sha256":"de1b6a1a12a0a7a3726bd043e9e5859b3ddea295e5c5dcec7fa87b33d25cef02","compression":null}},"int8lin":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_heldout_int8lin_pf16.json","transcript_sha256":"ebc0a7c29467ccf6e97f144886ab0d3c21c7f806ace8e3c41d80770355344d06","result":"FAIL","rows":130,"argmax_non_near_tie":[125,125],"argmax_near_tie":[4,5],"max_abs_dp":0.02733546495437622,"mean_of_run_mean_abs_dp":0.0029399254602956205,"row_max_abs_dp_quantiles":{"p50":0.0028653591871261597,"p90":0.008587327599525456,"max":0.02733546495437622},"rows_above":{"0.01":10,"0.02":2},"worst_10":[{"id":"tv4xh_legacy_holdout_05","q":0,"source":"transfer_v4_dev_sources_heldout","T":76,"max_abs_dp":0.02733546495437622,"near_tie":false},{"id":"tv4xh_composition_holdout_09","q":0,"source":"transfer_v4_dev_sources_heldout","T":122,"max_abs_dp":0.024029940366744995,"near_tie":false},{"id":"tv4h_15","q":0,"source":"transfer_v4_dev_head_heldout","T":49,"max_abs_dp":0.016689717769622803,"near_tie":false},{"id":"tv4xh_qnli_06","q":0,"source":"transfer_v4_dev_sources_heldout","T":48,"max_abs_dp":0.01626250147819519,"near_tie":false},{"id":"tv4h_06","q":0,"source":"transfer_v4_dev_head_heldout","T":90,"max_abs_dp":0.012899160385131836,"near_tie":false},{"id":"tv4xh_sciq_08","q":0,"source":"transfer_v4_dev_sources_heldout","T":112,"max_abs_dp":0.012246012687683105,"near_tie":false},{"id":"tv4h_33","q":0,"source":"transfer_v4_dev_head_heldout","T":67,"max_abs_dp":0.012146830558776855,"near_tie":false},{"id":"tv4xh_qnli_01","q":0,"source":"transfer_v4_dev_sources_heldout","T":59,"max_abs_dp":0.011427253484725952,"near_tie":false},{"id":"tv4h_17","q":0,"source":"transfer_v4_dev_head_heldout","T":57,"max_abs_dp":0.011220306158065796,"near_tie":false},{"id":"tv4xh_paws_09","q":0,"source":"transfer_v4_dev_sources_heldout","T":102,"max_abs_dp":0.010669350624084473,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":null,"ms_per_call_warm":{"n":792,"median":37.10564550419804,"p10":33.18940781173296,"p90":44.29100419511087},"load_seconds_median":0.39198004199715797,"contended":true,"other_gpu_processes_start":5,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_int8lin_pf16","main_mlirb_bytes":1040063350,"main_mlirb_sha256":"2281e56d413cbc38cde4bd37953fd5476726a1f37953f940eba9dadcc0bfdb77","aimodelc_bytes":2039283435,"aimodelc_tree_sha256":"87648a5d644b71e6863305f67be352e1a9a29847a6a0dc58d994e00aa45d8a8e","compression":{"scheme":"int8lin","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":186,"fp16_layers":[],"fp16_linear_modules":[],"head":"none in the graph (the pointer head runs on the host, head/)"}},"vs_fp16":{"rows":130,"max_abs_dp":0.028595760464668274,"mean_row_max_abs_dp":0.004137265409987706,"p50":0.003038167953491211,"p90":0.00875835418701172,"argmax_equal":129}},"int8mix":{"transcript":"$ZOO_WORK_ROOT/_kev/results/readout_heldout_int8mix_pf16.json","transcript_sha256":"c0b93960cc9f86e5fe710401cf75f46301b99e1c7f90cbe2a251bd77e93992e1","result":"PASS","rows":130,"argmax_non_near_tie":[125,125],"argmax_near_tie":[5,5],"max_abs_dp":0.0042969584465026855,"mean_of_run_mean_abs_dp":0.0008201541722668574,"row_max_abs_dp_quantiles":{"p50":0.001016378402709961,"p90":0.002225518226623535,"max":0.0042969584465026855},"rows_above":{"0.01":0,"0.02":0},"worst_10":[{"id":"tv4h_11","q":0,"source":"transfer_v4_dev_head_heldout","T":299,"max_abs_dp":0.0042969584465026855,"near_tie":false},{"id":"tv4xh_composition_holdout_09","q":0,"source":"transfer_v4_dev_sources_heldout","T":122,"max_abs_dp":0.004092991352081299,"near_tie":false},{"id":"tv4xh_qnli_06","q":0,"source":"transfer_v4_dev_sources_heldout","T":48,"max_abs_dp":0.003649115562438965,"near_tie":false},{"id":"tv4h_05","q":0,"source":"transfer_v4_dev_head_heldout","T":55,"max_abs_dp":0.003253042697906494,"near_tie":false},{"id":"tv4h_17","q":0,"source":"transfer_v4_dev_head_heldout","T":57,"max_abs_dp":0.002789214253425598,"near_tie":false},{"id":"tv4xh_emotion_02","q":0,"source":"transfer_v4_dev_sources_heldout","T":53,"max_abs_dp":0.002777695655822754,"near_tie":false},{"id":"tv4sh_00","q":0,"source":"transfer_v4_dev_score_heldout","T":98,"max_abs_dp":0.0026823878288269043,"near_tie":false},{"id":"tv4xh_qnli_01","q":0,"source":"transfer_v4_dev_sources_heldout","T":59,"max_abs_dp":0.0026788413524627686,"near_tie":false},{"id":"tv4sh_10","q":0,"source":"transfer_v4_dev_score_heldout","T":98,"max_abs_dp":0.00245097279548645,"near_tie":false},{"id":"tv4h_00","q":0,"source":"transfer_v4_dev_head_heldout","T":59,"max_abs_dp":0.0023977160453796387,"near_tie":false}],"reset_bit_equal":true,"finite":true,"min_pos_cos_21_rows":null,"ms_per_call_warm":{"n":792,"median":21.789895996334963,"p10":19.821628800127655,"p90":23.59552089474164},"load_seconds_median":0.2865322920115432,"contended":true,"other_gpu_processes_start":1,"gpu_lock_start":0,"bundle":{"name":"kev_0_8b_decode_int8mix_pf16","main_mlirb_bytes":1273276920,"main_mlirb_sha256":"84b37b90783782a80f4a348a69e7cccbe96156d1b8a5907b6f25a399beb7dadf","aimodelc_bytes":2272483979,"aimodelc_tree_sha256":"a551fda4c99dea9db6841efa22cec3d7067deb61a4ae7d8c6791b2d2832ef000","compression":{"scheme":"int8mix","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","excluded":"fp16: the embedding table, the GDN conv1d, every RMSNorm (RMSNorm, RMSNormPlusOne, RMSNormGated), SDPA, RoPE, the GDN A_log / dt_bias","excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.rms_norm.RMSNormGated","coreai_models.primitives.macos.rms_norm.RMSNormPlusOne","coreai_models.primitives.macos.rope.RoPE","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"int8_linear_modules":93,"fp16_layers":[0,1,2,3,4,5,6,7,8,9,10,11],"fp16_linear_modules":["model.layers.0.linear_attn.in_proj_qkv","model.layers.0.linear_attn.in_proj_z","model.layers.0.linear_attn.in_proj_b","model.layers.0.linear_attn.in_proj_a","model.layers.0.linear_attn.out_proj","model.layers.0.mlp.gate_proj","model.layers.0.mlp.up_proj","model.layers.0.mlp.down_proj","model.layers.1.linear_attn.in_proj_qkv","model.layers.1.linear_attn.in_proj_z","model.layers.1.linear_attn.in_proj_b","model.layers.1.linear_attn.in_proj_a","model.layers.1.linear_attn.out_proj","model.layers.1.mlp.gate_proj","model.layers.1.mlp.up_proj","model.layers.1.mlp.down_proj","model.layers.2.linear_attn.in_proj_qkv","model.layers.2.linear_attn.in_proj_z","model.layers.2.linear_attn.in_proj_b","model.layers.2.linear_attn.in_proj_a","model.layers.2.linear_attn.out_proj","model.layers.2.mlp.gate_proj","model.layers.2.mlp.up_proj","model.layers.2.mlp.down_proj","model.layers.3.self_attn.q_proj","model.layers.3.self_attn.k_proj","model.layers.3.self_attn.v_proj","model.layers.3.self_attn.o_proj","model.layers.3.mlp.gate_proj","model.layers.3.mlp.up_proj","model.layers.3.mlp.down_proj","model.layers.4.linear_attn.in_proj_qkv","model.layers.4.linear_attn.in_proj_z","model.layers.4.linear_attn.in_proj_b","model.layers.4.linear_attn.in_proj_a","model.layers.4.linear_attn.out_proj","model.layers.4.mlp.gate_proj","model.layers.4.mlp.up_proj","model.layers.4.mlp.down_proj","model.layers.5.linear_attn.in_proj_qkv","model.layers.5.linear_attn.in_proj_z","model.layers.5.linear_attn.in_proj_b","model.layers.5.linear_attn.in_proj_a","model.layers.5.linear_attn.out_proj","model.layers.5.mlp.gate_proj","model.layers.5.mlp.up_proj","model.layers.5.mlp.down_proj","model.layers.6.linear_attn.in_proj_qkv","model.layers.6.linear_attn.in_proj_z","model.layers.6.linear_attn.in_proj_b","model.layers.6.linear_attn.in_proj_a","model.layers.6.linear_attn.out_proj","model.layers.6.mlp.gate_proj","model.layers.6.mlp.up_proj","model.layers.6.mlp.down_proj","model.layers.7.self_attn.q_proj","model.layers.7.self_attn.k_proj","model.layers.7.self_attn.v_proj","model.layers.7.self_attn.o_proj","model.layers.7.mlp.gate_proj","model.layers.7.mlp.up_proj","model.layers.7.mlp.down_proj","model.layers.8.linear_attn.in_proj_qkv","model.layers.8.linear_attn.in_proj_z","model.layers.8.linear_attn.in_proj_b","model.layers.8.linear_attn.in_proj_a","model.layers.8.linear_attn.out_proj","model.layers.8.mlp.gate_proj","model.layers.8.mlp.up_proj","model.layers.8.mlp.down_proj","model.layers.9.linear_attn.in_proj_qkv","model.layers.9.linear_attn.in_proj_z","model.layers.9.linear_attn.in_proj_b","model.layers.9.linear_attn.in_proj_a","model.layers.9.linear_attn.out_proj","model.layers.9.mlp.gate_proj","model.layers.9.mlp.up_proj","model.layers.9.mlp.down_proj","model.layers.10.linear_attn.in_proj_qkv","model.layers.10.linear_attn.in_proj_z","model.layers.10.linear_attn.in_proj_b","model.layers.10.linear_attn.in_proj_a","model.layers.10.linear_attn.out_proj","model.layers.10.mlp.gate_proj","model.layers.10.mlp.up_proj","model.layers.10.mlp.down_proj","model.layers.11.self_attn.q_proj","model.layers.11.self_attn.k_proj","model.layers.11.self_attn.v_proj","model.layers.11.self_attn.o_proj","model.layers.11.mlp.gate_proj","model.layers.11.mlp.up_proj","model.layers.11.mlp.down_proj"],"head":"none in the graph (the pointer head runs on the host, head/)"}},"vs_fp16":{"rows":130,"max_abs_dp":0.0027962028980255127,"mean_row_max_abs_dp":0.0005706999612112458,"p50":0.0005280077457427979,"p90":0.001020634174346924,"argmax_equal":130}}},"bisect":{"transcript":"$ZOO_WORK_ROOT/_kev/results/r3_int8_bisect.json","sha256":"0a33b8f5974dd61ca24d365008818cc212f276e7b88fd01d0a77bed9ab1f34c4","chosen":null,"reproduction_check":{"gate_worst":0.04313388466835022,"torch_all_int8_worst":0.04200497269630432,"ratio":0.9738277231293697,"reproduced":true,"pearson_per_row":0.9723470816657634,"rows_above_bar_gate":10,"rows_above_bar_torch":9},"instrument_floor":{"worst_max_abs_dp":3.4570693969726562e-06,"bit_equal_p1_rows":60,"rows":60}},"generated_at":"2026-10-03T19:33:44+09:00"}}