{"what":"d1-3B: the two shipped decoders alone on the Mac GPU (AOT h16c with --expect-frequent-reshapes, Python runtime, SpecializationOptions.default()), against the provider's fp32 oracle through the host's readout: the fixture's 393 answerable questions, at most 40 rows a process and a re-run of each process's first row (round 6a). mac_fp16 = the Mac's decoder (fp16, the attention projections fp32; static S = 64, dynamic positions); iphone_int8mlp = the iPhone's decoder (the MLP linears int8 per block of 32, the conv-mixer projections fp16, the attention projections fp32; the same contract). export_record = the export that wrote each bundle (the .aimodel, metadata.json, tokenizer/, head/).","sources":[{"file":"results/r6a_readout_fp16_pf64.json","bytes":570025,"sha256":"a65331d3218918ff8c196830c196798c82038b847a6c841a53cceaf56781094d"},{"file":"results/r6a_export_fp16_pf64.json","bytes":36420,"sha256":"905e06b67c486e171e9ba2edbac4b9fdddf5017a28e279c9f31fdb8caef34535"},{"file":"results/r6a_readout_int8mlp_pf64.json","bytes":572833,"sha256":"16f28d47362ccdc5533fa3851c1113a6a4c261638e63d37b6bd2cd2b8f526dc5"},{"file":"results/r6a_export_int8mlp_pf64.json","bytes":42107,"sha256":"3862b78f1c50f7882bb827d5be5adcf3fe26d249e965445d5ba1421fb3febba4"}],"trimmed":{"removed":["per-call ms lists (call_ms, first_call_of_process_ms; each transcript keeps its medians)","hidden-dump file names and hashes (npz, npz_key, npz_sha256, hidden_npz) and the hidden rows' lowest positions","the oracle's probabilities inside the transcripts (probs_oracle: they are in fixtures-d1-3b.json)","token ids and rendered prompt text (row_ids, ids, text, instructions, state, criteria)","other sessions' processes seen by the drivers (other_gpu_processes*, other_gpu_jobs*, busiest, top)","the red arms' block (in gate-d1-3b-red.json)"],"serialization":"compact json (no indentation)","full_copies":"the source files above, in the lane directory $ZOO_WORK_ROOT/_d1_3b/ (not in the repository)","paths":"local paths rewritten: the lane directory as $ZOO_WORK_ROOT/_d1_3b, the repository relative, the home directory as ~, the checkpoint snapshot as <snapshot>, the overlay venv as <overlay venv>, temporary directories as $TMPDIR, Xcode as $DEVELOPER_DIR, the phone's app container as <app container>, the phone's ids as <udid> / <coredevice id>","process_ids":"removed (keys pid / pids; 'pid <n>' in text)"},"mac_fp16":{"export_record":{"mode":"fp16","name":"d1_3b_decode_fp16_pf64","toy":false,"toy_seed":null,"hf_id":"LiquidAI/d1-3B","revision":"da1fe36a861f24690f27f622dca1d8688503d113","prefill_chunk":64,"fp16_layers":null,"quant_block":32,"argv":["fp16","--prefill-chunk","64","--aot","--record","$ZOO_WORK_ROOT/_d1_3b/results/r6a_export_fp16_pf64.json"],"started":"2026-10-08T16:27:15+09:00","script_sha256":"16218c7f43fe02a1622122240e3342f8b7381e21edc7d8d83e7a5561283bfcf1","module_sha256":"bacdc9f3db17cc336ba25899a403a1e99075dd3ccd52b1a99166c8e8aa29cf89","option_rows_sha256":"78d2c1132c5f58ef38a647871a17f8e5b1105a87dc8d84630cf5300e60a27ad9","export":{"bundle":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_fp16_pf64","name":"d1_3b_decode_fp16_pf64","aimodel":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_fp16_pf64/d1_3b_decode_fp16_pf64.aimodel","toy":false,"load":{"load_report":{"checkpoint_files":["model.safetensors"],"checkpoint_keys":707,"checkpoint_keys_under_prefix":266,"module_tensors":266,"unread_checkpoint_keys_under_prefix":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_embedding":true,"module_has_lm_head_weight":false,"meta_params":[],"fp32_params":["model.layers.13.self_attn.k_proj.weight","model.layers.13.self_attn.out_proj.weight","model.layers.13.self_attn.q_proj.weight","model.layers.13.self_attn.v_proj.weight","model.layers.17.self_attn.k_proj.weight","model.layers.17.self_attn.out_proj.weight","model.layers.17.self_attn.q_proj.weight","model.layers.17.self_attn.v_proj.weight","model.layers.2.self_attn.k_proj.weight","model.layers.2.self_attn.out_proj.weight","model.layers.2.self_attn.q_proj.weight","model.layers.2.self_attn.v_proj.weight","model.layers.21.self_attn.k_proj.weight","model.layers.21.self_attn.out_proj.weight","model.layers.21.self_attn.q_proj.weight","model.layers.21.self_attn.v_proj.weight","model.layers.24.self_attn.k_proj.weight","model.layers.24.self_attn.out_proj.weight","model.layers.24.self_attn.q_proj.weight","model.layers.24.self_attn.v_proj.weight","model.layers.27.self_attn.k_proj.weight","model.layers.27.self_attn.out_proj.weight","model.layers.27.self_attn.q_proj.weight","model.layers.27.self_attn.v_proj.weight","model.layers.5.self_attn.k_proj.weight","model.layers.5.self_attn.out_proj.weight","model.layers.5.self_attn.q_proj.weight","model.layers.5.self_attn.v_proj.weight","model.layers.9.self_attn.k_proj.weight","model.layers.9.self_attn.out_proj.weight","model.layers.9.self_attn.q_proj.weight","model.layers.9.self_attn.v_proj.weight"],"dtype":"float16"},"weights":{"file":"<snapshot>/model.safetensors","bytes":6247065504,"sha256":"50e03317847caf6df9a9aee27ed40f20554a86a21e60d1d47ba41a422b546c0c"},"text_config":{"num_hidden_layers":30,"num_full_layers":8,"num_conv_layers":22,"hidden_size":2048,"ff_dim":10752,"vocab_size":128000,"rope_theta":1000000.0,"tie_embedding":true,"head_dim":64,"num_attention_heads":32,"num_key_value_heads":8,"conv_L_cache":3,"conv_state_width":2,"norm_eps":1e-05,"full_attention_layers":[2,5,9,13,17,21,24,27]}},"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":64,"n_image_tokens":2816,"spec":{"input_names":["input_ids","position_ids","image_embeds"],"output_names":["hidden"],"state_names":["keyCache","valueCache","convState"],"reference_inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,128],"int32"],"image_embeds":[[2816,2048],"float16"],"k_cache":[[8,1,8,2048,64],"float16"],"v_cache":[[8,1,8,2048,64],"float16"],"conv_state":[[22,1,2048,2],"float16"]},"dynamic":{"input_ids":null,"position_ids":{"1":"seq_pos"},"image_embeds":null,"k_cache":{"3":"k_seq"},"v_cache":{"3":"v_seq"},"conv_state":null}},"fp32_params":["model.layers.13.self_attn.k_proj.weight","model.layers.13.self_attn.out_proj.weight","model.layers.13.self_attn.q_proj.weight","model.layers.13.self_attn.v_proj.weight","model.layers.17.self_attn.k_proj.weight","model.layers.17.self_attn.out_proj.weight","model.layers.17.self_attn.q_proj.weight","model.layers.17.self_attn.v_proj.weight","model.layers.2.self_attn.k_proj.weight","model.layers.2.self_attn.out_proj.weight","model.layers.2.self_attn.q_proj.weight","model.layers.2.self_attn.v_proj.weight","model.layers.21.self_attn.k_proj.weight","model.layers.21.self_attn.out_proj.weight","model.layers.21.self_attn.q_proj.weight","model.layers.21.self_attn.v_proj.weight","model.layers.24.self_attn.k_proj.weight","model.layers.24.self_attn.out_proj.weight","model.layers.24.self_attn.q_proj.weight","model.layers.24.self_attn.v_proj.weight","model.layers.27.self_attn.k_proj.weight","model.layers.27.self_attn.out_proj.weight","model.layers.27.self_attn.q_proj.weight","model.layers.27.self_attn.v_proj.weight","model.layers.5.self_attn.k_proj.weight","model.layers.5.self_attn.out_proj.weight","model.layers.5.self_attn.q_proj.weight","model.layers.5.self_attn.v_proj.weight","model.layers.9.self_attn.k_proj.weight","model.layers.9.self_attn.out_proj.weight","model.layers.9.self_attn.q_proj.weight","model.layers.9.self_attn.v_proj.weight"],"quantization":null,"ops":{"graph_type":"GraphOp","ops":1590,"by_op":{"coreai.constant":374,"coreai.transpose":242,"coreai.decomposable.broadcasting_batch_matmul":166,"coreai.reshape":143,"coreai.decomposable.broadcasting_mul":123,"coreai.slice":110,"coreai.invoke":85,"coreai.decomposable.broadcasting_add":76,"coreai.concat":59,"coreai.cast":52,"coreai.silu":30,"coreai.split":22,"coreai.conv2d":22,"coreai.read_handle":19,"coreai.write_handle":17,"coreai.slice_update":16,"coreai.shrink_dims":16,"coreai.decomposable.broadcasting_sub":2,"coreai.decomposable.broadcasting_maximum":2,"coreai.decomposable.broadcasting_minimum":2,"coreai.graph":1,"coreai.create_token":1,"coreai.get_shape":1,"coreai.decomposable.broadcasting_greater":1,"coreai.not":1,"coreai.gather_nd":1,"coreai.broadcast_in_dims":1,"coreai.gather_along_axis":1,"coreai.decomposable.broadcasting_where":1,"coreai.cos":1,"coreai.sin":1,"coreai.output":1},"printed_chars":202009,"method":"regex over str(prog.get_graph('main'))"},"seconds":{"load":8.237683915998787,"quantize":0.4091410000110045,"export":22.952836374985054,"optimize":0.04498308396432549,"save":5.360625708010048,"total":41.986240749945864},"du_aimodel":"5.2G","du_bundle":"5.2G","aimodel_files":{"main.hash":32,"main.mlirb":5562548035,"metadata.json":105},"main_mlirb":{"bytes":5562548035,"sha256":"d4ed7ae09f048ef7597310b03e1ab88934d9ce5cd4c9df3376e521b5a2b174bb"},"main_hash_hex":"d4ed7ae09f048ef7597310b03e1ab88934d9ce5cd4c9df3376e521b5a2b174bb","head":{"file":"option_rows.safetensors","bytes":17491440,"sha256":"e43cfaee023b4c1c2bab48b873aa2195bc6e5f86fe9f30049a89a1e0ad948522","n":2134,"hidden":2048,"readback":{"ids_equal":true,"rows_bit_equal":true},"source":{"hf_id":"LiquidAI/d1-3B","revision":"da1fe36a861f24690f27f622dca1d8688503d113","file":"model.safetensors","file_sha256":"50e03317847caf6df9a9aee27ed40f20554a86a21e60d1d47ba41a422b546c0c","key":"model.language_model.embed_tokens.weight","header":{"dtype":"BF16","shape":[128000,2048],"data_offsets":[0,524288000]},"data_start":97728,"conversion":"bf16 -> fp32 (exact)","tokenizer_json_sha256":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","candidates":{"A..Z":"26/26","a..z":"26/26","0..9":"10/10","00..99":"100/100","100..999":"900/900","#0..#199":"0/200","AA..ZZ":"587/676","' A..Z'":"26/26","' a..z'":"26/26","' 0..9'":"0/10","' 00..99'":"0/100","' 100..999'":"0/900","' #0..#199'":"0/200","' AA..ZZ'":"429/676","noul":"5/6"}}},"metadata_sha256":"0fb3d5b208dcc02df38448eb9429b6f4245742aed7ec11a24d0f2148dea32ecb","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","tokenizer_config.json":"ef6770d12dd9ac58d334e979bf160c5cdfd41f13d205fd5d5516857645feb0ea","chat_template.jinja":"86f4770449a4797c9b4212d110b0cc70fb993c1ce960095bcfb12eea22f61cca"},"tokenizer_verbatim_from_snapshot":true,"license_sha256":"4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c","disk":{"before_load":{"time":"2026-10-08T16:27:16+09:00","free_bytes":316391776256,"free_gib":294.7,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_load":{"time":"2026-10-08T16:27:24+09:00","free_bytes":316390952960,"free_gib":294.7,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_export":{"time":"2026-10-08T16:27:48+09:00","free_bytes":316323561472,"free_gib":294.6,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_save":{"time":"2026-10-08T16:27:53+09:00","free_bytes":305424015360,"free_gib":284.4,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc/d1_3b_decode_fp16_pf64.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":24.247334917075932,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_fp16_pf64/d1_3b_decode_fp16_pf64.aimodel","--output","$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-08T16:28:00+09:00","free_bytes":305117806592,"free_gib":284.2,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after":{"time":"2026-10-08T16:28:25+09:00","free_bytes":294421028864,"free_gib":274.2,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}}},"returncode":0,"stdout_tail":["Compiling h16c: 1 of 1"],"stderr_tail":[],"main_hash_hex":"3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17","runtime_cache_entry":"~/Library/Caches/coreai-cache/26A428/python/3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17","stats":{"operationDistribution":[{"name":"constant","count":1930},{"name":"slice","count":290},{"name":"reshape","count":274},{"count":243,"name":"transpose"},{"name":"concat","count":234},{"name":"broadcast_to","count":187},{"name":"batch_matmul","count":168},{"name":"broadcasting_batch_matmul","count":168},{"name":"mul","count":167},{"count":164,"name":"broadcasting_mul"},{"name":"broadcast_in_dims","count":159},{"name":"add","count":114},{"name":"broadcasting_add","count":110},{"count":85,"name":"invoke"},{"count":67,"name":"cast"},{"name":"broadcasting_divide","count":34},{"count":34,"name":"divide"},{"name":"broadcasting_sub","count":33},{"name":"sub","count":33},{"name":"exp","count":31},{"name":"silu","count":30},{"count":22,"name":"conv2d"},{"count":22,"name":"split"},{"count":19,"name":"read_handle"},{"count":17,"name":"write_handle"},{"count":16,"name":"get_shape"},{"name":"shrink_dims","count":16},{"count":16,"name":"slice_update"},{"count":9,"name":"broadcast_shapes"},{"name":"reduce","count":7},{"name":"gather_nd","count":4},{"count":4,"name":"reduce_sum"},{"count":3,"name":"gather_along_axis"},{"count":3,"name":"not"},{"name":"reduce_mean","count":3},{"count":3,"name":"reduce_product"},{"count":3,"name":"rsqrt"},{"name":"broadcasting_greater","count":2},{"count":2,"name":"broadcasting_maximum"},{"count":2,"name":"broadcasting_minimum"},{"count":2,"name":"expand_dims"},{"count":2,"name":"greater"},{"name":"maximum","count":2},{"count":2,"name":"minimum"},{"name":"range","count":2},{"name":"broadcasting_where","count":1},{"name":"cos","count":1},{"name":"create_token","count":1},{"name":"sin","count":1},{"name":"softmax","count":1},{"count":1,"name":"where"}],"computeTypes":["Bool","Float16","Float32","Int32","UInt32","UInt64"],"storageTypes":[{"name":"Float16","count":2613312517},{"name":"Float32","count":83886114},{"count":322,"name":"Int32"},{"count":85,"name":"UInt32"},{"name":"UInt64","count":1}]},"du_aimodelc":"9.9G","digest":{"bytes":10600779479,"tree_sha256":"63174182934412a3e4fbba7f3a0ed04d492bdd98c068d9f5d9a7adc2db782479","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"3127f44d7f075bab54c0fdbeffa2e0a00ab4a0d6c49ef224747216eff4522cf7","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"7ab03af9005f856b3ed2da5c2a4ddf5ef542dbdb79760cef9e1724b2c2662869","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"abeab6036db0831ea8daae3e7ef374523152c576d6db10cec619eb122aaa361a","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"830005a4bbf962d5a7c6cf9ed5cb012d4611ea21e5a9873d25fe695bd73b8fb0","main-h16c.mlirb":"3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17","main.hash":"09be89ec238fab77a3478e257ecdedbc17a7043c41880f651f18f7534b511cb3","metadata.json":"844ee4cdba8c65b05f13589f7ba44b10a30c9a57c762d718fa46a110394d64b6","stats.json":"8107be886f9b1c0247b4e471c9b7840b043051906694477b4eccc09b1d066c65"},"file_bytes":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":58571,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":318573,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":10599799820,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":597502,"main-h16c.mlirb":2848,"main.hash":32,"metadata.json":193,"stats.json":1940}}},"finished":"2026-10-08T16:28:30+09:00"},"readout":{"schema":"d1-decoder-readout-gate/1","gate":"the decoder alone on the Mac GPU (the oracle's row ids), the fp16 slot hidden read out through host.py","bundle":{"bundle":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_fp16_pf64","name":"d1_3b_decode_fp16_pf64","kind":"decision-backbone","compression":null,"prefill_chunk":64,"max_ctx":4096,"toy":null,"metadata_sha256":"0fb3d5b208dcc02df38448eb9429b6f4245742aed7ec11a24d0f2148dea32ecb","main_mlirb":{"bytes":5562548035,"sha256":"d4ed7ae09f048ef7597310b03e1ab88934d9ce5cd4c9df3376e521b5a2b174bb"},"tokenizer_sha256":{"chat_template.jinja":"86f4770449a4797c9b4212d110b0cc70fb993c1ce960095bcfb12eea22f61cca","tokenizer.json":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","tokenizer_config.json":"ef6770d12dd9ac58d334e979bf160c5cdfd41f13d205fd5d5516857645feb0ea"},"head_sha256":{"option_rows.json":"133b9d7150b4dc225da8893d6e052e78198714a11f18e42821738b3e6c1a5acf","option_rows.safetensors":"e43cfaee023b4c1c2bab48b873aa2195bc6e5f86fe9f30049a89a1e0ad948522"},"license_sha256":"4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c","aimodelc":{"path":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc/d1_3b_decode_fp16_pf64.h16c.aimodelc","bytes":10600779479,"tree_sha256":"63174182934412a3e4fbba7f3a0ed04d492bdd98c068d9f5d9a7adc2db782479"},"aimodelc_main_hash":"3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17"},"chunk":64,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options","reset":"every process re-runs its first row last: hidden rows bit-equal","finite":"every hidden value of every row finite, no row all zero","rows":"every expected row present"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0","tokenizers":"0.23.0rc0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","readout":"host.py: z = h_slot . E[id] in float64 over the group ids (head/option_rows, fp32), group max, softmax over the options","gpu":"shared with other sessions, _GPU_LOCK read only (the ms are contended reference values)"},"summary":{"runs":393,"expected_runs":393,"missing_runs":[],"questions":393,"argmax_equal":393,"questions_non_near_tie":392,"argmax_equal_non_near_tie":392,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.003938001526033352,"max_abs_dp_non_near_tie":0.003938001526033352,"mean_of_run_mean_abs_dp":0.00018400777830741502,"mean_question_max_abs_dp":0.00026774901064529136,"worst_run":{"id":"tv4s_16","name":"decision","max_abs_dp":0.003938001526033352},"finite_all":true,"all_zero_runs":0,"hidden_rows":14,"min_pos_cos":0.9999677870841723,"hidden_max_abs_diff":0.057308197021484375,"tokens":74504,"padded_tokens":88320,"calls":1380,"pad_waste":0.15643115942028984,"ms_per_call_median":34.36177095863968,"ms_per_call_warm_median":34.359687531832606},"by_source":{"card":{"runs":5,"expected_runs":5,"missing_runs":[],"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.00017047680369253193,"max_abs_dp_non_near_tie":0.00017047680369253193,"mean_of_run_mean_abs_dp":3.18884354659313e-05,"mean_question_max_abs_dp":4.6936821274445e-05,"worst_run":{"id":"card_refund","name":"urgency","max_abs_dp":0.00017047680369253193},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9999817602196224,"hidden_max_abs_diff":0.057308197021484375,"tokens":268,"padded_tokens":320,"calls":5,"pad_waste":0.1625,"ms_per_call_median":34.601874998770654,"ms_per_call_warm_median":34.601874998770654},"own":{"runs":36,"expected_runs":36,"missing_runs":[],"questions":36,"argmax_equal":36,"questions_non_near_tie":36,"argmax_equal_non_near_tie":36,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0007200608570492872,"max_abs_dp_non_near_tie":0.0007200608570492872,"mean_of_run_mean_abs_dp":7.281127390524976e-05,"mean_question_max_abs_dp":0.00011219921728365965,"worst_run":{"id":"own_fiveq_09","name":"request_followed","max_abs_dp":0.0007200608570492872},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.999987723779683,"hidden_max_abs_diff":0.057308197021484375,"tokens":18249,"padded_tokens":19520,"calls":305,"pad_waste":0.06511270491803278,"ms_per_call_median":34.49329105205834,"ms_per_call_warm_median":34.494166960939765},"own_long":{"runs":8,"expected_runs":8,"missing_runs":[],"questions":8,"argmax_equal":8,"questions_non_near_tie":8,"argmax_equal_non_near_tie":8,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0008319770606668442,"max_abs_dp_non_near_tie":0.0008319770606668442,"mean_of_run_mean_abs_dp":0.00015793572686840208,"mean_question_max_abs_dp":0.00022130298625980867,"worst_run":{"id":"long_34k","name":"cancellations","max_abs_dp":0.0008319770606668442},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9999677870841723,"hidden_max_abs_diff":0.057308197021484375,"tokens":19882,"padded_tokens":20096,"calls":314,"pad_waste":0.01064888535031847,"ms_per_call_median":34.78697902755812,"ms_per_call_warm_median":34.78697902755812},"semif":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":144,"questions_non_near_tie":144,"argmax_equal_non_near_tie":144,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0021829575071747875,"max_abs_dp_non_near_tie":0.0021829575071747875,"mean_of_run_mean_abs_dp":0.00020145048179849684,"mean_question_max_abs_dp":0.00030217572269774747,"worst_run":{"id":"semif_d51b05bfb9d6d7f09331","name":"answer","max_abs_dp":0.0021829575071747875},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9999942033909348,"hidden_max_abs_diff":0.057308197021484375,"tokens":12177,"padded_tokens":18368,"calls":287,"pad_waste":0.33705357142857145,"ms_per_call_median":34.28929194342345,"ms_per_call_warm_median":34.28787493612617},"tv4":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":59,"argmax_equal_non_near_tie":59,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.002702106589851039,"max_abs_dp_non_near_tie":0.002702106589851039,"mean_of_run_mean_abs_dp":0.00016711811524566025,"mean_question_max_abs_dp":0.0003205346486071876,"worst_run":{"id":"tv4_031","name":"answer","max_abs_dp":0.002702106589851039},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9999932391365841,"hidden_max_abs_diff":0.057308197021484375,"tokens":7969,"padded_tokens":9984,"calls":156,"pad_waste":0.20182291666666666,"ms_per_call_median":33.97058299742639,"ms_per_call_warm_median":33.97058299742639},"tv4s":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":20,"questions_non_near_tie":20,"argmax_equal_non_near_tie":20,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.003938001526033352,"max_abs_dp_non_near_tie":0.003938001526033352,"mean_of_run_mean_abs_dp":0.00031608226632903004,"mean_question_max_abs_dp":0.0004741233994935574,"worst_run":{"id":"tv4s_16","name":"decision","max_abs_dp":0.003938001526033352},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9999935669626713,"hidden_max_abs_diff":0.057308197021484375,"tokens":2436,"padded_tokens":2560,"calls":40,"pad_waste":0.0484375,"ms_per_call_median":33.953749982174486,"ms_per_call_warm_median":33.953749982174486},"tv4x":{"runs":120,"expected_runs":120,"missing_runs":[],"questions":120,"argmax_equal":120,"questions_non_near_tie":120,"argmax_equal_non_near_tie":120,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.002957088207970582,"max_abs_dp_non_near_tie":0.002957088207970582,"mean_of_run_mean_abs_dp":0.0001909443450137039,"mean_question_max_abs_dp":0.00022461025325132514,"worst_run":{"id":"tv4x_composition_holdout_15","name":"decision","max_abs_dp":0.002957088207970582},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.999993144524632,"hidden_max_abs_diff":0.057308197021484375,"tokens":13523,"padded_tokens":17472,"calls":273,"pad_waste":0.2260187728937729,"ms_per_call_median":34.18120904825628,"ms_per_call_warm_median":34.15204200427979}},"near_ties":[{"id":"tv4_013","name":"answer","oracle_top2_margin":0.01051943645398079,"argmax_equal":true,"max_abs_dp":0.00040011450731231024,"probs":[0.008882935492610093,0.010400798423212744,0.4854876435849487,0.4952286224992285]}],"checks":{"all_runs":true,"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_mean_abs_dp":true,"d_reset_bit_equal_all_processes":true,"e_finite":true,"red_required_arms_red":true,"red_graph_dp_equals_oracle_dp":true,"red_base_bit_equal_gate_runs":true},"result":"PASS","timing":{"contended":true,"S":64,"load_seconds":{"first_process":7.92180675000418,"median":4.234104582981672,"all":[7.92180675000418,0.808044874924235,7.421910290955566,0.8421641669701785,0.8637314999941736,9.540093832998537,0.9153531660558656,10.434901666012593,1.0462988750077784,9.200583625002764]},"ms_per_call_warm_runs":{"n":1354,"median":34.359687531832606,"p10":33.79362132400274,"p90":36.517903709318496},"ms_per_call_all_runs":{"n":1380,"median":34.36177095863968,"p10":33.79394584335387,"p90":40.42542978422717},"row_wall_seconds_warm_runs":{"n":383,"median":0.07286325003951788,"p10":0.0717319334158674,"p90":0.14020714145153762}},"seconds":{"gpu_processes":532.8244618749013,"scoring":0.2797640839125961,"total":533.2467927079415},"generated_at":"2026-10-08T07:38:19+00:00","note":"round 6a: chunk width S from the bundle's metadata; --compare-with the same mode's S=16 transcript (the rounding order changes with S); ms contended, not the ladder","load":{"start":{"at":"2026-10-08T16:29:26+09:00","loadavg":"{ 10.60 8.50 12.77 }","swapusage":"total = 7168.00M  used = 5483.69M  free = 1684.31M  (encrypted)"},"end":{"at":"2026-10-08T16:38:18+09:00","loadavg":"{ 12.56 8.95 10.41 }","swapusage":"total = 7168.00M  used = 5475.69M  free = 1692.31M  (encrypted)"}},"contract":{"function":"main","inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,-1],"int32"],"image_embeds":[[2816,2048],"float16"]},"outputs":{"hidden":[[1,64,2048],"float16"]},"states":{"keyCache":[[8,1,8,-1,64],"float16"],"valueCache":[[8,1,8,-1,64],"float16"],"convState":[[22,1,2048,2],"float16"]},"dynamic":"-1 = dynamic: position_ids [1, seq] with seq in S..max_context_length-1; the KV sequence axis 2048..max_context_length (a host allocates it at max_context_length)"},"oracle":{"path":"$ZOO_WORK_ROOT/_d1_3b/oracle/records_oracle.json","sha256":"067a4f5658937da0e20e307deaa15a38deb55d0f12b2f9993b609461d0430af1","schema":"d1-oracle/1","toy":null,"hidden_dir":"$ZOO_WORK_ROOT/_d1_3b/oracle/hidden"},"runtime_cache_entry":{"path":"~/Library/Caches/coreai-cache/26A428/python/3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17","exists":true,"kib":10352340},"descriptor_same_in_every_process":true,"red_rule":"red = the perturbed rows against their base rows fail the gate's own bar (FACTS §7): (a) an argmax moves on a question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over the arm's rows of the row's mean |dp| > 0.002","script":{"path":"conversion/d1/readout_gate.py","sha256":"713ba1e75f03beca460edb8a70ff5f7590fa534c109a8b9d28d904ba3afa6515"},"runs":[{"id":"tv4_000","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.967055480182726e-05,"mean_abs_dp":2.9835277400894113e-05,"oracle_top2_margin":0.9175063266514067,"near_tie":false,"n_options":4,"probs":[0.012278654065537022,0.030389674486733683,0.9479918105347979,0.009339860912931474],"hidden_sha256":"920b2985f95e9b42eb7a93b6d347f2959084573074ef6fc621f1ddea0dfd7698","first_in_process":true,"shard":"shard_00","index":0,"hidden":{"min_pos_cos":0.9999932391365841,"min_pos_cos_index":96,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4_001","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.0175534518607208e-05,"mean_abs_dp":1.008776725930008e-05,"oracle_top2_margin":0.9880834835402893,"near_tie":false,"n_options":4,"probs":[0.0011564291320507315,0.0030649005979227193,0.9919467820819102,0.0038318881881162898],"hidden_sha256":"2be029505c60d3b2900a078e4561cecb1a19406d7772c2580317cff5822e048d","first_in_process":false,"shard":"shard_00","index":1},{"id":"tv4_002","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.2378731218854952e-05,"mean_abs_dp":6.1893656094118635e-06,"oracle_top2_margin":0.9589307483192843,"near_tie":false,"n_options":4,"probs":[0.9731254585085373,0.0052785401589175085,0.014203343252693467,0.007392658079851795],"hidden_sha256":"a840ec6df5a84c9c376e82984c1e66f505993ad8a0812337b3de093083f1aa40","first_in_process":false,"shard":"shard_00","index":2},{"id":"tv4_003","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.29857122444755e-05,"mean_abs_dp":5.17876641313959e-05,"oracle_top2_margin":0.8790862718882508,"near_tie":false,"n_options":4,"probs":[0.9198334848974701,0.04092974686133629,0.029832238371388625,0.00940452986980487],"hidden_sha256":"a1d0087dab2cc60ed9b1435fb958699be8913aa5c6b79dfb38cf7dcd4ff2ba4c","first_in_process":false,"shard":"shard_00","index":3},{"id":"tv4_004","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":3.4379300668296864e-05,"mean_abs_dp":1.718965033416784e-05,"oracle_top2_margin":0.9788776201055156,"near_tie":false,"n_options":4,"probs":[0.0029011714115998863,0.00476326850651488,0.9855790032016438,0.006756556880241421],"hidden_sha256":"4089d9f605fa8d6361b29909178a50be568f962499c5db34517bbbb9964a2d6d","first_in_process":false,"shard":"shard_00","index":4},{"id":"tv4_005","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.007332563357377e-05,"mean_abs_dp":2.0036662816794257e-05,"oracle_top2_margin":0.9301866143096604,"near_tie":false,"n_options":4,"probs":[0.9589055821928747,0.02864539577568856,0.008210920524387374,0.004238101507049422],"hidden_sha256":"b7f92098231d18285d173b8afcb9d5d96e1e37fec52ee2c4d580326e6e5b4489","first_in_process":false,"shard":"shard_00","index":5},{"id":"tv4_006","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":249,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002006292140671953,"mean_abs_dp":0.00014593624594762527,"oracle_top2_margin":0.5148134998751737,"near_tie":false,"n_options":4,"probs":[0.04289392491847002,0.7215465064078608,0.02878153428530951,0.20677803438835965],"hidden_sha256":"43c1c08aeb4a99c490fffebfc81167a25991d793d6a0317455b0c6cd153ad3e4","first_in_process":false,"shard":"shard_00","index":6},{"id":"tv4_007","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00034829425755977805,"mean_abs_dp":0.00017903946771418984,"oracle_top2_margin":0.7992202960971314,"near_tie":false,"n_options":4,"probs":[0.023356839709381592,0.08467378692227344,0.8845473061120761,0.007422067256268974],"hidden_sha256":"c3913f9ecd9019d921ce60368f1337609227cfc3ebc6d45a199eb7d22c477648","first_in_process":false,"shard":"shard_00","index":7},{"id":"tv4_008","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":113,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002668971960996247,"mean_abs_dp":0.00013344859804983403,"oracle_top2_margin":0.8940057521242946,"near_tie":false,"n_options":4,"probs":[0.04695963084270544,0.9404691120459661,0.004616300194915986,0.00795495691641258],"hidden_sha256":"c46e90dd51eb5313db17de839ec4b5a04bc3b80567e6fb384b518e73c1ae5969","first_in_process":false,"shard":"shard_00","index":8},{"id":"tv4_009","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":339,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.1102317089006775e-05,"mean_abs_dp":5.55115854447509e-06,"oracle_top2_margin":0.9850214404934367,"near_tie":false,"n_options":4,"probs":[0.0021453089408164923,0.003962005899630948,0.9894484240290693,0.004444261130483219],"hidden_sha256":"51f10d5d6aa746f28c37db4da0a9ebaaf92ba05b1b40e5325d6c529344eba05a","first_in_process":false,"shard":"shard_00","index":9},{"id":"tv4_010","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000335829945334154,"mean_abs_dp":0.00021939454254259888,"oracle_top2_margin":0.1758312758643772,"near_tie":false,"n_options":4,"probs":[0.395350471814418,0.21392744390845214,0.1718735540131782,0.21884853026395173],"hidden_sha256":"24167194fa554bd3f170df294e158023321bdc4ca7141fc0621be0c64e0ed59f","first_in_process":false,"shard":"shard_00","index":10},{"id":"tv4_011","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":129,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00016783019215305117,"mean_abs_dp":8.3915096076536e-05,"oracle_top2_margin":0.0715867449236845,"near_tie":false,"n_options":4,"probs":[0.017223033154964318,0.04268407415063492,0.43409960837505207,0.5059932843193488],"hidden_sha256":"74fb1df2587de86dda81da02714bbf38082a1688d18ea58edf280d2db8cc35c6","first_in_process":false,"shard":"shard_00","index":11},{"id":"tv4_012","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":9.722304638726254e-05,"mean_abs_dp":5.071650159269611e-05,"oracle_top2_margin":0.92417841818824,"near_tie":false,"n_options":4,"probs":[0.02463104463696939,0.004975223311234988,0.021410353398546658,0.948983378653249],"hidden_sha256":"1836327884f9f03539226142361e01c14f5d13e11e2283d1b6e91ef031bfc426","first_in_process":false,"shard":"shard_00","index":12},{"id":"tv4_013","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":183,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00040011450731231024,"mean_abs_dp":0.00020005725365611696,"oracle_top2_margin":0.01051943645398079,"near_tie":true,"n_options":4,"probs":[0.008882935492610093,0.010400798423212744,0.4854876435849487,0.4952286224992285],"hidden_sha256":"38dfd87bd11bc300eec1bcd8f96fc7bc6eabe92bd8c479ce6b8a3196bdf416f8","first_in_process":false,"shard":"shard_00","index":13},{"id":"tv4_014","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":4.860337306811929e-05,"mean_abs_dp":2.7270626984252522e-05,"oracle_top2_margin":0.8982278774496655,"near_tie":false,"n_options":4,"probs":[0.03639545381755646,0.020643162509288507,0.934718795807757,0.00824258786539815],"hidden_sha256":"6baf9a4bdd4045f94be23964b1ae82594f3a73e9b3790508713a1786d2a01ebe","first_in_process":false,"shard":"shard_00","index":14},{"id":"tv4_015","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":194,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018714929091334653,"mean_abs_dp":9.357464545667804e-05,"oracle_top2_margin":0.7573631279404294,"near_tie":false,"n_options":4,"probs":[0.8599115699761357,0.028185069434286815,0.10229005194530605,0.009613308644271377],"hidden_sha256":"0e4e1d7b0b052e48a3eae444bffac1e3f30b834bd38b5818d8c971eb21e4ae41","first_in_process":false,"shard":"shard_00","index":15},{"id":"tv4_016","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":7.623177789716634e-05,"mean_abs_dp":4.138073180802405e-05,"oracle_top2_margin":0.8801963176754999,"near_tie":false,"n_options":4,"probs":[0.010452355244417972,0.011062981156636682,0.049084798064292806,0.9293998655346526],"hidden_sha256":"047c972277fb3f36a3bb8600a78edcf86f96ac9e4b59d1b51ae03dc6b9fa8379","first_in_process":false,"shard":"shard_00","index":16},{"id":"tv4_017","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.771439533481693e-05,"mean_abs_dp":3.38571976674115e-05,"oracle_top2_margin":0.955853033961402,"near_tie":false,"n_options":4,"probs":[0.004066542322866343,0.014593233679943127,0.9705528225227483,0.010787401474442137],"hidden_sha256":"9b08eb36a5d647d76abeb57e7c497ec8471b92c11e91a22f57a1d84a30678b86","first_in_process":false,"shard":"shard_00","index":17},{"id":"tv4_018","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.0448937359014465e-06,"mean_abs_dp":3.0704293184965342e-06,"oracle_top2_margin":0.9849916555226311,"near_tie":false,"n_options":4,"probs":[0.0013990570237260788,0.0047933396012887295,0.9897930850337561,0.004014518341229169],"hidden_sha256":"999b134013de911344f7785ddc04ffecd48a4a13bc57a5e657f02262e03fda6d","first_in_process":false,"shard":"shard_00","index":18},{"id":"tv4_019","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":150,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001235714254065906,"mean_abs_dp":0.0006682831286560949,"oracle_top2_margin":0.5069606504483266,"near_tie":false,"n_options":4,"probs":[0.12663986123079923,0.6469416062205012,0.13883232240614019,0.08758621014255939],"hidden_sha256":"e1bd3355e7d8a5758bd2c2a497ff6711977ae1a395cd41419c9d02f9dd065bcd","first_in_process":false,"shard":"shard_00","index":19},{"id":"tv4_020","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0011586458255319343,"mean_abs_dp":0.0005797507975949081,"oracle_top2_margin":0.43055736666492805,"near_tie":false,"n_options":4,"probs":[0.015942007294819833,0.07457710927375218,0.6689432950831161,0.24053758834831188],"hidden_sha256":"675d84c83bb082cabad4d994f48403658f8fe25d20395c9d0168065f1a7eceee","first_in_process":false,"shard":"shard_00","index":20},{"id":"tv4_021","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004139431173018904,"mean_abs_dp":0.0002069715586509209,"oracle_top2_margin":0.26756356697958295,"near_tie":false,"n_options":4,"probs":[0.21914105294359032,0.12107808861706139,0.48599692317396376,0.17378393526538458],"hidden_sha256":"2b9b9a50aa6b3a954826e96b19a9920798dc12ab8021116575cb02720f752f9b","first_in_process":false,"shard":"shard_00","index":21},{"id":"tv4_022","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":154,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003374236916355855,"mean_abs_dp":0.00017918084526309663,"oracle_top2_margin":0.10257574535656738,"near_tie":false,"n_options":4,"probs":[0.26874271385380377,0.38589026994546943,0.283853471879819,0.06151354432090779],"hidden_sha256":"0be381ee1489f483c0cb5e9022e888957493316bc5f8448538632d701d6d53fd","first_in_process":false,"shard":"shard_00","index":22},{"id":"tv4_023","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":107,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018568203496521773,"mean_abs_dp":0.00011401595945796303,"oracle_top2_margin":0.4294777143114449,"near_tie":false,"n_options":4,"probs":[0.5977788921309886,0.16798961571987397,0.10705678403609922,0.12717470811303824],"hidden_sha256":"79d22e3deb22d29635292c99914ca6225dbeee71087458f8fab054b92663e29a","first_in_process":false,"shard":"shard_00","index":23},{"id":"tv4_024","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0005315202110872663,"mean_abs_dp":0.0002657601055436193,"oracle_top2_margin":0.6636511103865234,"near_tie":false,"n_options":4,"probs":[0.09529000507033476,0.10263123279884671,0.03646841888166405,0.7656103432491544],"hidden_sha256":"9893c787fc9866f4856f581bc3b67d12aaec0a00bb928105aeac2e806a701638","first_in_process":false,"shard":"shard_00","index":24},{"id":"tv4_025","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":193,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00026109170088689515,"mean_abs_dp":0.0001593700783957576,"oracle_top2_margin":0.04727997159987629,"near_tie":false,"n_options":4,"probs":[0.285029862967323,0.34132915060450775,0.29358044999633603,0.08006053643183306],"hidden_sha256":"859d706902605bd6da47ad5cd1e4239f0e8efa0c2ba3bff50dace6135940f2f8","first_in_process":false,"shard":"shard_00","index":25},{"id":"tv4_026","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":109,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.8454967534452535e-06,"mean_abs_dp":1.422748376741546e-06,"oracle_top2_margin":0.9854219242740557,"near_tie":false,"n_options":4,"probs":[0.0018302240048755668,0.004111137332440252,0.9897421056865721,0.004316532976111985],"hidden_sha256":"9bd87dbce65954201a7802229ab660043a5be296ca05a9dd4a7800e9745d0b1c","first_in_process":false,"shard":"shard_00","index":26},{"id":"tv4_027","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008539092772172352,"mean_abs_dp":0.00042702664384867907,"oracle_top2_margin":0.02766328769190496,"near_tie":false,"n_options":4,"probs":[0.0058304706993581356,0.41181322893537,0.1993175295999239,0.3830387707653479],"hidden_sha256":"f75b7552a9df7428f2918bd410d0110575af31cfe4cae67bdce3d896094b01b5","first_in_process":false,"shard":"shard_00","index":27},{"id":"tv4_028","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":316,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004496707158329616,"mean_abs_dp":0.00027549703678305185,"oracle_top2_margin":0.08139784238225001,"near_tie":false,"n_options":4,"probs":[0.18183010235157068,0.33212764755424,0.41285008210048957,0.07319216799369968],"hidden_sha256":"e44c7cf71fffabb5dc7afd30fdfeee4cb4694ae32a1d403adc182294b36b93a2","first_in_process":false,"shard":"shard_00","index":28},{"id":"tv4_029","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00047767280178567795,"mean_abs_dp":0.0002388364008928355,"oracle_top2_margin":0.7765665787029383,"near_tie":false,"n_options":4,"probs":[0.844354946578135,0.04039012091764989,0.04690792407641241,0.0683470084278026],"hidden_sha256":"2ea732328645fa570755c67ff0d37b349261b3cccbea94173f63c0b573d5133b","first_in_process":false,"shard":"shard_00","index":29},{"id":"tv4_030","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.749623422366071e-05,"mean_abs_dp":4.374811711185507e-05,"oracle_top2_margin":0.9245924496230566,"near_tie":false,"n_options":4,"probs":[0.008611997456101098,0.025260956161074544,0.9499746619529774,0.01615238442984693],"hidden_sha256":"2744956f1cddd448571fee7ee4ed76b10fd0fffa1d110b1f4db954ca80db1f26","first_in_process":false,"shard":"shard_00","index":30},{"id":"tv4_031","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002702106589851039,"mean_abs_dp":0.0013510532949255393,"oracle_top2_margin":0.12507160244912263,"near_tie":false,"n_options":4,"probs":[0.554417196514493,0.43474160885944557,0.006884417451257974,0.003956777174803441],"hidden_sha256":"9f5c26f98ad6d3c4410570afafa181368f35febaf079cd4543a1822766fffb69","first_in_process":false,"shard":"shard_00","index":31},{"id":"tv4_032","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00010624372375378499,"mean_abs_dp":5.3121861876870486e-05,"oracle_top2_margin":0.9326063916622778,"near_tie":false,"n_options":4,"probs":[0.003425941597154515,0.008590816794790557,0.027778149915869556,0.9602050916921853],"hidden_sha256":"44d389d6efb4fcd5d11372a8540f7b4d07b8ecfbd8624a8afce70e1859421394","first_in_process":false,"shard":"shard_00","index":32},{"id":"tv4_033","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":311,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":2.3971806760947345e-05,"mean_abs_dp":1.1985903380447652e-05,"oracle_top2_margin":0.9615159028252174,"near_tie":false,"n_options":4,"probs":[0.003551741529768449,0.0056471692000324295,0.01466309836665114,0.9761379909035479],"hidden_sha256":"9d8a4b6821a7d788bb413cd3414069f4746e51e8a1f596f80e07c1b2b2839f87","first_in_process":false,"shard":"shard_00","index":33},{"id":"tv4_034","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0002655330039398329,"mean_abs_dp":0.0001405512946625921,"oracle_top2_margin":0.2533182700310583,"near_tie":false,"n_options":4,"probs":[0.08337395616484372,0.12549980338945926,0.2690410165756689,0.5220852238700281],"hidden_sha256":"b89f31ea64c63e0b016e57ec4a30a673a44b6bb5fff987f31da0e15acc1661b7","first_in_process":false,"shard":"shard_00","index":34},{"id":"tv4_035","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":126,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004963145830121918,"mean_abs_dp":0.0002549408644390203,"oracle_top2_margin":0.23375552424918428,"near_tie":false,"n_options":4,"probs":[0.36394044590173313,0.029321112423166974,0.596710501581347,0.010027940093752944],"hidden_sha256":"8e508d24a812e5008ebbac0f1850c7e8c1b57902085a0234e315e1fe47ec4f16","first_in_process":false,"shard":"shard_00","index":35},{"id":"tv4_036","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":156,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002188314600011232,"mean_abs_dp":0.00010941573000058588,"oracle_top2_margin":0.12129537667015589,"near_tie":false,"n_options":4,"probs":[0.15911346217510006,0.4433204586324602,0.32233870135006665,0.07522737784237297],"hidden_sha256":"6f65ff4e49bec3ce87904328cedabc55949027c6a6ca8e0b5657385cd0e4a4d6","first_in_process":false,"shard":"shard_00","index":36},{"id":"tv4_037","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001106783830310265,"mean_abs_dp":5.533919151552236e-05,"oracle_top2_margin":0.6711215583174963,"near_tie":false,"n_options":4,"probs":[0.015286384904964562,0.07314154485736298,0.7912667383754971,0.1203053318621754],"hidden_sha256":"756512a8aaf9dbf5f64aaceceec6a8a999992c7123d3ebfe68a2a92b7a5d1a57","first_in_process":false,"shard":"shard_00","index":37},{"id":"tv4_038","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.802737258104763e-05,"mean_abs_dp":3.4598805064895875e-05,"oracle_top2_margin":0.9219558028864439,"near_tie":false,"n_options":4,"probs":[0.002608072562448326,0.028241604229897112,0.950295363593685,0.018854959613969442],"hidden_sha256":"2634c5ea63491239ba6631d7347c27276ddbc449b1a00642cf74eccba6ce51f3","first_in_process":false,"shard":"shard_00","index":38},{"id":"tv4_039","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":87,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004989780456401416,"mean_abs_dp":0.00024948902282008836,"oracle_top2_margin":0.5889692850440698,"near_tie":false,"n_options":4,"probs":[0.7903157688407301,0.20037557336328626,0.00567825894734184,0.003630398848641799],"hidden_sha256":"90a10e0a051d84741113a1d0384b7fcc2a1f945d502dc55c3d8280b3f438b2a6","first_in_process":false,"shard":"shard_00","index":39},{"id":"tv4_040","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":222,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0007399666439259489,"mean_abs_dp":0.0003699833219629606,"oracle_top2_margin":0.2663605121271405,"near_tie":false,"n_options":4,"probs":[0.053333282085219584,0.31419193430581843,0.050766200580085366,0.5817085830288766],"hidden_sha256":"f8688f3e5c689b517348da0336dee4ede8a7690ba15c2a66d35afdd567c545a3","first_in_process":true,"shard":"shard_01","index":0},{"id":"tv4_041","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":229,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007013927214962745,"mean_abs_dp":0.00035069636074813726,"oracle_top2_margin":0.4427181728774944,"near_tie":false,"n_options":4,"probs":[0.0766884143157115,0.1732416878956411,0.6166707961625153,0.13339910162613206],"hidden_sha256":"b4685d18ddc8a8d4ca9863120ce19e1ed92c0002caf3acb234c5c0f2b078969e","first_in_process":false,"shard":"shard_01","index":1},{"id":"tv4_042","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.372437714954106e-05,"mean_abs_dp":4.1862188574820405e-05,"oracle_top2_margin":0.8585390325266999,"near_tie":false,"n_options":4,"probs":[0.9148341560454468,0.05620383178761728,0.01403608793095778,0.014925924235978008],"hidden_sha256":"02d3fa91360c16bab6ddcce2aeb2455ea64428e558a9d321d5bc192290e11e03","first_in_process":false,"shard":"shard_01","index":2},{"id":"tv4_043","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005862436675126981,"mean_abs_dp":0.0002931218337563733,"oracle_top2_margin":0.5045756833909496,"near_tie":false,"n_options":4,"probs":[0.03250869664407037,0.7003665455356092,0.07020895738068504,0.1969158004396355],"hidden_sha256":"614ef21dc6d4e3be04003d4c257276c75dd2f853b0b27b5d1e0e94704935acb5","first_in_process":false,"shard":"shard_01","index":3},{"id":"tv4_044","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":2.1713885500359263e-06,"mean_abs_dp":1.2107367862015722e-06,"oracle_top2_margin":0.9832281448196667,"near_tie":false,"n_options":4,"probs":[0.001567362481961449,0.004597183110812893,0.0053045835733921305,0.9885308708338335],"hidden_sha256":"c6cd50015e28728d54f39cc2b61a3de01806847ddecd80c696c0cab5e7cb4249","first_in_process":false,"shard":"shard_01","index":4},{"id":"tv4_045","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":112,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00025598838464802043,"mean_abs_dp":0.00014365167962300356,"oracle_top2_margin":0.3822788183711481,"near_tie":false,"n_options":4,"probs":[0.015693469872277693,0.20288933904174844,0.584711823300135,0.1967053677858388],"hidden_sha256":"8d406a7201df429aa825a9ea933052ffbe7a1b5edd19b67bf18728795c6a4f48","first_in_process":false,"shard":"shard_01","index":5},{"id":"tv4_046","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.8533165153610263e-05,"mean_abs_dp":1.4266582576818792e-05,"oracle_top2_margin":0.9119621800258289,"near_tie":false,"n_options":4,"probs":[0.9511563031161523,0.006895591136839934,0.03915282400816212,0.002795281738845562],"hidden_sha256":"0ebd56c9797cbcb7a95769260898a199fcacc706bbb5fd827e1767cc1a050957","first_in_process":false,"shard":"shard_01","index":6},{"id":"tv4_047","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002504382680203199,"mean_abs_dp":0.00012764627650748567,"oracle_top2_margin":0.6958231902791943,"near_tie":false,"n_options":4,"probs":[0.02307026517036586,0.12574391193350296,0.8220623155649257,0.029123507331205407],"hidden_sha256":"fb8a0601ac936bd765c3838babdf2b9e003a7ff3f6f43b958ffe8df5944cb076","first_in_process":false,"shard":"shard_01","index":7},{"id":"tv4_048","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":141,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00016815181944329627,"mean_abs_dp":8.702987902518616e-05,"oracle_top2_margin":0.9368998612950754,"near_tie":false,"n_options":4,"probs":[0.009364615030567685,0.960627435660146,0.023473008496092962,0.006534940813193377],"hidden_sha256":"761241eb40f42aa28315111fb30d1235ad20ca0656b4bc03868e36a6985f818b","first_in_process":false,"shard":"shard_01","index":8},{"id":"tv4_049","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":7.167211598646528e-05,"mean_abs_dp":3.8282168968632865e-05,"oracle_top2_margin":0.8649083955667569,"near_tie":false,"n_options":4,"probs":[0.0022242796929883833,0.0596079494866399,0.013787652896730236,0.9243801179236414],"hidden_sha256":"6021929a093c4a9577688f2af6d728cc38808aecf643a320d7d2cd145ff0fe86","first_in_process":false,"shard":"shard_01","index":9},{"id":"tv4_050","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00016258078810360566,"mean_abs_dp":8.129039405183145e-05,"oracle_top2_margin":0.407839880593036,"near_tie":false,"n_options":4,"probs":[0.2786338040940451,0.6867610926533837,0.013232274502803872,0.02137282874976728],"hidden_sha256":"10f835dff70276a1ac51328aa4d349106dfa5ca5adaad298dc135f16db8ab8db","first_in_process":false,"shard":"shard_01","index":10},{"id":"tv4_051","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0003497443003569467,"mean_abs_dp":0.00017487215017848528,"oracle_top2_margin":0.6243594070338851,"near_tie":false,"n_options":4,"probs":[0.007420923748178998,0.028622147894202005,0.17013504545647676,0.7938218829011423],"hidden_sha256":"035f268007095c095fab8b2aa637828113f7fba6f96738c4d4e12458f6c267d6","first_in_process":false,"shard":"shard_01","index":11},{"id":"tv4_052","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":146,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010625714708611955,"mean_abs_dp":5.312857354309295e-05,"oracle_top2_margin":0.8549797972832431,"near_tie":false,"n_options":4,"probs":[0.9189342565396695,0.06415349563538314,0.0070279948311083576,0.009884252993839021],"hidden_sha256":"34d65e5b06d9a5d567b01b93b262104506251194f499cadf6a26f2fc38e2dd79","first_in_process":false,"shard":"shard_01","index":12},{"id":"tv4_053","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":370,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.988224442463828e-05,"mean_abs_dp":3.1384890772858005e-05,"oracle_top2_margin":0.7270944760158131,"near_tie":false,"n_options":4,"probs":[0.024946691686156732,0.8417591940525468,0.01854992064931837,0.11474419361197807],"hidden_sha256":"379495856ccbf58a38ca9a223cd1b174a11ffde81c585241d5bfe82c8721f000","first_in_process":false,"shard":"shard_01","index":13},{"id":"tv4_054","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":114,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003670921162240437,"mean_abs_dp":0.0002541216875096633,"oracle_top2_margin":0.050236600003381204,"near_tie":false,"n_options":4,"probs":[0.1281805414815234,0.33255187013659426,0.2575511859253488,0.2817164024565335],"hidden_sha256":"65b220770b31e966c7c403b3c01c2661ea1fdc5726f04d2f437869f85ac7059f","first_in_process":false,"shard":"shard_01","index":14},{"id":"tv4_055","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":174,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005678370279930678,"mean_abs_dp":0.00028964675867225453,"oracle_top2_margin":0.034482426730795546,"near_tie":false,"n_options":4,"probs":[0.2558063798558529,0.3447529491197904,0.31129773848375125,0.08814293254060539],"hidden_sha256":"b5c93640a1a5d4faae6e826598f4decd7a42d8ed349a1c223a59b37638a6a12b","first_in_process":false,"shard":"shard_01","index":15},{"id":"tv4_056","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":5.7197517136176046e-05,"mean_abs_dp":2.859875856808542e-05,"oracle_top2_margin":0.929137078840534,"near_tie":false,"n_options":4,"probs":[0.0036645157378785652,0.017698057858917135,0.024800474283918027,0.9538369521192862],"hidden_sha256":"9c2abcf26c1f44f42dce60df4b35e5ce966b00d12806840a613f74fe1b014c78","first_in_process":false,"shard":"shard_01","index":16},{"id":"tv4_057","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0008196770120304508,"mean_abs_dp":0.0004098385060152315,"oracle_top2_margin":0.46040710392162176,"near_tie":false,"n_options":4,"probs":[0.015838259721235766,0.04576170444175482,0.23973681539334582,0.6986632204436636],"hidden_sha256":"a1a5e53140f2d4bde60ca546b65ae638903ef1fc9847f53a8548a55ee87d4db3","first_in_process":false,"shard":"shard_01","index":17},{"id":"tv4_058","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":152,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008898472318427109,"mean_abs_dp":0.00045172935300100484,"oracle_top2_margin":0.3868433028253313,"near_tie":false,"n_options":4,"probs":[0.07252327597828262,0.26697712494422726,0.6522050703567212,0.00829452872076899],"hidden_sha256":"fcf99f7a4f54f5780a05130205b606cfbb7cfae4cc0af734276f79eab7493799","first_in_process":false,"shard":"shard_01","index":18},{"id":"tv4_059","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":116,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.405308345110324e-05,"mean_abs_dp":1.2026541725503265e-05,"oracle_top2_margin":0.9575450347147793,"near_tie":false,"n_options":4,"probs":[0.9730305482453451,0.007071494827120832,0.0044428476083007385,0.015455109319233428],"hidden_sha256":"c84250edf4d2840c9379bb7efc65c793a533d6fda0d93567201491db16531b42","first_in_process":false,"shard":"shard_01","index":19},{"id":"tv4x_emotion_00","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":57,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006272844685603074,"mean_abs_dp":0.00022055478681566482,"oracle_top2_margin":0.35913577951608466,"near_tie":false,"n_options":6,"probs":[0.1392300294408167,0.22038481406896207,0.5783037923132598,0.025746649754957596,0.01881920895767744,0.017515505464326387],"hidden_sha256":"b136c9a916cbb810df77fb1cd0b27e983281fd4cd75dccaaced03057625051e3","first_in_process":false,"shard":"shard_01","index":20},{"id":"tv4x_emotion_01","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00048619251783765804,"mean_abs_dp":0.0001819536784342738,"oracle_top2_margin":0.06860383828577754,"near_tie":false,"n_options":6,"probs":[0.2605709675501977,0.32840177735902315,0.037054340974504774,0.17286278086083212,0.07785573595728913,0.12325439729815318],"hidden_sha256":"dbe8f18060e0353f97097ff37e9035f66e7a39c91e471712d64ed925a0aae21f","first_in_process":false,"shard":"shard_01","index":21},{"id":"tv4x_emotion_02","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":51,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011274763800905863,"mean_abs_dp":3.7582546003016e-05,"oracle_top2_margin":0.8937235386954242,"near_tie":false,"n_options":6,"probs":[0.9239259008132776,0.00887889927166791,0.006025920730873223,0.027236315007863356,0.03004704001962193,0.0038859241566959455],"hidden_sha256":"3028dcfe996777a0f56892a3c3924afeae8826d4bbcd78d7d1773e2efd7efee4","first_in_process":false,"shard":"shard_01","index":22},{"id":"tv4x_emotion_03","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004167270509275456,"mean_abs_dp":0.00013933640043964235,"oracle_top2_margin":0.7205916931033775,"near_tie":false,"n_options":6,"probs":[0.8166458638313606,0.09529661500367366,0.07846458674058691,0.0035695509719020795,0.0037183345353429954,0.0023050489171336725],"hidden_sha256":"5c415f9c8a3c758f5ab6bd5a513bea5e98a1139a6b35af18f4a3785ac62d5249","first_in_process":false,"shard":"shard_01","index":23},{"id":"tv4x_emotion_04","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.6130652232755374e-05,"mean_abs_dp":1.8710217410933927e-05,"oracle_top2_margin":0.9436175051795138,"near_tie":false,"n_options":6,"probs":[0.009803308045087531,0.9629262873702715,0.019388471922958977,0.003236462660189437,0.0018615634581403413,0.0027839065433521715],"hidden_sha256":"e91398d5a79b114da0d458c7f368aca84de7560a51e2a441e4bc48ac66a5564c","first_in_process":false,"shard":"shard_01","index":24},{"id":"tv4x_emotion_05","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0001664967420739183,"mean_abs_dp":5.871847390558635e-05,"oracle_top2_margin":0.5267872368274699,"near_tie":false,"n_options":6,"probs":[0.12882020315921144,0.15754180840290297,0.014985988713731009,0.684598810301487,0.008022376213036402,0.006030813209631286],"hidden_sha256":"9eb3ce613c0aba051b82b46e3647a9d7d6b1f8448574fc3754ecd9fc8099fbf8","first_in_process":false,"shard":"shard_01","index":25},{"id":"tv4x_emotion_06","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":52,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001351822572492889,"mean_abs_dp":4.529047463164437e-05,"oracle_top2_margin":0.4531383187251465,"near_tie":false,"n_options":6,"probs":[0.012585520959507216,0.2592017195654352,0.7120749662081598,0.006717253093637844,0.004777689470187829,0.004642850703072127],"hidden_sha256":"3eaa09c3f080bc172c6d9a20019e02efce3ed240f5500948e700f24ff8e35478","first_in_process":false,"shard":"shard_01","index":26},{"id":"tv4x_emotion_07","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00041939619386999194,"mean_abs_dp":0.00013979873129001827,"oracle_top2_margin":0.8020359796556499,"near_tie":false,"n_options":6,"probs":[0.882493773478836,0.015189429067367154,0.007624044997456859,0.08118715356268373,0.009040754916206739,0.004464843977449571],"hidden_sha256":"461ef698fe3f64bd5a3c37885b04af5ad8b9b1f2e6c27bc0e205115fcd9742b0","first_in_process":false,"shard":"shard_01","index":27},{"id":"tv4x_emotion_08","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011646469110528079,"mean_abs_dp":4.3453638227554436e-05,"oracle_top2_margin":0.4470593629065237,"near_tie":false,"n_options":6,"probs":[0.22241785548773266,0.6695975894607339,0.09863387844719029,0.003958740321865926,0.0025063762179549174,0.0028855600645222895],"hidden_sha256":"c3b1a36b9d9e8f275fcc603431f62497860dbad0fe244f951a342406b6b3f0af","first_in_process":false,"shard":"shard_01","index":28},{"id":"tv4x_emotion_09","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003968591962635326,"mean_abs_dp":0.00013228639875452879,"oracle_top2_margin":0.7069872082119006,"near_tie":false,"n_options":6,"probs":[0.7932538623084846,0.038360176179125266,0.08577736271852034,0.061995980716162115,0.007657572056665102,0.012955046021042534],"hidden_sha256":"6c85912915e656cf48a430734a3ecded1f87d1fc3520cfe8246e5ed5ca86499e","first_in_process":false,"shard":"shard_01","index":29},{"id":"tv4x_emotion_10","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":56,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002954275517480509,"mean_abs_dp":9.847585058271313e-05,"oracle_top2_margin":0.4519727177002277,"near_tie":false,"n_options":6,"probs":[0.6886250323019475,0.03313586999496217,0.01739947590543537,0.23619021394266754,0.013966179885972967,0.010683227969014374],"hidden_sha256":"524cf3b65b3da6fb7052cc633e28d642a9424fc2ce3e65726969526722f1bff4","first_in_process":false,"shard":"shard_01","index":30},{"id":"tv4x_emotion_11","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":63,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.759191778895392e-06,"mean_abs_dp":3.824375472031433e-06,"oracle_top2_margin":0.9419592280403212,"near_tie":false,"n_options":6,"probs":[0.015226724382678174,0.9579047702349257,0.01592956131389053,0.0035518262677321573,0.002704351026348256,0.004682766774425214],"hidden_sha256":"444718bfb6892e9f0dd28ce8872b3a3483342bcd1fc0ab69d192ee578d6fcfc1","first_in_process":false,"shard":"shard_01","index":31},{"id":"tv4x_emotion_12","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00014221887680299083,"mean_abs_dp":4.759357432504094e-05,"oracle_top2_margin":0.5647145684451723,"near_tie":false,"n_options":6,"probs":[0.03355569929945177,0.18692607282695622,0.7514412096387174,0.0159603547334232,0.0065320118209212215,0.005584651680530163],"hidden_sha256":"65ebfb1fc8b479551c0bb1abf7f94ff26d6ae930aa55cadd47f4952c53858be4","first_in_process":false,"shard":"shard_01","index":32},{"id":"tv4x_emotion_13","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.0008900383595003003,"mean_abs_dp":0.00030045158998048154,"oracle_top2_margin":0.25219666980289757,"near_tie":false,"n_options":6,"probs":[0.3491504255209412,0.013731110667876594,0.010681164805390882,0.0037807854984965215,0.6030649850747024,0.01959152843259245],"hidden_sha256":"72c2415b8ad60d8d224948e26184a95b50880976e8d8d86705b6d0b393797a6e","first_in_process":false,"shard":"shard_01","index":33},{"id":"tv4x_emotion_14","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00016267905989453268,"mean_abs_dp":5.526167446184223e-05,"oracle_top2_margin":0.6706157015798525,"near_tie":false,"n_options":6,"probs":[0.07825856389813197,0.11838417825379695,0.7887837573061012,0.00612525607903029,0.004590728354430325,0.0038575161085093158],"hidden_sha256":"43541bb6d825cae42bb671d2294fe38fbcd633c75842540603e33ed0a285296f","first_in_process":false,"shard":"shard_01","index":34},{"id":"tv4x_emotion_15","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":59,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003035570287798217,"mean_abs_dp":0.00010118567625994946,"oracle_top2_margin":0.637767381720322,"near_tie":false,"n_options":6,"probs":[0.00832480233087558,0.8105743935076404,0.17221974991339284,0.003384567497543242,0.0022759200964105404,0.0032205666541373387],"hidden_sha256":"8f0aef732ab32e8c5812a6eb898ef16005805c5d1088d7e57418398eabd60098","first_in_process":false,"shard":"shard_01","index":35},{"id":"tv4x_emotion_16","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":65,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00019541010480783605,"mean_abs_dp":6.513670160260262e-05,"oracle_top2_margin":0.9295570190736654,"near_tie":false,"n_options":6,"probs":[0.9462764067775663,0.011069132092112317,0.01695932467075175,0.016295087957284137,0.005502089703853302,0.003897958798432194],"hidden_sha256":"90849f78d11ad579507f6b0c3ede9fde4f56cb2b177472b4263e4bf49d95fc39","first_in_process":false,"shard":"shard_01","index":36},{"id":"tv4x_emotion_17","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001951090123983451,"mean_abs_dp":8.670825760143513e-05,"oracle_top2_margin":0.5585129847508995,"near_tie":false,"n_options":6,"probs":[0.6750586964514771,0.03129018815964046,0.10980225232664043,0.05072760675239486,0.11639826382669366,0.016722992483153523],"hidden_sha256":"acc1b85cb5e90a8f14c716e09e77a9d714d0657a91896f6d6a212b6f3ed24c03","first_in_process":false,"shard":"shard_01","index":37},{"id":"tv4x_emotion_18","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.667061915813008e-05,"mean_abs_dp":1.6794494524185733e-05,"oracle_top2_margin":0.9019695522988173,"near_tie":false,"n_options":6,"probs":[0.929670402477949,0.010477521524204593,0.006027137746728612,0.019190530801342812,0.02767646785051393,0.006957939599261058],"hidden_sha256":"daf666ba9a463a604b2891aba26edee4a9f46f5a2dca69f1eee450dba25d089f","first_in_process":false,"shard":"shard_01","index":38},{"id":"tv4x_emotion_19","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014640701567791847,"mean_abs_dp":4.880233855931599e-05,"oracle_top2_margin":0.9096538982162927,"near_tie":false,"n_options":6,"probs":[0.9305998039836597,0.020768917989030435,0.010165246234605838,0.01862722640228562,0.015719026353586796,0.004119779036831665],"hidden_sha256":"7369b55f13ebc23b4be3766fdb33d6770592f3c86b26e9860fc4122acb0142de","first_in_process":false,"shard":"shard_01","index":39},{"id":"tv4x_qnli_00","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.617832068700296e-05,"mean_abs_dp":2.6178320686872855e-05,"oracle_top2_margin":0.967676077803126,"near_tie":false,"n_options":2,"probs":[0.9838642172222498,0.016135782777750053],"hidden_sha256":"60f2ca72b9bcdec5e973ba2ab02bbd5636e55ac0755b647a2e23afd212237882","first_in_process":true,"shard":"shard_02","index":0,"hidden":{"min_pos_cos":0.999993144524632,"min_pos_cos_index":93,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4x_qnli_01","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.527245568652365e-05,"mean_abs_dp":9.527245568644038e-05,"oracle_top2_margin":0.8131166435987733,"near_tie":false,"n_options":2,"probs":[0.09353695065629974,0.9064630493437001],"hidden_sha256":"22127f0b5f82e33dfff4e4fab10d67fd9f5cb555189e73af97bb0688311c959b","first_in_process":false,"shard":"shard_02","index":1},{"id":"tv4x_qnli_02","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014716247870694144,"mean_abs_dp":0.00014716247870690674,"oracle_top2_margin":0.8113164357001165,"near_tie":false,"n_options":2,"probs":[0.9058053803287651,0.09419461967123485],"hidden_sha256":"029e957eceb8e3f16bd6db7647d37b14734018c2f2570d80fb78fbd8b71a7a31","first_in_process":false,"shard":"shard_02","index":2},{"id":"tv4x_qnli_03","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.401936141450017e-05,"mean_abs_dp":8.401936141447242e-05,"oracle_top2_margin":0.8867196111326728,"near_tie":false,"n_options":2,"probs":[0.056556175072249124,0.9434438249277509],"hidden_sha256":"a7ee9bdbdfda11e67bb61b1a577698b7a2f9e6eafe3963ad3c7453d470538f8b","first_in_process":false,"shard":"shard_02","index":3},{"id":"tv4x_qnli_04","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":157,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00038882379194582484,"mean_abs_dp":0.0003888237919457832,"oracle_top2_margin":0.7598955557679068,"near_tie":false,"n_options":2,"probs":[0.8803366016758992,0.11966339832410083],"hidden_sha256":"1f20207023b3133f474d1a7d01c0364a83ffe74e3ccc82248586b53137999e48","first_in_process":false,"shard":"shard_02","index":4},{"id":"tv4x_qnli_05","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003730258367763417,"mean_abs_dp":0.0003730258367762862,"oracle_top2_margin":0.2604357567584594,"near_tie":false,"n_options":2,"probs":[0.6305909042160059,0.36940909578399395],"hidden_sha256":"14661c5b39b5b845b350a2dc79995ed6569831e5ab875846df93ebcff3073781","first_in_process":false,"shard":"shard_02","index":5},{"id":"tv4x_qnli_06","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002627005653228487,"mean_abs_dp":0.00026270056532282093,"oracle_top2_margin":0.7349581738845581,"near_tie":false,"n_options":2,"probs":[0.8677417875076019,0.13225821249239816],"hidden_sha256":"da54d3ca49834b70bbe510901d52275e710c0c345be808a7250e7ac72ac85c9f","first_in_process":false,"shard":"shard_02","index":6},{"id":"tv4x_qnli_07","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":64,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007914552398928243,"mean_abs_dp":0.0007914552398927965,"oracle_top2_margin":0.06001024267368793,"near_tie":false,"n_options":2,"probs":[0.5292136660969512,0.47078633390304886],"hidden_sha256":"4721a58589147d03596b561d6c9ec8408806916aca26f1bd959629ffb748b5f7","first_in_process":false,"shard":"shard_02","index":7},{"id":"tv4x_qnli_08","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00024185755612261828,"mean_abs_dp":0.00024185755612261134,"oracle_top2_margin":0.79770824729877,"near_tie":false,"n_options":2,"probs":[0.10138773390673757,0.8986122660932624],"hidden_sha256":"5afd1887a0ac5ac32dcdd93953754c6ed549564cb4fda80c25989cb103a80cc6","first_in_process":false,"shard":"shard_02","index":8},{"id":"tv4x_qnli_09","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.516380979121797e-06,"mean_abs_dp":3.5163809790923067e-06,"oracle_top2_margin":0.9488240328840618,"near_tie":false,"n_options":2,"probs":[0.02558446717699003,0.97441553282301],"hidden_sha256":"2d5873e4f0f4f7fb6f04d55990b090b940840e28f42a1d70cb9344f3f975d318","first_in_process":false,"shard":"shard_02","index":9},{"id":"tv4x_qnli_10","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.3103101844237917e-05,"mean_abs_dp":2.3103101844203222e-05,"oracle_top2_margin":0.966925876394352,"near_tie":false,"n_options":2,"probs":[0.01651395870097978,0.9834860412990202],"hidden_sha256":"b8029a3dbb2fdc15f866aa82515977db64a08165411b6fdeab4b625ef40a9338","first_in_process":false,"shard":"shard_02","index":10},{"id":"tv4x_qnli_11","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":59,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.381931735076034e-05,"mean_abs_dp":5.3819317350680546e-05,"oracle_top2_margin":0.9082556399693326,"near_tie":false,"n_options":2,"probs":[0.9541816393020169,0.045818360697983],"hidden_sha256":"dbb9e301961a91a234433271b885a57e14fed22c8162c16242b96e4164c6a045","first_in_process":false,"shard":"shard_02","index":11},{"id":"tv4x_qnli_12","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000560649105013411,"mean_abs_dp":0.0005606491050133833,"oracle_top2_margin":0.3794817865854016,"near_tie":false,"n_options":2,"probs":[0.6903015423977142,0.30969845760228587],"hidden_sha256":"52948a66b383f34784985136588aa18839627499c3ea5d1369cdc7e288bd0980","first_in_process":false,"shard":"shard_02","index":12},{"id":"tv4x_qnli_13","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.3029727837436953e-05,"mean_abs_dp":2.3029727837423075e-05,"oracle_top2_margin":0.7759343256201553,"near_tie":false,"n_options":2,"probs":[0.8879441330822403,0.11205586691775983],"hidden_sha256":"7a45b842bb359ffb30129d78bd909e221051acdda9fabeaa13d4439b1c724e58","first_in_process":false,"shard":"shard_02","index":13},{"id":"tv4x_qnli_14","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011430292517944363,"mean_abs_dp":0.00011430292517940199,"oracle_top2_margin":0.7304054579392312,"near_tie":false,"n_options":2,"probs":[0.8650884260444361,0.13491157395556383],"hidden_sha256":"5fbdd6516eca9986a61f27d65443613b980afb1eb3a6b07ec7c4a17ea40cc48d","first_in_process":false,"shard":"shard_02","index":14},{"id":"tv4x_qnli_15","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.605437600159243e-05,"mean_abs_dp":1.6054376001543857e-05,"oracle_top2_margin":0.9337811649427256,"near_tie":false,"n_options":2,"probs":[0.9668745280953612,0.03312547190463876],"hidden_sha256":"1c649e9d2ed85ecc1c4929c7342cab69dfbbc79ce498ee6fbd74fc617913aeae","first_in_process":false,"shard":"shard_02","index":15},{"id":"tv4x_qnli_16","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003607183625402577,"mean_abs_dp":0.00036071836254022993,"oracle_top2_margin":0.43803847138581853,"near_tie":false,"n_options":2,"probs":[0.7193799540554495,0.28062004594455053],"hidden_sha256":"12a54b6445206ee541a79c420f61140478532274518a92d5215cff57f6e8f5a0","first_in_process":false,"shard":"shard_02","index":16},{"id":"tv4x_qnli_17","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.3203686118304359e-05,"mean_abs_dp":1.3203686118269664e-05,"oracle_top2_margin":0.9211082745803174,"near_tie":false,"n_options":2,"probs":[0.9605409336040405,0.0394590663959596],"hidden_sha256":"5902cbdddda719773cf55426dde421e9d7dccbe34067a5983b0bebd64de3217f","first_in_process":false,"shard":"shard_02","index":17},{"id":"tv4x_qnli_18","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.53918440694412e-05,"mean_abs_dp":5.53918440694412e-05,"oracle_top2_margin":0.8523963888636283,"near_tie":false,"n_options":2,"probs":[0.07385719741225531,0.9261428025877447],"hidden_sha256":"9c8e8a2a2d8a47ee4f979d98555d609dda30b1d0895b3c7f91c7449f47c872e2","first_in_process":false,"shard":"shard_02","index":18},{"id":"tv4x_qnli_19","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00017325971160808695,"mean_abs_dp":0.0001732597116080592,"oracle_top2_margin":0.7034361806374811,"near_tie":false,"n_options":2,"probs":[0.8518913500303487,0.1481086499696514],"hidden_sha256":"e4f22b7345c1324cb873aee564c9559c218c866c283e28376144ff66bdbd54e6","first_in_process":false,"shard":"shard_02","index":19},{"id":"tv4x_paws_00","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00019982288320874453,"mean_abs_dp":0.00019982288320868208,"oracle_top2_margin":0.8207418652737444,"near_tie":false,"n_options":2,"probs":[0.08982889024633645,0.9101711097536634],"hidden_sha256":"2742f6f748fdb96d4b256b4bec83873ac91f158fe8374a4b9022a210b8f65fb4","first_in_process":false,"shard":"shard_02","index":20},{"id":"tv4x_paws_01","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":139,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005904962029685268,"mean_abs_dp":0.000590496202968499,"oracle_top2_margin":0.07396326897051358,"near_tie":false,"n_options":2,"probs":[0.5375721306882253,0.4624278693117747],"hidden_sha256":"2244fbfdaa2d78a8f36a5d5ea6bf3ef3ff633b660252afa26dccb4fb6e11a60f","first_in_process":false,"shard":"shard_02","index":21},{"id":"tv4x_paws_02","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":116,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.722045172100842e-06,"mean_abs_dp":4.722045172055739e-06,"oracle_top2_margin":0.9416461714567297,"near_tie":false,"n_options":2,"probs":[0.02918163631680719,0.9708183636831927],"hidden_sha256":"7dfb4131bbd721040399d14d8f4813f3d2ecf26bcae052ef5f833d57777ba059","first_in_process":false,"shard":"shard_02","index":22},{"id":"tv4x_paws_03","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.166283865707609e-05,"mean_abs_dp":3.166283865704486e-05,"oracle_top2_margin":0.9530152365841582,"near_tie":false,"n_options":2,"probs":[0.976475955453422,0.023524044546577916],"hidden_sha256":"3e671b13bb6f8a3028513034bc1a85809fdbce7a44abbcf9fb5590e99676eb50","first_in_process":false,"shard":"shard_02","index":23},{"id":"tv4x_paws_04","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.798938216443176e-05,"mean_abs_dp":9.798938216443176e-05,"oracle_top2_margin":0.9194586795437724,"near_tie":false,"n_options":2,"probs":[0.9598273291540507,0.040172670845949415],"hidden_sha256":"68f92e16d419a8b8b0eed0cec590214465c5c63a247195294656a792da1a6f87","first_in_process":false,"shard":"shard_02","index":24},{"id":"tv4x_paws_05","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":107,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00033143680013691945,"mean_abs_dp":0.00033143680013691945,"oracle_top2_margin":0.3452288520436342,"near_tie":false,"n_options":2,"probs":[0.6722829892216802,0.32771701077831983],"hidden_sha256":"fec0be5b97a8a3c2096306d55b9702c6ee99caecc7e32097f74d3adff49fb67c","first_in_process":false,"shard":"shard_02","index":25},{"id":"tv4x_paws_06","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":112,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006472775541622733,"mean_abs_dp":0.0006472775541622178,"oracle_top2_margin":0.15852749594349458,"near_tie":false,"n_options":2,"probs":[0.42008897447409055,0.5799110255259096],"hidden_sha256":"3b806e97beb2fd33ebf7dc050f6cbe96fa0fd2b470e75b6e4ce89f3c82cad3ad","first_in_process":false,"shard":"shard_02","index":26},{"id":"tv4x_paws_07","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.5250802460741184e-05,"mean_abs_dp":2.525080246072557e-05,"oracle_top2_margin":0.9577955021898784,"near_tie":false,"n_options":2,"probs":[0.9788725002924785,0.021127499707521518],"hidden_sha256":"e2c1638efdaac5a8fa652e898f14299aa303937a8ed73ee835437b99101cc42e","first_in_process":false,"shard":"shard_02","index":27},{"id":"tv4x_paws_08","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.5190438405085824e-06,"mean_abs_dp":7.5190438404201115e-06,"oracle_top2_margin":0.9480983046539663,"near_tie":false,"n_options":2,"probs":[0.9740416332831427,0.025958366716857334],"hidden_sha256":"d425d3ea2191f0fdbb225e73d3f7ef2c3fcc215b94c0604f7afb48de9c9047ad","first_in_process":false,"shard":"shard_02","index":28},{"id":"tv4x_paws_09","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":140,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.079693928848442e-05,"mean_abs_dp":9.079693928847749e-05,"oracle_top2_margin":0.8694028325570654,"near_tie":false,"n_options":2,"probs":[0.9346106193392443,0.06538938066075578],"hidden_sha256":"297dc8edb7798923b5631bbe87a7438f419da172b08fe76ce0270994dc8c12a0","first_in_process":false,"shard":"shard_02","index":29},{"id":"tv4x_paws_10","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.852575602684207e-05,"mean_abs_dp":5.852575602677962e-05,"oracle_top2_margin":0.8572341261731035,"near_tie":false,"n_options":2,"probs":[0.07132441115742151,0.9286755888425785],"hidden_sha256":"2015763eccceb42a90ad371a6260b5f6d499b9df9d3ee001c5bc63bf22aed509","first_in_process":false,"shard":"shard_02","index":30},{"id":"tv4x_paws_11","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.296607106349463e-05,"mean_abs_dp":2.2966071063470345e-05,"oracle_top2_margin":0.9195431167829624,"near_tie":false,"n_options":2,"probs":[0.9597485923204178,0.04025140767958232],"hidden_sha256":"e386def3b22305559163567bdc2139c130be861bb66cf4fc9e85bd68621fa4c4","first_in_process":false,"shard":"shard_02","index":31},{"id":"tv4x_paws_12","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.634706747536383e-06,"mean_abs_dp":5.634706747475668e-06,"oracle_top2_margin":0.9457556296060624,"near_tie":false,"n_options":2,"probs":[0.9728721800962837,0.02712781990371627],"hidden_sha256":"b941fcb9919cdbbb9774614b0f1a25efcdee186135e556a61c20e35c4123b8c2","first_in_process":false,"shard":"shard_02","index":32},{"id":"tv4x_paws_13","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":132,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.542202685855809e-05,"mean_abs_dp":1.5422026858530336e-05,"oracle_top2_margin":0.5612642409230675,"near_tie":false,"n_options":2,"probs":[0.7806166984346752,0.21938330156532482],"hidden_sha256":"6c965f66b6c2a5bed53ecb9a9192b14d5c465d3d643137116c32fcb77670cd95","first_in_process":false,"shard":"shard_02","index":33},{"id":"tv4x_paws_14","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.081485562355212e-05,"mean_abs_dp":2.0814855623520895e-05,"oracle_top2_margin":0.9160329963674159,"near_tie":false,"n_options":2,"probs":[0.04200431667191556,0.9579956833280844],"hidden_sha256":"bd6a4a4ced581cb7f3377e1f5e6910a68a67141caea4e3295857005f50803367","first_in_process":false,"shard":"shard_02","index":34},{"id":"tv4x_paws_15","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.371290067661793e-05,"mean_abs_dp":4.3712900676576294e-05,"oracle_top2_margin":0.9209250852704204,"near_tie":false,"n_options":2,"probs":[0.9604188297345336,0.03958117026546639],"hidden_sha256":"39e6fcd093a081beafb5a0c073128bd3a4dfb292c7b018598980c757f02f38df","first_in_process":false,"shard":"shard_02","index":35},{"id":"tv4x_paws_16","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.557574646353203e-05,"mean_abs_dp":5.557574646345917e-05,"oracle_top2_margin":0.9487658868484278,"near_tie":false,"n_options":2,"probs":[0.9744385191706774,0.025561480829322702],"hidden_sha256":"fa17783130c86d285be591f8ee43579e8e07816424741515a81dbffaf1ef73bc","first_in_process":false,"shard":"shard_02","index":36},{"id":"tv4x_paws_17","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000539944798817249,"mean_abs_dp":0.0005399447988171657,"oracle_top2_margin":0.19665281051821115,"near_tie":false,"n_options":2,"probs":[0.5977864604602884,0.40221353953971156],"hidden_sha256":"6f2432a9adb402237fc54d4aef886fee1d4d4c912181cb0e2248d7dd7292a886","first_in_process":false,"shard":"shard_02","index":37},{"id":"tv4x_paws_18","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.3739480009648808e-06,"mean_abs_dp":1.3739480009457988e-06,"oracle_top2_margin":0.9666590415731975,"near_tie":false,"n_options":2,"probs":[0.01667185316140222,0.9833281468385978],"hidden_sha256":"9b59cff0f93a187bc345a7c1d11376904ce5b014d535f3ed121be054fd562cc9","first_in_process":false,"shard":"shard_02","index":38},{"id":"tv4x_paws_19","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.424388918871202e-05,"mean_abs_dp":3.424388918871202e-05,"oracle_top2_margin":0.9576614130338148,"near_tie":false,"n_options":2,"probs":[0.9787964626277187,0.021203537372281336],"hidden_sha256":"1adafc31c35622456d7e6c2b78f59ab3bb3710f622af3a96ae33def0038c5cc5","first_in_process":false,"shard":"shard_02","index":39},{"id":"tv4x_sciq_00","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":163,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.483475425340558e-06,"mean_abs_dp":3.741737712668978e-06,"oracle_top2_margin":0.9809464409661907,"near_tie":false,"n_options":4,"probs":[0.0014305394178400366,0.9877479077971338,0.004032769412257148,0.006788783372769041],"hidden_sha256":"6cfe77ee05ce6a62a387a3014181ff847f89fffbc767c34285dce909e39a57e8","first_in_process":true,"shard":"shard_03","index":0},{"id":"tv4x_sciq_01","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":71,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00011871472663871341,"mean_abs_dp":6.167417039390673e-05,"oracle_top2_margin":0.7009752393049276,"near_tie":false,"n_options":4,"probs":[0.01131421566576461,0.12266758741083598,0.04260315694587208,0.8234150399775273],"hidden_sha256":"a1af468c1fefe4de5361ad1cb32fb388a8fdfd09cc480604f73be5c492d12e6f","first_in_process":false,"shard":"shard_03","index":1},{"id":"tv4x_sciq_02","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":118,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.429035426303571e-05,"mean_abs_dp":7.1451771315126505e-06,"oracle_top2_margin":0.9777094276883521,"near_tie":false,"n_options":4,"probs":[0.9862829292042915,0.008599709158504585,0.0022630253163835354,0.0028543363208204777],"hidden_sha256":"9f18c1ca959a67509a53aaf8aa2f7a823cd6b9341ba60124f1f767693d0f14d0","first_in_process":false,"shard":"shard_03","index":2},{"id":"tv4x_sciq_03","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":109,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":1.1346524935285096e-05,"mean_abs_dp":5.6732624676117025e-06,"oracle_top2_margin":0.9846019844932212,"near_tie":false,"n_options":4,"probs":[0.0014972049962084768,0.0038635274184795863,0.0050251470559652565,0.9896141205293466],"hidden_sha256":"934554accb942f33609e3176299925880771acee062a1ca4650462075f6507fd","first_in_process":false,"shard":"shard_03","index":3},{"id":"tv4x_sciq_04","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":134,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.8333579034129954e-06,"mean_abs_dp":3.4166789516961436e-06,"oracle_top2_margin":0.9820033764228081,"near_tie":false,"n_options":4,"probs":[0.0016774122084924244,0.9883928411398616,0.006380290776047058,0.0035494558755989067],"hidden_sha256":"1aca3efe10279397cccb79bc33c1d1e01ec5fd8b1ef7bf437467f5d1c82a941a","first_in_process":false,"shard":"shard_03","index":4},{"id":"tv4x_sciq_05","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.1686728168177645e-05,"mean_abs_dp":2.0843364084100532e-05,"oracle_top2_margin":0.9571367735230473,"near_tie":false,"n_options":4,"probs":[0.9686391980231353,0.011459617140709439,0.011019634657738376,0.008881550178416919],"hidden_sha256":"9257f1645c4381267820dd433b8a363dca741451587e38ac5a220e42c09b8833","first_in_process":false,"shard":"shard_03","index":5},{"id":"tv4x_sciq_06","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":204,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.005925849847607e-06,"mean_abs_dp":4.502962924945054e-06,"oracle_top2_margin":0.9905170864635728,"near_tie":false,"n_options":4,"probs":[0.001212276734080581,0.9935100936651083,0.002981065497869905,0.002296564102941132],"hidden_sha256":"8cf23e532fd5ac72bb482b1e5bf1281d3e6de9b2f58c9a8fe99ac494ecb38595","first_in_process":false,"shard":"shard_03","index":6},{"id":"tv4x_sciq_07","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.3173648220686793e-05,"mean_abs_dp":1.1586824110359389e-05,"oracle_top2_margin":0.9897740900749374,"near_tie":false,"n_options":4,"probs":[0.0011261573521293166,0.9932006193149681,0.0033948012139491295,0.0022784221189535245],"hidden_sha256":"f3a71bc81bd63d767a5ad0799d9e6c6a93ca38f93641d9f8a3b68af6fc47774e","first_in_process":false,"shard":"shard_03","index":7},{"id":"tv4x_sciq_08","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.7724922537376564e-05,"mean_abs_dp":8.862461268758647e-06,"oracle_top2_margin":0.9845569664084634,"near_tie":false,"n_options":4,"probs":[0.9897090519471201,0.005179560592221231,0.002666306408616232,0.002445081052042562],"hidden_sha256":"597369a85a3d4b61bb08e3c42d0415ad484e125ec5635afe1e4acf34ee7dcbf5","first_in_process":false,"shard":"shard_03","index":8},{"id":"tv4x_sciq_09","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":1.9402150119640638e-05,"mean_abs_dp":9.701075059796033e-06,"oracle_top2_margin":0.9786206076387443,"near_tie":false,"n_options":4,"probs":[0.00362465743433867,0.005293516649111899,0.006217855551881874,0.9848639703646677],"hidden_sha256":"c676fbe51d77072bab987a7a2c775cd609e6d6ad530449114a8b1c205f215c1d","first_in_process":false,"shard":"shard_03","index":9},{"id":"tv4x_sciq_10","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":3.845760155302136e-06,"mean_abs_dp":2.0246476654837368e-06,"oracle_top2_margin":0.9825931896772335,"near_tie":false,"n_options":4,"probs":[0.0032641306239696633,0.004979768004639256,0.0041844040160491204,0.9875716973553419],"hidden_sha256":"a60ea12c5003aefb102f8f5a560bd2d8fcf611e8bb6b912f83723834fbba5c3d","first_in_process":false,"shard":"shard_03","index":10},{"id":"tv4x_sciq_11","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":378,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.529410207505414e-05,"mean_abs_dp":2.7647051037526962e-05,"oracle_top2_margin":0.9561027133595873,"near_tie":false,"n_options":4,"probs":[0.01680891173802268,0.007212778331923517,0.9728307100963879,0.003147599833665973],"hidden_sha256":"671b53262512f09ea1b662c672b4157cab04aec165be00f5c88d706a59e8872d","first_in_process":false,"shard":"shard_03","index":11},{"id":"tv4x_sciq_12","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":202,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.818901721239996e-05,"mean_abs_dp":9.094508606170192e-06,"oracle_top2_margin":0.9850719395803337,"near_tie":false,"n_options":4,"probs":[0.0006952405140343348,0.0040975515104979205,0.9901274696711425,0.0050797383043251895],"hidden_sha256":"2e4e2efde4fc03a18abec839ec7698affe2729e559e4581cd996a7ee9673bb76","first_in_process":false,"shard":"shard_03","index":12},{"id":"tv4x_sciq_13","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":163,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.68221669669877e-05,"mean_abs_dp":1.841108348348919e-05,"oracle_top2_margin":0.978102360961836,"near_tie":false,"n_options":4,"probs":[0.9851683426322448,0.007010023736923892,0.002960590481675065,0.00486104314915626],"hidden_sha256":"f47bbd892d120e2dfcadff68419a5868f98c569582aff174774bf7b73eae4c28","first_in_process":false,"shard":"shard_03","index":13},{"id":"tv4x_sciq_14","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":141,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":7.173026982719086e-06,"mean_abs_dp":4.694330864458626e-06,"oracle_top2_margin":0.9785787223861179,"near_tie":false,"n_options":4,"probs":[0.0032886264392608822,0.005217280058903498,0.006455206861740591,0.9850388866400951],"hidden_sha256":"c3711568b8d950412007ae3136bc7160cbf844f7c146cabe863f78d9eb11b5b4","first_in_process":false,"shard":"shard_03","index":14},{"id":"tv4x_sciq_15","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":277,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.944509185665023e-05,"mean_abs_dp":1.4722545928314598e-05,"oracle_top2_margin":0.98899586086744,"near_tie":false,"n_options":4,"probs":[0.0011582305553441668,0.9924906523295377,0.002901479683536364,0.0034496374315817037],"hidden_sha256":"63a7f401dc34d891cc29c995ab2f81e786edd1935422299cb70388f0f5c06b55","first_in_process":false,"shard":"shard_03","index":15},{"id":"tv4x_sciq_16","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":352,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.000673039976480605,"mean_abs_dp":0.000336519988240299,"oracle_top2_margin":0.6374411741262631,"near_tie":false,"n_options":4,"probs":[0.04048897154085078,0.11949089433008747,0.08208741121887848,0.7579327229101832],"hidden_sha256":"3efd6c8de3a08d65bdb3e0fd70560181b0424274076e512cca1b9ce139c822b6","first_in_process":false,"shard":"shard_03","index":16},{"id":"tv4x_sciq_17","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":185,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.7074928149860575e-05,"mean_abs_dp":1.853746407490931e-05,"oracle_top2_margin":0.977520044286674,"near_tie":false,"n_options":4,"probs":[0.9860319880372161,0.008450382814689546,0.0038053775760888437,0.0017122515720055308],"hidden_sha256":"198729d65ed362387fe479b83974993ea4c4405d4810710acefb1f41d7549d9f","first_in_process":false,"shard":"shard_03","index":17},{"id":"tv4x_sciq_18","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.788921257642878e-05,"mean_abs_dp":1.0267218701289936e-05,"oracle_top2_margin":0.9656887266619748,"near_tie":false,"n_options":4,"probs":[0.9757854503709543,0.01006467998320064,0.007325679845160749,0.006824189800684276],"hidden_sha256":"f096f35fef74c983e2b511f1e6af034bfc85cd28973443570adbb0fc12606d31","first_in_process":false,"shard":"shard_03","index":18},{"id":"tv4x_sciq_19","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.94717720692195e-06,"mean_abs_dp":3.2450818998063e-06,"oracle_top2_margin":0.9916925929078704,"near_tie":false,"n_options":4,"probs":[0.9942898602916022,0.0026070872759878247,0.001797890970792394,0.0013051614616175421],"hidden_sha256":"abfd96aeecc568bc7debba2aecebd25c61c2212c4a035da1654b6a9470c94026","first_in_process":false,"shard":"shard_03","index":19},{"id":"tv4x_legacy_holdout_00","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.230971291672379e-05,"mean_abs_dp":2.230971291669083e-05,"oracle_top2_margin":0.9635829051438438,"near_tie":false,"n_options":2,"probs":[0.9818137622848386,0.018186237715161403],"hidden_sha256":"b9bf06a15985080b3dd4ebeaaf51aef3990c699658c2f5deb6074324490946c2","first_in_process":false,"shard":"shard_03","index":20},{"id":"tv4x_legacy_holdout_01","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.515124885031476e-06,"mean_abs_dp":6.5151248849395355e-06,"oracle_top2_margin":0.9796117050734497,"near_tie":false,"n_options":2,"probs":[0.010187632338390177,0.9898123676616098],"hidden_sha256":"4193484a72998dd093cc9e2e04ec04bc88afbf941d33e7af013a8e805ba5a903","first_in_process":false,"shard":"shard_03","index":21},{"id":"tv4x_legacy_holdout_02","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.693672459759238e-05,"mean_abs_dp":1.6936724597470948e-05,"oracle_top2_margin":0.9564229434436572,"near_tie":false,"n_options":2,"probs":[0.9781945349972312,0.021805465002768972],"hidden_sha256":"ad98d892a8867c41f2f32864965b9835528d1c96d6f587280be2f5cddc3cec25","first_in_process":false,"shard":"shard_03","index":22},{"id":"tv4x_legacy_holdout_03","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.558842644861866e-06,"mean_abs_dp":4.558842644846253e-06,"oracle_top2_margin":0.9698785191123516,"near_tie":false,"n_options":2,"probs":[0.01506529928646908,0.984934700713531],"hidden_sha256":"d2afcd8a0adf1715dc64a36c4e1c6aa2aa7a8741ecd8cb76772674455385a316","first_in_process":false,"shard":"shard_03","index":23},{"id":"tv4x_legacy_holdout_04","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.1893027017449709e-05,"mean_abs_dp":1.18930270174393e-05,"oracle_top2_margin":0.963808624235394,"near_tie":false,"n_options":2,"probs":[0.9819162051447144,0.01808379485528558],"hidden_sha256":"af589b31c1bd608c6436b37b9f83e372d1dc625cec352c4adffbadca80f18486","first_in_process":false,"shard":"shard_03","index":24},{"id":"tv4x_legacy_holdout_05","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.332751080509805e-06,"mean_abs_dp":6.3327510804638346e-06,"oracle_top2_margin":0.9798986789108202,"near_tie":false,"n_options":2,"probs":[0.01004432779350936,0.9899556722064905],"hidden_sha256":"22cad755784e13061b707cb2496c2f36573a0343774b68ad75d11d733b1f11c8","first_in_process":false,"shard":"shard_03","index":25},{"id":"tv4x_legacy_holdout_06","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.507481711737819e-06,"mean_abs_dp":8.507481711699655e-06,"oracle_top2_margin":0.9308847344232876,"near_tie":false,"n_options":2,"probs":[0.9654338597299321,0.034566140270067915],"hidden_sha256":"d37e13e35aa14adde8b974a4f778efff0a45a8c2c9fe5b010fba21866b39554f","first_in_process":false,"shard":"shard_03","index":26},{"id":"tv4x_legacy_holdout_07","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.806065986590752e-06,"mean_abs_dp":4.8060659864988114e-06,"oracle_top2_margin":0.9809323842367962,"near_tie":false,"n_options":2,"probs":[0.009538613947588436,0.9904613860524116],"hidden_sha256":"3c66dc1890bd3334bf4eeae59df41e641e415ee4cd8c5c4f0c329c20452c522a","first_in_process":false,"shard":"shard_03","index":27},{"id":"tv4x_legacy_holdout_08","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.639499133323808e-05,"mean_abs_dp":3.6394991333235475e-05,"oracle_top2_margin":0.9719924049726297,"near_tie":false,"n_options":2,"probs":[0.9859598074949817,0.0140401925050184],"hidden_sha256":"a66e8d34c9a340ecb0f1c06ca87f5edef10b0453505edd985b1fc1dd77cdab74","first_in_process":false,"shard":"shard_03","index":28},{"id":"tv4x_legacy_holdout_09","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.8970833588758474e-05,"mean_abs_dp":2.8970833588709034e-05,"oracle_top2_margin":0.977575169351521,"near_tie":false,"n_options":2,"probs":[0.011241386157828182,0.9887586138421718],"hidden_sha256":"7fd45fac1c9445962d8d115ae10617951fe8f46a37e675660559d80f8e25f313","first_in_process":false,"shard":"shard_03","index":29},{"id":"tv4x_legacy_holdout_10","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.511414219739347e-05,"mean_abs_dp":8.511414219737785e-05,"oracle_top2_margin":0.9679284058204262,"near_tie":false,"n_options":2,"probs":[0.9838790887680158,0.016120911231984336],"hidden_sha256":"7d5572ab6e24f838e8d91cb74dbb90d6def29428a5f3772e2736362a15e6c698","first_in_process":false,"shard":"shard_03","index":30},{"id":"tv4x_legacy_holdout_11","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.4639475141530198e-05,"mean_abs_dp":2.4639475141467748e-05,"oracle_top2_margin":0.9685686046272135,"near_tie":false,"n_options":2,"probs":[0.01569105821125184,0.9843089417887483],"hidden_sha256":"77296c139e7d961821d862e3ba840a052186908688e5208c7fa3703fbe23986a","first_in_process":false,"shard":"shard_03","index":31},{"id":"tv4x_legacy_holdout_12","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.876883898722987e-06,"mean_abs_dp":8.876883898691762e-06,"oracle_top2_margin":0.966612234838911,"near_tie":false,"n_options":2,"probs":[0.9833149943033542,0.016685005696645824],"hidden_sha256":"88ba3f6063fd736da4ac45bba952cbd7b29904524b48a3b0c0b0e4ed3fe0f4bf","first_in_process":false,"shard":"shard_03","index":32},{"id":"tv4x_legacy_holdout_13","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.3839362745357922e-05,"mean_abs_dp":1.3839362745353585e-05,"oracle_top2_margin":0.9734283318284275,"near_tie":false,"n_options":2,"probs":[0.013271994723040876,0.986728005276959],"hidden_sha256":"41bffb432088f5969ecfe618ed7660cbf3edac64a1737db658bfb988692bb87d","first_in_process":false,"shard":"shard_03","index":33},{"id":"tv4x_legacy_holdout_14","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010004891538005489,"mean_abs_dp":0.00010004891538000632,"oracle_top2_margin":0.9391565521574167,"near_tie":false,"n_options":2,"probs":[0.9696783249940883,0.03032167500591163],"hidden_sha256":"10502a507dba7f861382bf11c920444382d1c440d841267786aba95b695d1e2f","first_in_process":false,"shard":"shard_03","index":34},{"id":"tv4x_legacy_holdout_15","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.243865821545011e-05,"mean_abs_dp":3.243865821544491e-05,"oracle_top2_margin":0.9806683876941003,"near_tie":false,"n_options":2,"probs":[0.009698244811165338,0.9903017551888347],"hidden_sha256":"a84802dbcae3065d349e197557dfca0f7c0e4d1713cd069265a346505b48d6ca","first_in_process":false,"shard":"shard_03","index":35},{"id":"tv4x_legacy_holdout_16","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.011545976150387e-06,"mean_abs_dp":3.011545976123499e-06,"oracle_top2_margin":0.9767388481315413,"near_tie":false,"n_options":2,"probs":[0.9883724356117468,0.01162756438825322],"hidden_sha256":"3125d4db178aa4802179a0a070ff9b783735098be5ce7160cc4a897055d17f20","first_in_process":false,"shard":"shard_03","index":36},{"id":"tv4x_legacy_holdout_17","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.1786870164010033e-05,"mean_abs_dp":1.1786870163925899e-05,"oracle_top2_margin":0.9737978337394874,"near_tie":false,"n_options":2,"probs":[0.013089296260092403,0.9869107037399076],"hidden_sha256":"03b5b36e6c611ff9b9f1272a2d885e7f65e0eb4413d062ac0edeb89fb035a671","first_in_process":false,"shard":"shard_03","index":37},{"id":"tv4x_legacy_holdout_18","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001303356526188848,"mean_abs_dp":0.00013033565261885878,"oracle_top2_margin":0.9502284696119041,"near_tie":false,"n_options":2,"probs":[0.9752445704585709,0.024755429541429112],"hidden_sha256":"70336b429fd66ef9665a4f283dc892040261a59355274fcffd38b9bf36fbef38","first_in_process":false,"shard":"shard_03","index":38},{"id":"tv4x_legacy_holdout_19","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.2709486394153757e-05,"mean_abs_dp":1.2709486394067021e-05,"oracle_top2_margin":0.9806576856541841,"near_tie":false,"n_options":2,"probs":[0.009658447686513827,0.9903415523134861],"hidden_sha256":"9ce0f82968c71897584bb4f4fbbcc18f7b84b06e07748b3083e648b5ae4ba4ff","first_in_process":false,"shard":"shard_03","index":39},{"id":"tv4x_composition_holdout_00","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009652316777889469,"mean_abs_dp":0.0009652316777888914,"oracle_top2_margin":0.3083575869539718,"near_tie":false,"n_options":2,"probs":[0.653213561799197,0.34678643820080296],"hidden_sha256":"4b34d4f3de931fc92973fb4d84ea805079d95cc5faf00163b57ea7a8b598a03d","first_in_process":true,"shard":"shard_04","index":0},{"id":"tv4x_composition_holdout_01","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.5780553052402446e-05,"mean_abs_dp":1.5780553052357343e-05,"oracle_top2_margin":0.9234987591559782,"near_tie":false,"n_options":2,"probs":[0.9617335990249367,0.038266400975063154],"hidden_sha256":"77f2b1120a2b0e37f853454d8e5320bdc9c873bc20a9ee607e3b8c551926ba67","first_in_process":false,"shard":"shard_04","index":1},{"id":"tv4x_composition_holdout_02","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009652316777889469,"mean_abs_dp":0.0009652316777888914,"oracle_top2_margin":0.3083575869539718,"near_tie":false,"n_options":2,"probs":[0.653213561799197,0.34678643820080296],"hidden_sha256":"4b34d4f3de931fc92973fb4d84ea805079d95cc5faf00163b57ea7a8b598a03d","first_in_process":false,"shard":"shard_04","index":2},{"id":"tv4x_composition_holdout_03","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001061774161565776,"mean_abs_dp":0.0010617741615657483,"oracle_top2_margin":0.18137856727329082,"near_tie":false,"n_options":2,"probs":[0.5896275094750797,0.41037249052492036],"hidden_sha256":"dff731be10dbeffd8902b5a490a79f5a0a3069dbab21bca01034db3d3842e6e4","first_in_process":false,"shard":"shard_04","index":3},{"id":"tv4x_composition_holdout_04","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.335859019954924e-05,"mean_abs_dp":4.335859019947638e-05,"oracle_top2_margin":0.9085835772192891,"near_tie":false,"n_options":2,"probs":[0.0457515699805549,0.954248430019445],"hidden_sha256":"655d29e48f77908d7a7da46316fe6f64cf9fb476050a9be74baf2a4728f69fbc","first_in_process":false,"shard":"shard_04","index":4},{"id":"tv4x_composition_holdout_05","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":154,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.4681688940209057e-05,"mean_abs_dp":2.4681688940179566e-05,"oracle_top2_margin":0.9617516402681077,"near_tie":false,"n_options":2,"probs":[0.980900501822994,0.019099498177006048],"hidden_sha256":"a0aa5866dd75b27ab17e48f495a9e81a278f543d2da7ccb0806fdfc7e1452b4f","first_in_process":false,"shard":"shard_04","index":5},{"id":"tv4x_composition_holdout_06","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.335859019954924e-05,"mean_abs_dp":4.335859019947638e-05,"oracle_top2_margin":0.9085835772192891,"near_tie":false,"n_options":2,"probs":[0.0457515699805549,0.954248430019445],"hidden_sha256":"655d29e48f77908d7a7da46316fe6f64cf9fb476050a9be74baf2a4728f69fbc","first_in_process":false,"shard":"shard_04","index":6},{"id":"tv4x_composition_holdout_07","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.912775083218637e-05,"mean_abs_dp":7.912775083215168e-05,"oracle_top2_margin":0.9085517076396206,"near_tie":false,"n_options":2,"probs":[0.04580327393102189,0.9541967260689781],"hidden_sha256":"5753aebed301b1fc697fa38e5f4a0ecc5bdbb9b0378b3897efa1c1713950ca6b","first_in_process":false,"shard":"shard_04","index":7},{"id":"tv4x_composition_holdout_08","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00037513298030664166,"mean_abs_dp":0.00037513298030660003,"oracle_top2_margin":0.6069969730568061,"near_tie":false,"n_options":2,"probs":[0.8038736195087096,0.19612638049129033],"hidden_sha256":"b501aa2cf74557dc76974518f80b99ad412200ee5a1ed8eee43d06830d316305","first_in_process":false,"shard":"shard_04","index":8},{"id":"tv4x_composition_holdout_09","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":180,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.768967362872578e-06,"mean_abs_dp":4.768967362830945e-06,"oracle_top2_margin":0.9425995631950593,"near_tie":false,"n_options":2,"probs":[0.9713045505648925,0.028695449435107507],"hidden_sha256":"9e6efb02502b08cc13536b9b755ff189c313bf1145cf9449be4e691ff2db930d","first_in_process":false,"shard":"shard_04","index":9},{"id":"tv4x_composition_holdout_10","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00037513298030664166,"mean_abs_dp":0.00037513298030660003,"oracle_top2_margin":0.6069969730568061,"near_tie":false,"n_options":2,"probs":[0.8038736195087096,0.19612638049129033],"hidden_sha256":"b501aa2cf74557dc76974518f80b99ad412200ee5a1ed8eee43d06830d316305","first_in_process":false,"shard":"shard_04","index":10},{"id":"tv4x_composition_holdout_11","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007205762853866315,"mean_abs_dp":0.0007205762853865483,"oracle_top2_margin":0.6005328768420692,"near_tie":false,"n_options":2,"probs":[0.8009870147064211,0.1990129852935788],"hidden_sha256":"e2d988de68565f60b939d8773d3d7f32b8501c7efebe771ff9d48820229fe054","first_in_process":false,"shard":"shard_04","index":11},{"id":"tv4x_composition_holdout_12","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002086239169303644,"mean_abs_dp":0.0020862391693035748,"oracle_top2_margin":0.5443443270299198,"near_tie":false,"n_options":2,"probs":[0.2299140756543437,0.7700859243456564],"hidden_sha256":"4fcb7a3ec8e3615dab3a0a0dc382d1da621f8e715f12863efd459e8831c9e12c","first_in_process":false,"shard":"shard_04","index":12},{"id":"tv4x_composition_holdout_13","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.479338954727809e-05,"mean_abs_dp":3.4793389547158396e-05,"oracle_top2_margin":0.9560703155565883,"near_tie":false,"n_options":2,"probs":[0.021999635611253107,0.978000364388747],"hidden_sha256":"6460adaa2c8afaaa58e401883e1a61edb703e94364e4dcba45b34ed874a03b80","first_in_process":false,"shard":"shard_04","index":13},{"id":"tv4x_composition_holdout_14","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002086239169303644,"mean_abs_dp":0.0020862391693035748,"oracle_top2_margin":0.5443443270299198,"near_tie":false,"n_options":2,"probs":[0.2299140756543437,0.7700859243456564],"hidden_sha256":"4fcb7a3ec8e3615dab3a0a0dc382d1da621f8e715f12863efd459e8831c9e12c","first_in_process":false,"shard":"shard_04","index":14},{"id":"tv4x_composition_holdout_15","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002957088207970582,"mean_abs_dp":0.002957088207970554,"oracle_top2_margin":0.5847640916754826,"near_tie":false,"n_options":2,"probs":[0.21057504237022928,0.7894249576297707],"hidden_sha256":"8452877036502ae50f7533f5f030d8029a1d9004c696ac6b1231b64911138971","first_in_process":false,"shard":"shard_04","index":15},{"id":"tv4x_composition_holdout_16","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00026746370661734353,"mean_abs_dp":0.00026746370661729496,"oracle_top2_margin":0.7741124000066352,"near_tie":false,"n_options":2,"probs":[0.11267633629006515,0.8873236637099349],"hidden_sha256":"6bd81f6f53335ddf31825999737ad9b0c1400614002bf8a408bdadf6e307bddf","first_in_process":false,"shard":"shard_04","index":16},{"id":"tv4x_composition_holdout_17","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006705509750142558,"mean_abs_dp":0.0006705509750141864,"oracle_top2_margin":0.6852267962646376,"near_tie":false,"n_options":2,"probs":[0.15671605089266702,0.843283949107333],"hidden_sha256":"67b641ca84f1f684ab94213309e5d0181cd479833fd3e558d2856c34f4cb6d76","first_in_process":false,"shard":"shard_04","index":17},{"id":"tv4x_composition_holdout_18","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00026746370661734353,"mean_abs_dp":0.00026746370661729496,"oracle_top2_margin":0.7741124000066352,"near_tie":false,"n_options":2,"probs":[0.11267633629006515,0.8873236637099349],"hidden_sha256":"6bd81f6f53335ddf31825999737ad9b0c1400614002bf8a408bdadf6e307bddf","first_in_process":false,"shard":"shard_04","index":18},{"id":"tv4x_composition_holdout_19","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00024233577453537658,"mean_abs_dp":0.00024233577453536964,"oracle_top2_margin":0.7745030800582808,"near_tie":false,"n_options":2,"probs":[0.11250612419632416,0.8874938758036758],"hidden_sha256":"1bcf188b7edca20308c02dad7298604537977e7050473c35a57cbf591fddd695","first_in_process":false,"shard":"shard_04","index":19},{"id":"tv4s_00","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022050946128954196,"mean_abs_dp":0.0001470063075263778,"oracle_top2_margin":0.14786413611853794,"near_tie":false,"n_options":3,"probs":[0.42343417428127716,0.5708573652701466,0.005708460448576206],"hidden_sha256":"825f1414dfbb49e641fcb6e11baabeeccea2272f4a0e860e373809ee878819a6","first_in_process":false,"shard":"shard_04","index":20,"hidden":{"min_pos_cos":0.9999935669626713,"min_pos_cos_index":82,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4s_01","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001253363006913366,"mean_abs_dp":8.355753379419752e-05,"oracle_top2_margin":0.9603374696968201,"near_tie":false,"n_options":3,"probs":[0.01649697543683095,0.9770617289783831,0.006441295584786039],"hidden_sha256":"981f03ba44438b6e89171c5ce7f2b92d5f86ae6d5c7d73fdaedb0ae7096f4b72","first_in_process":false,"shard":"shard_04","index":21},{"id":"tv4s_02","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00044927336156797004,"mean_abs_dp":0.0002995155743786753,"oracle_top2_margin":0.32987924818870906,"near_tie":false,"n_options":3,"probs":[0.3321863347435049,0.6629606467456735,0.004853018510821741],"hidden_sha256":"afbb7ccd9f3a7ad0f4829ef22cacd1fa64ae2b2da8322d13b166b5447922f091","first_in_process":false,"shard":"shard_04","index":22},{"id":"tv4s_03","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007763252730743492,"mean_abs_dp":0.0005175501820495605,"oracle_top2_margin":0.8223114159276116,"near_tie":false,"n_options":3,"probs":[0.0034546869671605797,0.9086574662059403,0.08788784682689907],"hidden_sha256":"dc740a4ee41a94878d601304ac34d3ed356bce17c472a3cfbf7717bc9db96914","first_in_process":false,"shard":"shard_04","index":23},{"id":"tv4s_04","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2132629585058794e-05,"mean_abs_dp":1.4755086390072156e-05,"oracle_top2_margin":0.48158885519616973,"near_tie":false,"n_options":3,"probs":[0.2547244069341292,0.7363527833599398,0.00892280970593099],"hidden_sha256":"de7e27a9c2b7b3731dde46326f19728290c2954b7cc72c1d1fc1398f20dc1da7","first_in_process":false,"shard":"shard_04","index":24},{"id":"tv4s_05","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.013667370379338e-05,"mean_abs_dp":3.342444913584172e-05,"oracle_top2_margin":0.9705965340279149,"near_tie":false,"n_options":3,"probs":[0.005799485297682429,0.9823563508396558,0.011844163862661726],"hidden_sha256":"be2a93a3832482b606987db322322026a2151cc350b9065960d6de29b063c87c","first_in_process":false,"shard":"shard_04","index":25},{"id":"tv4s_06","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000525566268819444,"mean_abs_dp":0.0003503775125463342,"oracle_top2_margin":0.11280309184966997,"near_tie":false,"n_options":3,"probs":[0.4400725550539563,0.551851497292834,0.008075947653209561],"hidden_sha256":"597592305dcc4fa6b44c0103402daf8ed684ea668f473a00eaa9a6d82dc4cb46","first_in_process":false,"shard":"shard_04","index":26},{"id":"tv4s_07","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005167785014601328,"mean_abs_dp":0.00034451900097340105,"oracle_top2_margin":0.7816282229653853,"near_tie":false,"n_options":3,"probs":[0.04271091760158631,0.8689689430822535,0.08832013931616027],"hidden_sha256":"3d59ae34ef3e794e24fd5811f2b4f8b315378d9e723db10a040b6dea09cf9420","first_in_process":false,"shard":"shard_04","index":27},{"id":"tv4s_08","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001178159648369953,"mean_abs_dp":7.854397655797953e-05,"oracle_top2_margin":0.7841390923197227,"near_tie":false,"n_options":3,"probs":[0.10364968050784745,0.8875703242954268,0.008779995196725725],"hidden_sha256":"4302041aafca3152c52745f798f85fe1bdf28cb9a8564e984a2232b73da94bf6","first_in_process":false,"shard":"shard_04","index":28},{"id":"tv4s_09","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.086410799262417e-05,"mean_abs_dp":1.3909405328423342e-05,"oracle_top2_margin":0.9808333378930814,"near_tie":false,"n_options":3,"probs":[0.005713505202666117,0.9875448064601742,0.0067416883371597],"hidden_sha256":"c2882334041c6fe5e4d0eb1c6ca49f140547092fa86f667f0778802c6e4e644d","first_in_process":false,"shard":"shard_04","index":29},{"id":"tv4s_10","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009157530853800555,"mean_abs_dp":0.0006105020569200688,"oracle_top2_margin":0.4147412288568358,"near_tie":false,"n_options":3,"probs":[0.2853849771312857,0.7019449010317038,0.012670121837010566],"hidden_sha256":"e4032052b797088415e6fcf164d7da441b9f5439742d8edea20ccd51b6151191","first_in_process":false,"shard":"shard_04","index":30},{"id":"tv4s_11","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003931511722014003,"mean_abs_dp":0.000262100781467584,"oracle_top2_margin":0.5728354456080552,"near_tie":false,"n_options":3,"probs":[0.015499368026395642,0.779039592029395,0.20546103994420942],"hidden_sha256":"3260e255a882b349f7c2541e5584d35c8a07d8a4afd797b7df0f5f9cd4be1c21","first_in_process":false,"shard":"shard_04","index":31},{"id":"tv4s_12","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000943106289413409,"mean_abs_dp":0.0006287375262756072,"oracle_top2_margin":0.6046066435315647,"near_tie":false,"n_options":3,"probs":[0.7999474230448184,0.19722227384065197,0.0028303031145296594],"hidden_sha256":"944b7346fe18a4eb0e065e2c321264466665c5013d6e8499a6b57416f1028429","first_in_process":false,"shard":"shard_04","index":32},{"id":"tv4s_13","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002220091133238039,"mean_abs_dp":0.0001480060755492266,"oracle_top2_margin":0.8289862535749726,"near_tie":false,"n_options":3,"probs":[0.005156247001274795,0.9121310003505888,0.08271275264813645],"hidden_sha256":"8717792f60ae365ba1e3066d25d2ae42c42a9b7f470946ccbcfec9172c3613f3","first_in_process":false,"shard":"shard_04","index":33},{"id":"tv4s_14","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2454457328846544e-05,"mean_abs_dp":1.4969638219189396e-05,"oracle_top2_margin":0.9642054851284572,"near_tie":false,"n_options":3,"probs":[0.012391816058439068,0.9765534681789902,0.011054715762570697],"hidden_sha256":"edfcccfead02c328d533cd9da62c98b628899fc3b2d29df1844174e565f0067c","first_in_process":false,"shard":"shard_04","index":34},{"id":"tv4s_15","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.186812457528948e-05,"mean_abs_dp":6.124541638349047e-05,"oracle_top2_margin":0.9505295735609528,"near_tie":false,"n_options":3,"probs":[0.01638178329086242,0.9671442095562028,0.016474007152934865],"hidden_sha256":"5c966c239dc3378d49e623c36f198151ad78c7f0a4dd3e7adba5b38a0722f130","first_in_process":false,"shard":"shard_04","index":35},{"id":"tv4s_16","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003938001526033352,"mean_abs_dp":0.0026253343506888647,"oracle_top2_margin":0.13584732588717008,"near_tie":false,"n_options":3,"probs":[0.43415178300943086,0.5621426772581272,0.0037055397324420375],"hidden_sha256":"b453aa066e3f06ff906a010fd9567d07b1c06afaf64916f2949497a68477f413","first_in_process":false,"shard":"shard_04","index":36},{"id":"tv4s_17","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.3278457389734435e-05,"mean_abs_dp":1.551897159310685e-05,"oracle_top2_margin":0.9172811742066569,"near_tie":false,"n_options":3,"probs":[0.006845830544245207,0.9552247895003279,0.03792937995542691],"hidden_sha256":"93e45ce9b7d9c17879a32421498b27d78401441af3709500dc8eea4a33898e64","first_in_process":false,"shard":"shard_04","index":37},{"id":"tv4s_18","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.278862783809126e-05,"mean_abs_dp":4.8525751892014e-05,"oracle_top2_margin":0.9227063857483483,"near_tie":false,"n_options":3,"probs":[0.9600412371407516,0.03747570770450632,0.002483055154742175],"hidden_sha256":"e6d096c68e0a8ef4ba99a2dda510a4359d59cca15adaf0709e2b976ec1d57fd5","first_in_process":false,"shard":"shard_04","index":38},{"id":"tv4s_19","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.531859336591836e-05,"mean_abs_dp":2.3545728910586507e-05,"oracle_top2_margin":0.9819012797443343,"near_tie":false,"n_options":3,"probs":[0.006564607271109844,0.9885215176882236,0.004913875040666572],"hidden_sha256":"b941113fd4799e8af7465b33067b4c551c3b30d726b4f6802a3835fe0090b273","first_in_process":false,"shard":"shard_04","index":39},{"id":"semif_a3f18f3a63d45345942b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.1469271791186434e-05,"mean_abs_dp":1.431284786084242e-05,"oracle_top2_margin":0.9357222628286423,"near_tie":false,"n_options":3,"probs":[0.012136207799222254,0.026050424635484536,0.9618133675652933],"hidden_sha256":"59690eec4f0c13b6abbc1411ccb9bea698a48bbee6df03325c3fd7bf9f4e5ee1","first_in_process":true,"shard":"shard_05","index":0,"hidden":{"min_pos_cos":0.9999942033909348,"min_pos_cos_index":1,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"semif_f40beba9088c8db8bbd6","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.4924771695679073e-05,"mean_abs_dp":9.949847797144825e-06,"oracle_top2_margin":0.9328238790278026,"near_tie":false,"n_options":3,"probs":[0.9607169872787084,0.027884609564394897,0.011398403156896735],"hidden_sha256":"e0da945b52cc25e8a9dff93f62a5e378ea28aba3d8d840d71fc25509522e3820","first_in_process":false,"shard":"shard_05","index":1},{"id":"semif_0b43ea8e24e74d621f1b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.1670691865028893e-05,"mean_abs_dp":7.780461243374568e-06,"oracle_top2_margin":0.9216656021196827,"near_tie":false,"n_options":3,"probs":[0.03331372854298944,0.9549914482878533,0.011694823169157258],"hidden_sha256":"4e12397edfd1a1dd8a3641ff2e8e21b6430fd5f71f2b1978faf7f0b0a43efb9a","first_in_process":false,"shard":"shard_05","index":2},{"id":"semif_942ec65dac0bfb9874f9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00022426370598188639,"mean_abs_dp":0.00014950913732124835,"oracle_top2_margin":0.7123127373277265,"near_tie":false,"n_options":3,"probs":[0.8438141881761142,0.02426307419696414,0.13192273762692167],"hidden_sha256":"2c4724d7b2155eff945583e0e1cffd048d94d856143aaab0e32b848d84a1975a","first_in_process":false,"shard":"shard_05","index":3},{"id":"semif_e0c140e9222d1bce5647","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.119229819292251e-05,"mean_abs_dp":5.4128198795274154e-05,"oracle_top2_margin":0.9203280311161692,"near_tie":false,"n_options":3,"probs":[0.9533288853472699,0.013816445179608327,0.032854669473121864],"hidden_sha256":"c88bdd5616b6fa8679ac69de207a46551a0546e1c7f3dd4a2a93268b35abe3f2","first_in_process":false,"shard":"shard_05","index":4},{"id":"semif_8e8c3804a3c15ebb31e3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0015891505010201534,"mean_abs_dp":0.0010594336673467597,"oracle_top2_margin":0.23989506177672987,"near_tie":false,"n_options":3,"probs":[0.5983762421597949,0.04005170831547836,0.3615720495247267],"hidden_sha256":"129454b09d12e88265eed8209b80879e31ef8a8e79ffbdf524371b7235a2f9f3","first_in_process":false,"shard":"shard_05","index":5},{"id":"semif_56c419b12cd37cbe790c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002473280513329845,"mean_abs_dp":0.00016488536755536348,"oracle_top2_margin":0.8988903795476691,"near_tie":false,"n_options":3,"probs":[0.934195153195683,0.030858198547212515,0.03494664825710447],"hidden_sha256":"fafec302a9facd930e92a8d1c06dfcfe0eae21e6e51285866bb8e975a157f0e4","first_in_process":false,"shard":"shard_05","index":6},{"id":"semif_75b470586834df8c568d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":3.9029401224549565e-05,"mean_abs_dp":2.6019600816340933e-05,"oracle_top2_margin":0.6736416663487602,"near_tie":false,"n_options":3,"probs":[0.01880372030114218,0.15374830349567972,0.8274479762031781],"hidden_sha256":"cd754d53086d0c2df6cd937049c731ebd307e3189e9b0ce68d04143a3543a1e7","first_in_process":false,"shard":"shard_05","index":7},{"id":"semif_834f3b3e10cc61b33d25","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.219952027597948e-05,"mean_abs_dp":8.133013517293055e-06,"oracle_top2_margin":0.9604540417401267,"near_tie":false,"n_options":3,"probs":[0.010224840197946955,0.0146694395084033,0.9751057202936496],"hidden_sha256":"4de6fbd2d4280ef8e118dfc3d2ebbeeba8196b2d954c9d00e89cbf8511221fe8","first_in_process":false,"shard":"shard_05","index":8},{"id":"semif_76c7a79b1ff2363a60c5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.3205475271635976e-06,"mean_abs_dp":2.213698351417534e-06,"oracle_top2_margin":0.9636078315972091,"near_tie":false,"n_options":3,"probs":[0.9773776815918277,0.013764975528799234,0.008857342879373124],"hidden_sha256":"63c540ecfde068c3c9d5a9275fcc1da1b0aa25c9cf279d81bcee49ba7388d459","first_in_process":false,"shard":"shard_05","index":9},{"id":"semif_36c71bb649b7d5eccb1f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.060864569816914e-05,"mean_abs_dp":3.373909713214283e-05,"oracle_top2_margin":0.9650002382306553,"near_tie":false,"n_options":3,"probs":[0.9780551544015423,0.013131788612404213,0.00881305698605344],"hidden_sha256":"f93ae0636491c9e24591aee10be9651d81f10f2064086e3fb22fbc2ba3e76cfa","first_in_process":false,"shard":"shard_05","index":10},{"id":"semif_eac5121a4a1bf391a08c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":64,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011743894750693151,"mean_abs_dp":7.82926316712761e-05,"oracle_top2_margin":0.8647753102944388,"near_tie":false,"n_options":3,"probs":[0.04796409602429535,0.9129479166268055,0.039087987348899164],"hidden_sha256":"4392b337e8ee7c840d82df44fce1b75d4a12d97a766ee707f39202d1985dfad8","first_in_process":false,"shard":"shard_05","index":11},{"id":"semif_44574c44e243ee1b2f52","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.0759784860253988e-05,"mean_abs_dp":7.1731899068620125e-06,"oracle_top2_margin":0.9659716866106711,"near_tie":false,"n_options":3,"probs":[0.01034640890347426,0.011844024229873224,0.9778095668666524],"hidden_sha256":"d07b324cf107738de3e290cb744d6102b1bdbcdec8768b07c846d9382c9a87c1","first_in_process":false,"shard":"shard_05","index":12},{"id":"semif_2608370d01783735c3ea","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.5798521968020154e-05,"mean_abs_dp":3.719901464530167e-05,"oracle_top2_margin":0.9215895668087061,"near_tie":false,"n_options":3,"probs":[0.02066594480894145,0.028920287192775465,0.950413767998283],"hidden_sha256":"3e34d91096801929486ae9c41017317403d2fbbb96240f7a827ffd39300cd552","first_in_process":false,"shard":"shard_05","index":13},{"id":"semif_ec94ef909f7918695ab3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.707751752332847e-05,"mean_abs_dp":4.4718345015577754e-05,"oracle_top2_margin":0.9318247409749322,"near_tie":false,"n_options":3,"probs":[0.023701971439939507,0.02087721796124078,0.9554208105988198],"hidden_sha256":"9c69a15297c17330150edd3801b1b502366ced08a91108a21d4379699103ef40","first_in_process":false,"shard":"shard_05","index":14},{"id":"semif_03d919fd0e9b3a7cd1e6","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.2541793530475298e-05,"mean_abs_dp":8.361195686981219e-06,"oracle_top2_margin":0.8733675366733726,"near_tie":false,"n_options":3,"probs":[0.9245912357287426,0.024171951319128006,0.05123681295212939],"hidden_sha256":"b124a646b8e5b0b19eb9fa152ac1c045ebb115d59b4645057a16de41c8d87c3c","first_in_process":false,"shard":"shard_05","index":15},{"id":"semif_86a15618ba4cb90a307e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.1760474510791319e-05,"mean_abs_dp":7.840316340512512e-06,"oracle_top2_margin":0.9348419215531181,"near_tie":false,"n_options":3,"probs":[0.9581561718864129,0.023296876186595122,0.01854695192699201],"hidden_sha256":"e444124312779a7a949339de49495da1e123b4ecb70b7e628c040aada0ada725","first_in_process":false,"shard":"shard_05","index":16},{"id":"semif_3dd9c3ac98f8ddf7bdd3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.3539506604507848e-05,"mean_abs_dp":1.5693004402986727e-05,"oracle_top2_margin":0.9457231579098606,"near_tie":false,"n_options":3,"probs":[0.018856786534800114,0.9645874393215639,0.01655577414363605],"hidden_sha256":"7142a592fb3e011fa8d582059916007215b7d1daa06fcb978b29e51b3980b8ee","first_in_process":false,"shard":"shard_05","index":17},{"id":"semif_36273dd4777876a26338","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.7000066091315694e-05,"mean_abs_dp":1.800004406089679e-05,"oracle_top2_margin":0.9446003626832611,"near_tie":false,"n_options":3,"probs":[0.010798832446977906,0.9668900402630594,0.022311127289962727],"hidden_sha256":"208af093ee7eb62d86c6b71079fd64f252c8cf5d7b65864f60d1f2848f9b1333","first_in_process":false,"shard":"shard_05","index":18},{"id":"semif_07ee70cfe4b01330ce89","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00017564070560352474,"mean_abs_dp":0.00011709380373565424,"oracle_top2_margin":0.5487147306168284,"near_tie":false,"n_options":3,"probs":[0.026985200562276025,0.21200731001023654,0.7610074894274874],"hidden_sha256":"5c3bfd884eb2eec5da8f82ea9f2953c9c440ae872b5c87f2cd7f5ff7ca9d1b72","first_in_process":false,"shard":"shard_05","index":19},{"id":"semif_51ceab3ce622c7c759f1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.167147699749794e-05,"mean_abs_dp":1.4447651331702302e-05,"oracle_top2_margin":0.9406567372660729,"near_tie":false,"n_options":3,"probs":[0.01613310523520686,0.02158614608452823,0.9622807486802648],"hidden_sha256":"ee57bb43862e0d4ff0405288f4dd4a5fa2e7c396432c55798d7b4b1bae225ee3","first_in_process":false,"shard":"shard_05","index":20},{"id":"semif_b8091be929b1e489fcc4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00010149939691750376,"mean_abs_dp":6.766626461165125e-05,"oracle_top2_margin":0.9159577193463836,"near_tie":false,"n_options":3,"probs":[0.01395086831772432,0.9509108579921883,0.03513827369008734],"hidden_sha256":"8f518229bc62462c1143ef5efc35fc756962bca44121bf1ae8cb074c0c2c71f8","first_in_process":false,"shard":"shard_05","index":21},{"id":"semif_c89b86e3f55123bfa506","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.945500879811494e-05,"mean_abs_dp":3.963667253207431e-05,"oracle_top2_margin":0.9192964850975687,"near_tie":false,"n_options":3,"probs":[0.024735354711780747,0.947326637625112,0.02793800766310718],"hidden_sha256":"e5a79a893222f9843a303c0257327a07e3f022768a3a71938712e09f52ede8a9","first_in_process":false,"shard":"shard_05","index":22},{"id":"semif_3b0958cabc339005cf66","name":"answer","variant":"base","source":"semif","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002291561896069494,"mean_abs_dp":0.00015277079307125566,"oracle_top2_margin":0.8631264727730906,"near_tie":false,"n_options":3,"probs":[0.9115855529637362,0.0487748401889092,0.03963960684735453],"hidden_sha256":"b233fccd314ea24d53fd0ed06b83103d87c6ae15b3d381330385352c0d1f5164","first_in_process":false,"shard":"shard_05","index":23},{"id":"semif_f2b4ec4930322fe45118","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.6407648097314116e-05,"mean_abs_dp":2.427176539816084e-05,"oracle_top2_margin":0.9040631902962722,"near_tie":false,"n_options":3,"probs":[0.03923588440486526,0.943357867856025,0.017406247739109756],"hidden_sha256":"da7211ba9474e4226180291d38728ec81a4d86f4771ab133962b3fecf965f977","first_in_process":false,"shard":"shard_05","index":24},{"id":"semif_61ac20b07a5189231e15","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.8841964898957952e-05,"mean_abs_dp":1.2561309932665235e-05,"oracle_top2_margin":0.9283512811406358,"near_tie":false,"n_options":3,"probs":[0.015262802701624872,0.9565437583615224,0.028193438936852767],"hidden_sha256":"a19ec71f5d7a269ceefbf27a583793cb15f972052f8e49e77b9e3d2d865ecfcd","first_in_process":false,"shard":"shard_05","index":25},{"id":"semif_4b3d794c004566610181","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00012785676809368152,"mean_abs_dp":8.523784539581543e-05,"oracle_top2_margin":0.8413288291611345,"near_tie":false,"n_options":3,"probs":[0.036045450837060684,0.06120512130729042,0.9027494278556489],"hidden_sha256":"43bf9d4d1c4d7fdb613ba5049e9fcc2b403464843b111653193102eaeb768caf","first_in_process":false,"shard":"shard_05","index":26},{"id":"semif_b7c4e62d07986286fce0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004231455182535371,"mean_abs_dp":0.00028209701216906174,"oracle_top2_margin":0.4801013037391402,"near_tie":false,"n_options":3,"probs":[0.6733884435764005,0.13397325551956474,0.19263830090403486],"hidden_sha256":"dc4a55eb488c212702bd200553cc127de166ac3bc2ce230492ef7a107332a874","first_in_process":false,"shard":"shard_05","index":27},{"id":"semif_9473603b44af63beab79","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.069921055653957e-05,"mean_abs_dp":2.046614037099284e-05,"oracle_top2_margin":0.7943738126119272,"near_tie":false,"n_options":3,"probs":[0.888262299317294,0.09385759450545128,0.0178801061772548],"hidden_sha256":"ab9e51faa75ccdd7fe74ebd74785d360cf1bd56c27835111c5033921deef2d05","first_in_process":false,"shard":"shard_05","index":28},{"id":"semif_d41072dc35d8cd14c056","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00016770154153533667,"mean_abs_dp":0.00011180102769015737,"oracle_top2_margin":0.8433094179932079,"near_tie":false,"n_options":3,"probs":[0.03148795803010232,0.9060758092826907,0.062436232687207105],"hidden_sha256":"8dee057249bf4d88c90c4eff03215c3a9f9c606d644b2158e01063ee8e767f3f","first_in_process":false,"shard":"shard_05","index":29},{"id":"semif_33d6e5da58f0fee2490d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018126650991328042,"mean_abs_dp":0.0012084433994218973,"oracle_top2_margin":0.2551036769378849,"near_tie":false,"n_options":3,"probs":[0.5462157439015011,0.29361806181260414,0.16016619428589482],"hidden_sha256":"f738831e2528ab023412f1a1a1c8e19d4bf2a162c19b7d9758762c0241bd7436","first_in_process":false,"shard":"shard_05","index":30},{"id":"semif_823bf736b92e9d8fa66d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00012185771192207984,"mean_abs_dp":8.123847461471989e-05,"oracle_top2_margin":0.11576403794400497,"near_tie":false,"n_options":3,"probs":[0.07855712518463019,0.40295457725957345,0.5184882975557964],"hidden_sha256":"ec5e231fc1ecb1a95245e4ce0aad42972cbc3815c4d22f5b4734bae2a5228772","first_in_process":false,"shard":"shard_05","index":31},{"id":"semif_8a4c3de28be95c105ca1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00022001070863007094,"mean_abs_dp":0.0001466738057533465,"oracle_top2_margin":0.8881225713951193,"near_tie":false,"n_options":3,"probs":[0.014334209189989428,0.048988212596581586,0.936677578213429],"hidden_sha256":"eea4e46a70216d8bf5c734b2e7e0f03940652903751b0c5db364cacde8ff08df","first_in_process":false,"shard":"shard_05","index":32},{"id":"semif_ea7ec66a5e8eac886548","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.252636775224026e-05,"mean_abs_dp":1.5017578501521264e-05,"oracle_top2_margin":0.7170225342159673,"near_tie":false,"n_options":3,"probs":[0.8355985960196999,0.11859935732512457,0.045802046655175566],"hidden_sha256":"307449481e8414d4dbbf0b9e7ea37fa5ff69841fbf3574418f7e2354bd871a7d","first_in_process":false,"shard":"shard_05","index":33},{"id":"semif_25172cd6ddc49ab0ca70","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.616417670585572e-05,"mean_abs_dp":3.744278447055197e-05,"oracle_top2_margin":0.7725567740197001,"near_tie":false,"n_options":3,"probs":[0.055463550999553336,0.08599421997369039,0.8585422290267563],"hidden_sha256":"9b43883938e370fb2e6ec15a02e39119b9e1e120f5552d87ac2ee73fb0cd8a3d","first_in_process":false,"shard":"shard_05","index":34},{"id":"semif_fb73a85d386cf59b0c51","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005262501485071569,"mean_abs_dp":0.0003508334323381231,"oracle_top2_margin":0.41852063761693903,"near_tie":false,"n_options":3,"probs":[0.2799015213825074,0.02272169474072508,0.6973767838767675],"hidden_sha256":"3a0e8d4345c925fd13ddbf1c47c0cdaec39c034a08757f5788d1ccc6538e26c9","first_in_process":false,"shard":"shard_05","index":35},{"id":"semif_a02fd582d2881cc462b2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.385218520812137e-05,"mean_abs_dp":9.234790138745267e-06,"oracle_top2_margin":0.9347901747190903,"near_tie":false,"n_options":3,"probs":[0.014366944471752404,0.025408969380010565,0.960224086148237],"hidden_sha256":"1fd4c14423339c985927ee6da0a8918526f9d89981fae89b64f8ec8d0f8c3caf","first_in_process":false,"shard":"shard_05","index":36},{"id":"semif_19f37d5a97034bd7d04f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2254765718994485e-05,"mean_abs_dp":1.4836510479320405e-05,"oracle_top2_margin":0.9070000108695007,"near_tie":false,"n_options":3,"probs":[0.03760862605760574,0.944604734453316,0.01778663948907815],"hidden_sha256":"756d015908f88c560973599fc792011efef8334f2561b79a8bb1297bf892c0aa","first_in_process":false,"shard":"shard_05","index":37},{"id":"semif_fda0ca94652c6d739ecf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00016715802429556614,"mean_abs_dp":0.00011143868286369919,"oracle_top2_margin":0.8966135104773004,"near_tie":false,"n_options":3,"probs":[0.938034062223987,0.02083165163800822,0.041134286138004725],"hidden_sha256":"793fa73b3b787b14d8c2d62578bfe3ce439b2e2c7261dd43097c941c0b6d6369","first_in_process":false,"shard":"shard_05","index":38},{"id":"semif_d0e36c4cb10a29b3191c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013958476868553005,"mean_abs_dp":9.305651245704084e-05,"oracle_top2_margin":0.15133175652298392,"near_tie":false,"n_options":3,"probs":[0.40518655529182374,0.03805294540579169,0.5567604993023846],"hidden_sha256":"b37b32877aef5a95abd00efcee4878d81c26f2ebcac4cc0d7de119c22831ea1a","first_in_process":false,"shard":"shard_05","index":39},{"id":"semif_a0a39ace7e5337d038a7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.311828337897559e-05,"mean_abs_dp":1.541218891932689e-05,"oracle_top2_margin":0.9627691549734159,"near_tie":false,"n_options":3,"probs":[0.9766259590266253,0.009478316208495324,0.013895724764879398],"hidden_sha256":"a0935026fa6893203325db2c651a767941b9cb252f24bf4b6a59371c53afc226","first_in_process":true,"shard":"shard_06","index":0},{"id":"semif_eeae283841ce75dd25bf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.814861396589375e-05,"mean_abs_dp":2.5432409310608556e-05,"oracle_top2_margin":0.9621668481066167,"near_tie":false,"n_options":3,"probs":[0.013656234330689576,0.9757536034004749,0.01059016226883553],"hidden_sha256":"177d70eac3c1f8bdf4ece371002042856cda801cdec6116c08df623676bcff35","first_in_process":false,"shard":"shard_06","index":1},{"id":"semif_00bb4cc043f7d2b288d4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":4.9953871843255726e-05,"mean_abs_dp":3.330258122885045e-05,"oracle_top2_margin":0.9653603614640307,"near_tie":false,"n_options":3,"probs":[0.012864858352462706,0.00881883940392609,0.9783163022436112],"hidden_sha256":"07e0878f01ba76de456aa33f77307f94dd30849881c933c442b1a2e6058a3f26","first_in_process":false,"shard":"shard_06","index":2},{"id":"semif_1a81ead69e93e5d0c86a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008025769515977998,"mean_abs_dp":0.0005350513010651906,"oracle_top2_margin":0.2893138199873723,"near_tie":false,"n_options":3,"probs":[0.08924484928875517,0.31000926215474917,0.6007458885564957],"hidden_sha256":"3e4a49786415be51dd7600ebc3be606632f1c90fc45895b17eb6e77d74310180","first_in_process":false,"shard":"shard_06","index":3},{"id":"semif_4444dad25e438f128673","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000639521718126923,"mean_abs_dp":0.00042634781208463385,"oracle_top2_margin":0.15678293022597928,"near_tie":false,"n_options":3,"probs":[0.35842185908414925,0.12740665307934193,0.5141714878365088],"hidden_sha256":"bc0b0abc646aa2e136ca5b134206f2fcb813b13e4cfca9989675ceb4191db5d6","first_in_process":false,"shard":"shard_06","index":4},{"id":"semif_3226202ba1ada2570e9f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002555563099304048,"mean_abs_dp":0.00017037087328694578,"oracle_top2_margin":0.23422510399500152,"near_tie":false,"n_options":3,"probs":[0.536421784926457,0.16180355722802803,0.3017746578455151],"hidden_sha256":"e9ac499238fc3fdcf4d9652a8b7502a504abd3e3403c64afd5678f96ceecffdb","first_in_process":false,"shard":"shard_06","index":5},{"id":"semif_3f783eec5c90812479bc","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00015365927211918873,"mean_abs_dp":0.00010243951474613276,"oracle_top2_margin":0.5643859812186472,"near_tie":false,"n_options":3,"probs":[0.049198331278722016,0.7574456536993446,0.19335601502193348],"hidden_sha256":"192f382c52d282145c759c9afd4954de1e26c8d8aeb9f7403ed471ad6a06b80c","first_in_process":false,"shard":"shard_06","index":6},{"id":"semif_ffee861474a9665a093a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007911134018515065,"mean_abs_dp":0.0005274089345676247,"oracle_top2_margin":0.06305327507600167,"near_tie":false,"n_options":3,"probs":[0.4847263824808066,0.09229983286213984,0.42297378465705354],"hidden_sha256":"49e577fe954b410cfffea2087e8233e53494d7f91d7548eefd1887a97469cf7d","first_in_process":false,"shard":"shard_06","index":7},{"id":"semif_8c992e17e4fb54b044f5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.308983337292075e-06,"mean_abs_dp":5.539322224832471e-06,"oracle_top2_margin":0.950739141460235,"near_tie":false,"n_options":3,"probs":[0.012372895687156665,0.9691753920073765,0.018451712305466945],"hidden_sha256":"a4b6ca242b3e5cc61b35d96f300267b00a8ef70b677a30a7ed70f9d4d04fd724","first_in_process":false,"shard":"shard_06","index":8},{"id":"semif_695dd61195ca025c4f4c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012061974445110035,"mean_abs_dp":8.041316296741759e-05,"oracle_top2_margin":0.9013869599755495,"near_tie":false,"n_options":3,"probs":[0.9462622226420212,0.04466109081650877,0.009076686541469993],"hidden_sha256":"357b19f96322986ee5c5d33da463aafdce466c5232e4d42b374af550f1e9865e","first_in_process":false,"shard":"shard_06","index":9},{"id":"semif_e6155431ab7b0f8170d4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00037943498656678454,"mean_abs_dp":0.00025295665771123593,"oracle_top2_margin":0.3056350596030686,"near_tie":false,"n_options":3,"probs":[0.25700942078991557,0.5620324733303775,0.18095810587970698],"hidden_sha256":"9fd5a6104c04a2f4a82c01da20d94c9bb0136bfe2f0a0afa31a56658cfa03ca9","first_in_process":false,"shard":"shard_06","index":10},{"id":"semif_b49ecda75086d8c68d2a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008000190868123758,"mean_abs_dp":0.0005333460578749172,"oracle_top2_margin":0.4729230733547011,"near_tie":false,"n_options":3,"probs":[0.0702605699537122,0.22765504869830525,0.7020843813479826],"hidden_sha256":"91e1df83274c892ad92958b7acffb50d1f365f7d73718871614012e822bb8d9c","first_in_process":false,"shard":"shard_06","index":11},{"id":"semif_587ced261c6092cd1325","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022730657195790815,"mean_abs_dp":0.00015153771463852217,"oracle_top2_margin":0.7011503759417472,"near_tie":false,"n_options":3,"probs":[0.13174482670274731,0.8325001198390057,0.03575505345824683],"hidden_sha256":"2079aaf4b83e80ab1956b94746e04bb39106e7f25f65bbd9dc09ff3bee1fb2e2","first_in_process":false,"shard":"shard_06","index":12},{"id":"semif_533d4423d311de82b27d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.522964093167108e-05,"mean_abs_dp":6.348642728777494e-05,"oracle_top2_margin":0.9438360499050665,"near_tie":false,"n_options":3,"probs":[0.0236400998756841,0.9676577429719666,0.008702157152349397],"hidden_sha256":"2877bbffd61df6358bb20de917a939882b99ded3df7b220644b75e1ae1007321","first_in_process":false,"shard":"shard_06","index":13},{"id":"semif_2bf540d515a51755b25c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007537840374804983,"mean_abs_dp":0.0005025226916536609,"oracle_top2_margin":0.5672376377073012,"near_tie":false,"n_options":3,"probs":[0.18355226669852656,0.06684266460436256,0.749605068697111],"hidden_sha256":"ae201692eb06591951e7c5e9828f07c1ef8d1efd9b8f1a3fa4f96bf2823b4641","first_in_process":false,"shard":"shard_06","index":14},{"id":"semif_fdb9fc214aea2710763a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":65,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005245945340045521,"mean_abs_dp":0.00034972968933639587,"oracle_top2_margin":0.3787248844675741,"near_tie":false,"n_options":3,"probs":[0.10511431636394453,0.25855894325665135,0.6363267403794042],"hidden_sha256":"1af2a1c20c045e9595fd51c72d7ce0a9ac471372c0d1c91cce673954a6ff27d4","first_in_process":false,"shard":"shard_06","index":15},{"id":"semif_f6d68126b43f8c9ed183","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.939043304597046e-05,"mean_abs_dp":3.292695536394677e-05,"oracle_top2_margin":0.9339226656619631,"near_tie":false,"n_options":3,"probs":[0.026836099721224003,0.9608515044282198,0.012312395850556266],"hidden_sha256":"7a828c5de57568f9439d51da8ca91640cfc19e359f4367ecbe02d0a390a9c5db","first_in_process":false,"shard":"shard_06","index":16},{"id":"semif_44bae0c8a1615a096c43","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00019546930737035506,"mean_abs_dp":0.00013031287158018814,"oracle_top2_margin":0.8186553468489266,"near_tie":false,"n_options":3,"probs":[0.8954638052457854,0.02804410089416873,0.07649209386004589],"hidden_sha256":"f3c85744c29d4d7bae0f1938e1865e5ca715cd4cab7a7ed2e0923f980434adf7","first_in_process":false,"shard":"shard_06","index":17},{"id":"semif_2175379f4c69e6207626","name":"answer","variant":"base","source":"semif","type":"choice","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001156097673778611,"mean_abs_dp":0.0007707317825191018,"oracle_top2_margin":0.30882585389383266,"near_tie":false,"n_options":3,"probs":[0.5954457271533037,0.28825188289866915,0.11630238994802714],"hidden_sha256":"1f73b15b908fbabf62593bf290f87df87a690b86556296b363ce1e3abe1da99e","first_in_process":false,"shard":"shard_06","index":18},{"id":"semif_def9fe59d262357a673a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001262534371368973,"mean_abs_dp":0.0008416895809126579,"oracle_top2_margin":0.02681563599995257,"near_tie":false,"n_options":3,"probs":[0.42002200046409005,0.4491874634477848,0.13079053608812508],"hidden_sha256":"1ff4cda510633cc0290ccfd01d6196576730b800935185b2b4014ba91111484a","first_in_process":false,"shard":"shard_06","index":19},{"id":"semif_712f6ffb763eef7c9a12","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0003129803393967512,"mean_abs_dp":0.0002086535595978619,"oracle_top2_margin":0.4567225519567792,"near_tie":false,"n_options":3,"probs":[0.1941702737152666,0.15431946116665807,0.6515102651180753],"hidden_sha256":"31c543b0b13dc1a2fddc15569b60733bb59cb4ef5b62199ef463fcaae8e03463","first_in_process":false,"shard":"shard_06","index":20},{"id":"semif_e1d610bd14e3d16c09b9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005008875484687481,"mean_abs_dp":0.0003339250323125149,"oracle_top2_margin":0.7417275778640565,"near_tie":false,"n_options":3,"probs":[0.857529322868921,0.116802495066494,0.025668182064584984],"hidden_sha256":"4bd9055e8f4325c27aaa90eed002c024685b8f7a5e35ece891fc042808bab326","first_in_process":false,"shard":"shard_06","index":21},{"id":"semif_22ec57822db50ec121cf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.751644918374545e-05,"mean_abs_dp":5.167763278917983e-05,"oracle_top2_margin":0.9450968055693952,"near_tie":false,"n_options":3,"probs":[0.022465394591519035,0.9676992942927827,0.009835311115698268],"hidden_sha256":"39d1596926b781a8d49b57f799a19e969cf5ca6406417a9447abb2ddd198f88f","first_in_process":false,"shard":"shard_06","index":22},{"id":"semif_3e75f2b84e8e08e3d3f2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":7.032464342182632e-05,"mean_abs_dp":4.688309561459714e-05,"oracle_top2_margin":0.7381421705328148,"near_tie":false,"n_options":3,"probs":[0.09715686415938468,0.06765379686624787,0.8351893389743675],"hidden_sha256":"8c2004cda0b1f14d45c40fdc08ea4ffe33975de23b4bb6e74a551710b0ba86b0","first_in_process":false,"shard":"shard_06","index":23},{"id":"semif_ebd74aa05324a1556288","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.525820094693824e-06,"mean_abs_dp":4.350546729814386e-06,"oracle_top2_margin":0.9259538655050306,"near_tie":false,"n_options":3,"probs":[0.031615134239200184,0.9575750897066786,0.010809776054121185],"hidden_sha256":"7960d38b3873aeeab4fc7bd4b90729d725cdfc91b4fb6dddc5f013cbf2a57ad8","first_in_process":false,"shard":"shard_06","index":24},{"id":"semif_d7e84e4245a29817da1f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00023769042944077334,"mean_abs_dp":0.00015846028629385757,"oracle_top2_margin":0.9078157257003796,"near_tie":false,"n_options":3,"probs":[0.040634424387509076,0.01136700807160691,0.947998567540884],"hidden_sha256":"d692b6f02f1aa82469c8c8c5dfcef67b7a9c09b6c185eb96bd6a4180bf15467d","first_in_process":false,"shard":"shard_06","index":25},{"id":"semif_f41ae66b3c09630c7a42","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00035290411896993223,"mean_abs_dp":0.00023526941264663073,"oracle_top2_margin":0.8487522382658496,"near_tie":false,"n_options":3,"probs":[0.059742418522600545,0.031154811147407174,0.9091027703299923],"hidden_sha256":"9f8eb2ba59835bc5b7f1efec8094454148fc15b6fe1c15239d7c29172f5afe32","first_in_process":false,"shard":"shard_06","index":26},{"id":"semif_82c2cd97c160ad27951b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003946565183082784,"mean_abs_dp":0.0002631043455388245,"oracle_top2_margin":0.5084756035543895,"near_tie":false,"n_options":3,"probs":[0.6802166236076296,0.14799593230813707,0.17178744408423333],"hidden_sha256":"88efdc021766228423250d39afa8c8c86d2e86e6caf060afa54f5947b3845558","first_in_process":false,"shard":"shard_06","index":27},{"id":"semif_7ada97b6f99f4788e5c7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014130180216392896,"mean_abs_dp":9.42012014425823e-05,"oracle_top2_margin":0.6666932267467534,"near_tie":false,"n_options":3,"probs":[0.8087129513294926,0.04927220841222164,0.14201484025828567],"hidden_sha256":"57d902d8d794dda0a77d441ed64820dc6f3130c139edfbb221ad821c600f11ec","first_in_process":false,"shard":"shard_06","index":28},{"id":"semif_4d7a003342db5b0a17e3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.2755925237022125e-06,"mean_abs_dp":8.503950158396389e-07,"oracle_top2_margin":0.9548245786970802,"near_tie":false,"n_options":3,"probs":[0.017279392456111608,0.9721035989489094,0.010617008594978909],"hidden_sha256":"f0502f5a0c2d8e0d8c2c77521139738b6346a48614e6c667119f6fd13decb251","first_in_process":false,"shard":"shard_06","index":29},{"id":"semif_19d3d8d0bf773e88be7b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.662196871869529e-05,"mean_abs_dp":3.774797914575639e-05,"oracle_top2_margin":0.9590320722308011,"near_tie":false,"n_options":3,"probs":[0.974709491896997,0.009708000124052051,0.015582507978951084],"hidden_sha256":"9065210ef4a5557267f5f66baff9791ad002f7970283a58d7848deab2ee43cac","first_in_process":false,"shard":"shard_06","index":30},{"id":"semif_d6de731cca52359b1c87","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011972292376880023,"mean_abs_dp":7.981528251252423e-05,"oracle_top2_margin":0.14805592269858542,"near_tie":false,"n_options":3,"probs":[0.45711075980907134,0.3091389750160811,0.2337502651748476],"hidden_sha256":"27eb7cc9ebef086684ea4b3ee036e4d15d961098c942e88ee24c3317dadbdc0c","first_in_process":false,"shard":"shard_06","index":31},{"id":"semif_b45c63fb92729af27f05","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.04718543497448e-06,"mean_abs_dp":1.3647902899581222e-06,"oracle_top2_margin":0.9420362994502539,"near_tie":false,"n_options":3,"probs":[0.9634540284224903,0.02142100547055134,0.015124966106958435],"hidden_sha256":"bfd8a4d87e265d648ef2638460acebccdc565f3a127023729f126d51bccdca68","first_in_process":false,"shard":"shard_06","index":32},{"id":"semif_46b7029b9a704138b77a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":87,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.263630970062452e-05,"mean_abs_dp":3.509087313373002e-05,"oracle_top2_margin":0.9326528003885701,"near_tie":false,"n_options":3,"probs":[0.025265089783023192,0.016721743444472786,0.958013166772504],"hidden_sha256":"a3f9ba0113f5b28f5291ce0d1dfa8c56228ccb05396f869c4bc117f41c4e81a4","first_in_process":false,"shard":"shard_06","index":33},{"id":"semif_caf773b23934fa7362a1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.0731963494254906e-05,"mean_abs_dp":3.3821308996157214e-05,"oracle_top2_margin":0.9489755571809984,"near_tie":false,"n_options":3,"probs":[0.9668676313481896,0.017806525853644756,0.01532584279816558],"hidden_sha256":"23e2b54da7853344bb25af98997c5bffd717057666f5afe74d3f3729bc072aff","first_in_process":false,"shard":"shard_06","index":34},{"id":"semif_f46f392ef9e9e9df564b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0011356731773546347,"mean_abs_dp":0.0007571154515697646,"oracle_top2_margin":0.4778150088143989,"near_tie":false,"n_options":3,"probs":[0.01626296994645718,0.25409205623688014,0.7296449738166627],"hidden_sha256":"f0d20e63a4f0252da1838b4123eba2ca635c6df5624c97945386d368b45358fa","first_in_process":false,"shard":"shard_06","index":35},{"id":"semif_005dcf6d2c0f21d64796","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003755298986527489,"mean_abs_dp":0.00025035326576851313,"oracle_top2_margin":0.38216002159483015,"near_tie":false,"n_options":3,"probs":[0.10151572781020499,0.6400054879247236,0.2584787842650715],"hidden_sha256":"a53279a2d24f002f12721122957d5dd75d123a738363737878c2eb98b6360249","first_in_process":false,"shard":"shard_06","index":36},{"id":"semif_edb708bd0fe5fd67d6ee","name":"answer","variant":"base","source":"semif","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00037579160202522166,"mean_abs_dp":0.00025052773468345335,"oracle_top2_margin":0.44750541759432194,"near_tie":false,"n_options":3,"probs":[0.13598083124296936,0.6558461171589416,0.20817305159808905],"hidden_sha256":"25ae7fe9d62bfc44c2dda3a1cbcc95dcec097f11cb01dda378628351a0559a88","first_in_process":false,"shard":"shard_06","index":37},{"id":"semif_97a35499bd8717f3e786","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004305003102396121,"mean_abs_dp":0.0002870002068264173,"oracle_top2_margin":0.7893637810778941,"near_tie":false,"n_options":3,"probs":[0.027685719323045536,0.09105373471364955,0.8812605459633049],"hidden_sha256":"43f359a23746f19b3b8b2cdc39e8f59b5ac4c18b45cfa9f146cd3847f821d987","first_in_process":false,"shard":"shard_06","index":38},{"id":"semif_31eb4533d2c89959ec46","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003241269430585003,"mean_abs_dp":0.00021608462870571776,"oracle_top2_margin":0.7023103058071338,"near_tie":false,"n_options":3,"probs":[0.8169575205780701,0.11407505838138732,0.06896742104054247],"hidden_sha256":"14792d54332849e92bb0cf2a977b45aa3641f11c27dda15ae626c5d3282e2fbe","first_in_process":false,"shard":"shard_06","index":39},{"id":"semif_0aa24a6701ac9c56c4a0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018412177386255002,"mean_abs_dp":0.0012274784924169956,"oracle_top2_margin":0.04788238825654201,"near_tie":false,"n_options":3,"probs":[0.4389773519018687,0.4903552204270444,0.070667427671087],"hidden_sha256":"1f4584bfc27601b564efd337dc7549c9251d998fed30c74feb654aabb6198a03","first_in_process":true,"shard":"shard_07","index":0},{"id":"semif_796d5c0da6eaadae6a3f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2670844080490207e-05,"mean_abs_dp":1.5113896053612144e-05,"oracle_top2_margin":0.9559271118903919,"near_tie":false,"n_options":3,"probs":[0.012825834472526122,0.9715706445007734,0.015603521026700444],"hidden_sha256":"09d365619e229ee51b726f977a061649668eb4ae0421b784a356adcfb4420d17","first_in_process":false,"shard":"shard_07","index":1},{"id":"semif_599b3092b6fd18312d23","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.328700851336965e-05,"mean_abs_dp":3.552467234225337e-05,"oracle_top2_margin":0.9417958472977064,"near_tie":false,"n_options":3,"probs":[0.013100606630511942,0.022513521505244345,0.9643858718642437],"hidden_sha256":"9c19ac1fca62ff7e21cf6052ea7bb5f4492681fcdc040b4d6e9d4cdd34dc1099","first_in_process":false,"shard":"shard_07","index":2},{"id":"semif_960a4bcd27363e6db4b9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004338857378661798,"mean_abs_dp":0.0002892571585774116,"oracle_top2_margin":0.6142107425731358,"near_tie":false,"n_options":3,"probs":[0.14064842068616462,0.10373918945224653,0.755612389861589],"hidden_sha256":"97d33011329cb2fc0dc502957ab9f5aa7f442778580b699be5ffa4ff82ddd49d","first_in_process":false,"shard":"shard_07","index":3},{"id":"semif_c6a411e7f2bbdf377173","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001810260863580604,"mean_abs_dp":0.00012068405757203911,"oracle_top2_margin":0.8540853411132885,"near_tie":false,"n_options":3,"probs":[0.9120814288428417,0.058316463951971986,0.02960210720518633],"hidden_sha256":"2aeb4233b01c4de64023975e188d02d0905a458f8e8ef753367d94b35fdbcaba","first_in_process":false,"shard":"shard_07","index":4},{"id":"semif_e332aa927d0c2ed044aa","name":"answer","variant":"base","source":"semif","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000705867906443447,"mean_abs_dp":0.0004705786042956313,"oracle_top2_margin":0.6547585329407082,"near_tie":false,"n_options":3,"probs":[0.13966294563931833,0.0646455365436371,0.7956915178170446],"hidden_sha256":"b5000b89bf5e0bbc2d9676da51f26b6cf54ea7c411affd3a3a833c8e53ff8ca9","first_in_process":false,"shard":"shard_07","index":5},{"id":"semif_a0128e551d095f3449f8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.0904094112823846e-07,"mean_abs_dp":1.3936062743212518e-07,"oracle_top2_margin":0.9270749550688122,"near_tie":false,"n_options":3,"probs":[0.957129919330281,0.030055085629810133,0.012814995039908926],"hidden_sha256":"6f92ef7036ae4bff9ae7bbff67a8c1a2c52823bdd4cfa60835d0670c0643a045","first_in_process":false,"shard":"shard_07","index":6},{"id":"semif_46d6a0731bd288c370f0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00020158915737694683,"mean_abs_dp":0.0001343927715846451,"oracle_top2_margin":0.21947026315268053,"near_tie":false,"n_options":3,"probs":[0.596686985144864,0.3768375991936467,0.026475415661489383],"hidden_sha256":"1995f96ad3e809934eeec6bb2f89d436e4b7dccf35af19182baade959ec57144","first_in_process":false,"shard":"shard_07","index":7},{"id":"semif_21ca7a8b970e36b8e20f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.0674662756655664e-05,"mean_abs_dp":3.378310850443248e-05,"oracle_top2_margin":0.8425499157252577,"near_tie":false,"n_options":3,"probs":[0.9068322983805063,0.06419150861530143,0.028976193004192276],"hidden_sha256":"d3d1ac8a13d73eda19697e1f55feef9420b27189e829ca208d71db0e7bdf83da","first_in_process":false,"shard":"shard_07","index":8},{"id":"semif_4aa83af95bb69e90bc2d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00036083088922672646,"mean_abs_dp":0.00024055392615118798,"oracle_top2_margin":0.2723478587984883,"near_tie":false,"n_options":3,"probs":[0.20983348717557732,0.2589595046817893,0.5312070081426334],"hidden_sha256":"4868845e95e32a492dafe207ffbe2ad2d3e7e51495637ab9306f4cd472a7f059","first_in_process":false,"shard":"shard_07","index":9},{"id":"semif_cdfa9eed51fafcf09b72","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004598102604009835,"mean_abs_dp":0.0003065401736006279,"oracle_top2_margin":0.35764993020134866,"near_tie":false,"n_options":3,"probs":[0.26826622172652076,0.6266115892541144,0.10512218901936483],"hidden_sha256":"16e791cb8a9b55e1db7042ef35b911d6dea8a63667d151b26eaaa8ce6e6179ff","first_in_process":false,"shard":"shard_07","index":10},{"id":"semif_d51b05bfb9d6d7f09331","name":"answer","variant":"base","source":"semif","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0021829575071747875,"mean_abs_dp":0.0014553050047832379,"oracle_top2_margin":0.13898957633128284,"near_tie":false,"n_options":3,"probs":[0.5467922752725698,0.4121448916872784,0.0410628330401519],"hidden_sha256":"ac449dad4ca58f50588b20eedbd75bca52033ea921918a6f3d72645ee5e917d5","first_in_process":false,"shard":"shard_07","index":11},{"id":"semif_fdd902cc6e65fad7cb4f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011636618990673586,"mean_abs_dp":7.75774599378586e-05,"oracle_top2_margin":0.8581303373394734,"near_tie":false,"n_options":3,"probs":[0.03798713750549632,0.9100041537709611,0.052008708723542574],"hidden_sha256":"ed4106415367f30dc254d3f0ac34e50bef2be636294f40f3bc4202783a97285b","first_in_process":false,"shard":"shard_07","index":12},{"id":"semif_42384af9ae56beb37ff5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004190499574932005,"mean_abs_dp":0.0002793666383288096,"oracle_top2_margin":0.7064067239423519,"near_tie":false,"n_options":3,"probs":[0.11431768898554304,0.06435128909931365,0.8213310219151433],"hidden_sha256":"5e127abcd88a481d89fa3f09e08a43df581775a65f02feb4731230df643c16b9","first_in_process":false,"shard":"shard_07","index":13},{"id":"semif_cdf7f428857e1b841cb3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005926630936483601,"mean_abs_dp":0.00039510872909886974,"oracle_top2_margin":0.3444203525752899,"near_tie":false,"n_options":3,"probs":[0.13694980320977543,0.25874835649154143,0.6043018402986832],"hidden_sha256":"0d7116b15782023225598eaf25f6cdec6cac1b45f50edf701652c18ff2737dd1","first_in_process":false,"shard":"shard_07","index":14},{"id":"semif_774f7f9245f83c796d44","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001100834788642091,"mean_abs_dp":7.33889859094022e-05,"oracle_top2_margin":0.9309078473446127,"near_tie":false,"n_options":3,"probs":[0.025475637221016464,0.01829942293597267,0.9562249398430107],"hidden_sha256":"44b6d68d28ac347fe4a68b97ac811351d065aca27074ebb330fd20d5e659401c","first_in_process":false,"shard":"shard_07","index":15},{"id":"semif_92ec2ffa3dd46ada1335","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010485923522032814,"mean_abs_dp":6.990615681354746e-05,"oracle_top2_margin":0.6321995772739603,"near_tie":false,"n_options":3,"probs":[0.7766072642838301,0.07887994576821436,0.14451278994795547],"hidden_sha256":"334704efedadd56870db0bddd71fef25981287d3aaadae8bfcc81e8a734533bc","first_in_process":false,"shard":"shard_07","index":16},{"id":"semif_6149a17bc154f9c5b4a0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.7343938196386937e-05,"mean_abs_dp":3.156262546422789e-05,"oracle_top2_margin":0.9220314085305296,"near_tie":false,"n_options":3,"probs":[0.9491041655145257,0.026982449033272352,0.023913385452201993],"hidden_sha256":"3a2441b6d9869ff0c68ce881736d32e60a25475eee9bba487041ab2b9d1f422a","first_in_process":false,"shard":"shard_07","index":17},{"id":"semif_be866e1dbec5d42d9423","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":4.021399635952688e-06,"mean_abs_dp":2.6809330905865534e-06,"oracle_top2_margin":0.9256116863283881,"near_tie":false,"n_options":3,"probs":[0.028712359531779117,0.01696046785694828,0.9543271726112725],"hidden_sha256":"9dd9b37a707840f6d669e18b8c600d496fab2d34373619fb12c776219ff73a7a","first_in_process":false,"shard":"shard_07","index":18},{"id":"semif_97d2ed2591fc7638806c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00016021579403660557,"mean_abs_dp":0.00010681052935780182,"oracle_top2_margin":0.47570627138171984,"near_tie":false,"n_options":3,"probs":[0.22367419952941292,0.6993182389081732,0.07700756156241396],"hidden_sha256":"51b9349467d2540bc3c74dbe9c517941fc0bcde13e81e42ff6af2fa99a6c2c9c","first_in_process":false,"shard":"shard_07","index":19},{"id":"semif_4533dd91267b4d815690","name":"answer","variant":"base","source":"semif","type":"choice","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":9.976920725607297e-05,"mean_abs_dp":6.651280483743055e-05,"oracle_top2_margin":0.868350060217824,"near_tie":false,"n_options":3,"probs":[0.04429409231856542,0.042904445321005985,0.9128014623604285],"hidden_sha256":"82bd8d36449eead4efe4da1f279a3a72f7d9e0db89620e2015be87bb0d71754e","first_in_process":false,"shard":"shard_07","index":20},{"id":"semif_d880e65fef5e2993fdd0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001763686163136935,"mean_abs_dp":0.0011757907754246049,"oracle_top2_margin":0.22914503123859198,"near_tie":false,"n_options":3,"probs":[0.49748486247677937,0.2659530080157717,0.2365621295074489],"hidden_sha256":"92111558ce4547c76333bc185cb430c0460083ba110e33913c49936578cae32e","first_in_process":false,"shard":"shard_07","index":21},{"id":"semif_90494c76078ffc3c717a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0016082339109436683,"mean_abs_dp":0.0010721559406291399,"oracle_top2_margin":0.3375654329031006,"near_tie":false,"n_options":3,"probs":[0.23231255967039177,0.5725890369272681,0.19509840340234005],"hidden_sha256":"6620d376626933f316e421418226fd63c1065773ec289d017ba101319d98e531","first_in_process":false,"shard":"shard_07","index":22},{"id":"semif_9ac5cc82c4126c91174a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004365427776890085,"mean_abs_dp":0.00029102851845933436,"oracle_top2_margin":0.721630943498277,"near_tie":false,"n_options":3,"probs":[0.07230979040262354,0.8242675139754155,0.10342269562196095],"hidden_sha256":"be1356682dcf5cee2cea92212caf52f8b3ad5621e4e2f1ded4c8b0f07cc5c943","first_in_process":false,"shard":"shard_07","index":23},{"id":"semif_386e10b1c1ce906473a4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.412009567833497e-05,"mean_abs_dp":2.9413397118914265e-05,"oracle_top2_margin":0.9497366260379649,"near_tie":false,"n_options":3,"probs":[0.014823402612892868,0.967494834219165,0.017681763167941986],"hidden_sha256":"65444cd11614fadd8715620440113188501dd22290648855ed575610b57733c8","first_in_process":false,"shard":"shard_07","index":24},{"id":"semif_4da6aedc1139224eeff2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.254243918720089e-05,"mean_abs_dp":1.50282927914609e-05,"oracle_top2_margin":0.9739471851541249,"near_tie":false,"n_options":3,"probs":[0.009234747475482214,0.9831449988343024,0.007620253690215462],"hidden_sha256":"400c9ce0594fe1aa9e14fc31ddb8aa414f73717953b559208b966567fbf4226a","first_in_process":false,"shard":"shard_07","index":25},{"id":"semif_a22dab8ba073ff6cdb8d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.450420536392333e-05,"mean_abs_dp":4.9669470242617864e-05,"oracle_top2_margin":0.8182519231250659,"near_tie":false,"n_options":3,"probs":[0.05544275968866657,0.8814512144969024,0.06310602581443099],"hidden_sha256":"f3ec55e8955163189ea31e308e7c82bb5588c4b0a2034c0d788f1edf07f8ed2e","first_in_process":false,"shard":"shard_07","index":26},{"id":"semif_42c54fa2ad45743e24d8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.6299611736123616e-05,"mean_abs_dp":2.4199741157427308e-05,"oracle_top2_margin":0.9014139991441142,"near_tie":false,"n_options":3,"probs":[0.03312770969061111,0.9346002487126521,0.03227204159673683],"hidden_sha256":"1ad4e480926b3e52726318a0252e2eb4a8c7d1b8580b3dac993fe0bf136cb142","first_in_process":false,"shard":"shard_07","index":27},{"id":"semif_f3d13cf184f916677def","name":"answer","variant":"base","source":"semif","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.993455643506842e-05,"mean_abs_dp":5.3289704290036356e-05,"oracle_top2_margin":0.9155676995353127,"near_tie":false,"n_options":3,"probs":[0.9497775709925588,0.01615648613469075,0.03406594287275054],"hidden_sha256":"31ad261e32c9dd65e453109db9dac42992e2688fd8e3e93d00d4b61cc50fa698","first_in_process":false,"shard":"shard_07","index":28},{"id":"semif_1105577da4c8dad4609d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.8339464291972085e-05,"mean_abs_dp":1.2226309528031118e-05,"oracle_top2_margin":0.9671984103444269,"near_tie":false,"n_options":3,"probs":[0.011398053158212105,0.009977736240421748,0.9786242106013661],"hidden_sha256":"bb2bc22e73f4931a92f216546cd4816e23c0056b2c77b3e16745db195760cdbb","first_in_process":false,"shard":"shard_07","index":29},{"id":"semif_0ebd299739302d8b057b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004643256593127032,"mean_abs_dp":0.00030955043954182065,"oracle_top2_margin":0.10794600842693236,"near_tie":false,"n_options":3,"probs":[0.43782917448904096,0.23303612578940439,0.3291346997215546],"hidden_sha256":"c140cfa4de35b189a0ed38a4686b80ccc8b717cff870831a236dd382f282b7d3","first_in_process":false,"shard":"shard_07","index":30},{"id":"semif_eafc22c8c40df3932a8e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.821014805393546e-05,"mean_abs_dp":5.8806765369274115e-05,"oracle_top2_margin":0.8458232850367531,"near_tie":false,"n_options":3,"probs":[0.046951507611546685,0.05368215525140148,0.8993663371370518],"hidden_sha256":"45d06d4c9f420c1750180353f8bfd1213ffb3da91771cd4990de5e0d314ea054","first_in_process":false,"shard":"shard_07","index":31},{"id":"semif_0aa1447680e4f25862bb","name":"answer","variant":"base","source":"semif","type":"choice","tokens":103,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006786785623947578,"mean_abs_dp":0.00045245237492984775,"oracle_top2_margin":0.08837413898905816,"near_tie":false,"n_options":3,"probs":[0.34097097306960067,0.4287571450774988,0.23027188185290057],"hidden_sha256":"a5e36f6f7794b55621d94988a385581450b81b6efb09de5f1c9d88d851bbe3fb","first_in_process":false,"shard":"shard_07","index":32},{"id":"semif_980e76145c925747220c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001003326755586731,"mean_abs_dp":0.0006688845037244687,"oracle_top2_margin":0.030265359764135746,"near_tie":false,"n_options":3,"probs":[0.38311082122703793,0.3522396992531187,0.26464947951984336],"hidden_sha256":"fcc2f2ec5ba959c4c8193a46f9f287a63daaa635c7d1ce6e83fb64ec4b7d931e","first_in_process":false,"shard":"shard_07","index":33},{"id":"semif_4aee37a6a155747766aa","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008871177860195201,"mean_abs_dp":0.0005914118573462912,"oracle_top2_margin":0.09844443446324236,"near_tie":false,"n_options":3,"probs":[0.3481319816035348,0.4482367967664131,0.20363122163005207],"hidden_sha256":"0a8051c9ec9c13e495e0ae0d0c87c89e1fb52a046014a7ac0d6c2a99ce3376c5","first_in_process":false,"shard":"shard_07","index":34},{"id":"semif_6db17cc22bd656d78558","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00029175160570005243,"mean_abs_dp":0.0001945010704666831,"oracle_top2_margin":0.773589262631881,"near_tie":false,"n_options":3,"probs":[0.08479507185475607,0.056371837836352474,0.8588330903088914],"hidden_sha256":"3dc1b8190e4dc75fc942d06be4fa1a1db6ecf6eccaec78f46dc9b40a90b39be5","first_in_process":false,"shard":"shard_07","index":35},{"id":"semif_ef3e0f8e657bf59f8a29","name":"answer","variant":"base","source":"semif","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006685662852505825,"mean_abs_dp":0.0004457108568337124,"oracle_top2_margin":0.30690845818680684,"near_tie":false,"n_options":3,"probs":[0.5938092468147607,0.2858045402792887,0.12038621290595068],"hidden_sha256":"b7daf36d8bc17d03034adb6f68784869762dc22284a77cd5d7da568d24c50934","first_in_process":false,"shard":"shard_07","index":36},{"id":"semif_f756e1cb04e11766341e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":103,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000259413599736219,"mean_abs_dp":0.00017294239982409976,"oracle_top2_margin":0.6114943871073327,"near_tie":false,"n_options":3,"probs":[0.09949215884886234,0.1446067306795685,0.755901110471569],"hidden_sha256":"62416225007fead4643dff581b5a68c23ecd6bbee9204a9690c3fb449af5b749","first_in_process":false,"shard":"shard_07","index":37},{"id":"semif_d167e51bb1f499099645","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0019575044555841803,"mean_abs_dp":0.0013050029703894443,"oracle_top2_margin":0.23000797836991627,"near_tie":false,"n_options":3,"probs":[0.2655598675878728,0.23624955610702877,0.49819057630509833],"hidden_sha256":"d38124c4c24cb2cfc5ebae51487cfb9fd917c7493639c1f8b18091d665aac44f","first_in_process":false,"shard":"shard_07","index":38},{"id":"semif_674ca4e9346216255466","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010806760211490296,"mean_abs_dp":7.204506807659272e-05,"oracle_top2_margin":0.7924928211657615,"near_tie":false,"n_options":3,"probs":[0.8647135948645508,0.07238960142159578,0.06289680371385348],"hidden_sha256":"01882f57107d2a1ad645500ca21e6b10bc2e42c7887716c8210096b306248a58","first_in_process":false,"shard":"shard_07","index":39},{"id":"semif_1df851a202994a07f123","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.025826042427112e-05,"mean_abs_dp":1.3505506949534896e-05,"oracle_top2_margin":0.9421137806752105,"near_tie":false,"n_options":3,"probs":[0.9646200358877469,0.022476301801941292,0.012903662310311866],"hidden_sha256":"1e9d8ef3b868c3e0fa087eb1ecbd0af73f7bc73e53490c5a8d34ffeb7df7532e","first_in_process":true,"shard":"shard_08","index":0},{"id":"semif_b5e6788d95b8395d2888","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":3.25966059403493e-05,"mean_abs_dp":2.1731070626886232e-05,"oracle_top2_margin":0.9589997989900602,"near_tie":false,"n_options":3,"probs":[0.011348203805160547,0.014804590831495804,0.9738472053633437],"hidden_sha256":"2b0955469d35a74ce809262bb0e6650fc88a61b3b48087858e730f0f05b4a2dc","first_in_process":false,"shard":"shard_08","index":1},{"id":"semif_32f36cd0fbbfb3604586","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.236593011290427e-05,"mean_abs_dp":5.491062007523598e-05,"oracle_top2_margin":0.8345006167187339,"near_tie":false,"n_options":3,"probs":[0.9023692088820642,0.0677265489031765,0.029904242214759347],"hidden_sha256":"4dd9a7d0790c0b7f99c8a699aa0b2a7f1f639be76d6d17b99f23374e45552925","first_in_process":false,"shard":"shard_08","index":2},{"id":"semif_843185706adff0c60f8b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003709251351706744,"mean_abs_dp":0.0002472834234471093,"oracle_top2_margin":0.5507627206585595,"near_tie":false,"n_options":3,"probs":[0.7674932163519846,0.01651716378965365,0.21598961985836165],"hidden_sha256":"57c8f0c7cc8065a03b1c2ecb35449d79e939bda4261531a1d0e723314e1ff246","first_in_process":false,"shard":"shard_08","index":3},{"id":"semif_b531dd603bf080dab820","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005898181615258125,"mean_abs_dp":0.0003932121076838854,"oracle_top2_margin":0.701489376714385,"near_tie":false,"n_options":3,"probs":[0.8394042083775493,0.023789738374842023,0.13680605324760867],"hidden_sha256":"1afb84a12e80e7567c1f3b39b79c6aad33625c2627a90807af7c15b5cb9c78fe","first_in_process":false,"shard":"shard_08","index":4},{"id":"semif_be2dd67bc733c1213bef","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004012518330768211,"mean_abs_dp":0.0002675012220511863,"oracle_top2_margin":0.41848908396759443,"near_tie":false,"n_options":3,"probs":[0.13092710912169847,0.6435730753100151,0.22549981556828644],"hidden_sha256":"580792b6652ced8d8573ea799e1caedd13f7b7f58ee875d297fa9944b6fcdffe","first_in_process":false,"shard":"shard_08","index":5},{"id":"semif_b8e2750307df9761e716","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.151451145580913e-05,"mean_abs_dp":6.100967430383575e-05,"oracle_top2_margin":0.6699031354481804,"near_tie":false,"n_options":3,"probs":[0.8004600827967948,0.06912509117923807,0.13041482602396703],"hidden_sha256":"1e0bdc753cb5fe9faef6fa24a76d790421ce9812732734d75d309aaed43c4694","first_in_process":false,"shard":"shard_08","index":6},{"id":"semif_5d31b4bf2d0fc193f53b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001718780613911841,"mean_abs_dp":0.00011458537426080791,"oracle_top2_margin":0.2891301652654162,"near_tie":false,"n_options":3,"probs":[0.08480483602286902,0.31292157260043224,0.6022735913766988],"hidden_sha256":"c03d272606a3605287ff1b9a74d50107431082fe2ff6d103508b5093713c117e","first_in_process":false,"shard":"shard_08","index":7},{"id":"semif_a9230edfc6ec9e524af0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":1.1784926140092494e-05,"mean_abs_dp":7.856617426670506e-06,"oracle_top2_margin":0.9425808935221376,"near_tie":false,"n_options":3,"probs":[0.017466360785894936,0.01998672692918398,0.9625469122849212],"hidden_sha256":"ab8720288b5f0ff507abf9672410437a5001b26827f55f0eef6966d4d4c0425c","first_in_process":false,"shard":"shard_08","index":8},{"id":"semif_c100e84585149a04be49","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.5088353011932093e-05,"mean_abs_dp":1.0058902007910783e-05,"oracle_top2_margin":0.9447431163686603,"near_tie":false,"n_options":3,"probs":[0.9634640426399043,0.017785534145388977,0.018750423214706848],"hidden_sha256":"7bafda099be15ecd00b352d5eb6544b944c41f245c48cebb0c3fb073ba62d03e","first_in_process":false,"shard":"shard_08","index":9},{"id":"semif_579beed184ffc1104aa9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001495657917557347,"mean_abs_dp":9.97105278371634e-05,"oracle_top2_margin":0.8268440695720631,"near_tie":false,"n_options":3,"probs":[0.8867333780371982,0.05973543132789252,0.053531190634909256],"hidden_sha256":"f36e43115f837ab949bb040c902526d5227bb6857df9fe77b4e01e2ba540d5b5","first_in_process":false,"shard":"shard_08","index":10},{"id":"semif_c4e9bd463b6fde5be743","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.4959017301401376e-06,"mean_abs_dp":1.663934486837576e-06,"oracle_top2_margin":0.9288057413460095,"near_tie":false,"n_options":3,"probs":[0.02428133552427072,0.9530888173491308,0.022629847126598393],"hidden_sha256":"6e567811cd516e0b9dd30fa3c75a55d2682270b044212563ef81539c65ff70b5","first_in_process":false,"shard":"shard_08","index":11},{"id":"semif_b863998d4be2721a5dfd","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006216863938195427,"mean_abs_dp":0.0004144575958796963,"oracle_top2_margin":0.4760259113962742,"near_tie":false,"n_options":3,"probs":[0.0203151064845429,0.25122751738151133,0.7284573761339458],"hidden_sha256":"6f78bba3d61071ee294fad0a220dc203f5592fb2811b0c07d75ce92776f99900","first_in_process":false,"shard":"shard_08","index":12},{"id":"semif_37cebc3792e98049439f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00043956983977433417,"mean_abs_dp":0.0002930465598495792,"oracle_top2_margin":0.2339733215851162,"near_tie":false,"n_options":3,"probs":[0.3508921002576725,0.584095374019354,0.0650125257229736],"hidden_sha256":"54843350b4b67331641ccbf2d43e3f5739f0d1425a5b074c8e77fc1606fd3002","first_in_process":false,"shard":"shard_08","index":13},{"id":"semif_64d63f86e328aff1481c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.143897774657514e-05,"mean_abs_dp":4.095931849773931e-05,"oracle_top2_margin":0.9554370711800846,"near_tie":false,"n_options":3,"probs":[0.015487462362412985,0.013692638857247263,0.9708198987803398],"hidden_sha256":"e13ffb1cb70fa57ed6405b3994648c499d04120cefe5a705bd2d809276b2c9bf","first_in_process":false,"shard":"shard_08","index":14},{"id":"semif_6d64a8459f64eac7f4cc","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002834382529210977,"mean_abs_dp":0.00018895883528069826,"oracle_top2_margin":0.8876652124031529,"near_tie":false,"n_options":3,"probs":[0.9315653844778352,0.04439939010374484,0.02403522541841992],"hidden_sha256":"a461ceec0477ae595786de0112e5033dc6e1c44d52c9305682cc7ecfcd073577","first_in_process":false,"shard":"shard_08","index":15},{"id":"semif_2708193212d8a4a523c7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00042699429129133826,"mean_abs_dp":0.0002846628608608783,"oracle_top2_margin":0.77678402470049,"near_tie":false,"n_options":3,"probs":[0.033300278837830835,0.8713370562239717,0.09536266493819742],"hidden_sha256":"0bbb4bb6d5459c208c98d20d095c8f9fdd8ac58251df52388af4b37ebc8c2683","first_in_process":false,"shard":"shard_08","index":16},{"id":"semif_5a37d1dfcad20810a76a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.836806992289613e-05,"mean_abs_dp":5.224537994862402e-05,"oracle_top2_margin":0.7956891997585444,"near_tie":false,"n_options":3,"probs":[0.019158348341024634,0.8881880570395166,0.09265359461945875],"hidden_sha256":"558826e586d9d55bdb0d444acf97a4240871b5da8fac7dd79de7e3d2b2fdd360","first_in_process":false,"shard":"shard_08","index":17},{"id":"semif_acacd8b46acd25cb954b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.1439375631609904e-05,"mean_abs_dp":2.762625042102354e-05,"oracle_top2_margin":0.7741791224279037,"near_tie":false,"n_options":3,"probs":[0.09966377749163043,0.8738756268545359,0.02646059565383375],"hidden_sha256":"d1aefa13119bf3d0202908de75e4de059e9a2c4d4c0f48ba1443482f0f21a378","first_in_process":false,"shard":"shard_08","index":18},{"id":"semif_4089c199b8b796ec3ed8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.694708354126888e-05,"mean_abs_dp":4.4631389027503333e-05,"oracle_top2_margin":0.31988840025581594,"near_tie":false,"n_options":3,"probs":[0.607418284005418,0.10512365282273405,0.28745806317184797],"hidden_sha256":"b489a5458512c99424912cc968138d84b0b2b910e32baa1fb0bf39c263498885","first_in_process":false,"shard":"shard_08","index":19},{"id":"semif_02f2b95a5c5a4777a45c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.10305669200584e-05,"mean_abs_dp":6.0687044613375735e-05,"oracle_top2_margin":0.9451561864677233,"near_tie":false,"n_options":3,"probs":[0.016149603071865117,0.9644240597943853,0.0194263371337496],"hidden_sha256":"fe5996abea16454041e8ffa50aac93b931c0c6ce687a6b159219f255a00c21b8","first_in_process":false,"shard":"shard_08","index":20},{"id":"semif_ebea4681e3dcc7d92ca2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.323117298150268e-05,"mean_abs_dp":1.5487448654362874e-05,"oracle_top2_margin":0.6209814841559299,"near_tie":false,"n_options":3,"probs":[0.06293667561614946,0.7790264438372769,0.15803688054657364],"hidden_sha256":"5799ca8b97957599a19e48f8ce68d0d888ca2ebf2a814e64dd4f27c55c3e071d","first_in_process":false,"shard":"shard_08","index":21},{"id":"semif_b8c5b9b285cbdddc9f8d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00024175549628369097,"mean_abs_dp":0.00016117033085581364,"oracle_top2_margin":0.6299678225843708,"near_tie":false,"n_options":3,"probs":[0.017578913899399296,0.17645923289149984,0.8059618532091009],"hidden_sha256":"878c2a0f1660dd9fa86e30c02f01202856c84a0643c9b93bcd15b3ad1fa3b08c","first_in_process":false,"shard":"shard_08","index":22},{"id":"semif_7ed75d854d99b3a81916","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.693951420886027e-05,"mean_abs_dp":6.462634280592998e-05,"oracle_top2_margin":0.894601011388454,"near_tie":false,"n_options":3,"probs":[0.9314843906849187,0.03145430190762186,0.037061307407459436],"hidden_sha256":"cd642595812c47d61f36d9bd03335c76a97a69122770d22d451ef003e18dae18","first_in_process":false,"shard":"shard_08","index":23},{"id":"own_ticket_01","name":"team","variant":"base","source":"own","type":"choice","tokens":161,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.5207058134912863e-05,"mean_abs_dp":1.260352906746749e-05,"oracle_top2_margin":0.978724391038446,"near_tie":false,"n_options":4,"probs":[0.9848403647340807,0.004378834743562797,0.006078295929749095,0.004702504592607514],"hidden_sha256":"1491b1397e4e5522d348990363fba549108897f80a02b403a0f44786a29c382b","first_in_process":false,"shard":"shard_08","index":24},{"id":"own_ticket_01","name":"deadline","variant":"base","source":"own","type":"noul","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.095233235667521e-06,"mean_abs_dp":9.095233235662317e-06,"oracle_top2_margin":0.9854124568314981,"near_tie":false,"n_options":2,"probs":[0.9927153236489847,0.007284676351015295],"hidden_sha256":"f6f82308f2bca1728263ad8ced529ac861d18d7e7cb325b7f29879c7bf052086","first_in_process":false,"shard":"shard_08","index":25},{"id":"own_ticket_01","name":"mood","variant":"base","source":"own","type":"score","tokens":138,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.3534502055989606e-05,"mean_abs_dp":9.023001370609863e-06,"oracle_top2_margin":0.9818281952989258,"near_tie":false,"n_options":3,"probs":[0.989618104027715,0.007811462673011589,0.0025704332992732963],"hidden_sha256":"fda96a317e548ba17e4cb412323201b6c0d3e574b38246b1bd37fa44e3422cb9","first_in_process":false,"shard":"shard_08","index":26},{"id":"own_incident_02","name":"severity","variant":"base","source":"own","type":"score","tokens":201,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.500823441779602e-05,"mean_abs_dp":2.725845159827576e-05,"oracle_top2_margin":0.9326626788228828,"near_tie":false,"n_options":5,"probs":[0.007160239002149977,0.013794711150876996,0.953526760497784,0.020958128021037116,0.004560161328152097],"hidden_sha256":"ef64bfe808932e2199935a1fd4f66fe3f969aeb771f36d439634cf9b28f6e6a2","first_in_process":false,"shard":"shard_08","index":27},{"id":"own_incident_02","name":"cause","variant":"base","source":"own","type":"choice","tokens":193,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.711173392079608e-05,"mean_abs_dp":3.884469356835295e-05,"oracle_top2_margin":0.9230403231000168,"near_tie":false,"n_options":5,"probs":[0.95163864613064,0.009454782281800071,0.006307428121457532,0.004183343643976968,0.02841579982212535],"hidden_sha256":"6185e1bd290ff3b766193a9be8e45ad6a8a9052b8ded2d3fd6f262da1a18b494","first_in_process":false,"shard":"shard_08","index":28},{"id":"own_incident_02","name":"repeat","variant":"base","source":"own","type":"noul","tokens":155,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.139517233712418e-06,"mean_abs_dp":2.139517233693336e-06,"oracle_top2_margin":0.9695785051190727,"near_tie":false,"n_options":2,"probs":[0.9847913920767701,0.015208607923229964],"hidden_sha256":"014af37fc081165d27528c48a4725a84c6ccd8ed0c1714cedbbefb0214afe687","first_in_process":false,"shard":"shard_08","index":29},{"id":"own_email_03","name":"slot","variant":"base","source":"own","type":"choice","tokens":221,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00020687239233785348,"mean_abs_dp":0.00013791492822527499,"oracle_top2_margin":0.8965342472316058,"near_tie":false,"n_options":3,"probs":[0.03893610993231835,0.9358228315305697,0.02524105853711188],"hidden_sha256":"21c61e3eddcacd79ee0b6ad5d4fd96c296380c501aec52971494933e5bbb6719","first_in_process":false,"shard":"shard_08","index":30},{"id":"own_email_03","name":"rules_out_thursday","variant":"base","source":"own","type":"noul","tokens":203,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.4736049956008976e-05,"mean_abs_dp":2.473604995600724e-05,"oracle_top2_margin":0.9643501049842991,"near_tie":false,"n_options":2,"probs":[0.9821503164421935,0.017849683557806473],"hidden_sha256":"351e38412b4fea7ded4eec5dbfbe816a7d3570ce3f057be54ddb247a256915f3","first_in_process":false,"shard":"shard_08","index":31},{"id":"own_meeting_04","name":"script_owner","variant":"base","source":"own","type":"choice","tokens":169,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.267164374351708e-06,"mean_abs_dp":4.178109582856614e-06,"oracle_top2_margin":0.9810108457037208,"near_tie":false,"n_options":3,"probs":[0.006587921056937504,0.9876106422209313,0.005801436722131299],"hidden_sha256":"5267114518efc0b4a544f9ec082eff69238d83045171ea606d87a83bcbeccfff","first_in_process":false,"shard":"shard_08","index":32},{"id":"own_meeting_04","name":"tablet_decision","variant":"base","source":"own","type":"noul","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.5391681186915918e-05,"mean_abs_dp":1.5391681186893366e-05,"oracle_top2_margin":0.9660577759154095,"near_tie":false,"n_options":2,"probs":[0.9830134962765178,0.01698650372348207],"hidden_sha256":"853cac4767a10583cfbb42178fe72e4c9e9ec93d49b3c18a95b25b8fa412a44e","first_in_process":false,"shard":"shard_08","index":33},{"id":"own_meeting_04","name":"rota","variant":"base","source":"own","type":"choice","tokens":173,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.619158354335351e-05,"mean_abs_dp":3.0794389028913904e-05,"oracle_top2_margin":0.9651874400399899,"near_tie":false,"n_options":3,"probs":[0.006924753878009915,0.979170916680044,0.013904329441946117],"hidden_sha256":"e6c540bb76b59792eaaab29cd1e5631522c15d6c2948339118badea6db17bd11","first_in_process":false,"shard":"shard_08","index":34},{"id":"own_review_05","name":"stars","variant":"base","source":"own","type":"score","tokens":157,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000562218551487581,"mean_abs_dp":0.00022613282908933997,"oracle_top2_margin":0.33424306724969377,"near_tie":false,"n_options":5,"probs":[0.2783144218410621,0.6114659514782835,0.09701425203939568,0.010800758212308543,0.002404616428950138],"hidden_sha256":"68d487aefc1abae752fdacf812eac4a2891050ac9351c09b58ed9356937679f4","first_in_process":false,"shard":"shard_08","index":35},{"id":"own_review_05","name":"main_issue","variant":"base","source":"own","type":"choice","tokens":187,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.6951942702169736e-06,"mean_abs_dp":1.6162086070411987e-06,"oracle_top2_margin":0.9661892165317619,"near_tie":false,"n_options":10,"probs":[0.004332016681070756,0.9731312658332426,0.0030250154699195788,0.0018292328736896327,0.0028126284612041814,0.003185710610326654,0.0016634453104585475,0.0015088303204942638,0.0069487207762996585,0.00156313366329418],"hidden_sha256":"f5371b15e1aae9223effee3313a3d7a8cf7a94fcdd7f75ac9662015c84d644c5","first_in_process":false,"shard":"shard_08","index":36},{"id":"own_review_05","name":"would_rebuy","variant":"base","source":"own","type":"noul","tokens":150,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.802731906641597e-06,"mean_abs_dp":1.8027319065843511e-06,"oracle_top2_margin":0.9770772896738245,"near_tie":false,"n_options":2,"probs":[0.011463157894994351,0.9885368421050056],"hidden_sha256":"e58d8a43a6eb4f575eaf6da96cb491488e67b61c6e5747ba96dc5fdaa7129721","first_in_process":false,"shard":"shard_08","index":37},{"id":"own_order_06","name":"free_shipping","variant":"base","source":"own","type":"noul","tokens":248,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001244910334135263,"mean_abs_dp":0.00012449103341351242,"oracle_top2_margin":0.9421814505831572,"near_tie":false,"n_options":2,"probs":[0.028784783675007867,0.9712152163249921],"hidden_sha256":"58783a3dd3c9fb1a34cacccf9f22fd1d8434e7487bdb29f4ee777a8b2ad05535","first_in_process":false,"shard":"shard_08","index":38},{"id":"own_order_06","name":"method","variant":"base","source":"own","type":"choice","tokens":238,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.9302848283149245e-06,"mean_abs_dp":3.953523218870111e-06,"oracle_top2_margin":0.9874800471851845,"near_tie":false,"n_options":3,"probs":[0.9919564472797927,0.004468738809137947,0.0035748139110692947],"hidden_sha256":"3bd039d1ff3f4f67f308e4eb36548ff9600f189d2fb0b64728791a0b21b452c4","first_in_process":false,"shard":"shard_08","index":39},{"id":"own_invoice_07","name":"status","variant":"base","source":"own","type":"choice","tokens":200,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.243106286327958e-05,"mean_abs_dp":2.466994756017177e-05,"oracle_top2_margin":0.9534832020098472,"near_tie":false,"n_options":4,"probs":[0.009622362003750174,0.9698125231230219,0.016244664576391312,0.004320450296836586],"hidden_sha256":"f73f6430d6a82735500f1d2917a335ae4571c5e680d52561f48b685a3469d957","first_in_process":true,"shard":"shard_09","index":0},{"id":"own_invoice_07","name":"overdue","variant":"base","source":"own","type":"noul","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011999244374494289,"mean_abs_dp":0.00011999244374492901,"oracle_top2_margin":0.873984931273492,"near_tie":false,"n_options":2,"probs":[0.9371124580804909,0.06288754191950906],"hidden_sha256":"9d2168bf4b4287ba5d18555598a3a30704c640b8a19bf67722304232cf59d824","first_in_process":false,"shard":"shard_09","index":1},{"id":"own_invoice_07","name":"days_late","variant":"base","source":"own","type":"score","tokens":212,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0003244773815623825,"mean_abs_dp":0.00016223869078120035,"oracle_top2_margin":0.6100105675778147,"near_tie":false,"n_options":4,"probs":[0.014222776683660052,0.020949470924567646,0.17714562975430323,0.7876821226374691],"hidden_sha256":"bc9917cf5e1a0ca40e513b72c2e135c96fc6f1db422a9610f1a78e4c293dfc31","first_in_process":false,"shard":"shard_09","index":2},{"id":"own_sensor_08","name":"out_of_range","variant":"base","source":"own","type":"choice","tokens":210,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003907602658752285,"mean_abs_dp":0.00019538013293761078,"oracle_top2_margin":0.47726427438804064,"near_tie":false,"n_options":4,"probs":[0.1522756478378651,0.6299392972285162,0.11656758797912307,0.10121746695449567],"hidden_sha256":"2e5242ae90f1e3eac11b5ad0bd14332d7184cb596c3c841f2fa385ead5924f1c","first_in_process":false,"shard":"shard_09","index":3},{"id":"own_sensor_08","name":"alert","variant":"base","source":"own","type":"score","tokens":227,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010741154429050326,"mean_abs_dp":7.160769619363183e-05,"oracle_top2_margin":0.8319948315413686,"near_tie":false,"n_options":3,"probs":[0.8882622896172171,0.055633981251777716,0.05610372913100516],"hidden_sha256":"3c5741be41fdbe329a9df7a6b780e88ed4a944eee8227e701c76f1b5bc7d782a","first_in_process":false,"shard":"shard_09","index":4},{"id":"own_sensor_08","name":"all_ok","variant":"base","source":"own","type":"noul","tokens":213,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.2491032291026567e-07,"mean_abs_dp":3.24910322889449e-07,"oracle_top2_margin":0.8775311762487958,"near_tie":false,"n_options":2,"probs":[0.938765263214075,0.061234736785924934],"hidden_sha256":"7544b94b78c4c1a9ffcca879eda67f7e5d55eb3ea5e0e9635784cfd0aa0154e8","first_in_process":false,"shard":"shard_09","index":5},{"id":"own_fiveq_09","name":"signed","variant":"base","source":"own","type":"noul","tokens":143,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2719096103740455e-05,"mean_abs_dp":2.2719096103724842e-05,"oracle_top2_margin":0.9586839893129757,"near_tie":false,"n_options":2,"probs":[0.02068072443961588,0.9793192755603841],"hidden_sha256":"db2ee8fa73bfaf672e537e2f3a699f39bfb5ce76f4b90c152eb9b8bdf521d804","first_in_process":false,"shard":"shard_09","index":6},{"id":"own_fiveq_09","name":"where","variant":"base","source":"own","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.4389353850119413e-05,"mean_abs_dp":8.851616827211528e-06,"oracle_top2_margin":0.9790333070679461,"near_tie":false,"n_options":4,"probs":[0.005826830207387911,0.9848490618012883,0.004429932668508631,0.004894175322815152],"hidden_sha256":"c9a1d09fc3d30195359837ea37eda19912d2c21684bd4b602adf75ad744763ed","first_in_process":false,"shard":"shard_09","index":7},{"id":"own_fiveq_09","name":"damaged","variant":"base","source":"own","type":"noul","tokens":144,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.0839692966400795e-05,"mean_abs_dp":3.083969296632273e-05,"oracle_top2_margin":0.9486429121749174,"near_tie":false,"n_options":2,"probs":[0.9743522957804249,0.025647704219574966],"hidden_sha256":"f135d67df04d40ee30be6c6a37cda53afe17a30d3751b383b88a069731c3c746","first_in_process":false,"shard":"shard_09","index":8},{"id":"own_fiveq_09","name":"request_followed","variant":"base","source":"own","type":"score","tokens":146,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007200608570492872,"mean_abs_dp":0.00048004057136621,"oracle_top2_margin":0.26459297103932206,"near_tie":false,"n_options":3,"probs":[0.2637223291691452,0.2067413444173766,0.5295363264134783],"hidden_sha256":"affc1bb8abb6cb54352682fce575aacfd79a9ab210baab3b2aa6bcd0755cb49d","first_in_process":false,"shard":"shard_09","index":9},{"id":"own_fiveq_09","name":"photo","variant":"base","source":"own","type":"choice","tokens":139,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.870827501055828e-06,"mean_abs_dp":2.8708275010107254e-06,"oracle_top2_margin":0.9725187194604101,"near_tie":false,"n_options":2,"probs":[0.9862622305577061,0.013737769442293995],"hidden_sha256":"cabaee884814520db137a9d41f38326b3a0056a53ed4efcc614c37470edea31a","first_in_process":false,"shard":"shard_09","index":10},{"id":"own_long_log_10","name":"first_failure","variant":"base","source":"own","type":"choice","tokens":1652,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":5.174403173652253e-05,"mean_abs_dp":2.7166496985850098e-05,"oracle_top2_margin":0.8236539990551434,"near_tie":false,"n_options":6,"probs":[0.008931394252895148,0.016574220384516506,0.053919033488517974,0.026799167274918163,0.8776727901725683,0.01610339442658397],"hidden_sha256":"59c7199703501dfaa280e5956d2dff0f59b5fb327280df51d462e97d167f26c0","first_in_process":false,"shard":"shard_09","index":11,"hidden":{"min_pos_cos":0.999987723779683,"min_pos_cos_index":681,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_log_10","name":"impact","variant":"base","source":"own","type":"score","tokens":1643,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00015562702215987656,"mean_abs_dp":8.136432870661098e-05,"oracle_top2_margin":0.7135562371831476,"near_tie":false,"n_options":4,"probs":[0.09323801420762551,0.0593858685263817,0.8070633570045054,0.04031276026148732],"hidden_sha256":"5dbaec61f681bfad8bdde4e50bc0b7dff8dd6b7c10abc9d7c7ef2d0ffe71bad6","first_in_process":false,"shard":"shard_09","index":12,"hidden":{"min_pos_cos":0.999987723779683,"min_pos_cos_index":681,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_log_10","name":"resolved","variant":"base","source":"own","type":"noul","tokens":1641,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00016753878705139114,"mean_abs_dp":0.00016753878705130787,"oracle_top2_margin":0.8030306656886631,"near_tie":false,"n_options":2,"probs":[0.9013477940572803,0.09865220594271976],"hidden_sha256":"1964d7084b0434467f042ba45cd21c1a101fe65f7bd720ffa49b7a86e9be4d5d","first_in_process":false,"shard":"shard_09","index":13,"hidden":{"min_pos_cos":0.999987723779683,"min_pos_cos_index":681,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_contract_11","name":"convenience_notice","variant":"base","source":"own","type":"choice","tokens":1455,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.633371060426387e-06,"mean_abs_dp":4.839276250407721e-06,"oracle_top2_margin":0.9696129049453296,"near_tie":false,"n_options":4,"probs":[0.007679624264282164,0.007297667455873648,0.9773137273947201,0.00770898088512409],"hidden_sha256":"c6ee86e6bfb52170712e7ac43e8bbc6da253f5b618d760dfe9ac96254d9d77f2","first_in_process":false,"shard":"shard_09","index":14},{"id":"own_long_contract_11","name":"liability_cap","variant":"base","source":"own","type":"noul","tokens":1444,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.7543654109573055e-05,"mean_abs_dp":3.754365410943254e-05,"oracle_top2_margin":0.9687246536090245,"near_tie":false,"n_options":2,"probs":[0.9843247831504027,0.01567521684959712],"hidden_sha256":"4e2dffa0af0bc096ce684b204848be9148fd614f9a3575478e2edb45bf4a9f6e","first_in_process":false,"shard":"shard_09","index":15},{"id":"own_long_contract_11","name":"initial_term","variant":"base","source":"own","type":"score","tokens":1451,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00021224010325546216,"mean_abs_dp":0.00010612005162772284,"oracle_top2_margin":0.6988595928596064,"near_tie":false,"n_options":4,"probs":[0.012858615581925855,0.05749057078244641,0.8144183828585003,0.11523243077712736],"hidden_sha256":"868b5496141dcf654abcae42381c069aa206c61dc787a2f15bb467df2fbe2b8c","first_in_process":false,"shard":"shard_09","index":16},{"id":"own_long_minutes_12","name":"irrigation","variant":"base","source":"own","type":"choice","tokens":1375,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":3.177346937544012e-05,"mean_abs_dp":2.1182312916982343e-05,"oracle_top2_margin":0.971579292780383,"near_tie":false,"n_options":3,"probs":[0.0073513456231664815,0.010556949807473277,0.9820917045693602],"hidden_sha256":"e1a77ab4fc41bda100eb51d384a420e8ab7b34f686a4386ac998584709d9eb5c","first_in_process":false,"shard":"shard_09","index":17},{"id":"own_long_minutes_12","name":"chair","variant":"base","source":"own","type":"choice","tokens":1380,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.8856181323311247e-06,"mean_abs_dp":1.7240334492634726e-06,"oracle_top2_margin":0.9811541905303394,"near_tie":false,"n_options":4,"probs":[0.9861920774827602,0.0038479940277635926,0.004916665411689127,0.00504326307778721],"hidden_sha256":"247989767879b5b7e69fd6294603725c1cab1f867ae806e2299e28d8cd7969f5","first_in_process":false,"shard":"shard_09","index":18},{"id":"own_long_minutes_12","name":"lease_unanimous","variant":"base","source":"own","type":"noul","tokens":1356,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000384215382897235,"mean_abs_dp":0.0003842153828971448,"oracle_top2_margin":0.8679600847894678,"near_tie":false,"n_options":2,"probs":[0.06640417298816316,0.9335958270118367],"hidden_sha256":"3213f790e622e5d6bcc0953de396b91b47c527b39b1c01641e4afe88d50c1e10","first_in_process":false,"shard":"shard_09","index":19},{"id":"card_refund","name":"refund","variant":"base","source":"card","type":"noul","tokens":39,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.958319244634083e-06,"mean_abs_dp":8.95831924462194e-06,"oracle_top2_margin":0.9792455172612409,"near_tie":false,"n_options":2,"probs":[0.9896317169498651,0.010368283050134937],"hidden_sha256":"7c54add3a5a4624e24eafa7d42c2a5534af0ed7b22a929d36104aa50385da496","first_in_process":false,"shard":"shard_09","index":20,"hidden":{"min_pos_cos":0.9999942033909348,"min_pos_cos_index":1,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_refund","name":"team","variant":"base","source":"card","type":"choice","tokens":61,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.476310116658497e-06,"mean_abs_dp":6.317540077777246e-06,"oracle_top2_margin":0.9728582283795494,"near_tie":false,"n_options":3,"probs":[0.983499971781699,0.010642661793691754,0.005857366424609297],"hidden_sha256":"66641153891ab206a90c59de77f6697124692eeca5ada345cb1b441961353f3b","first_in_process":false,"shard":"shard_09","index":21,"hidden":{"min_pos_cos":0.9999817602196224,"min_pos_cos_index":51,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_refund","name":"urgency","variant":"base","source":"card","type":"score","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00017047680369253193,"mean_abs_dp":0.00011365120246166945,"oracle_top2_margin":0.22886812438079923,"near_tie":false,"n_options":3,"probs":[0.33873647248991473,0.5672736783645534,0.09398984914553184],"hidden_sha256":"b62a535ddadb78382897b47e6f53e65ca5d1ea962b73710d0c33eff000097470","first_in_process":false,"shard":"shard_09","index":22,"hidden":{"min_pos_cos":0.9999942033909348,"min_pos_cos_index":1,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_ticket_00","name":"team","variant":"base","source":"card","type":"choice","tokens":58,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.35084776786443e-05,"mean_abs_dp":2.9005651785804498e-05,"oracle_top2_margin":0.6698469077623299,"near_tie":false,"n_options":3,"probs":[0.15584711791653344,0.8256193566444818,0.01853352543898477],"hidden_sha256":"7c23a69f974618472a8e285946f770abb81e0c3cdc4e63ae5aaa5c3d4cb3fccf","first_in_process":false,"shard":"shard_09","index":23},{"id":"card_ticket_01","name":"team","variant":"base","source":"card","type":"choice","tokens":56,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.264195639756217e-06,"mean_abs_dp":1.5094637597834124e-06,"oracle_top2_margin":0.9806915560609186,"near_tie":false,"n_options":3,"probs":[0.006504245217802115,0.9871932253587542,0.0063025294234436095],"hidden_sha256":"10e7299e0f6a4869b56a4b20ea88b468363f83256fa7b472f7b60f991ad104b0","first_in_process":false,"shard":"shard_09","index":24},{"id":"long_15k","name":"temp_excursion","variant":"base","source":"own_long","type":"noul","tokens":1491,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.2536233383683175e-05,"mean_abs_dp":4.253623338364501e-05,"oracle_top2_margin":0.877992181442829,"near_tie":false,"n_options":2,"probs":[0.9389535544880309,0.06104644551196914],"hidden_sha256":"d9630ae624280c3cbde1b9491c78c062a6ad5360b127875f950a9c632b073275","first_in_process":false,"shard":"shard_09","index":25},{"id":"long_15k","name":"first_unit","variant":"base","source":"own_long","type":"choice","tokens":1515,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":6.318036450114056e-05,"mean_abs_dp":3.159018225057332e-05,"oracle_top2_margin":0.946464034757811,"near_tie":false,"n_options":4,"probs":[0.004980772208895604,0.004194790100449831,0.9687034722433655,0.02212096544728895],"hidden_sha256":"5e3c2ab2c2ff49fbcdfab681ec7b89a49e687be9a6490102a8718bf8d5094cc9","first_in_process":false,"shard":"shard_09","index":26},{"id":"long_15k","name":"severity","variant":"base","source":"own_long","type":"score","tokens":1526,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00012774087967681869,"mean_abs_dp":8.516058645117545e-05,"oracle_top2_margin":0.13607007226849527,"near_tie":false,"n_options":3,"probs":[0.11093340299042247,0.3763714642809789,0.5126951327285987],"hidden_sha256":"629ccef5e7d5322ab6885801be19d5bce26a7db12a1bdda37bd2b203a78f6580","first_in_process":false,"shard":"shard_09","index":27},{"id":"long_15k","name":"door_open","variant":"base","source":"own_long","type":"noul","tokens":1522,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018693803730990277,"mean_abs_dp":0.00018693803730987502,"oracle_top2_margin":0.8455142706260308,"near_tie":false,"n_options":2,"probs":[0.9229440733503252,0.07705592664967469],"hidden_sha256":"11c1f7dec0cd002cfb7e458f123220bbfdb508ea4c76e03c292a6367526a85fc","first_in_process":false,"shard":"shard_09","index":28},{"id":"long_34k","name":"refunded_twice","variant":"base","source":"own_long","type":"choice","tokens":3470,"calls":55,"padded_tokens":3520,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":4.785600890755415e-05,"mean_abs_dp":2.3928004453786615e-05,"oracle_top2_margin":0.9172464903835365,"near_tie":false,"n_options":4,"probs":[0.006621913887676964,0.028253888246782658,0.945413427633248,0.019710770232292448],"hidden_sha256":"ec66b2e94a068617afd62ba28fbddf51acb4e979c8770c6080467a68b2fda01c","first_in_process":false,"shard":"shard_09","index":29,"hidden":{"min_pos_cos":0.9999677870841723,"min_pos_cos_index":2007,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"big_refunds_approved","variant":"base","source":"own_long","type":"noul","tokens":3451,"calls":54,"padded_tokens":3456,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00027222058813392014,"mean_abs_dp":0.00027222058813386463,"oracle_top2_margin":0.27068175772038333,"near_tie":false,"n_options":2,"probs":[0.3649313417279422,0.6350686582720578],"hidden_sha256":"3d8e9db4edfef505bce58c1e0baa7ac0502862c115fa63fa4504d2ffa6b86aff","first_in_process":false,"shard":"shard_09","index":30,"hidden":{"min_pos_cos":0.9999677870841723,"min_pos_cos_index":2007,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"cancellations","variant":"base","source":"own_long","type":"score","tokens":3461,"calls":55,"padded_tokens":3520,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008319770606668442,"mean_abs_dp":0.00042313746546571857,"oracle_top2_margin":0.695961559659646,"near_tie":false,"n_options":4,"probs":[0.020315699478691867,0.13058775453680746,0.8281904557505189,0.02090609023398184],"hidden_sha256":"90b86dfe031801106c820a8779c003b19ce4e1377afce12539103c77c5c40d30","first_in_process":false,"shard":"shard_09","index":31,"hidden":{"min_pos_cos":0.9999677870841723,"min_pos_cos_index":2007,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"shipped_unpaid","variant":"base","source":"own_long","type":"noul","tokens":3446,"calls":54,"padded_tokens":3456,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00019797471749860573,"mean_abs_dp":0.00019797471749857798,"oracle_top2_margin":0.037437100559284564,"near_tie":false,"n_options":2,"probs":[0.5185205755621437,0.4814794244378563],"hidden_sha256":"90e33c55b474e2c38b2dba58fc1ca4dfdf155d656015f2e3ef8f2b90850e3b7a","first_in_process":false,"shard":"shard_09","index":32,"hidden":{"min_pos_cos":0.9999677870841723,"min_pos_cos_index":2007,"max_abs_diff":0.057308197021484375,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}}]},"processes":[{"shard":"shard_00","rows":40,"prompts":41,"load_seconds":7.92180675000418,"load_split_seconds":{"model":1.633823374984786,"main":6.287983375019394},"function_names":["main"],"process_wall_seconds":21.90389912505634,"reset_check":{"run":"tv4_000:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.0722280410118401},"contract_mismatch":[]},{"shard":"shard_01","rows":40,"prompts":41,"load_seconds":0.808044874924235,"load_split_seconds":{"model":0.005818584002554417,"main":0.8022262909216806},"function_names":["main"],"process_wall_seconds":196.81934929196723,"reset_check":{"run":"tv4_040:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.13918675004970282},"contract_mismatch":[]},{"shard":"shard_02","rows":40,"prompts":41,"load_seconds":7.421910290955566,"load_split_seconds":{"model":0.005712707992643118,"main":7.416197582962923},"function_names":["main"],"process_wall_seconds":17.17833233298734,"reset_check":{"run":"tv4x_qnli_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07183637493290007},"contract_mismatch":[]},{"shard":"shard_03","rows":40,"prompts":41,"load_seconds":0.8421641669701785,"load_split_seconds":{"model":0.006263624993152916,"main":0.8359005419770256},"function_names":["main"],"process_wall_seconds":5.735815916908905,"reset_check":{"run":"tv4x_sciq_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1070027919486165},"contract_mismatch":[]},{"shard":"shard_04","rows":40,"prompts":41,"load_seconds":0.8637314999941736,"load_split_seconds":{"model":0.006189333973452449,"main":0.8575421660207212},"function_names":["main"],"process_wall_seconds":176.07164004200604,"reset_check":{"run":"tv4x_composition_holdout_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.10674312501214445},"contract_mismatch":[]},{"shard":"shard_05","rows":40,"prompts":41,"load_seconds":9.540093832998537,"load_split_seconds":{"model":0.006993207964114845,"main":9.533100625034422},"function_names":["main"],"process_wall_seconds":24.523148291977122,"reset_check":{"run":"semif_a3f18f3a63d45345942b:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07203433394897729},"contract_mismatch":[]},{"shard":"shard_06","rows":40,"prompts":41,"load_seconds":0.9153531660558656,"load_split_seconds":{"model":0.008120957994833589,"main":0.907232208061032},"function_names":["main"],"process_wall_seconds":5.031953583005816,"reset_check":{"run":"semif_a0a39ace7e5337d038a7:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07274637511000037},"contract_mismatch":[]},{"shard":"shard_07","rows":40,"prompts":41,"load_seconds":10.434901666012593,"load_split_seconds":{"model":0.00675208296161145,"main":10.428149583050981},"function_names":["main"],"process_wall_seconds":37.402526666992344,"reset_check":{"run":"semif_0aa24a6701ac9c56c4a0:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.09212387504521757},"contract_mismatch":[]},{"shard":"shard_08","rows":40,"prompts":41,"load_seconds":1.0462988750077784,"load_split_seconds":{"model":0.007868750020861626,"main":1.0384301249869168},"function_names":["main"],"process_wall_seconds":7.256330166012049,"reset_check":{"run":"semif_1df851a202994a07f123:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.09444887505378574},"contract_mismatch":[]},{"shard":"shard_09","rows":33,"prompts":34,"load_seconds":9.200583625002764,"load_split_seconds":{"model":0.008682584040798247,"main":9.191901040961966},"function_names":["main"],"process_wall_seconds":37.78052149992436,"reset_check":{"run":"own_invoice_07:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1418224589433521},"contract_mismatch":[]}]},"iphone_int8mlp":{"export_record":{"mode":"int8mlp","name":"d1_3b_decode_int8mlp_pf64","toy":false,"toy_seed":null,"hf_id":"LiquidAI/d1-3B","revision":"da1fe36a861f24690f27f622dca1d8688503d113","prefill_chunk":64,"fp16_layers":null,"quant_block":32,"argv":["int8mlp","--prefill-chunk","64","--aot","--record","$ZOO_WORK_ROOT/_d1_3b/results/r6a_export_int8mlp_pf64.json"],"started":"2026-10-08T16:37:33+09:00","script_sha256":"16218c7f43fe02a1622122240e3342f8b7381e21edc7d8d83e7a5561283bfcf1","module_sha256":"bacdc9f3db17cc336ba25899a403a1e99075dd3ccd52b1a99166c8e8aa29cf89","option_rows_sha256":"78d2c1132c5f58ef38a647871a17f8e5b1105a87dc8d84630cf5300e60a27ad9","export":{"bundle":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_int8mlp_pf64","name":"d1_3b_decode_int8mlp_pf64","aimodel":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_int8mlp_pf64/d1_3b_decode_int8mlp_pf64.aimodel","toy":false,"load":{"load_report":{"checkpoint_files":["model.safetensors"],"checkpoint_keys":707,"checkpoint_keys_under_prefix":266,"module_tensors":266,"unread_checkpoint_keys_under_prefix":[],"module_tensors_not_in_checkpoint":[],"checkpoint_has_lm_head_key":false,"tie_embedding":true,"module_has_lm_head_weight":false,"meta_params":[],"fp32_params":["model.layers.13.self_attn.k_proj.weight","model.layers.13.self_attn.out_proj.weight","model.layers.13.self_attn.q_proj.weight","model.layers.13.self_attn.v_proj.weight","model.layers.17.self_attn.k_proj.weight","model.layers.17.self_attn.out_proj.weight","model.layers.17.self_attn.q_proj.weight","model.layers.17.self_attn.v_proj.weight","model.layers.2.self_attn.k_proj.weight","model.layers.2.self_attn.out_proj.weight","model.layers.2.self_attn.q_proj.weight","model.layers.2.self_attn.v_proj.weight","model.layers.21.self_attn.k_proj.weight","model.layers.21.self_attn.out_proj.weight","model.layers.21.self_attn.q_proj.weight","model.layers.21.self_attn.v_proj.weight","model.layers.24.self_attn.k_proj.weight","model.layers.24.self_attn.out_proj.weight","model.layers.24.self_attn.q_proj.weight","model.layers.24.self_attn.v_proj.weight","model.layers.27.self_attn.k_proj.weight","model.layers.27.self_attn.out_proj.weight","model.layers.27.self_attn.q_proj.weight","model.layers.27.self_attn.v_proj.weight","model.layers.5.self_attn.k_proj.weight","model.layers.5.self_attn.out_proj.weight","model.layers.5.self_attn.q_proj.weight","model.layers.5.self_attn.v_proj.weight","model.layers.9.self_attn.k_proj.weight","model.layers.9.self_attn.out_proj.weight","model.layers.9.self_attn.q_proj.weight","model.layers.9.self_attn.v_proj.weight"],"dtype":"float16"},"weights":{"file":"<snapshot>/model.safetensors","bytes":6247065504,"sha256":"50e03317847caf6df9a9aee27ed40f20554a86a21e60d1d47ba41a422b546c0c"},"text_config":{"num_hidden_layers":30,"num_full_layers":8,"num_conv_layers":22,"hidden_size":2048,"ff_dim":10752,"vocab_size":128000,"rope_theta":1000000.0,"tie_embedding":true,"head_dim":64,"num_attention_heads":32,"num_key_value_heads":8,"conv_L_cache":3,"conv_state_width":2,"norm_eps":1e-05,"full_attention_layers":[2,5,9,13,17,21,24,27]}},"trace_kv_len":2048,"max_ctx":4096,"functions":["main"],"query_len":64,"n_image_tokens":2816,"spec":{"input_names":["input_ids","position_ids","image_embeds"],"output_names":["hidden"],"state_names":["keyCache","valueCache","convState"],"reference_inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,128],"int32"],"image_embeds":[[2816,2048],"float16"],"k_cache":[[8,1,8,2048,64],"float16"],"v_cache":[[8,1,8,2048,64],"float16"],"conv_state":[[22,1,2048,2],"float16"]},"dynamic":{"input_ids":null,"position_ids":{"1":"seq_pos"},"image_embeds":null,"k_cache":{"3":"k_seq"},"v_cache":{"3":"v_seq"},"conv_state":null}},"fp32_params":["model.layers.13.self_attn.k_proj.weight","model.layers.13.self_attn.out_proj.weight","model.layers.13.self_attn.q_proj.weight","model.layers.13.self_attn.v_proj.weight","model.layers.17.self_attn.k_proj.weight","model.layers.17.self_attn.out_proj.weight","model.layers.17.self_attn.q_proj.weight","model.layers.17.self_attn.v_proj.weight","model.layers.2.self_attn.k_proj.weight","model.layers.2.self_attn.out_proj.weight","model.layers.2.self_attn.q_proj.weight","model.layers.2.self_attn.v_proj.weight","model.layers.21.self_attn.k_proj.weight","model.layers.21.self_attn.out_proj.weight","model.layers.21.self_attn.q_proj.weight","model.layers.21.self_attn.v_proj.weight","model.layers.24.self_attn.k_proj.weight","model.layers.24.self_attn.out_proj.weight","model.layers.24.self_attn.q_proj.weight","model.layers.24.self_attn.v_proj.weight","model.layers.27.self_attn.k_proj.weight","model.layers.27.self_attn.out_proj.weight","model.layers.27.self_attn.q_proj.weight","model.layers.27.self_attn.v_proj.weight","model.layers.5.self_attn.k_proj.weight","model.layers.5.self_attn.out_proj.weight","model.layers.5.self_attn.q_proj.weight","model.layers.5.self_attn.v_proj.weight","model.layers.9.self_attn.k_proj.weight","model.layers.9.self_attn.out_proj.weight","model.layers.9.self_attn.q_proj.weight","model.layers.9.self_attn.v_proj.weight"],"quantization":{"mode":"int8mlp","dtype":"int8","block":32,"kinds":["mlp"],"leaves":["feed_forward.gate_proj","feed_forward.up_proj","feed_forward.down_proj"],"linear":{"dtype":"int8","qscheme":"symmetric_with_clipping","granularity":{"type":"per_block","block_size":32,"axis":1}},"excluded_names":[".*\\.conv\\.in_proj$",".*\\.conv\\.out_proj$",".*lm_head$",".*self_attn\\.(q_proj|k_proj|v_proj|out_proj)$"],"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"fp16_layers":[],"quantized_linear_modules":90,"quantized_set_equals_intended":true,"quantized_params":1981808640,"fp32_attention_projections":["model.layers.2.self_attn.q_proj","model.layers.2.self_attn.k_proj","model.layers.2.self_attn.v_proj","model.layers.2.self_attn.out_proj","model.layers.5.self_attn.q_proj","model.layers.5.self_attn.k_proj","model.layers.5.self_attn.v_proj","model.layers.5.self_attn.out_proj","model.layers.9.self_attn.q_proj","model.layers.9.self_attn.k_proj","model.layers.9.self_attn.v_proj","model.layers.9.self_attn.out_proj","model.layers.13.self_attn.q_proj","model.layers.13.self_attn.k_proj","model.layers.13.self_attn.v_proj","model.layers.13.self_attn.out_proj","model.layers.17.self_attn.q_proj","model.layers.17.self_attn.k_proj","model.layers.17.self_attn.v_proj","model.layers.17.self_attn.out_proj","model.layers.21.self_attn.q_proj","model.layers.21.self_attn.k_proj","model.layers.21.self_attn.v_proj","model.layers.21.self_attn.out_proj","model.layers.24.self_attn.q_proj","model.layers.24.self_attn.k_proj","model.layers.24.self_attn.v_proj","model.layers.24.self_attn.out_proj","model.layers.27.self_attn.q_proj","model.layers.27.self_attn.k_proj","model.layers.27.self_attn.v_proj","model.layers.27.self_attn.out_proj"],"fp16_linear_modules":["model.layers.0.conv.in_proj","model.layers.0.conv.out_proj","model.layers.1.conv.in_proj","model.layers.1.conv.out_proj","model.layers.10.conv.in_proj","model.layers.10.conv.out_proj","model.layers.11.conv.in_proj","model.layers.11.conv.out_proj","model.layers.12.conv.in_proj","model.layers.12.conv.out_proj","model.layers.14.conv.in_proj","model.layers.14.conv.out_proj","model.layers.15.conv.in_proj","model.layers.15.conv.out_proj","model.layers.16.conv.in_proj","model.layers.16.conv.out_proj","model.layers.18.conv.in_proj","model.layers.18.conv.out_proj","model.layers.19.conv.in_proj","model.layers.19.conv.out_proj","model.layers.20.conv.in_proj","model.layers.20.conv.out_proj","model.layers.22.conv.in_proj","model.layers.22.conv.out_proj","model.layers.23.conv.in_proj","model.layers.23.conv.out_proj","model.layers.25.conv.in_proj","model.layers.25.conv.out_proj","model.layers.26.conv.in_proj","model.layers.26.conv.out_proj","model.layers.28.conv.in_proj","model.layers.28.conv.out_proj","model.layers.29.conv.in_proj","model.layers.29.conv.out_proj","model.layers.3.conv.in_proj","model.layers.3.conv.out_proj","model.layers.4.conv.in_proj","model.layers.4.conv.out_proj","model.layers.6.conv.in_proj","model.layers.6.conv.out_proj","model.layers.7.conv.in_proj","model.layers.7.conv.out_proj","model.layers.8.conv.in_proj","model.layers.8.conv.out_proj"],"fp16_linear_params":369098752,"seconds":6.7027142910519615},"ops":{"graph_type":"GraphOp","ops":1774,"by_op":{"coreai.constant":468,"coreai.transpose":242,"coreai.decomposable.broadcasting_batch_matmul":166,"coreai.reshape":143,"coreai.decomposable.broadcasting_mul":123,"coreai.slice":110,"coreai.blockwise_shift_scale":90,"coreai.invoke":85,"coreai.decomposable.broadcasting_add":76,"coreai.concat":59,"coreai.cast":52,"coreai.silu":30,"coreai.split":22,"coreai.conv2d":22,"coreai.read_handle":19,"coreai.write_handle":17,"coreai.slice_update":16,"coreai.shrink_dims":16,"coreai.decomposable.broadcasting_sub":2,"coreai.decomposable.broadcasting_maximum":2,"coreai.decomposable.broadcasting_minimum":2,"coreai.graph":1,"coreai.create_token":1,"coreai.get_shape":1,"coreai.decomposable.broadcasting_greater":1,"coreai.not":1,"coreai.gather_nd":1,"coreai.broadcast_in_dims":1,"coreai.gather_along_axis":1,"coreai.decomposable.broadcasting_where":1,"coreai.cos":1,"coreai.sin":1,"coreai.output":1},"printed_chars":227356,"method":"regex over str(prog.get_graph('main'))"},"seconds":{"load":9.729028083034791,"quantize":7.1878075420390815,"export":27.411592790973373,"optimize":0.049511666991747916,"save":3.900682457955554,"total":53.88817191601265},"du_aimodel":"3.5G","du_bundle":"3.5G","aimodel_files":{"main.hash":32,"main.mlirb":3704646757,"metadata.json":105},"main_mlirb":{"bytes":3704646757,"sha256":"2ab109907108870b8e7e191162d98bff7107b0ff338e3ead4854f490d4a6850b"},"main_hash_hex":"2ab109907108870b8e7e191162d98bff7107b0ff338e3ead4854f490d4a6850b","head":{"file":"option_rows.safetensors","bytes":17491440,"sha256":"e43cfaee023b4c1c2bab48b873aa2195bc6e5f86fe9f30049a89a1e0ad948522","n":2134,"hidden":2048,"readback":{"ids_equal":true,"rows_bit_equal":true},"source":{"hf_id":"LiquidAI/d1-3B","revision":"da1fe36a861f24690f27f622dca1d8688503d113","file":"model.safetensors","file_sha256":"50e03317847caf6df9a9aee27ed40f20554a86a21e60d1d47ba41a422b546c0c","key":"model.language_model.embed_tokens.weight","header":{"dtype":"BF16","shape":[128000,2048],"data_offsets":[0,524288000]},"data_start":97728,"conversion":"bf16 -> fp32 (exact)","tokenizer_json_sha256":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","candidates":{"A..Z":"26/26","a..z":"26/26","0..9":"10/10","00..99":"100/100","100..999":"900/900","#0..#199":"0/200","AA..ZZ":"587/676","' A..Z'":"26/26","' a..z'":"26/26","' 0..9'":"0/10","' 00..99'":"0/100","' 100..999'":"0/900","' #0..#199'":"0/200","' AA..ZZ'":"429/676","noul":"5/6"}}},"metadata_sha256":"864a2e13289fe5c6f012930fce4a6220db839d4eb23641d8d32023416f318e74","kind":"decision-backbone","tokenizer_sha256":{"tokenizer.json":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","tokenizer_config.json":"ef6770d12dd9ac58d334e979bf160c5cdfd41f13d205fd5d5516857645feb0ea","chat_template.jinja":"86f4770449a4797c9b4212d110b0cc70fb993c1ce960095bcfb12eea22f61cca"},"tokenizer_verbatim_from_snapshot":true,"license_sha256":"4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c","disk":{"before_load":{"time":"2026-10-08T16:37:34+09:00","free_bytes":275039531008,"free_gib":256.2,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_load":{"time":"2026-10-08T16:37:44+09:00","free_bytes":281572409344,"free_gib":262.2,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"9.9G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090__probe_r6a_ane_decoder":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_export":{"time":"2026-10-08T16:38:19+09:00","free_bytes":274289389568,"free_gib":255.5,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"20G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090__probe_r6a_ane_decoder":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after_save":{"time":"2026-10-08T16:38:23+09:00","free_bytes":270567501824,"free_gib":252.0,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"20G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090__probe_r6a_ane_decoder":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}}}},"aot":{"aimodelc":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc/d1_3b_decode_int8mlp_pf64.h16c.aimodelc","flags":["--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"seconds":130.57343341701198,"coreai_build":"<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","command":["<Metal toolchain v27.1.266.1>/usr/bin/coreai-build","compile","$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_int8mlp_pf64/d1_3b_decode_int8mlp_pf64.aimodel","--output","$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc","--platform","macOS","--preferred-compute","gpu","--architecture","h16c","--expect-frequent-reshapes"],"disk":{"before":{"time":"2026-10-08T16:38:30+09:00","free_bytes":270531616768,"free_gib":252.0,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7829511f4137136d5d242e2953a417e38a7173778551462e6a3ca862c0beac54":"8.1G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"20G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090__probe_r6a_ane_decoder":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"814M","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}},"after":{"time":"2026-10-08T16:40:41+09:00","free_bytes":247456350208,"free_gib":230.5,"coreai_cache_python":{"00aa128a7615bfabe425a6b86db63d6bf44aa3a440feb07d736cdd02ccbd70ff":"8.3G","1b7fc25e4a5a1399c7dd680d2c3d066a45ce9d2588688df6ba15a1f80d5acf54":"359M","22c66070c101671c057b44edf3c509191ec4047c5821e2a3d9f4dbb3efb4b794":"728M","29322c2bc4398833956be2ed0fa5556b61d34dc4715dee8c91f597351a2426ae":"727M","2ac10ae42f57fd52d07b4b7d5da081979db3ccf18310bcffc5d9f9c9130fabbf":"727M","2bd7faebac602c004d48666472d82fae89599b234bcffe4fe5f2ce3a4786577d":"5.2G","2d06d1d029e9f06f63b1175dfa7b92fe0091af4d5c6c6c1d12ef42b22ec683cc":"728M","31f1c7b96ed8ab381924b036cccd270c9a1df579a437d1349ab49490fd166036":"337M","34bad5d9071c4cfbc6da28ce7e6999f153fe959e2be85affa5d6d0e214d8250e":"216M","3b8d9a1914e1655ab335488aaa549b72c86c1c6d241ab997275af56488a3ed17":"9.9G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f":"7.8G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int4lin_r5a":"2.3G","3d2d9a151481ac39ce332afc1dc6c71dd92aafa93ead6ad106d7b60d28ebb87f__int8lin_b16_r5c":"8.0G","46e9e26c41c31447a60a2b198444f5320d51b790f256df0a61862391abcc602a":"727M","4756e8aef823910eb1a89665f2e1384ea103fc052c4ac153496bd18586381940":"727M","51503e89966b6096ab0c660b071ce81e097838d3447b6b1d1b017413ffcf2a0f":"217M","518573e81e837ce0a08229c22564e172766e2fcb7372e26b9a7a355c5e723824":"729M","51dd02877559cf7c0c5ab587103538ad365234e731f8f6bd0b8fd0830e6ff95d":"727M","52a4b0ccfaaa0dc08e783b665025a25579c631aeef10ab5a8e19f665d9a470ea":"180M","6b790a6e77c1f70b94b7f61c69afdb714118cc9963faea36f5c430abff63f6f3":"216M","6de6801431d7fa9ed049895d8e8eb1dd1a81107416cc065293a37019120feee7":"727M","71e6edbb6a7a5e21470b6780777cf7e7c1dd3cd4754c452c4dadba1ee2c83a9e":"9.9G","7829511f4137136d5d242e2953a417e38a7173778551462e6a3ca862c0beac54":"8.1G","7aade6cce742fd6792d86867a50b885ba11b0f8d8d55d939d198ec6dd40b6e8f":"215M","8532205b3f61d1b923af4f4936b6c50824f5dcb7765083164a39c9c203815453":"9.5G","8a171a6a249e5a9cf4e718c0b8c4626b48c4aa2552c11be50cbf1bff360f9457__probe_r6a_ane_tower":"814M","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116":"8.1G","8cc9c2617b3111f381420ce75db5f0047a3b58b770e5e3bedc6f78752c399116__int8mlp_b16_r5c":"8.3G","8d7089bfd527256f083226646e43725fc471aed91bd291afa341f0929eaa0c4d":"180M","91ff89a2876742257fe0d83c3d12aa56c7e69fd8e6db2d2a79a3cc5f19d6d1da":"1.4G","946d898f56a096da8929690af83d54bd435ede35289c234540b1fa0eec48399c":"20G","9e9017d6d5f731cabad83ce70adeac3c67fa7dc3d5edd5cc60cdd5ed1e3dfd8e":"727M","b0b674d190d22843a409857f390921d4798fade09b4381479342ec876019a9d3":"180M","b53bde419aad912fcddac94817786476e02a656378586a0f94c976f53f27a9b3":"728M","bd61a18199613eb219e7358764cd4e9f7a88a37b0f2e561564f485895e64c174":"217M","c0d09e5432b3df33e3c93ac381b5d1ef86d075203ab430341339882411828d31":"6.8G","cba72d97a69cc72c4f7873aba61fbb71f1bd583991f3dff87575a2eece112090__probe_r6a_ane_decoder":"9.9G","da2e4194f6a420079514a208bc4a13d57de16d38ae41119f70051cc6b8b94b71":"429M","db7c61fa41fa5e82932081ff166bac7208b62c4924477d6da314e5a0a6743ebe":"215M","dc3d237e16bb952758882afa4d7ceb55eb9820ee85667f65c4e1fda37c48b0e4":"215M","e30f084483af5f24f44a7d21831a21e1c826c183af2f96039ae3b8a4cc9f670f":"727M","ee0391645db2ca8c5eb3058ad264714960436638284f19259d38a77b17256feb":"1.6G","fdc4473f95da8a07e6d57da9f8b9dce024b5563678551edb2e54abd8ae2710aa":"216M","ff9730448b18a10647f4e37726abb9d499cc8c8875cf280575fd29a827d69aa8":"727M"}}},"returncode":0,"stdout_tail":["Compiling h16c: 1 of 1"],"stderr_tail":[],"main_hash_hex":"6b6e4de872f9b4ee33a610b8f5837085a7f968240f0c542f823a267354380933","runtime_cache_entry":"~/Library/Caches/coreai-cache/26A428/python/6b6e4de872f9b4ee33a610b8f5837085a7f968240f0c542f823a267354380933","stats":{"storageTypes":[{"count":1982496768,"name":"Int8"},{"count":694123524,"name":"Float16"},{"name":"Float32","count":83886114},{"name":"Int32","count":322},{"name":"UInt32","count":85},{"name":"UInt64","count":1}],"computeTypes":["Bool","Float16","Float32","Int32","Int8","UInt32","UInt64"],"operationDistribution":[{"name":"constant","count":2024},{"count":290,"name":"slice"},{"name":"reshape","count":274},{"name":"transpose","count":243},{"name":"concat","count":234},{"name":"broadcast_to","count":187},{"name":"batch_matmul","count":168},{"name":"broadcasting_batch_matmul","count":168},{"count":167,"name":"mul"},{"count":164,"name":"broadcasting_mul"},{"name":"broadcast_in_dims","count":159},{"name":"add","count":114},{"name":"broadcasting_add","count":110},{"count":90,"name":"blockwise_shift_scale"},{"count":85,"name":"invoke"},{"count":67,"name":"cast"},{"name":"broadcasting_divide","count":34},{"name":"divide","count":34},{"count":33,"name":"broadcasting_sub"},{"name":"sub","count":33},{"count":31,"name":"exp"},{"name":"silu","count":30},{"count":22,"name":"conv2d"},{"name":"split","count":22},{"count":19,"name":"read_handle"},{"name":"write_handle","count":17},{"count":16,"name":"get_shape"},{"count":16,"name":"shrink_dims"},{"count":16,"name":"slice_update"},{"count":9,"name":"broadcast_shapes"},{"count":7,"name":"reduce"},{"name":"gather_nd","count":4},{"name":"reduce_sum","count":4},{"name":"gather_along_axis","count":3},{"name":"not","count":3},{"count":3,"name":"reduce_mean"},{"count":3,"name":"reduce_product"},{"name":"rsqrt","count":3},{"name":"broadcasting_greater","count":2},{"name":"broadcasting_maximum","count":2},{"name":"broadcasting_minimum","count":2},{"name":"expand_dims","count":2},{"name":"greater","count":2},{"count":2,"name":"maximum"},{"count":2,"name":"minimum"},{"name":"range","count":2},{"name":"broadcasting_where","count":1},{"count":1,"name":"cos"},{"name":"create_token","count":1},{"count":1,"name":"sin"},{"name":"softmax","count":1},{"count":1,"name":"where"}]},"du_aimodelc":"8.1G","digest":{"bytes":8742889020,"tree_sha256":"607cbb07d9fec258a8ed5cde24670a1062db2a150a8ff93f5630ccc0ef8062e3","files":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":"be6e15ec3d2f39e9666d0ab2d53f125fc73ec1034b706ada703ff605d89a1421","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":"2679e4ea555bd048b721fcb381e2d6ce9c46df1679bca65ff308841aafa62857","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":"e5a85308d1554c94ca5d0720474ceb29aae27baec192ed6d14d757cfa5d5962e","main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":"24aef98cf182446aadb8f2e736207a176397d6a2dc906a0a7ac9e646f447eb64","main-h16c.mlirb":"6b6e4de872f9b4ee33a610b8f5837085a7f968240f0c542f823a267354380933","main.hash":"669cd47019f0b0d842b629ad6b38e79595488faea28e7ae13c18bf59ee5bc916","metadata.json":"d494fa3fa670edf2ba2049dac120ee6d7d35c8a128752677536719122170b14d","stats.json":"13b1461974bf9c26fd8c0873e9ef87aa747bda29403fd8f803240d02d42a2590"},"file_bytes":{"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/manifest.plist":70132,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/original_model_0.mpsgraph":358474,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/resources.bin":8741854580,"main-h16c-delegates/MPSGraph/mpsExecutable.mpsgraphpackage/specialized_model_1.mpsgraph":600266,"main-h16c.mlirb":3318,"main.hash":32,"metadata.json":193,"stats.json":2025}}},"finished":"2026-10-08T16:40:46+09:00"},"readout":{"schema":"d1-decoder-readout-gate/1","gate":"the decoder alone on the Mac GPU (the oracle's row ids), the fp16 slot hidden read out through host.py","bundle":{"bundle":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles/d1_3b_decode_int8mlp_pf64","name":"d1_3b_decode_int8mlp_pf64","kind":"decision-backbone","compression":{"scheme":"int8mlp","linear":"int8 per-block-32 symmetric_with_clipping (weight only)","quantized":"every MLP linear (feed_forward.gate_proj / up_proj / down_proj)","excluded":"fp32: the four attention projections of every full-attention layer; fp16: the conv-mixer projections (conv.in_proj / out_proj), the embedding table (the tied table of head/option_rows), the conv1d, every RMSNorm, SDPA","excluded_names":[".*\\.conv\\.in_proj$",".*\\.conv\\.out_proj$",".*lm_head$",".*self_attn\\.(q_proj|k_proj|v_proj|out_proj)$"],"excluded_types":["coreai_models.primitives.macos.rms_norm.RMSNorm","coreai_models.primitives.macos.sdpa.SDPA","torch.nn.modules.conv.Conv1d","torch.nn.modules.sparse.Embedding"],"quantized_linear_modules":90,"fp16_layers":[],"fp16_linear_modules":["model.layers.0.conv.in_proj","model.layers.0.conv.out_proj","model.layers.1.conv.in_proj","model.layers.1.conv.out_proj","model.layers.10.conv.in_proj","model.layers.10.conv.out_proj","model.layers.11.conv.in_proj","model.layers.11.conv.out_proj","model.layers.12.conv.in_proj","model.layers.12.conv.out_proj","model.layers.14.conv.in_proj","model.layers.14.conv.out_proj","model.layers.15.conv.in_proj","model.layers.15.conv.out_proj","model.layers.16.conv.in_proj","model.layers.16.conv.out_proj","model.layers.18.conv.in_proj","model.layers.18.conv.out_proj","model.layers.19.conv.in_proj","model.layers.19.conv.out_proj","model.layers.20.conv.in_proj","model.layers.20.conv.out_proj","model.layers.22.conv.in_proj","model.layers.22.conv.out_proj","model.layers.23.conv.in_proj","model.layers.23.conv.out_proj","model.layers.25.conv.in_proj","model.layers.25.conv.out_proj","model.layers.26.conv.in_proj","model.layers.26.conv.out_proj","model.layers.28.conv.in_proj","model.layers.28.conv.out_proj","model.layers.29.conv.in_proj","model.layers.29.conv.out_proj","model.layers.3.conv.in_proj","model.layers.3.conv.out_proj","model.layers.4.conv.in_proj","model.layers.4.conv.out_proj","model.layers.6.conv.in_proj","model.layers.6.conv.out_proj","model.layers.7.conv.in_proj","model.layers.7.conv.out_proj","model.layers.8.conv.in_proj","model.layers.8.conv.out_proj"],"head":"none in the graph (the tied rows of the readout ids are head/option_rows)"},"prefill_chunk":64,"max_ctx":4096,"toy":null,"metadata_sha256":"864a2e13289fe5c6f012930fce4a6220db839d4eb23641d8d32023416f318e74","main_mlirb":{"bytes":3704646757,"sha256":"2ab109907108870b8e7e191162d98bff7107b0ff338e3ead4854f490d4a6850b"},"tokenizer_sha256":{"chat_template.jinja":"86f4770449a4797c9b4212d110b0cc70fb993c1ce960095bcfb12eea22f61cca","tokenizer.json":"8096ecb9f54599d756c8de728a598a340bc1e43c0deb77ddd62456c38349fcee","tokenizer_config.json":"ef6770d12dd9ac58d334e979bf160c5cdfd41f13d205fd5d5516857645feb0ea"},"head_sha256":{"option_rows.json":"41bb7fc3120b09adf9cc11319b4ec566bc6989ff6ebdb30890e269e257ab8bba","option_rows.safetensors":"e43cfaee023b4c1c2bab48b873aa2195bc6e5f86fe9f30049a89a1e0ad948522"},"license_sha256":"4d28ca14dedc0b3d0fcc2b3339f0e79931faa33874f3d24f522183a8fc70068c","aimodelc":{"path":"$ZOO_WORK_ROOT/_d1_3b/exports/bundles_aotc/d1_3b_decode_int8mlp_pf64.h16c.aimodelc","bytes":8742889020,"tree_sha256":"607cbb07d9fec258a8ed5cde24670a1062db2a150a8ff93f5630ccc0ef8062e3"},"aimodelc_main_hash":"6b6e4de872f9b4ee33a610b8f5837085a7f968240f0c542f823a267354380933"},"chunk":64,"subset":"all","bar":{"max_abs_dp":0.02,"mean_of_run_mean_abs_dp":0.002,"near_tie_top2_margin":0.02,"argmax":"every question with an oracle top-2 margin above 0.02; near-ties listed apart","max_abs_dp_applies_to":"every option of every question, near-ties included","mean_of_run_mean_abs_dp_definition":"mean over runs (one run = one row = one question) of the mean |dp| over that question's options","reset":"every process re-runs its first row last: hidden rows bit-equal","finite":"every hidden value of every row finite, no row all zero","rows":"every expected row present"},"environment":{"versions":{"python":"3.11.13","numpy":"2.3.5","coreai-core":"1.0.0b2","coreai-torch":"0.4.1","coreai-models":"0.1.0","torch":"2.9.0","safetensors":"0.7.0","tokenizers":"0.23.0rc0"},"platform":"macOS-27.0-arm64-arm-64bit","macos_build":"26A428","chip":"Apple M4 Max","runtime":"coreai python runtime, AOT h16c GPU .aimodelc, SpecializationOptions.default(), no JIT","readout":"host.py: z = h_slot . E[id] in float64 over the group ids (head/option_rows, fp32), group max, softmax over the options","gpu":"shared with other sessions, _GPU_LOCK read only (the ms are contended reference values)"},"summary":{"runs":393,"expected_runs":393,"missing_runs":[],"questions":393,"argmax_equal":393,"questions_non_near_tie":392,"argmax_equal_non_near_tie":392,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.019660274378349274,"max_abs_dp_non_near_tie":0.019660274378349274,"mean_of_run_mean_abs_dp":0.0012615584218342564,"mean_question_max_abs_dp":0.0018242965018116246,"worst_run":{"id":"semif_33d6e5da58f0fee2490d","name":"answer","max_abs_dp":0.019660274378349274},"finite_all":true,"all_zero_runs":0,"hidden_rows":14,"min_pos_cos":0.9947377666772287,"hidden_max_abs_diff":0.6236745119094849,"tokens":74504,"padded_tokens":88320,"calls":1380,"pad_waste":0.15643115942028984,"ms_per_call_median":34.102458506822586,"ms_per_call_warm_median":34.10402103327215},"by_source":{"card":{"runs":5,"expected_runs":5,"missing_runs":[],"questions":5,"argmax_equal":5,"questions_non_near_tie":5,"argmax_equal_non_near_tie":5,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.004932724532862198,"max_abs_dp_non_near_tie":0.004932724532862198,"mean_of_run_mean_abs_dp":0.0007894183224688543,"mean_question_max_abs_dp":0.0011769364256183508,"worst_run":{"id":"card_ticket_00","name":"team","max_abs_dp":0.004932724532862198},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9997776553912043,"hidden_max_abs_diff":0.16211891174316406,"tokens":268,"padded_tokens":320,"calls":5,"pad_waste":0.1625,"ms_per_call_median":34.11612496711314,"ms_per_call_warm_median":34.11612496711314},"own":{"runs":36,"expected_runs":36,"missing_runs":[],"questions":36,"argmax_equal":36,"questions_non_near_tie":36,"argmax_equal_non_near_tie":36,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.0067648983722385925,"max_abs_dp_non_near_tie":0.0067648983722385925,"mean_of_run_mean_abs_dp":0.0005581196288609179,"mean_question_max_abs_dp":0.0009561333160396449,"worst_run":{"id":"own_sensor_08","name":"out_of_range","max_abs_dp":0.0067648983722385925},"finite_all":true,"all_zero_runs":0,"hidden_rows":3,"min_pos_cos":0.9997178009480513,"hidden_max_abs_diff":0.2262253761291504,"tokens":18249,"padded_tokens":19520,"calls":305,"pad_waste":0.06511270491803278,"ms_per_call_median":33.907833974808455,"ms_per_call_warm_median":33.9081248966977},"own_long":{"runs":8,"expected_runs":8,"missing_runs":[],"questions":8,"argmax_equal":8,"questions_non_near_tie":8,"argmax_equal_non_near_tie":8,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.008582613011315632,"max_abs_dp_non_near_tie":0.008582613011315632,"mean_of_run_mean_abs_dp":0.002069121047617599,"mean_question_max_abs_dp":0.002722812675803201,"worst_run":{"id":"long_34k","name":"shipped_unpaid","max_abs_dp":0.008582613011315632},"finite_all":true,"all_zero_runs":0,"hidden_rows":4,"min_pos_cos":0.9947377666772287,"hidden_max_abs_diff":0.6236745119094849,"tokens":19882,"padded_tokens":20096,"calls":314,"pad_waste":0.01064888535031847,"ms_per_call_median":34.770958474837244,"ms_per_call_warm_median":34.770958474837244},"semif":{"runs":144,"expected_runs":144,"missing_runs":[],"questions":144,"argmax_equal":144,"questions_non_near_tie":144,"argmax_equal_non_near_tie":144,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.019660274378349274,"max_abs_dp_non_near_tie":0.019660274378349274,"mean_of_run_mean_abs_dp":0.0012996018541948458,"mean_question_max_abs_dp":0.0019494027812922707,"worst_run":{"id":"semif_33d6e5da58f0fee2490d","name":"answer","max_abs_dp":0.019660274378349274},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9998259740136523,"hidden_max_abs_diff":0.13752365112304688,"tokens":12177,"padded_tokens":18368,"calls":287,"pad_waste":0.33705357142857145,"ms_per_call_median":33.94204203505069,"ms_per_call_warm_median":33.94204203505069},"tv4":{"runs":60,"expected_runs":60,"missing_runs":[],"questions":60,"argmax_equal":60,"questions_non_near_tie":59,"argmax_equal_non_near_tie":59,"near_tie_questions":1,"argmax_equal_near_tie":1,"max_abs_dp":0.008910118326611927,"max_abs_dp_non_near_tie":0.008910118326611927,"mean_of_run_mean_abs_dp":0.0011248894006540486,"mean_question_max_abs_dp":0.002232712554775843,"worst_run":{"id":"tv4_040","name":"answer","max_abs_dp":0.008910118326611927},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9989956661182413,"hidden_max_abs_diff":0.2218940258026123,"tokens":7969,"padded_tokens":9984,"calls":156,"pad_waste":0.20182291666666666,"ms_per_call_median":34.11418799078092,"ms_per_call_warm_median":34.12383346585557},"tv4s":{"runs":20,"expected_runs":20,"missing_runs":[],"questions":20,"argmax_equal":20,"questions_non_near_tie":20,"argmax_equal_non_near_tie":20,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.008809977689542725,"max_abs_dp_non_near_tie":0.008809977689542725,"mean_of_run_mean_abs_dp":0.0020789860949636213,"mean_question_max_abs_dp":0.0031184791424454265,"worst_run":{"id":"tv4s_00","name":"decision","max_abs_dp":0.008809977689542725},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9996697145184893,"hidden_max_abs_diff":0.16333898901939392,"tokens":2436,"padded_tokens":2560,"calls":40,"pad_waste":0.0484375,"ms_per_call_median":33.98439544253051,"ms_per_call_warm_median":33.98439544253051},"tv4x":{"runs":120,"expected_runs":120,"missing_runs":[],"questions":120,"argmax_equal":120,"questions_non_near_tie":120,"argmax_equal_non_near_tie":120,"near_tie_questions":0,"argmax_equal_near_tie":0,"max_abs_dp":0.015841057238656836,"max_abs_dp_non_near_tie":0.015841057238656836,"mean_of_run_mean_abs_dp":0.0013248695017167625,"mean_question_max_abs_dp":0.001481785047153981,"worst_run":{"id":"tv4x_composition_holdout_12","name":"decision","max_abs_dp":0.015841057238656836},"finite_all":true,"all_zero_runs":0,"hidden_rows":1,"min_pos_cos":0.9997857168488112,"hidden_max_abs_diff":0.13932740688323975,"tokens":13523,"padded_tokens":17472,"calls":273,"pad_waste":0.2260187728937729,"ms_per_call_median":34.12625007331371,"ms_per_call_warm_median":34.12283293437213}},"near_ties":[{"id":"tv4_013","name":"answer","oracle_top2_margin":0.01051943645398079,"argmax_equal":true,"max_abs_dp":0.006472445475024513,"probs":[0.009058762885649663,0.010429055487935343,0.4786150836026119,0.5018970980238031]}],"checks":{"all_runs":true,"a_argmax_non_near_tie":true,"b_max_abs_dp":true,"c_mean_of_run_mean_abs_dp":true,"d_reset_bit_equal_all_processes":true,"e_finite":true,"red_required_arms_red":true,"red_graph_dp_equals_oracle_dp":true,"red_base_bit_equal_gate_runs":true},"result":"PASS","timing":{"contended":true,"S":64,"load_seconds":{"first_process":8.394476332934573,"median":0.6078125624917448,"all":[8.394476332934573,0.6911258340114728,0.6305835000239313,0.6050396249629557,0.9260537920054048,0.590547583065927,0.5705066670197994,0.5826243340270594,0.5562187500763685,0.6105855000205338]},"ms_per_call_warm_runs":{"n":1354,"median":34.10402103327215,"p10":33.69159603025764,"p90":35.15045449603349},"ms_per_call_all_runs":{"n":1380,"median":34.102458506822586,"p10":33.68807982187718,"p90":35.18838755553588},"row_wall_seconds_warm_runs":{"n":383,"median":0.0726246670819819,"p10":0.07163366661407053,"p90":0.13868250022642314}},"seconds":{"gpu_processes":92.68812070798595,"scoring":0.2790575419785455,"total":93.09619062498678},"generated_at":"2026-10-08T07:42:28+00:00","note":"round 6a: chunk width S from the bundle's metadata; --compare-with the same mode's S=16 transcript (the rounding order changes with S); ms contended, not the ladder","load":{"start":{"at":"2026-10-08T16:40:55+09:00","loadavg":"{ 29.00 16.33 13.06 }","swapusage":"total = 7168.00M  used = 5475.69M  free = 1692.31M  (encrypted)"},"end":{"at":"2026-10-08T16:42:28+09:00","loadavg":"{ 13.26 14.23 12.57 }","swapusage":"total = 7168.00M  used = 5475.69M  free = 1692.31M  (encrypted)"}},"contract":{"function":"main","inputs":{"input_ids":[[1,64],"int32"],"position_ids":[[1,-1],"int32"],"image_embeds":[[2816,2048],"float16"]},"outputs":{"hidden":[[1,64,2048],"float16"]},"states":{"keyCache":[[8,1,8,-1,64],"float16"],"valueCache":[[8,1,8,-1,64],"float16"],"convState":[[22,1,2048,2],"float16"]},"dynamic":"-1 = dynamic: position_ids [1, seq] with seq in S..max_context_length-1; the KV sequence axis 2048..max_context_length (a host allocates it at max_context_length)"},"oracle":{"path":"$ZOO_WORK_ROOT/_d1_3b/oracle/records_oracle.json","sha256":"067a4f5658937da0e20e307deaa15a38deb55d0f12b2f9993b609461d0430af1","schema":"d1-oracle/1","toy":null,"hidden_dir":"$ZOO_WORK_ROOT/_d1_3b/oracle/hidden"},"runtime_cache_entry":{"path":"~/Library/Caches/coreai-cache/26A428/python/6b6e4de872f9b4ee33a610b8f5837085a7f968240f0c542f823a267354380933","exists":true,"kib":8538000},"descriptor_same_in_every_process":true,"red_rule":"red = the perturbed rows against their base rows fail the gate's own bar (FACTS §7): (a) an argmax moves on a question whose oracle top-2 margin is above 0.02, or (b) max |dp| > 0.02, or (c) the mean over the arm's rows of the row's mean |dp| > 0.002","script":{"path":"conversion/d1/readout_gate.py","sha256":"713ba1e75f03beca460edb8a70ff5f7590fa534c109a8b9d28d904ba3afa6515"},"runs":[{"id":"tv4_000","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013197150651203873,"mean_abs_dp":7.372729623692474e-05,"oracle_top2_margin":0.9175063266514067,"near_tie":false,"n_options":4,"probs":[0.012317559602245474,0.030557784835101284,0.9478103831813423,0.009314272381311035],"hidden_sha256":"94bccfd2a7bbc3fedc2c7d7a6ac3570dbf25b5a9f617eeba4fb5364bf8e8aafd","first_in_process":true,"shard":"shard_00","index":0,"hidden":{"min_pos_cos":0.9989956661182413,"min_pos_cos_index":96,"max_abs_diff":0.2218940258026123,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4_001","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":5.4412554975735716e-05,"mean_abs_dp":2.7206277487833706e-05,"oracle_top2_margin":0.9880834835402893,"near_tie":false,"n_options":4,"probs":[0.0011469111242165457,0.00304382684147775,0.9919810191023674,0.003828242931938439],"hidden_sha256":"8029f6a9b7a06011fd9a512bbdf101e1972cab492a2182f5ac5db15a5cd51973","first_in_process":false,"shard":"shard_00","index":1},{"id":"tv4_002","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000356064991457683,"mean_abs_dp":0.00018003259052688466,"oracle_top2_margin":0.9589307483192843,"near_tie":false,"n_options":4,"probs":[0.9727798744662309,0.00526216123810256,0.014503901808074881,0.007454062487591588],"hidden_sha256":"9050734a9c82723e9b53a0c259e1d80d8f2a51f10aa4e5a9405fe2422472a3cd","first_in_process":false,"shard":"shard_00","index":2},{"id":"tv4_003","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00036178738097181196,"mean_abs_dp":0.00020398720144506385,"oracle_top2_margin":0.8790862718882508,"near_tie":false,"n_options":4,"probs":[0.9195646832287427,0.040794011699545406,0.03007908187202458,0.009562223199687301],"hidden_sha256":"e46b5e0d0bfbfd9b93e50738e4f15f746d1bf43bf11afd4ad11f8cbad6122c79","first_in_process":false,"shard":"shard_00","index":3},{"id":"tv4_004","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":8.81238882555202e-06,"mean_abs_dp":4.406194412753567e-06,"oracle_top2_margin":0.9788776201055156,"near_tie":false,"n_options":4,"probs":[0.002898554705320874,0.004744195971485956,0.9856221948911377,0.006735054432055521],"hidden_sha256":"aa1b64274348edd02ac22b12bbc76d2f18cc381f96f4ecb62c7ca3f089cfb88c","first_in_process":false,"shard":"shard_00","index":4},{"id":"tv4_005","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011471903997416488,"mean_abs_dp":0.0005735951998708205,"oracle_top2_margin":0.9301866143096604,"near_tie":false,"n_options":4,"probs":[0.9577248930112409,0.029707458642541533,0.008292579973071867,0.004275068373145741],"hidden_sha256":"d4988013b5bc09cb6fdd20abcbb4b56f445445d5ecd5f6e63b9ba26e0b0b2f46","first_in_process":false,"shard":"shard_00","index":5},{"id":"tv4_006","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":249,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004621214757098535,"mean_abs_dp":0.002310607378549246,"oracle_top2_margin":0.5148134998751737,"near_tie":false,"n_options":4,"probs":[0.04278975837450032,0.726044298846848,0.028713219819969174,0.20245272295868266],"hidden_sha256":"646eeff9d5d2a1754235396625189440dea21578d0a935df7ce5ea605788defa","first_in_process":false,"shard":"shard_00","index":6},{"id":"tv4_007","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00029739178909787833,"mean_abs_dp":0.00015847694694448274,"oracle_top2_margin":0.7992202960971314,"near_tie":false,"n_options":4,"probs":[0.02355427695457015,0.08468132396828705,0.8843716786035336,0.007392720473609269],"hidden_sha256":"0e37a8ef38214dddb6f5c6c1d1008a4daddddb40e8db40dfdd3af38f7522f03d","first_in_process":false,"shard":"shard_00","index":7},{"id":"tv4_008","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":113,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006160222856725905,"mean_abs_dp":0.00030801114283627033,"oracle_top2_margin":0.8940057521242946,"near_tie":false,"n_options":4,"probs":[0.04624466246856736,0.9413520315277383,0.0045887441205847744,0.007814561883109675],"hidden_sha256":"5f8ade06266b3da98e3533b9fcd4c5307735ed8368e0be83dc11914f7469cb17","first_in_process":false,"shard":"shard_00","index":8},{"id":"tv4_009","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":339,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00016340729895880646,"mean_abs_dp":8.170364947943532e-05,"oracle_top2_margin":0.9850214404934367,"near_tie":false,"n_options":4,"probs":[0.002104083886741297,0.0038868869284991296,0.9896229336451171,0.004386095539642405],"hidden_sha256":"5b33a2183c7552b26f37eaaf039e03eea016c352e364570e697616add9f27cd1","first_in_process":false,"shard":"shard_00","index":9},{"id":"tv4_010","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020170937252622367,"mean_abs_dp":0.0011025697841856863,"oracle_top2_margin":0.1758312758643772,"near_tie":false,"n_options":4,"probs":[0.3952036819167722,0.21370677075008546,0.17378669439411012,0.21730285293903218],"hidden_sha256":"2f9562f2ede088cf32e404d6b4b92d7b1ef181d2a31efeaa6b541ac8f2cdce2d","first_in_process":false,"shard":"shard_00","index":10},{"id":"tv4_011","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":129,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00813731639189752,"mean_abs_dp":0.0040686581959487895,"oracle_top2_margin":0.0715867449236845,"near_tie":false,"n_options":4,"probs":[0.017019257920871188,0.04210160499271242,0.44240475495910264,0.49847438212731365],"hidden_sha256":"f557f7a68958a93c5960cb60e5b72f83edfb2bf1dbb7c798f606f741e0626176","first_in_process":false,"shard":"shard_00","index":11},{"id":"tv4_012","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0010283956663210692,"mean_abs_dp":0.0005141978331605519,"oracle_top2_margin":0.92417841818824,"near_tie":false,"n_options":4,"probs":[0.025162364469647886,0.004991895377591859,0.02198798021221962,0.9478577599405407],"hidden_sha256":"56389846f819bc83cc5261692fee686465b4308308421d3b6f42bfce12f4e562","first_in_process":false,"shard":"shard_00","index":12},{"id":"tv4_013","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":183,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.006472445475024513,"mean_abs_dp":0.0032362227375122843,"oracle_top2_margin":0.01051943645398079,"near_tie":true,"n_options":4,"probs":[0.009058762885649663,0.010429055487935343,0.4786150836026119,0.5018970980238031],"hidden_sha256":"bf00fdcdbc9bb4e9d08d477f0e47df74eb1ed15aee7e0c5f988eb70851ab4268","first_in_process":false,"shard":"shard_00","index":13},{"id":"tv4_014","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002437625597711615,"mean_abs_dp":0.0001218812798855955,"oracle_top2_margin":0.8982278774496655,"near_tie":false,"n_options":4,"probs":[0.03668781975039574,0.0206073210571306,0.9345361609660363,0.008168698226437222],"hidden_sha256":"a82556940c4ea6d11b893aa8452d3013fc79a963d2101b8cb9474b99fbd80d9f","first_in_process":false,"shard":"shard_00","index":14},{"id":"tv4_015","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":194,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0027188951650204407,"mean_abs_dp":0.0013594475825102169,"oracle_top2_margin":0.7573631279404294,"near_tie":false,"n_options":4,"probs":[0.8570055255202019,0.028306616318285367,0.1047158960285978,0.009971962132914873],"hidden_sha256":"ec0dfd1de98acbc53709699bd96b11c3a376b85cc66b49a77e47377d42acf018","first_in_process":false,"shard":"shard_00","index":15},{"id":"tv4_016","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00044478956467408626,"mean_abs_dp":0.00022239478233705224,"oracle_top2_margin":0.8801963176754999,"near_tie":false,"n_options":4,"probs":[0.010236990537425522,0.010847904805709577,0.04911296757450091,0.9298021370823639],"hidden_sha256":"563d62b9b37566eaccd644e32b64688f0540cac15c4b8cee8ba14eaef12f4890","first_in_process":false,"shard":"shard_00","index":16},{"id":"tv4_017","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00018508678763506925,"mean_abs_dp":0.00010960353634324738,"oracle_top2_margin":0.955853033961402,"near_tie":false,"n_options":4,"probs":[0.0040273202279488855,0.014460854398386723,0.9706701949150486,0.010841630458615706],"hidden_sha256":"50de2080bb23fff889a9df48f6ace330a10199f56f8c9ef3b59b6db005d3b85c","first_in_process":false,"shard":"shard_00","index":17},{"id":"tv4_018","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.4778488189203945e-05,"mean_abs_dp":1.3486677162825866e-05,"oracle_top2_margin":0.9849916555226311,"near_tie":false,"n_options":4,"probs":[0.0013967661926885247,0.004770606129199828,0.9897949697859496,0.004037657892162058],"hidden_sha256":"b6a42b1941507d25d8cd466a0828ff22cc05042bf7823e63dd5332816a323d94","first_in_process":false,"shard":"shard_00","index":18},{"id":"tv4_019","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":150,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.007496887614247466,"mean_abs_dp":0.003748443807123747,"oracle_top2_margin":0.5069606504483266,"near_tie":false,"n_options":4,"probs":[0.12037868787061767,0.6505263470317927,0.14136613565208217,0.08772882944550744],"hidden_sha256":"dcf93bc68883ea0c5d7dabb1be7578153fbeda67db9823d43dac572c0531a7c9","first_in_process":false,"shard":"shard_00","index":19},{"id":"tv4_020","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006034604495710694,"mean_abs_dp":0.0030173022478553564,"oracle_top2_margin":0.43055736666492805,"near_tie":false,"n_options":4,"probs":[0.015090248100480843,0.07282037874478363,0.6761365454043587,0.23595282775037674],"hidden_sha256":"2140f7edcfbba4960225311dd30ffd0507e40a559185edbe1fffeab9ba4323d1","first_in_process":false,"shard":"shard_00","index":20},{"id":"tv4_021","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0022312889317305795,"mean_abs_dp":0.001115644465865255,"oracle_top2_margin":0.26756356697958295,"near_tie":false,"n_options":4,"probs":[0.2179683098990448,0.12111939339574523,0.4885219657376019,0.1723903309676081],"hidden_sha256":"febf5689d2b8fdc0aed36aa5393e13d293436c940c5300cb34750b70982a74e8","first_in_process":false,"shard":"shard_00","index":21},{"id":"tv4_022","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":154,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005412873437141141,"mean_abs_dp":0.0027064367185705308,"oracle_top2_margin":0.10257574535656738,"near_tie":false,"n_options":4,"probs":[0.27186510921036905,0.3806789201076097,0.2859107431594915,0.06154522752252969],"hidden_sha256":"749376951a8a71624344893ea1e3dcc0b9be30e7a69e349e33ced92515dde9e9","first_in_process":false,"shard":"shard_00","index":22},{"id":"tv4_023","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":107,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002504493274332109,"mean_abs_dp":0.0012522466371660303,"oracle_top2_margin":0.4294777143114449,"near_tie":false,"n_options":4,"probs":[0.595148518791952,0.16829167294045347,0.10834171099185916,0.1282180972757354],"hidden_sha256":"bccf6da1fa523353fa54ce19add91bc3ea72d0a0bbc42cb27505e749d63c555f","first_in_process":false,"shard":"shard_00","index":23},{"id":"tv4_024","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0009282006955827482,"mean_abs_dp":0.0004641003477913602,"oracle_top2_margin":0.6636511103865234,"near_tie":false,"n_options":4,"probs":[0.09553981485992462,0.10280522176077696,0.03644130061463943,0.765213662764659],"hidden_sha256":"c3201e8139068b6162b26414cabc6161a19cb687c5e1658e618ba492aee85c95","first_in_process":false,"shard":"shard_00","index":24},{"id":"tv4_025","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":193,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0029656772408613308,"mean_abs_dp":0.0014828386204306758,"oracle_top2_margin":0.04727997159987629,"near_tie":false,"n_options":4,"probs":[0.28373736082024537,0.3440337361444822,0.2924360687564247,0.0797928342788477],"hidden_sha256":"a474876c6a473ccaffbc4d87bc00523f24ad0453b95d178eaae8f3af0c7e4bc0","first_in_process":false,"shard":"shard_00","index":25},{"id":"tv4_026","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":109,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013059998963571573,"mean_abs_dp":6.529999481785082e-05,"oracle_top2_margin":0.9854219242740557,"near_tie":false,"n_options":4,"probs":[0.0018476523725593926,0.004159554057132299,0.9896086602001829,0.004384133370125321],"hidden_sha256":"537ac9db15baf155072ae9ae6d4592d9d10c926a3e374a732558e33f379e9a58","first_in_process":false,"shard":"shard_00","index":26},{"id":"tv4_027","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008086967533298794,"mean_abs_dp":0.0040856533060361026,"oracle_top2_margin":0.02766328769190496,"near_tie":false,"n_options":4,"probs":[0.005746275631064914,0.4174092851902978,0.201038726669371,0.37580571250926637],"hidden_sha256":"dbb73da5592df95cf30c11e8de311c565c31c255837b2266d8d86fe65adc013a","first_in_process":false,"shard":"shard_00","index":27},{"id":"tv4_028","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":316,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0024549725594791294,"mean_abs_dp":0.0012274862797395161,"oracle_top2_margin":0.08139784238225001,"near_tie":false,"n_options":4,"probs":[0.1797329831022597,0.3317404843501684,0.41575472537580166,0.07277180717177036],"hidden_sha256":"5a61873b628a8f6b790072c3a050d7339aeae9fe94c675e1e1f47c1ae09a313b","first_in_process":false,"shard":"shard_00","index":28},{"id":"tv4_029","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0023463868669274524,"mean_abs_dp":0.0011731934334637348,"oracle_top2_margin":0.7765665787029383,"near_tie":false,"n_options":4,"probs":[0.8424862325129933,0.0407213691419297,0.04748591007520981,0.06930648826986721],"hidden_sha256":"9b5ca5b26a701d5686197fb0a399861cb0ec251963744b58db32697d29434cf3","first_in_process":false,"shard":"shard_00","index":29},{"id":"tv4_030","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004864092066539927,"mean_abs_dp":0.00024320460332700154,"oracle_top2_margin":0.9245924496230566,"near_tie":false,"n_options":4,"probs":[0.008537916648263878,0.02497135520592277,0.9503735749254077,0.01611715322040567],"hidden_sha256":"9910c75021a411315a294c83c54acae959184d63564ac8b49668958b7079ee1c","first_in_process":false,"shard":"shard_00","index":30},{"id":"tv4_031","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0017683882222465086,"mean_abs_dp":0.0008951186782997021,"oracle_top2_margin":0.12507160244912263,"near_tie":false,"n_options":4,"probs":[0.5588751052765261,0.430271114047348,0.006914062789385197,0.003939717886740669],"hidden_sha256":"6381fcd82dbdd4cf4acb49f9f0be9815af2e9011f1584cd6686c3723cd0c4303","first_in_process":false,"shard":"shard_00","index":31},{"id":"tv4_032","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.001379794835294712,"mean_abs_dp":0.0006898974176473553,"oracle_top2_margin":0.9326063916622778,"near_tie":false,"n_options":4,"probs":[0.0033573936350935535,0.008440589250473743,0.026510886863198926,0.9616911302512338],"hidden_sha256":"3e742c9a1b38b5fed3049835b08f35f2e4f9a4b3a2437aeec7e6034b464e2017","first_in_process":false,"shard":"shard_00","index":32},{"id":"tv4_033","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":311,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.000140398776308448,"mean_abs_dp":7.019938815420535e-05,"oracle_top2_margin":0.9615159028252174,"near_tie":false,"n_options":4,"probs":[0.0034789669617435377,0.005585926043509287,0.014632745508129976,0.9763023614866173],"hidden_sha256":"4daca80d51094b8fd070e9701213bc16a1c703b44f6e23baff13e183da71992a","first_in_process":false,"shard":"shard_00","index":33},{"id":"tv4_034","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.006053590415997734,"mean_abs_dp":0.0030267952079988635,"oracle_top2_margin":0.2533182700310583,"near_tie":false,"n_options":4,"probs":[0.08244065135034599,0.12408618336112658,0.2750790374062813,0.5183941278822461],"hidden_sha256":"10b634f5ff5103e56731b44dd22505a162f1d9e88140fb151dde3ddea19bbb2b","first_in_process":false,"shard":"shard_00","index":34},{"id":"tv4_035","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":126,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009287939785347366,"mean_abs_dp":0.0004643969892673791,"oracle_top2_margin":0.23375552424918428,"near_tie":false,"n_options":4,"probs":[0.3643729252972557,0.028999253717475245,0.5966229250435635,0.010004895941705486],"hidden_sha256":"81630853311d9b8fb6836a387d7d7f713aed6749175e2a4d0fc4727827fa73fc","first_in_process":false,"shard":"shard_00","index":35},{"id":"tv4_036","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":156,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0024118694446104705,"mean_abs_dp":0.0012561727330786636,"oracle_top2_margin":0.12129537667015589,"near_tie":false,"n_options":4,"probs":[0.15899162473376655,0.4411332942197072,0.324531739334676,0.07534334171185024],"hidden_sha256":"47f0e84c52080896cbede415cff883e6d858bfcc4a7d10dab2829164ee5567c5","first_in_process":false,"shard":"shard_00","index":36},{"id":"tv4_037","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008986386909936028,"mean_abs_dp":0.0004493193454968036,"oracle_top2_margin":0.6711215583174963,"near_tie":false,"n_options":4,"probs":[0.015255767555297475,0.07294326128831027,0.7922760554495217,0.11952491570687052],"hidden_sha256":"c80c169a9c34c1df7651379bb4b02ca7a1221393846358b87fc32bdd907e4abf","first_in_process":false,"shard":"shard_00","index":37},{"id":"tv4_038","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005278300340615209,"mean_abs_dp":0.0002724323436651834,"oracle_top2_margin":0.9219558028864439,"near_tie":false,"n_options":4,"probs":[0.0025898676716307506,0.0277437033005986,0.9507365013077395,0.018929927720031152],"hidden_sha256":"e0d356da1c166c92567428b9679f44cd05bbbe00c2c1d1831e59fb498d74dfcc","first_in_process":false,"shard":"shard_00","index":38},{"id":"tv4_039","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":87,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002927287097994169,"mean_abs_dp":0.001463643548997107,"oracle_top2_margin":0.5889692850440698,"near_tie":false,"n_options":4,"probs":[0.7869099228236345,0.2037747928490143,0.005680794632439027,0.003634489694912135],"hidden_sha256":"ddaa844bc87685d46f74afb968aedcdae38e3f3aa7ed3ee68b8ae5719eb54b83","first_in_process":false,"shard":"shard_00","index":39},{"id":"tv4_040","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":222,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.008910118326611927,"mean_abs_dp":0.004455059163305984,"oracle_top2_margin":0.2663605121271405,"near_tie":false,"n_options":4,"probs":[0.05414145920424271,0.3226190387792545,0.05118100395816406,0.5720584980583387],"hidden_sha256":"ba64329895789bd0e7480378e167b473792c8430380aaf1863f820e0f242a23a","first_in_process":true,"shard":"shard_01","index":0},{"id":"tv4_041","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":229,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005121723119235977,"mean_abs_dp":0.002560861559617978,"oracle_top2_margin":0.4427181728774944,"near_tie":false,"n_options":4,"probs":[0.07568814977010395,0.1701684898751513,0.621091126560255,0.13305223379448974],"hidden_sha256":"9ea01b6e8551b6692fbbfcaa9e7a5b6b2d143c4c9168f0a1db2244b59e9f0b48","first_in_process":false,"shard":"shard_01","index":1},{"id":"tv4_042","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0019192659092346354,"mean_abs_dp":0.0009596329546172773,"oracle_top2_margin":0.8585390325266999,"near_tie":false,"n_options":4,"probs":[0.9128311657590626,0.05781595685938738,0.014275822000795453,0.015077055380754448],"hidden_sha256":"6528b031ac43b4aa447fb371fe7d8500639c57d41c5383a57d5d774a7410062e","first_in_process":false,"shard":"shard_01","index":2},{"id":"tv4_043","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0018150651020764075,"mean_abs_dp":0.000968351091208625,"oracle_top2_margin":0.5045756833909496,"near_tie":false,"n_options":4,"probs":[0.03237563802323539,0.6991377241010455,0.07080058842124802,0.1976860494544711],"hidden_sha256":"588e7b3b832b7354d74abbb59e7250029e86d44bbad6e86b47c13957c1764ba0","first_in_process":false,"shard":"shard_01","index":3},{"id":"tv4_044","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0001739249684662436,"mean_abs_dp":8.696248423313617e-05,"oracle_top2_margin":0.9832281448196667,"near_tie":false,"n_options":4,"probs":[0.0015374048486155442,0.004523938166161808,0.005232253708719911,0.9887064032765027],"hidden_sha256":"b00ce0e8495896b7d16e40799cfe1cc006163f38893ca6859b198faf7613c89e","first_in_process":false,"shard":"shard_01","index":4},{"id":"tv4_045","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":112,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0019494404872056892,"mean_abs_dp":0.0009821135725219922,"oracle_top2_margin":0.3822788183711481,"near_tie":false,"n_options":4,"probs":[0.01573957150471402,0.20105250906623068,0.5869172521719888,0.19629066725706662],"hidden_sha256":"da64ab5a673f8591a2fe03bf2c1e11e5f56132e0a34d984cd83c1c6f0125996c","first_in_process":false,"shard":"shard_01","index":5},{"id":"tv4_046","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006594257591738817,"mean_abs_dp":0.00033148044444957175,"oracle_top2_margin":0.9119621800258289,"near_tie":false,"n_options":4,"probs":[0.9504683441918248,0.0069338012249008886,0.03980182406628641,0.002796030516987832],"hidden_sha256":"8846cd9087623e6eae3af2fbbba0f5922044cb5247de48efe277418175360816","first_in_process":false,"shard":"shard_01","index":6},{"id":"tv4_047","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0033104084275440704,"mean_abs_dp":0.0016552042137720447,"oracle_top2_margin":0.6958231902791943,"near_tie":false,"n_options":4,"probs":[0.022871447408469883,0.12930475862906735,0.819241319786117,0.028582474176345702],"hidden_sha256":"4aea8cfde2ce97cf077a487d1163a495698dc8bbbf5b00ee7e628838d4569bf4","first_in_process":false,"shard":"shard_01","index":7},{"id":"tv4_048","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":141,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006710963476086428,"mean_abs_dp":0.00033554817380431357,"oracle_top2_margin":0.9368998612950754,"near_tie":false,"n_options":4,"probs":[0.009133155001925589,0.9601513128570541,0.02423051889323586,0.006485013247784426],"hidden_sha256":"951844b6a7eba109e49e06e98d0ccea8bc3ab6565d5f040e1bfecb69a3042060","first_in_process":false,"shard":"shard_01","index":8},{"id":"tv4_049","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.001865468894659883,"mean_abs_dp":0.0009327344473299373,"oracle_top2_margin":0.8649083955667569,"near_tie":false,"n_options":4,"probs":[0.0021955986695278784,0.057900008806617706,0.013587133589566626,0.9263172589342877],"hidden_sha256":"3f148fc8d77e717146625cd08ef451cc01181090adefc5fe765df23af73ef3ac","first_in_process":false,"shard":"shard_01","index":9},{"id":"tv4_050","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005830913340872379,"mean_abs_dp":0.0029288409364389783,"oracle_top2_margin":0.407839880593036,"near_tie":false,"n_options":4,"probs":[0.2842156390703758,0.6807675985244077,0.013229578502759968,0.02178718390245665],"hidden_sha256":"307c1bf372269836c31b0955a1231c7d5964e71781d3fb57fb8c244a450d1c27","first_in_process":false,"shard":"shard_01","index":10},{"id":"tv4_051","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0030542560131828767,"mean_abs_dp":0.0015299893458599967,"oracle_top2_margin":0.6243594070338851,"near_tie":false,"n_options":4,"probs":[0.007425630963837511,0.02843379093956074,0.17286647618079698,0.7912741019158048],"hidden_sha256":"d9870127e6381f6c6cae6a5b3d6cd00cc1cf05f9bf96c5f376c0f3ca6ce221e1","first_in_process":false,"shard":"shard_01","index":11},{"id":"tv4_052","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":146,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003285316068393218,"mean_abs_dp":0.0001642658034196401,"oracle_top2_margin":0.8549797972832431,"near_tie":false,"n_options":4,"probs":[0.919369045293595,0.06390018504195238,0.006870537466437501,0.009860232198015143],"hidden_sha256":"aa3943ca821b4e1b6db696c4527628bdc63947c2858f0a8659fa39ccd6069b88","first_in_process":false,"shard":"shard_01","index":12},{"id":"tv4_053","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":370,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0022411723860550348,"mean_abs_dp":0.0011205861930275148,"oracle_top2_margin":0.7270944760158131,"near_tie":false,"n_options":4,"probs":[0.024309586842851866,0.8440402486830265,0.01837140937786095,0.11327875509626066],"hidden_sha256":"4c360ad80de8f2706e1e7f682c453a4c033590a876d63946142a02a824364275","first_in_process":false,"shard":"shard_01","index":13},{"id":"tv4_054","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":114,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001673541160959524,"mean_abs_dp":0.0009631865526956301,"oracle_top2_margin":0.050236600003381204,"near_tie":false,"n_options":4,"probs":[0.12820417735165549,0.3331642162866695,0.2583569695056455,0.2802746368560295],"hidden_sha256":"0e824ce2ee2c5b79dc9d530f0bfeb83d1d2972e85989ce4b25d33a5a0255d827","first_in_process":false,"shard":"shard_01","index":14},{"id":"tv4_055","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":174,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013088963487774552,"mean_abs_dp":0.0006544481743887207,"oracle_top2_margin":0.034482426730795546,"near_tie":false,"n_options":4,"probs":[0.2554423847157317,0.3447153403732318,0.31214725576576535,0.08769501914527113],"hidden_sha256":"246089832314b444a856f1ec375cbc6902866ac8ea137396ccb6a7290ae534af","first_in_process":false,"shard":"shard_01","index":15},{"id":"tv4_056","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00023486805257361132,"mean_abs_dp":0.00011752685405091046,"oracle_top2_margin":0.929137078840534,"near_tie":false,"n_options":4,"probs":[0.0036621264570335078,0.0178131571333327,0.02452220274331483,0.954002513666319],"hidden_sha256":"edef0046e0520ba8426c54994b1afe01a6d741156b20f52a015f8b5fbf24e798","first_in_process":false,"shard":"shard_01","index":16},{"id":"tv4_057","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.001101153048155945,"mean_abs_dp":0.0005505765240779691,"oracle_top2_margin":0.46040710392162176,"near_tie":false,"n_options":4,"probs":[0.016050737012130684,0.04646936983906487,0.2390981487412663,0.6983817444075381],"hidden_sha256":"8fd09fd9f8829b9288d985cc92455dfeeb99c675124e147de173579ad667f48d","first_in_process":false,"shard":"shard_01","index":17},{"id":"tv4_058","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":152,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004156091856389654,"mean_abs_dp":0.0021106533427058746,"oracle_top2_margin":0.3868433028253313,"near_tie":false,"n_options":4,"probs":[0.07529539639730087,0.26752285250461877,0.6489388257321742,0.008242925365906215],"hidden_sha256":"c5808a5e1d346ba822d6e9556284371e4fec6f54fdeadd3e9289d3a110c35694","first_in_process":false,"shard":"shard_01","index":18},{"id":"tv4_059","name":"answer","variant":"base","source":"tv4","type":"choice","tokens":116,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004905996121767897,"mean_abs_dp":0.0002452998060884265,"oracle_top2_margin":0.9575450347147793,"near_tie":false,"n_options":4,"probs":[0.9725158955497172,0.007203854089716052,0.0044987224467916125,0.01578152791377516],"hidden_sha256":"37ec42343847a842a4d04918be1a6405b2911f513549beda6edc8a7211ebe8bf","first_in_process":false,"shard":"shard_01","index":19},{"id":"tv4x_emotion_00","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":57,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009369246355243055,"mean_abs_dp":0.00034645659168252375,"oracle_top2_margin":0.35913577951608466,"near_tie":false,"n_options":6,"probs":[0.13959517409195982,0.22028332840473588,0.5779941521462958,0.02564509167770539,0.01884830144071475,0.01763395223858844],"hidden_sha256":"60c7ba6ce1832cf86caf184026fb6e46d9f2c813f490040c8df3ac3b00b9f2d1","first_in_process":false,"shard":"shard_01","index":20},{"id":"tv4x_emotion_01","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0035644509942139857,"mean_abs_dp":0.001288973379983599,"oracle_top2_margin":0.06860383828577754,"near_tie":false,"n_options":6,"probs":[0.26364922602657404,0.32599254973949254,0.036939812217678065,0.17311288177035453,0.07786424275812193,0.1224412874877789],"hidden_sha256":"d8ff23898dc4d3d0b7b8e049ed47f0a3352480538b2d1ec9ed6e961b156a3112","first_in_process":false,"shard":"shard_01","index":21},{"id":"tv4x_emotion_02","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":51,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000551915534030778,"mean_abs_dp":0.0001839718446769132,"oracle_top2_margin":0.8937235386954242,"near_tie":false,"n_options":6,"probs":[0.9243650687092994,0.008855022877918678,0.00599247457981993,0.02709498326331072,0.02982834484484939,0.0038641057248019806],"hidden_sha256":"8d9bafddb043d5dd2224008f19b59af794ac7b36878412007427075309fb3413","first_in_process":false,"shard":"shard_01","index":22},{"id":"tv4x_emotion_03","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007965045380979724,"mean_abs_dp":0.0002655015126993006,"oracle_top2_margin":0.7205916931033775,"near_tie":false,"n_options":6,"probs":[0.8158454486294244,0.09528532913564586,0.07932839437760934,0.0035365691586343034,0.0037201205662121067,0.0022841381324740216],"hidden_sha256":"f67c5941e1639a838e77338ed725d7fcf6644bfda3c4abdf9a4706d3112882e1","first_in_process":false,"shard":"shard_01","index":23},{"id":"tv4x_emotion_04","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004993403390693985,"mean_abs_dp":0.0001673501951464557,"oracle_top2_margin":0.9436175051795138,"near_tie":false,"n_options":6,"probs":[0.009723778174509267,0.9634817583615737,0.018980548806499534,0.003178847167339565,0.0018577437151298358,0.0027773237749480154],"hidden_sha256":"54a2d20e4a5c68dd698bc8c2b939a1873394db2f15c4dfe36c67b3f4dcec005b","first_in_process":false,"shard":"shard_01","index":24},{"id":"tv4x_emotion_05","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00048717465738118415,"mean_abs_dp":0.00018349296884948873,"oracle_top2_margin":0.5267872368274699,"near_tie":false,"n_options":6,"probs":[0.1292977191369497,0.1577083809811105,0.014851586457688616,0.6841732876136168,0.007985093875785566,0.005983931934848771],"hidden_sha256":"38cd3f4ef0586a7c65e7de623c918e4233fb3d253260191ce9919f1dda2e4299","first_in_process":false,"shard":"shard_01","index":25},{"id":"tv4x_emotion_06","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":52,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002326874096982845,"mean_abs_dp":0.0007756246989942783,"oracle_top2_margin":0.4531383187251465,"near_tie":false,"n_options":6,"probs":[0.012472749142810682,0.2570353426281701,0.7145370225623919,0.006609634194851536,0.004745169105732177,0.004600082366043483],"hidden_sha256":"f7810ea47ca7b93b7a0b210f53f38782e7063c077d89280cd1419ee28dbe4e18","first_in_process":false,"shard":"shard_01","index":26},{"id":"tv4x_emotion_07","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012448097965056526,"mean_abs_dp":0.0004188900805378322,"oracle_top2_margin":0.8020359796556499,"near_tie":false,"n_options":6,"probs":[0.8841579794692116,0.01516097569909447,0.007552455358070711,0.07979335059491584,0.008938364626453399,0.004396874252253932],"hidden_sha256":"871753beb73d8482b6f3a5b3421e4a5249b855b02c3dea653e97b6f206c2730c","first_in_process":false,"shard":"shard_01","index":27},{"id":"tv4x_emotion_08","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002169832032602681,"mean_abs_dp":0.0007232773442008874,"oracle_top2_margin":0.4470593629065237,"near_tie":false,"n_options":6,"probs":[0.2209196710722862,0.6717535252697592,0.09803020388451435,0.003915453885367191,0.002506935684200176,0.0028742102038729093],"hidden_sha256":"ee6d7f143b29927d820bfd3b842d20f24e630c0f14325443dcd919be173e750b","first_in_process":false,"shard":"shard_01","index":28},{"id":"tv4x_emotion_09","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000404423186520772,"mean_abs_dp":0.00016322375388824677,"oracle_top2_margin":0.7069872082119006,"near_tie":false,"n_options":6,"probs":[0.7928110911394286,0.03816364482505036,0.08572680307227754,0.06253457248959247,0.007699980421237849,0.013063908052413253],"hidden_sha256":"f472188c5cb70ed13ebe1c13fe34e556dfe906fe74b9834155c7282e5a4ba45d","first_in_process":false,"shard":"shard_01","index":29},{"id":"tv4x_emotion_10","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":56,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0026575716718781617,"mean_abs_dp":0.0009487746111730569,"oracle_top2_margin":0.4519727177002277,"near_tie":false,"n_options":6,"probs":[0.690980485064585,0.03336020830856984,0.017452349814814628,0.2336993153780936,0.01386893226562866,0.010638709168308362],"hidden_sha256":"015f86fab3daace2bb90ed9254c8562861e243de320b4a3db31ead3b7833b1b9","first_in_process":false,"shard":"shard_01","index":30},{"id":"tv4x_emotion_11","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":63,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002693861298220801,"mean_abs_dp":8.979537660733625e-05,"oracle_top2_margin":0.9419592280403212,"near_tie":false,"n_options":6,"probs":[0.015127206631764964,0.9581653971729689,0.01582162015015456,0.003513762855428854,0.0027024346567872163,0.004669578532895603],"hidden_sha256":"f4ac1649bbf7e3105c0244b5bea3d2f5d00fd78dfe1ff4a986e3f55027e69716","first_in_process":false,"shard":"shard_01","index":31},{"id":"tv4x_emotion_12","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000136636363308823,"mean_abs_dp":7.185636141929443e-05,"oracle_top2_margin":0.5647145684451723,"near_tie":false,"n_options":6,"probs":[0.03356012289268143,0.18700549643365694,0.7515554374683582,0.015828346423381417,0.0065024195871665885,0.005548177194755535],"hidden_sha256":"6ec312361a5c6d52139d52abe07604ac8e9eadbf1d6f0639de69de5d52bfc091","first_in_process":false,"shard":"shard_01","index":32},{"id":"tv4x_emotion_13","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.002386142863696872,"mean_abs_dp":0.0007953809545656136,"oracle_top2_margin":0.25219666980289757,"near_tie":false,"n_options":6,"probs":[0.34827128617503406,0.013700761603033929,0.010577334715846754,0.003739924391786553,0.604561089578899,0.01914960353539984],"hidden_sha256":"aefe77510ce9fde4a5912d13c2bb8af3e22a16280caa2436edc1af3e42166738","first_in_process":false,"shard":"shard_01","index":33},{"id":"tv4x_emotion_14","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007950650594919229,"mean_abs_dp":0.000268293900209566,"oracle_top2_margin":0.6706157015798525,"near_tie":false,"n_options":6,"probs":[0.07791791207984093,0.11912579984563512,0.7884091132288671,0.0060903365472410285,0.004599922825769031,0.0038569154726467198],"hidden_sha256":"220ac56b16a7fed860dd5473a3636622bebe02855cc242d7789bd2d1f7c26cb2","first_in_process":false,"shard":"shard_01","index":34},{"id":"tv4x_emotion_15","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":59,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003527815247336141,"mean_abs_dp":0.00011759384157788106,"oracle_top2_margin":0.637767381720322,"near_tie":false,"n_options":6,"probs":[0.008220473998410974,0.8106236180035942,0.17237809332635254,0.0033244000988052846,0.0022651987777513594,0.003188215795085586],"hidden_sha256":"f5cbca6b5c8c86da7f63db23777580fa4288e7a9fadd01b8a73250fa8285e0d0","first_in_process":false,"shard":"shard_01","index":35},{"id":"tv4x_emotion_16","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":65,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00017605176428714397,"mean_abs_dp":6.279861866553725e-05,"oracle_top2_margin":0.9295570190736654,"near_tie":false,"n_options":6,"probs":[0.946295765118087,0.011053134834254297,0.01703779458831242,0.01623829287180035,0.005506602685249723,0.00386840990229612],"hidden_sha256":"90e88fa19fd773ed40b2fa6b0335d51f61577a344b5f43b440482c774ff469cb","first_in_process":false,"shard":"shard_01","index":36},{"id":"tv4x_emotion_17","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010903272897996308,"mean_abs_dp":0.0005013498044545515,"oracle_top2_margin":0.5585129847508995,"near_tie":false,"n_options":6,"probs":[0.6753322128693388,0.03127381185310924,0.1110876886288384,0.04988477564158012,0.11577653841197086,0.016644972595162587],"hidden_sha256":"42912f0bc984b1a9094104d544bd6792086fa8f02b661288e16aaa382e15c0dd","first_in_process":false,"shard":"shard_01","index":37},{"id":"tv4x_emotion_18","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00026088213058483856,"mean_abs_dp":0.00011261679430912454,"oracle_top2_margin":0.9019695522988173,"near_tie":false,"n_options":6,"probs":[0.9298946139893757,0.010506269555048095,0.006015719232995259,0.019278724570249015,0.027461222536906388,0.0068434501154254835],"hidden_sha256":"15b14f26b16fbee6f8f3055050c5fe084c53fc98f122012e00c3b7aa565fa758","first_in_process":false,"shard":"shard_01","index":38},{"id":"tv4x_emotion_19","name":"emotion","variant":"base","source":"tv4x","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00032412941265919787,"mean_abs_dp":0.00011595722236842649,"oracle_top2_margin":0.9096538982162927,"near_tie":false,"n_options":6,"probs":[0.9304142764219354,0.021123628164348294,0.010199739461957884,0.018504271692054385,0.015652953722944177,0.004105130536759753],"hidden_sha256":"06c7425ef654ef1b5a90a94b249f653547f0e140dd7fd39ee163dcf854fa46a3","first_in_process":false,"shard":"shard_01","index":39},{"id":"tv4x_qnli_00","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.9886291061753953e-05,"mean_abs_dp":3.988629106165334e-05,"oracle_top2_margin":0.967676077803126,"near_tie":false,"n_options":2,"probs":[0.9838779251926246,0.016122074807375302],"hidden_sha256":"3474bbba42c377711c5f83a0fd6e3fbd331f74ca79d9f46358c7777d4cf3e4c7","first_in_process":true,"shard":"shard_02","index":0,"hidden":{"min_pos_cos":0.9997857168488112,"min_pos_cos_index":70,"max_abs_diff":0.13932740688323975,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4x_qnli_01","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006103756105809849,"mean_abs_dp":0.0006103756105809779,"oracle_top2_margin":0.8131166435987733,"near_tie":false,"n_options":2,"probs":[0.0928313025900324,0.9071686974099676],"hidden_sha256":"c5cc5a7cf9e762ef9d71dc2af9cce75d39afe71b00851f6c57744a3452bd3534","first_in_process":false,"shard":"shard_02","index":1},{"id":"tv4x_qnli_02","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006325474768631434,"mean_abs_dp":0.0006325474768630324,"oracle_top2_margin":0.8113164357001165,"near_tie":false,"n_options":2,"probs":[0.9050256703731953,0.09497432962680487],"hidden_sha256":"683f70e83121759dac0e3727f67a3e7656e6fc7878a79bd2a05ee2277cf3cb5d","first_in_process":false,"shard":"shard_02","index":2},{"id":"tv4x_qnli_03","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008335507675750961,"mean_abs_dp":0.0008335507675750857,"oracle_top2_margin":0.8867196111326728,"near_tie":false,"n_options":2,"probs":[0.05747374520123872,0.9425262547987614],"hidden_sha256":"bb9fd5e77910be085942f865da0e75433fe697855ef29d8fd02db0ccc70a2f6b","first_in_process":false,"shard":"shard_02","index":3},{"id":"tv4x_qnli_04","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":157,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001830686568650708,"mean_abs_dp":0.00018306865686504303,"oracle_top2_margin":0.7598955557679068,"near_tie":false,"n_options":2,"probs":[0.8797647092270884,0.12023529077291165],"hidden_sha256":"c87ac4d79e97777ff054fdc229369c37dd70658555b5f77ee597adbf676174c6","first_in_process":false,"shard":"shard_02","index":4},{"id":"tv4x_qnli_05","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003459740046881321,"mean_abs_dp":0.0034597400468812656,"oracle_top2_margin":0.2604357567584594,"near_tie":false,"n_options":2,"probs":[0.6336776184261109,0.36632238157388897],"hidden_sha256":"b4786e47454c722f8c394bccba7ee79de579f7300d5233a5eb6a8af96813c579","first_in_process":false,"shard":"shard_02","index":5},{"id":"tv4x_qnli_06","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004636591641467014,"mean_abs_dp":0.0004636591641466875,"oracle_top2_margin":0.7349581738845581,"near_tie":false,"n_options":2,"probs":[0.8679427461064257,0.13205725389357428],"hidden_sha256":"40827da0532b293c95770f05db466a69cddcbad7fc68ad043fcc39921598b55c","first_in_process":false,"shard":"shard_02","index":6},{"id":"tv4x_qnli_07","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":64,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011118706903517905,"mean_abs_dp":0.0011118706903517905,"oracle_top2_margin":0.06001024267368793,"near_tie":false,"n_options":2,"probs":[0.5288932506464922,0.4711067493535078],"hidden_sha256":"d06c436d813345d70a85b9fb5bc6afaec6ddb1bae93adb1b6a66ded73b359781","first_in_process":false,"shard":"shard_02","index":7},{"id":"tv4x_qnli_08","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006425404429697412,"mean_abs_dp":0.0006425404429697204,"oracle_top2_margin":0.79770824729877,"near_tie":false,"n_options":2,"probs":[0.10178841679358465,0.8982115832064153],"hidden_sha256":"bc34038136c7c281bb27bf8feb25d29a973386ca441f91d00171917ce850355e","first_in_process":false,"shard":"shard_02","index":8},{"id":"tv4x_qnli_09","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.5655257051161442e-05,"mean_abs_dp":1.5655257051125013e-05,"oracle_top2_margin":0.9488240328840618,"near_tie":false,"n_options":2,"probs":[0.02560363881502024,0.9743963611849797],"hidden_sha256":"3570bb26167d1274b99a02dd6ac3dec7180b0917492b253bbf03ae74a50091ba","first_in_process":false,"shard":"shard_02","index":9},{"id":"tv4x_qnli_10","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00014783499359076122,"mean_abs_dp":0.00014783499359074213,"oracle_top2_margin":0.966925876394352,"near_tie":false,"n_options":2,"probs":[0.01668489679641478,0.9833151032035853],"hidden_sha256":"357c7e6303a1090357953f2c72a1baa5b228b893b0697756edc16f2c13bfcfa4","first_in_process":false,"shard":"shard_02","index":10},{"id":"tv4x_qnli_11","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":59,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007524523087024948,"mean_abs_dp":0.0007524523087023699,"oracle_top2_margin":0.9082556399693326,"near_tie":false,"n_options":2,"probs":[0.9548802722933686,0.045119727706631266],"hidden_sha256":"30776b350edaafa8be4634186de6dd906f54ed7c75e57310e92a609cef07ba1e","first_in_process":false,"shard":"shard_02","index":11},{"id":"tv4x_qnli_12","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008611907376062655,"mean_abs_dp":0.0008611907376062378,"oracle_top2_margin":0.3794817865854016,"near_tie":false,"n_options":2,"probs":[0.6888797025550946,0.31112029744490544],"hidden_sha256":"8abb42807f4a272fdfeb351f057f31b8d335f2a82c3b927df62c8b08dd4c09a6","first_in_process":false,"shard":"shard_02","index":12},{"id":"tv4x_qnli_13","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018596385030624774,"mean_abs_dp":0.0018596385030624288,"oracle_top2_margin":0.7759343256201553,"near_tie":false,"n_options":2,"probs":[0.8861075243070152,0.11389247569298477],"hidden_sha256":"c5c8d05ce6f61f83f37609063b7a974474b6df5c060ee4ea810224a417a7a1e4","first_in_process":false,"shard":"shard_02","index":13},{"id":"tv4x_qnli_14","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00029650985149856646,"mean_abs_dp":0.0002965098514985526,"oracle_top2_margin":0.7304054579392312,"near_tie":false,"n_options":2,"probs":[0.864906219118117,0.13509378088188304],"hidden_sha256":"da921652b2ce78f07c91972b0fb9d0093a699c81b263e91fee1d1db0cdd0c476","first_in_process":false,"shard":"shard_02","index":14},{"id":"tv4x_qnli_15","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002517788545950417,"mean_abs_dp":0.00025177885459497923,"oracle_top2_margin":0.9337811649427256,"near_tie":false,"n_options":2,"probs":[0.9671423613259578,0.032857638674042254],"hidden_sha256":"0b96cb100d5d7ebbedcb82483ee15a76c9dc395d0d3c1f99732e0a10aa22ada5","first_in_process":false,"shard":"shard_02","index":15},{"id":"tv4x_qnli_16","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010727166712362068,"mean_abs_dp":0.0010727166712361513,"oracle_top2_margin":0.43803847138581853,"near_tie":false,"n_options":2,"probs":[0.7179465190216732,0.28205348097832694],"hidden_sha256":"40a2898ce59d992eb096a9154038e8edae98da424242d1c5f699f4b68d158ecd","first_in_process":false,"shard":"shard_02","index":16},{"id":"tv4x_qnli_17","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00014479161242102884,"mean_abs_dp":0.00014479161242100455,"oracle_top2_margin":0.9211082745803174,"near_tie":false,"n_options":2,"probs":[0.9606989289025797,0.039301071097420266],"hidden_sha256":"e108da8579d7876a4da65133bed0db5da6b0820d60a666154fd97b8333b4b54f","first_in_process":false,"shard":"shard_02","index":17},{"id":"tv4x_qnli_18","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0022346183299525313,"mean_abs_dp":0.0022346183299525244,"oracle_top2_margin":0.8523963888636283,"near_tie":false,"n_options":2,"probs":[0.07603642389813839,0.9239635761018616],"hidden_sha256":"26be8f78fba227859ef34f5e81a84ef46074dc31bdcf3090b44bffcbf5167f9f","first_in_process":false,"shard":"shard_02","index":18},{"id":"tv4x_qnli_19","name":"answers","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0025145515690981057,"mean_abs_dp":0.002514551569098092,"oracle_top2_margin":0.7034361806374811,"near_tie":false,"n_options":2,"probs":[0.8492035387496425,0.15079646125035753],"hidden_sha256":"9f27e0ae06ad371f3093ede62867aef0a24c4097d877dcae92571b3e3d415d50","first_in_process":false,"shard":"shard_02","index":19},{"id":"tv4x_paws_00","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008756876455379148,"mean_abs_dp":0.0008756876455378731,"oracle_top2_margin":0.8207418652737444,"near_tie":false,"n_options":2,"probs":[0.08875337971759,0.9112466202824101],"hidden_sha256":"3cfec852f49795996e3748693bcc3e7f10f150c4ebbb41d49c042129bdef39f6","first_in_process":false,"shard":"shard_02","index":20},{"id":"tv4x_paws_01","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":139,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012269468100810088,"mean_abs_dp":0.001226946810080981,"oracle_top2_margin":0.07396326897051358,"near_tie":false,"n_options":2,"probs":[0.5382085812953378,0.4617914187046622],"hidden_sha256":"568885347e8ae36935fc1d7627ddf4a7c17cf844b94eca83538300c2c48ba321","first_in_process":false,"shard":"shard_02","index":21},{"id":"tv4x_paws_02","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":116,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001193634772264085,"mean_abs_dp":0.00011936347722633044,"oracle_top2_margin":0.9416461714567297,"near_tie":false,"n_options":2,"probs":[0.02929627774886143,0.9707037222511384],"hidden_sha256":"61f7bf16edf5116cbb3fc2545429499118b164b46762613b5585f194b65e686d","first_in_process":false,"shard":"shard_02","index":22},{"id":"tv4x_paws_03","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001220949733434784,"mean_abs_dp":0.00012209497334346453,"oracle_top2_margin":0.9530152365841582,"near_tie":false,"n_options":2,"probs":[0.9766297132654226,0.02337028673457745],"hidden_sha256":"e681551c33bc0c543de3cd97129c709cdc820d9c65a29533b0a23dda40795b1a","first_in_process":false,"shard":"shard_02","index":23},{"id":"tv4x_paws_04","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":105,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.921192985516691e-05,"mean_abs_dp":9.921192985511834e-05,"oracle_top2_margin":0.9194586795437724,"near_tie":false,"n_options":2,"probs":[0.9598285517017413,0.04017144829825868],"hidden_sha256":"26bbdbdd48e8227ef2420be3ae62be19bb6d0a8e009003170143cba9d67c1edb","first_in_process":false,"shard":"shard_02","index":24},{"id":"tv4x_paws_05","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":107,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005634152021454952,"mean_abs_dp":0.0056341520214549246,"oracle_top2_margin":0.3452288520436342,"near_tie":false,"n_options":2,"probs":[0.678248578043272,0.32175142195672796],"hidden_sha256":"802c61fe76aefa6cca6790af778d3aa4defd86826d9151e51496624895b31e09","first_in_process":false,"shard":"shard_02","index":25},{"id":"tv4x_paws_06","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":112,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004862674604222961,"mean_abs_dp":0.004862674604222961,"oracle_top2_margin":0.15852749594349458,"near_tie":false,"n_options":2,"probs":[0.42559892663247567,0.5744010733675243],"hidden_sha256":"119785670bbd7ce29177b3eab138b017579d30d44c592cb06c0181e99532e801","first_in_process":false,"shard":"shard_02","index":26},{"id":"tv4x_paws_07","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002648072244143887,"mean_abs_dp":0.00026480722441437135,"oracle_top2_margin":0.9577955021898784,"near_tie":false,"n_options":2,"probs":[0.9791625583193536,0.02083744168064642],"hidden_sha256":"1fadde290ce377f665a1a3364b347cbef640239f1c489e58750bf984ef961a19","first_in_process":false,"shard":"shard_02","index":27},{"id":"tv4x_paws_08","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.939532876344302e-05,"mean_abs_dp":7.939532876332159e-05,"oracle_top2_margin":0.9480983046539663,"near_tie":false,"n_options":2,"probs":[0.9741285476557465,0.025871452344253625],"hidden_sha256":"b52e30a6ec77e21813a117a1917497c54d3a508a3a2c2c79ee7725e7ba3db7b0","first_in_process":false,"shard":"shard_02","index":28},{"id":"tv4x_paws_09","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":140,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0006878012007994899,"mean_abs_dp":0.0006878012007994275,"oracle_top2_margin":0.8694028325570654,"near_tie":false,"n_options":2,"probs":[0.9353892174793321,0.06461078252066782],"hidden_sha256":"a596315cba80cd826c6a6cca403a5d1fc5358163bdff2e7473e517c011ab8636","first_in_process":false,"shard":"shard_02","index":29},{"id":"tv4x_paws_10","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014963865998739445,"mean_abs_dp":0.0014963865998738474,"oracle_top2_margin":0.8572341261731035,"near_tie":false,"n_options":2,"probs":[0.0728793235133221,0.9271206764866778],"hidden_sha256":"9c6fa01404fe6a00ae2cbead1904b58f7e422724d1569d071d05348714d1df99","first_in_process":false,"shard":"shard_02","index":30},{"id":"tv4x_paws_11","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001770869079801493,"mean_abs_dp":0.0001770869079801389,"oracle_top2_margin":0.9195431167829624,"near_tie":false,"n_options":2,"probs":[0.9595944714835011,0.04040552851649897],"hidden_sha256":"a8cc7b237b2563849fa8cc7c55ea3e72cba68300c639c99f2350454e1324b71e","first_in_process":false,"shard":"shard_02","index":31},{"id":"tv4x_paws_12","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001239534029072939,"mean_abs_dp":0.00012395340290722798,"oracle_top2_margin":0.9457556296060624,"near_tie":false,"n_options":2,"probs":[0.972753861400124,0.027246138599876027],"hidden_sha256":"e8aaa0a6e0853d6a39857642ad3e32b392cefc16334f3271709126cbf89221d6","first_in_process":false,"shard":"shard_02","index":32},{"id":"tv4x_paws_13","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":132,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016925672419655646,"mean_abs_dp":0.0016925672419655508,"oracle_top2_margin":0.5612642409230675,"near_tie":false,"n_options":2,"probs":[0.7823246877034993,0.21767531229650075],"hidden_sha256":"e356648b392986613b95b5100da983d488f056f756cc49a0fb2edf453122e319","first_in_process":false,"shard":"shard_02","index":33},{"id":"tv4x_paws_14","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006013098327997471,"mean_abs_dp":0.0006013098327997297,"oracle_top2_margin":0.9160329963674159,"near_tie":false,"n_options":2,"probs":[0.042584811649091815,0.9574151883509082],"hidden_sha256":"47da6d77acec337d533067e41a6653150a94d59e1869d356444455ad64d30631","first_in_process":false,"shard":"shard_02","index":34},{"id":"tv4x_paws_15","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000625135448146609,"mean_abs_dp":0.0006251354481465916,"oracle_top2_margin":0.9209250852704204,"near_tie":false,"n_options":2,"probs":[0.9610876780833568,0.03891232191664325],"hidden_sha256":"8b2df03a6e4dc7543c697534913720722e0f4a0c6e98cea1686f6272475eac65","first_in_process":false,"shard":"shard_02","index":35},{"id":"tv4x_paws_16","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":1.9852786044282505e-05,"mean_abs_dp":1.9852786044249546e-05,"oracle_top2_margin":0.9487658868484278,"near_tie":false,"n_options":2,"probs":[0.9743630906381696,0.02563690936183037],"hidden_sha256":"d2ccd2c4710108130947edf58aa6afe8a51d60c7f1f3eba32c99550d05de12ec","first_in_process":false,"shard":"shard_02","index":36},{"id":"tv4x_paws_17","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0015464431284453184,"mean_abs_dp":0.0015464431284453184,"oracle_top2_margin":0.19665281051821115,"near_tie":false,"n_options":2,"probs":[0.599872848387551,0.40012715161244916],"hidden_sha256":"69c81e605c6143edf324c7117b4c9daa150908fba94d71d2ac10546cb20e42ce","first_in_process":false,"shard":"shard_02","index":37},{"id":"tv4x_paws_18","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.758051857292101e-05,"mean_abs_dp":8.758051857287417e-05,"oracle_top2_margin":0.9666590415731975,"near_tie":false,"n_options":2,"probs":[0.016758059731974174,0.9832419402680259],"hidden_sha256":"3e4b78d21db6bff75dab8bf5555842ba48e33c983fc4d6af8dc837398c0638c8","first_in_process":false,"shard":"shard_02","index":38},{"id":"tv4x_paws_19","name":"paraphrase","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011907874590666445,"mean_abs_dp":0.00011907874590665231,"oracle_top2_margin":0.9576614130338148,"near_tie":false,"n_options":2,"probs":[0.9789497852628141,0.021050214737185984],"hidden_sha256":"cd3acd7051e6472b8b636ca5c1c3d72e60e789c4e7151cb72ac1308289e4ca75","first_in_process":false,"shard":"shard_02","index":39},{"id":"tv4x_sciq_00","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":163,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":7.527447271571397e-05,"mean_abs_dp":3.76372363578752e-05,"oracle_top2_margin":0.9809464409661907,"near_tie":false,"n_options":4,"probs":[0.0014267986359288625,0.9878156987944242,0.00401226999875992,0.006745232570886992],"hidden_sha256":"127d5c7b27760ce6af112694f74aaa4828689137da7923f249357a60ec987965","first_in_process":true,"shard":"shard_03","index":0},{"id":"tv4x_sciq_01","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":71,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0005282298336531932,"mean_abs_dp":0.00034335213982475583,"oracle_top2_margin":0.7009752393049276,"near_tie":false,"n_options":4,"probs":[0.011030936028754717,0.12215972437074786,0.04274735506267809,0.8240619845378192],"hidden_sha256":"174a955da36fc6158d7c493ad5a6ceaa4326b150f812da1d1ff6b823b2964969","first_in_process":false,"shard":"shard_03","index":1},{"id":"tv4x_sciq_02","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":118,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001822676892379027,"mean_abs_dp":9.113384461888554e-05,"oracle_top2_margin":0.9777094276883521,"near_tie":false,"n_options":4,"probs":[0.9861149518693166,0.008730464693228215,0.002272245610900152,0.002882337826554856],"hidden_sha256":"14c6b4103fee7c7a4b5f8e04f25f36cdf9b1816e9285d1f778f1b3ab1dfd6594","first_in_process":false,"shard":"shard_03","index":2},{"id":"tv4x_sciq_03","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":109,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":7.825530033822581e-05,"mean_abs_dp":3.912765016915562e-05,"oracle_top2_margin":0.9846019844932212,"near_tie":false,"n_options":4,"probs":[0.0014803166903850917,0.0038287137876856104,0.004987247167309059,0.9897037223546201],"hidden_sha256":"483adea762ec0ea8780f1770e07b2e131c2e01bd7024f20e6a123eecc870c589","first_in_process":false,"shard":"shard_03","index":3},{"id":"tv4x_sciq_04","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":134,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.8686155385473115e-05,"mean_abs_dp":9.343077692705766e-06,"oracle_top2_margin":0.9820033764228081,"near_tie":false,"n_options":4,"probs":[0.0016695319620468002,0.9883841803686901,0.006401317514535552,0.0035449701547275716],"hidden_sha256":"71d29c7ed42b1cd0a53d0ef3a3ac312ab24f5f0d423d32903974e79e72fd3e9b","first_in_process":false,"shard":"shard_03","index":4},{"id":"tv4x_sciq_05","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005126522199135897,"mean_abs_dp":0.00025632610995680047,"oracle_top2_margin":0.9571367735230473,"near_tie":false,"n_options":4,"probs":[0.9680848590750535,0.011619606041646822,0.011199914913439083,0.009095619969860665],"hidden_sha256":"ec9e1eba2fe7d97056216f04423a113e066f0e3e40936124b37012661190bbb4","first_in_process":false,"shard":"shard_03","index":5},{"id":"tv4x_sciq_06","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":204,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.599646040519655e-05,"mean_abs_dp":1.7998230202604455e-05,"oracle_top2_margin":0.9905170864635728,"near_tie":false,"n_options":4,"probs":[0.0012099994417116948,0.9935370841996637,0.0029754491476947447,0.00227746721092989],"hidden_sha256":"26868acd33c97df31680e107ca46978b2f01663d5edc0cf3b406eab73148646e","first_in_process":false,"shard":"shard_03","index":6},{"id":"tv4x_sciq_07","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.1093204798973133e-05,"mean_abs_dp":5.546602399500553e-06,"oracle_top2_margin":0.9897740900749374,"near_tie":false,"n_options":4,"probs":[0.0011286977898510451,0.9931885388715463,0.0033956617471991987,0.0022871015914034485],"hidden_sha256":"21f5097625f9b63d05156ea1c2a5b73a38ef8ac45a72512eeec80fe9bd5ad871","first_in_process":false,"shard":"shard_03","index":7},{"id":"tv4x_sciq_08","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.445693915290216e-05,"mean_abs_dp":2.2924086127253996e-05,"oracle_top2_margin":0.9845569664084634,"near_tie":false,"n_options":4,"probs":[0.9897669951905903,0.005125353522041281,0.0026593423428168416,0.0024483089445515548],"hidden_sha256":"6ea12105b167cbacd5e94e01bee3ce10cf9b18e9c1f2f5715de6b3a4ab920670","first_in_process":false,"shard":"shard_03","index":8},{"id":"tv4x_sciq_09","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00012485531871675448,"mean_abs_dp":6.24276593584076e-05,"oracle_top2_margin":0.9786206076387443,"near_tie":false,"n_options":4,"probs":[0.003576851869364066,0.005250932572359501,0.0062027920250115436,0.9849694235332648],"hidden_sha256":"ab7fb263997c637a75ad96a20ef39d649c21ec2b8063e77cbf42c91201f53aba","first_in_process":false,"shard":"shard_03","index":9},{"id":"tv4x_sciq_10","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.0001017005464941656,"mean_abs_dp":5.085027324703672e-05,"oracle_top2_margin":0.9825931896772335,"near_tie":false,"n_options":4,"probs":[0.003235525496118858,0.0049303414988333735,0.004162129587611896,0.987672003417436],"hidden_sha256":"d0d4d6861cb7067f8ea09178b926346d651dedc954ae5d85e7660bfead880ff3","first_in_process":false,"shard":"shard_03","index":10},{"id":"tv4x_sciq_11","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":378,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00030605685790574544,"mean_abs_dp":0.00015302842895287304,"oracle_top2_margin":0.9561027133595873,"near_tie":false,"n_options":4,"probs":[0.016635511970714993,0.007067709457264438,0.9731920610563687,0.0031047175156519386],"hidden_sha256":"4d5f4f08de7c71f72f9516297cba947ab3ba086524fedaf4a3312a7932d2296c","first_in_process":false,"shard":"shard_03","index":11},{"id":"tv4x_sciq_12","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":202,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.478027549734918e-05,"mean_abs_dp":1.461730980401187e-05,"oracle_top2_margin":0.9850719395803337,"near_tie":false,"n_options":4,"probs":[0.0006896313343647874,0.004103588745822633,0.9901208784128576,0.0050859015069548786],"hidden_sha256":"99ab342db5afa3a845838031adf93f9226ad8fe6e8a62805fd10d55312dc61b8","first_in_process":false,"shard":"shard_03","index":12},{"id":"tv4x_sciq_13","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":163,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012485005598927845,"mean_abs_dp":6.242502799466048e-05,"oracle_top2_margin":0.978102360961836,"near_tie":false,"n_options":4,"probs":[0.9850066704092886,0.007106932957314175,0.0029857199663220085,0.004900676667075367],"hidden_sha256":"38273b0c53d7b9fc4448dd21b1125ef89743cf8b4b077e7ab21fe6e1ad27b09d","first_in_process":false,"shard":"shard_03","index":13},{"id":"tv4x_sciq_14","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":141,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":6.323515506589938e-05,"mean_abs_dp":3.382564521335423e-05,"oracle_top2_margin":0.9785787223861179,"near_tie":false,"n_options":4,"probs":[0.0032204774264077652,0.005218223173245354,0.006456961970439747,0.9851043374299071],"hidden_sha256":"b9174728526d1123a8ce3cb14f59ce2f2a06522992b8366407f7f9262a78429e","first_in_process":false,"shard":"shard_03","index":14},{"id":"tv4x_sciq_15","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":277,"calls":5,"padded_tokens":320,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.073807124085625e-05,"mean_abs_dp":7.0893699654846795e-06,"oracle_top2_margin":0.98899586086744,"near_tie":false,"n_options":4,"probs":[0.0011589821376340112,0.9924712250197233,0.002915184543642354,0.0034546082990002643],"hidden_sha256":"24d51a86d98e0412a0d0a492aaa5c1db4a37a54172c8e3713fc7a8923642cde7","first_in_process":false,"shard":"shard_03","index":15},{"id":"tv4x_sciq_16","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":352,"calls":6,"padded_tokens":384,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.00821808694737225,"mean_abs_dp":0.00433127329506728,"oracle_top2_margin":0.6374411741262631,"near_tie":false,"n_options":4,"probs":[0.040155561371839746,0.12676251740876116,0.0840403252330687,0.7490415959863304],"hidden_sha256":"767619c7dde2c5563336a55926b8cdbbb8a51a7b14c420991db67d120c1e8565","first_in_process":false,"shard":"shard_03","index":16},{"id":"tv4x_sciq_17","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":185,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.131412147529879e-05,"mean_abs_dp":2.94524735685274e-05,"oracle_top2_margin":0.977520044286674,"near_tie":false,"n_options":4,"probs":[0.9860462272305416,0.008440835036615043,0.003789387247985245,0.0017235504848581871],"hidden_sha256":"64f8ab8a51b5030cc51e9bbae5531638921ae047b3db28128609264305a34273","first_in_process":false,"shard":"shard_03","index":17},{"id":"tv4x_sciq_18","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.663289743371383e-05,"mean_abs_dp":4.331644871687318e-05,"oracle_top2_margin":0.9656887266619748,"near_tie":false,"n_options":4,"probs":[0.9758579287551856,0.010057148252075133,0.007285117234347158,0.00679980575839203],"hidden_sha256":"6bd8698cc157417ab16331e198b49d4af1e57729a52771127a53b2ec2c2fce13","first_in_process":false,"shard":"shard_03","index":18},{"id":"tv4x_sciq_19","name":"answer","variant":"base","source":"tv4x","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":2.470734745363412e-05,"mean_abs_dp":1.2353673726799388e-05,"oracle_top2_margin":0.9916925929078704,"near_tie":false,"n_options":4,"probs":[0.9943205148162627,0.0025896953130356954,0.001786148660183118,0.0013036412105184093],"hidden_sha256":"57d2aba0fd07cea6221fc817040b455bd33772a58654bf3f9681fbaacd652d04","first_in_process":false,"shard":"shard_03","index":19},{"id":"tv4x_legacy_holdout_00","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001696658765496517,"mean_abs_dp":0.00016966587654962048,"oracle_top2_margin":0.9635829051438438,"near_tie":false,"n_options":2,"probs":[0.9819611184484716,0.018038881551528472],"hidden_sha256":"159a623540256671c83a3ed4eb8961d55449e01bc0aa5f995f6dcc543e1e9336","first_in_process":false,"shard":"shard_03","index":20},{"id":"tv4x_legacy_holdout_01","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.425285185759655e-05,"mean_abs_dp":8.425285185751068e-05,"oracle_top2_margin":0.9796117050734497,"near_tie":false,"n_options":2,"probs":[0.010278400315132621,0.9897215996848674],"hidden_sha256":"2c1c589a927799921dda92079016adc61cbf397222c760ef551660e35123d352","first_in_process":false,"shard":"shard_03","index":21},{"id":"tv4x_legacy_holdout_02","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018061937366764536,"mean_abs_dp":0.0001806193736676124,"oracle_top2_margin":0.9564229434436572,"near_tie":false,"n_options":2,"probs":[0.9783920910954962,0.0216079089045038],"hidden_sha256":"9c6b916ac060ce21d0719587ef03b89e281a2f18bfb62486e809eef08d59a263","first_in_process":false,"shard":"shard_03","index":22},{"id":"tv4x_legacy_holdout_03","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.9640641291118754e-06,"mean_abs_dp":1.9640641291066713e-06,"oracle_top2_margin":0.9698785191123516,"near_tie":false,"n_options":2,"probs":[0.01506270450795332,0.9849372954920467],"hidden_sha256":"c08e1d9c43e63743a9d9057b52ff9d7077b2e126803a84d183d4976e538e4eda","first_in_process":false,"shard":"shard_03","index":23},{"id":"tv4x_legacy_holdout_04","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.44456037089225e-05,"mean_abs_dp":7.444560370888087e-05,"oracle_top2_margin":0.963808624235394,"near_tie":false,"n_options":2,"probs":[0.9819787577214059,0.018021242278594106],"hidden_sha256":"2c81cab7341d4d0c064ff41c388330e84f712dbaa7e9e71777dbdb7a7e1671f3","first_in_process":false,"shard":"shard_03","index":24},{"id":"tv4x_legacy_holdout_05","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.865593333541353e-05,"mean_abs_dp":1.865593333540052e-05,"oracle_top2_margin":0.9798986789108202,"near_tie":false,"n_options":2,"probs":[0.010032004611254456,0.9899679953887455],"hidden_sha256":"86f4f8b2337e432f3a367504d3cbaf1e0bfec507b69ea8388e9067c0b13dc55b","first_in_process":false,"shard":"shard_03","index":25},{"id":"tv4x_legacy_holdout_06","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001970480799928273,"mean_abs_dp":0.0019704807999281967,"oracle_top2_margin":0.9308847344232876,"near_tie":false,"n_options":2,"probs":[0.9634718864117157,0.03652811358828445],"hidden_sha256":"5c8668912e97d8830e4c0dede46f180622320c1d5f5648f53f1e0454a472815d","first_in_process":false,"shard":"shard_03","index":26},{"id":"tv4x_legacy_holdout_07","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.2186397359846843e-06,"mean_abs_dp":2.2186397359014176e-06,"oracle_top2_margin":0.9809323842367962,"near_tie":false,"n_options":2,"probs":[0.00953602652133783,0.9904639734786622],"hidden_sha256":"2d5979d8b9ab277d267e9e65fddb841ad3b5b777d4d40d7c5b32437501b87892","first_in_process":false,"shard":"shard_03","index":27},{"id":"tv4x_legacy_holdout_08","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011307065711058106,"mean_abs_dp":0.00011307065711055331,"oracle_top2_margin":0.9719924049726297,"near_tie":false,"n_options":2,"probs":[0.9861092731434254,0.013890726856574581],"hidden_sha256":"08fe97ef83662be808c3825db48c1a3d506eb1ac9087476a531d86c2002b53c1","first_in_process":false,"shard":"shard_03","index":28},{"id":"tv4x_legacy_holdout_09","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.094615587519555e-05,"mean_abs_dp":2.0946155875098404e-05,"oracle_top2_margin":0.977575169351521,"near_tie":false,"n_options":2,"probs":[0.011233361480114619,0.9887666385198854],"hidden_sha256":"af67768443479dcf833b05661b4866707f05ebaa97e45cb48a7175c5b0fda85a","first_in_process":false,"shard":"shard_03","index":29},{"id":"tv4x_legacy_holdout_10","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":4.889547561035684e-06,"mean_abs_dp":4.889547560992316e-06,"oracle_top2_margin":0.9679284058204262,"near_tie":false,"n_options":2,"probs":[0.9839690924577741,0.016030907542225907],"hidden_sha256":"ba44933a2ddff932fabd4fd719b30cd689bbc0bd2187e7f4493a7571c705cbf1","first_in_process":false,"shard":"shard_03","index":30},{"id":"tv4x_legacy_holdout_11","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.6614781623984664e-05,"mean_abs_dp":4.661478162395864e-05,"oracle_top2_margin":0.9685686046272135,"near_tie":false,"n_options":2,"probs":[0.01566908290476926,0.9843309170952307],"hidden_sha256":"74f935d552c82602cda7ae1eb6efbe8ef70fccf5bfaa36fef2188a5c9fa75c33","first_in_process":false,"shard":"shard_03","index":31},{"id":"tv4x_legacy_holdout_12","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00039188129133743005,"mean_abs_dp":0.00039188129133735025,"oracle_top2_margin":0.966612234838911,"near_tie":false,"n_options":2,"probs":[0.9829142361281182,0.017085763871881914],"hidden_sha256":"718bd8eed96394748bf81d63171da1f6d38f9fd9ec66ce81ff7a7f1f6b75d736","first_in_process":false,"shard":"shard_03","index":32},{"id":"tv4x_legacy_holdout_13","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.9565444967683905e-05,"mean_abs_dp":4.9565444967637935e-05,"oracle_top2_margin":0.9734283318284275,"near_tie":false,"n_options":2,"probs":[0.013335399530753918,0.9866646004692461],"hidden_sha256":"1ad85244b9224aebf3b8387311ca8a03c6ce1f557e9828b175b7411e6579d866","first_in_process":false,"shard":"shard_03","index":33},{"id":"tv4x_legacy_holdout_14","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011329991462208922,"mean_abs_dp":0.0011329991462208158,"oracle_top2_margin":0.9391565521574167,"near_tie":false,"n_options":2,"probs":[0.9684452769324875,0.031554723067512426],"hidden_sha256":"92b803946dcd5c5b75f1b328e8458cbd701693c2286f766a38d5c3e84d5a2013","first_in_process":false,"shard":"shard_03","index":34},{"id":"tv4x_legacy_holdout_15","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.3406579176165074e-05,"mean_abs_dp":3.34065791761113e-05,"oracle_top2_margin":0.9806683876941003,"near_tie":false,"n_options":2,"probs":[0.009699212732125956,0.990300787267874],"hidden_sha256":"b4ec5dd56a509f1a44d9cd580dae7deff2d7347cf546fe74944c917b684a137a","first_in_process":false,"shard":"shard_03","index":35},{"id":"tv4x_legacy_holdout_16","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":9.28878465966676e-05,"mean_abs_dp":9.288784659664331e-05,"oracle_top2_margin":0.9767388481315413,"near_tie":false,"n_options":2,"probs":[0.9884623119123673,0.011537688087632697],"hidden_sha256":"04f3516fb32b29fa206b2ab225dc425fba0e042445c5cfad125b7c75910b0247","first_in_process":false,"shard":"shard_03","index":36},{"id":"tv4x_legacy_holdout_17","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001453276207398213,"mean_abs_dp":0.00014532762073975278,"oracle_top2_margin":0.9737978337394874,"near_tie":false,"n_options":2,"probs":[0.01295575550951656,0.9870442444904834],"hidden_sha256":"d0fce49ad90cf62c6054a806cf18c4baac0db0836154ee8482ca8fd0fe3ed5b0","first_in_process":false,"shard":"shard_03","index":37},{"id":"tv4x_legacy_holdout_18","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012317750348943557,"mean_abs_dp":0.0012317750348943123,"oracle_top2_margin":0.9502284696119041,"near_tie":false,"n_options":2,"probs":[0.9738824597710577,0.026117540228942214],"hidden_sha256":"c2789482407375b01f14c8d1957980d22d81ae4cdcb57f259eed38cdfd7e75e9","first_in_process":false,"shard":"shard_03","index":38},{"id":"tv4x_legacy_holdout_19","name":"decision","variant":"base","source":"tv4x","type":"noul","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.419474306831649e-05,"mean_abs_dp":3.4194743068253174e-05,"oracle_top2_margin":0.9806576856541841,"near_tie":false,"n_options":2,"probs":[0.00970535191597617,0.9902946480840238],"hidden_sha256":"950d53836450e868ee201a9b3b1bb4f21e3cf37ad950b1c3ee6b4f98b07be1ea","first_in_process":false,"shard":"shard_03","index":39},{"id":"tv4x_composition_holdout_00","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005806694085613295,"mean_abs_dp":0.005806694085613295,"oracle_top2_margin":0.3083575869539718,"near_tie":false,"n_options":2,"probs":[0.6599854875625992,0.34001451243740083],"hidden_sha256":"b5169c49b614c43972b923e28882b263154e1ed73733aa472dc114ae4e540415","first_in_process":true,"shard":"shard_04","index":0},{"id":"tv4x_composition_holdout_01","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":3.228283168821289e-05,"mean_abs_dp":3.228283168814697e-05,"oracle_top2_margin":0.9234987591559782,"near_tie":false,"n_options":2,"probs":[0.9617816624096773,0.03821833759032276],"hidden_sha256":"9632b77c295515df6dbda136ef2b11562901f6712804ca7b3b9102d5a551b2c8","first_in_process":false,"shard":"shard_04","index":1},{"id":"tv4x_composition_holdout_02","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005806694085613295,"mean_abs_dp":0.005806694085613295,"oracle_top2_margin":0.3083575869539718,"near_tie":false,"n_options":2,"probs":[0.6599854875625992,0.34001451243740083],"hidden_sha256":"b5169c49b614c43972b923e28882b263154e1ed73733aa472dc114ae4e540415","first_in_process":false,"shard":"shard_04","index":2},{"id":"tv4x_composition_holdout_03","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":162,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0052253077517230295,"mean_abs_dp":0.0052253077517230295,"oracle_top2_margin":0.18137856727329082,"near_tie":false,"n_options":2,"probs":[0.5959145913883684,0.40408540861163156],"hidden_sha256":"6a42ad97c74f529363c8591e2bb3581bac282d0f96739b28800a2ee57008e471","first_in_process":false,"shard":"shard_04","index":3},{"id":"tv4x_composition_holdout_04","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.422484314941995e-05,"mean_abs_dp":6.42248431494026e-05,"oracle_top2_margin":0.9085835772192891,"near_tie":false,"n_options":2,"probs":[0.045772436233504916,0.9542275637664952],"hidden_sha256":"db3596fd4676f234299bea88221d8ac1f855c6b6457107a2992945f4befb9a89","first_in_process":false,"shard":"shard_04","index":4},{"id":"tv4x_composition_holdout_05","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":154,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0001519793999603669,"mean_abs_dp":0.00015197939996031487,"oracle_top2_margin":0.9617516402681077,"near_tie":false,"n_options":2,"probs":[0.9807238407340935,0.01927615926590646],"hidden_sha256":"83f91143910479ea3a8d8db754c89b37798fa7474f306b81150ca4e73db7ed55","first_in_process":false,"shard":"shard_04","index":5},{"id":"tv4x_composition_holdout_06","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.422484314941995e-05,"mean_abs_dp":6.42248431494026e-05,"oracle_top2_margin":0.9085835772192891,"near_tie":false,"n_options":2,"probs":[0.045772436233504916,0.9542275637664952],"hidden_sha256":"db3596fd4676f234299bea88221d8ac1f855c6b6457107a2992945f4befb9a89","first_in_process":false,"shard":"shard_04","index":6},{"id":"tv4x_composition_holdout_07","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":153,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00012723110553961026,"mean_abs_dp":0.00012723110553959638,"oracle_top2_margin":0.9085517076396206,"near_tie":false,"n_options":2,"probs":[0.04585137728572938,0.9541486227142707],"hidden_sha256":"e410c422cdd60dd69eec33e2e26e8b7cd51b55e7de857fcd76223066a35764fa","first_in_process":false,"shard":"shard_04","index":7},{"id":"tv4x_composition_holdout_08","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.011441992183612415,"mean_abs_dp":0.011441992183612415,"oracle_top2_margin":0.6069969730568061,"near_tie":false,"n_options":2,"probs":[0.8149404787120155,0.18505952128798456],"hidden_sha256":"fa9c0aa2c78b2ddebef3769470c9d4a90efa14a03d4eb2f69d3cd5ada0b385e9","first_in_process":false,"shard":"shard_04","index":8},{"id":"tv4x_composition_holdout_09","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":180,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":8.875762002893625e-05,"mean_abs_dp":8.875762002892411e-05,"oracle_top2_margin":0.9425995631950593,"near_tie":false,"n_options":2,"probs":[0.9713885392175586,0.028611460782441443],"hidden_sha256":"2ad13cd7cd43b7fb23d5a66d71d89d42149ffd822d00afe65de7bb48bcd30d44","first_in_process":false,"shard":"shard_04","index":9},{"id":"tv4x_composition_holdout_10","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.011441992183612415,"mean_abs_dp":0.011441992183612415,"oracle_top2_margin":0.6069969730568061,"near_tie":false,"n_options":2,"probs":[0.8149404787120155,0.18505952128798456],"hidden_sha256":"fa9c0aa2c78b2ddebef3769470c9d4a90efa14a03d4eb2f69d3cd5ada0b385e9","first_in_process":false,"shard":"shard_04","index":10},{"id":"tv4x_composition_holdout_11","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.011678946828239473,"mean_abs_dp":0.011678946828239459,"oracle_top2_margin":0.6005328768420692,"near_tie":false,"n_options":2,"probs":[0.8119453852492741,0.18805461475072596],"hidden_sha256":"49fc9c05dc46f8f9a6176405337ba1d90e476f3517c6405405a09abd60f5951e","first_in_process":false,"shard":"shard_04","index":11},{"id":"tv4x_composition_holdout_12","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.015841057238656836,"mean_abs_dp":0.015841057238656767,"oracle_top2_margin":0.5443443270299198,"near_tie":false,"n_options":2,"probs":[0.2436688937236969,0.7563311062763032],"hidden_sha256":"720073fec42f0fe6dd8bc5161b88b2383b2cd3958c9b7c47057717a8da109c8d","first_in_process":false,"shard":"shard_04","index":12},{"id":"tv4x_composition_holdout_13","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00010550876301851364,"mean_abs_dp":0.0001055087630185015,"oracle_top2_margin":0.9560703155565883,"near_tie":false,"n_options":2,"probs":[0.02207035098472432,0.9779296490152756],"hidden_sha256":"92e72aeb185400b42f072727a0bf0f4676b65b7596b06befa4208f1264de9547","first_in_process":false,"shard":"shard_04","index":13},{"id":"tv4x_composition_holdout_14","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.015841057238656836,"mean_abs_dp":0.015841057238656767,"oracle_top2_margin":0.5443443270299198,"near_tie":false,"n_options":2,"probs":[0.2436688937236969,0.7563311062763032],"hidden_sha256":"720073fec42f0fe6dd8bc5161b88b2383b2cd3958c9b7c47057717a8da109c8d","first_in_process":false,"shard":"shard_04","index":14},{"id":"tv4x_composition_holdout_15","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":137,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.01283064758982333,"mean_abs_dp":0.012830647589823288,"oracle_top2_margin":0.5847640916754826,"near_tie":false,"n_options":2,"probs":[0.220448601752082,0.779551398247918],"hidden_sha256":"ae935fb1e8a6a5cba8d833b395461952574b3646f9fbef20c7389bf770b4361b","first_in_process":false,"shard":"shard_04","index":15},{"id":"tv4x_composition_holdout_16","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013776966796802403,"mean_abs_dp":0.0013776966796802334,"oracle_top2_margin":0.7741124000066352,"near_tie":false,"n_options":2,"probs":[0.11432149667636263,0.8856785033236373],"hidden_sha256":"6a287142855626446b735eb99317a7b9c1ad6fe9e4fd936b3f8832d0033fd77c","first_in_process":false,"shard":"shard_04","index":16},{"id":"tv4x_composition_holdout_17","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006489069495368982,"mean_abs_dp":0.0006489069495368704,"oracle_top2_margin":0.6852267962646376,"near_tie":false,"n_options":2,"probs":[0.15803550881721812,0.8419644911827819],"hidden_sha256":"42d261517e7d5e48ace2b1dfe1ff5fd0b2e9529d52c09ba0f4d606825d9a255a","first_in_process":false,"shard":"shard_04","index":17},{"id":"tv4x_composition_holdout_18","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0013776966796802403,"mean_abs_dp":0.0013776966796802334,"oracle_top2_margin":0.7741124000066352,"near_tie":false,"n_options":2,"probs":[0.11432149667636263,0.8856785033236373],"hidden_sha256":"6a287142855626446b735eb99317a7b9c1ad6fe9e4fd936b3f8832d0033fd77c","first_in_process":false,"shard":"shard_04","index":18},{"id":"tv4x_composition_holdout_19","name":"decision","variant":"base","source":"tv4x","type":"choice","tokens":166,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001415843209520276,"mean_abs_dp":0.0014158432095202692,"oracle_top2_margin":0.7745030800582808,"near_tie":false,"n_options":2,"probs":[0.11416430318037982,0.8858356968196202],"hidden_sha256":"71e578d1568b823c8bdbc570ae7d35d427f699eaea623967608f9dd9d391f9b9","first_in_process":false,"shard":"shard_04","index":19},{"id":"tv4s_00","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008809977689542725,"mean_abs_dp":0.005873318459695101,"oracle_top2_margin":0.14786413611853794,"near_tie":false,"n_options":3,"probs":[0.4144036871304449,0.5797728950483572,0.0058234178211978484],"hidden_sha256":"90c1cfa55afa198df4baa8b16920fb4f3ffd6549afb740d115727865fa707edd","first_in_process":false,"shard":"shard_04","index":20,"hidden":{"min_pos_cos":0.9996697145184893,"min_pos_cos_index":91,"max_abs_diff":0.16333898901939392,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"tv4s_01","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006414541219789127,"mean_abs_dp":0.00042763608131925895,"oracle_top2_margin":0.9603374696968201,"near_tie":false,"n_options":3,"probs":[0.01599627058766799,0.9775778467996706,0.00642588261266139],"hidden_sha256":"1d39df05a4e314b611d93021ca18297a5c97a433ab14e3fcaac4c873748b0cd0","first_in_process":false,"shard":"shard_04","index":21},{"id":"tv4s_02","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004768837566669015,"mean_abs_dp":0.0031792250444460115,"oracle_top2_margin":0.32987924818870906,"near_tie":false,"n_options":3,"probs":[0.32786904309658804,0.6672836938604509,0.004847263042961017],"hidden_sha256":"ba92aff3954bae3bacba029bf6c276fb767f78830ce89c2b27f7117cef900fe4","first_in_process":false,"shard":"shard_04","index":22},{"id":"tv4s_03","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001715784490293748,"mean_abs_dp":0.0011438563268624659,"oracle_top2_margin":0.8223114159276116,"near_tie":false,"n_options":3,"probs":[0.003493025062397573,0.9077071529911426,0.08879982194645973],"hidden_sha256":"c120203ee4222cedc6a41900b51c34432bc0f41826479d7e3aa840b6404fad42","first_in_process":false,"shard":"shard_04","index":23},{"id":"tv4s_04","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008112631733705883,"mean_abs_dp":0.005408421155803959,"oracle_top2_margin":0.48158885519616973,"near_tie":false,"n_options":3,"probs":[0.24663390783000835,0.7443061915359952,0.009059900633996438],"hidden_sha256":"5fca19510d531bc9e92b7df6b492e2fad3a3a129c27e718c21476d8bd70a9cca","first_in_process":false,"shard":"shard_04","index":24},{"id":"tv4s_05","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00014881618451811907,"mean_abs_dp":9.921078967874807e-05,"oracle_top2_margin":0.9705965340279149,"near_tie":false,"n_options":3,"probs":[0.005817056463643881,0.9822576713288415,0.011925272207514668],"hidden_sha256":"c625af361cf83909ac1ff9cbc77f7bd3c7fdce52820c101ff6c56734ed2c17da","first_in_process":false,"shard":"shard_04","index":25},{"id":"tv4s_06","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004852940907309644,"mean_abs_dp":0.003235293938206447,"oracle_top2_margin":0.11280309184966997,"near_tie":false,"n_options":3,"probs":[0.4346940478778272,0.5570128395611968,0.008293112560975972],"hidden_sha256":"6cf866e2d75c68330113f300ebf5a9b6acdf54d8caa68fbb73ef45611b098e91","first_in_process":false,"shard":"shard_04","index":26},{"id":"tv4s_07","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0010917620295595037,"mean_abs_dp":0.0007278413530396552,"oracle_top2_margin":0.7816282229653853,"near_tie":false,"n_options":3,"probs":[0.04303723127672653,0.8702511699381328,0.08671159878514063],"hidden_sha256":"902b4ef350ed33513948d472b4864045acbeb0015464a815071b74229b78089d","first_in_process":false,"shard":"shard_04","index":27},{"id":"tv4s_08","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0014675499256652236,"mean_abs_dp":0.0009783666171101126,"oracle_top2_margin":0.7841390923197227,"near_tie":false,"n_options":3,"probs":[0.1021128981557456,0.8891385067883983,0.008748595055856104],"hidden_sha256":"3429b05848cb60918a14f9840a87799ec6353a528746dec03d44695928a7d55c","first_in_process":false,"shard":"shard_04","index":28},{"id":"tv4s_09","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.456330421105271e-05,"mean_abs_dp":3.637553614067752e-05,"oracle_top2_margin":0.9808333378930814,"near_tie":false,"n_options":3,"probs":[0.005686260079305114,0.9876202338723779,0.006693506048317078],"hidden_sha256":"502de3e254fd601d4f1499b910874e726f207f5eec8ad7feaed2fa3b0ac78b65","first_in_process":false,"shard":"shard_04","index":29},{"id":"tv4s_10","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004333524875561934,"mean_abs_dp":0.002889016583707982,"oracle_top2_margin":0.4147412288568358,"near_tie":false,"n_options":3,"probs":[0.2819672053411038,0.7052459931251085,0.012786801533787749],"hidden_sha256":"e8a33a73fa449a1bdd479ccbdb35f0841fd7a4a9344006c77491f39faf553db2","first_in_process":false,"shard":"shard_04","index":30},{"id":"tv4s_11","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008739359512600686,"mean_abs_dp":0.00582623967506712,"oracle_top2_margin":0.5728354456080552,"near_tie":false,"n_options":3,"probs":[0.015641599142791993,0.7872867651206703,0.1970716357365377],"hidden_sha256":"72ff578e9e73db5b30f6c131bf3dd91396a4824fa625426a4812d242e47f5b92","first_in_process":false,"shard":"shard_04","index":31},{"id":"tv4s_12","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00792337653557007,"mean_abs_dp":0.005282251023713372,"oracle_top2_margin":0.6046066435315647,"near_tie":false,"n_options":3,"probs":[0.7929671527986617,0.20416717598119544,0.002865671220142823],"hidden_sha256":"bd205ef245cd67aaaf1192d7e30446bfd9d327e984a12006ccce2f9af48ce016","first_in_process":false,"shard":"shard_04","index":32},{"id":"tv4s_13","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":122,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004497838169964824,"mean_abs_dp":0.0029985587799765726,"oracle_top2_margin":0.8289862535749726,"near_tie":false,"n_options":3,"probs":[0.0052434822570418165,0.9163195941514628,0.07843692359149543],"hidden_sha256":"33b68cb75f839bcc4f55a8ee6f7023dbafd16ab9a4dd1f3570a9e170c9a5111a","first_in_process":false,"shard":"shard_04","index":33},{"id":"tv4s_14","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000490645403502632,"mean_abs_dp":0.00032709693566847914,"oracle_top2_margin":0.9642054851284572,"near_tie":false,"n_options":3,"probs":[0.012005090135500534,0.9770665680398217,0.010928341824677704],"hidden_sha256":"45281a94e9984fbc701b7f44ba4a3668935faae8b828ba462ea2c309899d6011","first_in_process":false,"shard":"shard_04","index":34},{"id":"tv4s_15","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":123,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00040182665453973587,"mean_abs_dp":0.00026788443635984704,"oracle_top2_margin":0.9505295735609528,"near_tie":false,"n_options":3,"probs":[0.016083656518152422,0.9669917489566331,0.016924594525214436],"hidden_sha256":"87efcddfbb73f0f24bd19cb4b92e644179abfb54d61271e66c326cddfade5e4a","first_in_process":false,"shard":"shard_04","index":35},{"id":"tv4s_16","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0027943402251338623,"mean_abs_dp":0.0018628934834226112,"oracle_top2_margin":0.13584732588717008,"near_tie":false,"n_options":3,"probs":[0.42741944125826364,0.5688428615861941,0.00373769715554229],"hidden_sha256":"e5cb8a6daea7bd8a3f302af058542f9db36a5f936cfcce103a1327ac264ca30c","first_in_process":false,"shard":"shard_04","index":36},{"id":"tv4s_17","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00018871089211916752,"mean_abs_dp":0.00012580726141282373,"oracle_top2_margin":0.9172811742066569,"near_tie":false,"n_options":3,"probs":[0.007057819893754109,0.9551787683211509,0.03776341178509498],"hidden_sha256":"b5edafc8eff58c90fc27b76d08683eb5da6b49e1ede141aeca6c1ea35b3202d5","first_in_process":false,"shard":"shard_04","index":37},{"id":"tv4s_18","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0012127318630387185,"mean_abs_dp":0.0008084879086924828,"oracle_top2_margin":0.9227063857483483,"near_tie":false,"n_options":3,"probs":[0.9588965729619778,0.03858895423081402,0.0025144728072081326],"hidden_sha256":"24a1e89b3d90ace3e0a6120ba2832c5e82ecc95bdd52c31abe3be5d15d3c18e3","first_in_process":false,"shard":"shard_04","index":38},{"id":"tv4s_19","name":"decision","variant":"base","source":"tv4s","type":"score","tokens":121,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001229107634230786,"mean_abs_dp":8.194050894870144e-05,"oracle_top2_margin":0.9819012797443343,"near_tie":false,"n_options":3,"probs":[0.0065705229127393265,0.9886091098582808,0.004820367228979905],"hidden_sha256":"b793e320ec5d05316474e93b6d02d0f73f5ab4306b4a2353c9d102268cb6c015","first_in_process":false,"shard":"shard_04","index":39},{"id":"semif_a3f18f3a63d45345942b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00016843437373637027,"mean_abs_dp":0.00011228958249090311,"oracle_top2_margin":0.9357222628286423,"near_tie":false,"n_options":3,"probs":[0.012088734354520582,0.025948674535824997,0.9619625911096544],"hidden_sha256":"e868b90f7ae7e554539fa902dbf37c7727c69cf2117728a97cf5df5a498c0f7f","first_in_process":true,"shard":"shard_05","index":0,"hidden":{"min_pos_cos":0.9998259740136523,"min_pos_cos_index":1,"max_abs_diff":0.13752365112304688,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"semif_f40beba9088c8db8bbd6","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00034510490118144777,"mean_abs_dp":0.00023006993412098253,"oracle_top2_margin":0.9328238790278026,"near_tie":false,"n_options":3,"probs":[0.9610653052224822,0.02760213876800682,0.011332556009510957],"hidden_sha256":"3528f43df629b1e874fd12ab778a25705f19845748a1f7bd8d712407a2aa798e","first_in_process":false,"shard":"shard_05","index":1},{"id":"semif_0b43ea8e24e74d621f1b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.507864879487737e-05,"mean_abs_dp":6.338576586327992e-05,"oracle_top2_margin":0.9216656021196827,"near_tie":false,"n_options":3,"probs":[0.03325470136169396,0.9550748562447832,0.011670442393522872],"hidden_sha256":"28c0dcdd954cfd122a34057a525d39b89ff378846431fef6d5a937cd801fa367","first_in_process":false,"shard":"shard_05","index":2},{"id":"semif_942ec65dac0bfb9874f9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001792012578001434,"mean_abs_dp":0.0011946750520009433,"oracle_top2_margin":0.7123127373277265,"near_tie":false,"n_options":3,"probs":[0.8458032238266677,0.02427454691089699,0.1299222292624353],"hidden_sha256":"1d7cf48720a502dbd7370a996d153614f8dab6d72b8a51c14d9a056765b71c4b","first_in_process":false,"shard":"shard_05","index":3},{"id":"semif_e0c140e9222d1bce5647","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008709690673850412,"mean_abs_dp":0.0005806460449233504,"oracle_top2_margin":0.9203280311161692,"near_tie":false,"n_options":3,"probs":[0.954118662116462,0.013638012537630705,0.032243325345907375],"hidden_sha256":"2ac3fd7eae6048cfa8c281ffaab36047b265328d764da380af8b67171a36a98f","first_in_process":false,"shard":"shard_05","index":4},{"id":"semif_8e8c3804a3c15ebb31e3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.009250889250971839,"mean_abs_dp":0.006167259500647862,"oracle_top2_margin":0.23989506177672987,"near_tie":false,"n_options":3,"probs":[0.5913898328124007,0.039376378912920985,0.3692337882746784],"hidden_sha256":"c72c729ab4b231fb3310d1ed216db7ce2ad6f17557182283a65ca5635eb83227","first_in_process":false,"shard":"shard_05","index":5},{"id":"semif_56c419b12cd37cbe790c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008931712445379159,"mean_abs_dp":0.0005954474963586406,"oracle_top2_margin":0.8988903795476691,"near_tie":false,"n_options":3,"probs":[0.934840996388888,0.030604174676069008,0.03455482893504308],"hidden_sha256":"90ac045d6364479b464e723e54fe84bda5857e61f89489026b4c286bcb96c355","first_in_process":false,"shard":"shard_05","index":6},{"id":"semif_75b470586834df8c568d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002678483606260995,"mean_abs_dp":0.00017856557375078735,"oracle_top2_margin":0.6736416663487602,"near_tie":false,"n_options":3,"probs":[0.01875486049858581,0.15354829189512353,0.8276968476062906],"hidden_sha256":"814e4d8ce98f9e42d83e0b53bdf02d94fd8aecb1cf8da046a3916f7a4d72dfc7","first_in_process":false,"shard":"shard_05","index":7},{"id":"semif_834f3b3e10cc61b33d25","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":2.3981624050166753e-05,"mean_abs_dp":1.5987749366773787e-05,"oracle_top2_margin":0.9604540417401267,"near_tie":false,"n_options":3,"probs":[0.010223952415613824,0.014639896449748748,0.9751361511346374],"hidden_sha256":"0e7319518d8bfff39eabd16f137f4edf60be9667a2268193091b9a59230136f5","first_in_process":false,"shard":"shard_05","index":8},{"id":"semif_76c7a79b1ff2363a60c5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.771377474295125e-05,"mean_abs_dp":3.847584982863995e-05,"oracle_top2_margin":0.9636078315972091,"near_tie":false,"n_options":3,"probs":[0.9774320748190435,0.01372539246785622,0.008842532713100258],"hidden_sha256":"5747d1285d7b5b08e627c05d392977631475634dab3f360f0667dca9aedd0892","first_in_process":false,"shard":"shard_05","index":9},{"id":"semif_36c71bb649b7d5eccb1f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.913867004716302e-05,"mean_abs_dp":5.2759113364749324e-05,"oracle_top2_margin":0.9650002382306553,"near_tie":false,"n_options":3,"probs":[0.9781849017172877,0.013043986182379659,0.00877111210033265],"hidden_sha256":"640d59e9bb26c2b7828eb6d6aea109e9128bbe9dd1eaf16a16fd811c5cee884b","first_in_process":false,"shard":"shard_05","index":10},{"id":"semif_eac5121a4a1bf391a08c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":64,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008210893895767013,"mean_abs_dp":0.0005473929263844976,"oracle_top2_margin":0.8647753102944388,"near_tie":false,"n_options":3,"probs":[0.04850213935801705,0.9120093882897219,0.03948847235226115],"hidden_sha256":"365ea1a598194ecbaabf969a30c43aeb1dc2a7b70d6461b68e17866e2c44b03d","first_in_process":false,"shard":"shard_05","index":11},{"id":"semif_44574c44e243ee1b2f52","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":7.330236617957009e-05,"mean_abs_dp":4.8868244119741146e-05,"oracle_top2_margin":0.9659716866106711,"near_tie":false,"n_options":3,"probs":[0.010340237126683149,0.011773029769177839,0.9778867331041391],"hidden_sha256":"55e171eaa707061f3c1e3293a7181bcd5fb49cab934c08ddab3a73324a3131d0","first_in_process":false,"shard":"shard_05","index":12},{"id":"semif_2608370d01783735c3ea","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00015989441144992345,"mean_abs_dp":0.00010659627429988304,"oracle_top2_margin":0.9215895668087061,"near_tie":false,"n_options":3,"probs":[0.020718361115308535,0.028971966775890222,0.9503096721088011],"hidden_sha256":"94f54fa8075d92ce91a40bf80da24386a33d5dd9b46cba7fc6b138bf83006478","first_in_process":false,"shard":"shard_05","index":13},{"id":"semif_ec94ef909f7918695ab3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013993643089349472,"mean_abs_dp":9.329095392895832e-05,"oracle_top2_margin":0.9318247409749322,"near_tie":false,"n_options":3,"probs":[0.023653779663282755,0.02071839578948075,0.9556278245472366],"hidden_sha256":"ceacd76ecbd0b088dd1f3394c1ddd3e837a64da8a82f9bd347271b56840957ac","first_in_process":false,"shard":"shard_05","index":14},{"id":"semif_03d919fd0e9b3a7cd1e6","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018232973022648188,"mean_abs_dp":0.00012155315348431316,"oracle_top2_margin":0.8733675366733726,"near_tie":false,"n_options":3,"probs":[0.9246157223789596,0.02400159127920298,0.05138268634183735],"hidden_sha256":"809b8d63fe8cb282ecb43cd3e50b7b99ccee45b6b5ef6871d687621f12af4deb","first_in_process":false,"shard":"shard_05","index":15},{"id":"semif_86a15618ba4cb90a307e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002594300152818363,"mean_abs_dp":0.00017295334352121264,"oracle_top2_margin":0.9348419215531181,"near_tie":false,"n_options":3,"probs":[0.9584038414271839,0.02309219629769674,0.018503962275119336],"hidden_sha256":"bae37c45f1ccdd1991fc8d5b8812bf8cbef4678b96c55325afc1e3c2120de8aa","first_in_process":false,"shard":"shard_05","index":16},{"id":"semif_3dd9c3ac98f8ddf7bdd3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0002510366896911753,"mean_abs_dp":0.00016735779312744675,"oracle_top2_margin":0.9457231579098606,"near_tie":false,"n_options":3,"probs":[0.019044086560965956,0.9643444249467233,0.016611488492310676],"hidden_sha256":"cf9f8152825be3fad14c86e8344a1302e2255ba63f8f13a45f0e12466796ae07","first_in_process":false,"shard":"shard_05","index":17},{"id":"semif_36273dd4777876a26338","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001880669698671733,"mean_abs_dp":0.00012537797991147027,"oracle_top2_margin":0.9446003626832611,"near_tie":false,"n_options":3,"probs":[0.010698052049047294,0.9671023321210545,0.02219961582989824],"hidden_sha256":"ecbb0d4905abefc83f255b5297b4b441c1db7e9301abca95566771fe55d66477","first_in_process":false,"shard":"shard_05","index":18},{"id":"semif_07ee70cfe4b01330ce89","name":"answer","variant":"base","source":"semif","type":"choice","tokens":66,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0073674564269810006,"mean_abs_dp":0.004911637617987366,"oracle_top2_margin":0.5487147306168284,"near_tie":false,"n_options":3,"probs":[0.02673355372297131,0.20500130851737922,0.7682651377596494],"hidden_sha256":"9c86e8d7d5f92da8b8165b2a3c9c46f063d81a6f40b52a386e20e17bbb66cf42","first_in_process":false,"shard":"shard_05","index":19},{"id":"semif_51ceab3ce622c7c759f1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00017998158290843946,"mean_abs_dp":0.00011998772193893305,"oracle_top2_margin":0.9406567372660729,"near_tie":false,"n_options":3,"probs":[0.016069274008645107,0.02149166720517923,0.9624390587861757],"hidden_sha256":"fa872c08dc06ee04ff24ade136d36ac19391ddbbdebe57f14811a2df8b5f5e2d","first_in_process":false,"shard":"shard_05","index":20},{"id":"semif_b8091be929b1e489fcc4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00035349124747419136,"mean_abs_dp":0.00023566083164948173,"oracle_top2_margin":0.9159577193463836,"near_tie":false,"n_options":3,"probs":[0.013806200322000773,0.95136584863658,0.03482795104141918],"hidden_sha256":"86d7e5a24c5a851eb0ea805539e3d493a8c24ddbc9ace68cba61f3f9270689b7","first_in_process":false,"shard":"shard_05","index":21},{"id":"semif_c89b86e3f55123bfa506","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004741731842643482,"mean_abs_dp":0.0003161154561762437,"oracle_top2_margin":0.9192964850975687,"near_tie":false,"n_options":3,"probs":[0.024373668946487514,0.9477413558005783,0.027884975252934138],"hidden_sha256":"49b5e848e81fca96c9e872bfcc76038fc9a4c4b32329869af5a3c40d48ea6b8b","first_in_process":false,"shard":"shard_05","index":22},{"id":"semif_3b0958cabc339005cf66","name":"answer","variant":"base","source":"semif","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002769750120430181,"mean_abs_dp":0.0001846500080286186,"oracle_top2_margin":0.8631264727730906,"near_tie":false,"n_options":3,"probs":[0.9118484781881455,0.04841126136820957,0.039740260443644815],"hidden_sha256":"34d0b7f283f154eb39b82cd0da1d89f903c345d086c7b7a1ac038b6ff7d1b574","first_in_process":false,"shard":"shard_05","index":23},{"id":"semif_f2b4ec4930322fe45118","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009072286307142408,"mean_abs_dp":0.0006048190871427982,"oracle_top2_margin":0.9040631902962722,"near_tie":false,"n_options":3,"probs":[0.03856382922985258,0.944228688838642,0.017207481931505446],"hidden_sha256":"f6015b1b455e759c280820283d00dacef0ad6a47a40db4164c9cd6e66f3a1e40","first_in_process":false,"shard":"shard_05","index":24},{"id":"semif_61ac20b07a5189231e15","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0001131491655539385,"mean_abs_dp":7.543277703590985e-05,"oracle_top2_margin":0.9283512811406358,"near_tie":false,"n_options":3,"probs":[0.015289691915970577,0.956440511036401,0.02826979704762834],"hidden_sha256":"d202d454f39a98f6c28bd4049c460ebb0981c7192028d7db657aa2819c55ae8a","first_in_process":false,"shard":"shard_05","index":25},{"id":"semif_4b3d794c004566610181","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00035913239742731834,"mean_abs_dp":0.00023942159828492285,"oracle_top2_margin":0.8413288291611345,"near_tie":false,"n_options":3,"probs":[0.0364448193834514,0.0610670278582172,0.9024881527583314],"hidden_sha256":"239874c9a6c8e9b6352df89434e5afe59d38b2e60f1b141a3e487cc0b9f1f16e","first_in_process":false,"shard":"shard_05","index":26},{"id":"semif_b7c4e62d07986286fce0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0035127010833623518,"mean_abs_dp":0.0023418007222415493,"oracle_top2_margin":0.4801013037391402,"near_tie":false,"n_options":3,"probs":[0.6698201637993195,0.13360568869502976,0.19657414750565075],"hidden_sha256":"28b330896a07f8c7ca2e0a646681c22a14c68a043a65cc77c0999c98e80e5036","first_in_process":false,"shard":"shard_05","index":27},{"id":"semif_9473603b44af63beab79","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00037143791876426135,"mean_abs_dp":0.00024762527917617075,"oracle_top2_margin":0.7943738126119272,"near_tie":false,"n_options":3,"probs":[0.8878601621879731,0.09410224702710901,0.018037590784917762],"hidden_sha256":"fd87f0b897e4ac02b37fe274fd0157f747e6d196d5edde5fd6183a728e6bed38","first_in_process":false,"shard":"shard_05","index":28},{"id":"semif_d41072dc35d8cd14c056","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005556818063850999,"mean_abs_dp":0.00037045453759006544,"oracle_top2_margin":0.8433094179932079,"near_tie":false,"n_options":3,"probs":[0.031135272395705873,0.9064637895475405,0.06240093805675359],"hidden_sha256":"1e961f93c0a40861929fede9f7ba213f20ce710d0516e8950a96e91e6bf00569","first_in_process":false,"shard":"shard_05","index":29},{"id":"semif_33d6e5da58f0fee2490d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.019660274378349274,"mean_abs_dp":0.013106849585566155,"oracle_top2_margin":0.2551036769378849,"near_tie":false,"n_options":3,"probs":[0.5676886833789831,0.2839410946029395,0.1483702220180774],"hidden_sha256":"6fdc7fa60d64142377d57e40724305d6a9d8b1428aeb560c0812c97044b5dfbc","first_in_process":false,"shard":"shard_05","index":30},{"id":"semif_823bf736b92e9d8fa66d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004206107808862769,"mean_abs_dp":0.0028040718725751887,"oracle_top2_margin":0.11576403794400497,"near_tie":false,"n_options":3,"probs":[0.07860345087255084,0.4069925016685935,0.5144040474588557],"hidden_sha256":"b3ac7faf5e66cae661472d881a5e1c630060866a3d80bad7257e8325fb3ce557","first_in_process":false,"shard":"shard_05","index":31},{"id":"semif_8a4c3de28be95c105ca1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.000530116294816918,"mean_abs_dp":0.0003534108632112347,"oracle_top2_margin":0.8881225713951193,"near_tie":false,"n_options":3,"probs":[0.014308595891786744,0.04827051453032559,0.9374208895778877],"hidden_sha256":"d15a969037bf6e230e983a3966f77fe50790ebdcbec72c20c99054011d62a61d","first_in_process":false,"shard":"shard_05","index":32},{"id":"semif_ea7ec66a5e8eac886548","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009172888302788618,"mean_abs_dp":0.0006115258868525352,"oracle_top2_margin":0.7170225342159673,"near_tie":false,"n_options":3,"probs":[0.836538411217731,0.11777768532979706,0.04568390345247201],"hidden_sha256":"116d5dd3e678ca7fc10d06aa8443f94537067d29ef618b8874664349c1772437","first_in_process":false,"shard":"shard_05","index":33},{"id":"semif_25172cd6ddc49ab0ca70","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004858814824450941,"mean_abs_dp":0.003239209882967329,"oracle_top2_margin":0.7725567740197001,"near_tie":false,"n_options":3,"probs":[0.05355539114282612,0.0830672646110024,0.8633773442461714],"hidden_sha256":"1e75f59a441cba4ebfdb1c5c1296d06f05452271346f94acf65c6ef3dcf2c003","first_in_process":false,"shard":"shard_05","index":34},{"id":"semif_fb73a85d386cf59b0c51","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0012075841171805823,"mean_abs_dp":0.0008050560781203246,"oracle_top2_margin":0.41852063761693903,"near_tie":false,"n_options":3,"probs":[0.2782364445807105,0.02265293727683437,0.6991106181424552],"hidden_sha256":"02062761f31a162838e869a661f2f57a091e65eb7ca72c0e269be0480472adcc","first_in_process":false,"shard":"shard_05","index":35},{"id":"semif_a02fd582d2881cc462b2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002448668719982061,"mean_abs_dp":0.0001632445813320952,"oracle_top2_margin":0.9347901747190903,"near_tie":false,"n_options":3,"probs":[0.014362666057103209,0.025179470786589636,0.9604578631563072],"hidden_sha256":"e8009c8e205c4ffc2d5b1180d8ee0014341bc853b3a8b2b4b74800fb2022b5d4","first_in_process":false,"shard":"shard_05","index":36},{"id":"semif_19f37d5a97034bd7d04f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004780516559276249,"mean_abs_dp":0.0003187011039517222,"oracle_top2_margin":0.9070000108695007,"near_tie":false,"n_options":3,"probs":[0.03724991586562452,0.9450736099632794,0.017676474171096152],"hidden_sha256":"49d31bd17b44c5070dc1c41d85786e6174c51a1adaa19a844d75ccb0c249175e","first_in_process":false,"shard":"shard_05","index":37},{"id":"semif_fda0ca94652c6d739ecf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003976822851722017,"mean_abs_dp":0.0002651215234481877,"oracle_top2_margin":0.8966135104773004,"near_tie":false,"n_options":3,"probs":[0.9382631742626432,0.020881114300138105,0.0408557114372188],"hidden_sha256":"0c7b3c1a91fc3f86622127eb14108668c81bbbed8682323b71af3b646dcbc7f6","first_in_process":false,"shard":"shard_05","index":38},{"id":"semif_d0e36c4cb10a29b3191c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009805533412050038,"mean_abs_dp":0.0006537022274700072,"oracle_top2_margin":0.15133175652298392,"near_tie":false,"n_options":3,"probs":[0.40434558671930426,0.038147231148936936,0.5575071821317588],"hidden_sha256":"27db62f775dc46e457f914bbe31a4333e0cf45565841a22a87f6513000a99d52","first_in_process":false,"shard":"shard_05","index":39},{"id":"semif_a0a39ace7e5337d038a7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00028333257178303484,"mean_abs_dp":0.00018888838118866504,"oracle_top2_margin":0.9627691549734159,"near_tie":false,"n_options":3,"probs":[0.9769324098817873,0.009405666915838877,0.01366192320237388],"hidden_sha256":"30d331714c3ac8713493133cc0c20a631165a1df64ef4a6925040e0dfe7b7e7d","first_in_process":true,"shard":"shard_06","index":0},{"id":"semif_eeae283841ce75dd25bf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.242261023050368e-05,"mean_abs_dp":5.494840682032711e-05,"oracle_top2_margin":0.9621668481066167,"near_tie":false,"n_options":3,"probs":[0.01367421568310739,0.9757093294042103,0.010616454912682262],"hidden_sha256":"07e0bddc05929e4123d6d3db8b9a61c078ae7eeb75c1b4f7e94142f0bed49bcf","first_in_process":false,"shard":"shard_06","index":1},{"id":"semif_00bb4cc043f7d2b288d4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00013606786301501028,"mean_abs_dp":9.071190867668566e-05,"oracle_top2_margin":0.9653603614640307,"near_tie":false,"n_options":3,"probs":[0.012808548190575761,0.008789035574641285,0.978402416234783],"hidden_sha256":"a7ccc564344b682605bd66833c7fb9f3ed0570af164a4e0a071192a865bcf40b","first_in_process":false,"shard":"shard_06","index":2},{"id":"semif_1a81ead69e93e5d0c86a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006035034051619037,"mean_abs_dp":0.004023356034412705,"oracle_top2_margin":0.2893138199873723,"near_tie":false,"n_options":3,"probs":[0.0877345094747073,0.316846873157966,0.5954186173673267],"hidden_sha256":"bfd291f425d0e9d02fc8047d8920ab990e67e94eebaf7b568202b8c9dc04afbc","first_in_process":false,"shard":"shard_06","index":3},{"id":"semif_4444dad25e438f128673","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.01043015323648433,"mean_abs_dp":0.006953435490989544,"oracle_top2_margin":0.15678293022597928,"near_tie":false,"n_options":3,"probs":[0.36821249060250666,0.12587913423021901,0.5059083751672744],"hidden_sha256":"dd892a78095335a40f92d9f32506104e8ef57075e235d280853214d8a0c51c72","first_in_process":false,"shard":"shard_06","index":4},{"id":"semif_3226202ba1ada2570e9f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0017959776073845912,"mean_abs_dp":0.0011973184049230794,"oracle_top2_margin":0.23422510399500152,"near_tie":false,"n_options":3,"probs":[0.5380512957578316,0.1608776464676023,0.3010710577745661],"hidden_sha256":"9c0406855ae07dc88eed727e262fb2bfc86534ec0ae84423153dc18968e28f16","first_in_process":false,"shard":"shard_06","index":5},{"id":"semif_3f783eec5c90812479bc","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0074621410701543045,"mean_abs_dp":0.004974760713436194,"oracle_top2_margin":0.5643859812186472,"near_tie":false,"n_options":3,"probs":[0.04855018020035195,0.7650614540416181,0.18638836575803006],"hidden_sha256":"926ad73baa1767c253aed124282393d8f29b53e3b9831df07a51c2c406318f16","first_in_process":false,"shard":"shard_06","index":6},{"id":"semif_ffee861474a9665a093a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00417639533378189,"mean_abs_dp":0.0027842635558546123,"oracle_top2_margin":0.06305327507600167,"near_tie":false,"n_options":3,"probs":[0.48856747949365603,0.0931446950334696,0.4182878254728745],"hidden_sha256":"d6990368e6e6ab3a6afd444e2f0887539b1538b1b21a712de14595c5fbac8990","first_in_process":false,"shard":"shard_06","index":7},{"id":"semif_8c992e17e4fb54b044f5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":8.597485800365678e-05,"mean_abs_dp":5.731657200244884e-05,"oracle_top2_margin":0.950739141460235,"near_tie":false,"n_options":3,"probs":[0.012455081154695295,0.969096569924361,0.018448348920943724],"hidden_sha256":"93e7349a8ca919f11b8e184b147d9a4c0cfd809d28a9b502e1cdab56740ba22c","first_in_process":false,"shard":"shard_06","index":8},{"id":"semif_695dd61195ca025c4f4c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":76,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002730090642422134,"mean_abs_dp":0.0001820060428281648,"oracle_top2_margin":0.9013869599755495,"near_tie":false,"n_options":3,"probs":[0.9458685938333279,0.04497408250555635,0.009157323661115845],"hidden_sha256":"4ab1e8767ec875de96c91c067d9e11f60d904f67ac0fb7c0cbf72ddfd6b26660","first_in_process":false,"shard":"shard_06","index":9},{"id":"semif_e6155431ab7b0f8170d4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0009128269983050274,"mean_abs_dp":0.0006085513322033609,"oracle_top2_margin":0.3056350596030686,"near_tie":false,"n_options":3,"probs":[0.2558640217155707,0.5628851994143266,0.18125077887010269],"hidden_sha256":"716f77adeba165d8b19dbe043e44e13460bd0b1c7d40dec03baf62cd5036bc44","first_in_process":false,"shard":"shard_06","index":10},{"id":"semif_b49ecda75086d8c68d2a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00025870488128520996,"mean_abs_dp":0.00017246992085682053,"oracle_top2_margin":0.4729230733547011,"near_tie":false,"n_options":3,"probs":[0.0703833994195136,0.2281025840251839,0.7015140165553025],"hidden_sha256":"61b03907160c0d3db59d1839aecb45fbed7cda78b82d938d9d448dd29ec41e68","first_in_process":false,"shard":"shard_06","index":11},{"id":"semif_587ced261c6092cd1325","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0072206492039775805,"mean_abs_dp":0.004813766135985077,"oracle_top2_margin":0.7011503759417472,"near_tie":false,"n_options":3,"probs":[0.12532066842713568,0.8399480756149412,0.03473125595792315],"hidden_sha256":"e6736ea784056f137a7ebf4540bb53916c0e4043d3d3b3c5b84e01bd18a77f9a","first_in_process":false,"shard":"shard_06","index":12},{"id":"semif_533d4423d311de82b27d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.9028680209833698e-05,"mean_abs_dp":1.2685786806510118e-05,"oracle_top2_margin":0.9438360499050665,"near_tie":false,"n_options":3,"probs":[0.023707434745758555,0.9675629434645502,0.008729621789691208],"hidden_sha256":"eb143effdee26d5f33d7366c7a9a1411507b10940bd9b5ac04c74f5666e4b2c5","first_in_process":false,"shard":"shard_06","index":13},{"id":"semif_2bf540d515a51755b25c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004177236687536778,"mean_abs_dp":0.0027848244583578894,"oracle_top2_margin":0.5672376377073012,"near_tie":false,"n_options":3,"probs":[0.18054940702192795,0.06491450355594379,0.7545360894221282],"hidden_sha256":"f616be78397e762db3a8306e81ca1db592c1ef224a24c7d555e85c1c790fc1f0","first_in_process":false,"shard":"shard_06","index":14},{"id":"semif_fdb9fc214aea2710763a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":65,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0014601616255731087,"mean_abs_dp":0.0009734410837154012,"oracle_top2_margin":0.3787248844675741,"near_tie":false,"n_options":3,"probs":[0.10622223659367531,0.25666628882026155,0.6371114745860631],"hidden_sha256":"4524a92bc49ea976538b229721b07078fa8bb9c679a4aba2ef781dc754d0a5db","first_in_process":false,"shard":"shard_06","index":15},{"id":"semif_f6d68126b43f8c9ed183","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.877090949557185e-05,"mean_abs_dp":4.584727299705831e-05,"oracle_top2_margin":0.9339226656619631,"near_tie":false,"n_options":3,"probs":[0.026810677423715068,0.9608290043568014,0.012360318219483484],"hidden_sha256":"0f0e79b5534eacf9ab91ea0ef0db44320ac4b16a73a66ca81d86bd02ba622cfe","first_in_process":false,"shard":"shard_06","index":16},{"id":"semif_44bae0c8a1615a096c43","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0010884342402149166,"mean_abs_dp":0.0007256228268099155,"oracle_top2_margin":0.8186553468489266,"near_tie":false,"n_options":3,"probs":[0.8942240696032184,0.028074507067078147,0.07770142332970338],"hidden_sha256":"35b3645671e28de93434a76004bfaebd1c6ae14123de0745d32ca793805ecaed","first_in_process":false,"shard":"shard_06","index":17},{"id":"semif_2175379f4c69e6207626","name":"answer","variant":"base","source":"semif","type":"choice","tokens":72,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005117459359435972,"mean_abs_dp":0.0034116395729573193,"oracle_top2_margin":0.30882585389383266,"near_tie":false,"n_options":3,"probs":[0.5908041797592872,0.28990336098208785,0.11929245925862499],"hidden_sha256":"9d841f4c9870c2b1a88b6309bfd6d8f06ef8cf38d1b4610c7d094df7a1a5fdc2","first_in_process":false,"shard":"shard_06","index":18},{"id":"semif_def9fe59d262357a673a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0005106217414568293,"mean_abs_dp":0.0003404144943045621,"oracle_top2_margin":0.02681563599995257,"near_tie":false,"n_options":3,"probs":[0.42133201540700665,0.447414307334959,0.13125367725803433],"hidden_sha256":"3b0e3cf8e20973031ad1584e82e9bf18a1bd77eca3d9a4c8fbd8456235cb42b8","first_in_process":false,"shard":"shard_06","index":19},{"id":"semif_712f6ffb763eef7c9a12","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0011991582638044562,"mean_abs_dp":0.0007994388425363504,"oracle_top2_margin":0.4567225519567792,"near_tie":false,"n_options":3,"probs":[0.19361043059200228,0.15399312636551463,0.652396443042483],"hidden_sha256":"9796a77c638f9f469f7a80168af3a22a68c47c054bab0bebd67f587a7826517c","first_in_process":false,"shard":"shard_06","index":20},{"id":"semif_e1d610bd14e3d16c09b9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":78,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0024000592811285193,"mean_abs_dp":0.0016000395207522871,"oracle_top2_margin":0.7417275778640565,"near_tie":false,"n_options":3,"probs":[0.8556291261009532,0.11827257847206031,0.026098295426986363],"hidden_sha256":"2a93ca5fa134bd22d2a16241115bcd764467dd9e714d4c8a248028bd247c64c9","first_in_process":false,"shard":"shard_06","index":21},{"id":"semif_22ec57822db50ec121cf","name":"answer","variant":"base","source":"semif","type":"choice","tokens":68,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":3.9266642816504493e-05,"mean_abs_dp":2.6177761877702044e-05,"oracle_top2_margin":0.9450968055693952,"near_tie":false,"n_options":3,"probs":[0.02249645730015372,0.9676610444864154,0.009842498213430776],"hidden_sha256":"39cd7a86a41d42ae5a6e9e73b35e452134dae96dcd5b12b90f5b26a3d2af861c","first_in_process":false,"shard":"shard_06","index":22},{"id":"semif_3e75f2b84e8e08e3d3f2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009761652498094797,"mean_abs_dp":0.0006507768332063198,"oracle_top2_margin":0.7381421705328148,"near_tie":false,"n_options":3,"probs":[0.09668049134089175,0.06708367979150935,0.8362358288675988],"hidden_sha256":"c0da2e7996ba10cbc7845aa3aaeba9cb9383247fec1457e918ddc339b84ab54b","first_in_process":false,"shard":"shard_06","index":23},{"id":"semif_ebd74aa05324a1556288","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":9.342452154546821e-05,"mean_abs_dp":6.228301436366397e-05,"oracle_top2_margin":0.9259538655050306,"near_tie":false,"n_options":3,"probs":[0.031521491788831284,0.9576147326417164,0.010863775569452309],"hidden_sha256":"79426d9ee8af50d3123732feddd9dfe173ebc596fbe36217e6da505912b6347f","first_in_process":false,"shard":"shard_06","index":24},{"id":"semif_d7e84e4245a29817da1f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0006338022810928878,"mean_abs_dp":0.0004225348540618726,"oracle_top2_margin":0.9078157257003796,"near_tie":false,"n_options":3,"probs":[0.03991878549474366,0.011211154253838795,0.9488700602514176],"hidden_sha256":"9ff6837ad8ab6d75c241cbbb56ac170ad13ab4c2dd4cfe936298cf9411594792","first_in_process":false,"shard":"shard_06","index":25},{"id":"semif_f41ae66b3c09630c7a42","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00033697917876414005,"mean_abs_dp":0.00022465278584276582,"oracle_top2_margin":0.8487522382658496,"near_tie":false,"n_options":3,"probs":[0.060180870558292195,0.030915526665040733,0.9089036027766672],"hidden_sha256":"d37f0fe9bfa92361c1101cedf226ed1037bf4a2f4c0f5bae9c2880578cec5833","first_in_process":false,"shard":"shard_06","index":26},{"id":"semif_82c2cd97c160ad27951b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":69,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00220015995037931,"mean_abs_dp":0.001466773300252892,"oracle_top2_margin":0.5084756035543895,"near_tie":false,"n_options":3,"probs":[0.6794374000683251,0.14679553617171298,0.17376706375996193],"hidden_sha256":"7beb491a543f773635064a5ea604b623c85d508c61c7ff8e4da79eeb11055a90","first_in_process":false,"shard":"shard_06","index":27},{"id":"semif_7ada97b6f99f4788e5c7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002185650505469372,"mean_abs_dp":0.0014571003369796022,"oracle_top2_margin":0.6666932267467534,"near_tie":false,"n_options":3,"probs":[0.810685152351461,0.049553866634577906,0.1397609810139612],"hidden_sha256":"08b4faab07a4d62b80264989cfa1418a8daa2a083407ea379e4b2e1f430342d8","first_in_process":false,"shard":"shard_06","index":28},{"id":"semif_4d7a003342db5b0a17e3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.222303730841201e-05,"mean_abs_dp":8.14869153895464e-06,"oracle_top2_margin":0.9548245786970802,"near_tie":false,"n_options":3,"probs":[0.017276740099180437,0.9721166458846209,0.010606614016198685],"hidden_sha256":"996a45115de30dae556ef4a617701c0c23976070e2860b6d23623ac9d45b9291","first_in_process":false,"shard":"shard_06","index":29},{"id":"semif_19d3d8d0bf773e88be7b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":70,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011567185422522996,"mean_abs_dp":7.711456948348491e-05,"oracle_top2_margin":0.9590320722308011,"near_tie":false,"n_options":3,"probs":[0.9747685417825035,0.009673812393132315,0.015557645824364169],"hidden_sha256":"a46595aa2f9b012f48c928f2fdfb91d4a6720319a59249336250f089df2b364f","first_in_process":false,"shard":"shard_06","index":30},{"id":"semif_d6de731cca52359b1c87","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0032568472726709485,"mean_abs_dp":0.002171231515113984,"oracle_top2_margin":0.14805592269858542,"near_tie":false,"n_options":3,"probs":[0.4585043202016542,0.30589992025249696,0.2355957595458489],"hidden_sha256":"e194a21fd241748b55e2cd20a9e1fc9e1b4476f2a407be66bd23ac98b42df018","first_in_process":false,"shard":"shard_06","index":31},{"id":"semif_b45c63fb92729af27f05","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00011351796330483221,"mean_abs_dp":7.567864220322379e-05,"oracle_top2_margin":0.9420362994502539,"near_tie":false,"n_options":3,"probs":[0.9635687756986752,0.021324482305522988,0.015106741995801828],"hidden_sha256":"f63048d9204dd6cfe4dc852b2381929fb942443d5bdb50a70944ee92a63a2445","first_in_process":false,"shard":"shard_06","index":32},{"id":"semif_46b7029b9a704138b77a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":87,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0004613438337847109,"mean_abs_dp":0.0003075625558564739,"oracle_top2_margin":0.9326528003885701,"near_tie":false,"n_options":3,"probs":[0.025056308100309142,0.016511821584612223,0.9584318703150787],"hidden_sha256":"999051c671bd2cf4cb496417f9d2c05899fe84924430e2deaad351fad8e230d3","first_in_process":false,"shard":"shard_06","index":33},{"id":"semif_caf773b23934fa7362a1","name":"answer","variant":"base","source":"semif","type":"choice","tokens":81,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00023082323464651555,"mean_abs_dp":0.00015388215643103638,"oracle_top2_margin":0.9489755571809984,"near_tie":false,"n_options":3,"probs":[0.9670477226193419,0.017667722838373385,0.015284554542284574],"hidden_sha256":"a583478f4f88198a133b8d8baaca58f2faac3d00b6b30be7f656762a28810c1c","first_in_process":false,"shard":"shard_06","index":34},{"id":"semif_f46f392ef9e9e9df564b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005323303274746782,"mean_abs_dp":0.0035488688498311826,"oracle_top2_margin":0.4778150088143989,"near_tie":false,"n_options":3,"probs":[0.016146689843829297,0.2582889414543652,0.7255643687018055],"hidden_sha256":"7a256ad227730b9d4fd9a62fbbb1474218273026addb6618eb64d4ce1f99bc99","first_in_process":false,"shard":"shard_06","index":35},{"id":"semif_005dcf6d2c0f21d64796","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005685738739791613,"mean_abs_dp":0.003790492493194427,"oracle_top2_margin":0.38216002159483015,"near_tie":false,"n_options":3,"probs":[0.10157360983132298,0.6459490147010405,0.25247737546763654],"hidden_sha256":"fe25848b17da9a6f3020eb4f7cc2ce8d349d5a953785de2d11cc499669c20ec5","first_in_process":false,"shard":"shard_06","index":36},{"id":"semif_edb708bd0fe5fd67d6ee","name":"answer","variant":"base","source":"semif","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.002831503978846406,"mean_abs_dp":0.0018876693192308909,"oracle_top2_margin":0.44750541759432194,"near_tie":false,"n_options":3,"probs":[0.1388825004846176,0.655880023713887,0.20523747580149537],"hidden_sha256":"06d66f6e9f353d2832b83307dd3b8c237ee530d7ea806965cde1873fec820ade","first_in_process":false,"shard":"shard_06","index":37},{"id":"semif_97a35499bd8717f3e786","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0031488202752610883,"mean_abs_dp":0.0020992135168407047,"oracle_top2_margin":0.7893637810778941,"near_tie":false,"n_options":3,"probs":[0.028151752196940463,0.09416702242525529,0.8776812253778042],"hidden_sha256":"3fca995d05b78382fc4e4af1541567a7cceba45c49c6f00f3aa7d7e757bf031d","first_in_process":false,"shard":"shard_06","index":38},{"id":"semif_31eb4533d2c89959ec46","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0005182866457192625,"mean_abs_dp":0.0003455244304795685,"oracle_top2_margin":0.7023103058071338,"near_tie":false,"n_options":3,"probs":[0.8162764290509662,0.11416176576620375,0.06956180518282995],"hidden_sha256":"2636a6ca0bb86f51793074f7491d354ab20736bf5e23d3a04331b8a8a3baffec","first_in_process":false,"shard":"shard_06","index":39},{"id":"semif_0aa24a6701ac9c56c4a0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.008332555492610638,"mean_abs_dp":0.005555036995073703,"oracle_top2_margin":0.04788238825654201,"near_tie":false,"n_options":3,"probs":[0.4477302442738626,0.48018144719580824,0.07208830853032906],"hidden_sha256":"524defe29c1791d3fc5e6d7ba2b2210ed5a116cf2e3f6c89c2006e06bbfcc3c5","first_in_process":true,"shard":"shard_07","index":0},{"id":"semif_796d5c0da6eaadae6a3f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00011660074840057037,"mean_abs_dp":7.773383226701107e-05,"oracle_top2_margin":0.9559271118903919,"near_tie":false,"n_options":3,"probs":[0.012849312801917995,0.9716410960757016,0.015509591122380364],"hidden_sha256":"9674dfaffb610bccb85e5a66cb8b4da3348a464cb2ecb3f34e9e8d0feff95c03","first_in_process":false,"shard":"shard_07","index":1},{"id":"semif_599b3092b6fd18312d23","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00047504542369491976,"mean_abs_dp":0.0003166969491299783,"oracle_top2_margin":0.9417958472977064,"near_tie":false,"n_options":3,"probs":[0.013005392607172641,0.02218697711340202,0.9648076302794253],"hidden_sha256":"38469cd5339a82c2ef47c6de3517b9c447237bfd851c4a7a5c9e96d07fb6c4d1","first_in_process":false,"shard":"shard_07","index":2},{"id":"semif_960a4bcd27363e6db4b9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.002115203148829403,"mean_abs_dp":0.0014101354325529354,"oracle_top2_margin":0.6142107425731358,"near_tie":false,"n_options":3,"probs":[0.13899216629872432,0.10371412642872348,0.7572937072725522],"hidden_sha256":"a58f453b2cfb03b2a0885083cf28f5920b9ee25c93709f28706e197baf83b9ea","first_in_process":false,"shard":"shard_07","index":3},{"id":"semif_c6a411e7f2bbdf377173","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0009688575524694487,"mean_abs_dp":0.0006459050349796117,"oracle_top2_margin":0.8540853411132885,"near_tie":false,"n_options":3,"probs":[0.9132313124816692,0.05738636338281876,0.029382324135512113],"hidden_sha256":"dbc9c6b12e0a60d04ef600cc4aae5e24a172b34b54a7557d05f1fcd4019515bc","first_in_process":false,"shard":"shard_07","index":4},{"id":"semif_e332aa927d0c2ed044aa","name":"answer","variant":"base","source":"semif","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0038920841310218224,"mean_abs_dp":0.0025947227540145576,"oracle_top2_margin":0.6547585329407082,"near_tie":false,"n_options":3,"probs":[0.13763236086126965,0.06348990509710738,0.798877734041623],"hidden_sha256":"3ac6abd882944d2fe80b391f3f0fac10156345dba13e405d4f51db181e945262","first_in_process":false,"shard":"shard_07","index":5},{"id":"semif_a0128e551d095f3449f8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012372143779404432,"mean_abs_dp":8.24809585293276e-05,"oracle_top2_margin":0.9270749550688122,"near_tie":false,"n_options":3,"probs":[0.957006154056187,0.03013372070037132,0.012860125243441562],"hidden_sha256":"53f0c241f6905341d240ad1426a9d19029b411f169e0ef4d815a61743de379b5","first_in_process":false,"shard":"shard_07","index":6},{"id":"semif_46d6a0731bd288c370f0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00196279675117661,"mean_abs_dp":0.0013085311674510755,"oracle_top2_margin":0.21947026315268053,"near_tie":false,"n_options":3,"probs":[0.5945466547525275,0.3788453437582673,0.02660800148920515],"hidden_sha256":"5a0e0b66ab21eaf36bd65fe9eabf8d9ad70265b9474e136877ef08da6f1ab43a","first_in_process":false,"shard":"shard_07","index":7},{"id":"semif_21ca7a8b970e36b8e20f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":106,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002623791566509284,"mean_abs_dp":0.0001749194377672833,"oracle_top2_margin":0.8425499157252577,"near_tie":false,"n_options":3,"probs":[0.9065304774182613,0.06422047513532929,0.029249047446409368],"hidden_sha256":"fce7d68332d9c0ff923e20428468a61a489c4571b9a91119175525abfc8d6114","first_in_process":false,"shard":"shard_07","index":8},{"id":"semif_4aa83af95bb69e90bc2d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.005133748684604061,"mean_abs_dp":0.003422499123069365,"oracle_top2_margin":0.2723478587984883,"near_tie":false,"n_options":3,"probs":[0.20506056938019998,0.2615117933357212,0.5334276372840787],"hidden_sha256":"0a3450485aada14e84961f0c7759cefec3f203e7f203129d317e0ae5814066e1","first_in_process":false,"shard":"shard_07","index":9},{"id":"semif_cdfa9eed51fafcf09b72","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004974415711401892,"mean_abs_dp":0.0033162771409345434,"oracle_top2_margin":0.35764993020134866,"near_tie":false,"n_options":3,"probs":[0.26554307410748906,0.6311261947051153,0.10333073118739569],"hidden_sha256":"69d0f9883a44efabfaf9beac0599d3c5e63ced7eeb1d5d3c2981daea26ebea11","first_in_process":false,"shard":"shard_07","index":10},{"id":"semif_d51b05bfb9d6d7f09331","name":"answer","variant":"base","source":"semif","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.005833550273285915,"mean_abs_dp":0.0038890335155239452,"oracle_top2_margin":0.13898957633128284,"near_tie":false,"n_options":3,"probs":[0.5541266102652984,0.40415210617517583,0.041721283559525736],"hidden_sha256":"e419ec3156ccc758e28296eef6317066e4a5b9ceacfc441baf5cc4140d54f01b","first_in_process":false,"shard":"shard_07","index":11},{"id":"semif_fdd902cc6e65fad7cb4f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":6.49867837320614e-05,"mean_abs_dp":4.332452248808488e-05,"oracle_top2_margin":0.8581303373394734,"near_tie":false,"n_options":3,"probs":[0.03782431063400553,0.9101639169007593,0.052011772465235244],"hidden_sha256":"89286ff83a8d441488d5259b7479f4074f6d1398c12f473122efffe89953ca45","first_in_process":false,"shard":"shard_07","index":12},{"id":"semif_42384af9ae56beb37ff5","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004388529903381788,"mean_abs_dp":0.0029256866022544883,"oracle_top2_margin":0.7064067239423519,"near_tie":false,"n_options":3,"probs":[0.1119812307881701,0.06271826735079813,0.8253005018610319],"hidden_sha256":"6d07e331f6b98487c4878dc678d851281a4b1100aa2dbb6294b7e9b45ae6f523","first_in_process":false,"shard":"shard_07","index":13},{"id":"semif_cdf7f428857e1b841cb3","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001017816533522009,"mean_abs_dp":0.0006785443556812654,"oracle_top2_margin":0.3444203525752899,"near_tie":false,"n_options":3,"probs":[0.13801981469874222,0.2587715388245943,0.6032086464766636],"hidden_sha256":"8724b520108e8cd0fbbb7209b47e84724f0817ebe0508c6f2c18183774ef1fb6","first_in_process":false,"shard":"shard_07","index":14},{"id":"semif_774f7f9245f83c796d44","name":"answer","variant":"base","source":"semif","type":"choice","tokens":77,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0008094154517634955,"mean_abs_dp":0.0005396103011756949,"oracle_top2_margin":0.9309078473446127,"near_tie":false,"n_options":3,"probs":[0.02491961957282977,0.017935941653531778,0.9571444387736384],"hidden_sha256":"f5558cf7d61474e661d86ea3d90efd4f572bc78e7340bdbe697dc4d000a65516","first_in_process":false,"shard":"shard_07","index":15},{"id":"semif_92ec2ffa3dd46ada1335","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008653615632899792,"mean_abs_dp":0.000576907708859963,"oracle_top2_margin":0.6321995772739603,"near_tie":false,"n_options":3,"probs":[0.7774728695499854,0.0782620147634337,0.14426511568658087],"hidden_sha256":"dd475a8e12b8c4a846e556e63dfd475f0780334e70be73dac7ad244876bb339a","first_in_process":false,"shard":"shard_07","index":16},{"id":"semif_6149a17bc154f9c5b4a0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003083565142050215,"mean_abs_dp":0.00020557100947001203,"oracle_top2_margin":0.9220314085305296,"near_tie":false,"n_options":3,"probs":[0.9493651780905343,0.02693590036169984,0.02369892154776579],"hidden_sha256":"415b21afd5195364557a1ca87f5f84937b24e22226b22e3c351bfcb1b45cbbc9","first_in_process":false,"shard":"shard_07","index":17},{"id":"semif_be866e1dbec5d42d9423","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002623562749463071,"mean_abs_dp":0.0001749041832975635,"oracle_top2_margin":0.9256116863283881,"near_tie":false,"n_options":3,"probs":[0.028518793873025812,0.016895251316125896,0.9545859548108483],"hidden_sha256":"e0862749e61fa8af5d4f4be01054853f58f9163f9c117a3cdf091c0980d9f014","first_in_process":false,"shard":"shard_07","index":18},{"id":"semif_97d2ed2591fc7638806c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.005374550655593807,"mean_abs_dp":0.0035830337703958762,"oracle_top2_margin":0.47570627138171984,"near_tie":false,"n_options":3,"probs":[0.21818842497532095,0.7040662293615425,0.07774534566313648],"hidden_sha256":"72e661d78c2bce488f28c281fdab3f8bd95617589fe81790560834af2c1e31a3","first_in_process":false,"shard":"shard_07","index":19},{"id":"semif_4533dd91267b4d815690","name":"answer","variant":"base","source":"semif","type":"choice","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00045821935204126696,"mean_abs_dp":0.0003054795680275252,"oracle_top2_margin":0.868350060217824,"near_tie":false,"n_options":3,"probs":[0.0446821128741758,0.04307441332469313,0.9122434738011311],"hidden_sha256":"32a642929dfc77a55db0edfeb1e4d5c321d9d1d6ea6f3b8f1ec824eff8d4719c","first_in_process":false,"shard":"shard_07","index":20},{"id":"semif_d880e65fef5e2993fdd0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":95,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0011080913352120303,"mean_abs_dp":0.0007387275568080479,"oracle_top2_margin":0.22914503123859198,"near_tie":false,"n_options":3,"probs":[0.49658406163398316,0.26682135108992183,0.236594587276095],"hidden_sha256":"063b489dce4b1cf50611a433ef5407c9a48259007b64e8dc5afd614598bf173c","first_in_process":false,"shard":"shard_07","index":21},{"id":"semif_90494c76078ffc3c717a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.003428973859812001,"mean_abs_dp":0.002285982573207982,"oracle_top2_margin":0.3375654329031006,"near_tie":false,"n_options":3,"probs":[0.23539786328304102,0.5675518291565125,0.1970503075604465],"hidden_sha256":"8b2fc7212f85b1b00e6de31127613d0fea69cfc4814f04cd09df69e72a4ad2bb","first_in_process":false,"shard":"shard_07","index":22},{"id":"semif_9ac5cc82c4126c91174a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0030712321196562087,"mean_abs_dp":0.0020474880797707643,"oracle_top2_margin":0.721630943498277,"near_tie":false,"n_options":3,"probs":[0.07309451608317055,0.8216328246334483,0.10527265928338103],"hidden_sha256":"c1fcb3da291fe76986d23aac9c274733b51ac427fe70a3fc02053a4a74384d7e","first_in_process":false,"shard":"shard_07","index":23},{"id":"semif_386e10b1c1ce906473a4","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.000156132213549931,"mean_abs_dp":0.00010408814236661836,"oracle_top2_margin":0.9497366260379649,"near_tie":false,"n_options":3,"probs":[0.014747450532831032,0.9676068463370366,0.017645703130132306],"hidden_sha256":"f0892ba2a72315384246fac700610d3a82fd61fa6813a75e76744b72afa8254d","first_in_process":false,"shard":"shard_07","index":24},{"id":"semif_4da6aedc1139224eeff2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":110,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":1.2489763058298915e-05,"mean_abs_dp":8.326508705524226e-06,"oracle_top2_margin":0.9739471851541249,"near_tie":false,"n_options":3,"probs":[0.009224508429073575,0.983175878726839,0.007599612844087511],"hidden_sha256":"52293f7cb9e76c9f51928fbb044148c31baea2c5881ebb15482cd200edaf4c66","first_in_process":false,"shard":"shard_07","index":25},{"id":"semif_a22dab8ba073ff6cdb8d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":100,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006007694358335369,"mean_abs_dp":0.00040051295722234637,"oracle_top2_margin":0.8182519231250659,"near_tie":false,"n_options":3,"probs":[0.055009639895853814,0.8819774797273721,0.06301288037677417],"hidden_sha256":"ee71ee1c930d47707ad978f5f33bc18d89d044d796f3b256069f83ef5843df58","first_in_process":false,"shard":"shard_07","index":26},{"id":"semif_42c54fa2ad45743e24d8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":86,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00015611624108725214,"mean_abs_dp":0.00010407749405811721,"oracle_top2_margin":0.9014139991441142,"near_tie":false,"n_options":3,"probs":[0.03316011202896957,0.9347099032698353,0.0321299847011951],"hidden_sha256":"7709adf0a5b3bafb0cdf1bb6139804e1351f3a2dc228aa3a2077c0c2120ebab6","first_in_process":false,"shard":"shard_07","index":27},{"id":"semif_f3d13cf184f916677def","name":"answer","variant":"base","source":"semif","type":"choice","tokens":99,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0003821201691905918,"mean_abs_dp":0.0002547467794604211,"oracle_top2_margin":0.9155676995353127,"near_tie":false,"n_options":3,"probs":[0.9500797566053143,0.015964018468683932,0.03395622492600173],"hidden_sha256":"e9705a10afcaef563a5a4a8033e21b5825cc072a1d95e0f499797fbef7ee1d32","first_in_process":false,"shard":"shard_07","index":28},{"id":"semif_1105577da4c8dad4609d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002398522742169673,"mean_abs_dp":0.00015990151614464604,"oracle_top2_margin":0.9671984103444269,"near_tie":false,"n_options":3,"probs":[0.011258299329175069,0.009895977259533935,0.9788457234112911],"hidden_sha256":"27a765b05fc032bd80a2f037ad31c9765340914d76ab8f280fd7ed42d161b58a","first_in_process":false,"shard":"shard_07","index":29},{"id":"semif_0ebd299739302d8b057b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":92,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002324188588827325,"mean_abs_dp":0.001549459059218226,"oracle_top2_margin":0.10794600842693236,"near_tie":false,"n_options":3,"probs":[0.4396890374185556,0.23274832891156672,0.3275626336698777],"hidden_sha256":"481fa2118c200326ce450366147f667123c87e50a0ab0e5f618607ee15ebce5f","first_in_process":false,"shard":"shard_07","index":30},{"id":"semif_eafc22c8c40df3932a8e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":80,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009653083771981041,"mean_abs_dp":0.0006435389181320671,"oracle_top2_margin":0.8458232850367531,"near_tie":false,"n_options":3,"probs":[0.04684000714673626,0.05274013719095992,0.9004198556623039],"hidden_sha256":"fddbaebcbc942323d97bdb9e6f39f2b19f25e543f65d23683d389b7cb73cf6cb","first_in_process":false,"shard":"shard_07","index":31},{"id":"semif_0aa1447680e4f25862bb","name":"answer","variant":"base","source":"semif","type":"choice","tokens":103,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0023107159005960054,"mean_abs_dp":0.001540477267063976,"oracle_top2_margin":0.08837413898905816,"near_tie":false,"n_options":3,"probs":[0.34276550200828204,0.4270797519486802,0.23015474604303773],"hidden_sha256":"e668a82a19f6de5dd256c977b38c5978ab0a70e0b326f4534a2245e3d75fc422","first_in_process":false,"shard":"shard_07","index":32},{"id":"semif_980e76145c925747220c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":104,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.007800886491597214,"mean_abs_dp":0.00520059099439818,"oracle_top2_margin":0.030265359764135746,"near_tie":false,"n_options":3,"probs":[0.38355565541706127,0.3452433572442066,0.2712009873387322],"hidden_sha256":"443ecd4e2279f3530f03e8c130d4d2fe0439a05fc5c46498a463a3c0cd315581","first_in_process":false,"shard":"shard_07","index":33},{"id":"semif_4aee37a6a155747766aa","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004936201973839538,"mean_abs_dp":0.0032908013158929514,"oracle_top2_margin":0.09844443446324236,"near_tie":false,"n_options":3,"probs":[0.3440828974157148,0.45041251054386866,0.20550459204041646],"hidden_sha256":"325ec8b4411de22cecdb6bef5f46dde845a32bd12ad6deaea7d8cba6854471b8","first_in_process":false,"shard":"shard_07","index":34},{"id":"semif_6db17cc22bd656d78558","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005890218502844075,"mean_abs_dp":0.00039268123352298456,"oracle_top2_margin":0.773589262631881,"near_tie":false,"n_options":3,"probs":[0.08500505273950973,0.05591756337521379,0.8590773838852765],"hidden_sha256":"8fddace8b36a45fdc23d61a9c4263ee872a0967374d3efed4f7dabbf18f6b581","first_in_process":false,"shard":"shard_07","index":35},{"id":"semif_ef3e0f8e657bf59f8a29","name":"answer","variant":"base","source":"semif","type":"choice","tokens":97,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.002727198611144621,"mean_abs_dp":0.0018181324074297612,"oracle_top2_margin":0.30690845818680684,"near_tie":false,"n_options":3,"probs":[0.5954034718110613,0.2835050237315586,0.1210915044573801],"hidden_sha256":"104f38046be889625665b2224fa73c5d9353cbb89e9bbd5797ea3ad4bdcb29c2","first_in_process":false,"shard":"shard_07","index":36},{"id":"semif_f756e1cb04e11766341e","name":"answer","variant":"base","source":"semif","type":"choice","tokens":103,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009773143511814242,"mean_abs_dp":0.0006515429007876208,"oracle_top2_margin":0.6114943871073327,"near_tie":false,"n_options":3,"probs":[0.0988738993947726,0.1442773789246789,0.7568487216805485],"hidden_sha256":"6fcb681af2e62f6d0645098bcaf308fbbc58c943b822ae1b75db55b872c62637","first_in_process":false,"shard":"shard_07","index":37},{"id":"semif_d167e51bb1f499099645","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.006788570040815012,"mean_abs_dp":0.004525713360543369,"oracle_top2_margin":0.23000797836991627,"near_tie":false,"n_options":3,"probs":[0.2594365234387829,0.2407383736031458,0.4998251029580713],"hidden_sha256":"e15c68708c647f942e8deade6ee47e574d1f3e9b3fad4a1de969a46043da67d3","first_in_process":false,"shard":"shard_07","index":38},{"id":"semif_674ca4e9346216255466","name":"answer","variant":"base","source":"semif","type":"choice","tokens":73,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0018160999386019894,"mean_abs_dp":0.0012107332924013354,"oracle_top2_margin":0.7924928211657615,"near_tie":false,"n_options":3,"probs":[0.8666377624052677,0.07153646070439738,0.061825776890334985],"hidden_sha256":"2a4244bd912a54080356f415a82e656472a7ef4b36db3809708baa9e8655e3e5","first_in_process":false,"shard":"shard_07","index":39},{"id":"semif_1df851a202994a07f123","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00029732711401209144,"mean_abs_dp":0.0001982180760081107,"oracle_top2_margin":0.9421137806752105,"near_tie":false,"n_options":3,"probs":[0.9649076678515881,0.02226728661711953,0.012825045531292233],"hidden_sha256":"9797a2a974eef317bea688abb38059c8dbc43d3efe5faf51bfc53b8039c90953","first_in_process":true,"shard":"shard_08","index":0},{"id":"semif_b5e6788d95b8395d2888","name":"answer","variant":"base","source":"semif","type":"choice","tokens":84,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00017931554918226844,"mean_abs_dp":0.00011954369945486067,"oracle_top2_margin":0.9589997989900602,"near_tie":false,"n_options":3,"probs":[0.011330490751548404,0.014675584941865943,0.9739939243065856],"hidden_sha256":"21753d6b1f2336e8ac8a8bdfba6972168073fb992cea992ccd7ee71c1dc3ed04","first_in_process":false,"shard":"shard_08","index":1},{"id":"semif_32f36cd0fbbfb3604586","name":"answer","variant":"base","source":"semif","type":"choice","tokens":74,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00064832702446882,"mean_abs_dp":0.00043221801631255244,"oracle_top2_margin":0.8345006167187339,"near_tie":false,"n_options":3,"probs":[0.9029351699764201,0.06721809609079048,0.02984673393278934],"hidden_sha256":"de59565551d6231bd8dcffd9871325b5c9271f18b9b9cd774a5b914d50eab800","first_in_process":false,"shard":"shard_08","index":2},{"id":"semif_843185706adff0c60f8b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":75,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0020259590370597935,"mean_abs_dp":0.0013506393580398277,"oracle_top2_margin":0.5507627206585595,"near_tie":false,"n_options":3,"probs":[0.7652069197159221,0.016406576253485808,0.21838650403059212],"hidden_sha256":"23942849c81a9b4fe2486bd2f3c82b4fae263317ef0cb427d4a51878e52e6f32","first_in_process":false,"shard":"shard_08","index":3},{"id":"semif_b531dd603bf080dab820","name":"answer","variant":"base","source":"semif","type":"choice","tokens":91,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004278404832802618,"mean_abs_dp":0.0002852269888534581,"oracle_top2_margin":0.701489376714385,"near_tie":false,"n_options":3,"probs":[0.8385075110449737,0.023739634970107823,0.13775285398491863],"hidden_sha256":"3b415d978b99a80b8bba3bb04ca62e4e52ab2ed770b6a36a36c6193e3b1b8c92","first_in_process":false,"shard":"shard_08","index":4},{"id":"semif_be2dd67bc733c1213bef","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00690829619000044,"mean_abs_dp":0.00460553079333358,"oracle_top2_margin":0.41848908396759443,"near_tie":false,"n_options":3,"probs":[0.13354397547885352,0.6370660309530914,0.22938999356805495],"hidden_sha256":"735e99fe00fc51909c92b2d8612f7680fbd2f4de365b5408bdd8283b26d8d565","first_in_process":false,"shard":"shard_08","index":5},{"id":"semif_b8e2750307df9761e716","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0035566572124605678,"mean_abs_dp":0.002371104808307096,"oracle_top2_margin":0.6699031354481804,"near_tie":false,"n_options":3,"probs":[0.8039661331960638,0.06793108126312754,0.12810278554080853],"hidden_sha256":"93d4d5c4fa2a3b286ba6ebed02385dc3b7940fc47c797800b3489bc431009008","first_in_process":false,"shard":"shard_08","index":6},{"id":"semif_5d31b4bf2d0fc193f53b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0035963904767345456,"mean_abs_dp":0.0023975936511563913,"oracle_top2_margin":0.2891301652654162,"near_tie":false,"n_options":3,"probs":[0.08344964301644926,0.3107303505795765,0.6058200064039742],"hidden_sha256":"0bf7c2385db72da2bc7175e60cfa209791e8d3f35f26c2ea1aaa5bf20aaeebb2","first_in_process":false,"shard":"shard_08","index":7},{"id":"semif_a9230edfc6ec9e524af0","name":"answer","variant":"base","source":"semif","type":"choice","tokens":82,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0007635706230484374,"mean_abs_dp":0.0005090470820322789,"oracle_top2_margin":0.9425808935221376,"near_tie":false,"n_options":3,"probs":[0.017087689423445424,0.01959290442832467,0.9633194061482299],"hidden_sha256":"3e734eb7100a708b949b32d6181c1a0d39710e6b05a09484f39391721f9e7cd0","first_in_process":false,"shard":"shard_08","index":8},{"id":"semif_c100e84585149a04be49","name":"answer","variant":"base","source":"semif","type":"choice","tokens":88,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.000350797885072196,"mean_abs_dp":0.0002338652567148112,"oracle_top2_margin":0.9447431163686603,"near_tie":false,"n_options":3,"probs":[0.9638292491154273,0.017665476888007403,0.018505273996565177],"hidden_sha256":"4e81fdbc829ac8bb69569900c97b972791cd5a06bbed8c1b56ee0f3652ee642b","first_in_process":false,"shard":"shard_08","index":9},{"id":"semif_579beed184ffc1104aa9","name":"answer","variant":"base","source":"semif","type":"choice","tokens":79,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0017746591338307915,"mean_abs_dp":0.0011831060892204977,"oracle_top2_margin":0.8268440695720631,"near_tie":false,"n_options":3,"probs":[0.8848091531116117,0.060714015412125025,0.054476831476263206],"hidden_sha256":"97cb62e488c2d6159a2b1d69cbc06acaed505abe5d709f2ad5a3a1e74dfc5955","first_in_process":false,"shard":"shard_08","index":10},{"id":"semif_c4e9bd463b6fde5be743","name":"answer","variant":"base","source":"semif","type":"choice","tokens":67,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00047387842151669357,"mean_abs_dp":0.0003159189476778142,"oracle_top2_margin":0.9288057413460095,"near_tie":false,"n_options":3,"probs":[0.024040923014318608,0.9535630734820874,0.022396003503594088],"hidden_sha256":"9b90c9b66dd9f9cce37a9582e2706978ac17285c37d0ea4f88a80f0dee10a1c9","first_in_process":false,"shard":"shard_08","index":11},{"id":"semif_b863998d4be2721a5dfd","name":"answer","variant":"base","source":"semif","type":"choice","tokens":96,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.008796089903942539,"mean_abs_dp":0.005864059935961717,"oracle_top2_margin":0.4760259113962742,"near_tie":false,"n_options":3,"probs":[0.020076861854213653,0.24329135850171751,0.7366317796440688],"hidden_sha256":"f19ce10a884de745948cf171345c67c51f619c3932074e1ff4f151ddb899aa64","first_in_process":false,"shard":"shard_08","index":12},{"id":"semif_37cebc3792e98049439f","name":"answer","variant":"base","source":"semif","type":"choice","tokens":93,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00403646042053607,"mean_abs_dp":0.002690973613690704,"oracle_top2_margin":0.2339733215851162,"near_tie":false,"n_options":3,"probs":[0.34686616075868393,0.5885714042796644,0.0645624349616517],"hidden_sha256":"d2b7ab12487df76a187bd3e305a0e7a148a856ee14b59d75f9b853c22bb006e1","first_in_process":false,"shard":"shard_08","index":13},{"id":"semif_64d63f86e328aff1481c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0005145059686755271,"mean_abs_dp":0.00034300397911704697,"oracle_top2_margin":0.9554370711800846,"near_tie":false,"n_options":3,"probs":[0.015155164494588726,0.013448991778649265,0.9713958437267619],"hidden_sha256":"edfe688f6a6314d585a0b7d7147fe01f471f286b2103bde1e58ba9e0f2f6971f","first_in_process":false,"shard":"shard_08","index":14},{"id":"semif_6d64a8459f64eac7f4cc","name":"answer","variant":"base","source":"semif","type":"choice","tokens":98,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00047462014451349344,"mean_abs_dp":0.0003164134296757039,"oracle_top2_margin":0.8876652124031529,"near_tie":false,"n_options":3,"probs":[0.9323234428752698,0.04389087072770938,0.02378568639702077],"hidden_sha256":"5d60bd56317888535ac7d1b54703e76f0cc6e70c2c0cdbab368854d620695254","first_in_process":false,"shard":"shard_08","index":15},{"id":"semif_2708193212d8a4a523c7","name":"answer","variant":"base","source":"semif","type":"choice","tokens":102,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0026865226775868445,"mean_abs_dp":0.0017910151183912065,"oracle_top2_margin":0.77678402470049,"near_tie":false,"n_options":3,"probs":[0.03260310664027562,0.8744505731928499,0.09294632016687457],"hidden_sha256":"7a8a02ee1eb207033d035c17710e8ac3824d4af988d86c503892d7001753281e","first_in_process":false,"shard":"shard_08","index":16},{"id":"semif_5a37d1dfcad20810a76a","name":"answer","variant":"base","source":"semif","type":"choice","tokens":90,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0008809096527288007,"mean_abs_dp":0.0005872731018192375,"oracle_top2_margin":0.7956891997585444,"near_tie":false,"n_options":3,"probs":[0.01932615158123637,0.8873855154567107,0.09328833296205295],"hidden_sha256":"621c88fceded332078e6b2ddf802fc743cbac8189cfc95fa63f3a58f87d18afa","first_in_process":false,"shard":"shard_08","index":17},{"id":"semif_acacd8b46acd25cb954b","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003389322176451104,"mean_abs_dp":0.0002259548117634127,"oracle_top2_margin":0.7741791224279037,"near_tie":false,"n_options":3,"probs":[0.10003979286459219,0.8737444623449979,0.026215744790409803],"hidden_sha256":"bbe96507aa66e8783483ba0c80a23099cdcdb7b22f4c8b9e45d6e976ac15f32a","first_in_process":false,"shard":"shard_08","index":18},{"id":"semif_4089c199b8b796ec3ed8","name":"answer","variant":"base","source":"semif","type":"choice","tokens":101,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0016217060226894064,"mean_abs_dp":0.0010811373484595859,"oracle_top2_margin":0.31988840025581594,"near_tie":false,"n_options":3,"probs":[0.6059493438194635,0.10496601349178632,0.28908464268875017],"hidden_sha256":"ac968a6f55ff8336756d85079ba5ec1ab32d2c464521261a6161df426485ddee","first_in_process":false,"shard":"shard_08","index":19},{"id":"semif_02f2b95a5c5a4777a45c","name":"answer","variant":"base","source":"semif","type":"choice","tokens":89,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00012167062908008397,"mean_abs_dp":8.11137527200421e-05,"oracle_top2_margin":0.9451561864677233,"near_tie":false,"n_options":3,"probs":[0.016150273284472244,0.9643934197322253,0.019456306983302446],"hidden_sha256":"78ddaf1ba9ba8424e4afc271fef646b26c2f59e1f88cabb34ee9e74c60fa8b6c","first_in_process":false,"shard":"shard_08","index":20},{"id":"semif_ebea4681e3dcc7d92ca2","name":"answer","variant":"base","source":"semif","type":"choice","tokens":94,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0020934464523746643,"mean_abs_dp":0.0013956309682497624,"oracle_top2_margin":0.6209814841559299,"near_tie":false,"n_options":3,"probs":[0.062314041399558415,0.7775632076205974,0.1601227509798442],"hidden_sha256":"d8cd6aa2bddec5106e0a4bf699b47c88136e529267bb1c0cf6e47934f09b191d","first_in_process":false,"shard":"shard_08","index":21},{"id":"semif_b8c5b9b285cbdddc9f8d","name":"answer","variant":"base","source":"semif","type":"choice","tokens":85,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0026031455974440953,"mean_abs_dp":0.0017354303982960832,"oracle_top2_margin":0.6299678225843708,"near_tie":false,"n_options":3,"probs":[0.017553634062808143,0.1788389317184579,0.8036074342187338],"hidden_sha256":"666b4fa2efe9f0c3136124c14c9128ef3eec1ba2ddf2afa8ddea251df1232af5","first_in_process":false,"shard":"shard_08","index":22},{"id":"semif_7ed75d854d99b3a81916","name":"answer","variant":"base","source":"semif","type":"choice","tokens":83,"calls":2,"padded_tokens":128,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00018181064741740194,"mean_abs_dp":0.00012120709827824483,"oracle_top2_margin":0.894601011388454,"near_tie":false,"n_options":3,"probs":[0.931763140846545,0.03143821412702442,0.03679864502643061],"hidden_sha256":"3bcccecf176c8bb0e407942ceb2efa7c0b875dc618b1701813a89c9b232acd8b","first_in_process":false,"shard":"shard_08","index":23},{"id":"own_ticket_01","name":"team","variant":"base","source":"own","type":"choice","tokens":161,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010601184023983912,"mean_abs_dp":5.300592011994341e-05,"oracle_top2_margin":0.978724391038446,"near_tie":false,"n_options":4,"probs":[0.9849211695161856,0.004359099019594032,0.006034782358075691,0.004684949106144705],"hidden_sha256":"f2eee52f5bdc9b0d9492526dd97579e1510e60d62057006061fe5ccb0ac5613e","first_in_process":false,"shard":"shard_08","index":24},{"id":"own_ticket_01","name":"deadline","variant":"base","source":"own","type":"noul","tokens":149,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.083317922149888e-07,"mean_abs_dp":5.083317921599113e-07,"oracle_top2_margin":0.9854124568314981,"near_tie":false,"n_options":2,"probs":[0.9927067367475413,0.007293263252458848],"hidden_sha256":"139b63076c487bb98a719d77995fd84b2a93791f7490893c78a061eba93c153b","first_in_process":false,"shard":"shard_08","index":25},{"id":"own_ticket_01","name":"mood","variant":"base","source":"own","type":"score","tokens":138,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00022622118261150792,"mean_abs_dp":0.0001508141217410073,"oracle_top2_margin":0.9818281952989258,"near_tie":false,"n_options":3,"probs":[0.9898578597123825,0.007612835252342446,0.0025293050352750857],"hidden_sha256":"f391db8f0bc74f9346eb8ded29e55f4acaa9fa34e0d4a420baed5f61e3f001fd","first_in_process":false,"shard":"shard_08","index":26},{"id":"own_incident_02","name":"severity","variant":"base","source":"own","type":"score","tokens":201,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00045367242854843504,"mean_abs_dp":0.00018146897141938512,"oracle_top2_margin":0.9326626788228828,"near_tie":false,"n_options":5,"probs":[0.007129696334645113,0.01375040052851166,0.9540454411607502,0.020560200964927187,0.004514261011165883],"hidden_sha256":"d3f1489ca49c66b132b00470fbab513ee4fc2e39f4239c8bd840169b0700c1b1","first_in_process":false,"shard":"shard_08","index":27},{"id":"own_incident_02","name":"cause","variant":"base","source":"own","type":"choice","tokens":193,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.001061049979421802,"mean_abs_dp":0.00042441999176875603,"oracle_top2_margin":0.9230403231000168,"near_tie":false,"n_options":5,"probs":[0.952602584376141,0.00933554919752003,0.006232605676461755,0.004131341959700536,0.02769791879017659],"hidden_sha256":"c8cf8b50d048fd14ff7523c0a9b52e476a6ea4f7b3467e2698212a697576170e","first_in_process":false,"shard":"shard_08","index":28},{"id":"own_incident_02","name":"repeat","variant":"base","source":"own","type":"noul","tokens":155,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.158777503233569e-05,"mean_abs_dp":5.1587775032324416e-05,"oracle_top2_margin":0.9695785051190727,"near_tie":false,"n_options":2,"probs":[0.9848408403345688,0.015159159665431325],"hidden_sha256":"63b99c8ecdb05c3ec85e7912e535235e8af00860b8c33282f81b0b4eaf154dd1","first_in_process":false,"shard":"shard_08","index":29},{"id":"own_email_03","name":"slot","variant":"base","source":"own","type":"choice","tokens":221,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0006128990014052649,"mean_abs_dp":0.0004085993342701315,"oracle_top2_margin":0.8965342472316058,"near_tie":false,"n_options":3,"probs":[0.03946668626106954,0.9350030601368265,0.02553025360210379],"hidden_sha256":"cf09ae61136d8b4434f113f45633055a5a53a56de58b682eb749578ccd98863e","first_in_process":false,"shard":"shard_08","index":30},{"id":"own_email_03","name":"rules_out_thursday","variant":"base","source":"own","type":"noul","tokens":203,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00012996869341545467,"mean_abs_dp":0.00012996869341538007,"oracle_top2_margin":0.9643501049842991,"near_tie":false,"n_options":2,"probs":[0.9820450837987342,0.017954916201265922],"hidden_sha256":"b69ee94729c657801bf1241a1847e5442e922841e79aa716e8739bb64dbe79fd","first_in_process":false,"shard":"shard_08","index":31},{"id":"own_meeting_04","name":"script_owner","variant":"base","source":"own","type":"choice","tokens":169,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.610693474733132e-05,"mean_abs_dp":1.7404623164877137e-05,"oracle_top2_margin":0.9810108457037208,"near_tie":false,"n_options":3,"probs":[0.006619636287583563,0.9876001660931307,0.005780197619285723],"hidden_sha256":"acff0e64b240a46fecdfb9f25169cc4d0fe042b12b27a95bc99fd4aa1558767f","first_in_process":false,"shard":"shard_08","index":32},{"id":"own_meeting_04","name":"tablet_decision","variant":"base","source":"own","type":"noul","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00013856433127612533,"mean_abs_dp":0.0001385643312760837,"oracle_top2_margin":0.9660577759154095,"near_tie":false,"n_options":2,"probs":[0.9831674522889808,0.016832547711019157],"hidden_sha256":"d8945ab0e6916a8fbe1400da114d915d5f9c6dcb8cb59b8fceea3b3d64f711a4","first_in_process":false,"shard":"shard_08","index":33},{"id":"own_meeting_04","name":"rota","variant":"base","source":"own","type":"choice","tokens":173,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00012760489101371064,"mean_abs_dp":8.506992734248157e-05,"oracle_top2_margin":0.9651874400399899,"near_tie":false,"n_options":3,"probs":[0.006890258047678497,0.9792523299875143,0.013857411964807189],"hidden_sha256":"aa72b2f99168c2076aa9819e204ebb5a97ecb08ed064584d0671d1506a0c4f8c","first_in_process":false,"shard":"shard_08","index":34},{"id":"own_review_05","name":"stars","variant":"base","source":"own","type":"score","tokens":157,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004339355328092487,"mean_abs_dp":0.0017383302152377208,"oracle_top2_margin":0.33424306724969377,"near_tie":false,"n_options":5,"probs":[0.282091558617667,0.6095323330884533,0.09530542120039787,0.010660579807629045,0.0024101072858527394],"hidden_sha256":"c71a7836377b52c8102c703f4a06ae6fb2fde81c4728617527e30116933a5fb2","first_in_process":false,"shard":"shard_08","index":35},{"id":"own_review_05","name":"main_issue","variant":"base","source":"own","type":"choice","tokens":187,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":5.3430202833193094e-05,"mean_abs_dp":1.3612769454118627e-05,"oracle_top2_margin":0.9661892165317619,"near_tie":false,"n_options":10,"probs":[0.004344700374219557,0.9731202090171479,0.003001102177772546,0.0018195082326197029,0.002807287486482413,0.0031906593252626282,0.0016545359567184021,0.0015030060735731956,0.006997455784862635,0.0015615355713410495],"hidden_sha256":"51f253daca60970d882ba1f8167918e5faf4575b7a8515b040a14993dc964b4e","first_in_process":false,"shard":"shard_08","index":36},{"id":"own_review_05","name":"would_rebuy","variant":"base","source":"own","type":"noul","tokens":150,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.7607767826243416e-05,"mean_abs_dp":4.760776782622e-05,"oracle_top2_margin":0.9770772896738245,"near_tie":false,"n_options":2,"probs":[0.011508962930914067,0.988491037069086],"hidden_sha256":"38002b5a34a60115998e31b6fa0c7d89a09df85b7ec224cf6d7d7fd7b66cc74a","first_in_process":false,"shard":"shard_08","index":37},{"id":"own_order_06","name":"free_shipping","variant":"base","source":"own","type":"noul","tokens":248,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00024514346909679935,"mean_abs_dp":0.000245143469096782,"oracle_top2_margin":0.9421814505831572,"near_tie":false,"n_options":2,"probs":[0.028664131239324594,0.9713358687606753],"hidden_sha256":"96e5897c6503d54c35d16d8cca38520fe7b84ca62453b26aa7b8579cc61baffb","first_in_process":false,"shard":"shard_08","index":38},{"id":"own_order_06","name":"method","variant":"base","source":"own","type":"choice","tokens":238,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.333518086849455e-05,"mean_abs_dp":4.8890120578943024e-05,"oracle_top2_margin":0.9874800471851845,"near_tie":false,"n_options":3,"probs":[0.9920238521758329,0.004431125791015359,0.003545022033151843],"hidden_sha256":"8c1608117b77410efd046b2ddc42b546b8d95a0e4eb556b8a1d0e411fad38fc6","first_in_process":false,"shard":"shard_08","index":39},{"id":"own_invoice_07","name":"status","variant":"base","source":"own","type":"choice","tokens":200,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0003038573598911354,"mean_abs_dp":0.00015192867994556596,"oracle_top2_margin":0.9534832020098472,"near_tie":false,"n_options":4,"probs":[0.009467312318435438,0.9700739494200498,0.016197374328073742,0.004261363933441127],"hidden_sha256":"9083efd0353925fa5ec96c939a1c04ba3dce6a35aeeab6608bc8c6dd8856c0e1","first_in_process":true,"shard":"shard_09","index":0},{"id":"own_invoice_07","name":"overdue","variant":"base","source":"own","type":"noul","tokens":179,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0007098452696669005,"mean_abs_dp":0.0007098452696668936,"oracle_top2_margin":0.873984931273492,"near_tie":false,"n_options":2,"probs":[0.9362826203670791,0.06371737963292087],"hidden_sha256":"c9ac9657ea612720600e5a429e96d72e2b9858133b62e14f446da68807c5b97d","first_in_process":false,"shard":"shard_09","index":1},{"id":"own_invoice_07","name":"days_late","variant":"base","source":"own","type":"score","tokens":212,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":3,"argmax_oracle":3,"argmax_equal":true,"max_abs_dp":0.003641274821665763,"mean_abs_dp":0.0018206374108328737,"oracle_top2_margin":0.6100105675778147,"near_tie":false,"n_options":4,"probs":[0.014591941034970126,0.021391172778186533,0.18030051575260242,0.7837163704342409],"hidden_sha256":"744a155b3b5a7409f06e03105b23814f18dfd5515b9d9e539c549d441e5d5d40","first_in_process":false,"shard":"shard_09","index":2},{"id":"own_sensor_08","name":"out_of_range","variant":"base","source":"own","type":"choice","tokens":210,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0067648983722385925,"mean_abs_dp":0.003382449186119303,"oracle_top2_margin":0.47726427438804064,"near_tie":false,"n_options":4,"probs":[0.1495192067501536,0.6366955808640196,0.11523439242426277,0.09855081996156406],"hidden_sha256":"2d5732f4a474b39ccc9044026571f12ac179f5c4c14a89c7e02e48783e5bc713","first_in_process":false,"shard":"shard_09","index":3},{"id":"own_sensor_08","name":"alert","variant":"base","source":"own","type":"score","tokens":227,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.003918327202090177,"mean_abs_dp":0.0026122181347268242,"oracle_top2_margin":0.8319948315413686,"near_tie":false,"n_options":3,"probs":[0.8842365508708364,0.057161328071128885,0.05860212105803468],"hidden_sha256":"03b8388a08733f5e30d063e910bdffe4b6d2bf507fb96898ecade611bb020e61","first_in_process":false,"shard":"shard_09","index":4},{"id":"own_sensor_08","name":"all_ok","variant":"base","source":"own","type":"noul","tokens":213,"calls":4,"padded_tokens":256,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0002333505854283241,"mean_abs_dp":0.00023335058542830328,"oracle_top2_margin":0.8775311762487958,"near_tie":false,"n_options":2,"probs":[0.9385322375389696,0.06146776246103035],"hidden_sha256":"62ea508e070fc198e124b4fcd362697ff5fe2c7e4745149cfb2c159d5ddf4875","first_in_process":false,"shard":"shard_09","index":5},{"id":"own_fiveq_09","name":"signed","variant":"base","source":"own","type":"noul","tokens":143,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.00022772101247549997,"mean_abs_dp":0.00022772101247548956,"oracle_top2_margin":0.9586839893129757,"near_tie":false,"n_options":2,"probs":[0.02088572635598767,0.9791142736440124],"hidden_sha256":"08d9bef5b9a2f7726391ab8ffd634bd81f8d0eab79fed9bac71c501aaacceccd","first_in_process":false,"shard":"shard_09","index":6},{"id":"own_fiveq_09","name":"where","variant":"base","source":"own","type":"choice","tokens":148,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":2.3556797054717464e-05,"mean_abs_dp":1.54356927109414e-05,"oracle_top2_margin":0.9790333070679461,"near_tie":false,"n_options":4,"probs":[0.005847992447190703,0.9848764741805618,0.0044126635397083655,0.004862869832539103],"hidden_sha256":"c9aeacc9b3406d2883f6db5d76c2ea4da1981ff711116b4e71fa1cb15ef3cd4d","first_in_process":false,"shard":"shard_09","index":7},{"id":"own_fiveq_09","name":"damaged","variant":"base","source":"own","type":"noul","tokens":144,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00031856297991338794,"mean_abs_dp":0.0003185629799133359,"oracle_top2_margin":0.9486429121749174,"near_tie":false,"n_options":2,"probs":[0.974640019067372,0.02535998093262798],"hidden_sha256":"bd38e400525a5d7652bbf5b85bd2f3002a67a4be5f104c1dd2dff19c9d09efc5","first_in_process":false,"shard":"shard_09","index":8},{"id":"own_fiveq_09","name":"request_followed","variant":"base","source":"own","type":"score","tokens":146,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.001447841636173508,"mean_abs_dp":0.0009652277574490054,"oracle_top2_margin":0.26459297103932206,"near_tie":false,"n_options":3,"probs":[0.2643376031707962,0.20517919412751376,0.53048320270169],"hidden_sha256":"7fb5ca2d2d50ee70ae1cada0d6d1c685b5525a6994cff38b59505134d4c6c810","first_in_process":false,"shard":"shard_09","index":9},{"id":"own_fiveq_09","name":"photo","variant":"base","source":"own","type":"choice","tokens":139,"calls":3,"padded_tokens":192,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.498287637353052e-05,"mean_abs_dp":7.498287637351404e-05,"oracle_top2_margin":0.9725187194604101,"near_tie":false,"n_options":2,"probs":[0.9863343426065786,0.013665657393421463],"hidden_sha256":"9b2e318fbe9faa8127639d479596825cd85868fa4157ae4029a974ca40dbd513","first_in_process":false,"shard":"shard_09","index":10},{"id":"own_long_log_10","name":"first_failure","variant":"base","source":"own","type":"choice","tokens":1652,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":4,"argmax_oracle":4,"argmax_equal":true,"max_abs_dp":0.0007644778790389939,"mean_abs_dp":0.00025482595967966205,"oracle_top2_margin":0.8236539990551434,"near_tie":false,"n_options":6,"probs":[0.008829215091975859,0.01644919997126062,0.05387205015412446,0.026570632630961304,0.8783892544544368,0.015889647697240987],"hidden_sha256":"bdc53d1441cec63ad080ccd1c7cc59ca68f04eb49b5fc7cdfd3229710f33db65","first_in_process":false,"shard":"shard_09","index":11,"hidden":{"min_pos_cos":0.9997178009480513,"min_pos_cos_index":261,"max_abs_diff":0.2262253761291504,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_log_10","name":"impact","variant":"base","source":"own","type":"score","tokens":1643,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.004178411420219907,"mean_abs_dp":0.002089205710109924,"oracle_top2_margin":0.7135562371831476,"near_tie":false,"n_options":4,"probs":[0.09537877803767322,0.0604081748845074,0.8027293185621256,0.04148372851569375],"hidden_sha256":"aec668352a4751c0525ed7d5b7f3a305d756a78e0194e6e235e05c1aaca4eaeb","first_in_process":false,"shard":"shard_09","index":12,"hidden":{"min_pos_cos":0.9997178009480513,"min_pos_cos_index":261,"max_abs_diff":0.2262253761291504,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_log_10","name":"resolved","variant":"base","source":"own","type":"noul","tokens":1641,"calls":26,"padded_tokens":1664,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0013117792382811855,"mean_abs_dp":0.0013117792382810745,"oracle_top2_margin":0.8030306656886631,"near_tie":false,"n_options":2,"probs":[0.9028271120826127,0.0971728879173874],"hidden_sha256":"0c042a40823a84e010667bfefe61229fc122b5aba48daf2326445a261d80221d","first_in_process":false,"shard":"shard_09","index":13,"hidden":{"min_pos_cos":0.9997178009480513,"min_pos_cos_index":261,"max_abs_diff":0.2262253761291504,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"own_long_contract_11","name":"convenience_notice","variant":"base","source":"own","type":"choice","tokens":1455,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0002748434352978224,"mean_abs_dp":0.0001374217176488726,"oracle_top2_margin":0.9696129049453296,"near_tie":false,"n_options":4,"probs":[0.0075465196676341205,0.007167930583808889,0.9775956840843111,0.007689865664246013],"hidden_sha256":"a558ee57dcc4869357af30074d2690917e52f067c1d8fd6b71699e530c7a7683","first_in_process":false,"shard":"shard_09","index":14},{"id":"own_long_contract_11","name":"liability_cap","variant":"base","source":"own","type":"noul","tokens":1444,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":5.7992382577753965e-05,"mean_abs_dp":5.799238257772274e-05,"oracle_top2_margin":0.9687246536090245,"near_tie":false,"n_options":2,"probs":[0.9843043344219345,0.01569566557806552],"hidden_sha256":"e7727349a7ca25ea7281227aa53672ec7298ef012a81204c3e9688a4bfe69d13","first_in_process":false,"shard":"shard_09","index":15},{"id":"own_long_contract_11","name":"initial_term","variant":"base","source":"own","type":"score","tokens":1451,"calls":23,"padded_tokens":1472,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0009683413216943215,"mean_abs_dp":0.000628939167508918,"oracle_top2_margin":0.6988595928596064,"near_tie":false,"n_options":4,"probs":[0.013131875502292986,0.05677671488524657,0.8152726050616835,0.11481880455077689],"hidden_sha256":"f99e946a233086122f5f2be86e003e1eaae7704da478b5e1c761147da9ceb4e1","first_in_process":false,"shard":"shard_09","index":16},{"id":"own_long_minutes_12","name":"irrigation","variant":"base","source":"own","type":"choice","tokens":1375,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00027356716240989343,"mean_abs_dp":0.00018237810827327096,"oracle_top2_margin":0.971579292780383,"near_tie":false,"n_options":3,"probs":[0.007248543759270469,0.010354411039583863,0.9823970452011456],"hidden_sha256":"38ad6ef82ca144fa4156d885061e8f2b42136883b4a3073a88f3ca7a4abf21c2","first_in_process":false,"shard":"shard_09","index":17},{"id":"own_long_minutes_12","name":"chair","variant":"base","source":"own","type":"choice","tokens":1380,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.00010338376195873078,"mean_abs_dp":5.1691880979406264e-05,"oracle_top2_margin":0.9811541905303394,"near_tie":false,"n_options":4,"probs":[0.9862983468628512,0.003825816991118933,0.004886278509750151,0.004989557636279632],"hidden_sha256":"75116309dd39310985a9c5bc2578143c66b9c956c3e78716675fb4fad12154c1","first_in_process":false,"shard":"shard_09","index":18},{"id":"own_long_minutes_12","name":"lease_unanimous","variant":"base","source":"own","type":"noul","tokens":1356,"calls":22,"padded_tokens":1408,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.001130716524755826,"mean_abs_dp":0.001130716524755826,"oracle_top2_margin":0.8679600847894678,"near_tie":false,"n_options":2,"probs":[0.06715067413002193,0.9328493258699782],"hidden_sha256":"6ee7a9c53cf250c8be31e8150dbc373509da732540b368c722297f97d07eb6b0","first_in_process":false,"shard":"shard_09","index":19},{"id":"card_refund","name":"refund","variant":"base","source":"card","type":"noul","tokens":39,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":7.191058084943247e-05,"mean_abs_dp":7.191058084939084e-05,"oracle_top2_margin":0.9792455172612409,"near_tie":false,"n_options":2,"probs":[0.9896946692114699,0.010305330788530197],"hidden_sha256":"99da4b773c0e1d7919828ebcb8555b88e7f548852640da4b0430765d3010c948","first_in_process":false,"shard":"shard_09","index":20,"hidden":{"min_pos_cos":0.9997776553912043,"min_pos_cos_index":27,"max_abs_diff":0.16211891174316406,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_refund","name":"team","variant":"base","source":"card","type":"choice","tokens":61,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":6.155015469000436e-05,"mean_abs_dp":4.103343646005668e-05,"oracle_top2_margin":0.9728582283795494,"near_tie":false,"n_options":3,"probs":[0.9835667192872184,0.010634730901953939,0.005798549810827583],"hidden_sha256":"03767127fb9009478ba628149de8b1d5d829f4fcbf735d6d93bf2b2530b4811b","first_in_process":false,"shard":"shard_09","index":21,"hidden":{"min_pos_cos":0.9998136768773541,"min_pos_cos_index":17,"max_abs_diff":0.12644195556640625,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_refund","name":"urgency","variant":"base","source":"card","type":"score","tokens":54,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0007779649500007646,"mean_abs_dp":0.0005186433000004912,"oracle_top2_margin":0.22886812438079923,"near_tie":false,"n_options":3,"probs":[0.337798065837446,0.5677139400714755,0.09448799409107857],"hidden_sha256":"f51b5c5fccaaf6b4cfbc0f5d664c7991e0e17b70efcb07fb3570398b1a273460","first_in_process":false,"shard":"shard_09","index":22,"hidden":{"min_pos_cos":0.9998136768773541,"min_pos_cos_index":17,"max_abs_diff":0.12644195556640625,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"card_ticket_00","name":"team","variant":"base","source":"card","type":"choice","tokens":58,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.004932724532862198,"mean_abs_dp":0.0032884830219081194,"oracle_top2_margin":0.6698469077623299,"near_tie":false,"n_options":3,"probs":[0.15118859280692865,0.8305955896550227,0.018215817538048636],"hidden_sha256":"890ef7a4b2f522df737cf8b5d6795c512ecfe5c3d63e3ea26a78150344e0449f","first_in_process":false,"shard":"shard_09","index":23},{"id":"card_ticket_01","name":"team","variant":"base","source":"card","type":"choice","tokens":56,"calls":1,"padded_tokens":64,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":4.053190968935413e-05,"mean_abs_dp":2.7021273126212955e-05,"oracle_top2_margin":0.9806915560609186,"near_tie":false,"n_options":3,"probs":[0.0065117982180930385,0.9872281567394655,0.0062600450424414675],"hidden_sha256":"43b7c94fd485c3ef73b0d72c002b7642a85bf7bce95d614fd6a2f3a8943e5413","first_in_process":false,"shard":"shard_09","index":24},{"id":"long_15k","name":"temp_excursion","variant":"base","source":"own_long","type":"noul","tokens":1491,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0004428039335171857,"mean_abs_dp":0.0004428039335171649,"oracle_top2_margin":0.877992181442829,"near_tie":false,"n_options":2,"probs":[0.9394388946549317,0.060561105345068346],"hidden_sha256":"dc592e0a1b44e6bd9cb51c17cb98ce7ea806812ac340efcc881fb516765e1351","first_in_process":false,"shard":"shard_09","index":25},{"id":"long_15k","name":"first_unit","variant":"base","source":"own_long","type":"choice","tokens":1515,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001725066638257923,"mean_abs_dp":8.625333191286514e-05,"oracle_top2_margin":0.946464034757811,"near_tie":false,"n_options":4,"probs":[0.004974077304042945,0.004180576999534178,0.968488693220807,0.02235665247561588],"hidden_sha256":"cbecffbe538856eee12eb1733e93dbea2ef14a27ffce12ad456680b810d79b17","first_in_process":false,"shard":"shard_09","index":26},{"id":"long_15k","name":"severity","variant":"base","source":"own_long","type":"score","tokens":1526,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.00295749175274429,"mean_abs_dp":0.001971661168496235,"oracle_top2_margin":0.13607007226849527,"near_tie":false,"n_options":3,"probs":[0.11121860385735383,0.37917149604646866,0.5096099000961776],"hidden_sha256":"e02776e95a53d19b3e36b7ef734eb29a9e103e9bcc92ad0bf89443cd2ddb0c0e","first_in_process":false,"shard":"shard_09","index":27},{"id":"long_15k","name":"door_open","variant":"base","source":"own_long","type":"noul","tokens":1522,"calls":24,"padded_tokens":1536,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.0008249991903853671,"mean_abs_dp":0.0008249991903853462,"oracle_top2_margin":0.8455142706260308,"near_tie":false,"n_options":2,"probs":[0.9235821345034008,0.07641786549659926],"hidden_sha256":"377481da3a238c7b1e765975efa0ca3a51c4a72775668e20832663392e16dd7c","first_in_process":false,"shard":"shard_09","index":28},{"id":"long_34k","name":"refunded_twice","variant":"base","source":"own_long","type":"choice","tokens":3470,"calls":55,"padded_tokens":3520,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.0001473261468686246,"mean_abs_dp":7.366307343431447e-05,"oracle_top2_margin":0.9172464903835365,"near_tie":false,"n_options":4,"probs":[0.006548491371802775,0.0282129720795555,0.9453875345450858,0.01985100200355589],"hidden_sha256":"5b772400873c7551f87b7db141cb103f87bdeeefdfb0e6e8754269467b4ad0b8","first_in_process":false,"shard":"shard_09","index":29,"hidden":{"min_pos_cos":0.9947377666772287,"min_pos_cos_index":253,"max_abs_diff":0.6236745119094849,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"big_refunds_approved","variant":"base","source":"own_long","type":"noul","tokens":3451,"calls":54,"padded_tokens":3456,"finite":true,"all_zero":false,"argmax":1,"argmax_oracle":1,"argmax_equal":true,"max_abs_dp":0.0004197998476305198,"mean_abs_dp":0.00041979984763049205,"oracle_top2_margin":0.27068175772038333,"near_tie":false,"n_options":2,"probs":[0.36423932129217784,0.6357606787078222],"hidden_sha256":"0ae523f842a07de21e9d349f37d4c8e78e140a6355a63654653621b4c68c756f","first_in_process":false,"shard":"shard_09","index":30,"hidden":{"min_pos_cos":0.9947377666772287,"min_pos_cos_index":253,"max_abs_diff":0.6236745119094849,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"cancellations","variant":"base","source":"own_long","type":"score","tokens":3461,"calls":55,"padded_tokens":3520,"finite":true,"all_zero":false,"argmax":2,"argmax_oracle":2,"argmax_equal":true,"max_abs_dp":0.008234960860138196,"mean_abs_dp":0.004151174824248769,"oracle_top2_margin":0.695961559659646,"near_tie":false,"n_options":4,"probs":[0.020368790396786574,0.13963187989034426,0.8194771948768851,0.02052213483598403],"hidden_sha256":"94d3bd605d72db6301289b7bd15007f7acb40ef19f9b716584c5fa7b029a6b3c","first_in_process":false,"shard":"shard_09","index":31,"hidden":{"min_pos_cos":0.9947377666772287,"min_pos_cos_index":253,"max_abs_diff":0.6236745119094849,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}},{"id":"long_34k","name":"shipped_unpaid","variant":"base","source":"own_long","type":"noul","tokens":3446,"calls":54,"padded_tokens":3456,"finite":true,"all_zero":false,"argmax":0,"argmax_oracle":0,"argmax_equal":true,"max_abs_dp":0.008582613011315632,"mean_abs_dp":0.008582613011315604,"oracle_top2_margin":0.037437100559284564,"near_tie":false,"n_options":2,"probs":[0.5273011632909579,0.4726988367090421],"hidden_sha256":"404a92cd5d45f5b3ed370432fe464afc44f94458915c859c0e120a1dcbae98d8","first_in_process":false,"shard":"shard_09","index":32,"hidden":{"min_pos_cos":0.9947377666772287,"min_pos_cos_index":253,"max_abs_diff":0.6236745119094849,"positions_below":{"0.99":0,"0.9":0,"0.5":0}}}]},"processes":[{"shard":"shard_00","rows":40,"prompts":41,"load_seconds":8.394476332934573,"load_split_seconds":{"model":1.9499847909901291,"main":6.444491541944444},"function_names":["main"],"process_wall_seconds":28.23362183396239,"reset_check":{"run":"tv4_000:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.0725947089958936},"contract_mismatch":[]},{"shard":"shard_01","rows":40,"prompts":41,"load_seconds":0.6911258340114728,"load_split_seconds":{"model":0.00651062501128763,"main":0.6846152090001851},"function_names":["main"],"process_wall_seconds":5.000625291024335,"reset_check":{"run":"tv4_040:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1415914579993114},"contract_mismatch":[]},{"shard":"shard_02","rows":40,"prompts":41,"load_seconds":0.6305835000239313,"load_split_seconds":{"model":0.005583500023931265,"main":0.625},"function_names":["main"],"process_wall_seconds":4.835478125023656,"reset_check":{"run":"tv4x_qnli_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07410366600379348},"contract_mismatch":[]},{"shard":"shard_03","rows":40,"prompts":41,"load_seconds":0.6050396249629557,"load_split_seconds":{"model":0.005965916905552149,"main":0.5990737080574036},"function_names":["main"],"process_wall_seconds":5.538862042012624,"reset_check":{"run":"tv4x_sciq_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.10848366701975465},"contract_mismatch":[]},{"shard":"shard_04","rows":40,"prompts":41,"load_seconds":0.9260537920054048,"load_split_seconds":{"model":0.006420417106710374,"main":0.9196333748986945},"function_names":["main"],"process_wall_seconds":5.697763042058796,"reset_check":{"run":"tv4x_composition_holdout_00:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.10552433296106756},"contract_mismatch":[]},{"shard":"shard_05","rows":40,"prompts":41,"load_seconds":0.590547583065927,"load_split_seconds":{"model":0.008561208029277623,"main":0.5819863750366494},"function_names":["main"],"process_wall_seconds":4.485607541981153,"reset_check":{"run":"semif_a3f18f3a63d45345942b:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07174429099541157},"contract_mismatch":[]},{"shard":"shard_06","rows":40,"prompts":41,"load_seconds":0.5705066670197994,"load_split_seconds":{"model":0.00537858298048377,"main":0.5651280840393156},"function_names":["main"],"process_wall_seconds":4.4842809999827296,"reset_check":{"run":"semif_a0a39ace7e5337d038a7:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07212712499313056},"contract_mismatch":[]},{"shard":"shard_07","rows":40,"prompts":41,"load_seconds":0.5826243340270594,"load_split_seconds":{"model":0.0056056249886751175,"main":0.5770187090383843},"function_names":["main"],"process_wall_seconds":4.4832052499987185,"reset_check":{"run":"semif_0aa24a6701ac9c56c4a0:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.0718186660669744},"contract_mismatch":[]},{"shard":"shard_08","rows":40,"prompts":41,"load_seconds":0.5562187500763685,"load_split_seconds":{"model":0.005741833010688424,"main":0.55047691706568},"function_names":["main"],"process_wall_seconds":5.227409958955832,"reset_check":{"run":"semif_1df851a202994a07f123:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.07217241707257926},"contract_mismatch":[]},{"shard":"shard_09","rows":33,"prompts":34,"load_seconds":0.6105855000205338,"load_split_seconds":{"model":0.005820791004225612,"main":0.6047647090163082},"function_names":["main"],"process_wall_seconds":21.751631667022593,"reset_check":{"run":"own_invoice_07:0:base","bit_equal":true,"hidden_max_abs_diff":0.0,"wall_seconds":0.1432776659494266},"contract_mismatch":[]}]}}